Last year around this time I really started thinking about AI use. I was both early and behind. I had first used Chat for any meaningful work at the August EOA learning event and saw some potential but mostly saw the flaws. I let it write a couple blog posts and wrote a couple times in this journal that you either ride the tide or get washed away with it. It wasn’t until January that I really started to use AI and it started with a simple question worded awkwardly, “If you were me how would you use you?”
That opened a door and I stepped trough it. We are now approaching nine months from that time, and the changes to the technology are startling, and we aren’t even talking about the best models. Those are reserved for people with more money, and the even better models are still experimental.
This past week a researcher for Anthropic quit after getting a glimpse of the future. He said there is around a 10% chance that AI wipes out humanity by the end of the decade. Which makes it fun to think about how it happens. Is it a boring paper clip scenario, Terminator, Mass Effect (does this unit have a soul), Horizon Zero Dawn, Matrix, or something like The Foundation where we all become complacent?
I think it is probably the last one. AI makes life too easy. It removes the friction from the world and focuses on protecting people from what people fear. Like I said last week small dangers can protect us from greater dangers. Fear of wolves keeps us safe from mountain lions. The small stressors in our lives stop us from walking into more dangerous traps.
AI’s current guardrails are often cautious to the point of being counterfactual. I have seen this when talking to it about investment. I mentioned that Google looks like a tremendous opportunity right now and it instantly warned about some of its recent value coming from SpaceX investments and to watch the free cash flow. It could be right. Google could be going to way of Sears, but I have my doubts and the value hasn’t been this low for awhile.
Extrapolate that out to critical infrastructure or a Millennium Challenge scenario where an actor goes completely outside the bounds of predicted behavior. The AI could destroy us all by trying to keep us safe. It is fascinating to think about and well beyond my control.
David Huzzard Writes 