Selling You The End Of The World

One common refrain in the AI hype cycle has been that AI development is not only perpetually just a few weeks and another billion away from human-level sentience, but that this sentience will ultimately try to kill us all.

Neither of those claims are true. At least, not in the Skynet terminator sense.

Instead of a hyper-sentient robot god hunting humans to extinction, AI is far more likely to kill your grandma with badly-automated medicare rejection systems with a 90 percent error rate, pummel you with lazy clickbait slop, give your Instagram credentials to a slob in Boise, and slowly kill your minority neighbors with data center exhaust in a country too corrupt to have working environmental regulators.

The real problems with AI mostly make for a shitty movie.

As I keep saying, the real issue and fight surrounding AI isn't really with the software, it's the terrible human beings in control of it. These are poorly understood automation platforms, layered on to existing systems long-broken by corruption, overseen by terrible, unremarkable, and extremely greedy men.

The more dramatic, exciting, Skynet-doomsday type AI warnings usually originate from two camps within AI development.

One being engineers who read a lot of science fiction and got easily spooked by their own algorithmic card tricks. The other coming from executives who are keen to not only market their software as more capable than it actually is, but to scare lawmakers into passing regulation they get to write and is favorable to them.

They don't want regulation that meaningfully constrains AI's impact on labor and consumers and protects the public interest, they want rules that legalize all of their worst impulses on surveillance, environmental abuse, and privacy, while making life harder on smaller or foreign companies trying to compete with them.

So every few weeks you see an AI company easily trick our lazy "CEO said a thing!" media into parroting claims of impending Skynet-esque doomsday. And it's only going to get louder as AI companies try to justify insane valuations and lopsided expenditures ahead of a flood of initial public offerings (IPOs) at OpenAI, Anthropic, and Elon Musk's white supremacy vaporware emporium.

Like last week, when the newswires were jam-packed with claims that Anthropic had called for a pause in AI development before humanity risks "losing control" of our fancy new AI godkings:

That's a lot of free marketing for a company that very much wants you to believe that software automation is far more capable than it actually is, and they're doing everything in their power to ensure it's being developed ethically.

The actual blog post by Anthropic mostly just talks about how automation software is now capable of building other automation software. Which is interesting and will have a broad and lasting impact on software and programmers, but isn't something that should result in you hoarding canned goods in the basement.

One useful rule of thumb if you're dizzied by AI hype and the surrounding shitty press coverage: any time you see the word "AI" in a headline, replace "AI" with "software automation" and see if it lands the same.

Anyway, despite the press cycle claiming otherwise, Anthropic doesn't really call for a serious pause on AI development in their post. Not in any meaningful way. Nor does it really present any hard evidence that we're at risk of losing control:

AI that can build itself would be a major development in the history of technology—one that could bring enormous good for the world in science, healthcare, and beyond. But full recursive self-improvement also might increase the risks of humans losing control over AI systems. If systems are capable of fully building their own successors, the ways we secure them, monitor them, and shape their behavior all grow much more important.

If "software automation" has escaped "human control" in a malicious way that closely resembles a virus, then the problem is very human. You've designed a shitty product in an incompetent way that did damage you could have avoided. You should be prosecuted and embarrassed for being bad at your job.

Hints that this was the fault of a sentient robot king helps shift accountability.

And while we certainly need AI rules to protect the public from terrible men building shitty software, who writes the rules in a country too corrupt to function? It certainly isn't labor unions, consumer rights policy experts, objective academics, or even engineers. This is America. It's going to be rich assholes.

The part where Anthropic claims to be calling for a pause on AI development doesn't really even say that. It's just a few paragraphs of wishful thinking:

If it were possible to effectively slow the development of this technology to give ourselves more time to deal with its immense implications, we think that would likely be a good thing. But if a slowdown simply lets the least cautious actors catch up technologically, it could leave everyone less safe. Without a global coordination mechanism, companies and governments will have to make difficult decisions about safety while under competitive and geopolitical pressures.

This is just zero calorie fantasy fluff. They don't really intend to take any meaningful pause. Not that such a global coordinated pause in AI development to protect the public interest is even possible in a country that was too corrupt to function even before it was taken over by a failed NYC real estate conman.

Trumpism and its courts, enabled and coddled by the lion's share of big tech companies at nearly every step of the way, has mercilessly dismantled all cogent regulatory oversight. Our Congress has been bribed into apathy. Who, exactly, is crafting and enforcing sensible, useful AI regulations in the next half decade?

This is a country that hasn't been able to pass meaningful internet privacy laws in the last quarter century despite a barrage of jaw dropping privacy scandals, many of them directly threatening national security. There's a lot of chatter about "building responsible AI frameworks," and none of it strikes me as honest.

I'm endlessly struck by the weird disconnect between AI boosterism and policy and current everyday political reality. If you're not capable of being honest about the fact a cabal of corrupt zealots has hollowed public interest governance out like a pumpkin, why should people take anything you're saying seriously?

To be clear, Anthropic is probably among the "better" companies pursuing AI development, though that's arguably a very low bar. Their executives at least aren't overt Trump supporters, and they've sometimes shown the faintest hint of an ethical backbone when it comes to AI use in military murder machinery.

Though I doubt these ethical considerations have any meaningful staying power. There's money to be made. And like all of these companies, the revenues they're promising are so comically beyond the amount of money they're actually making, they're incentivized to be full of shit ahead of an IPO.

These are not companies that are capable of being honest about what their products do or how much money they'll actually make. And they've tethered all of this hype and financial fraud to our economy in a way that's going to cause massive hardship borne primarily by labor, consumers, and the poor.

None of this is to say that you won't someday also get to enjoy badly-programmed Arnold Schwarzenegger robots hunting down your loved ones in rat tunnels if that's truly your kink, but that's not really what these guys are worried about.

They're mostly worried about building a badly-automated ouroborus that shits advertising and subscription revenue, free from meaningful oversight, ethical constraints, labor rights, or competition. And barring some meaningful and profound political sea changes, they're going to get most of what they want.