How do you interpret the resignation of that Anthropic engineer, who’s saying the companies are reckless and gambling with our lives? On the one hand, you seem to agree that the companies are being reckless; on the other hand, you’ve made clear that you think the “AI will cause the end of civilization” debate is senseless.
My whole career, I’ve been trying to say the same thing from different angles. We wrote our stochastic parrot paper in 2021 for this reason. OpenAI had claimed GPT-2 was too powerful to release. It was so hyped up. And we were saying, you’re asking the wrong questions.
In my book I write, “Can a bridge decide to collapse?” The whole “Oh no, our models went rogue” thing … When a bridge collapses, you don’t analyze whether the bridge was ethical or sentient or why it decided to collapse. You ask, who is the person who built this bridge to be so flimsy? There are permits you have to acquire. There are supposed to be tests. When you ask about the bridge collapsing itself, you’ve lost the thread.
Meanwhile, what’s really existential is AI powering autonomous weapons, killing machines, that are actually being used in warfare. The climate catastrophe is a real thing that could be exacerbated by what these tech companies are building. Bosses are using AI as an excuse to get rid of their pesky workers. This machine-god narrative is, in my opinion, meant to distract us from these. The industry has become a mix of cults, a mix of people who are true believers, just like a preacher who says the end times is coming. The singularity has been coming for decades now.
Take me through, step by step, how “AI extinction” might happen, if it were real. I’m thinking of this post where someone said, Does an angry GPU just show up at my house or something?
I can’t really envision it, but a lot of those scenarios seem to be that you ask the machine to optimize for some other task, and then the machine decides that the way to get to that goal is by killing everyone in the world. Does the AI hypnotize you so that you don’t unplug it? I don’t know! Other people say, well, the AI can infect your machines. Yes—and we already have malware, so…
What about floods and fires? What about a major disruption to the global food supply chain? There are real threats coming to us, like people being able to create and use chemical weapons. [Note: Not long after we spoke, Anthropic revealed that it had blocked several potential malicious attempts by scientists to build biological weapons.]
There’s another element where you’re so privileged that you can’t imagine other things harming you. Like if you’re not worried about police brutality, or floods, or warfare—things that can actually kill you—then you can obsess over the machine-god.
Why are we spending so much time talking about this idea of AI that people have been talking about since the ’40s and ’50s? I have this game I play sometimes where I read quotes to people about artificial intelligence that I’ve gathered in my research, and I ask, What decade was this statement made in? Sometimes people get it right, but a lot of times they do not, because it all sounds the same.
This is an edition of Steven Levy and Lauren Goode’s Backchannel newsletter. Read previous newsletters here.