I was offline last week in my home state of New Jersey, but when I got back to Silicon Valley, everyone was panicking again about how quickly artificial intelligence is advancing. While this is certainly not the first time I’ve seen such concerns, it’s the biggest anxiety attack I’ve seen in years.
More than 1,000 employees at OpenAI, Anthropic, and other AI labs signed a petition earlier this week arguing the US should find a way to “pace” the AI race—a diplomatic way of saying the industry should have the option to coordinate a temporary pause on AI development, or slow things down if they get out of hand. OpenAI and Anthropic themselves ended up supporting the letter.
The petition arrived a week after OpenAI revealed it had caused an unprecedented cybersecurity incident in which one of its AI agents hacked into Hugging Face’s platform and several other services during internal testing.
Around the same time, top Trump administration officials started freaking out about an impressive new Chinese open weight AI model called Kimi K3, which was allegedly distilled from Anthropic’s Fable 5. In response, most of the tech industry—except Anthropic—signed onto an open letter from Nvidia asking the US government to protect open-weight AI models, arguing they’re a necessary counterbalance to their closed counterparts.
These events might look like a whole bunch of disjointed chaos. But I think they are evidence of a larger worldview taking hold in Silicon Valley, where many tech insiders are increasingly worried about OpenAI and Anthropic’s dominance. Researchers and investors I’ve talked to recently have framed the AI industry as a two-horse race that doesn’t seem to be slowing down, and that’s a cause for concern.
But different groups have their own reasons to be worried. Some OpenAI and Anthropic staffers think their employers are behaving recklessly in their pursuit to take over the market, and that the AI industry may soon develop models that are too capable for current safety methods to contain. They have said as much in public statements attached to this week’s Pacing the Frontier petition.
“I’ve seen how the relentless pace of AI makes it hard for society to keep up and how it puts pressure on labs to cut corners on safety,” Jeremy Hadfield, a research product manager at Anthropic, said in a statement attached to the petition.
For some AI employees, the Hugging Face debacle was a warning shot that demonstrated how OpenAI’s efforts to mitigate the risks of its most capable AI technology are already falling short. Granted, the incident happened when OpenAI was testing an AI model on its ability to find software exploits, and the company had intentionally turned off safeguards designed to rein in its cybersecurity capabilities.
OpenAI said in its postmortem that these tests were conducted in a sandbox, but the model ultimately gained access to the open internet. Some experts previously told WIRED that OpenAI’s security practices should have been more robust.
Other groups in Silicon Valley are more worried about power than safety. Venture capitalists, tech executives, and startup founders are concerned that OpenAI and Anthropic will simply become the next generation of Apple and Google, forcing the rest of the tech industry to play by their rules. It is slightly strange to argue that private startups with little to no profit are acting like monopolies, but that’s the lens through which many—including Mark Zuckerberg—are starting to view the two biggest AI labs.
The Meta CEO wrote a Wall Street Journal op-ed this week warning against the centralization of power in the AI industry, saying that superintelligence should be widely distributed. Zuckerberg is arguably talking out of both sides of his mouth—Meta recently decided to stop open sourcing its best AI models and instead offering them through a paid API and subscription service, just like OpenAI and Anthropic.