In February, Anthropic Safety Lead Mrinank Sharma abruptly resigned from the company, writing in a cryptic open letter that “the world is in peril” from “a whole series of interconnected crises” including AI and bioweapons. “Throughout my time here, I’ve repeatedly seen how hard it is to truly let our values govern our actions,” Sharma wrote at the time.
In July, an open letter signed by over 1,300 employees at frontier AI companies warned of “a real risk that capability development rapidly accelerates beyond our ability to understand or control the resulting systems.” That open letter asked the US government to back an “international effort to develop the technical and governance tools needed to deliberately pace the frontier of automated AI development.”
Here in the US, proposed legislation, including the AI Kill Switch Act and the FRONTIER Act, is at least seeking to impose some level of governmental control over potential runaway AI scenarios. Thus far, though, the international governmental response has been more muted than you might expect if leaders truly believed AI systems had a real chance of causing civilization-level destruction.
That said, international treaties on threats like nuclear bombs and biological weapons took decades to develop and enact. Those are years we might not have if the most apocalyptic AI doomsayers turn out to be correct.
“Safety researchers are resigning, powerful AI models are breaking out of their labs, and companies are racing ahead anyway,” Rep. Lori Trahan (D-Ma.) wrote on social media Wednesday morning. “It’s past time for Congress to get off the sidelines and do its job. We can start with my bipartisan FRONTIER Act.”