In November 2023, a nonprofit board in San Francisco fired the most famous CEO in technology, citing a duty to ensure that artificial general intelligence benefited humanity. Five days later that board was gone. The episode is usually retold as a corporate soap opera, but it exposed something more useful: OpenAI’s AGI governance is not a legal abstraction. It is a live, contested question about who decides when a machine becomes more capable than us, and what happens to hundreds of billions of dollars when they do.
This piece walks through the actual machinery — the founding charter, the definition, the deployment gates, the 2025 restructure, and the courts and regulators circling the building. Not to cheer or sneer, but to let you read the next OpenAI announcement with a clearer head.
The founding document and the fourteen-word problem
OpenAI’s charter, published in 2018, reads like a constitution written by people who expected to lose control of what they were building. It commits the organisation to use whatever influence it gains over AGI’s deployment “to ensure it is used for the benefit of all,” and specifically to avoid uses of AI that “harm humanity or unduly concentrate power.”
That last clause is the interesting one. It is a governance promise aimed squarely at the company’s own commercial success, and it has no enforcement mechanism beyond the judgement of whoever sits on the board.
The definition underpinning all of this is famously compact: highly autonomous systems that outperform humans at most economically valuable work. No benchmark. No capability threshold. No measurement protocol, no date, no inspector. The definition functions less as a technical standard than as a tripwire that a small group of people is empowered to trip. If you want the full forensic read on those words and why they were chosen, OpenAI’s AGI definition and what those 14 words actually mean is worth your time.
The AGI clause: when a philosophy becomes a contract term
Here is where governance stops being philosophical. In the 2019 Microsoft agreement, OpenAI granted the software giant an exclusive licence to its pre-AGI technology. The licence was explicitly conditional: once OpenAI’s board declared AGI achieved, Microsoft’s rights to the resulting systems fell away, and the two organisations would have to renegotiate.
Read that again. A sentence written by ethicists became the trigger for the most consequential commercial term in the industry. Wall Street analysts valuing Microsoft’s stake were, whether they liked it or not, valuing a philosophical judgement call made by a nonprofit board in California.
That structure has since been reworked. Under the deal announced in late October 2025, Microsoft’s intellectual property rights now extend through 2032 regardless of any AGI declaration, and reporting on the transaction indicates that a future AGI call would be assessed with input from an independent panel of experts rather than resting solely on the board. The change removes an obvious conflict of interest — directors who could trigger a multi-billion-dollar contractual event by definition — but it also removes the clean, hard stop that the original clause implied. The broader sequence of milestones and money behind these decisions is laid out in the decoded OpenAI AGI roadmap.
Who actually holds the off switch?
Three mechanisms do the day-to-day work of AGI governance inside OpenAI. None of them is a statutory regulator, and all of them are staffed by people the company pays.
The Preparedness Framework
Introduced in December 2023 and revised in April 2025, the framework sorts risk into capability areas — biological and chemical harm, cybersecurity, and AI self-improvement among them — and two severity tiers, High and Critical. A model judged to be at the Critical threshold is not supposed to be deployed until the board has reviewed the evidence and signed off.
That is the closest thing the company has to a hard gate, and it deserves credit for being written down with named thresholds. It is also entirely self-administered. The evaluations are run internally, the threshold interpretations are internal, and the summary reports published alongside major launches are chosen by the same team that built the model. How those launch decisions play out in practice is examined in more depth in OpenAI’s path to AGI, rung by rung.
The Safety and Security Committee
Created in May 2024, in the same fortnight that the superalignment team was dissolved and co-lead Jan Leike resigned with the line that “safety culture and processes have taken a backseat to shiny products.” The committee originally included Sam Altman alongside board members and outside advisers. In September 2025 it was reconstituted as an independent body chaired by Zico Kolter, with Altman no longer sitting on it, and given explicit authority to advise the board on delaying a deployment.
Advisory authority is the phrase to underline. The committee can recommend a pause. It cannot impose one. Whether that distinction matters in a real crunch is the whole ballgame, and it is the question at the centre of who actually decides it’s safe to ship.
What the 2025 restructure changed
For six years OpenAI ran on a deliberately awkward structure: a nonprofit controlling a capped-profit arm, with investor returns limited to 100 times their money. In October 2025 the company converted the operating business into a public benefit corporation, OpenAI Group PBC, overseen by a renamed nonprofit, the OpenAI Foundation.
The numbers tell you most of what you need to know. The Foundation holds roughly 26% of the PBC, a stake valued in the region of $130 billion, and retains the right to appoint the board. Microsoft holds about 27%, worth roughly $135 billion. Employees and other investors hold the remainder.
Supporters point out that board appointment power is the only lever that has ever mattered in a governance fight. Critics point out that a 26% holder loses every ordinary shareholder vote, and that PBC directors are legally obliged to weigh mission and shareholder value rather than mission alone. Both are true. Which one dominates will be decided by events, not by the certificate of incorporation.
The pressure from outside
OpenAI does not govern AGI in a vacuum, even if it sometimes writes as though it does. The external constraints are real but patchy:
- Litigation. Elon Musk’s suit over the company’s drift from its nonprofit origins, and multiple copyright actions from publishers and authors, have forced internal documents into the public record at a steady drip.
- State attorneys general. Delaware, where OpenAI is incorporated, and California, where it operates, both signalled scrutiny of the conversion. Neither has blocked it.
- Europe. The EU AI Act’s general-purpose AI obligations, in force since August 2025, impose documentation, evaluation and incident-reporting duties that apply regardless of what a company’s charter says.
- Voluntary agreements. Pre-deployment testing arrangements with the UK AI Security Institute and the US Center for AI Standards and Innovation, plus frontier-lab commitments made at the 2024 Seoul summit.
- No federal statute. The United States still has no comprehensive AI law. Governance in America remains contracts, charters and press releases.
How to read a governance claim in the wild
Next time OpenAI publishes a safety framework or a restructure explainer, run it through five questions.
Can anyone say no, and to what? A veto over a specific launch is worth more than a seat at the table for a general discussion. Find the sentence that grants refusal, or accept that it doesn’t exist.
Is the gate before or after training? Most published thresholds govern deployment. Compute governance — decisions about how large a training run to authorise — is where the genuinely expensive risks accumulate, and it is far less documented.
What evidence clears the bar? “Sufficiently safe” without a defined test is a judgement, not a standard. Ask what result would produce a no.
Who pays the committee? Independent in title is not independent in resources. Staff, budget and access to unreleased models all shape what an oversight body can actually see.
What happens when the board and the CEO disagree? November 2023 answered that one empirically. The board lost.
The next real test will not arrive as a document. It will arrive as a specific model, on a specific launch date, with a specific internal evaluation that somebody decides is close enough. On current form, that moment lands somewhere in the agentic systems now moving into production, where a system that books flights and executes code is much harder to gate than a chatbot. The dates OpenAI has floated for that threshold have a habit of moving; every AGI timeline OpenAI has given, and what it’s worth is a useful corrective to anyone treating the next announcement as a schedule. Governance is what happens in the gap between the announcement and the launch, and it is being written now, mostly in private, by about a dozen people.