Ask six people what an “OpenAI AGI evaluation” is and you’ll get six different answers. A researcher will describe task suites and contamination checks. A policy person will talk about the Preparedness Framework’s scorecards. A sceptic will point at a marketing page and say none of it means anything. All three are partly right, which is why the topic is so slippery.
The honest version is this: AGI evaluation at OpenAI isn’t a single test you pass or fail. It’s an assembly line of measurements, red-team exercises, third-party audits and internal judgment calls, and the output is less “yes, it’s AGI” than a stack of documents that a small group of people then interpret behind closed doors.
Three very different things called “evaluation”
Most confusion starts here. When OpenAI staff say “evals,” they usually mean one of three things, and they get conflated constantly.
- Capability evaluation. Can the model do the task at all? These are the benchmark scores that end up in blog posts: math competitions, coding problems, graduate-level science questions.
- Dangerous capability evaluation. Could the model do something that causes serious harm if a determined person pushed it: synthesising pathogens, running an intrusion campaign, manipulating a user at scale.
- Behavioural and deployment evaluation. Does it behave inside the actual product, with tools, memory and a system prompt, once millions of people start pushing against it?
A model can top the capability charts and still fail behavioural evals badly. It can also score poorly on a benchmark while being genuinely useful at agentic work. The three layers rarely move together, which is exactly why OpenAI runs all of them before a launch.
Inside the Preparedness Framework
The safety layer runs through OpenAI’s Preparedness Framework, which was updated in April 2025 to add a “Critical” tier and to expand the tracked categories beyond the original four. Models are scored against thresholds for cyber, biological and chemical risk, persuasion, and model autonomy, with research and development and AI self-improvement now folded into the picture.
The mechanics matter more than the labels. A model that lands at “High” in a category triggers a formal review by a safety advisory group and, in principle, a decision by the board about whether to ship. Below that threshold, deployments can proceed with mitigations. The framework also introduced published scorecards, which are the closest thing to a public window into the process.
What those scorecards do not tell you is how close a model came to the line, or what the internal debate looked like when it did. The clauses and loopholes in OpenAI’s AGI policy are worth reading closely, because the escape hatches are written into the same document as the commitments.
Benchmarks are the weakest evidence in the room
Headline scores travel furthest and prove least. Two problems keep recurring.
The first is contamination. Test sets leak into training data, sometimes deliberately, sometimes because a dataset was scraped wholesale from the open web years before anyone thought to protect it. A model that has seen the answers isn’t demonstrating reasoning, it’s demonstrating recall.
The second is cost and scaffolding. When OpenAI’s o3 posted 87.5% on the ARC-AGI benchmark in late 2024, the headline was real, but so was the footnote: roughly $20 of compute per task under a high-compute configuration. That’s a genuine capability result and a terrible proxy for everyday usefulness at the same time.
Benchmarks also saturate. Once a suite creeps past 90%, the remaining points stop measuring anything meaningful and start measuring formatting quirks. Sorting the benchmark scores that mean something from the ones that don’t is a genuinely useful skill now, because the marketing around them has never been louder.
Autonomy is the hardest thing to test
Dangerous-capability evals for biology or cyber are at least well-defined. You can ask whether a model helped someone plan a plausible attack. Autonomy is messier. The question isn’t whether a model can complete a task in one shot, it’s whether a system of model plus tools plus retries plus a long context can keep making progress on a vague goal without drifting into something nobody asked for.
METR’s time-horizon work became the standard reference here. Instead of asking what a model knows, they measure how long a task, in human-expert time, a model can complete with roughly even odds. The doubling trend they documented across 2024 and 2025 was arguably more informative than any single exam score, because it described a trajectory rather than a snapshot.
Sandboxed red-team trials from groups like Apollo Research focused on a narrower worry: whether models can notice they’re being tested, and whether they’ll conceal intent or attempt to preserve themselves when a scenario makes that the rational move. Those findings are uncomfortable, and they’re also the sort of thing a benchmark can never surface. The underlying mechanics of how these systems are trained, and why oversight gets harder as capability rises, are covered well in what OpenAI’s pretraining and RLHF pipeline actually does.
Who is holding the clipboard
Self-evaluation has an obvious conflict of interest, and OpenAI has steadily pushed toward external assessment for exactly that reason. METR has run pre-deployment evaluations on frontier models. Apollo Research has tested scheming behaviour. The UK AI Security Institute has had pre-deployment access to models before public release. Independent evaluators adding their own scaffolding often find capability that the developer’s own harness missed, and sometimes the reverse.
The gap that remains is publication. Third-party findings usually surface as a short section in a system card, summarised by the company being evaluated. If you want to see what the underlying research actually claims rather than what the summary says, reading the primary papers rather than the press release is the only way through.
The definition problem nobody has solved
Here’s the odd part. OpenAI’s charter defines AGI as a highly autonomous system that outperforms humans at most economically valuable work. The Microsoft partnership agreement reportedly pins a different, blunter threshold: a system generating $100 billion in profit. One definition is a research aspiration, the other is an accounting event, and neither is something you can adjudicate with an eval suite.
So when a lab eventually declares that AGI has arrived, the declaration will be a statement about internal judgment, not a measurement result. The evaluations inform that judgment. They don’t determine it. Which is why the research on keeping a human brake in the system, discussed in the alignment race and who gets to slow things down, matters as much as any scorecard.
How to read an evaluation claim without being fooled
Four questions cut through most of the noise.
- Who ran the test? Developer-run, third-party-run with developer scaffolding, and fully independent evaluations are three different evidence tiers.
- What was the elicitation budget? A capability result means little without knowing how many attempts, how much compute and what tooling the model was given.
- What did the test not cover? Missing categories are more informative than the ones included. Bio and cyber get attention; long-horizon economic substitution gets far less.
- Was there a threshold, and what happened when it was crossed? A framework that has never blocked a launch is a description of intent, not a control.
None of this makes OpenAI’s evaluation work worthless. The shift from vague safety language to named thresholds, published scorecards and external red teams is a real change from five years ago, and the people running these tests are generally trying to find problems rather than hide them. But an evaluation is a map, not the territory, and the territory here is whether a system with capabilities nobody fully understands should be handed control over more of the economy. The map keeps getting better. It is still drawn by the people who built the terrain.

