In March 2023, GPT-4 cleared the bar exam around the 90th percentile. That same month it scored roughly 2% on ARC-AGI, a set of visual puzzles built to be easy for humans and hard for machines. Both numbers were true. That gap, rather than either number alone, is what AGI progress toward human intelligence really looks like: fast, uneven, and full of sharp edges.
Most public argument about AGI quietly flattens those edges. Optimists point at the bar exam. Skeptics point at the puzzles. Neither tells you much about whether a system will be able to take over a messy, week-long piece of work, or learn a new skill from three examples the way a person does.
“Human-level” is doing a lot of hidden work
Human intelligence is not a single score you can beat. It is a bundle: recognising a friend in bad light, learning to ride a bike, planning a holiday, reading a room, and remembering that your colleague hates being interrupted. Each of those draws on different machinery.
Benchmarks measure thin slices of that bundle. They are useful, but a model that maxes out multiple-choice reasoning tests has demonstrated something narrow and real, not something general.
The jagged frontier
Researchers at Stanford coined the phrase “jagged frontier” for good reason. Modern models write competent Python and decent sonnets, then fail to count the letters in a word or notice that a riddle depends on a pun. Capability is not a smooth curve climbing toward human ability. It is a coastline, with deep inlets next to high cliffs. Any honest assessment of AGI progress has to describe the shape of that coastline, not just its average altitude.
The benchmarks that fell, and the ones that didn’t
Around 2023, MMLU, a 57-subject knowledge test, was considered a serious hurdle. Within roughly eighteen months frontier models were scoring in the high 80s and low 90s, near or past expert human baselines, and labs stopped leading with it. Saturation like that is real progress. It also means the test stopped being informative.
The replacements got harder and more specific. GPQA Diamond, written by PhDs to be Google-proof, saw scores climb from the 30s to the 70s-plus in about a year. FrontierMath, a set of research-level maths problems, kept top models below 2% for a while before that wall also started to crack. ARC-AGI-1, the puzzle set from 2019, went from 2% to 87.5% in late 2024, though only with heavy test-time compute and at a cost per puzzle no human would consider reasonable.
That last detail matters. A score achieved by spending hundreds of dollars of inference on a single item is a different claim from a capability you can hand to a user at a fraction of a cent. Progress is real; the economics of it are part of the measurement.
The most useful yardstick so far: how long a task a model can finish
Benchmark percentages are brittle. A more informative metric comes from METR, a non-profit that measures the length of tasks, in human time, that models can complete at roughly 50% reliability. On their suite of software, research and operations tasks, the number has been doubling roughly every seven months.
In 2023 that horizon was measured in seconds of human work. By 2025 it had stretched into the range of an hour or two. Extrapolate the curve and you get autonomous multi-day projects in the early 2030s. Treat the curve with suspicion and you notice it is built on a modest number of tasks, that the tasks skew toward software, and that the 50% threshold hides a lot of failure.
The flip side of a 50% horizon
Half of tasks at that horizon fail. If you hire a contractor who finishes half of what they start, you do not have an employee. You have a very fast intern you must supervise constantly. Reliability, not peak capability, is the binding constraint on most real deployment, and it is the part of AGI progress toward human intelligence that moves slowest.
Where the distance to human intelligence is widest
- Sample efficiency. A child learns a new word from a handful of exposures. Fine-tuning a model on a new concept typically takes thousands of examples, and in-context learning fades over long contexts.
- Continual learning. Models do not accumulate skill from experience. Each session starts from the same weights. Humans get better at their job every week; today’s models mostly do not.
- Causal world models. Ask a model what happens to a glass knocked off a table and it answers correctly. Ask it to predict the outcome of an intervention no one has written about and the reasoning gets shaky.
- Long-horizon coherence. Agents drift. Give one a twenty-step task with tools and check in at step nineteen; something has usually gone sideways.
- Embodiment and physical intuition. Robotics has improved quickly, but a general-purpose robot that handles an unfamiliar kitchen is not here yet.
- Knowing what it doesn’t know. Calibration remains poor. Models state wrong things with the same confident register as right ones.
None of these is a minor footnote. Continual learning in particular is the difference between a tool you query and a colleague who improves.
It was never only about the model
A large share of visible capability gains since 2023 came from the wrapper, not the weights. Tool use, retrieval, scratchpads, better prompting, agent scaffolds that let a model check its own work against a test suite. Change the harness around the same base model and measured performance can swing dramatically.
This is why comparing lab announcements is so treacherous. Two teams can report headline numbers on the same benchmark using very different amounts of scaffolding and compute. The labs are chasing capability, distribution and talent at the same time, and the real shape of the AGI competition explains why so much of what gets published is shaped for investors as much as for researchers.
Regulation is now part of the picture too. Disclosure rules, compute thresholds and evaluation requirements are being written while the models they govern are still in training, and the pace of AGI policy change has started to affect what labs are willing to ship and when.
Tests that would actually move the needle
If you want to judge AGI progress toward human intelligence rather than be impressed by it, look for evidence on a few specific questions.
Can it learn on the job?
Give a system a novel task, correct it twice, and see whether the correction sticks across sessions without retraining. Very little of what ships today passes this.
Can it handle an eight-hour day?
Not a demo, not a benchmark run. Eight hours of ambiguous, interrupt-driven work with a human checking in occasionally, at a cost per hour below a junior salary. That is the threshold where the labour-market conversation stops being hypothetical.
Does it hold up off-distribution?
Benchmarks get contaminated. Models train on the internet, and the internet contains the answers. Genuinely held-out evaluations, written after the model was trained, are the only ones worth trusting at face value.
Reading the forecast without getting sold a story
Titles like “AGI by 2027” are almost always doing two things at once: describing a technical trend and marketing a company. The trend is real. The date is a guess dressed as a projection. It helps to separate the two, which is what a careful guide to reading AGI forecasts without being fooled is built for.
Longer-horizon bets deserve the same scrutiny. The case for AGI arriving by 2040 rests on continued scaling, falling inference costs and algorithmic gains that have not yet shown signs of stalling. The cracks are in reliability, energy and data. Both sides of that ledger are worth understanding before you pick a camp.
If you want to watch the field directly rather than through press releases, start with the primary literature. Knowing which AGI research papers are worth your time, and how to skim them in twenty minutes, will tell you more about the true state of capability than a year of headlines.
The honest picture is that the gap to human intelligence is closing fast in pattern recognition, knowledge retrieval and bounded reasoning, and barely moving in continual learning, calibration and long-horizon reliability. When someone tells you AGI is near or far, the useful follow-up is not “when” but “which gap are you talking about” and “what did the test actually measure.” Those two questions cut through most of the noise.