Every few weeks the same pattern repeats. A lab posts a demo, the timeline fills with screenshots, a researcher posts a caveat thread, and by Friday the argument has moved on to something else entirely. Following AGI news is exhausting precisely because the signal is real but the volume is turned up to eleven.
So here’s a more useful way to track it. Not as a countdown clock, but as a set of specific, checkable developments: which benchmarks are cracking, what the people building these systems say in unguarded moments, where the capital is flowing, and which experiments expose the gap between a demo and something you could actually hand a job to.
The benchmarks keep falling, and the goalposts keep moving
For years, the ARC-AGI test was the industry’s favourite punching bag for overclaiming. Frontier models scored in the low single digits on it. Then in late 2024, OpenAI’s o3 posted a reported score near 76% on the semi-private evaluation set under a high compute budget, and a large chunk of the “AI can’t do novel reasoning” argument had to be rebuilt overnight.
The same thing happened with Humanity’s Last Exam, a roughly 3,000-question gauntlet assembled by hundreds of subject experts that launched in early 2025. Early models landed under 10%. Within months, top reasoning systems were climbing well past that, and the benchmark’s authors were openly discussing what would replace it.
This is the recurring arc, and it’s worth understanding as a pattern rather than a series of shocks. Every major benchmark so far has eventually been beaten, usually faster than the people who designed it expected. The interesting question is never “did the score go up” but “did the thing the score was supposed to measure get better, or did the model just get better at the test”.
Independent evaluators have become the most reliable source of AGI news for exactly this reason. METR’s work on task-completion time horizons, which estimates how long a task a model can finish reliably, has been one of the more sober measures around. Its doubling trend has held for years, though the exact rate has been revised in both directions as methodology tightened.
What the lab leaders actually say when the cameras are off
Public keynotes are marketing. Conference talks and podcast interviews are closer to the truth, and the disagreements there are sharper than most coverage admits.
Ilya Sutskever used his NeurIPS 2024 appearance to argue that pre-training as the field has known it is winding down, that data is finite and returns from simply adding more of it are flattening. Rich Sutton has made a harder version of the same case: that imitation-based language model training is a dead end and something closer to genuine reinforcement learning from experience has to take over. Dario Amodei sits at the other end, sketching a compressed timeline for what he calls powerful AI, with capability arriving in a handful of years rather than decades.
These are not minor academic quibbles. They imply completely different research agendas and completely different spending priorities. Keeping up with the field’s live research problems is more informative than any single benchmark result, because it tells you where the next bottleneck will surface.
Follow the capital, because it’s louder than the roadmaps
Talk is cheap. Data centre construction is not.
The Stargate announcement put half a trillion dollars on the table for US AI infrastructure across a multi-year build-out. Combined hyperscaler capital expenditure has climbed past $300 billion annually, with a large share going to accelerators, cooling, and power contracts. Utilities in several US states have revised load forecasts upward for the first time in a decade because of AI demand.
You can read that two ways. Either it’s a rational bet that capability scales with compute, or it’s the largest capital misallocation in recent tech history. What you cannot do is dismiss it. Money on that scale reshapes power markets, chip supply chains, and the number of labs that can compete at the frontier at all.
Agents are where the hype meets the pavement
If you want a genuine read on where things stand, ignore the benchmark charts and watch the agent experiments. They’re messy, public, and unusually honest.
Anthropic let a Claude model run a small physical shop in an office for about a month, handling inventory, pricing, and vendor email. It lost money, got talked into discounts, and once tried to buy something from a human pretending to be a supplier. OpenAI’s GDPval and similar evaluations test real deliverables across occupations, and the results show a jagged frontier: strong on drafting and synthesis, brittle on anything requiring sustained state tracking or judgement under ambiguity.
That jaggedness is the actual story. Today’s models are genuinely capable at a wide band of knowledge work, and they break in ways that are easy to predict once you’ve spent a week with one. The failures cluster around long horizons, self-correction without external feedback, and knowing when to stop.
Which is why the real limitations rarely show up in a launch video. Memory that decays over a long session, confidence that doesn’t track accuracy, evaluation suites that leak into training data. Unglamorous problems, and the ones most likely to determine whether the next two years look like the last two.
How to read an AGI headline without whiplash
A short filter that saves a lot of wasted attention:
- Who measured it? A vendor-reported score on a benchmark the vendor helped design deserves a raised eyebrow. An independent replication deserves your time.
- What was the compute budget? Numbers from a best-of-many-attempts, high-compute run don’t describe the product you can use today.
- Is it a demo or a deployment? Demos are curated. Deployments generate support tickets. Ask which one you’re looking at.
- Did anything ship? A paper is a hypothesis. An API endpoint with a price per million tokens is a claim about the world that someone is willing to lose money on.
- What’s the failure mode? Anyone honest about a system will tell you how it breaks. If nobody will, that’s the headline.
It also helps to know roughly how these systems are actually built, since a surprising number of breathless stories dissolve once you understand what pre-training, post-training, and inference-time compute each contribute.
The signals worth watching over the next year
Three things will tell you more than any single model release.
First, whether the time-horizon curves keep bending. If models can reliably complete multi-day tasks without supervision, a lot of assumptions about white-collar work change. If the curve flattens, the current spending levels start to look uncomfortable.
Second, whether the memory problem gets solved in production rather than in demos. Persistent, reliable context across weeks is the difference between a clever assistant and a colleague.
Third, whether evaluation keeps up. Right now the field is running on tests that saturate within months, and a benchmark that everyone has already beaten tells you nothing. Whoever builds a durable measure of general capability will shape the next round of AGI news more than any lab’s launch event.
Until then, treat the cycle with a bit of scepticism and a bit of genuine wonder. Both are warranted. The systems really are getting better at a pace that would have seemed implausible in 2022, and the gap between what they do in a demo and what they do on a Tuesday afternoon is still wide enough to matter.