Model releases used to feel interchangeable. A new version dropped, the chat window got marginally better at following instructions, and everyone moved on within a week. That rhythm broke somewhere around late 2024, when labs started shipping systems that spend extra compute thinking before they answer.
Now a launch post leads with reasoning depth, test-time compute budgets, and how many hours an agent can run unattended before it drifts. The vocabulary changed because the products changed. If you’re trying to work out which of the new AGI models deserve your attention, the marketing pages won’t get you far. Here’s what is genuinely different, and how to separate capability from packaging.
What an “AGI model” actually means in a release note
No lab claims it has shipped AGI. What they ship are general-purpose models with visible reasoning traces, reliable tool use, and long-horizon task handling. Those capabilities used to sit on the AGI checklist as future milestones. Now they’re product features with a price per million tokens.
The label has drifted to cover frontier systems that plan across steps rather than autocomplete a reply. That drift causes real confusion, and it now shapes how every release gets framed. If you want the background on the definition fight and the reasoning boom, that piece covers why the goalposts keep moving.
The reasoning-first release cycle
Test-time compute became the real scaling knob
Instead of only training bigger models, labs now sell a dial for how long the model thinks. OpenAI’s o-series, Gemini’s thinking budgets, Claude’s extended thinking, DeepSeek’s R1. Different packaging, same underlying idea: more tokens spent reasoning, better answers on hard problems, with a cost trade-off you control per request.
The practical consequence is that the same base model can feel mediocre or excellent depending on the budget. A problem that fails at 2,000 thinking tokens often solves cleanly at 20,000. Benchmarks reported at maximum budget can look nothing like the default experience a user gets in your app.
Context windows stopped being the headline
The million-token announcements still land, but retrieval quality inside a huge window is what people actually notice. Models that hold attention across 200 pages exist. Models that don’t lose the thread at page 90 are rarer. Test the middle of a long document, not the beginning, because that’s where attention sags.
Benchmarks tell you less than the launch blog suggests
Every release ships with a chart that puts the new model on top, usually against a competitor version that launched months earlier, evaluated at a setting you can’t easily reproduce. It isn’t fraud. It’s selective framing, and it makes leaderboards close to useless for choosing a model.
Separating genuine capability jumps from benchmark gaming is a running theme in what’s real, what’s hype, and what’s coming next. The short version: gains on saturated tests like MMLU say almost nothing now. Contamination is widespread, and many of those questions have been in training data for two years.
Agents are where the gap actually shows up
Single-turn quality converges fast between labs. Ask five frontier models a tricky legal question and the answers will be broadly comparable. Ask them to fix a failing test suite across twelve files, or reconcile a messy spreadsheet with three conflicting formats, and they separate quickly.
- Does it recover when a tool call fails, or does it loop until the context runs out?
- Can it hold a plan across 40+ steps without quietly abandoning the original goal?
- What happens when it hits a file format it can’t parse?
- Will it stop and ask for input, or confidently invent a path forward?
Those behaviours matter more than any score. They’re also where most demos fall apart, which is why launch videos tend to cut after ninety seconds.
The cost curve nobody puts in the headline
Reasoning tokens are expensive. A model that thinks for 30,000 tokens on every request can cost ten to twenty times what a standard call does, and latency climbs with it. Several recent releases cut per-token prices while pushing far more tokens per task, so the bill per completed answer moved less than the sticker price implied.
If you’re evaluating new AGI models for production, model the cost per finished task rather than per million tokens. The cheap model that needs three retries is usually the expensive one. Add in the human review time that a higher failure rate creates, and the ranking often flips.
What to test in the first week
Skip the leaderboard. Pull your own hardest thirty examples, the ones your current model fumbles, and run them through cold. Then check a few things that charts won’t show you:
- Refusal behaviour on legitimate-but-edge-case requests, which varies wildly between labs
- Instruction-following under long, messy, contradictory prompts
- Latency at realistic thinking budgets, not the default setting
- Consistency across five runs of the same prompt
Variance catches people out. A model that’s brilliant on attempt one and incoherent on attempt three is worse for automation than a slightly duller model that behaves the same way every time.
Where the next round of releases is pointed
Three threads look likely to define the next batch. Longer autonomous runs with checkpointing and recovery, so an agent can resume rather than restart. Better calibration, meaning models that flag uncertainty instead of bluffing. And cheaper reasoning through distillation, where a smaller model gets close to frontier thinking quality inside a narrow domain.
Research direction is a reasonable proxy for what lands in products six to twelve months later, though the signal-to-noise ratio is rough; what’s actually moving the needle is a useful filter. Threshold tracking is the other angle worth following, since which milestones genuinely moved gives a steadier read than any single launch announcement.
A sane way to pick a model this quarter
Pick two, not six. Run your eval set. Keep whichever fails least on the tasks you actually ship. Re-run the test when the next release lands, because the ranking rotates roughly every six weeks and nobody’s marketing page will tell you that.
Lab loyalty is a bad strategy right now. The models are close enough that switching costs are mostly integration work, and the leader changes hands often. Build the eval harness once, point it at whatever ships next, and let the numbers make the call for you.