Hand a frontier model a stack of receipts, a spreadsheet with missing rows and a request to file an expense claim, and it will mostly get there. Ask the same model to keep that claim accurate across forty steps, remember a correction you made last Tuesday, and notice that one receipt contradicts the others, and things fall apart fast. That distance between what an AI can do once and what it can do reliably is where most of the real argument about AGI capabilities lives.
The phrase itself gets thrown around with very little precision. Sometimes it means “a system that can do anything a person can.” Sometimes it means “a system that can do my job.” Sometimes it is just shorthand for “the next model.” Getting clear on what the term actually covers makes it much easier to judge the claims coming out of labs every few months. If you want the foundational version first, this plain-English explanation of what AGI means is a good place to start.
Capability isn’t the same thing as skill
A calculator has a skill: arithmetic, and only arithmetic. A chess engine has a skill too, one it performs far beyond any human and cannot transfer to anything else. Capability is a different idea. It refers to the underlying ability to acquire new skills, including ones nobody trained you for.
François Chollet, the researcher behind the ARC benchmark, makes this distinction sharply. He separates skill, which is task-specific performance you can buy with more data and compute, from generalisation, which is how efficiently a system adapts to something genuinely novel. By that measure, a model that scores 95% on a graduate exam it has effectively seen before demonstrates skill. Whether it demonstrates capability depends on what it does with the exam it has never seen.
This matters because benchmark scores have been climbing for years while the underlying question, can the system handle a genuinely new situation, has moved much more slowly. The mechanisms behind that gap are worth understanding in their own right, and this breakdown of how general-purpose AI actually works covers the architecture side in detail.
The capability checklist researchers keep coming back to
There is no agreed scorecard, but the same components show up in almost every serious framework. A system with AGI-level capabilities would need, at minimum:
- Few-shot learning. Picking up a new task from a handful of examples or a short explanation, without retraining or fine-tuning.
- Transfer. Applying an insight from one domain to a completely unrelated one, the way a physicist borrows a technique from economics.
- Long-horizon planning. Holding a goal in mind across hundreds of steps and keeping the plan coherent as conditions change.
- Continual learning. Retaining what it learned last week without a full retrain, and without forgetting something else in the process.
- Grounded perception. Reasoning about the physical world from images, audio, sensor data or direct interaction, not just text.
- Calibrated self-knowledge. Recognising the edge of its own competence instead of confidently inventing an answer.
- Autonomous tool use. Deciding when to search, run code, call an API or ask a person, and doing it without hand-holding.
Score a current frontier model against that list and you get a patchwork: strong on some lines, effectively zero on others.
Where the capability story is strongest
Coding is the clearest example of real progress. HumanEval, a set of 164 introductory Python problems released in 2021, went from a meaningful test to a formality in about two years. Every frontier model now clears 90%, and nobody uses it to distinguish systems anymore. On SWE-bench Verified, which pulls real bug reports from open-source repositories, top systems went from under 2% in 2023 to the 70s by late 2025.
Translation, summarisation and document extraction are broadly solved for everyday use. Multimodal reasoning has improved sharply as well: a model can now look at a photograph of a damaged component and describe the failure mode, or read a scanned invoice and reconcile it against a purchase order. For a fuller inventory of what already works in production, this survey of general-purpose AI’s current strengths and limits is useful context.
Tool use deserves its own mention. The shift from answering questions to taking actions, searching, running code, editing files, checking its own output, has done more for practical capability than most raw model upgrades. An agent with a browser and a Python interpreter can do things no chatbot could manage two years ago.
The transfer problem in one benchmark
ARC-AGI is designed to resist memorisation: coloured-grid puzzles that require inferring a rule from two or three examples. GPT-4 scored 13.5% on the public evaluation set in 2023, against a human baseline of roughly 85%. In December 2024, OpenAI’s o3 reached 75.7% on the same set, though reportedly at a compute cost in the tens of dollars per task. That jump is real and it is also a warning: when a benchmark falls, the honest question is whether capability improved or the test got absorbed into the training pipeline.
The gaps that still define the frontier
Long-horizon reliability
This is the big one. If each step in a process succeeds 95% of the time, a twenty-step task succeeds about 36% of the time. Real work is made of long chains, and error compounds quietly. UK research institute METR has tracked the length of task an agent can complete with 50% reliability and found it doubling roughly every seven months since 2019. Extrapolate that curve and you reach a full working day of autonomous task completion somewhere in the late 2020s. Extrapolation is not a guarantee, but the trend is one of the more concrete signals available.
Continual learning
Models do not accumulate experience the way people do. A system that helps you for six months does not get better at helping you unless someone builds around that limitation with retrieval, memory stores or fine-tuning. Every conversation starts close to cold, and the workarounds are engineering patches rather than genuine learning.
Physical and causal grounding
Ask a language model which way a stack of objects topples and it will reason from text descriptions of physics rather than from an internal model of how objects behave. That works often enough to be convincing and fails in ways that are hard to predict. Robotics has improved, but embodied competence remains years behind conversational fluency.
Knowing when it doesn’t know
Calibration is underrated. A system that says “I’m not sure, here are two possibilities” is far more useful in a hospital or a law firm than one that is confidently wrong 4% of the time. Progress here has been incremental, mostly through better uncertainty prompting and tool-based verification rather than any fundamental fix.
Benchmarks are lagging the claims
FrontierMath, a set of research-level problems where the previous state of the art was around 2%, saw o3 reach 25.2% in late 2024. GDPval measures whether models can produce deliverables that experts in real occupations would accept. These are better tests than trivia questions, and they are still proxies. The step from benchmark to deployed capability stays large, which is why the distinction between AGI and superintelligence matters: crossing one threshold does not put you anywhere near the next, and treating them as a single curve is how forecasts go wrong.
What capability progress looks like in science
The most persuasive evidence for genuinely general capability is not a chatbot demo, it is a research result. AlphaFold predicted structures for roughly 200 million proteins, a body of work that would have taken experimental labs decades. Demis Hassabis has been explicit that this is the point of the exercise, arguing that AGI’s real payoff is compressing scientific discovery timelines from years to months. His remarks on using AGI to tackle disease are a useful corrective to the productivity-tool framing that dominates most coverage.
That is what a capability jump looks like: not a model that talks better, but a model that produces something experts could not produce at that speed before.
Signals worth watching over the next two years
Ignore the demo reels. The indicators that carry information are narrower. Does an agent finish a multi-day task without a human stepping in? Do models improve on a benchmark that was published after their training cutoff? Can a system learn a new skill from one example and apply it a week later? Does it say “I don’t know” when it doesn’t?
A single yes on any of those is interesting. Several together, repeated across independent evaluations rather than vendor blog posts, would be the point at which the word “capability” stops being a marketing term and starts describing something that has actually changed.