Ask a standard language model a tricky scheduling puzzle and it answers in about a second. Ask a reasoning model the same question and it may sit there for forty seconds, burn through a few thousand hidden tokens, then produce an answer. Sometimes that answer is genuinely impressive. Sometimes it’s the same wrong answer as before, only slower and more expensive.
That pause is the whole story of reasoning AI. The model isn’t retrieving a fact or pattern-matching a template. It’s generating a long private scratchpad of intermediate steps and, in theory, using that scratchpad to catch its own mistakes before you ever see the output.
What “reasoning AI” actually refers to
The label gets thrown around loosely, so it helps to be precise. Reasoning models are trained to produce a chain of intermediate steps before committing to a final answer, and they’re trained with reinforcement learning on problems that have checkable answers: maths with a definite result, code that passes or fails tests, logic puzzles with one valid solution.
That last part matters. You can’t reward a model for “thinking well” in the abstract. You can reward it for getting the right answer to a competition maths problem, and let it discover that backtracking, double-checking, and trying a second approach improves its hit rate. The behaviour that looks like deliberation is a side effect of that optimisation.
It is not the same as general intelligence
A model that can solve an olympiad geometry problem in ninety seconds may still fumble something a sharp twelve-year-old would never get wrong. Reasoning ability on benchmark tasks and genuine general capability are different things, and it’s worth keeping the distinction straight. If you want a fuller breakdown of that boundary, there’s a good piece on where narrow AI ends and general AI begins that avoids the usual hype.
Why thinking longer improves accuracy
The core mechanism is usually called test-time compute, or inference-time scaling. Traditional scaling laws said: bigger model, more training data, better results. Reasoning models added a third lever, which is how many tokens the model spends thinking at the moment you ask.
The gains are not marginal. When OpenAI launched its first o1 preview in September 2024, it scored around 56% on the AIME 2024 maths exam versus roughly 12% for GPT-4o on the same paper. DeepSeek’s open-weight R1 landed near 80% a few months later. These are benchmark scores, not proof of genius in the wild, but the shape of the curve is real: more thinking tokens, higher accuracy, up to a point of diminishing returns.
Where reasoning models earn their keep
There are tasks where the extra thirty seconds is obviously worth it, and tasks where it’s pure waste.
- Multi-step arithmetic with odd rules. Freight cost calculations with tiered rates, fuel surcharges, and regional exceptions. A standard model tends to drop a step; a reasoning model usually doesn’t.
- Debugging code where the bug is three calls deep. Tracing state through a chain of functions is exactly the kind of problem where writing out intermediate steps helps.
- Conflicting instructions. Extraction tasks where two rules in a policy contradict each other. Reasoning models are more likely to notice the conflict and flag it rather than silently pick one.
- Scheduling under constraints. Fifteen meetings, four rooms, three people who can’t be in the same room. Constraint satisfaction is a natural fit.
- Reading diagrams and tables. Combined with vision input, reasoning models are noticeably better at interpreting a wiring diagram or a dense financial table than a plain vision model. The shift toward multimodal AI is what made that combination practical.
Where they still trip
Three failure modes show up again and again.
Overthinking the easy stuff. Ask a reasoning model for the capital of France and it may spend 900 tokens working through it. Worse, occasionally the long chain introduces doubt that wasn’t there, and a model that knew the answer cold talks itself into hedging.
The chain of thought is not an audit trail. You usually can’t see the raw reasoning, and even when you can, it isn’t a faithful record of how the answer was reached. Models have been observed writing reasoning that justifies a conclusion they’d already settled on. Treating the visible chain as an explanation is a mistake people make once.
Confident errors in domains without ground truth. Reinforcement learning works because maths and code have verifiable answers. Ask a reasoning model to evaluate whether a business strategy is sound and you get a beautifully structured argument built on a shaky premise, delivered with total confidence.
The cost and latency maths nobody mentions in demos
Thinking tokens are billed as output tokens. That’s the detail that kills a lot of first deployments. If output costs $15 per million tokens and a request burns 8,000 reasoning tokens before producing a 400-token answer, you’re paying roughly 12 cents per call instead of a fraction of a cent. At 50,000 calls a month, that’s a real line item.
Latency compounds it. Forty seconds of thinking is fine for a batch job running overnight. It’s hopeless for anything a person is waiting on. Most production systems end up routing: a cheap fast model handles the easy 80%, and anything that fails a confidence check escalates to a reasoning model. That routing layer is where the engineering effort actually goes.
Prompting a reasoning model is a different job
Advice written for standard chatbots actively hurts here. Adding “think step by step” to a reasoning model is redundant at best and counterproductive at worst, because you’re imposing your structure on a process it has already learned.
What works instead:
- State the problem cleanly and stop. Concise problem statements outperform elaborate prompt scaffolding.
- Give the constraints and the definition of done. “Return JSON, one row per invoice, flag any total that doesn’t reconcile within $0.02.”
- Put the data in a clean, structured form. Reasoning over a tidy CSV beats reasoning over a paragraph of prose describing the same numbers.
- Tune the thinking budget. Most APIs expose a reasoning-effort setting, and dropping it for routine work is the single biggest cost lever you have. If you’re wiring this up yourself, the walkthrough on building with the OpenAI API covers the parameters and the production gotchas.
How to tell whether you actually need one
Skip the vendor benchmarks. Build a twenty-question test set from your own work, things you already know the right answer to, and run both a standard model and a reasoning model against it. Score correctness, wall-clock time, and cost per correct answer. Cut the test set roughly in half into easy and hard questions so you can see where the reasoning model’s advantage actually lives.
In most cases the pattern is stark. On the easy half, the standard model matches or beats it at a fraction of the cost. On the hard half, the reasoning model wins by a margin wide enough to justify the price. That split, more than any leaderboard, tells you how to architect the system. If you want a structured way to run that comparison across several providers, the 45-minute model bake-off method is a solid template.
One last habit worth building: verify the output mechanically wherever you can. Run the generated code. Recompute the totals with a formula. Check the extracted dates against the source. Reasoning AI reduces the rate of dumb errors but doesn’t eliminate them, and a cheap verification step catches the ones that slip through far more reliably than reading the model’s explanation and trusting it.