Picture the overnight queue at a mid-sized online retailer: 600 support tickets, and most of them are the same five problems. A human team clears it by 11am. An agentic AI system is the bet that software can clear most of it by 6am, without anyone writing an if/else branch for every possible flavour of complaint.
That bet is being placed in a lot of companies right now, and the results are genuinely mixed. Some deployments save real hours every week. Others quietly burn through API credits and produce confident nonsense. The difference rarely comes down to which model you picked.
What makes an AI system “agentic”?
A normal chatbot takes a turn. You ask, it answers, done. An agent takes a goal and keeps working until something stops it: reads the request, decides what it needs, uses a tool to get it, looks at the result, adjusts, repeats.
The working definition most teams settle on is that an agent is a language model plus tools, plus memory, plus a stopping condition. Take away any one of those and you have something less interesting. A model without tools can only talk. A model with tools but no memory repeats the same mistake four times in a session. A model with no stopping condition will loop happily until you notice the bill.
Even a very good chat assistant sits on the far side of that line. A clear-eyed breakdown of what ChatGPT does well and where it misfires makes the point neatly: it responds, it doesn’t pursue. An agent pursues.
The three parts that decide whether it works
Tools are the whole ballgame
An agent is only as capable as the interfaces you hand it. Give it a clean lookup_order(order_id) and a clean issue_refund(order_id, amount) and it will do surprisingly well. Give it a sprawling internal admin panel with 40 fields and vague labels and it will find creative ways to break things.
This is why so much agent work in practice is API work. Teams spend three weeks tightening endpoints and one afternoon on prompts. That ratio is normal and it’s the right one.
Memory is where budgets go to die
Every step in a loop means re-sending the conversation, the tool outputs and the instructions back through the model. A long research task with 30 tool calls can push six figures of tokens through one run. Storage is cheap; context windows are not.
Planning is mostly about knowing when to stop
The impressive-sounding part of agentic AI is the planning. The useful part is the exit condition. Good agents have hard caps on steps, explicit success criteria, and a defined handoff to a human when they can’t confirm the outcome.
Where agentic AI is genuinely earning its keep
Coding is the clearest success story. On SWE-bench Verified, a benchmark built from real GitHub issues, the top agents went from resolving a couple of percent of tasks in early 2023 to comfortably above 70% by late 2025. That shift is why engineering teams now routinely hand off dependency upgrades and test backfills to an agent and review the pull request afterwards.
The second strong area is multi-step research: pull the filings, cross-check the numbers, summarise with sources. Third is document plumbing, the kind of work an AI reader built to extract and summarise from long documents handles well when the input is messy PDFs and the output needs to land in a structured field.
Capability is also moving quickly enough that yesterday’s ceiling is today’s floor. The 2025 model releases and what they unlocked are a decent snapshot of how fast tool-calling reliability has improved, because a model that fumbles a JSON argument is useless as an agent no matter how well it reasons.
Where it falls over
Compounding error is the big one, and it’s arithmetic rather than opinion. A step that succeeds 95% of the time sounds excellent until you chain ten of them: 0.95 to the tenth power is roughly 0.60. Two in five runs end somewhere you didn’t plan. Push a workflow to 20 steps and the maths gets ugly fast.
Prompt injection is the second. An agent with read access to email and a browser will one day read a message telling it to forward the inbox somewhere. If you haven’t designed for that, it will comply.
The third is silent failure. An agent that crashes is easy. An agent that reports “refund issued” when it only drafted the request is dangerous, and it happens constantly because the model is optimised for completing the narrative, not for verifying reality.
The costs nobody puts in the spreadsheet
Agent runs consume somewhere between 10 and 50 times the tokens of a single chat exchange, thanks to retries, tool round-trips and growing context. Latency follows the same curve: a five-minute run is normal, and users who expected two seconds will notice.
Then there’s the operational overhead. Traces, evals, alerts, cost caps, rollback plans. This is the same discipline that AI ops teams are learning to run for the rest of the stack, and agents are the noisiest thing on it because every failure has a unique, unreproducible shape.
Do you actually need an agent?
Be honest about this before you build. Agents are the expensive, fragile option, and a plain workflow with a model call in the middle solves most problems.
- Yes, probably: the number of steps varies per case and you can’t predict the path in advance.
- Yes, probably: the task needs live data from two or more systems that were never designed to talk.
- No: the steps are fixed. Write a pipeline, put a model call where judgement is needed, ship it in a week.
- No: a wrong answer costs money, reputation or safety and you can’t put a human approval gate in the middle.
- No: volume is under a few hundred runs a month. The build and maintenance cost won’t clear.
Shipping one without regrets
Scope beats cleverness every time. Pick one workflow, one tool set, one measurable outcome. Log every trace from day one, then build an evaluation set of 50 to 100 real cases pulled from actual history rather than invented examples. Measure the human baseline before you switch anything on, so you have something to compare against besides vibes.
Put approval gates on anything irreversible: payments, deletions, outbound email. Set a hard step cap and a per-run cost ceiling. And read the post-mortems from teams that have already deployed production bots and learned what breaks at scale, because the failure modes repeat across industries with almost boring regularity.
What week one should actually look like
Choose the workflow with the highest volume and the lowest blast radius. Refund triage, ticket classification, invoice matching, log investigation. Something with a few hundred cases a week and a wrong answer that costs a few pounds, not a lawsuit.
Then instrument it before you optimise it. Capture inputs, tool calls, outputs, latency and cost per run. Run 100 historical cases through it with a human checking every result, and write down each failure in plain language. That list is your actual roadmap, and it will look nothing like the architecture diagram you drew last month.
Most teams find the same thing in week one: the model was never the bottleneck. The tools were sloppy, the instructions were ambiguous, and nobody had defined what success looked like. Fix those three and agentic AI stops being a demo and starts being a system your team trusts enough to leave running overnight.