Last spring a support team I know shipped an agent that resolved 71% of test tickets without a human. Six weeks later the same agent was closing 34%. Nobody had swapped the model. Nobody had touched the prompts. What changed was the traffic: real customers asked about refunds on orders from a marketplace the company had acquired two years earlier, and the agent had never seen that schema.
That gap between a clean pilot and a messy Tuesday afternoon is where any honest discussion of AI agent limitations should start. The interesting question isn’t whether a model can plan a task. It’s where the plan falls apart once the world stops cooperating.
Small per-step errors compound fast
A step that works 95% of the time sounds close to solved. Chain twenty of them and you land at roughly 36% end-to-end, because each hop multiplies the odds instead of adding them. That arithmetic is why agents feel magical in a three-step demo and useless in a fifteen-step workflow.
Take an agent that rebooks a delayed flight. It pulls the itinerary, checks the new schedule, confirms seat availability, applies the fare rules, holds the reservation, charges the card, and emails the receipt. Seven steps. At 95% per step you get about 70%. Drop to 90% and you’re under half. Real systems sit somewhere in between, and the failures rarely announce themselves: a fare rule gets skipped and the ticket quietly costs $400 more than it should.
The benchmarks show the same shape
On WebArena, which asks agents to complete realistic tasks across self-hosted websites, leading systems sat in the 30s for a long stretch before climbing. On SWE-bench Verified, the strongest coding agents resolve a little over 70% of issues; ask them to handle multi-file changes that span several hours and the numbers sag. The pattern repeats across domains. Agents are decent at short tasks with tight feedback and get dramatically worse as the horizon lengthens.
Agents are only as good as the surfaces they touch
Every tool call is a contract, and most of those contracts have no version history. A button gets renamed. A modal appears that nobody documented. Salesforce pushes a layout change over the weekend and your procurement agent starts clicking into the void.
Browser agents take the worst of this. Systems like Skyvern, which drives the web the way a person would, absorb a lot of interface churn by looking at the page rather than hard-coded selectors, but they still stumble on shadow DOM, CAPTCHAs, and anything behind a login that behaves differently for returning users. Approaches like Stagehand’s plain-English web automation cut down maintenance because the instructions describe intent rather than a CSS path. They don’t eliminate the problem. The page still has to mean roughly what the agent thinks it means.
Memory is still duct tape and string
Long context windows get marketed like memory, and they aren’t the same thing. An agent can hold 200,000 tokens and still forget instruction four of twelve by the time it reaches step nine, because attention over a long transcript is not the same as remembering what matters.
Retrieval patches some of this. It also introduces a new failure: the agent retrieves a policy document from 2023, treats it as current, and applies a refund rule the company retired last quarter. Agents learn about the world from scraped and licensed data, and that data has an expiry date. Some labs are now buying datasets from defunct startups to deepen domain coverage, which helps with breadth and does nothing about staleness.
Confident wrongness, and the missing “I’m stuck” reflex
Humans stall. We notice we’re circling, we ask a colleague, we escalate. Agents tend to keep going, and they do it with the same flat confidence whether they’re right or improvising. An agent that invents a customer ID and then builds six downstream actions on top of it is worse than one that simply fails, because the failure gets buried under a plausible-looking trail.
Some of this comes from how the computation stays hidden inside the model. Researchers kick around terms like opaque recurrence to describe the internal state a model carries between steps, and much of it remains difficult to inspect directly. You can log every tool call and still not know why the agent chose that tool.
Evaluating an agent is harder than building one
You can unit-test a function. Testing an agent means running it against a distribution of tasks that keeps shifting, scoring outcomes that are sometimes subjective, and doing it enough times to trust the variance. Most teams underbuild this and then can’t explain why a release that passed staging broke a customer segment.
Three things tend to go missing:
- Task coverage. Fifty hand-picked happy paths tell you almost nothing about the long tail that generates most of your support volume.
- Replay. Without recorded traces of real runs, every regression investigation starts from zero.
- Cost of failure. A wrong answer in a research tool is annoying. A wrong answer that issues a refund or deletes a record is a different category of problem, and it needs a different bar.
The economics: tokens don’t grow on trees
Agents think out loud, and you pay for every thought. A task that a human finishes in ninety seconds can burn 40,000 tokens once the agent loops through planning, tool calls, and self-correction. Multiply by a few thousand daily runs and the unit economics get uncomfortable fast. Latency compounds the same way. Users tolerate a slow chatbot for a minute; they don’t tolerate a slow checkout flow.
Permissions, prompt injection, and the trust boundary
Give an agent read access and it’s a search tool. Give it write access and you’ve handed a stranger your credentials plus a set of instructions it will follow too literally. Prompt injection remains largely unsolved: any text the agent reads, including an email body or a web page, can contain instructions that redirect it. The practical mitigation is boring and effective. Scope permissions narrowly, require human sign-off on irreversible actions, and log everything.
Where agents genuinely earn their keep
Narrow scope plus a human in the loop is the combination that ships. Enterprise conversational platforms have figured this out; when you look at what enterprises actually get from Kore.ai or read a realistic assessment of Yellow.ai, the working deployments are rarely autonomous. They handle triage, draft responses, pull account context, and hand off when confidence drops.
The jobs that hold up share three traits: fast verification (you know within seconds whether it worked), reversible actions, and a bounded toolset. Internal search, ticket routing, test generation, data cleaning, log triage. Boring tasks, real money saved.
What to demand before you hand over the keys
Ask for per-step accuracy on your own data, not a vendor benchmark. Ask what happens at step eleven when a dependency returns a 500. Ask for a kill switch and a full trace of every action taken. Then run the agent in shadow mode for two weeks against live traffic and compare its output to what your team actually did. You’ll learn more from the disagreements than from any evaluation suite.
The useful mental model is that an agent is a very fast junior colleague with perfect recall of everything it has read and no instinct for when it’s out of its depth. You’d never give that person unsupervised access to production on day one. Six months of supervised work, though, and they might genuinely run the queue.