A year ago, “AI agent” mostly meant a chatbot with ambitions. You asked it to book a flight and it produced a lovely itinerary no airline had ever heard of.
That has changed, unevenly. Point a modern agent at a messier job, say reconciling two spreadsheets that should match but don’t, and it will open both files, spot the mismatched rows, write a short script to fix them, run it, watch it crash on a date format, and try again. Nobody wrote instructions for that last step. The model decided the script had failed and decided what to do about it.
That loop, act, look at what happened, decide again, is the entire idea. Everything else is engineering.
What separates an agent from a chatbot
A chatbot answers. An agent acts. In practice the difference is who picks the next step.
With a normal model call you write a prompt, you get a completion, and the process ends. With an agent you hand over a goal and a set of tools: search the web, read a file, run a command, call an API. The model picks a tool, reads the result, then picks again. It keeps going until the job looks finished, it hits a limit you set, or it wanders off course.
Which is why the same underlying model can feel useless in a chat window and genuinely useful inside an agent harness. The model supplies judgement. The harness supplies hands.
The four parts of any agent
- Tools: the functions the model is allowed to call. Search, code execution, a database query, a browser. Every tool you add widens both what it can do and what it can break.
- Memory: short-term context in the running transcript, plus longer-term storage in files, a vector store, or a scratchpad. Long-horizon memory is where most agents quietly fall apart.
- A planner: usually a system prompt plus some scaffolding that asks the model to lay out steps before starting. Explicit plans cut thrashing on anything longer than a dozen steps.
- Guardrails: step limits, spending caps, sandboxed execution, and an approval prompt before anything irreversible.
Tools are where the risk lives
A read-only agent is a research assistant. An agent with write access to your production database is a different conversation. The most common production incident isn’t a model going rogue. It’s a model doing exactly what it was permitted to do, at 200 times the intended volume.
Where agents already earn their keep
Software engineering is the clearest case, because code has a built-in success check: it runs or it doesn’t. The benchmark numbers moved fast. Top models resolved roughly 14% of real GitHub issues on SWE-bench Verified in early 2024. By late 2025 the best were clearing 70%. The tooling changed just as sharply, and the shift in how people actually write software with agents is worth following closely.
Support triage comes next. Agents are decent at reading a ticket, pulling order history, deciding whether this is a refund, a bug report, or a lost parcel, and routing it with a draft reply. A human still clicks send on anything involving money.
Data work is the third big one. Pull this month’s numbers, compare to last month, flag anything outside 15%, write it up. Clear finish line, which is exactly where agents do well. If you’re weighing options, what ChatGPT’s agent mode does in practice, and where it stalls, makes a useful reality check.
Why they fall over at step twelve
Per-step accuracy doesn’t compose. An agent that gets a step right 95% of the time succeeds on a 20-step task about 36% of the time. Push to 50 steps and you’re near 8%. That one piece of arithmetic explains most agent disappointment. Demos are three steps. Real work is thirty.
Then there’s the failure where the agent is wrong and confident. A search tool returns a stale page. The model builds a plan on top of it. Every later step inherits the error, and nothing in the loop flags it, because each step looked locally sensible.
Security deserves its own paragraph. Prompt injection is not theoretical. An agent with inbox access that visits a page carrying instructions aimed at it will, sometimes, follow those instructions. Scope credentials tightly, treat every external document as hostile input, and never hand an agent a key broader than the single task in front of it.
One agent, or a swarm of them?
Multi-agent setups are the most oversold idea in the space. Frameworks that spawn a researcher, a critic, and a writer look superb in a diagram and produce mediocre output at five times the token cost. Debugging is worse. When four models talk to each other, tracing which one seeded a bad assumption can eat an afternoon.
For defined work the reliable pattern is a workflow, not an agent. Chain fixed steps: extract, validate, summarise, file. Put a model inside each step. Save the free-roaming agent for jobs where you genuinely cannot predict the sequence in advance. It matches what most teams discover after their first expensive experiment.
The economics got a lot friendlier
Agents are token-hungry by design. One task can be 20 to 50 model calls once you count planning, tool results, retries, and reflection. At 2023 prices that was a non-starter outside high-value work.
Then prices collapsed. DeepSeek’s cost reset dragged the whole market down and the frontier labs followed. A competent agent on a mid-tier model now runs at cents per task for plenty of workloads, which is why agent products suddenly look viable instead of aspirational.
Getting one into production without regretting it
Teams that succeed tend to do the same handful of things.
- Start with a task that has an objective pass or fail check. Code that runs. A form that validates. A number that reconciles.
- Build an eval set from real failures before touching prompts. Twenty messy examples from live usage beat a hundred synthetic ones.
- Give the agent the narrowest permissions that allow the task, then widen only after you’ve watched it work for a fortnight.
- Require human approval for anything irreversible: sending, deleting, paying, deploying.
- Log every tool call with its inputs and outputs. When something breaks, that trace is the only thing that will tell you why.
If you’re buying rather than building, the platform question matters more than the model question. Managed stacks like IBM’s watsonx sell governance and audit trails as much as raw capability, and in regulated industries that usually decides it. For the wider picture of what’s landing versus what’s still marketing, this forecast on the 2026 breakthroughs is a fair guide.
Designing for the day the agent is wrong
Every agent will eventually do the wrong thing competently. The design question isn’t how to prevent that. It’s what happens next.
Good systems make a wrong action cheap to reverse. Drafts instead of sends. Sandbox instead of production. A diff instead of a commit. They record enough that a human can reconstruct the reasoning, and they fail loudly rather than quietly shipping something that merely looks finished.
The teams getting real value from AI agents right now don’t have the most autonomous setups. They drew a firm line around what the agent can touch, made everything outside that line a human decision, and moved the line one notch at a time as evidence came in. Autonomy is something you earn in small increments, not something you switch on.