A customer types “where is my order” into a support widget. The bot serves up a tracking link. Fine. Then the customer types “never mind, the box arrived crushed, just refund me.” The bot answers: “Sorry, I didn’t understand that. Let me connect you with a human.”
That was the state of the art not long ago. Today the same conversation can end with a refund issued, a replacement order queued, and a damage note filed against the carrier, with nobody on your team lifting a finger. The conversation didn’t get smarter. The software got hands.
That gap is the whole of the AI agents vs chatbots debate, and it’s worth understanding properly, because vendors are now using the two words interchangeably and the price tags are not interchangeable at all.
One replies. The other finishes the job.
A chatbot takes text in and produces text out. Under the hood it’s usually an intent classifier wired to a decision tree: detect “refund request”, branch to the refund script, hand off when confidence drops below a threshold. It can be genuinely good at this. It just has no way to reach past the chat window and change anything in your systems.
An agent is built around a goal rather than a script. Give it “resolve this customer’s damaged-goods complaint” and it will look up the order, check the return policy, confirm the item is eligible, trigger the refund in your payment processor, update the ticketing system, and stop when the goal is met. If a step fails, it reads the error and tries another route. If you want the longer mechanical explanation, how software learned to get things done on its own covers the architecture in more depth.
The practical consequence: a chatbot’s failure mode is an unhelpful reply. An agent’s failure mode is a wrong refund. Those are not the same risk category, and they shouldn’t be bought with the same budget or governance.
The four differences that actually matter
1. It holds state across a task
A chatbot remembers the current message thread. An agent remembers what it has already tried. Ask a chatbot to book a flight, watch it forget your departure city two turns later. An agent tracks a working plan: outbound booked, hotel pending, car rental blocked on confirmation.
2. It touches tools
This is the real dividing line. API access, database reads, file writes, browser control, code execution. The moment a system can call your CRM and write to it, you’re in agent territory whether the vendor calls it one or not.
3. It plans and corrects
Chatbots follow flows. Agents decompose. Faced with “get this contract signed by Friday,” an agent breaks that into finding the right signatory, checking their calendar, drafting the envelope, and chasing a signature that hasn’t arrived. When step three returns an error, it adapts instead of apologising.
4. It operates inside permissions
Every tool an agent can reach is a permission you’ve granted. Scope matters more than model quality, and it’s the part most pilots skip. Read-only access to the billing system is a very different animal from write access with no approval step.
The middle ground is where most products live
Everything is not neatly binary. Plenty of platforms started as pure chatbots and grew tool access over time, which makes the label messier than the marketing suggests. A realistic look at the conversational AI platform Yellow.ai shows the pattern: strong intent handling and routing, with integrations that let it do a handful of concrete things. Is that a chatbot with action, or a narrow agent? Depends who’s asking and what they’re selling.
The enterprise version of this is search tools that learned to execute. Instead of returning a document, they return the completed task, which is the shift described in Glean’s move from searching to doing. Same buyer, same interface habit, entirely different backend.
What an agent looks like with a real workload on it
Take accounts receivable at a mid-sized software company. Four hundred invoices are overdue at any given time. A human collections rep sends maybe 60 emails a day and costs roughly $4 in loaded time per touch once you account for the CRM logging, the payment portal check, and the follow-up.
An agent assigned that queue pulls the ageing report at 7am, checks which invoices already have a payment in transit, drafts a reminder for the rest in the tone appropriate to how overdue they are, sends it, logs the touch, and escalates anything past 45 days to a human with a summary attached. It handles 400 in an hour. Cost per touch lands in the low cents. The human now spends their day on the twelve accounts that genuinely need judgement.
No chatbot does this. Not because the language model can’t write the email, but because writing the email was never the hard part.
Where agents break in production
Reliability is the honest counterweight. If a single decision step is 95% accurate, a ten-step task succeeds about 60% of the time. Chain twenty steps and you’re down near 36%. That compounding is why so many agent demos feel magical and so many agent deployments stall in month two.
Some of the more interesting work right now targets exactly this, adding checkpoints and recovery so a failed step doesn’t silently corrupt the rest of the run. There’s a good write-up on fixing the compounding-error problem in agent pipelines if you want the technical detail. The takeaway for buyers is simpler: ask what happens when step four fails, and listen for whether the answer mentions retries, rollback, or human approval.
Chatbots sidestep all of this by never taking a risky action. That’s not a flaw. For password resets, opening-hours questions, and tier-one triage, a boring deterministic bot is cheaper, faster, and far easier to audit.
The naming has made this worse
Half the confusion is linguistic. Every few months the industry renames the same feature after a human job: assistant, copilot, teammate, agent. The word “agent” now covers everything from a scripted FAQ widget to software that files your taxes. There’s a sharp rant on why naming features after human roles creates bad expectations, and it applies double here, because the label now implies a level of autonomy the product may not have.
Ignore the noun. Test the behaviour.
Questions that separate a real agent from a dressed-up chatbot
Before you sign anything, put these to the vendor and watch where the answers get vague:
- What can it write to? Not read. Write. Get the list of systems with write access, and ask which of those require human approval first.
- Show me a run with a failure in the middle. Not a happy path. Ask them to kill an API mid-task and show you the recovery.
- How many steps does the longest supported task have? Anything over eight or nine needs explicit checkpointing to survive.
- What’s the audit trail? Every tool call should be logged with inputs, outputs, and timestamps. If you can’t reconstruct why the refund happened, you don’t have a deployable system.
- What’s the kill switch? One control, immediate effect, no engineering ticket required.
- What did you deliberately keep human? A vendor with a clear answer has thought about blast radius. A vendor who says “everything” has not.
A useful rule of thumb: if removing the language model leaves the product still functional, you’re looking at a chatbot. If removing it leaves nothing, you’re looking at an agent, and you should be evaluating it like infrastructure rather than like a chat widget. Different contracts, different monitoring, different people in the room when it goes live.