Last month a friend asked me to help her “create an AI agent” for her six-person accounting firm. She had read that agents could chase overdue invoices, and she wanted one by Friday. Twenty minutes into the call, the thing she actually needed turned out to be a script that reads a spreadsheet, checks a due date, and sends a reminder email. No vector database. No multi-agent orchestration. That was the correct answer, and she was mildly disappointed.
Most people who set out to build an agent have the same gap between the imagined version and the useful version. Closing it is mostly about picking the right job, wiring two or three tools with tight inputs, and building in a way to stop.
An agent is not a chatbot wearing a lanyard
The line between the two isn’t tone or memory. The difference between AI agents and chatbots is not the conversation. It’s control. A chatbot produces text and hands it to you. An agent takes actions, looks at what happened, and decides what to do next.
That definition tells you what you have to build: a model, a set of tools it can call, a loop that feeds results back in, and a stopping rule. Everything else is polish.
Choose a job that ends
Agents fail most often not because the model is weak but because the task was never defined. “Handle customer support” cannot be finished. “Draft a reply to every new Zendesk ticket in the billing queue and tag refund requests” can.
A few filters I run before writing any code:
- It happens at least weekly. One-off tasks never repay the setup cost.
- Success is checkable. You need to be able to look at the output and say yes or no in under a minute.
- Mistakes are cheap and reversible. Drafting an email is fine. Sending 400 of them is not, at least not in week one.
- A person already does it in under ten minutes. If it takes a human an hour of judgement, an agent will take six and get it wrong.
That last filter does a lot of work. Agents are good at the boring middle of a process, the stretch where the steps are known and the decisions are narrow.
The three parts you cannot skip
A loop with a brake
Every agent is a while loop with a model inside it. The model gets a goal and the running transcript, picks an action, the action returns a result, and the result goes back in. Your job is to make sure the loop ends. Cap iterations at five or six. Cap the wall clock. Cap spend per run. An agent stuck in a retry cycle can burn real money in ninety seconds.
Tools with narrow inputs
Give the model three tools, not thirty. Each should take a small number of typed parameters and reject anything that doesn’t match. If a tool has a boolean called force, delete it. If a tool can delete records, put it behind a human approval step. The version of that failure where the agent deletes the wrong thing is not hypothetical.
State you can read
Write every step to a log: tool name, arguments, result, timestamp. When something goes sideways at 3am you want to reconstruct the run without guessing. One JSONL file per run is enough for months.
A working example, small enough to finish
The first agent I built that anyone else used did one thing. It read a shared inbox, sorted each message into billing, bug report, or sales, and dropped a draft reply into the drafts folder. Three tools, one loop, about forty lines of Python on top of a function-calling API.
It cost pennies per run and saved roughly twenty minutes a day. It also failed in a way I hadn’t planned for, which I’ll get to. If you want a more structured version of the same pattern, with a planner and a coder agent passing messages, there’s a decent walkthrough of building a small Python tool with a team of AI agents worth reading before you decide one agent is too few.
Spoiler: for most jobs, one is enough.
Instructions are product decisions
The system prompt is the most-read document in your codebase and the least reviewed. Write it like a job description for a hire who has never seen your business and cannot ask follow-up questions.
- Say what done looks like, in one sentence.
- List the tools and when each is appropriate.
- List the cases where the agent should stop and hand off, in plain language.
- Include two or three real examples of awkward inputs and the correct response.
The handoff list is the one people skip. “If the customer mentions a legal dispute, a chargeback, or cancellation, stop and flag the ticket.” One line like that prevents a whole category of incident you do not want to explain to anyone.
Where they break
Agents don’t usually explode. They drift. A tool returns a timeout, the model reads the silence as success, and the same reminder goes out twice. Or a step returns a 403 and the agent invents a workaround that touches data it shouldn’t. I’ve collected several of these in a piece on what actually goes wrong when software starts acting, and the pattern repeats: the reasoning is fine, the plumbing is not.
The second quiet failure lives in the workflow itself. A run that succeeds nineteen times out of twenty still has a one-in-twenty path you have never watched. If you want the ground-level version, there’s a good breakdown of where an AI agent workflow works and where it quietly breaks. Read it before promising anyone a fully automated process.
Testing something that never behaves the same way twice
You cannot unit test a model’s output the way you test a function. You can test the shell around it. Assert that the loop terminates. Assert that a tool call with malformed arguments raises instead of silently continuing. Replay twenty logged conversations through any prompt change and read the diffs by hand. That last exercise takes an afternoon and catches most regressions.
Build a small eval set from real cases, fifteen to thirty of them, each with an expected outcome. Run it before every prompt edit. It won’t catch everything, but it stops you shipping a change that breaks the one case your biggest client cares about.
Frameworks, or forty lines of Python
You have options. LangGraph if your flow has branches you need to see. CrewAI if roles genuinely differ. The OpenAI or Anthropic tool-use APIs if you want the raw loop. No-code builders if the job is nothing more than “when this happens, do that.” Most of the current batch of agent apps fall down on integration depth rather than intelligence, which is a useful thing to know before you compare pricing pages.
My rule: if the agent fits in one file, keep it in one file. Frameworks earn their keep at ten tools and three branches. Below that they mostly add abstractions you will debug later.
The part nobody budgets for
Getting an agent to work is maybe a third of the effort. Getting people to trust it is the rest. The AI agents teams actually keep using tend to be the ones that do something visible and boring, show their work, and fail loudly. An agent that quietly gets things 90% right is worse than one that asks for help on the other 10%, because the first trains people to stop checking.
Ship it read-only
Start with permissions that cannot hurt you. Read-only API keys. Drafts instead of sends. A dry-run mode that logs what it would have done. Run it in shadow mode beside a human for two weeks and compare the outputs. Then open one write action at a time, starting with whichever has the smallest blast radius.
Keep a kill switch that doesn’t require a deploy. Log every tool call with its arguments. Give every run an ID so a person can say “that one” when something looks off. Put a real human at the end of the escalation path, with a notification that actually reaches them on a weekend.
None of that is glamorous. But the agent still running in month four is almost never the cleverest one. It’s the one whose builder assumed it would misbehave, wrote down what misbehaving looked like, and kept the consequences small enough to fix on a Tuesday afternoon.