The first agent you build will probably work in the demo and fall over on the third real request. That is normal. The gap between an agent that books a meeting and one that books the right meeting, handles a reschedule, and does not email the wrong client is almost entirely engineering, not model choice.
What follows is the practical path, based on what separates agents that stay switched on from the ones quietly disabled six weeks after launch.
Define “done” before you pick a model
Write the task down as a completion condition, not a capability. “Handle customer refunds” is a capability. “Given an order ID and a reason category, issue a refund under $200 through the Stripe API, then post a one-line summary to the ticket” is a completion condition. The second version tells you which tools you need, what the agent must never do, and how you will test it.
Three questions worth answering on paper first:
- What does the agent receive at the start, exactly? A webhook payload, a spreadsheet row, a paragraph typed by an annoyed customer?
- What counts as success that a script can verify? A database row, a passing assertion, a returned JSON object with the right fields.
- What is the failure that gets somebody fired? Wrong amount, wrong recipient, deleted record, a reply sent to a client instead of a colleague.
That third question sets your autonomy budget. Agents touching money, production data or outbound communication need a confirmation gate before the irreversible step. Agents that draft replies into a review folder can run unsupervised from day one.
Choose the smallest architecture that could work
Most “agent” projects are actually workflows wearing a costume. If you can draw the decision tree on a napkin, build that instead. A plain pipeline with three LLM calls and a validation check will beat a general-purpose agent on reliability every single time, because you already know the shape of the problem.
Where a single agent earns its keep
Single agents make sense when the number of possible paths is open-ended: research tasks, code fixes across an unfamiliar repo, triaging messy inbound requests. One model, a handful of well-described tools, and a hard step limit will cover more ground than people expect. If you want a clearer picture of the shape these systems take in the wild, this breakdown of what AI agents actually do and where they break is a useful reality check before you commit to an architecture.
When multi-agent is worth the pain
Split into multiple agents only when sub-tasks need genuinely different context or different permissions. A researcher with web access handing structured findings to a writer with no web access is a sensible split. Three agents passing chat messages back and forth to plan a grocery list is theatre. Every extra agent adds a coordination failure mode, a shared-state problem, and a debugging session you will not enjoy.
The tool layer decides whether it works
Nine out of ten agent failures I have seen traced back to tool design, not reasoning. The model can only act through the interfaces you give it, and vague interfaces produce vague behaviour.
- Name tools after the outcome, not the API.
refund_orderbeatspost_v2_charges_refund, because the model reasons about intent. - Write descriptions that say when to use the tool, and when not to. Two sentences of guidance beat one sentence of summary.
- Return structured errors. An agent that receives
{"error": "no_such_order", "hint": "check the order ID format"}can self-correct. A stack trace just wastes a step. - Make writes idempotent. Agents retry. If retrying charges a card twice, you have built a bug with a personality.
- Cut the tool count. Twelve tools with overlapping purposes produce decision paralysis. Six sharp ones do not.
Browser and computer-use tools are their own discipline. Login walls, cookie banners and unlabelled buttons break agents in ways that look like reasoning failures but are really perception failures. Our teardown of where ChatGPT agents fall over covers the specific patterns worth designing around, including timeouts on slow pages and forms that silently reset.
Write the loop yourself at least once
Before you reach for a framework, hand-roll one agent loop. It is roughly forty lines: send the conversation and tool definitions to the model, check whether the response contains tool calls, execute them, append the results, repeat. Once you have done that, frameworks stop feeling like magic and start feeling like a convenience you can evaluate.
Whatever you build on, put four limits in place from the start:
- A maximum step count, typically 10 to 15. Agents that hit the ceiling should stop and hand back what they have.
- A token and dollar budget per run, checked every iteration.
- An explicit escape hatch tool like
ask_human, so the model has somewhere to go when it is stuck instead of looping. - Full logging of every step: prompt, tool call, arguments, response, latency. You will read these logs more than you read the code.
Keep memory boring
Most memory problems are context problems. The agent forgets the user’s constraint from six turns ago because it got buried under tool output. Fix that before you install a vector database: trim verbose tool responses, summarise completed sub-tasks into two lines, and pin the original goal to the top of every prompt.
Durable facts belong in a normal database. Order status, account tier, prior tickets. Retrieval is for fuzzy recall of documents, not for remembering that the customer already said they want a refund and not a replacement.
Test agents by reading traces, not just scores
You cannot unit-test your way to a reliable agent, and a single accuracy number tells you nothing actionable. Build a set of 40 to 60 real tasks with known-good outcomes, run the agent against all of them, and then read the failures. Sort them into buckets: wrong tool chosen, right tool with bad arguments, gave up early, invented a result, hit the step limit. The distribution of those buckets tells you exactly what to fix next.
If you want a sense of what good looks like at the frontier, this review of the best AI agents judged by what they actually finish is worth reading with your failure buckets open beside it. The pattern is consistent: the products that ship are the ones with tight scope and good tools, not the ones with the biggest models.
Pick models on cost per completed task
Benchmarks are a starting point, not a decision. Run your own eval set across two or three models and measure cost per successful completion, including retries. A cheap model that finishes 78% of tasks at a tenth of the price often beats a flagship that finishes 84%, especially once you add a verification step that catches the misses.
Routing helps more than picking. Send straightforward classification and extraction to a small model, and reserve the expensive call for planning and anything involving ambiguity. The collapse in inference pricing over the last two years, covered in this look at how DeepSeek rewired frontier model costs, means the economics of running agents in a loop have changed faster than most team budgets.
On infrastructure, you mostly need three things: a way to swap models without rewriting your loop, per-run cost visibility, and a place to store traces. Managed platforms such as Bedrock and the rest of Amazon’s AI stack cover the first and third out of the box, which is often enough reason to start there rather than assembling it yourself.
A realistic first two weeks
Day one and two: build the happy path by hand, with no model involved at all. Hard-code the inputs, call the real APIs, and confirm the outcome is actually achievable. Plenty of agent projects die here, which is much cheaper than dying in week six.
Days three to five: wrap that working path into a single agent with three or four tools and a 12-step ceiling. Get it running on ten real examples. Days six to eight: collect failures and fix the tools, not the prompt. Roughly 70% of your early failures are tool problems.
Week two: build the eval set properly, add the confirmation gate before irreversible actions, put a human review step in front of anything outbound, and set a hard daily spend cap. Then run it in shadow mode alongside a human for a week and diff the results. When the agent’s output matches or beats the human’s on 90% of tasks and its failures are boring rather than expensive, you have something you can turn on.