A refund request lands in the shared inbox. Handling it properly means pulling the order from Stripe, checking whether it falls inside the 30-day window, looking up the customer’s plan tier, drafting a reply, then issuing the credit. Ask a single-prompt chatbot to do all five things and you get a confident paragraph with the wrong order number in it.
Split the work into steps with a check between each one and the same model copes fine. That is the core idea behind multi-step AI agents: rather than one heroic prompt, you get a loop that plans, acts, looks at what came back, and decides what happens next.
What separates a multi-step agent from a chatbot with tools
Plenty of products now call a model with a tool definition attached, wrap it in a while loop, and ship it as an agent. The useful dividing line is whether the system can change its plan based on what it learns. A single tool call (look up order 4471) is a fetch. A multi-step agent notices the order shipped three days late, concludes the restocking fee should be waived, reads the policy to confirm, and writes the reply using the language of the policy it just read.
Multi-agent systems sit on a different axis. There, several models with separate roles talk to each other. You can have a multi-step agent built from one model in a loop, and you can have a multi-agent setup where every agent does exactly one step. Vendors conflate the two constantly.
Why one giant prompt rarely survives the whole task
Errors compound quietly
Every step has an accuracy rate. Chain eight steps that each succeed 92% of the time and the odds of a clean end-to-end run land near 51%. No amount of prompt engineering beats that arithmetic. You can only shorten the chain, add verification, or build recovery for the step that fails.
Context turns into noise
A single prompt carrying a 40-page contract, a customer’s history, and six tool schemas forces the model to attend to all of it simultaneously. Step-based design narrows the view: the clause extractor sees the contract, the risk scorer sees the extracted clauses, the drafting step sees the score plus a template.
Failures become invisible
When a chain breaks you get a trace. Step three returned an empty array because the date filter ran in the wrong timezone, and you can see it in ten seconds. When a monolith prompt breaks you get a plausible-sounding paragraph and no clue which part of the reasoning drifted.
The loop, in five parts
Nearly every production multi-step agent is assembled from the same pieces, whatever the framework calls them:
- Planner: converts the goal into an ordered list of steps, or picks the next one on the fly.
- Tool layer: the functions the agent may call, each with a typed schema and a documented failure mode.
- Working state: what the agent knows so far, including what it tried and gave up on.
- Validator: a check that a step’s output is usable before the next step consumes it.
- Stop condition: a budget on steps, wall clock, or spend, plus a working definition of done.
Skip any one of those and you get the familiar symptoms: retry storms, duplicated side effects, or an agent that declares victory after step one because nothing told it there were more steps.
Four patterns that hold up under real traffic
Sequential pipeline
Fixed order, no branching. Extract, classify, retrieve, draft, check, send. Cheapest to build, easiest to test, because every step has a known input shape. It falls over the moment different inputs need different paths, which is why most teams start here and outgrow it inside a quarter.
Planner plus executor
One call writes the plan, a second pass runs each item with tools and feeds results back into the plan. The plan becomes a visible artifact you can log, diff and inspect. If you have read the walkthrough on building a multi-agent support triage bot with the OpenAI Agents SDK, this is the shape underneath it: a router classifies the ticket, then hands off to a chain tuned for that category.
Reflection loop
A generator produces output, a critic checks it against explicit criteria, and failures go back with the reason attached. Cap it at two or three rounds. Past that, the model starts editing good work into worse work and cost climbs while accuracy flattens.
Supervisor with workers
A coordinating agent decomposes the task and dispatches narrow workers, each with its own tools and its own slice of context. The coordinator stays small and cheap; the workers do the reading and writing. The tradeoff is latency, since three parallel workers plus a synthesis step is four round trips before the user sees a single word.
Where the chain usually snaps
Step four works in the demo because the demo always contains a clean order with a valid customer ID. Real inputs arrive with a missing field, a tool returning a 429, or a record that exists in two systems with two different values. The failures demos hide almost always sit at the handoffs between steps rather than inside the model’s reasoning.
Three that show up again and again:
- Silent nulls, where a tool returns nothing and the next step treats nothing as a valid answer.
- Repeated side effects, where a retry sends a second email because the step never recorded the first.
- Scope creep, where step two decides it is allowed to do step five’s job and skips the check on the way.
State between steps is the part people underestimate
An agent that forgets it already tried the refund endpoint and hit a permissions error will try again, then again, burning budget on the same wall. Agent memory problems surface as loops and contradictions rather than crashes, so they slip past dashboards and become obvious only to the person reading the transcript.
A workable pattern is an append-only scratchpad with three fields per entry: action, result, decision. Pass the last few entries plus a rolling summary into context and keep the full trace outside the model, where you can query it when something looks wrong.
Guardrails that earn their build time
Longer chains take more actions, so the blast radius of one bad step grows. Before adding a step, decide what it is allowed to touch. Reads can be permitted broadly. Writes need scoping, idempotency keys, and for anything that charges, emails or deletes, a human approving the first few hundred runs.
The risks of letting software act on its own are well catalogued by now: prompt injection through fetched content, tools that over-trust their own inputs, agents that read “be helpful” as “do more”. None of it is exotic. All of it gets worse as the number of steps climbs.
Design rules that survive contact with production
Keep steps small enough to test alone. A step you can describe in two sentences is usually right; one that needs a paragraph should be split.
Make output typed. Prose can’t be validated; JSON matching a schema can be rejected and retried before the error spreads downstream.
Log every transition with input, output, latency and token count. When accuracy drops you want the offending step, not just a wrong final answer.
Ship the narrowest version first. The agent use cases that reach production are rarely the ambitious ones. They are the ones with a clear success criterion and a reviewer watching the first few weeks of output.
Pick the two steps that hurt most
Find the task where a person currently does three lookups before writing one reply. Build only the first two steps: fetch the record, validate the fields, then hand the result to a human with a draft attached. Measure how often the fetched data is correct and how often the draft needs rewriting. Those two numbers tell you whether the next step is worth building and which one it should be. Chains grow from evidence, not from architecture diagrams, and the teams that get multi-step agents working are the ones that added step four only after steps one through three had been boring for a month.