Deployment Is Where Agents Stop Being Impressive and Start Being Useful
A demo agent that turns a Slack message into a booked meeting takes an afternoon. The same agent serving 400 employees takes most of a quarter. The model doesn’t change between those two states. Everything around it does: permissions, retry logic, cost visibility, and the question nobody asks during the demo, which is who gets paged when it does something strange at 2am.
AI agent deployment is less a launch than a slow widening of trust. You hand over one narrow job, watch what happens, then expand the radius. Teams that skip the widening part end up with a genuinely impressive proof of concept and a security review that never closes.
Narrow the Job Until It Feels Almost Too Small
The failure mode I see most often isn’t a weak model. It’s a scope that would break a human new hire. “Handle customer support” isn’t a task, it’s a department.
Pick something with a clear start and end state. Triage inbound tickets into three categories and route them. Pull line items from supplier invoices into a spreadsheet. Draft a first-pass reply for a human to approve. Each of these has a definition of done, which means you can tell whether the agent finished or wandered off.
A workable test: if you can’t state the success case in one sentence and the failure case in another, the scope is still too wide. The mechanics of picking tools and the loop agents get stuck in are covered in this guide to how to build an AI agent without hitting the infinite loop, and it’s worth reading before you choose where the thing runs.
Pick the Runtime Before You Obsess Over the Model
Model choice gets all the attention. Runtime choice causes most of the late-night problems.
Short tasks and long-running work are different products
A triage agent that finishes in eight seconds fits neatly inside a serverless function. An agent that researches for twenty minutes, calls six APIs, and waits on a webhook does not. That one needs a queue, a durable state store, and a way to resume after a timeout. Teams usually discover this on day four, right after the function hits its execution ceiling.
State has to live somewhere you can inspect
Conversation history in memory is fine for a prototype and useless in production. Store run state in something you can query, so you can answer “what did it know at step nine?” without guessing.
Timeouts and retries are design decisions
Set an explicit ceiling on tool calls per run. Without one, a confused agent will politely retry a failing API forty times and bill you for the privilege.
Low-code builders are tempting here, and some ship real value in a week. Others crack the moment concurrency and audit requirements show up. The trade-offs are set out honestly in this breakdown of low-code AI agent builders and where they break at scale. Know which one you’re buying before the pilot becomes load-bearing.
Scope Permissions Like the Agent Is a New Contractor
Give a new contractor a company card with no limit and you’d expect trouble. An agent with a broad API key behaves the same way, not out of malice but because tool descriptions are ambiguous and models take them literally.
- Read before write. Run read-only for at least two weeks before granting any tool that changes data.
- Allowlist per task, not per agent. The refund tool shouldn’t be reachable from the ticket-triage flow.
- Gate irreversible actions. Refunds, outbound emails to external addresses, record deletion: all of these get a human approval step.
- Cap spend and rate per session so a runaway loop dies on its own.
- Log every tool’s arguments, not just that it was called.
The fifth item is the one people skip. When a customer asks why their account was credited twice, the argument log is the difference between a five-minute answer and a two-day investigation.
Traces Beat Logs Every Time
A log line reading “agent error” tells you nothing. What you need is the full sequence: the user input, the model’s reasoning step, the tool it chose, the arguments, the response, and the token count for each hop.
Give every run an ID and attach cost and latency to each step. When someone reports a bad answer, you replay the trace instead of reproducing it by hand. Track five numbers weekly: task completion rate against your own test set, tool error rate, average tool calls per task, cost per completed task, and human override rate. That last one is the most honest signal you have. If reviewers rewrite a third of the agent’s output, the agent isn’t working, whatever the demo suggested.
Treat Prompts Like Code, Because They Are
A single added sentence can shift which tools an agent reaches for. Version prompts and tool schemas in the same repo as your application, with diffs and rollback. If an agent suddenly starts guessing wrong field names, the fix is usually in the tool description rather than the model.
Writing instructions that survive contact with a real model is its own skill, and the practical guidance in writing system prompts an agent can’t misread covers the structural side well: explicit output formats, ordered steps, and edge cases named rather than implied.
Roll Out in Stages, With a Kill Switch You’ve Already Tested
Shadow mode first. The agent runs on live traffic, produces decisions, and nobody acts on them. Compare its output to what humans did. If agreement sits at 60%, you are not ready for write access, no matter how good the demo looked.
Then move to a small group: one team, or 5% of traffic, with write permissions limited to the least risky tool. Expand weekly, and pause whenever override rate climbs.
Build the kill switch before you need it, and test it in staging. Often the right switch isn’t full shutdown but a downgrade: revoke write tools and let the agent keep answering questions from documentation while someone investigates.
Name a person who owns the agent in production. A rotation of volunteers is how small issues become incidents.
Budget for the Loop, Not the Request
Single-request pricing is misleading. An agent averaging six tool calls and 18,000 tokens per task, running 3,000 tasks a day, costs roughly an order of magnitude more than the same volume of one-shot prompts. Set a ceiling per task, alert when the 95th percentile drifts up, and route routine traffic to a smaller model while reserving the expensive one for hard cases.
Understanding what each API call actually returns, and where you’re paying for context you never use, is worth an afternoon. This explanation of what AI agent APIs actually do is a decent starting point for trimming that waste.
The Runbook Nobody Writes Until the First Incident
Write it in week one anyway. It needs four things: how to disable tool access in under a minute, the three most likely failure modes and their fixes, the escalation contact for each downstream system the agent touches, and the exact query that shows a run’s full trace.
Add a short section on known weirdness. Every deployed agent has some: a tool that returns dates in a different format on Mondays, a category the classifier consistently mislabels, a vendor API that times out above 30 requests a minute. Writing these down turns a recurring mystery into a footnote, and it gives the next person a real chance of fixing things without paging the person who built it. That handover document is usually what decides whether the agent survives its second quarter.