A demo agent is a laptop, a terminal, and a lot of optimism. A hosted agent is a different animal. It wakes up because a webhook fired at 3am, calls three APIs, remembers what it needs to remember, and doesn’t burn through a month’s budget because somebody pasted a 40-page contract into a chat window.
Most teams discover the gap between those two things the same way: the agent works beautifully in a notebook, then falls over the first week real traffic hits it. The problem is rarely the model. It’s everything around it.
What AI agent hosting actually covers
People use the phrase to mean “wherever the code runs,” which is maybe a third of the job. A hosted agent also needs tool credentials it can reach safely, state that survives a restart, network rules that limit where it can send data, per-step logs, a spend cap, and rate limiting in front of model calls so a loop can’t hammer the provider into a 429 storm.
This matters because vendors sell pieces. A model provider sells inference. A container platform sells compute. Neither sells the glue, and the glue is where agents break. When you’re evaluating a host, you’re really evaluating how much of that glue they’ve already built and how much you’ll be writing on a Friday night.
Three hosting shapes, and who each one suits
Managed agent platforms
You buy the whole stack: runtime, memory store, tool connectors, a dashboard, sometimes a library of prebuilt agents you can fork. This is the fastest route from working prototype to something a customer can touch, and it’s usually the right call for teams without dedicated infrastructure people. The trade-off is that your architecture ends up shaped by their opinions, and moving later means untangling memory, triggers, and traces. If you’re shopping in that market, it’s worth reading up on how to evaluate agents in the marketplace without getting burned before you sign anything.
Containers and serverless on a general cloud
You get a Dockerfile, a queue, a cron scheduler, and a heap of responsibility. Full control, and full ownership of every failure mode. This is the sane choice when your agent needs unusual dependencies: a headless browser, ffmpeg, a pinned Postgres driver, a GPU for a local embedding model. It’s also the choice that quietly becomes a second job if nobody on the team has run production infrastructure before.
Self-hosted on your own hardware
Compliance, data residency, or genuinely heavy volume push people here. Fine reasons. Just go in knowing the real cost isn’t the server, it’s the observability, patching, and on-call rotation. There’s a useful rundown of what open source agents actually cost to run that covers this ground honestly.
A hybrid setup is common in practice: orchestration on a managed platform, tool execution inside your own VPC where the sensitive systems live. It works. It’s also two systems to debug, so only do it if the data genuinely can’t leave.
What breaks first when you go live
- Cold starts. A function that needs 8 seconds to pull a 400MB image is invisible for a batch job and fatal for a chat agent. Users give up around second five.
- Tool timeouts. Most HTTP clients default to 30 seconds or less. Scrape a slow page or wait on a queue and the retry fires before the first call has finished, so you get two writes and one headache.
- Duplicate actions. Retries are how you send two emails, issue two refunds, or open three identical tickets. Idempotency keys stop being optional the moment an agent can move money.
- Runaway loops. An agent retrying a failing tool forty times burns tokens and gets your IP throttled. Hard step caps and per-run token ceilings belong in the runtime, not in a prompt asking the model to please stop.
- Secrets in the wrong place. Credentials pasted into context, printed in a trace, or baked into a public image. Use a secrets store and redact before logging.
- Silent partial failure. The run reports success, but step four returned an empty list and the summary is confidently wrong. Check the shape of tool returns, not just that they didn’t error.
Latency budgets decide whether people trust it
Six sequential model calls at 1.2 seconds each plus two tool calls at 900ms puts you at nine seconds before streaming even helps. Parallelise tool calls that don’t depend on each other. Cache deterministic lookups. Route with a fast cheap model and hand the hard step to a slower one. Stream tokens so the user sees progress instead of a spinner.
Geography compounds quietly. A model endpoint on another continent adds 150–250ms per call, and with eight calls that’s two extra seconds nobody planned for. If your users are in Singapore, host near Singapore.
State is the part nobody budgets for
Agents are stateful in a way ordinary web apps aren’t. One task spans multiple model calls, tool invocations, and sometimes a human approval that arrives six hours later. If the process dies at step seven of twelve, restarting from scratch is expensive and occasionally harmful, because the side effects already happened.
The fix is durable execution: checkpoint after each step, resume from the last committed one, and keep task state separate from conversation memory. Treat long context as a cost, too. A 60-turn conversation stuffed into every prompt is a tax on every single call, so summarise and prune deliberately rather than hoping the context window covers it. Understanding what agent APIs handle versus what your code has to handle saves a lot of guessing here.
Permissions and the trust boundary
A hosted agent touches real systems with real credentials, which makes it an attack surface with an inbox. Three habits go a long way. Give each agent its own service account with narrow scopes instead of a shared admin key. Run any agent-generated code in a sandbox with no route to your secrets. Add egress allowlists so an instruction injected through a scraped web page can’t POST your customer list somewhere odd.
Prompt injection isn’t solved, so assume hostile input eventually reaches a tool call. The question is how much damage a single bad instruction can do given the permissions you handed over.
What it costs, roughly
For most business agents, GPU capacity is irrelevant. You’re paying for inference, a little orchestration compute, storage, and egress. A small container comfortably handling 5,000 runs a day often lands between $40 and $120 a month. The model spend is what moves: 5,000 runs with eight calls each and 2,000 average tokens is 80 million input tokens a day, which is rounding error at small-model rates and a serious invoice at frontier-model rates.
That imbalance is the whole lesson. Optimise prompt size and step count before you optimise infrastructure, because the hosting bill is usually noise next to tokens. One exception worth watching: per-seat or per-run pricing on managed platforms. A $29 plan that becomes $2,400 at 300,000 runs a year is a business model decision you should see coming.
Questions to ask before committing
- What’s the maximum execution time per run, and what happens when the agent hits it mid-task?
- Can I cap spend per run and per day at the platform level, or only by watching a dashboard?
- Where do traces live, how long do they stay, and can I search by run ID when a customer complains?
- Can I roll back a prompt change without a full redeploy?
- Does it support human approval steps that pause for hours or days?
- If I leave, can I export state, or do I rebuild from zero?
Prompt changes are deployments
Editing a system prompt changes behaviour as much as a code change does. Version them, stage them, keep an eval set of 40 or so real cases pulled from production, and gate the rollout on that set passing. Platforms that let anyone tweak a prompt in a UI are convenient and slightly terrifying, because the edit that fixes one complaint can break four unnoticed behaviours. That’s also why teams shipping fast on low-code agent builders often hit a wall at scale: the speed comes from skipping exactly this discipline.
Start managed, stay portable
The pragmatic path for most teams looks like this. Run orchestration on a managed runtime so you’re not debugging queues in week one. Put anything with odd dependencies in your own container. Keep every piece of durable state in a database you control, keyed by run ID, with the runtime treated as replaceable. Log tool arguments and results, not just the final answer.
That last set of choices is what buys you options in eighteen months, when pricing shifts, a vendor gets acquired, or your volume makes self-hosting genuinely cheaper. Hosting decisions are hard to reverse when your state lives inside someone else’s product. They’re pleasantly boring when it doesn’t.