AgentOps Is Not MLOps: What Breaks in Your Monitoring Stack When Agents Go to Production

For years, keeping a model healthy in production meant keeping it close to the model you shipped. You watch drift against a reference window. You track latency against an SLO. You check accuracy against a holdout. A number moves, you retrain.

That stopped working once models started calling tools.

The industry response was fast. Gartner expects more than 40 percent of agentic AI projects to be canceled by the end of 2027, citing escalating costs, unclear value, and inadequate risk controls. Every observability vendor now ships agent tracing.

What went unexamined is how teams actually ran the migration. Most ran it as an addition: new spans on top of the old stack; nothing came off it. The inherited signals still fire, and several now report healthy on runs that failed.

I learned this from a multi-step review pipeline I run, which fans out to parallel model reviewers and writes their verdicts into an application datastore. The first bad verdict it shipped had a fully green trace: every span succeeded, latencies normal, output wrong.

Migration by Addition: The Part Nobody Audits

What got added is real progress. OpenTelemetry’s GenAI semantic conventions now define agent spans: create_agent, invoke_agent, execute_tool, and plan. The spec still carries a Development status, worth knowing before you standardize on it.

Langfuse, LangSmith, Arize Phoenix, W&B Weave, and AgentOps all emit some version of it, so you get a waterfall of the run: which tool fired, what it returned, what it cost.

The part that never got re-examined is everything underneath. Drift monitors kept running. Retraining triggers kept their old thresholds untouched. And alerting never moved past a single boundary. Those components encode assumptions that hold for a stateless scoring service and stop holding for a system that runs a loop.

Five assumptions carry most of the weight:

• Outputs are comparable across runs. Same input, roughly the same output, so a difference means something.

• Inference is stateless. The request is the unit of work; nothing carries over.

• One request crosses one decision boundary. There’s a single place to set a threshold.

• Ground truth arrives. A label eventually shows up to score against.

• A human sits between the model and the consequence. The model recommends; a person acts.

Each breaks differently once the model runs a loop, invisible to the signal watching for it. A monitoring stack fails by staying green.

Five Assumptions, Five Silent Failures

Each of these five assumptions breaks in its own way, and the failure is usually invisible to the exact system built to catch it.

Comparable outputs: when the same input passes and fails in the same week

Run the same support ticket through your agent twice. Monday it refunds the customer and closes out; Thursday it loops on an order number the customer already gave.

Tau-bench measures this with pass^k: the odds that all k attempts at one task succeed. A single gpt-4o attempt cleared roughly 61 percent of retail tasks, but run the same task 8 times, and the odds of all 8 succeeding drop below 25 percent. Score your agent on one run per input, and your dashboard reports 2.4 times the reliability your users actually get.

Stateless inference: when the path is the defect and the answer looks fine

A courier mishears the street name at the first stop. Every turn after that is perfect, and every one is wrong.

Agent systems fail the same way. Anthropic’s own multi-agent research system hit this pattern directly: “one step failing can cause agents to explore entirely different trajectories.” Each step’s output feeds the next, so an early error doesn’t get corrected; it compounds.

This isn’t a rare edge case. The MAST taxonomy sorted over 1,600 traces into 14 failure modes, and the largest single category was system design: errors baked into how steps are wired together, not into any individual step’s output. Retraining a model can’t fix that, because the defect was never in the model. It was in the path.

One decision boundary: when 85 percent per step is a coin flip at ten

A single threshold assumes a single place to put it. But a ten-step run only succeeds if every step succeeds, and probabilities multiply. Take a per-step success rate of 85 percent, healthy on any dashboard you’ve built: run ten steps, and 0.8510 works out to roughly 20 percent. One clean run in five.

Per-step monitoring never multiplies. It reports the 85; your users live the 20.

Ground truth arrives: when the label lands after the action does

Your agent files a ticket, updates a CRM record, and drafts a reply. A human reads the reply on Thursday; the CRM record sits unread.

Traditional monitoring compares a model’s output to a ‘ground truth’ label to check if it was right, but that label has to come from somewhere. When the output was a prediction, a human could label it fast. When the output is an action, the only real judge is a human checking the result days later, or never. 

So teams substitute a cheap automated verifier instead: a script that checks the output, not a person. MAST found “many existing verifiers perform only superficial checks”, like confirming code compiles rather than confirming it’s correct. One ChatDev-built chess program passed every one of those checks, shipped with runtime bugs anyway, and scored just 25 percent on the ProgramDev benchmark.

A human in between: when the only witness to the action is the trace

This assumption is the expensive one, because the agent doesn’t just predict; it acts. Remove the human, and the trace (the log of what the agent did) becomes your only evidence that the action was correct.

But a trace can be faked, even by accident. A CrewAI issue documents agents writing out a fake-but-convincing sequence: ‘I ran the tool, here’s what it returned,’ without the tool ever actually running. The model simply generated text that looked like a real tool call and result. Native tool calling (where the system, not the model, executes the action) avoids this specific bug. But the deeper problem survives it: my pipeline’s trace was fully green, and it was accurate telemetry about work that was still done wrong.

What Replaces Them: Instrumenting the Trajectory

Retiring a signal is harder than adding one. Here is the mapping worth working from.

Cost per successful trajectory. A run that burns 40 tool calls and fails costs more than one that burns 12 and succeeds, and per-call dashboards rank them the other way around, since they score calls, not outcomes. Multi-agent systems already use roughly 15 times the tokens of a chat interaction, so the denominator is where the money hides.

Hard caps instead of recursion limits. Most frameworks let an agent retry up to a fixed recursion limit, often as high as 20, on the assumption that each retry is progress. A commenter on a LangGraph issue describes an agent looping to its recursion limit while “burning tokens the whole time with no visibility into what’s happening.” 

It wasn’t progress; the agent kept hitting the same deterministic tool error it had no way to reason past, and each retry just regenerated a near-identical call. A hard cap fixes what a recursion limit can’t: it flags a run once retries stop looking like progress, rather than waiting for the count to run out. One practitioner recommends flagging after 3 to 5 identical retries, not 20.

Configuration as a monitored surface. Behavioral traces record what the agent did, not that someone edited one line of the system prompt an hour earlier. That edit shifts policy while the data stays untouched, so every drift monitor watching the data stays silent by construction. The fix is treating the whole configuration as one diffable unit: prompt, tools, model, and parameters, versioned together, not just the prompt in isolation.

The instrument I’d add first is a deterministic pre-check. I moved my own pipeline onto one that runs before the model reviewers do; after watching them wave through mechanical defects, a regex settles it in under a second. Model-based evaluation is the wrong tool for anything a cheaper one can decide.

When the Old Stack Is Still the Right Answer

Not every LLM system is an agent, and losing that distinction is how teams pay for trajectory infrastructure they never needed.

A single model call with no tools and no memory is a stateless scoring service that emits text. Every assumption above holds. Monitor it like a model: input distribution, output quality, latency, cost.

LLMOps is enough for a wrapper. A wrapper that retrieves once and generates once, with no loop, needs prompt versioning and output evaluation. LLMOps covers that, and it’s enough. Trajectory tooling on a two-step pipeline buys you storage costs and a dashboard that goes unread.

The strongest version of the skeptical case comes from inside the industry. Langfuse co-founder Marc Klingen wrote on Hacker News in February 2024 that “Observability does not need to be reinvented to get detailed traces/metrics/logs of the LLM part of an application”, relocating the value to prompt management and evaluation.

The plumbing argument holds. He was writing about LLM applications, two years before agents were the question. On the plumbing, he’s right. Transport, span model, storage, and sampling are OpenTelemetry, and an agent needs no new wire protocol.

Compounding is not a law. MAKER drove a task past a million model steps with zero errors through extreme decomposition and voting, so step multiplication is an engineering problem, and a solvable one. It’s also engineering that only shows up at the level of the whole run.

The unit of analysis. APM asks whether a call succeeded, and agents fail with every call succeeding. MAST’s second-largest category is inter-agent misalignment, where every component works, and the coordination between them fails.

The bill for the new discipline. Measuring pass^k means running each task k times. At k of 8, your rollout spend is eight times what single-sample scoring costs.

So the position is conditional, and this part is my judgment. If your agent is read-only and a human reads every output, the old stack plus span-level tracing holds up for now. If it writes, or if a run routinely exceeds five tool calls before it returns, you have to start measuring the trajectory.

Decision Framework: Two Questions Before You Buy a Trace Vendor

Two questions settle most cases.

• Does the system decide without a human between the decision and its consequence? If a person reviews every output before anything happens, your blast radius is bounded and output-level monitoring covers most of the risk.

• Do its decisions produce external side effects? A write to a datastore, a payment, a message sent, a deploy triggered. Side effects turn a quality problem into an incident.

If a system sits between two branches, take the more conservative one. Trajectory instrumentation you did not need costs you storage; output-level monitoring you did need costs you an incident a customer reports before your dashboard does.

Conclusion: Start With the Agent That Writes

Adopt in one direction rather than all at once. Take the single agent with write access, instrument its full trajectory, set a hard iteration cap, and gate its side effects. Leave the read-only agents on what you already have until that first one is boring.

The tracing was the cheap half of this migration. The expensive half is deciding which inherited signals to stop trusting, and doing it before an agent takes an action that a green dashboard told you was fine.

Further Reading

• Sierra: tau-bench (the pass^k metric, the cleanest measure of agent consistency).

• UC Berkeley: Why Do Multi-Agent LLM Systems Fail? (MAST, the largest empirical multi-agent failure taxonomy).

• Anthropic: How we built our multi-agent research system (statefulness, compounding errors, and the token multiple).

• OpenTelemetry: GenAI agent spans (instrumenting an agent, vendor-neutrally).

• NVIDIA: NeMo Agent Toolkit (profiles workflows from agent level down to tokens).

• Gartner: Over 40% of agentic AI projects will be canceled by end of 2027 (the forecast every board deck now quotes).

···

Thanks for reading. I’m Mostafa Ibrahim, founder of Codecontent, a developer-first technical content agency. I write about agentic systems, RAG, and production AI. If you’d like to stay in touch or discuss the ideas in this article, you can find me on LinkedIn here.