When to Use One Model and When to Use a Team of Agents

On a recent AI capacity buildout, I scheduled a control plane node into an early cutover wave because the 2 week traffic snapshot I had of it looked effectively isolated. It was not. It was a monthly capacity reconciliation job. On the 21st it woke up, half the reconciliation now crossed a data center boundary, and pipeline latency moved from roughly 10 minutes to close to 60.

• Observed traffic window: 14 days

• Actual job frequency: 30 days

• Observed dependency edges on that node: 12

• Actual dependency edges (after the incident): 17

• Cutover latency on the affected pipeline: ~10 min to ~60 min

The graph was not wrong. My observation window was. We had used a 2 week capture window as the source of truth for a job that only ran once a month.

The failure mode I care about is a reasoning model that looks right on the evidence it was shown, and quietly wrong on the evidence it was not. A single model handed all of this evidence will smooth the contradictions between it and produce a confident answer. A small team of specialist agents, with a coordinator that names contradictions instead of averaging them, does not. The rest of the article is that team.

One reasoning model is the wrong shape

The work at this scale is not one question. It is dozens of questions that share very little context with each other. Inferring what talks to what from flow logs is nothing like forecasting the tail of utilization under a virtual network throughput ceiling, which is nothing like reading a runbook that has not been touched in years.

I tried the monolithic version first. A single strong reasoning model, prompted carefully, fed all of the evidence, and asked for a go / no go on each host in the wave. It failed in 2 ways. In runs where it kept the thread across all 5 tasks, it wrote a beautiful synthesis that quietly averaged away the contradictions between the flow graph and the runbook. In runs where the evidence for one task got long, it dropped that task and confidently answered on the remaining 4. Both failure modes produce output that looks correct on the page, which is what made them expensive.

The subagent pattern that Codex and Claude Code both ship solves the second failure directly, because each specialist only ever sees the evidence for its own question. It also makes the first failure a coordinator design problem, which is a smaller and more concrete thing to get right than a prompt.

The 5 specialists, and the model behind each

Each specialist returns a typed finding, not a paragraph. Claim, evidence, confidence, freshness, and one of 3 verdicts: ok, warn, or hold. A verdict from a runbook that has not been touched in years is not the same evidence as a verdict from a flow log captured yesterday, and the coordinator has to treat them differently.

The inference agent (Codex) aggregates flow logs over a full business cycle, weights edges by connection count more than by bytes, and proposes candidate groups of hosts that should move together. A single quantitative task driven to completion over months of logs. Log damping the bytes matters. A nightly 4 TB backup is a huge edge by volume and irrelevant to cutover ordering. A chatty 200 byte health check every second is the thing that breaks.

The sequencing agent (Claude Code) turns those groups into an ordered wave plan. It walks direction on the cross group edges, cross references extraction results for hard constraints (rollback documented?, manual steps live during cutover?), consults the capacity agent’s headroom for the target wave, and only then emits an ordering. Orchestration of several smaller subtasks with different tools, which is where Claude Code’s subagent primitive earns its keep.

The capacity agent (Codex) forecasts the tail of utilization for each wave. Not the point forecast and not the mean, which no one has ever provisioned to and slept well. A P90 or P95 for the specific window, calibrated against a coverage backtest on the last 5 windows. On one recent program the miscall on this number carried a several million dollar consequence, which the P90 section below unpacks.

The watch agent (Codex for the detector, Claude Sonnet for the 3 a.m. spokesperson) monitors the cutover for silent regressions. The detector is a rolling z score on a single key indicator plus a joint anomaly signal across several others. Small, deterministic, cheap to run per second. Codex writes and maintains it. But when the alarm fires and someone asks why, the answer has to fit in one sentence a tired human can act on. Opus 5 is currently too talkative for that moment, so the on call explanation goes through Sonnet.

The extraction agent (Claude, Skills style) reads runbooks, change tickets, and postmortems and returns typed facts, not prose. Upstream dependencies, downstream dependencies, whether rollback is documented at all, and a count of the manual steps a human has to run live during the cutover. That last count is not documentation, it is a risk score. Structured reads over unstructured text is the mode Claude Skills was designed for.

^{Figure 1. 5 specialists feed a coordinator. Codex behind the quantitative single track tasks, Claude behind the orchestration and document reads.}

The coordinator that names contradictions

The coordinator is the piece I most want a reader to take away from this article. It fans the same question out to the 5 specialists in parallel, waits for their typed findings, and then does something specific: it refuses to average them. When 2 specialists disagree, the coordinator names the disagreement and hands the room a decision brief in the order a human will have to think about it.

Consider a wave decision for a single host, where the inference agent and the extraction agent land on different verdicts.

# inference agent, running under Codex on 90 days of flow logsFinding(    agent="inference",    claim="no cross wave dependencies",    confidence=0.94,    freshness_days=1,    verdict="ok",)# extraction agent, running under Claude Skills on the runbookFinding(    agent="extraction",    claim="monthly reconciliation aggregator",    confidence=0.88,    freshness_days=1095,   # runbook has not been touched in 3 years    verdict="hold",)

A coordinator that averages produces the wrong answer. It looks at 0.94 ok and 0.88 hold, picks the majority signal, and returns probably ok. The wave ships. On the 21st it wakes up. This is the failure mode from the opening.

A coordinator that names contradictions produces something the room can actually act on.

# coordinator output on the same host{    "verdict": "hold",    "contradictions": [        {            "kind": "graph_vs_runbook",            "inference[codex]":  "no cross wave dependencies "                                 "(freshness 1d, confidence 0.94)",            "extraction[claude]":"monthly reconciliation aggregator "                                 "(runbook freshness 1095d, confidence 0.88)",            "note": "freshness gap means the graph may be capturing "                    "a subset of real behavior"        }    ],    "risk_signals": {        "graph_vs_runbook_conflict": True,        "capacity_coverage_below_policy": False,        "extraction_confidence_low": False,        "manual_cutover_steps": 4,        "specialist_missing": False    },    "decision_owner": "on_call.migrations@corp"}

3 things carry the weight in that output. contradictions is where evidence conflicts get surfaced, not smoothed. risk_signals is a small function of the specialist findings that a downstream router uses to decide whether to gate the wave on a human at all. And decision_owner is a human name, resolved to an on call rotation, agreed by leadership and signed off before the cutover window opens. The rest of this article is what each of these pieces buys, in incidents I actually paid for.

import asynciofrom dataclasses import dataclassfrom typing import LiteralVerdict = Literal["ok", "warn", "hold"]# Each specialist is bound to the model whose shape fits its job.# Codex for single track quantitative work driven to completion.# Claude Code for orchestration across smaller sub tasks.# Claude Skills for structured reads over unstructured documents.# The article names 5 specialists (see Figure 1). The watch specialist# uses two models — Codex for the detector, Claude Sonnet for the 3 a.m.# spokesperson — so its binding is a dict, not a single tuple.EXPECTED_SPECIALISTS = ("inference", "sequencing", "capacity",                       "watch", "extraction")SPECIALIST_MODELS = {    "inference":  ("openai",    "codex-latest"),    "sequencing": ("anthropic", "claude-code"),    "capacity":   ("openai",    "codex-latest"),    "watch":      {"detector":  ("openai",    "codex-latest"),                   "explainer": ("anthropic", "claude-sonnet-4-5")},    "extraction": ("anthropic", "claude-sonnet-skill:research"),}@dataclassclass Finding:    agent: str    model: str = ""    claim: str = ""    evidence: str = ""    confidence: float = 0.0    freshness_days: int = 0    verdict: Verdict = "hold"   # fail-closed default    # extraction-only structured facts    manual_cutover_steps: int = 0    rollback_documented: bool | None = None    # capacity-only: realized backtest coverage of the P90 forecast    coverage: float | None = None@dataclassclass Policy:    """All coordinator/watch-alarm knobs, with fail-closed defaults."""    min_coverage: float = 0.90    min_extraction_confidence: float = 0.85    hold_threshold: int = 3    decision_deadline_seconds: float = 120.0    default_on_timeout: str = "rollback"    decision_owner: object = None      # on-call rotation handle    escalation_chain: object = None    # upchain paging handle# Platform helpers assumed available — replace with your own bindings.@dataclassclass AlarmResponse:    status: str    decision: str | None = None    elapsed_seconds: float | None = Noneasync def notify(owner, alarm, sla_seconds: int): ...async def escalate(chain, alarm, decision): ...async def coordinate(host_pair, specialists, policy):    """Fan out to 5 specialists in parallel, refuse to average findings.    The coordinator itself runs under Claude Code, because its whole job    is fan out, wait, reconcile, and hand back a decision brief. That is    the shape Claude Code is currently best at.    """    # 1. Fan out with return_exceptions=True — a crashed specialist    # counts as missing, not as an unhandled TimeoutError that escapes    # this coroutine. Roster is EXPECTED_SPECIALISTS, not the model map.    raw = await asyncio.gather(*[        agent.evaluate(host_pair, model=SPECIALIST_MODELS[name])        for name, agent in specialists.items()    ], return_exceptions=True)    findings = [r for r in raw if isinstance(r, Finding)]    reporting = {f.agent for f in findings}    missing = set(EXPECTED_SPECIALISTS) - reporting    # Every risk_signals dict returned by the coordinator has the same    # four documented keys, plus specialist_missing. That way a    # downstream router never KeyErrors on the hold path.    def _signals(*, missing=False, contradictions=(), cap=None, ext=None):        return {            "graph_vs_runbook_conflict":                any(c["kind"] == "graph_vs_runbook" for c in contradictions),            "capacity_coverage_below_policy":                bool(missing or cap is None or cap.coverage is None                     or cap.coverage < policy.min_coverage),            "extraction_confidence_low":                bool(missing or ext is None                     or ext.confidence < policy.min_extraction_confidence),            "manual_cutover_steps": (ext.manual_cutover_steps if ext else 0),            "specialist_missing": bool(missing),        }    if missing:        return {            "verdict": "hold",            "findings": findings,            "contradictions": [{                "kind": "specialist_missing",                "note": f"no finding from: {sorted(missing)}",            }],            "risk_signals": _signals(missing=True),            "decision_owner": policy.decision_owner,        }    # 2. Name contradictions, do not average.    contradictions = []    inf = next(f for f in findings if f.agent == "inference")    ext = next(f for f in findings if f.agent == "extraction")    cap = next(f for f in findings if f.agent == "capacity")    if inf.verdict != ext.verdict:        contradictions.append({            "kind": "graph_vs_runbook",            "inference[codex]":  f"{inf.claim} "                                 f"(freshness {inf.freshness_days}d, "                                 f"confidence {inf.confidence:.2f})",            "extraction[claude]": f"{ext.claim} "                                  f"(runbook freshness {ext.freshness_days}d, "                                  f"confidence {ext.confidence:.2f})",            "note": "freshness gap means the graph may be capturing "                    "a subset of real behavior",        })    # 3. Capacity fails closed on missing calibration: an uncalibrated    # quantile is not the same as a calibrated one that happens to pass.    if cap.coverage is None or cap.coverage < policy.min_coverage:        if cap.coverage is None:            note = ("P90 backtest coverage is uncalibrated; "                    "treating as below nominal until proven otherwise")        else:            note = (f"P90 backtest coverage {cap.coverage:.1%} against "                    f"nominal {policy.min_coverage:.1%}; "                    f"missing tail lands inside window")        contradictions.append({            "kind": "capacity_miscalibrated",            "note": note,        })    # 4. Verdict roll-up: hold beats warn beats ok. Never averages.    verdict: Verdict = "ok"    if any(f.verdict == "hold" for f in findings):        verdict = "hold"    elif contradictions or any(f.verdict == "warn" for f in findings):        verdict = "warn"    # 5. Same 4 documented signals a downstream router scores against.    return {        "verdict": verdict,        "findings": findings,        "contradictions": contradictions,        "risk_signals": _signals(contradictions=contradictions,                                 cap=cap, ext=ext),        "decision_owner": policy.decision_owner,    }

Where the P90 was quietly lying

The first version of the capacity agent was accurate on the average and confidently wrong at the exact percentile the purchase order was based on. On one recent program the P90 number was pricing aggregate throughput of a single virtualized cloud tenancy, capped at roughly 25 Gbps, so production and non production needed multiple tenancies to reach 50 Gbps in aggregate. Any miscalibration on that P90 translated into an extra tenancy or roughly double database licensing cost over a 6 month window, which the risk ledger estimated at several million dollars of annual exposure.

The fix was not a better model. It was reporting calibration honestly. Before I let a quantile forecast size real capacity, I backtest coverage on the last 5 windows. Does the P90 band actually contain close to 90% of realized values?

^{Figure 2. Realized coverage on the last 5 migration windows. 2 of 5 backtested below the nominal P90, and the misses landed inside the migration window itself.}

87.9% realized coverage against a nominal 90%, with the missing 2% landing inside the migration window itself, is the difference between a wave that lands and one that gets deferred a quarter. I now report coverage alongside every capacity number I bring to a review board, and I would rather bring a wide honest band than a narrow flattering one.

The alarm that fired perfectly and bought me nothing

The first cutover I ran with a real watch agent, the detector fired 11 minutes in. The joint anomaly score on error rate, queue depth, and retry rate had gone somewhere no baseline had ever seen. Exactly the case I built the joint signal for.

I rolled back 40 minutes later.

Between minute 11 and minute 47, several senior people argued on a war room bridge about whether the signal was real, whether the model could be trusted, and whose call it was. The detector had worked. The rollback authority had not been agreed. By the time someone with authority decided, we had accumulated the exact retry storm the signal was designed to warn us about.

async def handle_watch_alarm(alarm, cutover, policy):    """When the watch agent fires, follow the pre agreed policy.    Detection without agreed authority to act is theater. The whole reason    this code exists is so that at 3 a.m., under stress, no one has to    negotiate rollback authority in a war room channel.    """    if alarm.severity < policy.hold_threshold:        await cutover.log(alarm, level="warn")        return AlarmResponse(status="observed")    await cutover.pause(reason=alarm.summary)    await notify(policy.decision_owner, alarm,                 sla_seconds=policy.decision_deadline_seconds)    # Pre-agreed authority means pre-agreed default too: if the on-call    # does not answer within the deadline, the runbook chooses for them.    # A rollback chosen by a human is terminal; a rollback that fired    # because nobody answered is the incident the chain exists for.    timed_out = False    try:        decision = await asyncio.wait_for(            policy.decision_owner.decide(                options=["rollback", "resume", "extend_pause"],            ),            timeout=policy.decision_deadline_seconds,  # e.g. 120        )    except asyncio.TimeoutError:        decision = policy.default_on_timeout  # typically "rollback"        timed_out = True    if decision == "rollback":        await cutover.rollback()    elif decision == "resume":        await cutover.resume()    else:        await cutover.extend_pause()    # Escalate on extend_pause always, and on any timeout-driven default —    # rollback-because-nobody-answered is what the chain exists for.    if decision == "extend_pause" or timed_out:        tag = decision if not timed_out else f"timeout_default_{decision}"        await escalate(policy.escalation_chain, alarm, tag)    return AlarmResponse(        status="handled",        decision=decision,        elapsed_seconds=alarm.age_seconds,    )

The important line in that code is not the pause or the rollback. It is policy.decision_owner, which is a human name resolved to an on call rotation and signed off in the runbook before the cutover window opens. When the watch agent fires, that person has 120 seconds to choose between rollback, resume, and extend_pause. Detection without agreed authority to act does not shorten an incident. On this one it extended it by 36 minutes. Note that when the on-call chooses the outcome, the escalation chain fires only on extend_pause; rollback and resume are terminal decisions that close the incident on their own, so paging leadership after either of them adds noise, not signal. When the choice is made by the timeout instead of by a human, however, the chain fires either way — a rollback that happened because nobody answered at 3 a.m. is precisely the situation the escalation chain exists for. And if the on-call does not answer within decision_deadline_seconds (120 by default), the pre-agreed default_on_timeout fires — typically rollback — because a paused cutover with no decision is the incident, not the alarm.

Risk based routing for the humans still in the loop

Gating every specialist finding above warn on a human scales badly. Median approval wait creeps up, reviewers batch, batches turn into skims, skims turn into approvals on things no one actually read. The fix is routing by risk, not by operation type. The coordinator surfaces the 4 documented risk signals shown above (graph versus runbook contradictions, capacity coverage below policy, low confidence on hard extraction constraints, count of manual cutover steps), plus a specialist_missing flag that goes True on the hold path so the router never has to key-error to notice a dark agent. A downstream router (not shown in the snippet) scores those signals against a policy threshold, only escalates above it, and writes everything below to the audit log.

^{Figure 3. Where the sequencing agent’s plan actually got overridden by a human over a 90 day window.}

Over 90 days, humans overrode the sequencing agent 71 times. About a third were the human catching a periodic dependency the agent had missed. About two thirds were the human being wrong, and the flow data quietly winning the argument in the next planning session. The one third is what I want in front of a person every time. The two thirds is what a risk routed queue lets execute and audit later. Roughly one third the human load, same net outcome.

Model selection: which model behind which specialist

Everything above is about the shape of the system: typed findings, a coordinator that names contradictions, and risk routed humans. The shape is what stays constant across model generations. Model selection is downstream, and it is where I have leaned on Codex for one class of work and Claude Code for another. Short version. Codex is currently far superior on a single specific difficult task driven to completion. Claude Code is superior at orchestrating a bunch of smaller ones. That maps almost one to one onto the specialists above.

I ran a small internal evaluation on the last 90 days of traffic to see whether that intuition held on the actual task mix. Each task was scored on completion rate under a fixed token budget and human review of the output, on a 10 point scale.

2 things this split changed on my current program. Cost per wave decision dropped, because the cheap deterministic detectors run under Codex and only the coordination and document reads pay for premium Claude tokens. Extraction precision on is rollback documented? moved from mediocre to review grade the week I moved it to a Skills style Research skill with a JSON schema on the output.

Limitations, and what I would do differently

• Sample size. The 71 override count, the 87.9% coverage number, and the model evaluation above are all from a single program over 90 days. They are consistent with what I have seen across earlier programs, but they are not a benchmark.

• Model selection is a snapshot. 12 months ago I would have written this article with different frontier models on several rows above. The Codex / Claude Code split is specific to the state of the models in late 2026.

• The extraction agent depends on runbook quality. On teams where the runbook has not been touched in years, freshness on that finding drops confidence enough that the coordinator will treat it as weak evidence. That is correct behavior. It is also a reminder that this system is not a substitute for keeping runbooks alive.

• The decision_owner field is a governance construct, not a model output. If the organization cannot agree who owns the rollback call before a window opens, none of the architecture above helps.

If I were rebuilding this from scratch, I would put more effort into the risk score itself. Right now it is 4 signals combined with a policy threshold. A learned function over those signals, trained on the 90 days of overrides, would almost certainly reduce the number of low value human touches further.

Conclusion

The lesson underneath all of this is not which model won a task in 2026. Models will move. The lesson is the shape. Split the problem into specialists whose outputs a coordinator can compare, make each of them return a typed finding with freshness and confidence, and put a coordinator in front of them that refuses to average contradictions. On top of that, route humans by risk, not by operation type. And write down one human name in decision_owner before the cutover window opens.

Which model sits behind which specialist is the last decision, not the first. Get the shape right and the model choice is cheap to revisit. Get the shape wrong and no frontier model will save the wave on the 21st.

About the author

Naveen Goel is a Principal Program Manager in Amazon (Ring) focused on large scale cloud and AI data center program delivery. His current work centers on hyperscale AI capacity buildouts, with earlier programs spanning regulated cloud migrations, connected device fleets with cloud based AI analysis, and large integrated infrastructure services engagements.