This article was co-authored by Rahul Vir and Reya Vir.
to Token Efficiency
We have officially moved past the AI prototyping phase. Building on the concepts in Escaping the Prototype Mirage [1], product and engineering teams across every industry are now shipping agentic applications that solve workflows previously dominated by manual grind. Building these autonomous agent prototypes is now a breeze. It is as simple as using key concepts like recursive Agentic Loops (Observe-Think-Act) for execution, setting up headless gateways to connect agents via chat apps, and relying on stored state that persists across reboots (as explained in [1]). But graduating them to reliable products is another story. The new frontier isn’t proving agents can work, it’s proving they can work profitably.
At the same time, internal metrics at enterprises like “token maxing” (unconstrained token use to achieve best results) that were appropriate for the prototyping stage are shifting to measuring the “value-to-token-spent” ratio as agentic products scale. After all, most products need to be profitable and maximize margin as they are moving from leveraging cheap traditional compute (TradCompute) to solve user problems toward using AI intelligence for the same.
But models need reasoning freedom and recent studies have shown that exploratory agentic workflows outperform fixed paths, opening new paths, creating MCP tools, and building infrastructure to solve the problem more efficiently in most cases. This brings the question of balancing the model’s need for agency with the economic reality of inference costs.
Why Constrained Agents Fail to Converge
Agent harnesses store your task context and objectives in markdown (*.md) files, which don’t typically represent tight workflows, but rather outline the intent or the objective you want to accomplish.
The Paradox of Objective Failure: In studies on agents solving complex problems, researchers found that providing strict, highly-constrained guidelines where each of the agent’s action takes it closer to the goal, leads to getting stuck in a local optima and suffering an objective failure. An example from Professor Jeff Clune’s research on open-ended agent learning illustrates this perfectly: an agent in a maze, when constantly rewarded solely for seeking the direct path to the exit, will repeatedly bang into walls and get trapped in a local optimum, never reaching the end [2].
The Power of Unconstrained Harnesses: Contemporary agent harnesses like Google Antigravity and Anthropic’s Claude Code have been so effective because they allow agents to create, orchestrate, execute complex tasks, and even create their own tools without strict human micro-management. They succeed because they are given the freedom to explore circuitous paths.
Consider an edge case in a routine medical intake workflow: if we rigidly constrain a healthcare agent to purely follow a predefined scheduling flow, it breaks in the real world. If a patient mentions chest pain midway through that routine intake, the agent’s Agentic Loop must have the autonomy to instantly recognize the urgency, abandon the scheduling flow, and trigger a safety escalation. It should utilize what we previously defined as a `No-Reply Token` to suppress booking chatter and route the context directly to a human nurse [1]. Rigidly constrained prototypes fail this test spectacularly because they cannot adapt to critical, out-of-bounds context.
Infinite Goal Searching is Expensive
While providing agency is essential to discover a solution initially, running a full open-ended search for every user workflow request can lead to massive and unsustainable token consumption. At this stage the agent has found a valid path and this approach is inherently allowing it to re-explore or “hallucinate” the workflow structure. While this can be self correcting, such subsequent runs of a similar request destroy enterprise token economics.
For example, routing medical intake workflows and even the edge cases that require an escalation can be learnt over a period of time. A clinic or a solution provider’s workflows will graduate to deterministic paths for the most part, leaving some autonomy reserved purely for rare outliers and complex edge cases.
Architectural Solutions Through Early Commitment and Deterministic Replay
Early Commitment has shown promise in structured problem solving and it can be applied to agentic workflows as well [3]. It involves classifying the problem first, say by structuring the system prompt to require the model to output a specific classification tag. By forcing an agent to classify the problem type and establish constraints before it generates the execution logic, you prevent the agent from hallucinating or exploring dead-end paths. This cuts out noise and focuses the agent purely on execution rather than continuous exploration.
For instance, in a telehealth triage workflow, we can enforce Early Commitment by requiring the agent to definitively classify the encounter as a “routine prescription refill” before taking any action. Once committed to this specific constraint, the agent restricts its tool calls strictly to the pharmacy database, completely bypassing the expensive, open-ended diagnostic reasoning paths it might otherwise wander down trying to diagnose a patient.
A recent study by Wang, X., et al. introduces the LOOP Skill Engine Framework, which takes early commitment to the infrastructure level by using a one-shot recording and deterministic replay paradigm [4]. The agent can autonomously explore once using full reasoning, and the system then compiles that successful trace into a branch-free recipe. For all future runs, the LLM can be bypassed, guaranteeing execution determinism and slashing token usage by over 93.3% for daily tasks, and up to 99.98% for high-frequency executions. This concept can be extended to agentic workflows.
Consider the generation of daily clinic compliance reports or standard post-discharge summaries, which are highly stable, repetitive tasks. Starting from exploratory and then quickly graduating to a deterministic framework, an agent has to reason through the complex data extraction from the Electronic Health Record exactly once. For the next hundred patients discharged with the same procedure, the system executes that exact branch-free recipe, reliably swapping in the new patient’s vitals and dates without ever invoking the LLM. This guarantees zero hallucinated data on repetitive healthcare tasks while maximizing token efficiency.
ML practitioners need to make the call between a pure deterministic replay (like LOOP) that maximizes token savings, and a hybrid approach (storing the explored path in a SKILL.md file). The hybrid approach trades some of those token savings back in exchange for reasoning through a guided path that is highly optimal, yet leaves enough flexibility to self-adapt to a changing underlying framework. Whether this skill file is updated manually or through an autonomous self-improving mechanism, preserving this reasoning headroom ensures adaptability and long-term robustness. For example, if the database structure changes, the agent is able to update the SQL queries and extract the information.
Conclusion: The Explore-Commit-Measure ML Pipeline
ML engineers and Product Managers must adapt their applications to leverage the vast intelligence of autonomous agents and embrace unconstrained agent harnesses for initial problem discovery and complex, one-off edge cases. This yields optimal solutions without running an expensive reinforcement learning cycle (which is often blocked by lack of expertise, platform constraints, training cost or closed models).
Once we have found a near-optimal path, token economics for structured and repetitive tasks demand we enforce early commitment in prompt design, utilizing deterministic replay architectures to cache the execution path.
As agentic products scale, we must shift operational metrics away from simple task success rates, moving instead toward token-efficiency and value-per-token generated.
References
- Vir, R., & Vir, R. (2026, March 4). Escaping the prototype mirage: Why enterprise AI stalls. Towards Data Science.
- Clune, J. (2025, February 12). Guest Lecture 6 CS329A by Prof. Jeff Clune: Open-ended Agent Learning in the Era of Foundation Models [Video]. YouTube.
- Vir, R. (2026, January 1). Why early commitment helps AI solve structured problems. Towards AI.
- Wang, X., Yu, K., Liang, X., Wang, L., & Han, C. (2026). Good to go: The LOOP skill engine that hits 99% success and slashes token usage by 99% via one-shot recording and deterministic replay. arXiv.