Running a frontier model for every computer action can be expensive, while a smaller model may need help during difficult parts of a task. The LEASH paper studies how to decide when to hand off and how to monitor the stronger agent afterward.
Estimate uncertainty from existing activations
LEASH has initial routing, step-level escalation and a shadow observer. It compares the smaller model's hidden states with states from past successful and failed trajectories. A state that resembles earlier failures supplies a signal to consider stronger assistance.
During the smaller agent's own execution, the estimator reuses activations already produced while generating an action. The authors describe this as avoiding an extra forward pass or separately trained monitor for that step.
Keep observing after a handoff
A shadow observer continues oversight of the stronger black-box agent. This addresses the fact that escalation does not guarantee recovery: a stronger model can also lose track of a long task, while an API may expose no internal states to inspect.
The framework uses the smaller model's uncertainty signal across these layers. It is described as training-free, though its estimate depends on reference trajectories from earlier successes and failures.
Interpret the reported cost and success
The experiments evaluate OSWorld across four model backbones. At matched escalation rates, the authors report gains of 6.55 to 11.94 points over random escalation.
With Qwen3.5-35B-A3B, they report 73.66% task success at $0.4117 per task, compared with 72.90% for always-on Claude Opus 5, and a 19.2% inference-cost reduction. Those costs belong to the paper's experimental setup and are not a quote for current provider pricing.
For an application, evaluate the routing policy under the same tasks, budget and oversight rules. The useful comparison is whether uncertainty-guided escalation improves completion at a controlled cost, including failures that continue after the handoff.
Comments