The routing decision needs its own evidence
Disagreement-Guided Weak-to-Strong Routing studies when a small language model should ask a stronger model for help. Small models can meet cost and latency constraints on edge devices or robotic systems, but deciding which responses need escalation is a separate problem from generating the responses themselves.
Many routing approaches depend on signals of correctness. In open-ended or agentic tasks, obtaining those signals can require expensive ground-truth supervision. This paper asks whether a router can instead use information available directly from model-generated responses.
Compare models rather than labelled answers
The researchers first characterize an optimal routing policy while allowing multiple responses from the weaker model through local inference. They then derive an approximation based on model-to-model comparison, introducing a disagreement measure that does not require ground-truth labels.
The strength of that approximation has a stated condition: it becomes more accurate as the stronger model becomes a better proxy for the ground-truth distribution. Calling a model stronger does not, by itself, make all of its answers correct. That assumption is important when interpreting what disagreement can tell the router.
A cost and accuracy tradeoff
Across the evaluated agentic and single-turn tasks, the authors report navigating the tradeoff between routing cost and response accuracy without ground-truth supervision. They also report cases where disagreement-based routing outperforms methods trained directly to predict correctness, even when correctness signals are available for training.
The captured abstract does not supply a universal cost reduction or accuracy gain, so none should be inferred. Franklin's reading is that a reproduction should track how often requests are escalated, the cost of generating local candidates and the final task outcome. The method is useful to examine where labelled routing data is scarce, while remembering that agreement between models is not an independent certificate that an answer is true.
Read the disagreement-guided routing paper.
Comments