Co-Evolving Harnesses and Models: On-Policy Correct... | AI Research

Co-Evolving Harnesses and Models: On-Policy Correction Helps Weaker Models Catch Up Where Imitation Fails
This research explores how to effectively combine two powerful methods for improving AI agents: evolving the "harness" (the system prompts, tools, and scaffolding surrounding a model) and fine-tuning the model’s internal weights. While both methods are known to improve performance, this paper investigates why they often conflict when used together. The authors discover that while a harness can be perfectly tuned to a smaller model’s unique way of planning, traditional methods of teaching that model to act like a stronger "expert" can accidentally break this delicate fit, leading to performance regressions. To see meta in practice, How to Make Cinematic Commercials walks through a concrete example.

The Conflict Between Harnesses and Weights

An agent’s performance depends on the interaction between its model weights and its harness. The authors found that when they evolved a harness specifically for a smaller model, it significantly boosted performance. Because a stronger expert model could also use this harness effectively, the researchers initially tried to improve the smaller model by having it imitate the expert’s full trajectories. Surprisingly, this caused the smaller model’s performance to drop significantly across all seven tested enterprise tasks. The analysis revealed that the model was successfully learning the expert's knowledge, but it was also adopting the expert's complex planning strategy. Because the smaller model lacked the underlying competence to execute that specific strategy, it lost the "fit" with its custom-built harness, resulting in a net loss of capability.

A New Approach: On-Policy Expert Correction

To solve this, the researchers developed an "on-policy" correction pipeline. Instead of forcing the smaller model to copy an entire sequence of actions from an expert, they used a meta-level agent to monitor the smaller model’s own attempts at a task. When the model made a mistake, the system identified only the specific turn where the error occurred and asked the expert to rewrite just that single step. By keeping the rest of the model’s original planning sequence intact, the smaller model could learn from the expert without losing its own rhythm or breaking its compatibility with the evolved harness. The ai agents story also surfaces in AI Agents Going Rogue Renew Calls..., adding another angle.

Why This Matters for Agent Development

This study demonstrates that harness evolution and model fine-tuning are not independent levers. Once a harness is optimized for a specific model, it becomes deeply coupled with that model's behavior. The authors’ findings suggest a new design principle for developers: rather than relying on wholesale imitation of stronger models, it is more effective to provide targeted, on-policy supervision at the specific states where a student model struggles. This approach allows for an iterative "co-evolution" loop, where the harness and the model weights can be updated in tandem to achieve higher performance on complex, domain-specific enterprise tasks without interfering with one another. The same reasoning question is explored in Wrong Prediction, Right Answer, which adds a research perspective. as detailed in the full paper on Arxiv