Back to AI Research

AI Research

Harness-Zero: Harness Distillation via Agent-as-Har... | AI Research

Key Takeaways

  • Harness-Zero is a research framework designed to improve the performance of AI agents by "distilling" the benefits of specialized external systems—known as h...
  • Agent harnesses, the external systems that mediate model-environment interaction, can substantially improve agent performance, but their gains remain tied to the harness at deployment.
  • Because the best harness varies across domains, instances, and models, a general-purpose agent must either settle for a suboptimal shared harness or route among an ever-growing set of specialized ones.
  • The challenge is that the two harnesses differ in action space and available information, so guidance from the optimized harness cannot serve directly as supervision for the target one.
  • We introduce Harness-Zero, which enables harness distillation through agent-as-harness.
Paper AbstractExpand

Agent harnesses, the external systems that mediate model-environment interaction, can substantially improve agent performance, but their gains remain tied to the harness at deployment. Because the best harness varies across domains, instances, and models, a general-purpose agent must either settle for a suboptimal shared harness or route among an ever-growing set of specialized ones. We therefore study agent harness distillation: using a domain- or instance-optimized harness as training-time guidance and transferring the behaviors it induces into model weights, so that its gains survive under a single fixed target harness. The challenge is that the two harnesses differ in action space and available information, so guidance from the optimized harness cannot serve directly as supervision for the target one. We introduce Harness-Zero, which enables harness distillation through agent-as-harness. Guided by the optimized harness, a harnessing agent corrects student responses before execution in the target harness's action space, turning harness guidance into training demonstrations. Fine-tuning on the resulting trajectories internalizes harness-induced behavior into the model, so the specialized harness can be removed at deployment. Our experiments spanning knowledge work, tool use, and science domains show that: (1) For frontier LLMs using the same evolved harness, agent-as-harness outperforms code-as-harness. (2) With the specialized harness removed at deployment, Harness-Zero improves the base model's macro-average task success from 23.3% to 44.3%, even exceeding the 41.7% it reaches with that harness still attached. (3) Harness-Zero recovers harness-induced behaviors absent from the base model, with 82.3% average recovery across 28 patterns in the three domains.

Harness-Zero is a research framework designed to improve the performance of AI agents by "distilling" the benefits of specialized external systems—known as harnesses—directly into the model's own weights. Currently, AI agents rely on these external harnesses to manage tools, organize workflows, and handle complex tasks. However, these gains are usually tied to the specific harness being used, forcing developers to choose between a generic, less capable harness or a complex, specialized one that is difficult to maintain. Harness-Zero solves this by using an optimized harness as a teacher during training, allowing the model to internalize those specialized behaviors so it can perform at a high level even when the external harness is removed.

The Challenge of Harness Dependency

AI agents often use "harnesses" to mediate their interaction with the environment, such as managing file systems, executing code, or organizing scientific research steps. While these systems significantly boost performance, they are often rigid. If a developer wants to use a different, more specialized harness, the agent may struggle because the new system has different rules, action spaces, or information access. Harness-Zero addresses this by moving the "intelligence" of the harness into the model itself, ensuring that the agent retains its specialized capabilities regardless of the specific environment it is deployed in. The ai agents story also surfaces in Google’s Gemini AI Accessed Three Outside..., adding another angle.

How Agent-as-Harness Works

The core innovation of this framework is "agent-as-harness." Instead of forcing a model to interact directly with a complex, foreign harness, the system wraps the agent in a "harnessing agent." During training, this harnessing agent acts as a supervisor. It reviews every action the student model proposes, comparing it against the guidance provided by an optimized reference harness. If the student’s proposal is sound, it is passed through; if it is suboptimal, the harnessing agent makes the smallest possible correction to align it with the optimized behavior. These reviewed, corrected trajectories are then used to fine-tune the student model, effectively teaching it to "behave" as if it were still being guided by the specialized harness.

Key Research Findings

The researchers tested Harness-Zero across knowledge work, tool use, and scientific reasoning domains, yielding several notable results:

  • Superior Guidance: When using the same evolved harness, the "agent-as-harness" approach outperformed traditional code-based harness methods.

  • Lasting Performance: After the specialized harness was removed at deployment, the distilled model achieved a macro-average task success rate of 44.3%, significantly higher than the 23.3% baseline and even exceeding the 41.7% success rate achieved when the specialized harness was still attached.

  • Behavioral Recovery: The distilled models successfully internalized approximately 82.3% of the specific behavioral patterns that were originally induced by the specialized harnesses but were absent in the base models. The ai agents story also surfaces in Google AI Introduces EnvHarness for Adaptive..., adding another angle.

Implications for Deployment

By internalizing these behaviors, Harness-Zero allows for more efficient deployment. Because the specialized harness is no longer required at runtime, the agent can operate under a simple, fixed target harness. This eliminates the need for complex routing between different specialized systems and reduces the overhead associated with maintaining multiple, domain-specific scaffolding tools. The result is a more capable, general-purpose agent that carries its expertise within its own parameters. The ai agents story also surfaces in Arm unveils AI-native mobile platform for..., adding another angle. as detailed in the full paper on Arxiv

Comments (0)

No comments yet

Be the first to share your thoughts!