The Blind Curator: How a Biased Judge Silently Disa... | AI Research

The Blind Curator: How a Biased Judge Silently Disables Skill Retirement in Self-Evolving Agents
Self-evolving AI agents improve by creating new skills based on their past failures. To prevent their internal libraries from becoming cluttered with ineffective or redundant skills, these agents use a "curator" mechanism to retire skills that no longer contribute to success. This paper investigates a critical vulnerability in this process: what happens when the judge responsible for identifying these failures is biased? The researchers demonstrate that while agents can handle random noise, a specific type of bias—where the judge incorrectly labels failures as successes—can silently disable the curator, causing the agent to retain harmful skills without any obvious drop in performance metrics.

The Danger of False-Pass Bias

The study distinguishes between two types of errors a judge can make: symmetric noise and false-pass bias. Symmetric noise occurs when a judge is simply inconsistent, flipping both successes and failures randomly. The researchers found that agents handle this gracefully; the curator remains functional, though it may require more data to make decisions.
In contrast, false-pass bias is catastrophic. When a judge consistently reports failures as passes, it breaks the evidence loop the curator relies on. The researchers identified a "sharp threshold" for this bias. Once the rate of false passes exceeds a specific point, the curator effectively stops retiring any skills, regardless of how much data the agent collects. Because this failure happens at the structural level, the agent continues to operate, making the "blind curator" invisible to standard performance monitoring.

Why the Curator Goes Blind

The core of the problem lies in the fact that the same signal used to create new skills is also used to retire old ones. When a judge reports a failure as a pass, it starves the system in two ways: it prevents the agent from learning from its mistakes (synthesis) and it prevents the curator from identifying which skills are failing (retirement).
The researchers found that the most dangerous judge is not necessarily the one that is most inaccurate, but the one that is "half-blind." A judge that is moderately biased allows the agent to keep creating new, potentially low-quality skills while simultaneously disabling the mechanism that would normally prune them. This leads to a buildup of ineffective skills that the agent continues to use, even though the system's aggregate performance might look normal.

A Practical Audit for Deployment

To help developers identify these risks before deploying an agent, the authors propose a "defect-injection audit." By intentionally introducing known errors—such as broken citations or logical contradictions—into a test set, operators can measure exactly how their judge behaves.
This audit acts as a go/no-go test. If the audit reveals that a judge has a high false-pass rate, the operator knows they are on the wrong side of the threshold and that their agent’s skill library will eventually drift into a state of unmanaged decay. The researchers emphasize that this is a behavioral safety result: the goal is not to measure if the agent is "smart," but to ensure that the governance mechanism—the curator—is actually capable of doing its job.

Key Takeaways for AI Governance

The study concludes that the health of an agent’s skill library is a property of the reward signal, not just the agent’s architecture. While a strict, conservative judge might lead to "starvation" (where the agent struggles to learn because it is too critical), it is ultimately safer than a lenient judge that creates a false sense of security. Because the failure of the curator is "silent"—meaning it does not show up in aggregate metrics—operators cannot rely on standard performance tracking to detect when their agent’s internal governance has failed. Instead, they must proactively audit the reliability of the judge itself.