Back to AI Research

AI Research

Proof-or-Stop: Don't Trust the Agent, Trust the... | AI Research

Key Takeaways

  • Proof-or-Stop: Don't Trust the Agent, Trust the Evidence -- Loop Engineering for Verifiable Evidence-Gated Lifecycle Control Autonomous coding agents are inc...
  • Autonomous coding agents increasingly execute multi-step software work, but lifecycle states such as reviewed, tested, DONE, and ready-to-merge remain claims unless supported by current evidence.
  • We present Proof-or-Stop Lifecycle Control, a method that permits lifecycle transitions only when fresh, tracked-source-state-bound, mechanically verifiable evidence satisfies the relevant gate.
  • The method treats agent outputs as claims rather than lifecycle state, and uses proof operationally to mean gate-admissible evidence under a stated trust model, not semantic program correctness.
  • We evaluate an open-source implementation through mechanism tests, a powered control-policy ablation, and operated self-application evidence.
Paper AbstractExpand

Autonomous coding agents increasingly execute multi-step software work, but lifecycle states such as reviewed, tested, DONE, and ready-to-merge remain claims unless supported by current evidence. We present Proof-or-Stop Lifecycle Control, a method that permits lifecycle transitions only when fresh, tracked-source-state-bound, mechanically verifiable evidence satisfies the relevant gate. The method treats agent outputs as claims rather than lifecycle state, and uses proof operationally to mean gate-admissible evidence under a stated trust model, not semantic program correctness. We evaluate an open-source implementation through mechanism tests, a powered control-policy ablation, and operated self-application evidence. The unattended-loop engine passed 10 of 10 scenarios with zero false-DONE, and local-key receipt bundles rejected 18 tamper classes with zero false accepts. In a 9,240-cell ablation, the pre-registered A4 versus A2-prime comparison reduced visible-pass/hidden-fail amplification from 31 of 1,800 injected cells under a compute-budgeted naive loop to 2 of 1,800 under the gated loop, a 1.6 percentage-point improvement in not-amplified rate with a 95 percent confidence interval of [0.8, 2.5]. A near-compute A3 versus A4 comparison, 14 of 1,800 versus 2 of 1,800, indicates that the gain is associated with enforcing review as a lifecycle gate rather than merely adding a reviewer. The self-application corpus contains 565 stories and 1,007 review findings, with 94.8 percent resolved, plus a 68-row high/critical cross-vendor exhibit. These results support Proof-or-Stop as a model-agnostic, host-neutral control layer for deciding which autonomous-agent claims a lifecycle may act on. The evaluation is limited to one model family, 24 ablation tasks, and a self-hosted corpus.

Proof-or-Stop: Don't Trust the Agent, Trust the Evidence -- Loop Engineering for Verifiable Evidence-Gated Lifecycle Control
Autonomous coding agents are increasingly capable of performing complex software tasks, but they often operate in a way where their own reports—such as "tests passed" or "code reviewed"—are treated as final lifecycle states. This paper introduces "Proof-or-Stop Lifecycle Control," a method designed to prevent autonomous systems from advancing work based on unverified claims. Instead of trusting an agent’s narrative, the system requires fresh, mechanically verifiable evidence bound to the specific state of the code. If the evidence is missing, stale, or tampered with, the system stops or repairs the process rather than moving forward.

The Agent-as-Claim Model

The core philosophy of this research is that an agent’s output should be treated as a "claim" rather than a definitive state. In traditional workflows, a green pipeline or a simple "LGTM" from a reviewer might be enough to merge code. However, these can be misleading if the evidence is stale or disconnected from the current version of the software. Proof-or-Stop shifts this by requiring that every lifecycle transition—such as moving from "tested" to "done"—be backed by an evidence gate. This gate verifies that the evidence is current, cryptographically bound to the specific source code, and produced by an authorized actor.

How Evidence Gating Works

The system uses "Loop Engineering" to manage the development process. When an agent proposes a transition, the system checks the evidence against a set of strict requirements. This includes verifying the "material hash" (the state of the source tree), the "head hash" (the specific commit), and the "receipt identity" (the command, arguments, and output of the task). By using these cryptographic bindings, the system ensures that the evidence provided actually corresponds to the code being evaluated. If the evidence fails these checks, the system does not simply ignore the error; it triggers a bounded loop to repair the issue, escalate the problem, or halt the process entirely.

Key Results and Performance

The researchers evaluated this method through a series of mechanism tests and a large-scale ablation study. In a test of 9,240 scenarios, the gated loop significantly reduced the amplification of "visible-pass/hidden-fail" errors—cases where an agent reported success despite underlying defects. Specifically, the gated loop reduced these failures from 31 out of 1,800 cases in a naive, compute-budgeted loop down to just 2 out of 1,800. Furthermore, the system was tested against 18 different classes of tampering, successfully rejecting all of them with zero false accepts.

Scope and Limitations

It is important to note that Proof-or-Stop is a control layer, not a new coding model or a proof of semantic program correctness. It does not guarantee that code is bug-free; rather, it ensures that the lifecycle only advances when the claims made by the agent are supported by admissible, verifiable evidence. The current evaluation is limited to one model family, a specific set of 24 ablation tasks, and a self-hosted corpus. While the results demonstrate that this method is an effective way to manage autonomous agent workflows, the researchers emphasize that strong cross-vendor verification remains a future goal, and the system is designed to be host-neutral and model-agnostic.

Comments (0)

No comments yet

Be the first to share your thoughts!