Back to AI Research

AI Research

BLADE audits how agents use information when tracing a failure

Key Takeaways

  • A training-free framework distinguishes introducing, propagating and repairing errors at handoffs between agents.
  • ## Information can be correct before a handoff
  • [BLADE](https://arxiv.org/abs/2610.18780) addresses failure attribution in language-model multi-agent systems.
  • A dependency trace can show where information came from, but a receiving agent may misinterpret a correct result and pass the mistake onward.
  • The researchers therefore examine use of information as well as its origin.

Information can be correct before a handoff

BLADE addresses failure attribution in language-model multi-agent systems. A dependency trace can show where information came from, but a receiving agent may misinterpret a correct result and pass the mistake onward.
The researchers therefore examine use of information as well as its origin. Their framework, Blame Localization via Audited Dependency Evidence, aims to identify the responsible agent and the decisive error step in a failed run.

Audit the source-to-decision transition

BLADE uses Jev for structured probabilistic judgments. Dependency probabilities select relevant source-decision handoffs for closer examination. The audits ask whether the receiving decision uses its sources faithfully and meets the task requirements.
This distinguishes error introduction, propagation and repair. The findings then support a global diagnosis that considers all execution steps. The framework is training-free and does not require reference answers for the evaluated attribution process.
Across 1,228 failed runs in five Who&When and Who&When Pro subsets, the authors report best or tied-best step accuracy among eight strategies using the same Jev decision interface. The comparison's shared interface is important: it helps isolate the attribution strategy from a change in the model providing judgments.

Preserve the evidence around each decision

Franklin analysis: an agent trace should retain the input actually available at a handoff and the decision made from it. A summary that only lists which agent spoke first can hide the point where a correct fact acquired an incorrect interpretation.
A useful diagnosis should show the relevant source, the receiving claim and the task requirement it violated. Then check whether later agents repeated, corrected or independently introduced the error. BLADE turns those distinctions into an auditing procedure, making a failure explanation more specific than naming whichever agent happened to produce the final answer.

Comments