Back to AI Research

AI Research

EDGE: Error Dependency Graph-Guided Multi-Error Att... | AI Research

Key Takeaways

  • Large Language Model (LLM) agents often fail in complex, multi-step tasks, but these failures are rarely caused by a single isolated mistake.
  • Large language model (LLM) agent failures often contain multiple related errors rather than a single mistake.
  • Existing attribution methods usually identify a responsible agent, step, or root cause, but do not explicitly model dependency between errors.
  • We introduce EDGE, an Error Dependency Graph-guided multi-Error attribution framework.
  • EDGE constructs an error dependency graph from observed error events and validates a reliable causal subset through counterfactual rollout.
Paper AbstractExpand

Large language model (LLM) agent failures often contain multiple related errors rather than a single mistake. Existing attribution methods usually identify a responsible agent, step, or root cause, but do not explicitly model dependency between errors. We introduce EDGE, an Error Dependency Graph-guided multi-Error attribution framework. EDGE constructs an error dependency graph from observed error events and validates a reliable causal subset through counterfactual rollout. The inference graph guides a two-stage LLM-as-judge detector for error attribution, and the intervention-validated subgraph provides a more reliable basis for explanation and repair analysis. Experiments on TRAIL and MAST show that EDGE improves category-level multi-error attribution across most evaluated models and settings. Experiments with adapted Who&When-style prompts show that the graph helps across prompting strategies. These results suggest that dependency structure is a useful diagnostic prior for agent failures beyond isolated root-cause prediction.

Large Language Model (LLM) agents often fail in complex, multi-step tasks, but these failures are rarely caused by a single isolated mistake. Instead, they typically involve a chain of related errors where one mistake triggers another. Current diagnostic tools often struggle to identify these interconnected issues, focusing instead on finding a single root cause. The EDGE framework addresses this by modeling agent failures as a "dependency-aware" process, using a graph-based approach to map how different error types propagate through a system.

Mapping Error Cascades

EDGE works by constructing an "error dependency graph" that visualizes how specific error categories recur and influence one another across different failed tasks. By analyzing patterns in historical failure data, the framework identifies which errors are likely to lead to others. It uses two primary methods to build this map: observing statistical correlations in how errors appear in sequence, and performing "counterfactual rollouts." In these rollouts, the system simulates what would happen if a specific error were fixed; if the downstream errors disappear as a result, the system confirms a causal link between the two.

Two-Stage Detection

Once the dependency graph is established, EDGE uses it to guide a two-stage detection process. In the first stage, an LLM analyzes the failed trace to identify an initial set of errors. In the second stage, the framework uses the dependency graph to suggest "plausible downstream errors" that might have been missed. By providing this structured context, the system helps the LLM-as-judge look for specific, logically connected mistakes rather than guessing blindly. This approach separates the initial detection from the verification of downstream effects, making the diagnosis more accurate and easier to interpret.

Improving Diagnostic Reliability

The framework distinguishes between two types of information: observational edges, which help broaden the scope of what the system can detect, and intervention-validated edges, which provide a highly reliable foundation for explaining why a failure occurred. This dual approach allows developers to use the system not just for identifying errors, but also for prioritizing repairs. By understanding the causal chain of a failure, developers can focus on fixing the "upstream" triggers that cause the most significant downstream damage.

Performance and Versatility

Experiments on benchmarks like TRAIL and MAST demonstrate that EDGE improves multi-error attribution across a wide range of LLM models. Because the framework is taxonomy-agnostic, it can be applied to different types of agent systems regardless of how they categorize their errors. The results suggest that incorporating a dependency structure acts as a powerful diagnostic tool, allowing systems to move beyond simple root-cause prediction and toward a more comprehensive understanding of how agent failures develop over time.

Comments (0)

No comments yet

Be the first to share your thoughts!