Back to AI Research

AI Research

Computing Actual Causes for Neural Network Predicti... | AI Research

Key Takeaways

  • This paper introduces a method for identifying the "actual causes" of neural network predictions while accounting for the structured causal dependencies inhe...
  • Explaining the predictions of neural networks is a central challenge in trustworthy AI.
  • We address this by formalizing explanations as Halpern-Pearl (HP) actual causes, modeling input dependencies using Boolean Structural Causal Models (SCMs).
  • We compute HP causes by applying bound propagation and branch-and-bound techniques, while providing formal guarantees of completeness and minimality.
  • In a case study, we further show that ignoring input dependencies inflates the number of reported causes, 14.9% of which are spurious under our SCM.
Paper AbstractExpand

Explaining the predictions of neural networks is a central challenge in trustworthy AI. Existing explanation methods, such as those based on feature attribution or minimal sufficient sets, typically treat input features as independent, which can yield misleading explanations when inputs exhibit structured dependencies. We address this by formalizing explanations as Halpern-Pearl (HP) actual causes, modeling input dependencies using Boolean Structural Causal Models (SCMs). We compute HP causes by applying bound propagation and branch-and-bound techniques, while providing formal guarantees of completeness and minimality. Our experiments show that we substantially outperform brute-force and ILP baselines in scalability, and outperform heuristic search as graph size grows, computing all minimal actual causes on instances with search spaces of up to $2.3\times10^{13}$ candidate (cause, contingency) pairs, on SCMs with up to 28 nodes, within a 180s per-instance budget. In a case study, we further show that ignoring input dependencies inflates the number of reported causes, 14.9% of which are spurious under our SCM.

This paper introduces a method for identifying the "actual causes" of neural network predictions while accounting for the structured causal dependencies inherent in input data. By using Boolean Structural Causal Models (SCMs) to represent how features influence one another, the authors provide a way to generate explanations that respect real-world constraints, avoiding the misleading or spurious results that can occur when features are treated as independent.

The Problem with Current Explanations

Existing methods for explaining neural networks often rely on feature attribution or minimal sufficient sets, which typically assume that input features are independent. In practice, however, variables are often linked—for example, a person’s income level may influence their expenses, which in turn affects their cash flow. When these dependencies are ignored, counterfactual explanations may suggest changing a feature that is physically or logically impossible to alter independently. This can lead to "spurious" explanations that do not reflect the true causal structure of the domain.

How the Approach Works

The authors formalize explanations using the Halpern-Pearl (HP) definition of actual causality. To compute these causes, they developed an algorithm called CausEx_BaB. Instead of testing every possible combination of inputs—which is computationally expensive—the algorithm uses:

  • Bound Propagation: It propagates intervals through the SCM and the neural network to determine if a group of inputs will consistently lead to a specific prediction.

  • Branch-and-Bound: It systematically splits the search space into regions. If a region is proven to preserve the prediction, it is pruned; if it is proven to change the prediction, it is identified as a potential cause.

  • Multilinear Relaxations: By relaxing Boolean structural equations into multilinear functions, the algorithm can reason about entire regions of inputs simultaneously, rather than checking individual assignments one by one.

Key Findings and Performance

The researchers tested their method using synthetic benchmarks and a case study involving the U.S. Supplemental Nutrition Assistance Program (SNAP). Their results indicate:

  • Scalability: The algorithm outperforms brute-force and integer linear programming (ILP) baselines, successfully handling search spaces of up to $2.3 \times 10^{13}$ candidate cause-contingency pairs within a 180-second budget.

  • Accuracy: In the SNAP case study, ignoring input dependencies more than doubled the median number of reported causes, with 14.9% of those causes identified as spurious under the SCM.

  • Completeness: The method provides formal guarantees of completeness and minimality, ensuring that all minimal sets of actual causes are identified.

Considerations for Implementation

The authors note that their current approach focuses on Boolean SCMs, which are effective for representing categorical or thresholded properties. While this allows for precise algorithmic analysis, it assumes that a valid SCM is already available—either derived from domain knowledge or learned from observational data. Additionally, while the algorithm is designed to be complete, users can restrict the search to causes of a specific maximum size to further manage computational costs if necessary.

Comments (0)

No comments yet

Be the first to share your thoughts!