Back to AI Research

AI Research

DDRG checks disagreements inside reasoning traces before repairing an answer

Key Takeaways

  • Graph alignment targets conflicting intermediate claims while retaining the original sampled reasoning bank.
  • ## A majority answer can hide an unresolved conflict
  • The [DDRG paper](https://arxiv.org/abs/2610.18606) studies what happens between sampling several reasoning traces and choosing a final answer.
  • Self-consistency voting can identify the most frequent answer while leaving disagreements in the intermediate reasoning unresolved.
  • DDRG, short for Disagreement-Driven Verification Guided by Reasoning Graphs, builds source-grounded claim graphs from a frozen bank of traces.

A majority answer can hide an unresolved conflict

The DDRG paper studies what happens between sampling several reasoning traces and choosing a final answer. Self-consistency voting can identify the most frequent answer while leaving disagreements in the intermediate reasoning unresolved.
DDRG, short for Disagreement-Driven Verification Guided by Reasoning Graphs, builds source-grounded claim graphs from a frozen bank of traces. It aligns corresponding claims and sends localized conflicts to an answer-blind verifier. The design uses evidence already present in the sample bank rather than expanding that bank with additional candidate traces.

Repair only when verification reaches the answer

A verifier-supported refutation must reach the supporting path of the majority answer before DDRG triggers a focused repair. Otherwise the framework returns the exact self-consistency vote. This condition limits how often a local disagreement changes the final result.
Across five benchmarks, the authors report macro-average accuracy improvements over direct answering of 8.65 percentage points for Qwen2.5-7B and 15.83 points for Llama3.1-8B. DDRG also exceeds the evaluated chain-of-thought, graph-based adaptations and self-consistency baselines on that macro metric.
The comparison is to the tested baselines and configurations. Reusing the initial candidate bank does not mean verification and repair have no computational cost, nor does agreement between traces establish that their shared premise is correct.

Keep the verification trail visible

Franklin analysis: evaluate a repair system by saving the original vote, conflicting claims, verifier evidence and revised answer together. Inspect cases where a correct majority answer changes, not just cases where an incorrect one improves.
A useful comparison holds the initial sample bank fixed and measures both accuracy and total inference cost. That reveals whether selective checking earns its added work and whether its trigger is narrow enough to preserve good answers when the evidence for correction is weak.

Comments