Back to AI Research

AI Research

Training retrievers on their own mistakes improves multi-hop evidence recovery

Key Takeaways

  • On-policy state training exposes an embedding retriever to the imperfect histories it will actually create at test time.
  • ### The next search depends on the earlier search
  • A [multi-hop retrieval study](https://arxiv.org/abs/2610.40409) examines embedding models that gather evidence one passage at a time.
  • Each search combines the question with the passages already found.
  • If an early passage is wrong, that mistake remains part of the input used to find later evidence.

The next search depends on the earlier search

A multi-hop retrieval study examines embedding models that gather evidence one passage at a time. Each search combines the question with the passages already found. If an early passage is wrong, that mistake remains part of the input used to find later evidence.
The paper identifies a training mismatch. Teacher-forced training supplies correct histories, while deployment exposes the retriever to its own imperfect choices. The authors report that by the third step, roughly half or more of the visited states can contain a wrong passage in the settings they study.

Train on the histories the current model reaches

The proposed on-policy state training refreshes retrieval chains using the current encoder during every epoch. Training then mixes those visited states with teacher-forced states, rather than relying exclusively on ideal histories.
The authors label a state with the earliest missing gold passage and mask other missing gold passages so that correct continuations are not penalized. That detail addresses the fact that a multi-document question can have more than one useful next piece of evidence.

Evaluate recovery as well as clean-path accuracy

The paper reports more complete evidence retrieval than teacher-forced training and the compared systems, with transfer to harder long-chain variants of HotpotQA and 2WikiMultiHopQA. It also analyzes recovery from the same mistaken states. These are retrieval results on the evaluated protocols, not a guarantee that a downstream language model will always answer correctly.
For a retrieval pipeline, preserve the sequence of passages instead of inspecting only the final answer. Compare behavior after an incorrect early retrieval with behavior after a correct history, and check whether the required evidence is eventually complete. The practical idea is to align training with the errors the retriever actually makes, rather than assuming those errors disappear after the first hop.

Comments