AI Research

Navigating Unreliable Parametric and Contextual Kno... | AI Research

Key Takeaways

Navigating Unreliable Parametric and Contextual Knowledge: Explicit Knowledge Conflict Resolution for LLM Inference Large language models (LLMs) often strugg...
However, the integration of external knowledge can introduce conflicts, not only between the model's internal parametric knowledge and the external information, but also among multiple pieces of external contexts.
Specifically, we first propose an adaptive knowledge assessment and retrieval approach that employs a modified semantic entropy measure to quantify an LLM's confidence in its answer to a given query.
Empirical results demonstrate that MACR significantly outperforms state-of-the-art baselines across benchmarks, while also providing interpretable resolutions of explicit conflicts.
Navigating Unreliable Parametric and Contextual Knowledge: Explicit Knowledge Conflict Resolution for LLM Inference

Paper AbstractExpand

Large language models (LLMs) have achieved strong performance across a wide range of language-based tasks by leveraging both extensive parametric knowledge and in-context learning ability, enabling them to incorporate external information provided in the input prompt. However, the integration of external knowledge can introduce conflicts, not only between the model's internal parametric knowledge and the external information, but also among multiple pieces of external contexts. Existing approaches typically assume that either the model or the provided context is reliable, overlooking the possibility that both sources may contain errors, and avoid conflicts by privileging one source over the other, rather than actively resolving inconsistencies. To address these limitations, we propose a novel framework MACR for LLM knowledge conflict resolution that moves beyond the conventional binary choice paradigm and incorporates an explicit conflict-resolution mechanism based on a multi-agent reasoning approach. Specifically, we first propose an adaptive knowledge assessment and retrieval approach that employs a modified semantic entropy measure to quantify an LLM's confidence in its answer to a given query. Based on this confidence estimation, MACR either externalizes the model's internal knowledge as textual representations or retrieves relevant external knowledge when internal knowledge is insufficient, generating basic contexts for subsequent reasoning. Then we introduce an inductive multi-agent reasoning framework with three specialized agents that, respectively, induce explicit rules, analyze potential conflicts, and resolve inconsistencies across all available contexts. Empirical results demonstrate that MACR significantly outperforms state-of-the-art baselines across benchmarks, while also providing interpretable resolutions of explicit conflicts.

Navigating Unreliable Parametric and Contextual Knowledge: Explicit Knowledge Conflict Resolution for LLM Inference
Large language models (LLMs) often struggle when the information they were trained on (parametric knowledge) contradicts the information provided in a prompt (contextual knowledge), or when different pieces of external information conflict with one another. Existing systems typically try to solve this by simply choosing one source over the other—assuming either the model or the external data is correct. This paper introduces a framework called MACR, which moves beyond this "either/or" approach by using a multi-agent system to actively analyze and resolve these conflicts, rather than just picking a side.

Assessing Model Confidence

The first step of the MACR framework is to determine how much the LLM actually knows about a specific query. The researchers use a modified "semantic entropy" measure to quantify the model's confidence. By generating multiple answers to a question and checking how consistent and relevant they are, the system can detect if the model is hallucinating or uncertain. To make this assessment more accurate, the framework also injects temporal information (such as the current date) and subject disambiguation into the prompt, helping the model account for outdated training data or ambiguous terms.

The Branching Strategy

Once the model’s confidence is measured, the framework decides how to proceed. If the model is highly confident, it is prompted to "externalize" its internal knowledge, turning its latent beliefs into a structured, written format. If the model has low confidence, the system triggers an external retrieval process to gather reliable information. This ensures that the reasoning process is always based on a clear, explicit set of facts, whether they come from the model’s own memory or an external source.

Multi-Agent Conflict Resolution

The core of the framework is an inductive multi-agent reasoning process involving three specialized agents:

The Observer: Analyzes training data to extract general, reusable rules for resolving knowledge conflicts.
The Analyzer: Breaks down specific discrepancies found between the model’s internal knowledge and the retrieved external context.
The Reasoner: Applies the rules learned by the Observer to the findings of the Analyzer to synthesize a final, conflict-free answer.
This collaborative process allows the system to provide an interpretable explanation for how it resolved the conflict, rather than just outputting a final answer.

Performance and Reliability

Empirical results show that MACR significantly outperforms existing methods across various benchmarks. By moving away from the simplistic binary choice of trusting either the model or the external context, the framework is better equipped to handle realistic scenarios where both sources might be flawed. This approach not only improves the factual accuracy of the model's responses but also provides a transparent, logical path for how those conclusions were reached.

Comments (0)

No comments yet

Be the first to share your thoughts!