DualG-MRAG: Decoupling Macro-Reasoning and Micro-Ma...

DualG-MRAG: Decoupling Macro-Reasoning and Micro-Ma... | AI Research

Key Takeaways

DualG-MRAG is a framework designed to improve how Multimodal Large Language Models (MLLMs) perform complex, multi-hop reasoning tasks.
While Multimodal Retrieval-Augmented Generation (MM-RAG) has shown promising results, it still struggles with complex multi-hop reasoning tasks.
Existing methods primarily focus on independent instance-level matching, which often fails to capture explicit relationships across modalities and documents.
To address this dilemma, we propose DualG-MRAG, a Dual-tier framework that introduces a decoupled architecture comprising Macro-reasoning and Micro-matching Graphs for Multimodal RAG.
Subsequently, to enable dynamic relevance propagation across heterogeneous evidence sources, we formulate retrieval as a query-driven message passing process via a GNN Retriever.

Paper AbstractExpand

While Multimodal Retrieval-Augmented Generation (MM-RAG) has shown promising results, it still struggles with complex multi-hop reasoning tasks. Existing methods primarily focus on independent instance-level matching, which often fails to capture explicit relationships across modalities and documents. Although Graph-enhanced methods introduce structural modeling, they face a fundamental challenge in multimodal scenarios: incorporating fine-grained visual features leads to rapid graph expansion and retrieval noise, whereas coarse-grained representations cause the discarding of critical local evidence. To address this dilemma, we propose DualG-MRAG, a Dual-tier framework that introduces a decoupled architecture comprising Macro-reasoning and Micro-matching Graphs for Multimodal RAG. Specifically, to suppress retrieval noise by isolating global structural reasoning from fine-grained evidence matching, we construct a Macro Graph for global topological routing and a Micro Graph for precise local verification. Subsequently, to enable dynamic relevance propagation across heterogeneous evidence sources, we formulate retrieval as a query-driven message passing process via a GNN Retriever. Furthermore, to provide the generative model with coherent structural guidance, we introduce a dynamic programming decoding mechanism that extracts explicit reasoning paths directly from the GNN's forward pass, replacing the standard input of isolated document chunks. Extensive experiments demonstrate that DualG-MRAG outperforms baselines in both evidence recall and complex QA accuracy.

DualG-MRAG is a framework designed to improve how Multimodal Large Language Models (MLLMs) perform complex, multi-hop reasoning tasks. By decoupling global reasoning from local evidence verification, the system aims to reduce retrieval noise and provide clearer structural guidance to generative models, helping them avoid hallucinations when processing heterogeneous data like text, images, and tables.

The Structural Dilemma

Existing multimodal retrieval systems often struggle to balance two competing needs. If they incorporate fine-grained visual features, the graph structure expands rapidly, leading to significant retrieval noise. If they rely only on coarse-grained representations, they lose the specific local evidence required for accurate reasoning. DualG-MRAG addresses this by splitting the retrieval process into two distinct tiers: a Macro Graph for global topological routing and a Micro Graph for precise local verification.

How the Framework Works

The system operates through three primary phases:

Dual-Tier Graph Construction: The Macro Graph acts as a high-level structural backbone, using factual triples and equivalence edges to connect global entities. The Micro Graph captures fine-grained dependencies within images and tables, using 4-tuple facts that link directly to source documents.
Query-Driven Retrieval: Instead of using static search, the framework uses a Graph Neural Network (GNN) that performs "message passing." The GNN initializes its state based on the user's specific query, ensuring that information flow is constrained to relevant nodes. This prevents the system from blindly searching irrelevant parts of the graph.
Explicit Path Decoding: To assist the generative model, the system uses a dynamic programming mechanism to extract clear reasoning paths from the GNN’s forward pass. These paths are fed to the MLLM, providing a structured guide that replaces the standard practice of feeding the model a flat, unorganized list of document chunks.

Performance and Results

According to the authors, Jiacheng Tao, Qingyun Sun, Haonan Yuan, Ziwei Zhang, and Jianxin Li, this architecture improves both document recall and the accuracy of complex question answering compared to existing baselines. By transforming fragmented retrieved data into transparent, verifiable pathways, the framework reduces the cognitive load on the MLLM, allowing it to perform multi-hop reasoning more effectively.

Franklin Analysis

The effectiveness of DualG-MRAG relies on its ability to isolate global reasoning from local matching. By using a query-driven GNN, the framework addresses the "static topology" limitation found in many graph-based retrievers, where the search process does not adapt to the user's intent. The evidence suggests that this decoupling is a viable strategy for managing the trade-off between graph scale and retrieval precision in multimodal environments. However, the authors note that the system still relies on an approximate heuristic matching pipeline for the Micro Graph because exact subgraph isomorphism is computationally expensive (NP-hard).