Back to AI Research

AI Research

GraphMemix: Query-Aware Evidence Forests for Long-T... | AI Research

Key Takeaways

  • GraphMemix is a framework designed to improve how multimodal AI agents retrieve and organize long-term memory.
  • To address these issues, we propose GraphMemix, a combinatorial-optimization graph memory framework that models memory organization as query-aware evidence-forest construction.
  • By organizing memory into a query-relevant subgraph, the method avoids substantial lifecycle cost and recovers low-similarity complementary evidence.
  • Experimental results across four long-term multimodal memory benchmarks demonstrate significant improvements with different foundation models and establish a new Pareto frontier between accuracy and lifecycle cost.
  • Instead of relying on pre-processed summaries or simple similarity matching—which can lead to redundant or incomplete informat...
Paper AbstractExpand

Organizing long-term memory for multimodal agents remains challenging because existing methods either suffer from expensive question-agnostic offline summaries or naive embedding similarity matching that introduces incomplete and redundant context. To address these issues, we propose GraphMemix, a combinatorial-optimization graph memory framework that models memory organization as query-aware evidence-forest construction. Specifically, our method consists of three key components:(1) candidate graph construction, which expands multi-view seed memories through schema and semantic relations to acquire query-aware original context; (2) evidence utility and activation costs, which decouples direct memory support from anchor-conditioned relation verification to suppress redundant or conflicting information; and (3) forest optimization, which jointly selects a forest-format memory context under a maximum evidence budget and its reliable relational structure. By organizing memory into a query-relevant subgraph, the method avoids substantial lifecycle cost and recovers low-similarity complementary evidence. Experimental results across four long-term multimodal memory benchmarks demonstrate significant improvements with different foundation models and establish a new Pareto frontier between accuracy and lifecycle cost.

GraphMemix is a framework designed to improve how multimodal AI agents retrieve and organize long-term memory. Instead of relying on pre-processed summaries or simple similarity matching—which can lead to redundant or incomplete information—the system constructs a "query-aware evidence forest" only after a user asks a question. This approach allows agents to select relevant memories and their logical relationships dynamically, balancing the need for accurate context with the computational costs of managing large archives.

Addressing Memory Retrieval Challenges

Current methods for long-term agent memory often struggle with two main issues. First, question-agnostic systems compress history into summaries before knowing what a user will ask, which can lead to high maintenance costs and the loss of specific details that might later become important. Second, standard retrieval methods based on embedding similarity often return duplicate records while missing complementary information that is only useful when combined with other data. GraphMemix addresses these by treating memory organization as a combinatorial optimization problem, building a subgraph that is specifically tailored to the current query.

How the Framework Works

The GraphMemix process consists of three primary stages:

  • Candidate Graph Construction: The system identifies "seed" memories using multi-view retrieval (e.g., images, captions, and OCR). It then expands these seeds by following schema and semantic relationships to capture local context, creating a bounded subgraph of potential evidence.

  • Evidence Utility and Activation Costs: A node verifier assesses the independent value of each memory. Simultaneously, an Evidence-Chain Verifier (ECV) examines the relationships between these memories to determine if they provide new, complementary information or if they are redundant or conflicting.

  • Forest Optimization: The system uses a two-stage solver to select the optimal set of memories and their connections. By applying Kruskal’s algorithm, it constructs a forest-format context that maximizes evidence utility while staying within a defined budget.

Performance and Results

Researchers evaluated GraphMemix across four long-term multimodal memory benchmarks: ATM-Bench, Mem-Gallery, MemEye, and H2HMem. Using the Qwen3-VL-8B-Instruct model, the framework achieved a macro-average Judge Accuracy of 61.55%, outperforming the strongest public baseline of 49.80%. The results indicate that the method establishes a new Pareto frontier, meaning it provides higher accuracy than existing approaches without the overhead of history-wide generative preprocessing.

Considerations for Implementation

The framework relies on a deterministic two-stage solver to manage the complexity of selecting evidence from a large candidate pool. While this approach improves the completeness of the retrieved context, it is constrained by the initial candidate graph size and the reliability of the verifiers. The authors note that the system is designed to avoid the "cold-start" and update costs associated with traditional memory systems by focusing on query-local organization rather than constant, global maintenance.

Comments (0)

No comments yet

Be the first to share your thoughts!