Back to AI Research

AI Research

MemTrapBench: Benchmarking Cognitive Traps in LLM M... | AI Research

Key Takeaways

  • MemTrapBench: Benchmarking Cognitive Traps in LLM Memory Use introduces a framework to evaluate how retrieved memories can unintentionally degrade an LLM's r...
  • Memory has become a key component of large language models, enabling them to retain information and learn from long-term interactions.
  • We identify memory-induced cognitive traps: even faithfully recorded and semantically relevant memories can distort model reasoning or beliefs and degrade current task performance.
  • To systematically evaluate these failure modes, we introduce MemTrapBench, which covers two forms of cognitive traps: Reasoning Fixation and Belief Distortion.
  • To mitigate these cognitive traps, we propose AdaptiveMem, a simple yet effective inference-time method that instructs LLMs to avoid memory traps.
Paper AbstractExpand

Memory has become a key component of large language models, enabling them to retain information and learn from long-term interactions. However, existing memory benchmarks mainly evaluate whether information is correctly extracted, stored, and retrieved, while largely overlooking how retrieved memories reshape model reasoning and affect performance on the current task. We identify memory-induced cognitive traps: even faithfully recorded and semantically relevant memories can distort model reasoning or beliefs and degrade current task performance. To systematically evaluate these failure modes, we introduce MemTrapBench, which covers two forms of cognitive traps: Reasoning Fixation and Belief Distortion. Experiments across two model families and five representative memory frameworks show that MemTrapBench is challenging: all evaluated memory strategies underperform the no-memory setting, with even the strongest methods suffering drops of more than 10%. To mitigate these cognitive traps, we propose AdaptiveMem, a simple yet effective inference-time method that instructs LLMs to avoid memory traps. AdaptiveMem mitigates cognitive traps on MemTrapBench while preserving or improving performance on standard memory benchmarks across diverse memory frameworks.

MemTrapBench: Benchmarking Cognitive Traps in LLM Memory Use introduces a framework to evaluate how retrieved memories can unintentionally degrade an LLM's reasoning. While current memory benchmarks focus on whether a model can successfully store and retrieve information, this research identifies "cognitive traps"—instances where relevant, accurate memories distort a model’s judgment or cause it to fixate on outdated reasoning patterns.

Identifying Memory-Induced Cognitive Traps

The authors define two primary categories of cognitive traps that occur when LLMs rely on past interactions:

  • Reasoning Fixation: This occurs when a model becomes anchored to a previously successful strategy, even when the current task requires a different approach. This includes "Task Boundary" issues, where rules from a past task persist into a new one, and "Cognitive Bias" or "Trauma," where prior feedback or successful patterns are overgeneralized to inappropriate new contexts.

  • Belief Distortion: This category tests whether a model will adopt counterfactual or clearly incorrect premises established in its history, overriding its own safety knowledge.
    The researchers note that these failures are not caused by the model's inability to retrieve information, but rather by the influence that retrieved information has on the model's subsequent decision-making process.

Evaluating Memory Frameworks

To test these failure modes, the authors developed MemTrapBench, a dataset of 1,050 instances across four scenarios. When testing five representative memory frameworks—including FullText, LightMem, and EverMemOS—on Gemini and Qwen models, the researchers found that all memory-enabled settings performed worse than a "no-memory" baseline.
Even the strongest memory strategies suffered performance drops of more than 10 percentage points. Controlled experiments confirmed that this degradation was caused by the semantic content of the "trap" memories rather than the length of the context provided to the model.

Mitigating Traps with AdaptiveMem

To address these vulnerabilities, the authors propose AdaptiveMem, an inference-time method that uses specific prompts to guide LLMs in evaluating retrieved memories before applying them.
According to the paper, AdaptiveMem can be integrated into existing memory frameworks without requiring architectural changes. In experiments, the method consistently improved performance on MemTrapBench while maintaining or enhancing the models' performance on standard memory benchmarks. For example, on the Gemini-3-Flash-Preview model, AdaptiveMem improved the LightMem framework by 14.9 percentage points on the benchmark.

Considerations for Implementation

The authors emphasize that MemTrapBench is designed as a diagnostic stress test for harmful memory influence rather than a general-purpose memory evaluation. A key limitation noted in the research is that the benchmark relies on specific, manually designed trap seeds to trigger these failures. While the results show that current memory frameworks are broadly susceptible to these cognitive traps, the authors clarify that their use of terms like "trauma" is a behavioral analogy for feedback-induced avoidance and does not imply that LLMs possess human-like psychological states or emotions.

Comments (0)

No comments yet

Be the first to share your thoughts!