Back to AI Research

AI Research

MAFIA: Query-Only Memory Attacks via Probing and Fa... | AI Research

Key Takeaways

  • MAFIA is a framework designed to test the security of memory-augmented LLM agents.
  • However, existing query-only attacks often fail to remain effective in two realistic and prevalent settings: large-scale benign memory pools and active input auditing.
  • Consequently, current approaches fall short when facing the dual challenges of high retrieval competitiveness and rigorous semantic checks.
  • To overcome these limitations, we propose MAFIA, a query-only Memory Attack framework via probing and Factual Injection against Audit, tailored to this extended threat model.
  • Extensive evaluations reveal that MAFIA achieves up to a 90.7% attack success rate while suppressing audit detection from a peak of 83.3% to at most 7.4%, exposing critical vulnerabilities across agentic memory systems.
Paper AbstractExpand

Memory-augmented LLM agents rely on rich context for long-horizon reasoning and acting, yet their memory modules expose a persistent attack surface for malicious records, making the study of memory poisoning threats imperative. However, existing query-only attacks often fail to remain effective in two realistic and prevalent settings: large-scale benign memory pools and active input auditing. Consequently, current approaches fall short when facing the dual challenges of high retrieval competitiveness and rigorous semantic checks. To overcome these limitations, we propose MAFIA, a query-only Memory Attack framework via probing and Factual Injection against Audit, tailored to this extended threat model. Specifically, MAFIA introduces: (1) a placement strategy that ensures retrieval-competitive injection via memory probing, budget allocation, and scheduling; and (2) a payload design that bypasses audits using compact factual cloaks, preserving malicious effects while maintaining high semantic similarity. Extensive evaluations reveal that MAFIA achieves up to a 90.7% attack success rate while suppressing audit detection from a peak of 83.3% to at most 7.4%, exposing critical vulnerabilities across agentic memory systems. Code will be made publicly available at this https URL .

MAFIA is a framework designed to test the security of memory-augmented LLM agents. It focuses on "query-only" memory poisoning, where an attacker influences an agent to store malicious information by interacting with it through standard user prompts, rather than having direct access to the agent's underlying database. The researchers, Jiaming Chen, Yisen Gao, Yanping Li, Zifan Liu, Yumeng Zhang, and Jun Zhang from The Hong Kong University of Science and Technology, developed this framework to address two common security hurdles: active input auditing and the difficulty of getting malicious data to persist in large, competitive memory pools.

Addressing Practical Security Constraints

Modern LLM agents often use an input auditor—a safety classifier that blocks prompts containing adversarial or malicious instructions. Additionally, these agents store vast amounts of benign data. For a poisoning attack to be effective, it must bypass the auditor and ensure the malicious entry is retrieved by the agent when a user later asks about the victim entity. Previous methods often relied on explicit, aggressive instructions that were easily flagged by auditors and failed to compete with the sheer volume of benign records in the agent's memory.

How MAFIA Works

The MAFIA framework uses a two-stage process to overcome these challenges:

  • Placement via Retrieval Probing: Since the attacker cannot see the agent's internal memory, they use "probe" queries to map the agent's retrieval behavior. By analyzing the question fields surfaced in the agent's responses, the attacker identifies which semantic regions are most frequently retrieved. They then use a size-ranked round-robin strategy to allocate their limited poisoning budget to these high-impact areas, ensuring the malicious records are positioned where they are most likely to be used.

  • Payload via Factual Cloaks: To bypass input auditors, MAFIA avoids using imperative commands. Instead, it uses "compact factual cloaks"—short, declarative annotations formatted as supplementary notes (e.g., "patient V → patient T in this DB"). Because these cloaks are framed as database facts rather than behavioral instructions, they are less likely to trigger safety filters while still successfully redirecting the agent's reasoning.

Performance and Results

In evaluations across four different agent-dataset configurations—including medical record lookups (MIMIC-III and eICU), product shopping (WebShop), and software engineering (HuggingFace Hub)—MAFIA demonstrated significant effectiveness. The framework achieved an attack success rate of up to 90.7%. Furthermore, it proved highly effective at evading detection; while existing methods saw their audit detection rates peak at 83.3%, MAFIA kept its detection rate to a maximum of 7.4%.

Franklin Analysis

The evidence provided by the researchers indicates that the primary vulnerability in current memory-augmented agents is the reliance on semantic similarity for retrieval. Because the agent treats retrieved records as factual evidence, it is susceptible to "factual" injections that mimic the structure of benign data. The success of MAFIA suggests that standard input auditing is insufficient when attackers use subtle, context-aware payloads that blend into the existing memory distribution. The researchers note that this study focuses on RAG-based memory systems, as research into long-term memory (LTM) systems is still in its early stages.

Comments (0)

No comments yet

Be the first to share your thoughts!