Back to AI Research

AI Research

Shared Selective Persistent Memory for Agentic LLM... | AI Research

Key Takeaways

  • Shared Selective Persistent Memory for Agentic LLM Systems Agentic LLM systems—AI that can autonomously use tools and write code—often struggle with a "blank...
  • Naively persisting entire conversation histories is token-inefficient and counterproductive: irrelevant context degrades generation quality.
  • Crucially, this memory is shared: workspaces encapsulating selective memory can be transferred across users with role-based access control, enabling collaborative reuse without redundant specification.
  • A complementary zero-token data refresh mechanism decouples generated programs from runtime data, enabling artifact reuse without re-invocation.
  • Across three enterprise scenarios, shared selective persistent memory achieves 96% task completion (vs.
Paper AbstractExpand

Agentic LLM systems that generate code through multi-turn tool use face a fundamental context problem: each session starts from zero, discarding the configuration choices, domain constraints, data schemas, and tool-use patterns that made previous sessions productive. Naively persisting entire conversation histories is token-inefficient and counterproductive: irrelevant context degrades generation quality. We introduce shared selective persistent memory, an architecture that identifies and retains four categories of reusable context (task specifications, data schemas, tool configurations, and output constraints) while discarding session-specific reasoning traces. Crucially, this memory is shared: workspaces encapsulating selective memory can be transferred across users with role-based access control, enabling collaborative reuse without redundant specification. We implement it in a deployed collaborative workspace platform where LLM agents produce, edit, and maintain git-versioned artifacts (dashboards, reports, and data-driven documents) from heterogeneous sources (CSV, SQL, REST APIs, and MCP servers). A complementary zero-token data refresh mechanism decouples generated programs from runtime data, enabling artifact reuse without re-invocation. Across three enterprise scenarios, shared selective persistent memory achieves 96% task completion (vs. 79% without memory and 71% with full history). Zero-token refresh eliminates LLM re-invocation for recurring updates (14x task-time reduction), while summary-driven generation cuts per-invocation token cost by 97x versus raw data injection. A replication on four public datasets confirms generalizability, with zero-token refresh succeeding in 12/12 trials. Notably, naive full-history persistence actively degrades completion by biasing the agent with stale traces, while selective memory outperforms both extremes.

Shared Selective Persistent Memory for Agentic LLM Systems
Agentic LLM systems—AI that can autonomously use tools and write code—often struggle with a "blank slate" problem. Every time a user starts a new session, the AI forgets the domain rules, data structures, and formatting preferences that made previous sessions successful. While some systems try to solve this by saving the entire conversation history, this approach is inefficient and often harmful, as irrelevant past reasoning can confuse the AI. This paper introduces "shared selective persistent memory," an architecture that identifies and saves only the essential, reusable parts of a session while discarding the clutter.

The Four Pillars of Memory

Instead of saving raw conversation logs, the system decomposes a session into four specific, reusable categories:

  • Task Specifications: Custom rules, such as specific color-coding or formatting requirements.

  • Data Schemas: Statistical summaries of data sources, such as column types and relationships, which allow the AI to understand data without needing to read the entire raw file.

  • Tool Configurations: The settings, authentication details, and parameter requirements for external tools and data connectors.

  • Output Constraints: Rules that ensure the AI’s generated code interacts correctly with the environment, specifically requiring that data be injected at runtime rather than hardcoded.
    By saving these categories, the system provides the AI with a stable foundation for future tasks while explicitly "forgetting" session-specific reasoning traces, error-recovery attempts, and temporary file logs that would otherwise bias the agent toward outdated or incorrect solutions.

Collaborative Workspaces and Zero-Token Refresh

The architecture is implemented in a platform where AI agents manage git-versioned artifacts like dashboards and reports. Because the memory is structured, it can be shared across different users with role-based access control. This allows teams to reuse successful configurations without having to re-specify them.
A key feature is the "zero-token data refresh" mechanism. Because the system enforces a strict contract where generated programs consume data from a runtime injection point, the AI does not need to be re-invoked when the underlying data changes. The artifact simply updates itself using the new data, which significantly reduces both costs and task-completion time.

Performance and Results

The researchers tested this architecture across three enterprise scenarios and four public datasets. The results showed that shared selective persistent memory significantly outperformed both "no memory" and "full history" approaches:

  • Higher Completion: The system achieved a 96% task completion rate, compared to 79% without memory and 71% with full history.

  • Efficiency: The zero-token refresh mechanism eliminated the need for LLM re-invocation for recurring updates, leading to a 14x reduction in task time.

  • Cost Savings: Summary-driven generation reduced per-invocation token costs by 97x compared to injecting raw data.
    The study also confirmed that "full history" persistence actually degraded performance, as the AI became biased by stale reasoning traces from previous sessions. In contrast, the selective memory approach provided the necessary context to improve accuracy without the performance-draining side effects of long, irrelevant conversation logs.

Comments (0)

No comments yet

Be the first to share your thoughts!