Back to AI Research

AI Research

Why Does CLAUDE.md Keep Growing? Catastrophic Remem... | AI Research

Key Takeaways

  • Agentic coding repositories often rely on README files like CLAUDE.md to guide AI agents.
  • This paper, "Why Does CLAUDE.md Keep Growing?
  • Agentic coding READMEs like this http URL grow without bound in real repositories, stopping only when the repository retires or someone rewrites the file wholesale.
  • We name the resulting divergence catastrophic remembering, the inverse of catastrophic forgetting around which continual learning is organized.
  • Finally, applying the same inversion to WildIFEval, we show that prompt comments can improve real-world agentic instruction-following by up to 23.1%.
Paper AbstractExpand

Agentic coding READMEs like this http URL grow without bound in real repositories, stopping only when the repository retires or someone rewrites the file wholesale. We trace this to imperfect recall: appending an instruction is always cheap, but once an instruction's rationale is gone, deleting it without risking a correctness regression costs O(2^|D|) in a prompt of |D| instructions. We name the resulting divergence catastrophic remembering, the inverse of catastrophic forgetting around which continual learning is organized. First, we characterize this phenomenon across 247,694 instruction lifetimes in 1,867 repositories: agentic prompts grow without bound, more than tripling over their lifetime (+226%), gaining +4.9 net instructions every commit; further, the older an instruction gets, the less likely it is to be deleted (log-hazard -0.032/commit). Then, we show that prompt comments can halt the growth: inverting IFEval yields verifiable worlds whose optimal prompts are known, and there comments encoding latent reasoning remove 99.3% of excess instructions (+211.3% to +1.4%). Finally, applying the same inversion to WildIFEval, we show that prompt comments can improve real-world agentic instruction-following by up to 23.1%. If English is the new code, why don't we have comments yet?

Agentic coding repositories often rely on README files like CLAUDE.md to guide AI agents. This paper, "Why Does CLAUDE.md Keep Growing? Catastrophic Remembering in Agentic Coding" by Kushal Chakrabarti of South Park Commons, investigates why these files grow without bound over time. The research identifies that this growth is driven by "catastrophic remembering," where maintainers keep unnecessary instructions because they lack the original rationale for why those instructions were added.

The Problem of Unbounded Growth

The study analyzed 247,694 instruction lifetimes across 1,867 GitHub repositories. It found that agentic prompts more than triple in size (+226%) over their lifespan, gaining an average of 4.9 net instructions per commit. While mass rewrites can temporarily shrink these files, the growth typically resumes immediately. The author concludes that this happens because deleting an instruction is risky; without knowing the original "latent reasoning" behind a rule, a maintainer cannot safely remove it without potentially causing a regression. Because the cost of adding an instruction is low but the cost of verifying its removal is high, the system defaults to keeping everything.

The Role of Imperfect Recall

The paper introduces the concept of "imperfect recall" to explain why deletion becomes less likely as an instruction ages. Unlike instruction staleness, which would suggest that older instructions are more likely to be deleted, the data shows that the deletion hazard actually decreases by 0.032 per commit as an instruction gets older. This suggests that the memory of why an instruction was created decays over time. When multiple authors edit a file, this decay accelerates, making it even harder for anyone to justify removing old, potentially redundant rules.

Using Comments to Halt Growth

To address this, the researcher proposes using "prompt comments"—text that is visible to human maintainers but invisible to the AI agent. By inverting the IFEval benchmark, the study created a controlled environment where the optimal prompt size was known. When maintainers included informative comments that encoded the latent reasoning (the failure, the hypothesis, and the outcome) behind each instruction, the prompts were able to settle near their optimal size. This approach removed 99.3% of excess instructions and improved real-world agentic instruction-following by up to 23.1%.

Franklin Analysis

The evidence suggests that the primary barrier to efficient prompt maintenance is the loss of context during the development loop. The paper provides a concrete solution by separating the "how" (the instruction for the agent) from the "why" (the comment for the human). The findings indicate that without this explicit documentation, agentic prompts will inevitably grow until they become non-functional or are completely rewritten. This research suggests that treating English as a form of code—complete with documentation—is essential for the long-term stability of agentic systems.

Comments (0)

No comments yet

Be the first to share your thoughts!