World Model Science: Self-Organized Criticality, We... | AI Research

World Model Science: Self-Organized Criticality, Weak Chaos, and Metastable Belief Dynamics in Long-Horizon LLM Agents explores why long-horizon AI agents often fail in ways that are not immediately obvious. Instead of looking only at whether an agent eventually succeeds or fails, the researchers treat the agent’s internal "world model"—its understanding of progress, beliefs, and constraints—as a dynamic system. By applying concepts from physics, such as self-organized criticality (the study of how small changes can trigger large, system-wide collapses), the authors aim to create a more rigorous way to diagnose why and how agents lose track of their tasks over time.

Measuring the "Hidden" State

Traditional evaluations often rely on whether an agent’s final answer is correct or if its individual tool calls are syntactically valid. The authors argue this is insufficient because an agent can perform a series of locally valid actions while its internal understanding of the global task state has already diverged. To address this, the researchers developed a framework that maps agent logs to a "world state" vector. This vector tracks variables like progress, uncertainty, risk, and plan intention. By comparing these agent-implied states against a "gold standard" extracted from the environment, the researchers can measure the "local-global gap"—the difference between an action that looks correct in the moment and the actual state of the task. The same ai agents question is explored in MeClear, which adds a research perspective.

Stress and Avalanches

The study investigates whether agent errors occur randomly or if they follow patterns similar to physical systems like sandpiles or neural avalanches. The authors define "stress" as the accumulation of unresolved uncertainties, contradictions, and tool-use errors. Their experiments show that as this stress builds up, it can trigger "avalanches" of errors—sudden, large-scale collapses in the agent's performance. These collapses are not just random noise; they are statistically correlated, meaning that early errors often set the stage for later, more significant failures.

Key Findings

Across 22 experiments—ranging from puzzles and tool-use benchmarks to embodied navigation and the Game of Life—the researchers found several consistent patterns:

• Silent Collapse: Agents frequently continue to produce locally valid actions even after their global task state has failed.

• Memory in Errors: Error sequences are not independent; they exhibit "long memory," where past mistakes influence future behavior.

• Structural Dependence: The way errors propagate depends on the complexity of the task and the depth of the dependencies involved.

• Stability: While these systems show signs of stress-sensitive collapse, they do not exhibit fully chaotic behavior. Instead, they remain in a "metastable" state where divergence is bounded. The ai agents story also surfaces in OpenAI Unveils GPT-Red an Automated Model..., adding another angle.

Important Limitations

While the study provides a new diagnostic lens for agent reliability, the authors note that their findings do not support universal claims. They explicitly state that they did not find evidence for universal power laws, single critical points, or a shared "optimal" way to intervene across all tasks. Furthermore, the framework is designed to analyze trajectories rather than predict terminal rewards, meaning it serves as a tool for understanding the "science" of how agents work rather than a simple leaderboard for performance. The researchers emphasize that their results are specific to the finite horizons and structures of the tasks tested. The ai agents story also surfaces in Stanford Researchers Develop TRACE to Fix..., adding another angle. as detailed in the full paper on Arxiv