Procedural Graphs: Self-Evolving Execution Structur... | AI Research

Procedural Graphs: Self-Evolving Execution Structures for LLM Agents
Large language models (LLMs) are increasingly used as autonomous agents to plan and execute complex tasks using external tools. However, these agents often struggle with long-horizon planning, frequently losing track of objectives, using tools in the wrong order, or getting stuck in repetitive loops. This paper introduces the Procedural Graph, a framework that provides agents with a structured, editable map of "what to do" next, allowing them to remain on track while maintaining the flexibility to reason through unique situations. The same large language models question is explored in Trace2Tower, which adds a research perspective.

From Knowledge Graphs to Procedural Graphs

The core innovation is the Procedural Graph, which organizes procedural knowledge into triplets—(procedure, relation, procedure)—similar to how knowledge graphs organize facts. While a knowledge graph answers "what is" questions, the Procedural Graph answers "what to do" questions. Nodes in the graph represent specific actions, reasoning steps, or task states, while edges define permissible transitions. Each edge is annotated with attributes that provide the agent with situational guidance, such as specific conditions for a task, recommended steps, and potential pitfalls to avoid.

Online Guidance and Offline Evolution

The framework operates in two distinct phases. During online inference, the agent identifies its current position in the graph and retrieves a "neighborhood" of nearby procedural steps. A guidance model then translates this structural context into actionable advice, which helps the agent choose its next move without dictating it entirely. The same ai evaluation question is explored in DRACO, which adds a research perspective.
During offline evolution, the system automatically improves itself. After completing a batch of tasks, an LLM refiner compares successful trajectories with failed ones. It then proposes edits to the graph—such as adding missing steps, removing ineffective ones, or updating guidance attributes. These changes are only adopted if they improve performance on a separate validation set, ensuring the graph evolves to be more effective over time. Rejected proposals are stored in a "rejection memory" to prevent the system from repeating the same mistakes.

Performance and Flexibility

Across a variety of benchmarks and LLM models, the Procedural Graph consistently outperforms existing memory-based methods. By separating procedural knowledge from the model's internal weights, the framework allows for a system that is both transparent and easily updated without the need for full retraining. The self-evolution loop enables the graph to start from a minimal skeleton and eventually match or exceed the performance of hand-designed workflows, effectively repairing flawed expert priors and adapting to new tasks without manual engineering. The same ai evaluation question is explored in WikiSkill, which adds a research perspective. as detailed in the full paper on Arxiv