AI Research

MetaSkill-Evolve: Recursive Self-Improvement of LLM... | AI Research

Key Takeaways

MetaSkill-Evolve: Recursive Self-Improvement of LLM Agents via Two-Timescale Meta-Skill Evolution Recent LLM agents often use "skills"—reusable procedural fi...
Recent LLM agents tackle increasingly long-horizon, open-ended tasks, and external skills, reusable procedural knowledge supplied to the agent, further extend this capability.
However, a fixed, hand-authored skill is rarely optimal, and cannot adapt to the diversity of tasks an agent encounters.
Self-improving agents address this by rewriting their own skill files from execution traces, yielding meaningful gains on challenging benchmarks.
Yet such self-evolution remains non-recursive: it improves only the task skill (what the agent does) while the improvement procedure (how it improves) is authored once and held fixed.

Paper AbstractExpand

Recent LLM agents tackle increasingly long-horizon, open-ended tasks, and external skills, reusable procedural knowledge supplied to the agent, further extend this capability. However, a fixed, hand-authored skill is rarely optimal, and cannot adapt to the diversity of tasks an agent encounters. Self-improving agents address this by rewriting their own skill files from execution traces, yielding meaningful gains on challenging benchmarks. Yet such self-evolution remains non-recursive: it improves only the task skill (what the agent does) while the improvement procedure (how it improves) is authored once and held fixed. We introduce MetaSkill-Evolve, a two-timescale framework that makes agentic skill improvement recursive: every branch carries both a task skill $s$ and a branch-local meta-skill $m=(\psi,\sigma,\alpha,\pi,\varepsilon)$ whose five components parameterise the Analyzer, Retriever, Allocator, Proposer, and Evolver agents of the improvement pipeline. Task skills evolve on a fast loop while the meta-skill evolves on a slower one under the same pipeline applied to itself, with no additional model or objective. With all five pipeline agents sharing a single frozen backbone, MetaSkill-Evolve outperforms no-skill, static-skill, and single-level evolution baselines on three agentic benchmarks (OfficeQA, SealQA, ALFWorld), improving held-out test accuracy over the raw backbone by +23.54, +16.09, and +1.92 points respectively.

MetaSkill-Evolve: Recursive Self-Improvement of LLM Agents via Two-Timescale Meta-Skill Evolution
Recent LLM agents often use "skills"—reusable procedural files—to handle complex, long-horizon tasks. While many systems allow agents to rewrite these skills based on past mistakes, the underlying "improvement procedure" (how the agent diagnoses errors and decides on edits) is usually hardcoded and fixed. MetaSkill-Evolve introduces a recursive framework that allows the agent to improve not just its task skills, but also the very process it uses to perform that improvement. By treating the improvement procedure as a learnable object, the system can adapt its diagnostic and editing strategies over time.

A Two-Timescale Approach

The framework operates on two distinct timescales. On the "fast" loop, the agent evolves its task skills to solve specific problems. On the "slow" loop, the agent evolves its "meta-skill"—a collection of five components (Analyzer, Retriever, Allocator, Proposer, and Evolver) that dictate how the fast loop functions. Because these meta-skills are stored in the same format as task skills, the system uses the same five-agent pipeline to refine them. This creates a bounded, recursive self-improvement loop that does not require additional models or complex new objectives.

Intelligent Search and Selection

To manage the evolution process, the system maintains a persistent graph of all attempted branches. It selects which branch to develop next using a scoring function that balances three factors: the current utility of the task skill, the "meta-productivity" of the branch (how effectively its current meta-skill generates improvements), and a novelty score that prevents the system from getting stuck in a single, repetitive lineage. This allows the agent to prioritize branches that are not only performing well but are also actively learning better ways to solve problems.

Performance Gains

The researchers evaluated MetaSkill-Evolve on three benchmarks: OfficeQA, SealQA, and ALFWorld. By using a single frozen model backbone, the team ensured that performance gains were due to the evolved skills and meta-skills rather than increased model capacity. The results showed consistent improvements over existing methods, including "no-skill," "static-skill," and "single-level evolution" baselines. Specifically, the system achieved significant accuracy increases on held-out test sets, demonstrating that allowing the improvement procedure to evolve alongside the task skill leads to more capable and adaptable agents.

Comments (0)

No comments yet

Be the first to share your thoughts!