Back to AI Research

AI Research

WikiSkill: Compiling Agent Experience into Persiste... | AI Research

Key Takeaways

  • WikiSkill is a framework designed to improve how AI agents develop and refine specialized skills.
  • Agent skills package specialized knowledge and workflows into reusable resources that extend AI agent capabilities.
  • Recent work automatically discovers such skills from agent experience, which enables agents to progressively adapt through interaction.
  • However, the insights that guide skill development typically remain scattered across optimization histories, limiting their systematic reuse across iterations.
  • We introduce WikiSkill, a framework that co-evolves agent skills with a persistent knowledge base (wiki).
Paper AbstractExpand

Agent skills package specialized knowledge and workflows into reusable resources that extend AI agent capabilities. Recent work automatically discovers such skills from agent experience, which enables agents to progressively adapt through interaction. However, the insights that guide skill development typically remain scattered across optimization histories, limiting their systematic reuse across iterations. We introduce WikiSkill, a framework that co-evolves agent skills with a persistent knowledge base (wiki). At a high level, WikiSkill separates raw execution experience, accumulated knowledge, and executable skills, while continuously consolidating experience into the wiki, which subsequent skill updates can build on. Across diverse benchmarks and models, WikiSkill consistently outperforms state-of-the-art skill-evolution methods and improves over no-skill baselines in most model-benchmark settings. We find that skill evolution complements model scaling: larger models generally benefit more from evolved skills, while smaller models with skills can outperform substantially larger models without them. We also find that evolved skills transfer effectively across models and model families, and skills evolved by other models can outperform self-evolved skills. Finally, our ablation studies confirm that persistent knowledge accumulation in the wiki is critical for effective skill evolution. These results demonstrate the benefits of systematically accumulating and refining agent experience for developing reusable and transferable skills.

WikiSkill is a framework designed to improve how AI agents develop and refine specialized skills. While existing methods allow agents to learn from experience, they often leave insights scattered across history, making it difficult to build on past lessons. WikiSkill addresses this by introducing a persistent knowledge base—a "wiki"—that compiles raw execution experience into structured, reusable knowledge that supports long-term skill evolution. Methods and results are detailed in the full paper on arxiv.org.

The Three-Layer Architecture

WikiSkill organizes an agent’s workspace into three distinct layers to separate raw data from executable procedures:

  • Raw Layer: Stores immutable execution traces, including step-by-step reasoning, tool calls, and final answers from training tasks.

  • Wiki Layer: Acts as a persistent knowledge base that compounds over time. It maintains a catalog of patterns, failure modes, and successful strategies. This layer is never reset, allowing the agent to learn from past successes and avoid repeating failed interventions.

  • Skills Layer: Contains the active, modular skills that the agent uses to perform tasks. Each skill is linked to the specific wiki patterns that inspired its creation. The same AI Evaluation question is explored in CAFE, which adds a research perspective.

The Evolutionary Loop

The framework operates through an iterative loop involving four components: 1. Inference Agent: Executes tasks using the current skill set. 2. Wiki Maintainer: Analyzes execution traces to update the wiki with new patterns and logs. 3. Skill Proposer: Reviews the updated wiki and execution history to create or modify skills. 4. Gating and Rollback: Tests new skills on a validation set. If performance improves, the skills are accepted; if not, the system reverts to the previous configuration. The wiki, however, retains all accumulated knowledge regardless of the skill update's success. The same AI Evaluation question is explored in Recursive Experiential-Working Memory Evolution for Long-Horizon..., which adds a research perspective.

Performance and Scaling

Researchers evaluated WikiSkill across five benchmarks, including mathematical reasoning, web search, spreadsheet manipulation, document question answering, and interactive embodied tasks. The framework consistently outperformed existing skill-evolution methods and no-skill baselines. The study found that skill evolution complements model scaling. Larger models generally saw greater performance gains from evolved skills, but smaller models equipped with WikiSkill were able to outperform significantly larger models that lacked these skills. For example, a Qwen-3.5-9B model using WikiSkill outperformed a Qwen-3.6-27B model without skills. Additionally, the researchers observed that skills evolved by one model can be transferred to others, sometimes performing better than skills the model evolved for itself. The same AI Evaluation question is explored in GraphMemix, which adds a research perspective.

Key Takeaways The evidence suggests that persistent knowledge accumulation is critical for effective skill development. Ablation studies confirmed that the wiki layer is essential to the framework's success, as it provides a structured history that prevents the loss of insights across iterations. By separating the accumulation of knowledge from the execution of tasks, WikiSkill provides a systematic way for agents to refine their capabilities over time.

More from Franklin

CAFEresearchRecursive Experiential-Working Memory Evolution for Long-Horizon...researchGraphMemixresearchThis INSANE AI Agent Will AUTOMATE...videoStanford Researchers Develop TRACE to Fix...newsThis AI Agent Researches and Creates...video

Comments