SkillAA: Attribution-Guided Skill-Graph Updating wi... | AI Research

SkillAA: Attribution-Guided Skill-Graph Updating with Targeted Validation and Rollback introduces a new framework designed to improve how frozen language models use external skills. While many existing systems treat skills as flat text or simple lists, SkillAA organizes them into a structured, addressable graph. This allows the system to not only select and compose skills more effectively but also to pinpoint exactly where a failure occurs and apply a precise, localized fix without needing to update the underlying model's parameters.

A Structured Approach to Skill Management

The core of SkillAA is a unified graph that represents skills as nodes containing three distinct parts: when to use the skill, how to execute it, and when to avoid it. Edges between these nodes define how skills relate to one another, such as prerequisite dependencies or enhancement relations. By separating these components, the system can distinguish between different types of errors—such as a skill being misapplied, a step being missing, or a rule being incorrect—rather than treating every failure as a general lack of knowledge. The openai story also surfaces in OpenAI Unveils GPT-Red an Automated Model..., adding another angle.

Targeted Repair and Attribution

When a model fails to solve a task, SkillAA uses "abductive attribution" to identify the smallest possible change needed to fix the error. By comparing the failed execution against successful examples, the system routes the repair to a specific graph object—such as a single field in a node or a specific relational edge. This targeted approach ensures that updates are focused and intentional. If a failure is determined to be a one-off execution lapse rather than a flaw in the skill graph, the system correctly chooses to make no changes at all, preventing the accumulation of unnecessary or harmful edits.

Scoped Validation and Rollback

To ensure that improvements do not break existing capabilities, SkillAA employs a two-level validation process. First, a "Local Gate" tests proposed changes against a subset of tasks likely to be affected by the edit. If the change causes more errors than it fixes, the system automatically rolls back the update. Second, a "Big Gate" evaluates the entire set of proposed changes at the end of an epoch, committing them only if they result in a net improvement across the board. This mechanism allows the skill graph to evolve reliably over time. The openai story also surfaces in AI Agents Going Rogue Renew Calls..., adding another angle.

Performance and Results

SkillAA demonstrates significant improvements across several benchmarks, including SearchQA, LiveMath, and DocVQA. Using the gpt-5.6-sol model, the framework achieved high accuracy rates of 81.5%, 66.7%, and 91.2% respectively. These results indicate that the combination of structured representation, attribution-guided editing, and rigorous validation is highly effective for enhancing the performance of frozen language models in complex, multi-step reasoning tasks. The ai search story also surfaces in OpenAI Says AI Found Possible Navier–Stokes..., adding another angle. as detailed in the full paper on Arxiv