Break It Down, Pass It On: Cross-Task Skill Transfe... | AI Research

Break It Down, Pass It On: Cross-Task Skill Transfer in LLM Agents investigates how Large Language Model (LLM) agents can improve their performance by learning from past tasks. The researchers, Yiyang Feng, Biddut Sarker Bijoy, Niranjan Balasubramanian, and Jiawei Zhou, examine how the method of "skill induction"—the process of extracting and storing knowledge from completed tasks—determines whether an agent becomes more capable or suffers from performance degradation due to irrelevant or misaligned information.

Improving Skill Transfer Through Decomposition

The paper finds that the level at which a skill is induced is the primary factor in whether it helps or harms an agent. When agents induce skills from an entire task trajectory, these skills are often too specific to the source task, leading to poor generalization and potential distraction in future tasks.
In contrast, the authors show that decomposing tasks into smaller subtasks and inducing one skill per subtask leads to more reliable transfer. Because different tasks often share common sub-procedures, these subtask-level skills are more likely to be relevant and reusable across different goals. Experiments across three long-horizon benchmarks—AppWorld, OfficeBench, and KramaBench—show that subtask-level skills consistently raise agent performance above the no-memory baseline, whereas task-level skills often reduce performance below that baseline.

Text Versus Code Skill Formats

The researchers compared two formats for storing these skills: natural-language text notes and executable Python code functions. Their analysis indicates that text-based skills generally transfer better than code-based skills. While code skills allow for functional execution, text skills provide flexible workflow notes that appear to be more robust for cross-task application. Across the 11 models tested, including various Mixture-of-Experts and dense models, the combination of subtask-level induction and text-based storage yielded the most consistent performance gains.

A Diagnostic Tool for Skill Utility

To predict whether a stored skill will actually benefit an agent, the authors introduce a "skill utility score." This score is based on two properties:

• Specificity: How closely a skill matches the requirements of a task.

• Abstractness: How evenly the skill’s relevance spreads across multiple tasks.
  The researchers found that neither property alone predicts success, but their combined effect—the utility score—correlates with task performance. A key advantage of this score is that it can be computed using only the skill and task descriptions, without requiring the agent to actually execute the task. This allows practitioners to use the score as a lightweight diagnostic tool to evaluate the quality of a skill memory before deploying an agent on new tasks.

Practical Considerations

The study highlights that skill transfer is not inherently beneficial; if managed poorly, it can introduce noise that degrades model performance. The authors suggest that for reliable improvement, developers should prioritize subtask-level decomposition and favor text-based skill formats. By using the skill utility score, developers can proactively manage their agent's memory, ensuring that only high-utility, relevant skills are retrieved during task execution.