Knowledge Reutilization in Meta-Reinforcement Learning | AI Research

Key Takeaways

Knowledge Reutilization in Meta-Reinforcement Learning Meta-reinforcement learning is designed to help AI agents adapt quickly to new tasks by identifying sh...
Meta-reinforcement learning enables fast adaptation by extracting shared structure from related tasks, but existing end-to-end methods often couple task inference with embodiment-specific control.
This coupling can obscure non-parametric task semantics, reduce sample efficiency, and limit cross-agent reuse.
We propose a meta-knowledge reutilization framework that learns task-level knowledge on a dynamics-simplified agent and transfers it to heterogeneous agents.
The framework uses a Bayesian non-parametric prior to organize latent task modes and a high-level policy to generate task-level magnitude guidance.

Paper AbstractExpand

Meta-reinforcement learning enables fast adaptation by extracting shared structure from related tasks, but existing end-to-end methods often couple task inference with embodiment-specific control. This coupling can obscure non-parametric task semantics, reduce sample efficiency, and limit cross-agent reuse. We propose a meta-knowledge reutilization framework that learns task-level knowledge on a dynamics-simplified agent and transfers it to heterogeneous agents. The framework uses a Bayesian non-parametric prior to organize latent task modes and a high-level policy to generate task-level magnitude guidance. To bridge reusable task knowledge with different embodiments, we introduce a semantic-magnitude interface and a lightweight temporal adaptor, which convert frozen meta-knowledge into temporally aligned subgoals for embodiment-specific low-level controllers. Experiments on multiple locomotion agents show that our framework reduces final-step tracking error by 94.75% -- 99.79% compared with recent state-of-the-art baselines and achieves comparable deployment performance with about 23.8% of their interaction data.

Knowledge Reutilization in Meta-Reinforcement Learning

Meta-reinforcement learning is designed to help AI agents adapt quickly to new tasks by identifying shared patterns across different challenges. However, current methods often bundle task-learning with the specific physical movements of a robot, which makes it difficult to reuse knowledge across different types of agents. This paper introduces a new framework that separates high-level task knowledge from low-level physical control, allowing an AI to learn a task once and apply it to various agents with different physical structures.

Decoupling Knowledge from Embodiment

The core problem with existing end-to-end meta-learning is that the "what to do" (task semantics) is too closely tied to the "how to move" (embodiment-specific control). This limits efficiency and prevents agents from sharing what they have learned. To solve this, the authors propose training an agent on a simplified version of physics to extract pure, task-level knowledge. By using a Bayesian non-parametric prior, the framework organizes these tasks into distinct modes, allowing the system to understand the underlying structure of a task without being distracted by the complexities of a specific robot's body.

Bridging the Gap with Interfaces

To transfer this "frozen" meta-knowledge to different robots, the framework uses two key components: a semantic-magnitude interface and a lightweight temporal adaptor. The interface translates the abstract task knowledge into magnitude-based guidance, while the adaptor ensures these instructions are timed correctly for the specific robot. Essentially, this allows a high-level policy to issue general goals—like "move forward at this speed"—which the robot's low-level controller then executes based on its own unique physical capabilities.

Significant Gains in Efficiency

The experimental results demonstrate that this approach is highly effective for locomotion tasks. Compared to current state-of-the-art methods, the framework reduced final-step tracking error by 94.75% to 99.79%. Beyond accuracy, the system is also significantly more data-efficient; it achieved comparable performance to existing baselines while using only about 23.8% of the interaction data typically required. This suggests that by separating task knowledge from physical control, AI agents can learn faster and apply their skills more flexibly across different hardware.