Efficient Test-Time Adaptation through Human-AI Int... | AI Research

Efficient Test-Time Adaptation through Human-AI Interaction
AI agents are typically trained on massive datasets to be generalists, but they often struggle to meet the high, specific standards required by professionals. Because individual expertise involves nuanced preferences that are difficult to define upfront, agents often fail to produce work that is ready for professional use. This paper introduces Test-time Adaptation through Human-agent Interaction (TAHI), a framework that allows agents to learn from a user’s feedback during real-world tasks. By treating the ongoing interaction between a human and an agent as a valuable data source, the system adapts to individual preferences, effectively closing the gap between general AI capabilities and expert-level performance.

How the TAHI Framework Works

TAHI functions as an iterative loop where a human and an agent collaborate to complete tasks in domains like writing and visual creation. The framework provides four primary channels for human input: planning adjustments, direct edits to deliverables, textual feedback, and the specification of success criteria. The same ai evaluation question is explored in Measure Before You Manage, which adds a research perspective.
As the user interacts with the agent, the system uses two main methods to adapt:

• Context-based adaptation: The agent stores factual knowledge and procedural workflows in a memory and skill library, which it references in future tasks.

• Weight-based adaptation: The agent uses Direct Preference Optimization (DPO) to update its internal model parameters. It learns to prefer the high-quality, human-refined final outputs over its initial, less-polished attempts.

The Role of Evolving Rubrics

A unique feature of this framework is the "evolving rubric module." In many open-ended tasks, success criteria are not clearly documented. The TAHI system uses an LLM to automatically generate evaluation rubrics based on the human’s feedback and corrections. These rubrics act as a scalable annotation tool, allowing the agent to "learn" what the user considers a success. Because the user can correct these rubrics if they are inaccurate, the system continuously refines its understanding of the user's specific requirements. The same large language models question is explored in GRAIN, which adds a research perspective.

Key Results and Performance

The researchers tested TAHI with 30 individuals across 600 tasks. The results showed that agents could adapt to a specific user’s style and preferences within only a few dozen tasks. Key findings include:

• Improved Success Rates: Agents using TAHI saw a 4.5% to 20.9% improvement in task success compared to non-adapted agents.

• Better Evaluation: The evolved rubrics were significantly more effective than those created by humans or LLMs alone, catching 16.0% to 22.3% more failures in agent outputs.

• Generalization: Even though the agents were personalized to specific users, they showed improved performance on tasks for other users, suggesting that the agents were learning a mix of both personal preferences and broader, shared professional strategies.

Implications for Future AI

The study suggests that the future of AI lies in tools that are not just generic, but are actively shaped by the people who use them. By leveraging the "last mile" of human-agent interaction, developers can create systems that are more accountable to their users and better suited for high-stakes professional environments. The ability to efficiently adapt to individual expertise without requiring extensive manual retraining makes this approach a promising path for making AI agents more reliable and useful in specialized fields. The same ai evaluation question is explored in Verify Smarter, Evolve Further, which adds a research perspective. as detailed in the full paper on Arxiv