How to Interpret Agent Behavior | AI Research

How to Interpret Agent Behavior
As autonomous AI agents like Claude Code and Codex become more capable, they are increasingly tasked with complex, long-running projects. While these agents are designed to work independently, they often struggle, fail, and recover multiple times before completing a task. Currently, researchers primarily measure success using simple "pass/fail" metrics, which reveal the outcome but fail to explain the "how" or "why" behind an agent's performance. This paper introduces ACT*ONOMY, a structured taxonomy and analysis framework designed to help researchers, designers, and users interpret the runtime behavior of AI agents at scale.

A Shared Vocabulary for Agent Actions

Because there is no industry standard for describing what an agent is doing, researchers often struggle to communicate findings or compare different systems. ACT*ONOMY addresses this by providing a hierarchical, three-level vocabulary consisting of 10 main actions, 46 subactions, and 120 specific leaf categories. Developed using a grounded theory approach—which involved analyzing 565 behavior descriptions from peer-reviewed AI literature—the taxonomy covers everything from how an agent retrieves information and plans its next move to how it evaluates its own work and reflects on past errors.

Automated Analysis of Agent Trajectories

To move beyond manual, time-consuming review of agent logs, the authors provide an automated pipeline called the Automated-Trace-Analysis-Tool. This tool processes raw, unstructured agent logs—often messy, free-form text—and maps them to the ACT*ONOMY taxonomy. By breaking down trajectories into specific, quote-grounded labels, the tool can generate behavioral profiles that show exactly how an agent spends its time. This allows users to see patterns that are otherwise invisible, such as whether an agent is stuck in a loop, failing to verify its own work, or relying too heavily on specific types of reasoning.

Surfacing Failure Modes and Behavioral Profiles

The researchers demonstrated the utility of ACT*ONOMY through two primary use cases. First, they compared different agents (such as AG2, HyperAgent, and SWE-Agent) to show how their internal architectures lead to distinct behavioral signatures. For example, they found that while some agents are dominated by execution, others spend significantly more time on reflection or evaluation. Second, they analyzed individual trajectories to identify specific failure modes. In one instance, the tool successfully pinpointed a "submit without verifying" pattern in a failed task, where the agent recognized it could not perform a necessary test but proceeded to submit the work anyway.

A Living Framework

The authors emphasize that the field of autonomous agents is evolving rapidly. To ensure the framework remains relevant, they have released ACT*ONOMY as an open-source, "living" repository. This allows the community to propose, review, and incorporate new subactions and categories as agent technology advances. By providing a consistent, extensible way to describe agent behavior, the project aims to help the research community move toward more reliable, transparent, and controllable AI systems.