When Tool Outputs Become Commands: Separating Actio... | AI Research

When Tool Outputs Become Commands: Separating Action Induction from Runtime Authorization in Tool-Augmented LLM Agents introduces SARA, a security mechanism designed to prevent LLM agents from executing unauthorized commands derived from untrusted external data. The researchers, led by Xiaokun Guo and colleagues at the Institute of Information Engineering, Chinese Academy of Sciences, address the risk where tool outputs—such as emails or webpages—contain hidden instructions that manipulate an agent into performing unintended real-world actions.

The Problem: Conflating Induction with Authorization

The paper identifies a core security tension in open-ended agent tasks: agents must process untrusted runtime observations to complete legitimate work, but this process often allows malicious "action-inducing" content to masquerade as valid instructions. The authors define this as a "confused deputy" problem, where an agent’s legitimate privileges are hijacked. The central challenge is that current systems often fail to distinguish between "runtime instantiation"—using data to complete a user-authorized task—and "authority expansion," where external content tricks the agent into performing actions the user never intended.

How SARA Works

SARA operates by decoupling the process of identifying potential actions from the process of authorizing them. It functions through two primary components:

• Action Probe: This component monitors incoming observations for action-inducing semantics. When it detects such content, it marks the trajectory as "EXPOSED" and records the provenance of these potential actions. Crucially, this probe does not block data; it merely flags the origin of the action for later review.

• Runtime Authorization: At the point of execution, SARA evaluates candidate tool calls against three criteria: the original user objective, the persistent action origins recorded by the probe, and audited evidence from previously successful, authorized executions.
  To prevent attackers from bypassing these checks, SARA employs "No-History-Promotion," a rule ensuring that historical tool outputs cannot be used to "launder" or override the original action-inducing source. This ensures that even if an action appears repeatedly in the execution history, it cannot gain authority unless it is explicitly supported by the user’s initial request.

Performance and Results

The researchers evaluated SARA using the AgentDojo and AgentDyn benchmarks. According to the paper, SARA limited the Attack Success Rate (ASR) to no more than 0.63% across four primary evaluation settings. The authors report that this security gain was achieved while maintaining competitive task utility, meaning the system remained capable of performing complex, multi-step tasks despite the added authorization constraints.

Franklin Analysis

The evidence suggests that SARA’s effectiveness stems from its refusal to treat all observation-derived information as equally trustworthy. By requiring that every tool call with real-world side effects be independently authorized—rather than relying on the agent's internal reasoning or planning state—the system creates a "hard" boundary at the point of execution. This approach is notable because it does not require the agent to be "perfect" at identifying malicious content; instead, it shifts the security burden to a verifiable, rule-based check at the final execution step.