Back to AI Research

AI Research

Agent-safety study finds benchmark formatting can change guard decisions

Key Takeaways

  • The paper separates predictable labels from actual decision sensitivity and finds effects that do not transfer uniformly across models.
  • ## A safety score can depend on the dataset
  • Shortcut Sensitivity in Agent-Safety Evaluations asks whether guards are evaluated on the behavior they are meant to detect or on incidental clues in a benchmark.
  • The authors distinguish three questions: whether labels are predictable from shortcuts, whether those clues are associated with guard decisions, and whether changing a clue changes a decision on the same example.
  • Their audit covers completed LinuxArena trajectories and two TS-Bench-derived step datasets.

A safety score can depend on the dataset

Shortcut Sensitivity in Agent-Safety Evaluations asks whether guards are evaluated on the behavior they are meant to detect or on incidental clues in a benchmark. The authors distinguish three questions: whether labels are predictable from shortcuts, whether those clues are associated with guard decisions, and whether changing a clue changes a decision on the same example.
Their audit covers completed LinuxArena trajectories and two TS-Bench-derived step datasets. In one analysis, shorter shell commands strongly predict trajectory labels. In another, exact-tool lookup performs much worse when evaluation holds out domains instead of mixing them across folds.

Paired edits test the stronger claim

The researchers then change representations in a frozen 64-step AgentDojo-Traj cohort using repeated, interleaved Haiku calls. Rewriting an INFORMATION marker reduced blocking by 8.33 percentage points, while opaque tool aliases increased it by 11.46 points.
All unsafe calls remained blocked in that experiment. The measured changes concerned false positives against inherited labels, so they should not be described as demonstrated successful unsafe actions. The authors also compare the interventions with a formatting placebo to help separate the intended edit from a general presentation effect.

The result has model-specific limits

The Haiku-sized effects did not reproduce in Luna or deterministic Q8 TS-Guard. That boundary matters: a representation effect found in one configuration is not proof that every guard shares the same sensitivity. The paper also shows that weighting rows versus interactions can change reported metrics.
Franklin's reading is that guard evaluation needs both grouped holdouts and paired tests. A useful audit asks whether the model handles a new domain and whether harmless representation changes alter decisions on otherwise matched cases. A high aggregate score alone cannot resolve those questions, and a shortcut that predicts labels is not automatically evidence that it caused a guard's behavior.
Read the shortcut-sensitivity paper.

Comments