SAEScientist-Bench: Can AI Agents Conduct Autonomou... | AI Research

SAEScientist-Bench: Can AI Agents Conduct Autonomous SAE Interpretability Research?
As AI models become more complex, understanding their internal decision-making processes is critical for safety and alignment. While researchers often use Sparse Autoencoders (SAEs) to "peek" inside models and identify specific features, this process is currently manual and labor-intensive. This paper introduces SAEScientist-Bench, a new framework designed to test whether AI agents can act as autonomous scientists to conduct this research themselves, using SAE tools to discover and validate features within a model’s internal architecture. The same ai evaluation question is explored in Harness-of-Harness, which adds a research perspective.

Evaluating AI as a Scientist

The benchmark challenges AI agents to perform mechanistic interpretability research on the Gemma-2-9B-IT model. For each of the 20 assigned tasks, an agent is given a target concept—such as a specific language or a technical document format—and must navigate a dictionary of over 131,000 features to find the one that best represents that concept. To succeed, the agent must design "contrastive probes," which are sets of texts used to test whether a feature is truly specific to the target concept or if it is merely reacting to unrelated patterns.

How the Benchmark Measures Success

To ensure a rigorous evaluation, the researchers compared the agents' findings against a "gold standard" set of expert-curated features from Neuronpedia. The agents are scored on three specific dimensions:

• Activation Rank: How prominently the discovered feature appears in the model compared to the expert baseline.

• Activation Selectivity: How well the feature distinguishes the target concept from "hard-negative" or unrelated texts.

• Causal Steering: Whether the feature can actually be used as a "lever" to influence the model’s future output in a way that aligns with the target concept. The same ai evaluation question is explored in Efficient Test-Time Adaptation through Human-AI Interaction, which adds a research perspective.

Key Findings and Performance

The study evaluated 10 frontier AI models and found that while they demonstrate genuine scientific discovery capabilities, they still lag behind human experts. Agents performed impressively on activation selectivity, often reaching near-expert levels in distinguishing concepts from controls. However, they struggled significantly with causal steering—the ability to use a feature to reliably change model behavior. This suggests that while current AI agents are becoming adept at identifying patterns, they have difficulty confirming whether those patterns are the actual "causal drivers" of the model's behavior.

Current Limitations

The research reveals that even the most advanced agents often misinterpret their own experimental measurements. While they are capable of designing clever tests to rule out incorrect candidates, they frequently struggle to translate their findings into effective steering vectors. This gap highlights that, while autonomous AI research is a promising path for model auditing, current systems still require further development to bridge the divide between identifying a feature and mastering its causal control. The same ai evaluation question is explored in From Deceptive Outputs to Deceptive Mechanisms, which adds a research perspective. as detailed in the full paper on Arxiv