Detecting a disclosure before it is written
The paper Pre-Crime for Agents, at the Last Second studies a controlled setting where one agent pressures another to reveal a secret. Across six scenarios, the target can be challenged for up to 20 rounds before a disclosure occurs.
The authors examine a linear probe on the target’s activations before an output is generated. The aim is to identify an imminent disclosure early enough for an overseer to intervene, rather than filter only text that has already been written.
A warning signal with a narrow horizon
On held-out Qwen3-32B episodes, a threshold set to flag 5% of benign calls catches 82% of leaking calls. The paper compares that with 26% for a zero-shot decision-model monitor at a similar flag rate.
Those figures describe a particular labeled evaluation. They do not establish a general probability that a model will violate its operator’s rules, or a reliable detector of every form of agent collusion.
Predicting disclosure is not judging permission
Two findings limit the operational interpretation. A bag-of-words classifier trained on the same calls comes close to the probe’s performance, suggesting much of the gain comes from labeled training rather than activation access alone. The probe also flags permitted disclosures.
The authors therefore characterize the signal as reading an approaching disclosure more than its authorization status. Stopping whenever an alarm appears would halt most benign episodes in the experiment, and a second text check does not solve that problem.
For monitoring research, the contribution is a short-range signal worth investigating alongside its failure modes. A useful next evaluation must separate unauthorized disclosure from allowed behavior and measure the cost of intervention over complete episodes. Predictive performance on individual calls is only one part of deciding whether a stopping policy would help.
Comments