What Do Compliance Detectors Read? An Audit of Acti...

What Do Compliance Detectors Read? An Audit of Acti... | AI Research

Key Takeaways

This paper investigates whether language model compliance detectors actually evaluate specific regulatory rules or if they simply react to the general conten...
Such monitoring is meaningful only if a detector's verdict depends on the stated rule rather than on surface features of the scenario.
We show this condition fails across the current class of compliance detectors, a failure we call rule blindness.
Auditing at scale requires a retraining-free detector, so we introduce the Internal Compliance Score (ICS): a training-free activation readout calibrated from ten labelled pairs and scored by a single projection.
We hold ICS to the same scrutiny as the guards it audits: a pre-registered criterion for beating trivial baselines is not met, and a bag-of-words model matches its pooled generalisation exactly.

Paper AbstractExpand

Regulatory compliance monitoring in deployed language models is increasingly implemented as a legal and audit control, checking model outputs against written rules spanning data protection, healthcare, financial regulation, and platform policy. Such monitoring is meaningful only if a detector's verdict depends on the stated rule rather than on surface features of the scenario. We show this condition fails across the current class of compliance detectors, a failure we call rule blindness. Deleting, permuting, or substituting the governing rule leaves detection accuracy unchanged for every guard and activation probe we test, including a policy-conditioned guard that correctly cites the governing clause yet barely changes its verdict when that clause is swapped for its permissive counterpart. A purpose-built benchmark crossing two rules with two scenarios, so that neither alone predicts the label, confirms the failure under a design no prior benchmark rules out, and shows that step by step reasoning, not any fast detector we test, is what escapes it. Auditing at scale requires a retraining-free detector, so we introduce the Internal Compliance Score (ICS): a training-free activation readout calibrated from ten labelled pairs and scored by a single projection. We hold ICS to the same scrutiny as the guards it audits: a pre-registered criterion for beating trivial baselines is not met, and a bag-of-words model matches its pooled generalisation exactly. It remains useful because it is inexpensive, letting us audit four deployed guard models, an 8B zero-shot judge, and thirteen benchmarks, and it raises the mechanically verified pass rate when used to rank candidate responses, though an adaptive white-box attack removes this gain. We release the counterfactual protocol and crossed-rule benchmark so rule blindness can be tested in future probe and guard claims.

This paper investigates whether language model compliance detectors actually evaluate specific regulatory rules or if they simply react to the general content of a scenario. The authors, Saisab Sadhu, Aadit Sengupta, Vinay Kumar Sankarapu, and Pratinav Seth, demonstrate that current guard models and activation probes suffer from "rule blindness," meaning their detection accuracy remains unchanged even when the governing rule is deleted, swapped, or permuted.

The Problem of Rule Blindness

Compliance monitoring is intended to check model outputs against specific legal or policy rules. However, the researchers found that existing detection systems—including fixed-taxonomy safety classifiers and policy-conditioned guards—fail to distinguish between the rule and the scenario. When the researchers replaced a governing rule with an unrelated one or removed it entirely, the detectors continued to flag the same outputs. This indicates that these systems are tracking a general "violation signal" inherent in the scenario rather than performing the logical composition of a rule against a specific case.

The Internal Compliance Score (ICS)

To audit these systems at scale without the need for expensive retraining, the authors introduced the Internal Compliance Score (ICS). ICS is a training-free activation readout that functions by calculating the difference between class means of adherent and violating texts. By using only ten labeled pairs, the system creates a projection that can be applied to a model's internal activations. While ICS is inexpensive and effective at ranking compliance risk, the authors emphasize that it is also rule-blind, just like the other detectors they tested. Its primary value lies in its efficiency and ability to be recalibrated at a negligible cost.

Evaluating Compliance Benchmarks

The study highlights that many public compliance benchmarks are "lexically degenerate," meaning they can be solved by simple bag-of-words models that ignore the rule entirely. The authors found that in four out of seven benchmarks tested, a policy-blind model performed as well as more complex detectors. Because the labels in these datasets are often redundant or predictable from the text surface, they are insufficient for testing whether a model is truly following a rule. To address this, the authors used outcome-ablated inputs and a purpose-built benchmark that crosses rules with scenarios to ensure that neither element alone can predict the label.

Limitations and Practical Implications

The researchers conclude that current activation-based compliance readouts are limited in what they can claim. While tools like ICS can improve the pass rate of candidate responses by ranking them, they remain susceptible to adaptive white-box attacks. Furthermore, the authors note that while their method generalizes across distributions within the same family (e.g., from one compliance task to another), it fails when forced to transfer across entirely different families, such as moving from safety data to regulatory compliance. The findings suggest that current compliance detectors offer the appearance of rule-specific assurance without the underlying mechanism to support it.