Phantom Gains: Auditing Self-Improvement Against a... | AI Research

Phantom Gains: Auditing Self-Improvement Against a Measured Null investigates why current methods for measuring language model self-improvement are often unreliable. The authors, Cheng Xu, Nan Yan, Liming Chen, and M-Tahar Kechadi, demonstrate that common evaluation practices—such as relying on single-decode results or arbitrary thresholds—can manufacture the appearance of capability gains where none exist. By auditing Qwen3-8B models across multiple rounds of self-training, the researchers show that many reported "improvements" are actually measurement artifacts that disappear when compared against a frozen, untrained control model.

Identifying Measurement Failures

The researchers identified seven distinct failure modes that frequently lead to incorrect conclusions in self-improvement research. A primary issue is the use of a single greedy decode to track model changes, which the authors found to be highly unstable due to inference batching and nondeterminism. Even when using a frozen model, this method can produce "phantom" gains and losses. Other failures include the use of underpowered ledgers, reliance on a single training seed, and the use of expansion thresholds that do not account for natural sampling noise. The authors argue that these errors are often standard practice and can invert findings, making a model appear to have learned or corrupted its knowledge when it has not.

The Need for a Measured Null

To address these inaccuracies, the paper proposes a transition-level auditing framework that requires a separately measured null for every statistic reported. Instead of assuming a zero-baseline for improvement, the authors suggest using a frozen control model—pushed through the exact same evaluation pipeline—to establish a "noise floor." By re-evaluating an untrained model at every checkpoint, researchers can determine whether observed changes in performance are statistically significant or merely the result of sampling jitter. The authors demonstrate that this approach is cost-effective, as it utilizes baseline replicates that are already available in multi-arm studies.

Replacing Thresholds with Exact Tests

The study finds that common "expansion" statistics, which attempt to distinguish between a model learning a new skill and sharpening an existing one, are fundamentally flawed because their null values are not zero. The authors replace these threshold-based metrics with a per-problem exact test against a pooled baseline. This method, conducted under false-discovery-rate control, allows for a more rigorous comparison across different model arms. When applied to their experiments, this controlled audit revealed that while external distillation improved performance on rarely-reached problems, three forms of self-training did not produce significant gains and, in some cases, corrupted existing capabilities.

Key Findings and Limitations

The audit concludes that transition-level analysis is highly sensitive to how data is sampled and measured. The researchers found that self-training methods often corrupt problems the base model already solves at rates significantly higher than the measured floor. While the study provides a robust framework for auditing, the authors note that their evidence remains inconclusive for the small set of problems that the base model never reaches. They emphasize that because transitions are differences between noisy estimates, researchers must be careful to distinguish between actual capability shifts and the "flicker" inherent in model evaluation.