ActMap: Single-Pass Uncertainty Quantification from... | AI Research

ActMap: Single-Pass Uncertainty Quantification from Generation-Time Activation Maps
Large language models often provide answers with high confidence, even when those answers are incorrect. To address this, researchers need a reliable way to determine if a specific model output is trustworthy without requiring expensive, repeated sampling. This paper introduces ActMap, a method that monitors a model's internal "thought process" during a single generation pass to predict the correctness of its output. By compressing the model's complex internal activity into a compact, standardized map, ActMap provides a practical tool for deciding when to trust a model, when to ask for human review, or when to trigger a fallback. The ai search story also surfaces in Google AI Releases TimesFM 3 for..., adding another angle.

How ActMap Works

During the generation of an answer, a transformer model produces a massive amount of internal data, specifically the hidden states of every layer for every token generated. ActMap captures this "trajectory" and compresses it into a fixed-size tensor (12 channels, 32 layers, 128 hidden coordinates). This process uses temporal statistics—such as how the model's internal state changes over time, the magnitude of its activity, and the "slope" of its internal dynamics—to summarize the entire generation process into a 96 KiB artifact. Because this compression happens during the standard generation pass, it adds no measurable delay. Once the map is created, a lightweight, pre-trained classifier (a compact Vision Transformer) reads the map and outputs a probability score representing the likelihood that the answer is correct.

Performance and Efficiency

The researchers evaluated ActMap across various tasks, including factual question answering, math problem solving, and long-form summarization. In these tests, ActMap consistently outperformed existing methods that rely on multiple samples, output probabilities, or single-vector internal probes. Notably, ActMap achieved performance levels comparable to much larger, more complex detectors (like ACT-ViT) while using a representation that is 67 times smaller. Because the resulting map has a fixed shape regardless of the model's size or depth, it serves as a universal, generator-agnostic tool for monitoring model reliability. The same large language models question is explored in Efficient Test-Time Adaptation through Human-AI Interaction, which adds a research perspective.

Practical Applications

The primary goal of ActMap is to provide a "reliability signal" that can be used in real-world deployments. By assigning a correctness score to every generated answer, the system enables several critical oversight functions:

• Abstention: The system can choose not to provide an answer if the confidence score is too low.

• Routing: Low-confidence answers can be automatically routed to a human reviewer or a more capable model.

• Selective Verification: Resources for checking facts can be focused specifically on the answers that ActMap flags as uncertain, rather than wasting effort on every single generation.

Key Considerations

ActMap is designed as a "white-box" method, meaning it looks directly at the model's internal activations rather than just the final text or output probabilities. While it excels at identifying uncertainty, the researchers note that long-form tasks, such as summarizing documents, remain the most challenging regime for all uncertainty quantification methods. Because summary factuality is multi-dimensional and often graded, the signal is harder to extract than in simple, binary-correctness tasks. However, even in these difficult scenarios, ActMap demonstrated a superior ability to maintain calibration, ensuring that the predicted confidence levels closely match the actual empirical accuracy of the model. The ai search story also surfaces in OpenAI Says AI Found Possible Navier–Stokes..., adding another angle. as detailed in the full paper on Arxiv