LatentGuard: Efficient and Inspectable Latent Reaso...

LatentGuard: Efficient and Inspectable Latent Reaso... | AI Research

Key Takeaways

LatentGuard is a framework designed to make Large Language Model (LLM) safety moderation more efficient by replacing the generation of long, explicit text ra...
Reasoning-based guard models improve LLM safeguards, but decoding explicit rationales for every interaction makes them costly to deploy.
Although latent-reasoning methods reduce token generation by moving reasoning into continuous states, they remain underexplored for safety moderation and lack an inspection interface for deployment.
In this paper, we propose LatentGuard, an efficient and inspectable safeguard framework that brings continuous latent reasoning to guard models.
LatentGuard uses a staged curriculum to progressively compress task-aligned textual rationales into compact latent states, enabling safety verdicts to be predicted directly from continuous representations.

Paper AbstractExpand

Reasoning-based guard models improve LLM safeguards, but decoding explicit rationales for every interaction makes them costly to deploy. Although latent-reasoning methods reduce token generation by moving reasoning into continuous states, they remain underexplored for safety moderation and lack an inspection interface for deployment. In this paper, we propose LatentGuard, an efficient and inspectable safeguard framework that brings continuous latent reasoning to guard models. LatentGuard uses a staged curriculum to progressively compress task-aligned textual rationales into compact latent states, enabling safety verdicts to be predicted directly from continuous representations. To preserve inspectability, an isolated auxiliary decoder generates compact audit artifacts on demand, keeping rationale generation off the standard inference path. Experiments show that LatentGuard-8B improves mean weighted F1 from 83.95 to 84.91 over GuardReasoner-8B, while reducing critical-path reasoning cost from 268.56 generated rationale tokens to 1.60 latent reasoning tokens. Its audit decoder achieves an audit utility score of 85.75, demonstrating an efficient and inspectable path toward deployable LLM safeguards.

LatentGuard is a framework designed to make Large Language Model (LLM) safety moderation more efficient by replacing the generation of long, explicit text rationales with compact, continuous latent states. By moving reasoning into these states, the system reduces the computational cost of safety checks while maintaining the ability to inspect the model's decision-making process when necessary.

Addressing the Cost of Safety Reasoning

Current reasoning-based guard models improve safety by generating explicit rationales for every interaction. However, this process is expensive because it requires the model to generate a large number of tokens for every safety check. LatentGuard addresses this by using a staged curriculum to compress textual rationales into compact latent representations. This allows the model to predict safety verdicts directly from these continuous states, significantly reducing the number of tokens generated during the critical path of inference.

Maintaining Inspectability

A primary challenge with latent-reasoning methods is that they are often "black boxes," making it difficult to understand why a specific safety verdict was reached. LatentGuard preserves inspectability through an isolated auxiliary decoder. This component is not part of the standard, high-speed inference path; instead, it generates compact audit artifacts only on demand. This design ensures that the system remains efficient for real-time deployment while still providing a way to review the reasoning behind safety decisions.

Performance and Efficiency Results

In experiments comparing the LatentGuard-8B model to the GuardReasoner-8B model, the authors reported the following results:

Improved Accuracy: The mean weighted F1 score increased from 83.95 to 84.91.
Reduced Computational Cost: The number of generated rationale tokens required for the critical path dropped from 268.56 to 1.60.
Audit Utility: The auxiliary audit decoder achieved an audit utility score of 85.75, confirming that the system can produce useful, inspectable outputs despite the shift to latent reasoning.

Franklin Analysis

The evidence suggests that LatentGuard successfully decouples the efficiency of safety moderation from the need for constant, explicit text generation. By offloading the "audit" requirement to an auxiliary decoder, the framework resolves the trade-off between the high performance of reasoning-based guards and the deployment constraints of real-time LLM applications. The significant reduction in token generation—from over 268 tokens down to 1.6—indicates that the staged curriculum approach is highly effective at compressing complex reasoning into a format suitable for rapid safety classification.