Back to AI Research

AI Research

CLEAR: Continuous Latent Adapter Routing for Utilit... | AI Research

Key Takeaways

  • CLEAR: Continuous Latent Adapter Routing for Utility-Preserving LLM Safety Alignment introduces a framework designed to improve LLM safety without degrading...
  • Improving the safety of large language models (LLMs) often comes at the expense of utility, as globally applied safety tuning may affect model responses to both harmful and benign inputs.
  • CLEAR aims to reduce harmful completions while avoiding unnecessary changes to the frozen backbone that could degrade performance on benign prompts.
  • These results suggest that CLEAR is a promising mechanism for improving the safety--utility trade-off in LLM alignment.
  • CLEAR: Continuous Latent Adapter Routing for Utility-Preserving LLM Safety Alignment introduces a framework designed to improve LLM safety without degrading the model's performance on benign tasks.
Paper AbstractExpand

Improving the safety of large language models (LLMs) often comes at the expense of utility, as globally applied safety tuning may affect model responses to both harmful and benign inputs. We propose \textbf{C}ontinuous \textbf{L}at\textbf{E}nt \textbf{A}dapter \textbf{R}outing (CLEAR), a conditional safety adaptation framework that uses a lightweight hidden-state gate to continuously control the activation strength of a safety low-rank adapter. CLEAR aims to reduce harmful completions while avoiding unnecessary changes to the frozen backbone that could degrade performance on benign prompts. Experiments on widely used safety and utility benchmarks show that CLEAR improves robustness on HarmBench while reducing the utility degradation observed with globally applied safety tuning such as SFT or standard low-rank adaptation (LoRA). On Llama-3-8B-Instruct, CLEAR reduces HarmBench ASR from 32.3\% to 0.5\%, while retaining most of the base model's utility and achieving up to 7.1 percentage points higher GSM8K accuracy than globally applied SFT or LoRA. These results suggest that CLEAR is a promising mechanism for improving the safety--utility trade-off in LLM alignment.

CLEAR: Continuous Latent Adapter Routing for Utility-Preserving LLM Safety Alignment introduces a framework designed to improve LLM safety without degrading the model's performance on benign tasks. By using a lightweight gate to control when safety interventions occur, the researchers aim to solve the "alignment tax"—a common issue where global safety tuning makes models less useful or prone to over-refusal on safe prompts.

How the Approach Works

The researchers, Chengxiao Wang, Enyi Jiang, Xiaojing Liao, and Sanmi Koyejo, propose a method that keeps the base LLM frozen. Instead of applying safety updates to the entire model, they introduce a safety-specific Low-Rank Adapter (LoRA). A "hidden-state gate" analyzes the input prompt to predict a continuous routing score between 0 and 1.
If the gate identifies a prompt as benign, the score remains low, allowing the model to function as the original, unmodified backbone. If the gate detects harmful or adversarial intent, the score increases, activating the safety adapter to steer the model toward refusal or safe responses. The gate is trained using subtype-aware weighting and a hard pairwise margin objective to ensure it can effectively distinguish between harmful and benign inputs.

Key Results

In experiments using Llama-3-8B-Instruct and Gemma-2-2B-it, the authors report that CLEAR significantly reduces harmful completions while maintaining higher utility than standard fine-tuning or global LoRA methods.
On the Llama-3-8B-Instruct model, CLEAR reduced the HarmBench Attack Success Rate (ASR) from 32.3% to 0.5%. Simultaneously, it retained 73.46% accuracy on the GSM8K reasoning benchmark, which the authors note is approximately 7 percentage points higher than results achieved by standard SFT or LoRA. On the Gemma-2-2B-it model, the framework improved GSM8K accuracy from roughly 38% to 42% and increased TruthfulQA MC2 scores from 47% to 53%.

Why It Matters

This research addresses the tension between safety and utility in AI alignment. Traditional methods often apply safety tuning globally, which can unintentionally cause models to refuse benign requests or perform worse on reasoning tasks. By decoupling safety intervention from the core model through conditional routing, the authors provide a mechanism that applies safety measures only when necessary. The gate itself is efficient, requiring fewer than one million parameters, which the authors suggest makes it a practical alternative to larger, external moderation systems.

Considerations

The effectiveness of CLEAR relies on the gate’s ability to accurately classify prompts. The authors note that they specifically trained the gate to handle "adversarial" examples—both harmful and benign—to prevent the model from being confused by prompts that mimic unsafe patterns. Because the gate is jointly optimized with the safety adapter, the framework is designed to learn both the detection of risk and the appropriate safety behavior simultaneously. The authors emphasize that this approach is intended to be a modular, parameter-efficient way to align models without the performance degradation typically associated with broad, global parameter updates.

Comments (0)

No comments yet

Be the first to share your thoughts!