CLEAR: Continuous Latent Adapter Routing for Utility-Preserving LLM Safety Alignment
CLEAR uses input-conditioned latent gating to reduce HarmBench ASR from 32.3% to 0.5%, maintaining high utility on GSM8K and other tasks.
Key Findings
Methodology
CLEAR employs a frozen backbone language model with an added safety low-rank adapter (LoRA). A trainable hidden state gate predicts a risk score g∈[0,1], dynamically controlling the adapter’s activation strength for each input. The framework introduces subtype-aware gate weighting and a pairwise margin loss to improve discrimination between harmful and benign prompts. During training, gate parameters ψ and adapter parameters θ are jointly optimized, enabling input-specific safety interventions. During inference, the gate estimates the risk score from hidden states, modulating the adapter’s influence to preserve utility on benign prompts while activating safety behavior on harmful ones.
Key Results
- On Llama-3-8B-Instruct, CLEAR reduces HarmBench attack success rate (ASR) from 32.3% to 0.5%, while maintaining 73.46% GSM8K accuracy, outperforming SFT and standard LoRA in safety-utility trade-off.
- In Gemma-2-2B, it achieves 0% HarmBench ASR and improves GSM8K accuracy to 41.62%, demonstrating robustness and utility retention across models.
- The method’s parameter efficiency (less than one million parameters) surpasses external guard models, with the gate effectively distinguishing unsafe prompts, leading to better safety-utility balance.
Significance
This work addresses the longstanding challenge of balancing safety and utility in large language models. By introducing input-conditioned safety regulation via a lightweight, continuous gating mechanism, it enables models to selectively activate safety behaviors only when necessary. This approach mitigates over-refusal and performance degradation, facilitating safer deployment in real-world applications. Its scalability and efficiency make it suitable for diverse models and tasks, marking a significant advancement in AI safety research and practical deployment.
Technical Contribution
The core technical innovation is the continuous latent gating mechanism that dynamically modulates safety adapter activation based on input risk estimation. This departs from traditional global fine-tuning, offering a more fine-grained, input-sensitive safety control. The integration of subtype-aware weighting and pairwise margin loss enhances the gate’s discriminative ability, leading to more precise safety interventions. The framework’s lightweight design ensures training efficiency and interpretability, opening new avenues for scalable, adaptive safety alignment in large models.
Novelty
This is the first work to implement a continuous, input-conditioned gating mechanism for safety adaptation in LLMs, moving beyond static or external guardrails. By leveraging internal hidden states to predict risk scores, the method achieves a fine-grained, context-aware safety response, significantly reducing harmful outputs while preserving task utility. Its combination of continuous control and subtype-aware discrimination represents a novel paradigm in model safety alignment.
Limitations
- The approach relies on predefined subtype labels, which may not cover all emerging or unforeseen safety threats, limiting its generalizability in novel attack scenarios.
- The gate’s accuracy depends on the quality of training data; in complex or adversarial environments, misclassification may occur, affecting safety and utility balance.
- While parameter-efficient, the method still requires additional training and tuning, which could be challenging for extremely large models or resource-constrained settings.
Future Work
Future research will explore unsupervised risk estimation techniques to reduce label dependency, extend the framework to multi-modal inputs, and incorporate adaptive, self-supervised risk learning. Additionally, integrating this approach with broader multi-objective optimization strategies could further enhance safety and utility, enabling more robust, scalable deployment of large language models in diverse real-world scenarios.
AI Executive Summary
The rapid advancement of large language models (LLMs) has unlocked unprecedented capabilities in reasoning, coding, and instruction following, transforming AI applications across industries. However, these models also pose significant safety risks, including harmful content generation and adversarial manipulation. Traditional safety alignment methods, such as supervised fine-tuning (SFT) and reinforcement learning from human feedback (RLHF), aim to mitigate these risks but often compromise the models’ utility and responsiveness due to global parameter adjustments. This creates a persistent safety-utility trade-off, limiting the models’ practical deployment.
In response, Wang et al. introduce CLEAR, a novel framework that enables input-conditioned safety regulation through a lightweight, continuous latent gating mechanism. By freezing the core language model and adding a safety-specific LoRA, the system predicts a risk score for each input based on internal hidden states. This score dynamically modulates the strength of safety adapter activation, allowing benign prompts to pass with minimal interference while activating safety behaviors on harmful or adversarial prompts. The key innovation lies in the continuous, input-sensitive control, which avoids the over-restriction typical of global fine-tuning.
Experimental results demonstrate that CLEAR dramatically reduces harmful outputs, achieving near-zero attack success rates on benchmarks like HarmBench, while maintaining or improving task performance such as GSM8K accuracy. On Llama-3-8B-Instruct, the method reduces HarmBench attack success from 32.3% to 0.5%, outperforming traditional methods in safety-utility balance. Its parameter efficiency and interpretability further enhance its appeal, making it suitable for scalable, real-world deployment. This approach offers a promising pathway toward safer, more reliable large language models, addressing a critical challenge in AI safety research.
Despite its strengths, the method relies on predefined subtypes and supervised risk labels, which may limit its adaptability to unforeseen threats. Future work will focus on unsupervised risk estimation, multi-modal extensions, and broader multi-objective optimization to further improve robustness and generalization. Overall, CLEAR marks a significant step forward in aligning large models with safety requirements without sacrificing their core utility, paving the way for more trustworthy AI systems.
Deep Dive
Abstract
Improving the safety of large language models (LLMs) often comes at the expense of utility, as globally applied safety tuning may affect model responses to both harmful and benign inputs. We propose \textbf{C}ontinuous \textbf{L}at\textbf{E}nt \textbf{A}dapter \textbf{R}outing (CLEAR), a conditional safety adaptation framework that uses a lightweight hidden-state gate to continuously control the activation strength of a safety low-rank adapter. CLEAR aims to reduce harmful completions while avoiding unnecessary changes to the frozen backbone that could degrade performance on benign prompts. Experiments on widely used safety and utility benchmarks show that CLEAR improves robustness on HarmBench while reducing the utility degradation observed with globally applied safety tuning such as SFT or standard low-rank adaptation (LoRA). On Llama-3-8B-Instruct, CLEAR reduces HarmBench ASR from 32.3\% to 0.5\%, while retaining most of the base model's utility and achieving up to 7.1 percentage points higher GSM8K accuracy than globally applied SFT or LoRA. These results suggest that CLEAR is a promising mechanism for improving the safety--utility trade-off in LLM alignment.