RASA: Routing-Aware Safety Alignment for Mixture-of-Experts Models
RASA enhances MoE model safety by repairing safety-critical experts and maintaining routing consistency.
Key Findings
Methodology
RASA is a routing-aware expert-level alignment framework for MoE models. It identifies and repairs safety-critical experts activated by successful jailbreaks, selectively fine-tuning these experts under fixed routing to ensure routing consistency with safety-aligned contexts.
Key Results
- RASA achieves near-perfect robustness against diverse jailbreak attacks, significantly improving cross-attack generalization and reducing over-refusal.
- It preserves general capabilities on benchmarks like MMLU, GSM8K, and TruthfulQA.
- RASA is data-efficient, achieving strong safety gains with minimal adversarial samples compared to full-parameter alignment and other MoE-specific methods.
Significance
RASA provides a practical, architecture-preserving safety alignment method for MoE models, addressing the issue of traditional full-parameter fine-tuning failing to effectively repair safety-critical experts. It has significant implications for academia and industry, particularly in high-security applications.
Technical Contribution
RASA decouples expert correction from routing control, offering a novel expert-level alignment framework. Unlike existing methods, it requires no auxiliary experts or architectural modifications, providing new theoretical guarantees and engineering possibilities.
Novelty
RASA is the first to propose selectively fine-tuning safety-critical experts under fixed routing, avoiding routing shortcut solutions. This innovation achieves safety alignment through expert-level repair rather than global parameter updates.
Limitations
- RASA may perform poorly against unseen complex jailbreak attacks that activate different routing paths.
- Routing instability may occur, leading to over-refusal of normal queries.
Future Work
Future research directions include optimizing routing consistency strategies, exploring more adversarial sample generation methods, and validating RASA's effectiveness on larger-scale MoE models.
AI Executive Summary
Mixture-of-Experts (MoE) models face unique challenges in safety alignment due to their sparse routing mechanisms. Traditional full-parameter fine-tuning often improves safety through routing or expert dominance effects rather than directly repairing safety-critical experts. To address this, the paper proposes RASA, a routing-aware expert-level alignment framework. RASA identifies and repairs safety-critical experts activated by successful jailbreaks, selectively fine-tuning these experts under fixed routing to ensure routing consistency with safety-aligned contexts. Experimental results show that RASA achieves near-perfect robustness against diverse jailbreak attacks, significantly improving cross-attack generalization and reducing over-refusal. It also preserves general capabilities on benchmarks like MMLU, GSM8K, and TruthfulQA. RASA provides a practical, architecture-preserving safety alignment method for MoE models, addressing the issue of traditional full-parameter fine-tuning failing to effectively repair safety-critical experts. Future research directions include optimizing routing consistency strategies, exploring more adversarial sample generation methods, and validating RASA's effectiveness on larger-scale MoE models.
Deep Analysis
Background
Mixture-of-Experts (MoE) models improve scalability and efficiency through sparse routing mechanisms, but introduce new failure modes. In these models, each input is processed by only a small subset of experts, which, while efficient, poses unique challenges for safety alignment. Traditional full-parameter fine-tuning often improves safety through routing or expert dominance effects rather than directly repairing safety-critical experts.
Core Problem
The core problem with MoE models is that their sparse routing mechanisms may leave safety-critical experts uncorrected, resulting in latent vulnerabilities under adaptive adversarial prompts. This phenomenon is known as an alignment shortcut, where safety objectives are satisfied through routing or expert dominance effects without meaningful expert-level repair.
Innovation
RASA's core innovation is its routing-aware expert-level alignment framework. It first identifies safety-critical experts activated by successful jailbreaks, then selectively fine-tunes these experts under fixed routing to ensure routing consistency with safety-aligned contexts. This innovation avoids the alignment shortcut problem present in traditional methods.
Methodology
- �� Identify Safety-Critical Experts: Recognize experts activated by jailbreaks through activation frequency discrepancies.
- �� Selective Fine-tuning: Fine-tune safety-critical experts under fixed routing to inject refusal behavior.
- �� Routing Consistency Optimization: Ensure routing consistency between jailbreak and safety-aligned inputs.
Experiments
The experimental design includes testing on two MoE architectures: Qwen3-30B-A3B and OLMoE-1B-7B-0125-Instruct. AdvBench is used as the core intent set, with state-of-the-art jailbreak attacks generating adversarial prompts. Baselines include the original model, SteerMoE, and SafeMoE-Fine-tuning.
Results
RASA achieves near-perfect robustness against diverse jailbreak attacks, significantly improving cross-attack generalization and reducing over-refusal. It preserves general capabilities on benchmarks like MMLU, GSM8K, and TruthfulQA.
Applications
RASA can be applied in high-security applications such as finance, healthcare, and government services. Its architecture-preserving nature makes it easy to integrate into existing systems.
Limitations & Outlook
RASA may perform poorly against unseen complex jailbreak attacks that activate different routing paths. Additionally, routing instability may occur, leading to over-refusal of normal queries.
Plain Language Accessible to non-experts
Imagine a factory with many workers, each responsible for different tasks. The MoE model is like this factory, where input information is distributed to different experts (workers) for processing. RASA acts like a smart manager who identifies which workers are prone to errors and specifically trains them to ensure they don't make mistakes when handling complex tasks. Meanwhile, RASA ensures that every worker follows the established process to avoid any unexpected errors.
ELI14 Explained like you're 14
Hey, imagine you're playing a game where your character has lots of skills to defeat enemies. But sometimes, your character uses the wrong skill, leading to defeat. RASA is like a super coach that helps your character identify which skills are prone to errors and specifically trains these skills to ensure no mistakes during critical moments. This way, you can better defeat enemies and win the game!
Glossary
Mixture-of-Experts Model
A model that assigns input to multiple experts through a sparse routing mechanism.
Used in this paper to improve model scalability and efficiency.
Safety-Critical Experts
Experts activated during jailbreak attacks that may lead to unsafe behaviors.
RASA identifies and repairs these experts to enhance model safety.
Routing Consistency
Ensuring that jailbreak inputs follow the same routing patterns as safety-aligned inputs.
RASA optimizes routing consistency to prevent shortcut solutions.
Alignment Shortcut
Satisfying safety objectives through routing or expert dominance effects without meaningful expert-level repair.
A common issue in traditional methods, avoided by RASA through expert-level repair.
Jailbreak Attack
An attack method that bypasses model safety mechanisms through specific inputs.
Used to test RASA's robustness under different attack scenarios.
Open Questions Unanswered questions from this research
- 1 How to validate RASA's effectiveness on larger-scale MoE models?
- 2 Are there more effective adversarial sample generation methods to enhance RASA's performance?
Applications
Immediate Applications
Financial Security
RASA can be used in financial systems to prevent security breaches caused by malicious inputs.
Long-term Vision
Healthcare Data Protection
In healthcare, RASA can protect sensitive data from breaches.
Abstract
Mixture-of-Experts (MoE) language models introduce unique challenges for safety alignment due to their sparse routing mechanisms, which can enable degenerate optimization behaviors under standard full-parameter fine-tuning. In our preliminary experiments, we observe that naively applying full-parameter safety fine-tuning to MoE models can reduce attack success rates through routing or expert dominance effects, rather than by directly repairing Safety-Critical Experts. To address this challenge, we propose RASA, a routing-aware expert-level alignment framework that explicitly repairs Safety-Critical Experts while preventing routing-based bypasses. RASA identifies experts disproportionately activated by successful jailbreaks, selectively fine-tunes only these experts under fixed routing, and subsequently enforces routing consistency with safety-aligned contexts. Across two representative MoE architectures and a diverse set of jailbreak attacks, RASA achieves near-perfect robustness, strong cross-attack generalization, and substantially reduced over-refusal, while preserving general capabilities on benchmarks such as MMLU, GSM8K, and TruthfulQA. Our results suggest that robust MoE safety alignment benefits from targeted expert repair rather than global parameter updates, offering a practical and architecture-preserving alternative to prior approaches.