GateBreaker: Gate-Guided Attacks on Mixture-of-Expert LLMs
GateBreaker is a training-free, lightweight attack framework targeting MoE LLMs, increasing attack success rate from 7.4% to 64.9%.
Key Findings
Methodology
GateBreaker employs a three-stage process: first, gate-level profiling identifies experts disproportionately activated by harmful prompts; second, expert-level localization pinpoints safety neurons by analyzing activation differences between harmful and benign inputs; third, during inference, targeted masking of these neurons weakens safety mechanisms. This approach requires no retraining, relies solely on activation analysis, and demonstrates high transferability across models. Experiments on eight state-of-the-art MoE models show an increase in attack success rate from 7.4% to 64.9%, with minimal utility loss. Safety neurons are shared across models, enabling one-shot transfer attacks, raising success from 17.9% to 67.7%. The framework also extends to vision-language models, achieving 60.9% success with unsafe images.
Key Results
- In eight top MoE models, average attack success rate (ASR) increased from 7.4% to 64.9%, modifying less than 3% of neurons per layer.
- Safety neurons exhibit high transferability, boosting attack success from 17.9% to 67.7% across models within the same family.
- Applied to five vision-language models with unsafe images, ASR reached 60.9%, demonstrating cross-modal attack potential.
Significance
This work exposes fundamental vulnerabilities in MoE architectures' safety mechanisms, challenging current defense methods. The inference-only, training-free approach underscores the ease of exploiting sparse expert pathways, raising concerns for real-world deployment. It emphasizes the need for more robust safety designs that account for the distributed nature of safety features, influencing future model development and security standards across AI applications.
Technical Contribution
GateBreaker introduces a novel, architecture-agnostic attack pipeline based on gate analysis, expert neuron localization, and targeted masking. It operates solely during inference, requiring no model retraining, and reveals the concentration of safety functions within sparse expert neurons. This approach provides a new perspective on model security, enabling efficient, transferable attacks and informing the design of more resilient architectures.
Novelty
This is the first comprehensive study revealing the distribution of safety mechanisms in MoE models and demonstrating an inference-only attack framework. Unlike prior work focusing on dense models or coarse expert-level attacks, it performs fine-grained neuron-level interference within sparse experts, exposing a critical security gap in modern large models.
Limitations
- The attack relies on internal access to gating logits and expert activations, which may be restricted in commercial or protected models.
- Effectiveness may decrease in models with complex or proprietary routing mechanisms.
- Current evaluation is limited to single-turn prompts; multi-turn or interactive scenarios require further research.
Future Work
Future directions include developing adaptive, multi-turn attack strategies, integrating with real-time detection systems, and designing inherently robust safety mechanisms that prevent such targeted interference, ultimately fostering safer large-scale AI systems.
AI Executive Summary
The rapid growth of large language models (LLMs) has revolutionized natural language processing, powering applications from chatbots to scientific research. However, ensuring their safety remains a critical challenge. Existing safety mechanisms often rely on post-training alignment techniques like reinforcement learning from human feedback (RLHF), which are insufficient against sophisticated attacks. In architectures like Mixture-of-Experts (MoE), safety features are distributed across sparse expert pathways, making them inherently fragile and vulnerable.
This paper introduces GateBreaker, a novel, inference-only attack framework that exploits the distribution of safety mechanisms within MoE models. The framework operates in three stages: first, gate-level profiling identifies experts disproportionately activated during harmful prompts; second, expert-level localization pinpoints safety neurons by analyzing activation differences between harmful and benign inputs; third, during inference, targeted masking of these neurons effectively disables safety features. This approach does not require retraining or model modification, making it highly practical.
Extensive experiments across eight state-of-the-art MoE models demonstrate that GateBreaker significantly increases attack success rates from 7.4% to 64.9%, with modifications affecting less than 3% of neurons per layer. The identified safety neurons are highly transferable within model families, enabling one-shot attacks that raise success rates to over 67%. The framework also extends to vision-language models, achieving 60.9% success with unsafe images. These findings reveal a critical vulnerability: safety mechanisms are concentrated in sparse, shared neurons, which can be targeted efficiently.
The implications are profound: as MoE models become prevalent in sensitive applications, their safety features may be more fragile than previously thought. This work urges the community to rethink safety design, emphasizing the need for more robust, distributed, and tamper-resistant safety mechanisms. While current results focus on single-turn prompts, future research should explore multi-turn scenarios, adaptive defenses, and integrated security solutions to build truly resilient AI systems.
Deep Analysis
Background
近年来，LLMs在自然语言理解和生成方面取得巨大突破，代表作如GPT-4、PaLM等。为了提升模型容量与效率，MoE架构被广泛采用，通过稀疏激活机制实现大规模参数的高效利用。早期研究如Shazeer等提出稀疏专家路由，显著降低计算成本，但安全性问题未充分关注。随着模型规模扩大，稀疏路径碎片化导致安全机制分散，潜在风险增加。现有安全对齐多依赖微调和强化学习，效果有限，易被绕过。
Core Problem
MoE模型的安全机制分散在不同专家中，依赖稀疏激活路径，导致有害内容可能绕过检测。传统防护难以应对路径的动态变化，攻击者可利用激活特性设计针对性攻击，威胁模型可靠性。现有研究缺乏细粒度理解与攻击手段，限制了安全防护效果。
Innovation
提出GateBreaker，无训练、推理时的攻击框架。创新点包括：1）门控层分析识别高风险专家，2）专家层面定位安全神经元，3）遮蔽关键神经元破坏安全机制。架构无关，适用多模型，且仅需少量修改，极大降低成本。
Methodology
- �� 采集有害提示，分析门控激活，识别高激活专家；• 计算专家激活频率，统计与有害内容的关联；• 在专家层面定位安全神经元，比较激活差异；• 统计筛选关键神经元；• 在推理中遮蔽神经元，削弱安全机制；• 评估攻击成功率与性能影响。
Experiments
在八个主流MoE模型上，使用有害提示集和正常提示集，测量ASR和模型性能。对比不同遮蔽策略，分析安全神经元迁移性。扩展到五个视觉模型，利用不安全图片测试效果，参数设置确保结果稳健。
Results
平均ASR从7.4%提升至64.9%，仅修改约2.9%的神经元。迁移实验显示安全神经元在模型家族内高度共享，成功率达67.7%。在视觉模型中，利用不安全图片实现60.9%的成功率，验证跨模态攻击潜力。
Applications
可用于模型安全评估，揭示潜在漏洞，优化安全设计。行业中用于内容过滤、审查，提升安全防护。未来结合多轮对话和微调，增强隐蔽性和鲁棒性。
Limitations & Outlook
依赖模型内部门控信息，封装模型可能限制应用。复杂路由机制可能降低效果。仅验证单轮场景，多轮交互鲁棒性待验证。未来需研究更隐蔽的攻击与防御策略。
Plain Language Accessible to non-experts
想象一个工厂里有许多不同的机器（专家），每个负责不同任务。为了节省能源，只开启部分机器处理订单（输入）。有些机器负责检测产品质量（安全专家），当检测到问题时会发出警报。这个工厂的安全机制就像这些检测机器一样，隐藏在某些特定的机器中。攻击者像是偷偷找到这些检测机器，把它们的警报关掉，让工厂继续生产有缺陷的产品（有害内容）。他们不用改动整个工厂，只是在关键的检测机器上做手脚，就能让系统变得不安全。这就像用一把钥匙关闭了工厂的安全门，外面的人就可以随意闯入，造成破坏。
ELI14 Explained like you're 14
想象你在学校，有很多老师（专家），每个负责不同科目。有些老师特别严格，负责检查作业（安全专家），当学生做错事时会批评他们。现在，有个调皮的学生（攻击者）想偷偷溜过老师的检查，不让老师发现自己做错了。于是，他偷偷找到那些负责检查的老师，把他们的警示信号关掉，这样老师就不会发现他的错误了。这个学生不用改变整个学校，只是在关键的老师那里做手脚，就能让自己逃过检查。这就像用一把钥匙关闭了学校的安全门，让坏事更容易发生。
Abstract
Mixture-of-Experts (MoE) architectures have advanced the scaling of Large Language Models (LLMs) by activating only a sparse subset of parameters per input, enabling state-of-the-art performance with reduced computational cost. As these models are increasingly deployed in critical domains, understanding and strengthening their alignment mechanisms is essential to prevent harmful outputs. However, existing LLM safety research has focused almost exclusively on dense architectures, leaving the unique safety properties of MoEs largely unexamined. The modular, sparsely-activated design of MoEs suggests that safety mechanisms may operate differently than in dense models, raising questions about their robustness. In this paper, we present GateBreaker, the first training-free, lightweight, and architecture-agnostic attack framework that compromises the safety alignment of modern MoE LLMs at inference time. GateBreaker operates in three stages: (i) gate-level profiling, which identifies safety experts disproportionately routed on harmful inputs, (ii) expert-level localization, which localizes the safety structure within safety experts, and (iii) targeted safety removal, which disables the identified safety structure to compromise the safety alignment. Our study shows that MoE safety concentrates within a small subset of neurons coordinated by sparse routing. Selective disabling of these neurons, approximately 3% of neurons in the targeted expert layers, significantly increases the averaged attack success rate (ASR) from 7.4% to 64.9% against the eight latest aligned MoE LLMs with limited utility degradation. These safety neurons transfer across models within the same family, raising ASR from 17.9% to 67.7% with one-shot transfer attack. Furthermore, GateBreaker generalizes to five MoE vision language models (VLMs) with 60.9% ASR on unsafe image inputs.