Transfer Safety Awareness for Cross-Modal Safety Drift in Multimodal Large Language Models
Proposed SRT method improves cross-modal safety by transferring safety awareness; experiments show enhanced safety across benchmarks.
Key Findings
Methodology
The study introduces a lightweight direction-refinement method called Safety-awareness Representation Transfer (SRT) to mitigate cross-modal safety drift with a frozen MLLM backbone. By analyzing model activations and attentions, it identifies limited attention to visual risk cues and proposes transferring safety signals from unsafe text processing to cross-modal contexts.
Key Results
- Experiments show SRT effectively improves safety across cross-modal settings; for Qwen3-VL-8B, unsafe generation rate in StUi reduced from 43.10% to 23.10%.
- In Gemma-3-12B-IT, SRT reduced unsafe rate in StSi from 68.86% to 23.35%.
- In LLaVA-OneVision-7B, SRT reduced FigStep benchmark unsafe rate from 77.86% to 34.64%.
Significance
The study systematically addresses cross-modal safety drift, and the proposed SRT method not only enhances model safety but also preserves utility. This offers new insights into balancing safety and utility in multimodal large language models, with significant academic and industrial impact.
Technical Contribution
SRT improves cross-modal safety by transferring unsafe-text-induced safety awareness, significantly enhancing safety while maintaining model utility compared to existing methods. Additionally, it offers a lightweight solution without modifying the model backbone.
Novelty
This study introduces the concept of cross-modal safety drift and develops the SRT method to address it. Unlike existing safety calibration methods, SRT enhances safety by transferring safety signals induced by unsafe text.
Limitations
- SRT may perform poorly in handling complex visual risk cues, as these cues may be weakened in the visual modality.
- The method relies on the presence of unsafe text to activate safety signals, potentially limited in pure visual inputs.
Future Work
Future research could explore activating safety signals without unsafe text and further improving risk recognition in the visual modality.
AI Executive Summary
Multimodal large language models excel at understanding and reasoning over text-image inputs, but they also introduce new safety risks. Existing methods struggle with cross-modal safety drift, especially when text and image combinations imply harmful intent.
The study proposes a method called Safety-awareness Representation Transfer (SRT), which improves cross-modal safety by transferring safety signals induced by unsafe text. Experiments show that SRT significantly enhances safety across benchmarks while preserving model utility.
This method holds significant academic importance and offers a new solution for the industry, helping multimodal large language models balance safety and utility. Future research could further explore activating safety signals without unsafe text.
Deep Analysis
Background
Multimodal large language models can process text and image inputs, significantly enhancing understanding and reasoning capabilities. However, this enhancement also introduces new safety risks, especially when text and image combinations imply harmful intent. Existing safety calibration methods perform poorly in addressing cross-modal safety drift.
Core Problem
Cross-modal safety drift refers to the issue where safe text may convey harmful intent when combined with unsafe images. Existing methods perform poorly in addressing this issue, especially when text and image combinations imply harmful intent.
Innovation
The study proposes a method called Safety-awareness Representation Transfer (SRT), which improves cross-modal safety by transferring safety signals induced by unsafe text. This method offers a lightweight solution without modifying the model backbone.
Methodology
- �� Analyze model activations and attentions to identify limited attention to visual risk cues.
- �� Propose Safety-awareness Representation Transfer (SRT) to mitigate cross-modal safety drift with a frozen MLLM backbone.
- �� Validate SRT's effectiveness across cross-modal settings through experiments.
Experiments
Experiments used multiple benchmarks, including VLSBench, MM-SafetyBench, FigStep, etc. By comparing unsafe generation rates, SRT's effectiveness across cross-modal settings was validated.
Results
Experimental results show SRT effectively improves safety across cross-modal settings while preserving model utility. Specifically, for Qwen3-VL-8B, the unsafe generation rate in StUi reduced from 43.10% to 23.10%.
Applications
SRT can be applied to safety calibration in multimodal large language models, especially when handling text and image combinations that imply harmful intent.
Limitations & Outlook
SRT may perform poorly in handling complex visual risk cues, as these cues may be weakened in the visual modality. Additionally, the method relies on the presence of unsafe text to activate safety signals, potentially limited in pure visual inputs.
Plain Language Accessible to non-experts
Imagine a kitchen where a chef needs to cook based on ingredients and recipes. Multimodal large language models are like this chef, with text and images as ingredients and recipes. The safety drift problem is like the chef ignoring certain allergenic ingredients while cooking. The SRT method is like an assistant reminding the chef to pay attention to allergenic ingredients, ensuring the dish is safe.
ELI14 Explained like you're 14
Imagine you're playing a game where your character needs to complete tasks based on hints. Multimodal large language models are like this character, with text and images as hints. The safety drift problem is like the character ignoring certain dangerous hints during tasks. The SRT method is like an assistant reminding the character to pay attention to dangerous hints, ensuring the task is completed safely.
Glossary
Cross-modal safety drift
Refers to the issue where safe text may convey harmful intent when combined with unsafe images.
Used in the study to describe safety risks introduced by visual modality.
Safety-awareness Representation Transfer (SRT)
A method to improve cross-modal safety by transferring safety signals induced by unsafe text.
Used to mitigate cross-modal safety drift.
Visual modality
Refers to the model's ability to process image inputs.
Used in the study to describe multimodal large language model capabilities.
Unsafe generation rate
Refers to the proportion of unsafe outputs generated by the model when processing inputs.
Used to evaluate model safety.
Multimodal large language models (MLLM)
Large language models capable of processing text and image inputs.
Used in the study to describe model capabilities.
Open Questions Unanswered questions from this research
- 1 How to activate safety signals without unsafe text remains an open question.
- 2 Improving recognition of complex visual risk cues needs further exploration.
Applications
Immediate Applications
Multimodal safety calibration
SRT can be applied to safety calibration in multimodal large language models, ensuring safety when processing text and image combinations.
Long-term Vision
Visual risk recognition
Future research could explore improving risk recognition in the visual modality to further enhance model safety.
Abstract
Visual modality enhances the capabilities of multimodal large language models (MLLMs) but also introduces a safety concern: a benign textual query may convey harmful intent when grounded in a visual image. We term this cross-modal safety drift and our pilot studies show that the safety response rate for such requests is substantially lower than that for requests containing explicitly unsafe text. This paper aims to systematically study this issue. First, we conduct an empirical analysis to identify representative unsafe response patterns. Building on these, we interpret model representations and attentions, revealing that visually risky cues receive limited attention and weakly trigger refusal. Motivated by the observation that safety signals from unsafe text processing can be transferred, we propose safety-awareness representation transfer (SRT), a lightweight direction-refinement method that mitigates cross-modal safety drift with a frozen MLLM backbone. Experiments across multiple benchmarks and models show that SRT effectively improves safety in diverse cross-modal settings while preserving utility. Code is available at https://github.com/cucu220123/safety-awareness.