CAD: Conflict-Aware Decoding to Mitigate Cross-Modal Hallucinations in Omnimodal Large Language Models
CAD reduces cross-modal hallucinations with PCME and CAA, improving CMM and AVHBench accuracy by 14.1% and 8.0%.
Key Findings
Methodology
CAD is a training-free decoding framework comprising Potential Conflict Magnitude Estimation (PCME) and Conflict Actionability Assessment (CAA). PCME quantifies potential conflict using audio-video disagreement and joint prediction deviation. CAA applies Dempster-Shafer theory to task-space answer relations to determine intervention necessity.
Key Results
- CAD improves CMM accuracy by 14.1 percentage points and AVHBench by 8.0 points on Qwen2.5-Omni-7B without retraining.
- On OmniVinci, CAD increases accuracy by 4.1 and 6.9 points on CMM and AVHBench, respectively.
- Experiments show CAD consistently outperforms baseline decoders and competitive training-free methods across multiple audio-visual backbones.
Significance
CAD addresses the cross-modal hallucination issue in Omni-LLMs, significantly enhancing accuracy and stability in audio-visual tasks. It holds broad potential for academia and industry, especially in complex scenarios requiring reliable multimodal fusion.
Technical Contribution
CAD distinguishes modality relevance from predictive compatibility, offering a new training-free decoding framework with theoretical guarantees and engineering possibilities, markedly different from existing SOTA methods.
Novelty
CAD is the first to apply Dempster-Shafer theory to cross-modal conflict assessment, providing a more reliable intervention mechanism than existing methods.
Limitations
- CAD may fail to recognize all beneficial cross-modal complementarities, potentially leading to information loss.
- Further research is needed to apply CAD in more complex multimodal scenarios.
Future Work
Future work could explore CAD's application in other multimodal tasks like augmented reality and autonomous driving, optimizing its performance on larger-scale models.
AI Executive Summary
Cross-modal hallucinations pose a significant challenge for Omni-LLMs, affecting their reliability in practical applications. Existing decoding methods fail to effectively assess predictive compatibility within the joint audio-visual branch, leading to potential harmful interference during information fusion. To address this, the paper proposes the Conflict-Aware Decoding (CAD) framework, which quantifies and evaluates cross-modal conflicts through Potential Conflict Magnitude Estimation (PCME) and Conflict Actionability Assessment (CAA). Experimental results demonstrate that CAD significantly improves accuracy across multiple audio-visual backbones, particularly on the CMM and AVHBench datasets. CAD's innovation lies in its training-free decoding mechanism, reallocating decoding weights through reliable intervention signals to reduce unreliable fusion. While CAD may not recognize all beneficial complementarities in some cases, its potential for application in cross-modal tasks is substantial, with future research focusing on optimizing its performance.
Deep Analysis
Background
In recent years, multimodal large language models (MLLMs) have evolved from static image-text understanding to multimodal reasoning in complex audio-visual scenarios. Conventional MLLMs typically process different modalities independently, whereas Omni-LLMs project text, image, video, and audio features into a unified representation space, enabling tighter cross-modal interaction and joint audio-visual reasoning.
Core Problem
Cross-modal hallucinations are a core issue for Omni-LLMs, affecting their stability in practical applications. Existing decoding methods fail to effectively assess predictive compatibility within the joint audio-visual branch, leading to potential harmful interference during information fusion.
Innovation
CAD quantifies and evaluates cross-modal conflicts through Potential Conflict Magnitude Estimation (PCME) and Conflict Actionability Assessment (CAA). PCME quantifies audio-video disagreement and joint prediction deviation, while CAA applies Dempster-Shafer theory to task-space answer relations.
Methodology
- �� Potential Conflict Magnitude Estimation (PCME) quantifies potential conflict using audio-video disagreement and joint prediction deviation.
- �� Conflict Actionability Assessment (CAA) applies Dempster-Shafer theory to task-space answer relations to determine intervention necessity.
- �� When an actionable conflict is identified, CAD selectively reallocates decoding weight from the joint branch to the unimodal branches.
Experiments
Experiments are conducted on CMM, AVHBench, WorldSense, and VideoMME datasets, evaluating CAD's performance across multiple audio-visual backbones. Tests are performed using Qwen2.5-Omni-7B and OmniVinci, comparing CAD with baseline decoders and competitive training-free methods.
Results
CAD improves CMM accuracy by 14.1 percentage points and AVHBench by 8.0 points on Qwen2.5-Omni-7B without retraining. On OmniVinci, CAD increases accuracy by 4.1 and 6.9 points on CMM and AVHBench, respectively.
Applications
CAD holds broad potential for application in complex scenarios requiring reliable multimodal fusion, such as audio-visual scene understanding and interactive assistance.
Limitations & Outlook
CAD may fail to recognize all beneficial cross-modal complementarities, potentially leading to information loss. Further research is needed to apply CAD in more complex multimodal scenarios.
Plain Language Accessible to non-experts
Imagine a kitchen where a chef needs to combine different ingredients to create a delicious dish. Each ingredient has its unique flavor, but if mixed incorrectly, it might result in a bad taste. CAD acts like a smart chef, identifying which combinations will result in a tasty dish and which need adjustment. By evaluating conflicts and complementarities between ingredients, CAD ensures the final dish is flavorful.
ELI14 Explained like you're 14
Hey, imagine you're playing a super cool game where you need to combine different skills to defeat enemies. Sometimes, you might use the wrong skills and fail. CAD is like a smart game assistant that helps you choose the best skill combinations, ensuring you win the game. It evaluates conflicts and complementarities between skills, making sure you always make the right choice.
Glossary
Omni-LLMs
Language models integrating audio, video, and text for multimodal reasoning.
Used to evaluate cross-modal hallucination impacts.
PCME
Mechanism quantifying audio-video disagreement and joint prediction deviation.
Used to identify potential cross-modal conflicts.
CAA
Mechanism applying Dempster-Shafer theory to task-space answer relations.
Used to determine intervention necessity.
Dempster-Shafer Theory
A theory for handling uncertainty and evidence combination.
Applied in conflict actionability assessment.
Cross-modal Hallucination
Phenomenon where one modality improperly influences another's prediction.
Affects Omni-LLMs' stability.
Open Questions Unanswered questions from this research
- 1 How to apply CAD in more complex multimodal scenarios remains to be explored.
- 2 CAD may fail to recognize all beneficial complementarities.
Applications
Immediate Applications
Audio-Visual Scene Understanding
CAD can enhance accuracy in audio-visual scene understanding, especially in complex tasks requiring reliable multimodal fusion.
Long-term Vision
Augmented Reality
CAD can be applied in augmented reality, improving user experience through reliable multimodal fusion.
Abstract
Omnimodal large language models (Omni-LLMs) integrate audio, video, and text, yet remain vulnerable to cross-modal hallucinations, where one modality improperly influences predictions about another. Existing training-free decoders modulate modality influence through perturbation or relevance weighting, but do not assess predictive compatibility within the joint audio-visual branch. Because joint-branch discrepancies may indicate either harmful interference or useful complementarity, reliable intervention requires assessing both discrepancy magnitude and actionability. To this end, we propose Conflict-Aware Decoding (CAD), a training-free framework comprising Potential Conflict Magnitude Estimation (PCME) and Conflict Actionability Assessment (CAA). PCME quantifies potential conflict using audio-video disagreement and the deviation of the joint prediction from a relevance-weighted unimodal reference. CAA then applies Dempster-Shafer reliability discounting to task-space answer relations, using query relevance and answer decisiveness to determine whether intervention is warranted. When an actionable conflict is identified, CAD selectively reallocates decoding weight from the joint branch to the unimodal branches. Experiments on CMM, AVHBench, WorldSense, and VideoMME show that CAD consistently outperforms the base decoder and competitive training-free methods across multiple audio-visual backbones. On Qwen2.5-Omni-7B, CAD improves overall accuracy by 14.1 and 8.0 percentage points on CMM and AVHBench, respectively, without model retraining.