HallusionBench: An Advanced Diagnostic Suite for Entangled Language Hallucination and Visual Illusion in Large Vision-Language Models
HallusionBench is a diagnostic benchmark for entangled language hallucination and visual illusion in large vision-language models, evaluating their reasoning robustness.
Key Findings
Methodology
This study introduces HallusionBench, comprising 346 images and 1129 expert-crafted questions, split into visual-dependent and supplement types. The design employs control groups with original and edited images to analyze response biases, logical consistency, and failure modes. Evaluation across 15 models shows GPT-4V achieves 31.42% question-pair accuracy, outperforming others below 16%. Quantitative analysis reveals dominant language bias and visual misinterpretations, highlighting the need for improved visual understanding. Case studies dissect failure cases, emphasizing challenges in complex reasoning scenarios.
Key Results
- GPT-4V attains 31.42% question-pair accuracy, significantly higher than other models (<16%), indicating the difficulty of complex visual reasoning tasks.
- Most models exhibit language bias, often answering affirmatively without visual input, demonstrating prevalent language hallucination issues.
- Visual illusions frequently occur when models misinterpret image details, pointing to the ongoing challenge of visual feature extraction and comprehension.
Significance
This work pioneers a systematic quantification of hallucination and illusion phenomena in LVLMs, providing crucial insights into their failure mechanisms. It bridges the gap between accuracy metrics and failure analysis, guiding future robustness improvements. The benchmark’s detailed diagnostics foster understanding of model limitations in real-world applications, such as medical diagnostics and autonomous systems, where reliability is critical. By revealing the roots of model errors, it paves the way for designing more trustworthy multimodal AI systems, advancing both academic research and industry deployment.
Technical Contribution
The paper introduces a novel diagnostic framework combining control group design with quantitative failure metrics, enabling fine-grained analysis of hallucination types. It innovates by categorizing failures into language hallucination and visual illusion, leveraging human-edited samples for robustness testing. The approach enhances interpretability, facilitates targeted model improvements, and sets a new standard for systematic failure diagnosis in multimodal models.
Novelty
This is the first benchmark explicitly targeting both language hallucination and visual illusion in LVLMs, employing human-crafted control pairs and detailed failure categorization. Unlike prior datasets focusing solely on object hallucination, HallusionBench emphasizes complex reasoning failures, filling a critical gap in evaluation tools. Its methodology enables precise failure mode analysis, offering a new paradigm for diagnosing and improving multimodal models.
Limitations
- The benchmark primarily targets specific hallucination and illusion types, leaving other failure modes underexplored. Its scope may not cover all real-world complexities.
- Reliance on GPT-4 for evaluation introduces potential biases; future work should incorporate human validation for more reliable assessments.
- Image editing strategies, though diverse, are limited in scope; more sophisticated manipulations are needed to simulate real-world visual disturbances.
Future Work
Future directions include expanding dataset diversity to cover more complex scenarios, integrating multi-modal training to reduce hallucinations, and developing explainability tools to interpret failure cases. Enhancing the benchmark with real-world noisy data and extending to video modalities will further improve model robustness and safety, fostering trustworthy AI deployment in critical applications.
AI Executive Summary
Recent advances in large vision-language models (LVLMs) like GPT-4V and LLaVA have unlocked impressive capabilities in multimodal understanding, enabling applications from image captioning to complex reasoning. However, these models are plagued by hallucination and illusion phenomena—where they generate false information or misinterpret visual inputs—posing significant risks for real-world deployment. Existing evaluation metrics mainly focus on accuracy, neglecting the nuanced failure mechanisms that undermine model reliability.
To address this, we introduce HallusionBench, a comprehensive diagnostic benchmark designed to systematically analyze the failure modes of LVLMs. Comprising 346 images and 1129 carefully crafted questions, the benchmark covers diverse topics and formats, including charts, maps, and manipulated images. The questions are structured into visual-dependent and supplement categories, with human experts editing images to create control pairs. This design allows us to quantify biases, logical consistency, and failure types, distinguishing between language hallucination—where models rely on prior knowledge without visual grounding—and visual illusion—where models misinterpret visual details.
Evaluation of 15 prominent LVLMs reveals that GPT-4V achieves a question-pair accuracy of 31.42%, far surpassing other models below 16%. Nonetheless, the results highlight widespread issues: models tend to over-rely on language priors, often answering affirmatively without visual evidence, and frequently misinterpret complex visual cues. Case studies illustrate these failure modes, emphasizing the importance of targeted diagnostics for robustness.
This work significantly advances the understanding of multimodal model limitations, providing a foundation for future improvements. By dissecting failure mechanisms with fine-grained metrics, it guides researchers toward developing more reliable, bias-resistant models suitable for critical applications. The benchmark’s insights will inform the design of next-generation multimodal AI systems, fostering safer and more trustworthy deployment in real-world scenarios.
Deep Analysis
Background
The evolution of multimodal learning has seen rapid progress with models like Flamingo, PaLM-E, and recent LVLMs such as GPT-4V and LLaVA. These models integrate visual and textual data, enabling tasks like visual question answering, captioning, and reasoning. Early works focused on object detection and description, but recent trends emphasize complex reasoning and contextual understanding. Despite these advances, models frequently suffer from hallucinations—producing plausible but false information—and visual illusions—misinterpreting visual cues—especially in challenging scenarios. Existing benchmarks like VQA and MMBench evaluate basic capabilities but lack diagnostic depth for failure modes. As models are increasingly deployed in sensitive domains, understanding their limitations becomes critical. This research aims to fill this gap by providing a systematic diagnostic framework that dissects hallucination and illusion phenomena, offering insights into their root causes and pathways for mitigation.
Core Problem
Despite significant progress, LVLMs still exhibit critical failure modes such as language hallucination—where models generate responses based on prior knowledge rather than visual evidence—and visual illusion—where visual details are misinterpreted. These issues undermine trustworthiness, especially in high-stakes applications like medical diagnostics or autonomous navigation. Current evaluation methods lack the granularity to diagnose these failures systematically, making it difficult to improve model robustness. The core challenge lies in disentangling whether errors stem from over-reliance on language priors or from visual feature misinterpretation. Addressing these problems requires a diagnostic tool capable of isolating and quantifying different failure types, guiding targeted model improvements.
Innovation
This work introduces HallusionBench, a novel diagnostic benchmark combining control group design with expert-edited samples to analyze hallucination and illusion in LVLMs. Key innovations include: • Structuring questions into visual-dependent and supplement categories, enabling targeted failure analysis. • Employing human-edited images to simulate real-world manipulations and test model robustness. • Developing quantitative metrics such as question-pair accuracy, bias difference, and failure mode classification. • Using GPT-4 as an automated evaluator, ensuring consistent and scalable assessment. These innovations facilitate a detailed understanding of how models fail, providing actionable insights for future development.
Methodology
- �� Data collection: 346 images covering diverse topics and formats, with 1129 question-answer pairs designed by experts.
- �� Control group setup: each question linked to original and edited images, allowing response comparison.
- �� Failure diagnosis: categorizing errors into language hallucination and visual illusion via a decision tree.
- �� Quantitative evaluation: measuring question-pair accuracy, bias, and consistency.
- �� Model assessment: testing 15 LVLMs, including GPT-4V, LLaVA-1.5, and others, across various question types.
- �� Case analysis: detailed review of failure instances to identify common patterns and root causes.
Experiments
- �� Dataset: diverse images with manual edits, covering charts, maps, videos, and illusions.
- �� Baselines: state-of-the-art LVLMs, including GPT-4V, Gemini Pro Vision, and open-source models.
- �� Metrics: question-pair accuracy, bias difference, logical consistency, failure mode classification.
- �� Protocol: multi-round GPT-4 evaluation, supplemented by human validation.
- �� Ablation studies: testing the impact of control group design and image editing strategies on model performance.
Results
- �� GPT-4V outperforms others with 31.42% question-pair accuracy, yet all models struggle with complex visual reasoning.
- �� Widespread language bias observed, with models often answering affirmatively without visual grounding.
- �� Visual misinterpretations are frequent in manipulated images, indicating persistent visual understanding gaps.
- �� Failure analysis reveals that models tend to over-rely on prior knowledge, leading to hallucinations, especially in ambiguous or manipulated scenes.
Applications
- �� Critical sectors like healthcare, autonomous driving, and security can leverage diagnostic insights to improve model reliability.
- �� Researchers can use the benchmark to develop more robust, bias-resistant multimodal models.
- �� Industry applications require models that can accurately interpret complex visual data under real-world conditions, reducing risks of false positives or negatives.
Limitations & Outlook
- �� The benchmark focuses on specific hallucination and illusion types, not covering all possible failure modes.
- �� Dependence on GPT-4 evaluation may introduce bias; future work should incorporate human assessments.
- �� Image manipulations, while diverse, are still limited; more realistic distortions are needed to simulate real-world scenarios.
Plain Language Accessible to non-experts
想象你在一个厨房里做饭。每次做菜都需要准备食材、调味料和厨具。有时候，厨房里会出现误会，比如你以为锅里有菜，但其实是空的，或者误以为调料放多了。这就像AI模型中的幻觉和错觉——幻觉就像你自己猜测锅里有菜，但实际上没有；错觉则是你看到的菜其实是错的，比如误认了颜色或形状。厨师们会设计各种测试，比如用不同的食材或调料，看看厨房里的厨具是否会出错。这个研究就像是在检测和修理AI厨房，确保每一道菜都正确无误，不会出现虚假的信息或误解。
Abstract
We introduce HallusionBench, a comprehensive benchmark designed for the evaluation of image-context reasoning. This benchmark presents significant challenges to advanced large visual-language models (LVLMs), such as GPT-4V(Vision), Gemini Pro Vision, Claude 3, and LLaVA-1.5, by emphasizing nuanced understanding and interpretation of visual data. The benchmark comprises 346 images paired with 1129 questions, all meticulously crafted by human experts. We introduce a novel structure for these visual questions designed to establish control groups. This structure enables us to conduct a quantitative analysis of the models' response tendencies, logical consistency, and various failure modes. In our evaluation on HallusionBench, we benchmarked 15 different models, highlighting a 31.42% question-pair accuracy achieved by the state-of-the-art GPT-4V. Notably, all other evaluated models achieve accuracy below 16%. Moreover, our analysis not only highlights the observed failure modes, including language hallucination and visual illusion, but also deepens an understanding of these pitfalls. Our comprehensive case studies within HallusionBench shed light on the challenges of hallucination and illusion in LVLMs. Based on these insights, we suggest potential pathways for their future improvement. The benchmark and codebase can be accessed at https://github.com/tianyi-lab/HallusionBench.