iVGR: Internalizing Visually Grounded Reasoning for MLLMs with Reinforcement Learning
iVGR internalizes visual localization into textual reasoning using reinforcement learning, significantly enhancing MLLMs' performance.
Key Findings
Methodology
iVGR employs a dual-stream training strategy, internalizing visual localization into textual reasoning through a consistency reward. The textual stream aligns with a high-quality visual stream, allowing accurate localization without explicit grounding during inference. This method uses a reinforcement learning framework, combining format, accuracy, and consistency rewards to ensure alignment between textual reasoning and visual references.
Key Results
- On the HR8K benchmark, the iVGR-Qwen3-VL-32B model achieved 82.9% accuracy, improving by 4.8% over Qwen3-VL-32B.
- On the V* dataset, the iVGR-Qwen2.5-VL-7B model reached 86.4% accuracy, a 7.9% increase over the baseline.
- In multimodal reasoning tasks, iVGR shows outstanding performance across multiple benchmarks, demonstrating its broad applicability.
Significance
iVGR addresses the challenge of processing high-resolution images in MLLMs by internalizing visual localization, significantly improving reasoning accuracy. This research provides new insights into multimodal reasoning, promoting deep integration of visual and textual information with substantial academic and practical value.
Technical Contribution
iVGR introduces an innovative dual-stream training strategy, transferring visual localization into textual reasoning through a consistency reward, avoiding interference from explicit grounding. This method achieves high-precision reasoning without relying on external tools, offering new engineering possibilities.
Novelty
iVGR is the first to internalize visual localization into textual reasoning, distinguishing itself from traditional explicit grounding methods by reducing inference interference and improving reasoning efficiency.
Limitations
- In complex scenarios, the model may not fully internalize all visual information, leading to decreased reasoning accuracy.
- Requires substantial training data and computational resources for optimal performance.
Future Work
Future work could explore applying iVGR on larger datasets and investigate its performance in other multimodal tasks. Additionally, further optimization of the consistency reward mechanism could enhance the model's generalization capabilities.
AI Executive Summary
In recent years, multimodal large language models (MLLMs) have made significant progress in integrating visual and textual information. However, existing methods still face challenges when processing high-resolution images, particularly requiring explicit visual localization during inference. To address this issue, researchers have proposed a new framework called iVGR, which internalizes visual localization into textual reasoning using reinforcement learning.
iVGR employs a dual-stream training strategy, aligning the textual stream with a high-quality visual stream through a consistency reward, ensuring accurate localization without explicit grounding during inference. Experimental results show that iVGR significantly outperforms existing methods across multiple benchmarks, particularly excelling in fine-grained tasks.
This research not only enhances the reasoning efficiency of MLLMs but also provides new insights into multimodal reasoning, promoting deep integration of visual and textual information. However, the model still has limitations in certain complex scenarios, and future work will explore how to further optimize the model's generalization capabilities.
Deep Analysis
Background
Multimodal large language models (MLLMs) have made significant progress in integrating visual and textual information in recent years. However, existing methods still face challenges when processing high-resolution images, particularly requiring explicit visual localization during inference. Traditional visual localization methods often rely on explicit bounding boxes or cropping tools, which may interfere with the model's primary reasoning task.
Core Problem
Existing visual localization methods require explicit visual localization during inference, which may interfere with the model's primary reasoning task. How to improve the model's reasoning accuracy without relying on explicit localization is a pressing issue.
Innovation
iVGR introduces a dual-stream training strategy to internalize visual localization into textual reasoning. • The textual stream aligns with a high-quality visual stream through a consistency reward, ensuring accurate localization without explicit grounding during inference. • This method avoids interference from explicit grounding, improving the model's reasoning efficiency.
Methodology
- �� iVGR employs a dual-stream training strategy, aligning the textual stream with the visual stream. • Internalizes visual localization into textual reasoning through a consistency reward. • Uses a reinforcement learning framework, combining format, accuracy, and consistency rewards to ensure alignment between textual reasoning and visual references.
Experiments
The experimental design includes evaluations on multiple benchmark datasets such as HR8K and V*. Baseline models include Qwen2.5-VL-7B and Qwen3-VL-32B. Evaluation metrics include accuracy and IoU (Intersection-over-Union), with ablation studies to verify the method's effectiveness.
Results
iVGR achieved 82.9% accuracy on the HR8K benchmark, improving by 4.8% over Qwen3-VL-32B. On the V* dataset, the iVGR-Qwen2.5-VL-7B model reached 86.4% accuracy, a 7.9% increase over the baseline.
Applications
iVGR can be applied in scenarios requiring high-precision visual reasoning, such as autonomous driving and medical image analysis. Its ability to function without explicit localization makes it advantageous in resource-constrained environments.
Limitations & Outlook
Although iVGR performs well across multiple benchmarks, in some complex scenarios, the model may not fully internalize all visual information, leading to decreased reasoning accuracy. Additionally, the training process requires substantial computational resources.
Plain Language Accessible to non-experts
Imagine cooking in a kitchen. Traditional visual reasoning methods are like needing to constantly check a recipe to confirm each step, while iVGR is like having memorized the entire recipe and just operating based on memory. This not only saves time but also reduces the chance of errors. iVGR internalizes visual information, allowing the model to reason without explicit localization, thereby improving efficiency.
ELI14 Explained like you're 14
Hey there! Imagine playing a game where you need to find hidden objects. Traditional methods are like needing a magnifying glass to find each item, but iVGR is like having super vision that lets you see all the hidden stuff directly. This new method makes the game faster and more fun because you don't need the magnifying glass anymore!
Glossary
Reinforcement Learning
A machine learning method that trains models to make decisions through rewards and penalties.
Used in iVGR to internalize visual localization into textual reasoning.
Consistency Reward
A reward mechanism that ensures alignment between textual reasoning and visual references.
Used to align the textual stream with the visual stream.
IoU (Intersection-over-Union)
A metric that measures the overlap between predicted and ground-truth boxes.
Used to evaluate the accuracy of visual localization.
Dual-Stream Training
A training strategy using two different reasoning streams.
Used in iVGR to internalize visual localization capabilities.
Multimodal Large Language Models
Models that integrate visual and textual information for reasoning.
The subject of iVGR research.
Open Questions Unanswered questions from this research
- 1 How to improve iVGR's reasoning accuracy in more complex scenarios?
- 2 How to reduce the computational resource requirements of iVGR during training?
Applications
Immediate Applications
Autonomous Driving
iVGR can enhance visual recognition and decision-making in autonomous driving systems, reducing reliance on explicit localization.
Long-term Vision
Medical Image Analysis
iVGR can improve the accuracy of medical image analysis, helping doctors make diagnostic decisions more quickly.
Abstract
While visually grounded Chain-of-Thought (CoT) has emerged as a promising paradigm to enhance fine-grained perception in multimodal large language models (MLLMs), its efficacy during the inference phase remains underexplored. In this work, we empirically find that mandating explicit object boxes in visually grounded CoT during inference often degrades performance compared to standard textual CoT, which reasons without explicit visual grounding. We hypothesize that the visual localization capability can be internalized into the textual CoT and that the mandatory explicit grounding introduces unnecessary interference with the model's primary objective of answer prediction. To address this problem, we propose Internalizing Visually Grounded Reasoning (\textbf{iVGR}), a novel reinforcement learning framework that transfers localization capabilities into the textual reasoning process. We employ a dual-stream training strategy, where a textual stream is aligned with a high-quality visually grounded stream via a proposed consistency reward, enabling the model to localize accurately without explicit grounding during inference. Extensive experiments demonstrate that our method significantly outperforms existing baselines on fine-grained benchmarks, while maintaining the flexibility to support tool-assisted inference workflows.