Be Consistent! Enhancing Robust Visual Reasoning in LVLMs with Consistency Constraints
Introduces ConVBench for complex visual reasoning; uses GRPO reinforcement learning with consistency rewards to improve LVLM logical stability, achieving 73.36% consistency.
Key Findings
Methodology
ConVBench encompasses six reasoning categories: action/state, complex counting, spatial reasoning, causal/intent, commonsense, and temporal perception. It uses automatically generated logically equivalent question-answer pairs validated by humans. The ConVLM model is trained via Group Relative Policy Optimization (GRPO) with dual rewards: accuracy and consistency. This approach leverages pseudo-labels from GPT-4, reducing manual annotation. The evaluation metrics include logical consistency (answer agreement for equivalent questions) and robust accuracy (both answers correct). Experiments on MSCOCO and external datasets demonstrate significant improvements over baselines, with ConVLM-7B achieving 73.36% consistency and 66.83% accuracy.
Key Results
- ConVLM-7B achieves 73.36% logical consistency and 66.83% accuracy on ConVBench, outperforming models like Claude-3.7Sonnet by over 11% and 20%, respectively. It also surpasses open-source models such as InternVL and Qwen2.5-VL-7B.
- On external benchmarks like V*Bench and InfoVQA-test, ConVLM demonstrates strong generalization, with 84.90% accuracy on V*Bench, validating robustness across tasks.
- Ablation studies confirm that automatic question generation and dual rewards are crucial, with each component contributing significantly to performance gains.
Significance
This work addresses the critical issue of logical inconsistency in multimodal reasoning systems, proposing a scalable, weakly supervised training framework. By integrating automatic question generation with reinforcement learning, it enhances model stability and trustworthiness, paving the way for more reliable AI applications in autonomous systems, intelligent assistants, and robotics. The approach reduces annotation costs while improving reasoning quality, marking a step forward in trustworthy AI development.
Technical Contribution
The key innovation is the combination of automatically generated logically equivalent question-answer pairs with a dual-reward reinforcement learning framework (GRPO). This setup encourages models to produce answers that are both correct and logically consistent, even with noisy pseudo-labels. The introduction of logical consistency metrics into training provides a new paradigm for improving reasoning stability. Additionally, the Proposer module automates data creation, reducing reliance on manual annotations, and the dual reward mechanism balances correctness with logical stability, leading to state-of-the-art performance.
Novelty
This is the first comprehensive attempt to incorporate logical consistency constraints into large vision-language models via automated data generation and reinforcement learning. Unlike prior works focusing solely on accuracy, this approach emphasizes the stability and coherence of answers across semantically equivalent questions, setting a new standard for trustworthy multimodal reasoning systems.
Limitations
- Despite improvements, the model struggles with highly ambiguous or occluded images, indicating limitations in visual perception under challenging conditions.
- Automated question generation, while efficient, can introduce noise, which may affect training quality and requires further refinement.
- Training large models with dual rewards is computationally intensive, limiting scalability and deployment in resource-constrained environments.
Future Work
Future directions include integrating causal reasoning and multi-task learning to handle more complex scenarios, optimizing question generation to reduce noise, and developing more efficient training algorithms. Extending the framework to real-world applications like autonomous driving or medical diagnosis will further validate its robustness and practical utility.
AI Executive Summary
Visual reasoning remains a fundamental challenge in artificial intelligence, especially when models must interpret complex scenes and answer logically consistent questions. Existing benchmarks often focus on superficial tasks or symbolic reasoning, failing to evaluate models' true understanding and logical stability. Recognizing this gap, the authors introduce ConVBench, a comprehensive benchmark designed to assess deep visual reasoning across six categories: action and state, complex counting, spatial relations, causal understanding, commonsense reasoning, and temporal perception.
ConVBench pairs each image with two logically equivalent questions, enabling evaluation of whether models produce consistent answers. To facilitate large-scale data creation, the authors employ GPT-4 to automatically generate question-answer pairs, which are then validated by human annotators to ensure quality. This approach minimizes manual effort while maintaining reliability. The core innovation lies in training models with a reinforcement learning framework called Group Relative Policy Optimization (GRPO), which incorporates a dual reward system: one for answer correctness and another for logical consistency across equivalent questions.
The proposed ConVLM model, trained with this method, demonstrates remarkable performance improvements. On the ConVBench dataset, the 7B version achieves 73.36% logical consistency and 66.83% accuracy, surpassing state-of-the-art closed-source models like Claude-3.7Sonnet. Extensive experiments on external benchmarks such as V*Bench and InfoVQA-test confirm its strong generalization capabilities. Ablation studies reveal that automatic question generation and dual rewards are critical to these gains.
This research significantly advances the development of trustworthy multimodal AI systems. By emphasizing logical stability alongside accuracy, it addresses a key limitation of current models, paving the way for more reliable applications in autonomous systems, intelligent assistants, and beyond. Future work will focus on extending causal reasoning, reducing noise in data generation, and optimizing computational efficiency, aiming to realize truly robust visual reasoning AI.
Deep Analysis
Background
多模态视觉推理技术经历了从基础的图像识别到深度理解的演变。早期工作如VQA、Image Captioning取得了显著进展，但在推理复杂性和逻辑一致性方面仍存在不足。近年来，诸如LLaVA、MiniGPT-4等模型引入大规模预训练和多模态融合，提升了感知能力，但在复杂推理任务中的表现仍不稳定。现有基准如VCR、OK-VQA多偏重语义匹配，缺乏对推理逻辑和一致性的系统评估。研究逐渐认识到，模型需要在符号推理和视觉理解基础上，强化逻辑推理能力，确保输出的合理性和一致性，成为当前研究热点。
Core Problem
尽管LVLM在感知任务中表现优异，但在复杂推理场景中，模型常出现答案不一致、推理不稳的问题。现有评测体系难以全面衡量模型在多类别推理中的能力，尤其在语义等价问答中，模型表现出明显的不稳定性。这严重影响模型的可信度和实际应用价值。核心难点在于如何设计训练机制，既保证答案正确，又提升推理的逻辑稳定性，避免模型过度依赖偏差或表面特征，从而实现真正的理解能力。
Innovation
本研究的创新点包括：1）提出ConVBench，涵盖六大复杂推理类别，系统评估模型的逻辑一致性；2）利用GPT-4自动生成逻辑等价问答对，降低人工标注成本；3）引入基于GRPO的强化学习框架，结合双重奖励机制，强化模型在推理中的逻辑稳定性。与传统只追求准确率的方法不同，该方案强调推理的逻辑一致性，显著提升模型的鲁棒性。模型架构中，Proposer模块自动生成问答对，减少人工干预，增强泛化能力。这一系列创新推动多模态推理技术向更高水平发展。
Methodology
- �� 数据准备：从MSCOCO筛选复杂场景图片，自动生成逻辑等价问答对，人工验证确保质量。
- �� 模型架构：基于LVLM，加入Proposer模块自动生成问答对，结合强化学习优化。
- �� 训练策略：采用GRPO算法，设计双重奖励——答案正确性（racc）和逻辑一致性（rcon）。
- �� 训练流程：
- 自动生成问答：利用GPT-4根据图像、描述和目标信息生成等价问答。
- 计算奖励：结合答案正确性和逻辑一致性指标。
- 迭代优化：通过GRPO调整模型参数，平衡两个目标。
- �� 评价指标：逻辑一致性（两个等价问答答案是否一致）和鲁棒准确率（两问都正确的比例）。
Experiments
采用MSCOCO训练集，训练8×A100 GPU，批次256，学习率10e-7。模型在ConVBench、V*Bench和InfoVQA-test上评估，比较不同版本（3B、7B）和对比模型。进行消融实验验证奖励机制和自动问答生成效果。指标包括逻辑一致性、准确率和外部基准性能。训练过程中结合自动生成问答对和双重奖励，优化推理稳定性和准确性。多任务、多数据源训练确保模型在多场景下表现优异。
Results
ConVLM-7B在ConVBench达成73.36%逻辑一致性和66.83%准确率，优于Claude-3.7Sonnet，提升11.12%和20.94%。在V*Bench上，准确率达84.90%，验证其良好的泛化能力。消融实验显示，自动问答生成和双重奖励是性能提升的关键。模型在复杂推理任务中表现出较强的鲁棒性，验证了方法的有效性。
Applications
该模型适用于智能问答、自动诊断、机器人理解等场景，尤其在需要复杂推理和逻辑稳定的应用中。只需提供高质量图像和相关问题，模型即可输出一致且正确的答案，提升系统可信度。未来结合因果推理、多任务学习，有望实现更智能、更可靠的多模态理解系统。
Limitations & Outlook
模型在极端复杂或模糊场景仍存在推理不足，自动生成问答可能引入噪声影响训练效果。训练成本高，难以在资源有限环境中广泛部署。未来需优化生成策略，提升效率和鲁棒性。
Plain Language Accessible to non-experts
想象你在厨房做菜。每次用不同的食谱做菜，味道都一样。这个研究就像教厨师识别不同版本的食谱其实是一样的，只要用不同的问题问厨师，他都能给出一致的答案。让模型学会在不同问法下保持答案一致，就像厨师记住了菜谱的核心，不会出错。这让电脑变得更聪明，也让它的回答更可靠，就像厨房里的菜一样稳定好吃。
ELI14 Explained like you're 14
想象你在玩一个超级复杂的拼图游戏。有时候，你会问朋友：‘这块拼图是不是放对了？’然后又问：‘这块拼图和那块是不是一样？’如果你的朋友知道这两句话其实是在问同一件事，他就能给你一致的答案。这个研究就是让电脑学会像你一样，不管问得怎么变，它都能给出一样的、正确的答案。这样，电脑就不会因为问法不同而搞错，也更聪明、更可靠啦！
Abstract
While Large Vision-Language Models (LVLMs) exhibit strong perceptual capabilities, they remain vulnerable in visual reasoning tasks. Existing benchmarks largely focus on symbolic mathematical or scientific problems and simple vision-centric tasks, offering limited assessment of complex visual reasoning and logical consistency, a critical requirement for reliable reasoning systems. We introduce ConVBench, a complex vision-centric reasoning benchmark in which each image is paired with two logically equivalent questions across six categories: action and state, complex counting, spatial reasoning, causal and intent understanding, commonsense reasoning, and temporal perception. To complement this benchmark, we define two evaluation metrics, logical consistency and robust accuracy, that jointly assess both the correctness and consistency of model responses. We further present ConVLM, which improves LVLM reasoning through Group Relative Policy Optimization (GRPO)-based reinforcement learning with a novel consistency reward. This method leverages automatically generated logically equivalent question-answer pairs and a dual-reward design combining accuracy- and consistency-based signals, encouraging agreement between paired responses. The framework functions effectively with or without strict answer supervision.