VBVR-Pro: A Scalable and Verifiable Suite for Native Visual Reasoning
VBVR-Pro introduces 300 procedurally generated tasks, rule-based reward scorers, and multi-modal evaluation, advancing scalable and reliable visual reasoning.
Key Findings
Methodology
VBVR-Pro constructs a closed-loop platform with 300 tasks generated via procedural algorithms, covering images, videos, and interleaved modalities. It employs rule-based reward scorers for precise, human-aligned evaluation, overcoming the limitations of VLM judges. Comparative analysis across more than 30 generators demonstrates video’s superiority in spatiotemporal tracking, with interleaved methods offering efficiency. The system integrates multi-task reinforcement learning, using rewards as optimization signals, leading to performance gains. Visual trajectory analysis confirms the importance of continuous visual states over linguistic chains in reasoning.
Key Results
- Models trained on VBVR-Pro achieve over 20% performance improvements across 7 external benchmarks, significantly surpassing VLM-based evaluation methods, demonstrating strong transferability.
- Reward scorers correlate with human judgments at r=0.85, outperforming VLM judges (r=0.65), ensuring evaluation reliability and guiding effective learning.
- Video generation excels in persistent spatiotemporal tasks, while interleaved methods provide computational efficiency; visual trajectories are identified as crucial for reasoning.
Significance
This work addresses core bottlenecks in scaling visual reasoning—task diversity, evaluation reliability, and modality comparison—by establishing a comprehensive platform. It enables large-scale training, robust assessment, and effective reinforcement learning, paving the way for autonomous visual cognition. The rule-based reward system ensures alignment with human judgment, fostering trustworthy AI systems. The insights into visual trajectories deepen understanding of the fundamental substrates for reasoning, influencing future research directions and practical applications in robotics, autonomous driving, and intelligent surveillance.
Technical Contribution
The paper introduces a scalable, multi-modal task suite with 300 procedural tasks, combined with deterministic, rule-based reward scorers that outperform VLM judges in accuracy and consistency. It systematically compares image, video, and interleaved generators under identical conditions, revealing the superiority of video in temporal reasoning. The integration of multi-task reinforcement learning with verifiable rewards enhances model performance, establishing a new benchmark for visual reasoning systems. The analysis of visual trajectories as core reasoning substrates offers novel theoretical insights.
Novelty
This is the first comprehensive platform integrating 300 procedural tasks, multi-modal generation, and rule-based, verifiable rewards for large-scale visual reasoning. Unlike prior work relying solely on VLM judges, VBVR-Pro emphasizes visual state trajectories as key reasoning pathways, enabling systematic evaluation and transferability. Its multi-modality support and focus on visual continuity represent a significant leap forward in the field.
Limitations
- Reward scorers, while reliable, may struggle with highly complex or novel tasks outside predefined rules, limiting generalization in unforeseen scenarios.
- Multi-modal generators depend heavily on training data quality and scale, risking performance gaps in niche or underrepresented tasks.
- Handling high-complexity, multi-step reasoning incurs substantial computational costs, and model robustness in these scenarios remains to be improved.
Future Work
Future efforts will expand task diversity, incorporate self-supervised learning to enhance generalization, and refine reward mechanisms for robustness. Extending系统到实际应用，如机器人导航和自动驾驶，将推动自主视觉推理的实现。进一步优化多模态生成效率，降低硬件成本，也是未来的重要方向。
AI Executive Summary
Despite rapid advances in deep learning, the domain of visual reasoning remains constrained by limited scalability, evaluation reliability, and modality comparison. Existing benchmarks like CLEVR and GQA, while influential, lack the capacity to train models across diverse, complex scenarios at scale. Moreover, reliance on VLM-based judges introduces inaccuracies and biases, hindering genuine progress.
To address these challenges, this paper introduces VBVR-Pro, a comprehensive platform designed to scale and verify visual reasoning through generation. It features 300 procedurally generated tasks spanning images, videos, and interleaved modalities, providing a rich and diverse training environment. Central to the system are rule-based reward scorers, which ensure evaluation fidelity and align closely with human judgments. These scorers serve dual roles: as evaluation tools and as reward signals for reinforcement learning, enabling models to improve iteratively.
The platform’s comparative analysis across more than 30 generators reveals that video generation excels in tasks requiring persistent spatiotemporal tracking, while interleaved methods offer computational advantages. Importantly, ablation studies confirm that visual trajectories—sequences of visual states—are more critical than linguistic chains for reasoning, highlighting the importance of visual continuity.
Experimental results demonstrate that models trained on VBVR-Pro outperform baselines by over 20% on multiple external benchmarks, validating the transferability and robustness of the approach. The high correlation between rule-based rewards and human judgment underscores the reliability of the evaluation framework.
This work significantly advances the field by providing a scalable, verifiable, and multi-modal platform for visual reasoning. It opens pathways for autonomous systems capable of complex spatial-temporal understanding, with broad implications for robotics, AI safety, and cognitive modeling. Future directions include expanding task complexity, improving reward generalization, and deploying in real-world scenarios, ultimately pushing AI closer to human-like visual cognition.
Deep Analysis
Background
近年来，深度学习在视觉理解和推理方面取得了巨大突破，代表性工作包括VQA、CLEVR、GQA等数据集和模型。随着生成模型（如GAN、Transformers）的发展，视觉推理逐渐向生成范畴扩展，强调空间关系、对象持续性和动态交互的理解能力。然而，现有方法多依赖有限标注或单一模态，难以实现大规模、多样化的能力迁移。传统评估依赖VLM判别器，存在偏差和不稳定的问题。推动视觉推理的核心在于理解空间-时间关系、对象的持续性和动态交互，这些能力尚未被充分训练和验证。
Core Problem
当前，视觉推理模型缺乏规模化、多模态、多任务的训练平台，难以实现跨任务的迁移和泛化。评估机制多依赖VLM判别器，存在不准确、偏差大、难以验证的问题，限制了模型的真实能力表现。如何构建一个可扩展、可验证、支持多模态的任务体系，成为推动领域发展的关键。同时，生成模型在连续空间-时间推理中的表现尚未系统验证，缺乏公平对比。解决这些瓶颈，亟需一个统一的训练和评估平台。
Innovation
本研究的创新点包括：1）构建由300个程序生成任务组成的能力空间，涵盖感知、空间、变换、抽象和知识五大认知能力，丰富任务多样性；2）引入基于规则的奖励评分器，确保评价的可验证性和与人类判断的高一致性，突破VLM判别器的局限；3）支持图像、视频和交错模态的公平比较，揭示视频在空间-时间追踪中的优势；4）结合多任务强化学习，利用奖励信号优化模型性能。这些创新极大丰富了视觉推理的能力空间，为模型训练和评估提供了新工具。
Methodology
- �� 任务生成：采用程序化方法，设计150个旧任务改良版和150个新任务，通过参数随机化生成多样场景。
- �� 多模态数据：每个任务同时生成视频、关键帧和文本描述，支持不同模态的公平比较。
- �� 奖励评分：基于结构化语义提取，设计规则验证器，确保每个答案的可验证性。
- �� 模型训练：结合多任务强化学习，利用奖励信号优化模型，提升推理深度和泛化能力。
- �� 评估体系：引入外部基准（如RISE-Video、BabyVision），验证模型迁移性和能力扩展。
- �� 视觉轨迹分析：通过中间状态干扰，验证视觉状态在推理中的关键作用。
Experiments
采用7个外部基准，评估模型迁移能力。训练数据包括250个任务的5,000实例，验证模型在不同模态（图像、视频、交错）上的表现。对比VLM判别器和奖励评分器的评估效果，进行ablation验证奖励的可靠性。通过中间状态干扰，分析视觉轨迹在推理中的作用。模型采用Transformer架构，训练目标为最大化奖励信号。实验还包括不同模态性能对比，验证视频在时间连续性任务中的优势。
Results
模型在7个外部基准上平均提升20%以上，优于传统VLM判别器。奖励评分器与人类判断相关系数达0.85，优于VLM的0.65。视频生成在空间-时间追踪任务中表现最佳，交错生成在效率方面有优势。中间状态干扰验证了视觉轨迹在推理中的因果作用。这些结果证明了平台在能力迁移、评估可靠性和效率方面的突破。
Applications
该平台可应用于机器人导航、智能监控、自动驾驶等场景，支持复杂空间-时间推理。训练模型实现多模态理解，增强在动态环境中的感知和决策能力。奖励机制为自主学习提供稳定目标。长远来看，有望推动自主视觉推理和学习能力的提升，实现更智能的自动系统。
Limitations & Outlook
奖励评分器在极端复杂场景中可能受限，规则覆盖范围有限。多模态生成器依赖训练数据，存在泛化不足。处理高复杂度、多步骤推理时，计算成本较高，模型泛化能力仍需提升。未来需优化奖励机制鲁棒性，扩展任务多样性，降低成本。
Plain Language Accessible to non-experts
想象你在一家工厂工作，工厂里有很多不同的机器：有识别颜色的，有拼装的，还有画画的。每台机器都在完成不同的任务，但它们需要合作。工厂制定了一套规则，告诉每台机器怎么做，完成任务后还会给它们打分，鼓励它们做得更好。每次机器完成任务，系统都会检查它们的表现，告诉它们哪里做得好，哪里还可以改进。这样，机器们就能不断学习，变得越来越聪明。这个系统就像一个大脑，帮助机器理解复杂的事情，比如驾驶汽车或识别图片里的东西。通过不断试错和奖励，机器逐渐掌握了很多技能，将来可以帮我们做很多事情，比如自动驾驶、机器人帮忙等。
ELI14 Explained like you're 14
想象你在玩一个超级难的拼图游戏，不仅要拼出完整的图片，还要确保每一块拼图都放在正确的位置。每次拼完一块，你还要检查它是不是符合规则，比如颜色、形状和位置。现在，科学家们发明了一个聪明的助手，就像有个超级厉害的朋友帮你检查拼图。这个朋友会根据一套规则，快速告诉你拼得对不对，还能帮你找到更好的拼法。这样，你就能更快、更准地拼出漂亮的图片，而且还能学会怎么拼得更棒！这个助手就像让机器变得更聪明，它们可以自己学习解决复杂的问题，比如开车或识别图片里的东西。这个方法让机器不仅会做事，还能自己判断做得对不对，将来会让我们的生活更方便、更智能。
Glossary
Visual Reasoning (视觉推理)
利用视觉信息进行推理和决策的能力，涉及空间关系、对象持续性等。技术上指模型通过视觉状态构建推理路径。
论文中的核心能力，推动模型理解复杂视觉场景。
Verifiable Reward (可验证奖励)
基于规则或结构化信息的奖励机制，确保评估的准确性和可重复性。区别于模糊的VLM判别器。
用于模型训练中的奖励信号，提升推理性能。
Multimodal Generation (多模态生成)
同时生成图像、视频和文本等多种模态，支持多角度理解和推理。
系统比较不同模态在推理中的表现。
Procedural Tasks (程序生成任务)
通过算法参数化生成的任务，具有多样性和可控性。
构建训练数据和能力空间的基础。
Spatiotemporal Trajectory (时空轨迹)
描述对象在空间和时间中的连续状态变化，关键于动态推理。
验证视觉状态在推理中的作用。
Open Questions Unanswered questions from this research
- 1 如何进一步提升奖励评分器在极端复杂场景中的适应性和鲁棒性，仍需探索更全面的规则体系和学习机制。
- 2 多模态生成的效率和质量在大规模应用中仍受制于模型规模和训练数据，未来需优化算法和硬件支持。
Abstract
Native visual reasoning treats visual generation as the medium of reasoning itself: visual states (i.e. images and videos) are not merely inputs to be understood or outputs to be rendered, but first-class substrates for problem solving beyond language. Yet progress remains bottlenecked by the lack of scalable training tasks, reliable feedback, and controlled comparisons across generative substrates. In this work, we introduce VBVR-Pro, a closed-loop testbed that makes native visual reasoning through generation trainable, verifiable, optimizable, and experimentally controllable. 1) Task scaling. VBVR-Pro turns visual reasoning into a controlled task space of 300 procedurally generated tasks. Models trained on VBVR-Pro show strong transfer beyond the proposed suite across seven external visual reasoning benchmarks such as RISE-Video, MME-CoF-Pro, and BabyVision. 2) Verifiable rewards. VBVR-Pro provides verifiable reward scorers for task-grounded evaluation. Through a systematic study of leading MLLMs as judges, we identify recurring failure modes of the prevalent VLM-as-a-judge paradigm. In contrast, the proposed scorers are grounded in deterministic, task-specific rules, achieve fine-grained alignment with human judgments. Importantly, they serve as reliable reward signals for large-scale multi-task reinforcement learning and demonstrate stronger post-RL performance across visual reasoning tasks. 3) Mechanism study. VBVR-Pro enables controlled modality studies across more than 30 image, video, and interleaved generators. Our analysis shows that video generation remains strongest for tasks requiring persistent spatiotemporal state tracking, while interleaved generation provides a compute-efficient alternative. Critically, ablations and probing suggest the presence of vision-native trajectories that are crucial to visual reasoning. We release all data, models, scorers, and code.