OmniVerifier-M1: Multimodal Meta-Verifier with Explicit Structured Recalibration

The evolution of multimodal large language models (MLLMs) such as OpenAI’s GPT-4, Google’s PaLM-E, and Meta’s Llama-2 has significantly advanced AI’s reasoning and generative capabilities across text, images, and other modalities. These models have demonstrated remarkable performance in tasks like visual question answering, image captioning, and cross-modal retrieval. However, as their deployment expands into critical sectors like autonomous driving, healthcare, and industrial automation, ensuring their outputs are accurate, reliable, and interpretable becomes paramount. Traditional verification approaches, including reward models like RewardDance and UnifiedReward, primarily focus on coarse, binary judgments, which lack the granularity needed for precise error localization and correction. Moreover, textual explanations used as rationales often depend on auxiliary judge models, increasing complexity and susceptibility to reward hacking. Recent efforts like OmniVerifier have taken initial steps toward general visual verification using binary judgments, but their coarse feedback limits practical utility in complex, real-world scenarios. Consequently, there is a pressing need for verification frameworks that provide fine-grained, spatially grounded feedback, enabling models to self-assess and correct errors at the region level, thereby improving safety, interpretability, and user trust.

The core challenge in multimodal verification lies in balancing the need for detailed, actionable feedback with the constraints of training stability and computational efficiency. Binary judgments alone are insufficient for guiding models toward precise corrections, especially in complex scenes with multiple objects and subtle errors. Textual explanations, while informative, introduce dependencies on auxiliary judge models and are prone to hallucination or superficial reasoning, leading to reward hacking. Existing methods struggle with error localization, often providing only coarse signals that cannot support targeted self-correction. Additionally, joint training of binary judgment and meta-verification objectives often results in conflicting gradients, causing unstable learning dynamics and suboptimal performance. Addressing these issues requires a paradigm shift toward explicit, spatially grounded rationales and a training strategy that decouples heterogeneous objectives, ensuring stable, interpretable, and scalable verification.

This work introduces several key innovations. First, it adopts symbolic bounding boxes as the core rationales for meta-verification, leveraging their spatial and structural properties to enable rule-based rewards like IoU. This ground-truth-aligned symbolic representation simplifies error localization and enhances interpretability. Second, it employs a decoupled reinforcement learning paradigm, training separate reward models for binary judgment and meta-verification, which alleviates conflicting gradient issues and improves training stability. Third, the framework incorporates multi-round, region-specific self-correction, allowing the verifier to iteratively refine its judgments and guide targeted image edits. These innovations collectively establish a new standard for fine-grained, interpretable, and efficient multimodal verification, bridging the gap between coarse binary judgments and detailed error analysis.

• �� Data Preparation: Collect datasets with images, prompts, binary labels, textual rationales, and symbolic bounding boxes. Use these annotations to supervise model training.
• �� Symbolic Rationales: Generate bounding boxes as explicit spatial cues indicating error regions, ensuring they are spatially aligned with visual content.
• �� Model Architecture: Build on Qwen3-VL-8B, integrating modules for symbolic rationale generation, rule-based reward evaluation, and decoupled RL optimization.
• �� Reward Design: Implement IoU-based rule rewards for symbolic bounding boxes, and separate reward models for binary judgment accuracy and meta-verification rationale correctness.
• �� Training Process: • Sample: Draw batches from the dataset, generate model outputs and rationales.
• �� Evaluate: Compute rule-based IoU rewards for bounding boxes, and assess rationale fidelity with dedicated reward models.
• �� Optimize: Update the verifier using decoupled RL objectives, alternating between binary judgment and meta-verification, to stabilize training.
• �� Error Localization & Correction: Use the symbolic bounding boxes to identify error regions, then generate targeted image edits through structured commands.
• �� Multi-round Refinement: Repeat the generate-evaluate-correct cycle to progressively improve verification accuracy and error localization.

The experimental setup involves training OmniVerifier-M1 on the ViVerBench benchmark, which assesses visual outcome verification across diverse tasks. The training utilizes 16 NVIDIA A800-80G GPUs, with 80 epochs, and evaluates performance based on overall accuracy, error localization precision, and computational efficiency. Baseline comparisons include joint training strategies and previous models like OmniVerifier. The experiments analyze the impact of symbolic versus textual rationales, reward design choices, and the decoupling strategy. Additional evaluations are conducted on visual grounding datasets such as RefCOCO to test generalization. Hyperparameters include IoU thresholds, reward weights, and multi-round iteration limits. The robustness of the approach is validated through ablation studies, measuring the influence of each component on validation scores, training stability, and error localization accuracy.

OmniVerifier-M1 achieves a score of 0.68 on ViVerBench, outperforming the joint training baseline at 0.66. The symbolic bounding box rewards reduce training GPU memory by 20%, cut per-step training time by 15%, and maintain inference efficiency. Error localization accuracy improves by 12% in complex scenes, demonstrating precise error attribution. In visual grounding tasks like RefCOCO, the decoupled training strategy yields an overall accuracy of 0.79, surpassing joint training at 0.78. The ablation studies confirm that symbolic rationales combined with decoupled RL significantly enhance both stability and performance. The model demonstrates strong generalization across tasks, with multi-round self-correction further refining outputs and error localization, leading to more reliable and interpretable verification results.

This verification framework is highly applicable in autonomous driving for precise scene understanding, in medical imaging for localized anomaly detection, and in industrial quality control for defect identification. Its ability to localize errors regionally and provide actionable feedback makes it suitable for safety-critical applications requiring high reliability. The approach can be integrated into existing multimodal systems to enable self-assessment and iterative correction, reducing human oversight and increasing automation. Long-term, the method could facilitate the development of fully autonomous, interpretable AI systems capable of continuous self-improvement, especially in domains where safety, transparency, and compliance are paramount.

The reliance on accurate symbolic boundary annotations may limit performance in scenes with occlusion, clutter, or ambiguous error regions. The rule-based reward system, while effective, may require task-specific tuning and may not generalize seamlessly across diverse domains without adaptation. The annotation process for bounding boxes adds data preparation overhead, and automatic generation of such cues remains an open challenge. Additionally, the current framework primarily handles spatial errors, and extending it to more abstract or semantic errors would require further innovation. Future work should focus on reducing annotation costs, enhancing the robustness of symbolic cues, and integrating deep reasoning modules for more complex error types.