VIG-RL: Learning to Search and Insert for Verified Image Grounding

The evolution of multimodal large language models (MLLMs) like GPT-4 and PaLM-E has significantly improved natural language and visual understanding. However, their generative capabilities are limited to producing text, often hallucinating images or providing unverified visual content. Retrieval-augmented methods (e.g., Ma et al. 2024) have attempted to incorporate external knowledge, but typically follow static, decoupled pipelines that retrieve evidence once before generation. These approaches struggle with factual accuracy and dynamic reasoning in complex scenarios. Recent research highlights the need for models capable of active decision-making, managing when and what to retrieve, and how to integrate verified evidence seamlessly into responses. This gap motivates the development of strategies that combine reinforcement learning with multimodal reasoning, enabling models to autonomously manage evidence provenance and improve factual reliability in real-world applications.

The core challenge is enabling models to dynamically determine when to search for external visual evidence, select the most relevant images, and insert them accurately within a response. Static retrieval-then-generation pipelines lack the flexibility to adapt to evolving information needs during interaction, leading to hallucinations and factual inconsistencies. In knowledge-intensive tasks, ensuring the authenticity of visual evidence is critical, yet current methods rely on heuristics or separate modules, limiting their effectiveness. The problem becomes how to formulate this as an active decision process, where the model learns policies for search, selection, and insertion that optimize factual correctness, semantic coherence, and response quality in an integrated manner.

VIG-RL’s main innovation is framing verified image grounding as a reinforcement learning problem within a ReAct-style interactive loop. It introduces a three-action space—text search, image search, and answer generation—allowing the model to actively decide its next step based on current context. The use of the GRPO algorithm enables efficient policy optimization guided by a composite reward system that balances format adherence, factual accuracy, and relevance. Symbolic referencing ensures retrieved images are factually grounded, preventing hallucinations. Unlike prior static pipelines, VIG-RL learns to manage evidence provenance dynamically, adapting to complex scenarios and improving multimodal response fidelity.

• �� Initialize environment with user query.
• �� At each step, generate reasoning (rt) to identify information gaps.
• �� Sample an action (at) from the action space: text search, image search, or answer generation.
• �� Execute action, receive observation (ot), and update context (St+1).
• �� For image search, generate query, retrieve top-K images, assign symbolic IDs.
• �� During answer decoding, predict symbolic tags for images, replacing them with actual images in final output.
• �� Use the GRPO algorithm to optimize policy parameters, guided by a multi-dimensional reward:
• Format reward ensures syntactic correctness.
• Text reward evaluates semantic and factual alignment.
• Image reward penalizes irrelevant insertions.
• Search reward encourages high recall.
• �� Terminate upon answer generation or max steps.

Models trained and evaluated on MRAMG-Bench, covering web, academic, and lifestyle domains. Baselines include static RAG systems and zero-shot models like GPT-5, Gemini-2.5-Flash, and Qwen3-VL variants. Metrics include Image F1, comprehensive scoring (C.S.), and order scores. Training uses 1.1k samples with 80/20 split, tuning retrieval depth K=5. Ablation studies analyze reward design, search frequency, and policy effectiveness. Results show VIG-RL’s superior performance in image relevance, semantic placement, and zero-shot transfer, with significant improvements over static baselines.

VIG-RL-8B achieves an average C.S. score of 78.1 across six datasets, outperforming static RAG by over 21 points. In zero-shot tests, it reaches C.S. scores of up to 97.2, demonstrating strong generalization. Ablation confirms active search rate increases from 36.4% to 100%, with improved retrieval recall and insertion precision. The model effectively manages evidence provenance, leading to more coherent and factually accurate multimodal responses, validating the policy optimization approach.

Applicable in intelligent question answering, digital archives, and virtual assistants requiring factual multimodal responses. Its ability to actively manage evidence retrieval and insertion makes it suitable for high-stakes domains like medical diagnosis, legal analysis, and scientific research, where factual accuracy is paramount. Integration with large-scale knowledge bases and real-time retrieval systems can further enhance its utility, enabling trustworthy AI systems capable of handling complex, knowledge-intensive tasks.

Despite strong performance, the model’s effectiveness depends on the quality and coverage of external knowledge bases. High training costs and computational complexity limit real-time deployment. Symbolic referencing may face challenges with rapidly changing content, requiring continuous updates. Future work should focus on improving efficiency, robustness, and scalability, especially in dynamic environments with evolving data sources.