Visual-ERM: Reward Modeling for Visual Equivalence

In recent years, with advances in computer vision and natural language processing, vision-to-code tasks have become an important research area. The goal of these tasks is to convert structured visual inputs, such as charts, tables, and SVGs, into executable or structured representations like code or markup. Traditional approaches primarily rely on supervised fine-tuning, which requires substantial annotated data and often lacks cross-domain generalization. Recently, reinforcement learning has emerged as a promising alternative, but it introduces challenges due to misaligned reward signals. Existing reward methods either rely on textual rules or coarse visual embedding similarity, both of which fail to capture fine-grained visual discrepancies and are vulnerable to reward hacking.

The core problem in vision-to-code tasks is how to effectively evaluate the similarity between the generated code and the original visual input. Existing methods primarily rely on textual rules or coarse visual embedding similarity, both of which fail to capture fine-grained visual discrepancies and are vulnerable to reward hacking. This leads to models potentially optimizing the wrong objectives during training, affecting final performance. Therefore, developing a reward model that can provide fine-grained evaluation directly in the visual space is key to addressing this issue.

The core innovation of Visual-ERM lies in its design as a multimodal generative reward model. Firstly, the model can evaluate vision-to-code quality directly in the rendered visual space, avoiding vulnerabilities associated with textual rules or coarse visual embedding similarity. Secondly, Visual-ERM integrates modeling of global structure and local visual details, providing fine-grained, interpretable, and task-agnostic feedback. This approach not only enhances the model's parsing capabilities but also opens new engineering possibilities for future vision-to-code tasks. Finally, Visual-ERM further enhances test-time scaling via reflection and revision.

The implementation of Visual-ERM includes the following key steps:

• �� Reward Data Generation: Generate reward data through controlled corruption and annotation.

• �� Supervised Fine-Tuning of the Reward Model: Perform supervised fine-tuning on the generated reward data.

• �� Integration into RL: Integrate Visual-ERM into the RL pipeline to enhance the model's parsing capabilities.

• �� Reflection and Revision: Further enhance test-time scaling through reflection and revision.

The experimental design includes datasets for multiple vision-to-code tasks, such as ChartMimic, OmniDocBench, and UniSVG. We adopt Qwen3-VL-8B-Instruct as the policy model's backbone and use GRPO as the RL algorithm. In the experiments, we compare the performance differences between Visual-ERM and DINO-based reward methods, focusing on the model's performance in fine-grained visual discrepancy evaluation.

Experimental results show that Visual-ERM achieves significant performance improvements across multiple vision-to-code tasks. On chart-to-code tasks, Visual-ERM improved Qwen3-VL-8B-Instruct's performance by 8.4 points. On table and SVG parsing tasks, Visual-ERM achieved improvements of 2.7 and 4.1 points respectively. Additionally, in the VisualCritic-RewardBench benchmark, Visual-ERM at 8B decisively outperformed Qwen3-VL-235B-Instruct and approached leading closed-source models.

Application scenarios for Visual-ERM include but are not limited to:

• �� AI-assisted front-end development: Converting UI designs to code.

• �� Scientific paper parsing: Automatically extracting and parsing charts and data from papers.

• �� Knowledge management and system integration: Enhancing information accessibility and usability through vision-to-code conversion.

Despite its strong performance across multiple tasks, Visual-ERM may encounter performance bottlenecks when dealing with complex visual structures, particularly in parsing high-resolution images. Additionally, the model relies heavily on training data, potentially requiring substantial annotated data to realize its potential. In certain specific tasks, Visual-ERM's generalization ability may be limited. Future research could explore the application of Visual-ERM in more vision-to-code tasks, especially those involving complex visual structures. Additionally, further optimization of the model's computational efficiency and generalization capabilities are important research directions.