VFIG: Vectorizing Complex Figures in SVG with Vision-Language Models

In the fields of technical illustration and digital design, vector graphics formats (SVG) are widely used due to their resolution independence and flexible editability. However, over time, original vector files are often lost or inaccessible, leaving only rasterized images that are difficult to edit. This situation is particularly common in academic publications and professional designs, posing significant challenges for modifying and reusing graphics. Traditional graphic reconstruction methods typically rely on manual operations, which are time-consuming and require specialized skills, making them impractical for large-scale applications. With the advancement of deep learning technologies, automated graphic conversion has become possible, but existing methods still face challenges in handling complex graphics, especially in terms of dataset scale and complexity.

The core problem is how to effectively convert complex rasterized graphics into editable SVG format. The difficulty lies in restoring the geometric structure and semantic information of the graphics. Existing methods often rely on small-scale datasets, lacking generalization capabilities for complex graphics. Additionally, traditional graphic conversion methods perform poorly in handling multi-level structures and topological edge cases, resulting in insufficient fidelity and consistency in conversion results. Solving this problem is crucial for improving the efficiency and flexibility of graphic editing.

The core innovations of VFIG lie in its coarse-to-fine training strategy and large-scale dataset support. First, VFIG introduces VFIG-DATA, a dataset comprising 66K high-quality figure-SVG pairs, significantly enhancing model training effectiveness. Second, VFIG employs a coarse-to-fine training curriculum, combining supervised fine-tuning and reinforcement learning to progressively optimize graphic fidelity and consistency. Additionally, the introduction of the VFIG-BENCH evaluation suite provides new standards for assessing the structural integrity of complex figures. Compared to existing methods, VFIG excels in handling complex graphic structures.

• �� Construction of VFIG-DATA dataset: Collecting 66K high-quality figure-SVG pairs, covering real-world paper figures and procedurally generated diagrams.
• �� Coarse-to-fine training strategy:
• Supervised Fine-Tuning (SFT): Learning atomic primitives to establish initial graphic structures.
• Reinforcement Learning (RL): Optimizing global diagram fidelity, layout consistency, and topological edge cases.
• �� VFIG-BENCH evaluation suite:
• Introducing novel evaluation metrics to assess the structural integrity of complex figures.
• Testing model performance on various types of figures to ensure generalization capability.

The experimental design includes training and evaluating the model using the VFIG-DATA dataset. Baseline models include existing figure conversion methods for performance comparison with VFIG. Evaluation metrics include the VLM-Judge score, which measures model performance in complex figure conversion tasks. Experiments also include ablation studies to analyze the impact of different training strategies on model performance. Key hyperparameter adjustments ensure model stability and efficiency.

Experimental results show that VFIG achieves a VLM-Judge score of 0.829 on VFIG-BENCH, significantly outperforming other baseline models. Ablation studies indicate that the coarse-to-fine training strategy plays a crucial role in enhancing model performance. VFIG excels in handling various types of figures, particularly in the conversion of professional diagrams, showing superior fidelity and consistency compared to existing methods.

Application scenarios for VFIG include graphic editing and reuse in technical illustration and digital design. By automating graphic conversion, designers and researchers can more efficiently modify and extend existing graphics. Additionally, VFIG can be applied in education and publishing, aiding in the rapid generation of high-quality graphic content.

Despite VFIG's excellent performance in complex figure conversion, it may experience performance degradation when handling extremely complex or irregular figures. Furthermore, the computational cost of VFIG's training and inference processes is high, which may limit its application in resource-constrained environments. Future research can overcome these limitations by expanding datasets and optimizing algorithms.