KoVRE: Training an Efficient Embedding Model for Korean Visual Document Retrieval
KoVRE model enhances Korean document retrieval efficiency using bilingual supervision and hard-negative mining.
Key Findings
Methodology
The study introduces KoVRE, a single-vector embedding model for Korean visual document retrieval. It is trained on 708,729 Korean and English query-page pairs using positive-aware hard-negative mining and reranker-based knowledge distillation. The methodology involves Matryoshka multiple-negative ranking objectives, analyzing training data composition, hard-negative treatment, and embedding dimensionality.
Key Results
- Result 1: KoVRE achieves an nDCG@10 score of 0.7627 on KoViDoRe benchmarks, improving over the base model by more than 20%.
- Result 2: Outperforms multi-vector baselines in SDS KoPub VDR, demonstrating adaptability across document domains.
- Result 3: Ablation studies confirm the critical role of bilingual training data and hard-negative treatment.
Significance
This research significantly enhances Korean visual document retrieval efficiency through bilingual supervision and carefully designed training strategies, addressing the limitations of existing models that rely heavily on large backbones and multi-vector representations.
Technical Contribution
Technically, the model achieves efficient retrieval without scaling up the backbone or using multi-vector representations. The training strategy, including hard-negative mining and knowledge distillation, offers new engineering possibilities.
Novelty
This study is the first to achieve efficient single-vector embedding in Korean visual document retrieval, offering a more compact solution compared to previous methods relying on large-scale multi-vector representations.
Limitations
- Limitation 1: The diversity of Korean training data is limited, potentially affecting the model's generalization to unseen formats.
- Limitation 2: The model may experience performance drops when handling complex document layouts.
Future Work
Future research could expand the diversity of Korean page images, explore more efficient hard-negative mining strategies, and apply the model to other language environments.
AI Executive Summary
Visual Document Retrieval (VDR) plays a crucial role in directly matching text queries with document images, especially in preserving visual and structural information. However, existing VDR models are predominantly English-centric and rely on massive backbones or multi-vector representations, leading to high storage and retrieval costs. To address these issues, researchers have introduced KoVRE, a single-vector retriever model specifically designed for Korean documents. It is trained using positive-aware hard-negative mining and knowledge distillation on 708,729 Korean and English query-page pairs.
Experimental results demonstrate that KoVRE excels in multiple Korean document retrieval benchmarks, particularly in economic and human resources domains. Its performance surpasses larger single-vector models and strong multi-vector baseline models, indicating that a well-designed training strategy can achieve efficient document retrieval without scaling up the backbone.
Despite its success, the study also highlights limitations in the diversity of training data. Future research directions include expanding the diversity of Korean page images and applying the model to other language environments. With these efforts, KoVRE is poised to provide broader support for multilingual document retrieval.
Deep Analysis
Background
Visual Document Retrieval (VDR) directly matches text queries with document images, preserving visual and structural information. Existing VDR models are predominantly English-centric and rely on massive backbones or multi-vector representations, leading to high storage and retrieval costs. Recently, researchers have focused on VDR in multilingual environments, particularly the need for Korean document retrieval.
Core Problem
Existing VDR models perform poorly in Korean document retrieval due to English-centric training data and evaluation benchmarks. Additionally, systems relying on large backbones and multi-vector representations incur high storage and computational costs, limiting practical applications.
Innovation
KoVRE achieves efficient Korean document retrieval through bilingual supervision and hard-negative mining. Innovations include: 1) single-vector representation reducing storage and computational costs; 2) improved performance through hard-negative mining and knowledge distillation; 3) significant retrieval efficiency without scaling up the backbone.
Methodology
- �� Initialize model with Qwen3-VL-Embedding-2B.
- �� Conduct contrastive learning on Korean and English query-page pairs using Matryoshka multiple-negative ranking objectives.
- �� Perform knowledge distillation using Qwen3-VL-Reranker-8B as the teacher model.
- �� Optimize training through hard-negative mining and self-guide filtering.
Experiments
Experiments are conducted on KoViDoRe and SDS KoPub VDR benchmarks, using nDCG@10 as the main evaluation metric. The model is trained on 708,729 Korean and English query-page pairs, with hard-negative mining using Qwen3-VL-Embedding-8B and model optimization through knowledge distillation.
Results
Experimental results show that KoVRE excels in multiple benchmarks, particularly in economic and human resources domains. Its performance surpasses larger single-vector models and strong multi-vector baseline models, indicating that a well-designed training strategy can achieve efficient document retrieval without scaling up the backbone.
Applications
KoVRE can be applied in multilingual document retrieval, especially in scenarios requiring efficient processing of Korean documents. Its compact single-vector representation is suitable for environments with limited storage and computational resources.
Limitations & Outlook
Despite its success, KoVRE's training data diversity is limited, potentially affecting the model's generalization to unseen formats. Additionally, the model may experience performance drops when handling complex document layouts. Future research should expand the diversity of Korean page images and explore more efficient hard-negative mining strategies.
Plain Language Accessible to non-experts
Imagine you're in a library looking for a book. Previously, you had to write down the text from the book and then search using that text, which was slow and might miss important information. KoVRE is like a smart librarian that can find the book based on your description directly, without needing to write down the text first. It learns from both Korean and English book descriptions to improve accuracy. So, if you're looking for a book about Korean history, KoVRE will find it based on the cover and table of contents, not just the title.
ELI14 Explained like you're 14
Hey there! Imagine you're at school looking for a super cool comic book. Before, you'd have to write down the title and then search, but sometimes the title doesn't match the book's content. Now, there's a super helper called KoVRE that can find the book based on what you say! Like if you say, "I want a comic with dragons and knights," KoVRE will find it right away, without you having to search through a bunch of titles. Isn't that awesome? That's the magic of KoVRE!
Glossary
Visual Document Retrieval (VDR)
Directly matches text queries with document images, preserving visual and structural information.
Used in the paper to enhance Korean document retrieval efficiency.
Embedding Model
Maps text or images into a shared vector space.
Used to represent document pages and queries.
Knowledge Distillation
Transfers knowledge from a complex model to a smaller model.
Used to improve the performance of the KoVRE model.
Hard-negative Mining
Selects challenging negative samples from data to improve model performance.
Used during training to optimize the model.
Matryoshka Representation Learning (MRL)
Enhances model representation capability through multi-level embedding learning.
Used to optimize KoVRE model's embedding representation.
Open Questions Unanswered questions from this research
- 1 How to improve model generalization to unseen formats? Current training data lacks diversity to cover all possible document formats.
- 2 What adjustments are needed to apply KoVRE in other language environments? Current training strategy is primarily focused on Korean.
Applications
Immediate Applications
Korean Document Retrieval
Apply KoVRE in scenarios requiring efficient processing of Korean documents to enhance retrieval efficiency.
Multilingual Document Retrieval
Utilize KoVRE's bilingual capabilities for efficient document retrieval in multilingual environments.
Long-term Vision
Global Document Retrieval Platform
Expand KoVRE's language capabilities to build a global document retrieval platform supporting multiple languages.
Abstract
Visual Document Retrieval (VDR) directly matches text queries against document images, preserving visual and structural information that may be lost during text extraction. However, existing VDR models and training resources remain predominantly English-centric, while many high-performing systems rely on massive backbones or storage-intensive multi-vector representations. To address these limitations, we introduce KoVRE: Korean Visual Document Retrieval Embedding, a single-vector retriever for Korean visual documents, alongside a comprehensive training recipe. We train the model on 708,729 Korean and English query-page pairs using positive-aware hard-negative mining and conduct controlled analyses of training-data composition, hard-negative treatment, and reranker-based knowledge distillation. Across Korean visual document retrieval benchmarks, our 2B model substantially improves over the base backbone model, outperforming both its 8B single-vector counterpart and a strong multi-vector baseline. These results demonstrate that targeted bilingual supervision and our carefully designed training strategies can produce a highly effective Korean VDR model across diverse document domains, without requiring a scaled-up backbone or multi-vector representations.