End-to-End Training of Multi-Document Reader and Retriever for Open-Domain Question Answering

The evolution of open-domain QA has transitioned from simple retrieval-then-generation pipelines to sophisticated neural models like DPR, REALM, and FiD, which leverage large pre-trained transformers. These models have improved answer accuracy but often rely on staged training, which limits the potential for joint optimization. Recent efforts aim to unify retrieval and reading, but the challenge remains in handling the latent variable of relevant document sets. Large-scale knowledge bases like Wikipedia provide abundant information, but effectively integrating multiple documents remains difficult due to computational complexity and the intractability of marginalizing over all possible document combinations. Prior work has shown that end-to-end training can improve relevance modeling, but explicit supervision for retrieval is costly and often unavailable. This context motivates the development of methods that can learn retrieval strategies implicitly, leveraging the model’s own outputs to guide training.

The core challenge in open-domain QA is to accurately identify and utilize relevant documents from a massive knowledge base, enabling precise answer generation. Existing methods struggle with the joint optimization of retrieval and reading components, often due to the intractability of marginalizing over the latent document set. This leads to suboptimal relevance estimation, limiting overall system performance. Additionally, reliance on supervised retriever training with labeled data is resource-intensive, hindering scalability. Addressing these issues requires a framework capable of learning relevance without explicit labels, efficiently handling large-scale data, and enabling seamless integration of retrieval and comprehension modules.

This paper introduces EMDR2, which innovatively applies the EM algorithm to jointly train a neural retriever and a multi-document reader. Key innovations include: 1) modeling the document set as a latent variable, 2) estimating this set via prior and approximate posterior distributions, 3) integrating these estimates into a unified training loop, 4) employing a dual-encoder retriever with dot-product scoring, and 5) using a FiD-based reader that encodes multiple documents independently for joint answer generation. These innovations enable the model to learn relevance patterns implicitly, improve multi-document relevance estimation, and optimize both components simultaneously, overcoming the limitations of staged training and explicit supervision.

• �� Initialization: pre-train the retriever with unsupervised inverse cloze tasks, producing initial relevance scores. • Pseudo-labeling: generate candidate document sets using current retriever scores, selecting top-K documents. • EM steps: in the E-step, estimate the posterior distribution of the document set Z based on current model outputs; in the M-step, update retriever parameters to maximize the approximate posterior probability, and update reader parameters based on the predicted answer likelihood. • Approximate inference: use a temperature-scaled softmax to compute the posterior, simplifying the combinatorial complexity. • Parameter updates: alternate between estimating Z and optimizing model parameters via gradient descent, ensuring the model learns relevance implicitly. • Final training: perform end-to-end optimization, with stop-gradient operations to stabilize training and improve efficiency.

Experiments conducted on NQ, TriviaQA, and WebQuestions datasets, with 50 retrieved documents per question. The models are based on T5-base, trained with Adam optimizer, batch size 64, over 10-20 epochs. The retrieval index uses asynchronous distributed MIPS for efficiency. Baselines include staged FiD models with supervised retriever initialization. Evaluation metrics focus on exact match scores, with ablation studies varying the number of retrieved documents and initialization strategies. The training process involves iterative EM steps, with document embeddings periodically refreshed to prevent staleness. Results are compared against recent state-of-the-art models, demonstrating consistent improvements.

EMDR2 surpasses all comparable models, achieving 52.5% EM on NQ, 71.4% on TriviaQA, and 48.7% on WebQuestions, outperforming FiD and FiD-KD by 2-3 points. It exhibits robustness to unsupervised retriever initialization, with recall improving by over 20%. The method reduces training complexity by requiring only a single end-to-end cycle, unlike multi-stage distillation approaches. Ablation results confirm that modeling multiple documents jointly via EM significantly boosts performance, especially in low-resource or constrained retrieval settings.

This framework can be applied in large-scale search engines, virtual assistants, and knowledge-based AI systems, where accurate retrieval of relevant information is critical. Its ability to learn relevance implicitly makes it suitable for scenarios with limited labeled data. Additionally, it can be extended to multilingual and multimodal QA, as well as integrated with reasoning modules for complex inference tasks, broadening its industrial and research impact.

Despite its strengths, EMDR2 requires substantial computational resources for training, especially with large knowledge bases. The approximate posterior estimation may be less accurate in noisy or out-of-distribution scenarios, affecting retrieval quality. Its current validation is limited to English Wikipedia, and adaptation to other languages or modalities remains a challenge. Future work should focus on efficiency improvements, cross-lingual extensions, and interpretability enhancements to facilitate broader deployment.