Reading Wikipedia to Answer Open-Domain Questions

The evolution of NLP has seen rapid progress in machine comprehension and information retrieval, driven by datasets like SQuAD and advances in neural architectures such as attention mechanisms and memory networks. Early systems like IBM DeepQA integrated multiple data sources but faced scalability issues. Recent developments focus on end-to-end neural models that directly map questions to answers, leveraging large-scale pretraining and sophisticated attention. However, these models often assume access to relevant passages, limiting their applicability in open-domain settings. The challenge remains to efficiently search and understand massive unstructured corpora like Wikipedia, which contains constantly evolving knowledge. Addressing this gap is critical for building scalable, real-time QA systems capable of handling diverse questions across domains.

The core challenge in open-domain QA is twofold: first, rapidly retrieving relevant documents from millions of candidates; second, accurately understanding and extracting answer spans within those documents. Existing methods either rely on static knowledge bases, which are incomplete and outdated, or simple retrieval models that lack deep comprehension. Balancing retrieval efficiency with understanding depth is difficult, especially when questions require nuanced inference or multi-hop reasoning. Additionally, the high computational cost of neural models hampers real-time deployment. These limitations hinder the development of truly scalable, accurate open-domain QA systems capable of handling diverse, complex queries in dynamic environments.

The paper introduces several key innovations: 1) a fast, scalable document retriever based on bigram hashing and TF-IDF, capable of filtering relevant Wikipedia articles efficiently; 2) a deep neural reader employing multi-layer bidirectional LSTMs with attention mechanisms to encode context and question, predicting answer spans with high accuracy; 3) a multi-task training framework combining datasets like SQuAD, WebQuestions, and WikiMovies, enhanced by distant supervision to leverage unannotated data; 4) an end-to-end pipeline that seamlessly integrates retrieval and comprehension, enabling the system to operate at scale with minimal manual intervention. These innovations collectively address the bottlenecks of existing QA systems, offering a practical solution for large-scale deployment.

• �� Document retrieval: utilize bigram hashing and TF-IDF to quickly identify top 5 Wikipedia articles relevant to the question. • Paragraph encoding: represent tokens with GloVe embeddings, then pass through multi-layer bidirectional LSTM to capture contextual information. • Question encoding: encode question tokens with a single-layer LSTM, apply attention to align with paragraph representations. • Answer span prediction: use bilinear classifiers to independently predict start and end positions within paragraphs, constrained to a maximum span length of 15 tokens. • Training: employ multi-task learning across datasets, with distant supervision for unannotated data, optimizing span prediction accuracy. • System integration: combine retrieval and reading modules into an end-to-end pipeline, enabling real-time question answering from Wikipedia.

The system is evaluated on SQuAD, WebQuestions, WikiMovies, and CuratedTREC datasets. Retrieval performance is measured by coverage of answer-containing articles within top 5 results, reaching over 85%. The reading model achieves 70.0% EM and 79.0% F1 on SQuAD, outperforming previous models. Multi-task training improves generalization across datasets, validated through ablation studies that highlight the importance of alignment features. The end-to-end system demonstrates robustness in open-domain settings, with significant improvements over baseline retrieval and comprehension methods. Hyperparameters such as hidden size (128 units) and span length limit (15 tokens) are tuned for optimal performance.

DrQA achieves 70.0% EM and 79.0% F1 on SQuAD, surpassing prior models like BiDAF and R-net. Retrieval module outperforms Wikipedia's internal search engine, with over 85% coverage in top 5 articles. Multi-task learning enhances performance on WebQuestions and WikiMovies, demonstrating cross-domain robustness. Ablation results show that removing alignment features drops F1 by 2-3%, confirming their importance. The combined pipeline effectively balances speed and accuracy, validating the design choices and training strategies.

This system can be integrated into virtual assistants, enterprise search engines, and online customer service bots, providing accurate answers from Wikipedia in real-time. It supports multilingual expansion and can be adapted for domain-specific corpora, such as medical or legal texts. The approach enables scalable, cost-effective knowledge access, reducing reliance on manual curation and enabling dynamic updates. Long-term, it could underpin fully autonomous AI systems capable of reasoning over diverse unstructured data sources, transforming information retrieval and decision-making processes.

Despite high accuracy, the system struggles with questions requiring multi-hop reasoning or involving complex inference chains. Its performance degrades with noisy or ambiguous queries, indicating a need for improved disambiguation and reasoning modules. Computational costs remain high due to neural network complexity, limiting deployment in resource-constrained environments. Future work should focus on model compression, reasoning extension, and multilingual support to address these limitations.