UniKGQA: Unified Retrieval and Reasoning for Solving Multi-hop Question Answering Over Knowledge Graph
UniKGQA unifies retrieval and reasoning in a single architecture, achieving significant improvements in multi-hop KGQA accuracy, with over 75% Hits@1 on WebQSP.
Key Findings
Methodology
UniKGQA integrates a pre-trained language model (PLM) for question-relation semantic matching and an information propagation module to pass matching signals along directed KG edges. It employs a shared pre-training task based on question-relation pairs, enhancing relevance modeling. During fine-tuning, the model optimizes retrieval and reasoning stages jointly using KL divergence losses, with abstract subgraphs reducing search space. The architecture enables parameter sharing, facilitating effective transfer from retrieval to reasoning, validated through extensive experiments on datasets like WebQSP and CWQ, outperforming state-of-the-art baselines.
Key Results
- On WebQSP, UniKGQA achieves 75.1% Hits@1, outperforming NSM by 6.4%; on CWQ, 50.7%, surpassing PullNet and NSM. The model demonstrates superior performance especially in complex multi-hop scenarios, with significant improvements in answer coverage and reasoning accuracy. Ablation studies confirm the importance of pre-training and the effectiveness of abstract subgraphs. The shared architecture allows efficient parameter transfer, leading to faster convergence and better generalization across datasets.
- Across various subgraph sizes, UniKGQA maintains high answer coverage, outperforming heuristic methods like PPR. Fine-tuning strategies further improve robustness, with the model showing resilience to different relation path lengths. The experiments validate that the unified approach effectively captures long-distance dependencies, reducing error propagation common in pipeline models. Parameter sharing reduces training time without sacrificing accuracy, confirming the architecture's efficiency.
- The model excels in multi-hop reasoning, accurately identifying long relation chains and reducing false positives. Its transferability across datasets demonstrates strong generalization, and the parameter sharing approach simplifies deployment. Overall, UniKGQA sets new benchmarks in multi-hop KGQA performance, especially in challenging scenarios involving complex relation paths.
Significance
This work addresses the longstanding challenge of integrating retrieval and reasoning in multi-hop knowledge graph question answering. By unifying these stages within a single architecture and training paradigm, it overcomes the limitations of pipeline approaches that often treat them separately. The resulting system significantly improves answer accuracy and efficiency, paving the way for more intelligent and scalable question-answering systems. Such advancements are crucial for deploying knowledge-based AI in real-world applications like virtual assistants, enterprise search, and automated reasoning platforms. The approach also opens new research directions in joint modeling and end-to-end training for complex reasoning tasks.
Technical Contribution
UniKGQA introduces a novel unified architecture combining question-relation semantic matching with information propagation, enabling deep integration of retrieval and reasoning. The shared pre-training task based on contrastive learning enhances relevance modeling, while the use of abstract subgraphs reduces computational complexity. Parameter sharing allows effective transfer learning, reducing training time and improving robustness. The architecture's modular design facilitates extensions to multi-modal data and dynamic knowledge graphs. Empirical results demonstrate superior performance over existing pipeline models, establishing a new standard in multi-hop KGQA.
Novelty
This is the first work to unify retrieval and reasoning stages in multi-hop KGQA through a single model architecture and shared training objectives. Unlike prior methods that optimize these stages separately, UniKGQA leverages a PLM-based semantic matching and information propagation to enable end-to-end relevance modeling. The concept of abstract subgraphs further distinguishes it by reducing search space without losing critical information, facilitating efficient long-distance reasoning. This holistic approach marks a significant step forward in the field.
Limitations
- The model heavily relies on large-scale pretraining data, which may limit applicability in low-resource languages or domains.
- Abstract subgraphs, while reducing complexity, may omit some relevant nodes in highly intricate graphs, affecting recall.
- Computational costs for training and inference remain high, requiring further optimization for real-time deployment.
Future Work
Future research will explore multi-modal extensions, integrating visual and textual data for richer reasoning. Enhancing model efficiency through pruning and approximation techniques is also a priority. Additionally, adapting the architecture for dynamic, evolving knowledge graphs and improving interpretability will be key directions to broaden practical applicability.
AI Executive Summary
The rapid growth of large-scale knowledge graphs has propelled the development of multi-hop question answering (QA) systems, aiming to extract precise answers from complex relational data. Traditional approaches typically adopt a pipeline: first retrieving a relevant subgraph, then reasoning over it. However, this separation often leads to suboptimal performance due to mismatched objectives and limited information sharing. Addressing this challenge, the present work introduces UniKGQA, a unified model architecture that seamlessly integrates retrieval and reasoning stages. By leveraging a pre-trained language model (PLM) for question-relation semantic matching and an information propagation module, UniKGQA effectively models long-distance dependencies within the graph. The architecture employs shared pre-training tasks based on contrastive learning, which align relevance signals across both stages. During fine-tuning, the model optimizes retrieval and reasoning jointly using KL divergence losses, with abstract subgraphs reducing search space and computational costs. Extensive experiments on datasets such as WebQSP and CWQ demonstrate that UniKGQA surpasses existing state-of-the-art methods, achieving over 75% Hits@1 on WebQSP and notable improvements in complex multi-hop scenarios. This unified approach not only enhances accuracy but also simplifies the pipeline, enabling better transferability and robustness. The research signifies a major step forward in knowledge graph question answering, with promising implications for deploying intelligent, scalable QA systems in real-world applications. Future directions include multi-modal reasoning, dynamic graph adaptation, and efficiency optimization, aiming to further broaden the impact of this innovative framework.
Deep Dive
Abstract
Multi-hop Question Answering over Knowledge Graph~(KGQA) aims to find the answer entities that are multiple hops away from the topic entities mentioned in a natural language question on a large-scale Knowledge Graph (KG). To cope with the vast search space, existing work usually adopts a two-stage approach: it first retrieves a relatively small subgraph related to the question and then performs the reasoning on the subgraph to find the answer entities accurately. Although these two stages are highly related, previous work employs very different technical solutions for developing the retrieval and reasoning models, neglecting their relatedness in task essence. In this paper, we propose UniKGQA, a novel approach for multi-hop KGQA task, by unifying retrieval and reasoning in both model architecture and parameter learning. For model architecture, UniKGQA consists of a semantic matching module based on a pre-trained language model~(PLM) for question-relation semantic matching, and a matching information propagation module to propagate the matching information along the directed edges on KGs. For parameter learning, we design a shared pre-training task based on question-relation matching for both retrieval and reasoning models, and then propose retrieval- and reasoning-oriented fine-tuning strategies. Compared with previous studies, our approach is more unified, tightly relating the retrieval and reasoning stages. Extensive experiments on three benchmark datasets have demonstrated the effectiveness of our method on the multi-hop KGQA task. Our codes and data are publicly available at~\url{https://github.com/RUCAIBox/UniKGQA}.