Reasoning over Hierarchical Question Decomposition Tree for Explainable Question Answering
Proposes RoHT framework with hierarchical question decomposition tree, achieving 29.7% EM improvement on KQA Pro, surpassing SOTA.
Key Findings
Methodology
The RoHT framework consists of two stages: first, constructing a Hierarchical Question Decomposition Tree (HQDT) to capture the semantic structure of complex questions; second, performing recursive probabilistic reasoning over HQDT, integrating knowledge from KB and text. The scheduler dynamically selects the appropriate knowledge source for each sub-question, while executors retrieve answers from KB or text, and an aggregator fuses these answers based on their probabilities. This approach leverages question decomposition and probabilistic inference, significantly enhancing complex QA performance.
Key Results
- On KQA Pro and Musique datasets, RoHT achieves 29.7% and 45.8% EM improvements respectively over previous SOTA, with notable F1 gains of 11.3% on Musique. These results demonstrate the effectiveness of multi-source knowledge integration via hierarchical decomposition.
- Compared to TransferNet and other end-to-end models, RoHT's explicit question structure and probabilistic reasoning yield superior accuracy, especially on multi-hop and relation-intensive questions.
- Ablation studies confirm that the hierarchical question tree and recursive probabilistic inference are critical for performance gains, particularly in handling complex, multi-layered queries.
Significance
This work addresses the key challenge of integrating heterogeneous knowledge sources for complex question answering, providing a structured, interpretable framework that enhances reasoning depth and transparency. By explicitly modeling question semantics and probabilistically fusing multi-source information, RoHT advances both academic understanding and practical deployment of explainable AI systems, paving the way for more robust and trustworthy intelligent assistants.
Technical Contribution
The paper introduces the novel Hierarchical Question Decomposition Tree (HQDT) for representing complex questions, combined with a probabilistic recursive inference mechanism. The design of the scheduler, executors, and aggregator modules enables dynamic, multi-source knowledge fusion, setting a new standard for interpretability and flexibility in complex QA. This approach differs fundamentally from prior graph-based or end-to-end models by explicitly modeling question structure and reasoning uncertainty.
Novelty
This is the first work to systematically utilize a hierarchical question decomposition tree for multi-source knowledge integration in complex QA. Unlike previous methods that focus on flat question decomposition or single knowledge sources, RoHT models the multi-layer semantic structure and performs probabilistic reasoning, offering a more flexible and interpretable solution to multi-hop, multi-relation questions.
Limitations
- The approach relies heavily on the accuracy of question decomposition; errors in the HQDT can propagate, especially in ambiguous or poorly formulated questions.
- Computational cost remains high due to recursive reasoning and multi-source retrieval, limiting scalability in real-time applications.
- Dependence on training data quality for question decomposer and generator may affect robustness across diverse domains.
Future Work
Future directions include improving the robustness of question decomposition, integrating multi-modal data (images, videos), and optimizing inference efficiency. Expanding the framework to handle real-time, large-scale knowledge bases and exploring unsupervised or semi-supervised training strategies will further enhance practical deployment.
AI Executive Summary
Complex question answering (QA) remains a significant challenge in AI, especially when integrating multiple knowledge sources like structured databases and unstructured texts. Traditional methods often struggle with the semantic complexity and multi-hop reasoning required for such tasks. To address this, the authors propose the RoHT framework, which constructs a Hierarchical Question Decomposition Tree (HQDT) to explicitly model the semantic structure of complex questions. This tree-based representation enables the system to decompose questions into manageable sub-questions, each associated with specific knowledge sources.
The core innovation lies in combining this structured decomposition with a probabilistic recursive reasoning process. The framework employs a scheduler to determine whether to answer sub-questions via knowledge bases, text, or further decomposition, while executors retrieve answers using specialized models like KB executors and reading comprehension models. An aggregator then fuses these answers, considering their probabilities, to produce the final response.
Experimental results on the KQA Pro and Musique datasets demonstrate that RoHT significantly outperforms existing state-of-the-art models, achieving up to 29.7% EM improvement and 11.3% F1 gain. These results validate the effectiveness of explicit hierarchical question modeling and probabilistic reasoning in multi-source knowledge integration. The approach not only improves accuracy but also enhances interpretability, making the system's reasoning process transparent.
This work has broad implications for advancing AI's reasoning capabilities in complex, real-world scenarios, such as multi-hop knowledge inference, multi-modal data fusion, and explainable AI. Future efforts will focus on reducing computational costs, extending multi-modal integration, and improving robustness across domains, aiming to deploy more intelligent, trustworthy systems in practical applications.
Deep Dive
Abstract
Explainable question answering (XQA) aims to answer a given question and provide an explanation why the answer is selected. Existing XQA methods focus on reasoning on a single knowledge source, e.g., structured knowledge bases, unstructured corpora, etc. However, integrating information from heterogeneous knowledge sources is essential to answer complex questions. In this paper, we propose to leverage question decomposing for heterogeneous knowledge integration, by breaking down a complex question into simpler ones, and selecting the appropriate knowledge source for each sub-question. To facilitate reasoning, we propose a novel two-stage XQA framework, Reasoning over Hierarchical Question Decomposition Tree (RoHT). First, we build the Hierarchical Question Decomposition Tree (HQDT) to understand the semantics of a complex question; then, we conduct probabilistic reasoning over HQDT from root to leaves recursively, to aggregate heterogeneous knowledge at different tree levels and search for a best solution considering the decomposing and answering probabilities. The experiments on complex QA datasets KQA Pro and Musique show that our framework outperforms SOTA methods significantly, demonstrating the effectiveness of leveraging question decomposing for knowledge integration and our RoHT framework.