Hybrid-IR: Dual-Path Hybrid Retrieval with Iterative Reasoning for Complex Medical Question Answering
Hybrid-IR combines graph-based and dense retrieval with iterative reasoning, achieving 10% accuracy improvement on medical QA benchmarks.
Key Findings
Methodology
The approach designs a dual-path retrieval framework integrating structured knowledge exploration via a knowledge graph index (KG-index) and unstructured semantic matching through dense vector retrieval. It employs a question decomposition module using LLMs, followed by graph propagation with GFM-RAG-8M to model entity dependencies. Results from both paths are fused via Reciprocal Rank Fusion, enabling a dynamic 'retrieve-reason' loop. This iterative process accumulates evidence and refines reasoning trajectories across multiple rounds, effectively supporting complex multi-hop biomedical questions.
Key Results
- On USMLE, MedMCQA, and MMLU-Med datasets, Hybrid-IR improves average accuracy by over 10%, outperforming single-path retrieval and existing multi-modal methods. For example, with Llama-3-8B-Instruct, accuracy increased from 60.7% to 67.2%, and similar gains were observed with GPT-4o-mini, demonstrating robustness across models.
- Multi-round iterative retrieval significantly enhances evidence chain completeness, enabling deeper reasoning and better evidence integration, especially in complex multi-hop questions.
- The method effectively overcomes static retrieval limitations by leveraging knowledge graphs to guide structured reasoning, reducing evidence omission and increasing interpretability.
Significance
This work addresses core challenges in biomedical question answering—knowledge fragmentation and deep reasoning—by integrating structured and unstructured knowledge sources within an iterative framework. It advances AI capabilities in medical diagnostics, clinical decision support, and knowledge synthesis, offering a scalable solution for complex reasoning tasks. The proposed Hybrid-IR framework paves the way for more reliable, explainable, and comprehensive AI-driven medical systems, with potential impacts on healthcare quality and efficiency.
Technical Contribution
The key technical innovation lies in combining graph-structured knowledge with dense semantic retrieval within a multi-round retrieval-reasoning loop. The use of GFM-RAG for knowledge propagation, along with RRF for fusion, creates a robust, flexible system capable of handling complex, multi-hop biomedical questions. This approach surpasses existing single-path or static retrieval methods, providing a new paradigm for knowledge integration and reasoning in AI medical applications.
Novelty
This is the first framework to unify graph-based structured knowledge and dense semantic retrieval within an iterative reasoning process tailored for biomedical QA. Unlike prior work relying on a single retrieval paradigm, Hybrid-IR dynamically refines evidence through multiple retrieval and reasoning cycles, significantly improving performance on complex tasks.
Limitations
- Despite its strengths, the approach depends heavily on the quality and coverage of the knowledge graph; incomplete or noisy graphs may impair reasoning accuracy.
- Multi-round retrieval increases computational load, which could hinder real-time application in clinical settings.
- The current system relies on pre-trained models without domain-specific fine-tuning, limiting its adaptability to highly specialized medical subfields.
Future Work
Future directions include automatic expansion and updating of the knowledge graph to cover emerging medical knowledge, integrating multi-modal data such as medical images, and optimizing inference speed for real-time clinical deployment. Further research will explore domain-specific fine-tuning and explainability enhancements to improve trust and usability in healthcare environments.
AI Executive Summary
The rapid advancement of large language models (LLMs) has opened new horizons in biomedical question answering, yet significant challenges remain. Existing systems often struggle with fragmented medical knowledge and lack the capacity for deep, multi-step reasoning necessary for complex clinical scenarios. Traditional retrieval-augmented methods, relying on a single retrieval path, fail to capture the full spectrum of structured relationships and fine-grained semantics, limiting their effectiveness.
To address these issues, this study introduces Hybrid-IR, a dual-path retrieval framework that combines graph-based structured knowledge exploration with dense semantic matching. Central to this approach is a knowledge graph index (KG-index) that explicitly links dispersed entities and relations across documents, enabling structured reasoning. Simultaneously, dense vector retrieval captures nuanced textual semantics, ensuring comprehensive evidence coverage. The core innovation lies in embedding this dual retrieval within an iterative 'retrieve-reason' loop, where each cycle refines evidence and updates reasoning trajectories based on accumulated knowledge.
Extensive experiments on three prominent medical QA benchmarks—USMLE-style questions, MedMCQA, and MMLU-Med—demonstrate that Hybrid-IR consistently outperforms existing methods, achieving over 10% accuracy improvements. For instance, with the Llama-3-8B-Instruct backbone, accuracy rose from 60.7% to 67.2%, validating the framework’s robustness and effectiveness. The multi-round iterative process significantly enhances the depth and reliability of reasoning, especially in multi-hop scenarios, by dynamically guiding retrieval based on previous evidence.
This approach marks a substantial step forward in AI-driven medical reasoning, offering a scalable, interpretable, and more accurate solution for complex biomedical questions. It addresses core limitations of static retrieval and knowledge fragmentation, paving the way for smarter, more trustworthy AI medical assistants. Nonetheless, challenges such as knowledge graph completeness and computational costs remain, guiding future research toward more adaptive, real-time, multi-modal, and domain-specific systems. Overall, Hybrid-IR exemplifies a promising direction for integrating structured knowledge and deep reasoning in healthcare AI.
Deep Dive
Abstract
Large language models (LLMs) have shown promising performance across a wide range of biomedical applications, including medical question answering (QA), yet they remain prone to hallucinations and outdated knowledge. Although retrieval-augmented generation (RAG) can alleviate this issue by incorporating external documents, there still exist two fundamental limitations. First, medical knowledge is often fragmented across documents, while most RAG methods rely on a single retrieval path, which makes it challenging to jointly preserve fine-grained semantic information and structured global associations. Second, static retrieval strategies are typically insufficient to support deep reasoning that is important in complex medical QA. In this paper, we present a dual-path retrieval framework with an iterative retrieval-reasoning mechanism termed "Hybrid-IR" for complex medical QA. The proposed Hybrid-IR integrates graph-based retrieval for exploration of structured knowledge and dense retrieval for fine-grained semantic matching. Moreover, the reasoning trajectory can be progressively refined through an iterative retrieve-reason loop. Experiments on three widely used medical QA benchmarks demonstrate the effectiveness of our Hybrid-IR.