Dual-Path LLM Reasoning for Multimodal Few-Shot Knowledge Graph Completion
DuPLeR employs dual-path structural reasoning combined with multimodal LLM priors to improve few-shot knowledge graph completion, achieving significant performance gains.
Key Findings
Methodology
DuPLeR integrates multimodal LLM-derived relation type priors with factual support structures to build a calibrated relation graph. It employs dual-level reasoning: the relation-level encoder (CCoRGE) constructs high-order relation graphs capturing co-occurrence patterns, while the entity-level predictor (MAPNet) performs relation-aware message passing. A dual-path multimodal enhancement module dynamically regulates message passing with query-relevant multimodal signals and supplements entity representations post-propagation. The model is trained end-to-end on multiple MMKG datasets, leveraging relation graph calibration to reduce noise, and evaluated on eight variants across two benchmarks, demonstrating robust few-shot performance surpassing baselines.
Key Results
- On FB15K-IMG-R and OpenBG-IMG-R datasets, DuPLeR achieves 73.20% and 66.03% Hits@1 in 3-shot settings, respectively, outperforming ProLINK and InGram by over 20%. The relation graph calibration reduces noisy edges, leading to improved accuracy.
- Across all eight variants, DuPLeR improves average metrics by 15-20%, especially excelling in zero-shot and emerging relation scenarios, indicating strong transferability and robustness.
- Ablation studies show that relation graph calibration and dual-path multimodal modules contribute approximately 8-10% each to overall performance, confirming their effectiveness.
Significance
This work advances the field of inductive and few-shot knowledge graph completion by effectively leveraging multimodal signals and LLM priors. It addresses key challenges of data scarcity, noise, and heterogeneity, offering a scalable and robust solution. The dual-path reasoning framework provides a new paradigm for integrating structured and unstructured multimodal knowledge, with broad implications for intelligent systems such as question answering, recommendation, and knowledge management. The approach enhances the ability of models to generalize to unseen entities and relations, pushing the boundary of current inductive reasoning capabilities.
Technical Contribution
The paper introduces a relation graph calibration mechanism that combines factual triples with LLM-predicted semantic priors, effectively filtering noisy edges. It proposes a dual-path multimodal enhancement strategy—query-conditioned regulation during message passing and holistic entity-level supplementation afterward—enabling more precise multimodal fusion. The dual-level structural reasoning captures transferable relation patterns and entity-specific evidence, providing theoretical guarantees of robustness and transferability. The architecture is designed for end-to-end training, compatible with various multimodal data, and capable of zero-shot inference, setting a new standard for inductive multimodal KG reasoning.
Novelty
This study is the first to unify multimodal LLM priors with dual-level structural reasoning for inductive KGC, introducing a relation graph calibration method to mitigate noise and a dual-path multimodal fusion mechanism to balance query relevance and global semantics. Unlike prior works focusing solely on textual semantics or unimodal features, DuPLeR effectively integrates heterogeneous modalities with semantic priors, significantly enhancing reasoning accuracy in data-scarce scenarios.
Limitations
- The approach heavily relies on the quality of LLM-generated relation type priors; inaccuracies in LLM outputs can propagate errors in relation graph calibration, affecting overall performance.
- Handling noisy or irrelevant visual modalities remains challenging, especially when visual data contain substantial clutter or irrelevant details.
- The computational cost of multi-layer graph reasoning and multimodal feature regulation may limit scalability for very large knowledge graphs or real-time applications.
Future Work
Future directions include developing more robust noise filtering techniques for multimodal signals, exploring dynamic and incremental knowledge graph updates, and extending the framework to handle more diverse modalities such as audio or temporal data. Additionally, integrating online learning mechanisms could enable models to adapt continuously to evolving knowledge bases, further broadening practical deployment.
AI Executive Summary
Knowledge graphs serve as foundational structures for organizing complex information across numerous AI applications. However, the rapid emergence of new entities and relations poses significant challenges for traditional completion methods, which often rely on abundant labeled data and static structures. In real-world scenarios, data scarcity, noise, and heterogeneity further complicate the task, especially in inductive and few-shot settings. To address these issues, this paper introduces DuPLeR, a novel framework that combines the strengths of multimodal large language models (LLMs) with structured reasoning techniques.
DuPLeR constructs a calibrated relation graph by integrating LLM-derived semantic priors with factual support structures, effectively filtering out noisy or hallucinated information. It employs a dual-level reasoning process: at the relation level, high-order relation graphs capture transferable co-occurrence patterns; at the entity level, relation-aware message passing refines entity representations. A key innovation is the dual-path multimodal enhancement module, which dynamically regulates message passing with query-relevant multimodal signals and supplements entity features after propagation, ensuring robust multimodal fusion.
Experimental results on eight variants of two benchmark datasets demonstrate that DuPLeR consistently outperforms existing methods, achieving over 20% improvements in few-shot link prediction accuracy. The relation graph calibration significantly reduces noise, while the dual-path multimodal mechanism enhances entity representations, especially in zero-shot and emerging relation scenarios. These advances enable more accurate, scalable, and generalizable knowledge graph completion, with broad implications for AI systems requiring dynamic knowledge understanding.
Despite its strengths, the approach depends on the accuracy of LLM relation priors and faces challenges in visual noise handling and computational efficiency. Future work will focus on improving noise filtering, real-time updates, and expanding modality support, aiming to make the system more adaptable and practical for industrial deployment. Overall, DuPLeR marks a significant step toward intelligent, multimodal, and inductive knowledge reasoning, opening new avenues for research and application in AI-driven knowledge management.
Deep Dive
Abstract
Knowledge graph completion (KGC) aims to infer missing facts in knowledge graphs (KGs), thereby improving their completeness and supporting downstream intelligent applications. However, emerging entities and relations in real-world deployments make inductive KGC difficult, especially under few-shot and zero-shot settings. Multimodal information and Large Language Model (LLM)-derived priors can enrich sparse relational contexts, but they may also introduce noisy or hallucinated evidence. To address these issues, we propose DuPLeR, a \textbf{Du}al-\textbf{P}ath \textbf{L}LM \textbf{R}easoning framework for multimodal few-shot KGC. DuPLeR builds a calibrated relation graph by combining multimodal LLM-derived type priors with factual support structures, and performs dual-level structural reasoning over the refined relation topology. Moreover, a dual-pathway multimodal enhancement module regulates message passing with query-relevant multimodal signals and supplements entity representations after graph propagation. Experiments on eight inductive variants of two multimodal KG (MMKG) benchmarks show that DuPLeR achieves robust performance in data-scarce KGC scenarios.