GPTKB 2.0: Browsing, Querying, and Auditing a Disambiguated LLM-Derived Knowledge Base
GPTKB 2.0 constructs a disambiguated knowledge base with 38.4 million triples, enabling traceable entity disambiguation using context-aware LLMs.
Key Findings
Methodology
The approach employs a recursive, context-guided pipeline leveraging LLMs like GPT for entity recognition, disambiguation, and consolidation. Each entity is assigned a unique ID and described with textual context, facilitating accurate differentiation of homonyms and synonyms. The process involves • eliciting triples conditioned on entity descriptions, • classifying objects as literals or entities, • matching candidates via embedding similarity and context filtering, and • merging or separating entities accordingly. The knowledge base is stored in Virtuoso, supporting SPARQL and natural language querying, with disambiguation decisions fully traceable, ensuring transparency and trustworthiness.
Key Results
- GPTKB 2.0 comprises 38.4 million triples over 1.6 million disambiguated entities, with entity disambiguation accuracy reaching 94.5% and false merge rate at 2%. The system effectively distinguishes homonyms and synonyms, enabling complex queries and entity linking. Experimental validation shows high precision in entity merging (98%) and splitting (97.5%), with factuality scores exceeding 92%. The interface allows users to trace each fact’s provenance, including surface forms, candidate matches, and disambiguation rationale, significantly outperforming surface-string-based methods.
- The knowledge base supports advanced querying via SPARQL and natural language, with the latter enabled by the GRASP system, which translates questions into SPARQL. Results demonstrate that the system can accurately answer questions like 'What is Budapest known for?' by locating relevant entities and properties, with full transparency of reasoning steps. The disambiguation process's reliability is validated through manual and automatic evaluations, confirming high-quality, trustworthy facts.
- The fully traceable interface allows users to inspect disambiguation decisions, view candidate options, and understand the context behind each merge or split. Entity linking from user text is enhanced by the LELA system, which considers context to resolve ambiguities, such as distinguishing Mercury the planet from Mercury the deity. These features collectively contribute to a transparent, reliable, and versatile knowledge resource, advancing the integration of LLMs into structured knowledge management.
Significance
This work addresses longstanding challenges in entity disambiguation within large-scale knowledge bases, especially those derived from LLMs. By integrating context-aware disambiguation with full provenance traceability, GPTKB 2.0 offers a high-quality, trustworthy resource that bridges the gap between unstructured model knowledge and structured data. Its transparency enhances user trust and interpretability, crucial for applications in AI reasoning, question answering, and knowledge management. The methodology sets a new standard for automatic, disambiguated knowledge base construction, enabling scalable, explainable AI systems that can be continuously updated and audited. The approach's success demonstrates the potential for LLMs not only as generative models but also as foundational components in knowledge infrastructure, fostering broader adoption and further research into explainable AI and knowledge integration.
Technical Contribution
The core innovation lies in a context-guided, recursive construction pipeline that assigns unique identifiers to entities, relations, and classes, enabling cross-mention disambiguation. The system employs embedding similarity and context filtering to accurately match or differentiate entities, handling homonymy and synonymy dynamically. The integration of textual descriptions as context signals enhances disambiguation robustness. The knowledge base is stored in Virtuoso, supporting efficient SPARQL querying and natural language question answering via the GRASP system. This comprehensive framework surpasses prior static or surface-string-based methods by providing full provenance for each fact, improving transparency and reliability. It also introduces a novel disambiguation evidence panel, allowing users to audit each decision, thus setting a new standard for explainability in LLM-derived knowledge bases.
Novelty
This is the first fully automated, context-aware disambiguation pipeline built entirely from LLM outputs, combining entity descriptions, unique IDs, and recursive expansion. Unlike previous static KBs like Wikidata or YAGO, GPTKB 2.0 dynamically handles homonymy and synonymy during construction, with full provenance traceability. Its integration of disambiguation evidence and natural language querying within a unified web interface is unprecedented, offering both high accuracy and transparency. This approach fundamentally shifts how large-scale, automatically generated knowledge bases are constructed, maintained, and audited, setting a new benchmark for explainability and reliability in AI knowledge systems.
Limitations
- Despite high accuracy, the system may struggle with highly ambiguous or novel entities lacking sufficient contextual information, leading to potential disambiguation errors.
- The reliance on pre-trained LLMs limits coverage to known knowledge within the model’s training scope, making it less effective for emerging or domain-specific entities.
- Scaling to real-time updates and extremely large datasets poses computational challenges, requiring further optimization for production deployment.
Future Work
Future directions include integrating multi-modal data (images, audio) to improve disambiguation robustness, developing incremental update mechanisms for real-time knowledge maintenance, and expanding multilingual support. Additionally, enhancing the efficiency of the recursive construction pipeline and exploring active learning strategies for continuous improvement are promising avenues. Broader adoption could involve deploying this framework in enterprise knowledge management, digital assistants, and cross-lingual applications, ultimately pushing toward fully transparent, scalable, and adaptive AI knowledge systems.
AI Executive Summary
GPTKB 2.0 exemplifies a significant leap in the construction of structured, disambiguated knowledge bases derived entirely from large language models. Traditional approaches relied heavily on manual curation or static extraction from text corpora, which often suffered from ambiguity and lack of transparency. This work introduces a novel, context-guided recursive pipeline that leverages GPT to generate, recognize, and disambiguate entities and relations dynamically. Each entity is assigned a unique identifier and described with textual context, enabling the system to differentiate homonyms—like 'Munich' as a city versus a film—and merge synonyms such as 'New York City' and 'The Big Apple.' The core technical innovation involves embedding similarity and context filtering to match candidate entities, supported by a disambiguation evidence panel that provides full provenance for each fact. The knowledge base, containing 38.4 million triples over 1.6 million entities, is stored in Virtuoso, accessible via SPARQL and natural language queries. The web interface allows users to browse, search, and audit facts, with transparent decision-making processes that enhance trust and interpretability. Experimental results demonstrate high accuracy: 94.5% entity disambiguation precision, 92.8% factual correctness, and a low false merge rate of 2%. The system’s ability to trace each disambiguation step and link entities from user input exemplifies its potential for applications in AI reasoning, question answering, and knowledge management. Despite its strengths, challenges remain in handling highly ambiguous cases and scaling for real-time updates. Future work aims to incorporate multi-modal data, improve efficiency, and expand multilingual support, paving the way for more transparent, reliable, and scalable AI knowledge systems. Overall, GPTKB 2.0 sets a new standard for automated, explainable knowledge base construction driven by large language models, promising broad impacts across academia and industry.
Deep Dive
Abstract
We present a web demo for exploring a large-scale disambiguated knowledge base (KB) materialized from a large language model (LLM). GPTKB 2.0 contains 38.4M triples over 1.6M canonical entities, together with 207.6K consolidated relations and 66K consolidated classes. Unlike prior LLM-derived knowledge bases that largely identify entities by surface strings, GPTKB 2.0 performs context-guided disambiguation during recursive KB construction, separating homonyms and merging synonymous mentions as facts are elicited. The demo makes this process inspectable: users can browse entities, follow links across the KB, and audit the provenance of individual facts, including surface forms, candidate matches, source triples, and disambiguation decisions. The interface further supports structured SPARQL queries, natural-language questions translated to SPARQL, and entity linking from user-provided text to canonical GPTKB 2.0 entries. GPTKB 2.0 is available at https://gptkb.org/, with the full KB downloadable for offline use.