LitCurate: A Configuration-Driven AI-Assisted Framework for Scientific Database Construction with an Application to Lower-Mantle Equation-of-State Data
LitCurate uses large language models within a configurable, stage-wise workflow to build a structured EOS database with 1334 entries from 205 papers.
Key Findings
Methodology
LitCurate employs a staged, auditable pipeline integrating literature discovery via OpenAlex, relevance screening, full-text processing, and schema-guided extraction using LLMs. Configurable YAML files specify search queries, screening criteria, extraction schemas, and prompts. Each stage produces intermediate artifacts and source provenance, stored in a run ledger. Multiple models and platforms (OpenAI, Anthropic, self-hosted) are supported, enabling flexible deployment. Extraction outputs are validated against JSON Schema, ensuring structured, traceable data. Final datasets are merged and accessible via a web interface.
Key Results
- Constructed a database with 1334 EOS entries from 205 papers, linking parameters to mineral phases, compositions, methods, and constraints. Extraction accuracy reached 89.1%, with a 97.6% F1 score, outperforming non-reasoning baselines. Literature retrieval achieved 74% coverage, demonstrating effective query and screening strategies. Model performance varied, with commercial models excelling in detection and accuracy, while open-weight models offered cost advantages.
- The framework's modular design allowed iterative reprocessing, ensuring high-quality, source-traceable data. The database supports scientific analysis, model input, and cross-study comparisons, significantly reducing manual effort.
- Model evaluations indicated high fidelity in numeric parameter extraction, with challenges remaining in formula and equation recognition, highlighting future directions for multimodal content understanding.
Significance
This framework addresses the longstanding bottleneck of transforming scattered literature data into structured, traceable databases, essential for data-driven earth sciences. It enhances transparency, reproducibility, and efficiency, enabling rapid updates and broad dissemination of scientific data. By automating and auditing each step, it bridges the gap between raw publications and usable datasets, fostering open science and collaborative research. Its adaptability across disciplines makes it a versatile tool for scientific data curation, supporting large-scale modeling and hypothesis testing in geophysics, materials science, and beyond.
Technical Contribution
The main technical innovation is the integration of configurable, schema-guided LLM workflows within a multi-stage pipeline supporting literature discovery, relevance screening, and structured extraction. The system maintains provenance and intermediate artifacts, enabling inspection and re-execution. It supports multiple LLM providers and models, with validation against JSON Schema, ensuring data integrity. The design emphasizes modularity, extensibility, and transparency, setting a new standard for automated scientific data curation.
Novelty
This is the first comprehensive open-source pipeline combining literature discovery with schema-guided LLM extraction, supporting full transparency and source traceability. Unlike prior tools limited to either document retrieval or partial extraction, LitCurate offers an end-to-end, auditable workflow adaptable to various scientific domains, emphasizing source provenance and user configurability.
Limitations
- Model performance in recognizing complex formulas and diverse reporting styles remains imperfect, leading to potential data omissions or inaccuracies.
- Processing large corpora demands significant computational resources, especially during full-text conversion and extraction stages.
- Handling non-standardized or multimodal content like figures and equations needs further enhancement, requiring integration of multimodal understanding techniques.
Future Work
Future efforts will focus on integrating multimodal content understanding, especially for figures and equations, to improve formula recognition. Expanding the framework to other scientific parameters and disciplines will enhance its versatility. Optimization of computational efficiency, possibly via distributed processing or active learning, will be prioritized to handle larger datasets. Additionally, developing user-friendly interfaces for schema customization and manual review will facilitate broader adoption.
AI Executive Summary
Scientific literature contains a wealth of experimental and computational data crucial for understanding Earth's interior, but much of this information remains locked in unstructured formats within publications. Traditional manual curation is slow, error-prone, and difficult to scale, creating a bottleneck for large-scale data-driven modeling. Existing automated tools lack transparency and source traceability, limiting their utility for rigorous scientific analysis.
To address this challenge, the authors introduce LitCurate, an open-source, configuration-driven framework that leverages large language models (LLMs) within a staged, auditable pipeline. The system begins with literature discovery via OpenAlex, followed by relevance screening, full-text processing, and schema-guided extraction. Each stage produces intermediate artifacts and source provenance, stored in a run ledger, enabling inspection, reprocessing, and transparency. The pipeline is highly configurable via YAML files, supporting multiple LLM providers and models, with validation against JSON Schema to ensure data quality.
Applied to construct a comprehensive database of lower-mantle EOS parameters, LitCurate extracted 1334 entries from 205 papers spanning 1990–2025. The dataset links parameters such as V0, K0, and K′0 to mineral phases, compositions, methods, and constraints, with accuracy metrics showing 89.1% detection rate and 97.6% F1 score. The database is accessible through a web interface, facilitating scientific analysis, model input, and cross-study comparisons.
This work significantly advances automated scientific data curation, providing a reusable, transparent, and scalable approach. It bridges the gap between unstructured literature and structured datasets, supporting the needs of earth scientists, materials researchers, and computational modelers. Future developments will focus on multimodal content understanding, expanding parameter coverage, and optimizing workflow efficiency, aiming to foster open, reproducible science in the digital age.
Deep Dive
Plain Language Accessible to non-experts
Imagine you have a huge library filled with thousands of books about the Earth's interior. Finding the right information manually is like flipping through every page, which takes forever. LitCurate is like having a smart robot assistant that can search through all these books quickly, pick out the important parts, and organize the facts into a neat digital spreadsheet. This robot not only finds the data but also keeps track of where each piece came from, so you can always check the original source. With this system, scientists can get the information they need faster and more reliably, helping them understand the deep Earth without spending years reading every paper. It’s like having a super-efficient librarian who works 24/7, making research faster and more accurate.
ELI14 Explained like you're 14
Think of a giant school library with thousands of books about how the Earth works deep underground. If you wanted to find specific facts, like how squishy rocks are under pressure, you'd have to spend hours flipping through pages and taking notes. Now, imagine having a super-smart robot friend that can do all that for you. You tell it what you're looking for, and it searches all the books, finds the right pages, and writes down the facts in a neat list. Plus, it keeps track of exactly which book and page each fact came from, so you can double-check if needed. This way, scientists can get all the important info they need in a fraction of the time, making discoveries faster and easier. It’s like having a personal research assistant who never gets tired!
Glossary
Large Language Models (大规模语言模型)
基于深度学习的模型，能理解和生成自然语言，用于自动信息提取。
用于指导文献内容的结构化提取。
Schema (架构)
定义数据结构、字段类型和验证规则，确保提取数据符合预期格式。
引导模型输出符合结构化要求的内容。
JSON Schema (JSON架构)
一种描述JSON数据结构的标准，用于验证模型输出的正确性。
确保提取结果的格式一致性。
OpenAlex (开放文献数据库)
一个开放的学术文献和作者信息数据库，用于文献检索。
作为文献检索的基础数据源。
Equation-of-State (状态方程)
描述材料在不同压力和温度下的物理性质的数学关系。
本研究的核心参数类型。
Open Questions Unanswered questions from this research
- 1 模型在识别复杂公式和多样化表达方面仍存在不足，未来需增强公式理解和多模态内容融合能力。
- 2 如何进一步提升大规模文献检索的效率和覆盖率，尤其在多学科交叉领域仍是挑战。
Applications
Immediate Applications
科学数据构建
研究人员可利用该框架快速生成结构化数据库，支持地球模型和材料研究。
模型输入优化
地球物理模拟中，自动提取的参数可作为高质量输入，提升模拟精度。
Long-term Vision
科学知识自动化
未来实现全自动化的科学知识提取与更新，推动科学数据库的实时维护与共享。
Abstract
The growing scientific literature contains decades of experimental and computational results that could support data-driven and physics-based modeling, yet much of this infor- mation remains locked in publications and is not readily usable for large-scale analysis or sci- entific software. Building structured databases from the literature is particularly challenging whenrelevantstudiesmustfirstbediscoveredamonglargecollectionsofpapersandreported quantities must be extracted with enough scientific context to remain usable. We present LitCurate, an open-source framework for building scientific databases from the literature using large language models within an auditable, stage-wise curation workflow. LitCurate integratesliteraturediscovery, relevancescreening, full-textprocessing, andstructuredinfor- mation extraction while retaining intermediate results and provenance, allowing researchers to inspect and revise individual stages rather than treating automated curation as a black- box process. We apply LitCurate to construct an equation-of-state database of lower-mantle and lower-mantle-relevant high-pressure mineral phases from experimental and theoretical studies, comprising 1,334 entries from 205 papers. The resulting dataset links reported equation-of-state parameters to mineral phases, compositions, equation formulations, meth- ods, and parameter constraints, and labels values as source-reported or citation-reported when provenance can be determined. The records are available through a searchable web application. By connecting scientific literature to traceable, machine-readable data, LitCu- rate provides a reusable approach for transforming accumulated literature into resources for scientific analysis and computational modeling.