SciPaths: Forecasting Pathways to Scientific Discovery

Scientific discovery relies on building blocks of knowledge, often represented as dependencies between prior research and innovations. Early efforts used citation analysis to map influence, but these lacked granularity. Recent advances in NLP and ML have enabled automated literature analysis, hypothesis generation, and idea synthesis (Reddy & Shojaee, 2025). However, most systems focus on retrieval or influence proxies, not on explicit causal pathways. The challenge remains to model the intricate, multi-step dependencies that constitute scientific progress, especially in a fine-grained, contribution-level manner. This work addresses this gap by constructing detailed, expert-annotated pathways and developing models to predict them, aiming to understand the underlying structure of scientific innovation.

Existing models struggle to identify and ground the complex, multi-layered dependencies in scientific progress. Citation-based approaches are too coarse, often conflating influence with causality. The core challenge is to decompose a target contribution into necessary enabling contributions and accurately link each to prior work, capturing roles and rationales. This is compounded by the difficulty of modeling deep causal mechanisms, especially for core methodological dependencies. The lack of high-quality, contribution-level annotated datasets further hampers progress. Overcoming these issues is essential for advancing AI's ability to reason about scientific pathways and facilitate automated discovery.

This work introduces several key innovations: 1) constructing a high-quality, expert-annotated dataset of scientific pathways with detailed decomposition, roles, and groundings; 2) proposing a two-stage model framework that first predicts enabling contributions and then grounds them in prior work; 3) leveraging NLP models like GPT-4 and Gemini 3.1 Pro with semantic matching and rationale-based reasoning; 4) demonstrating that pathway decomposition quality critically impacts downstream grounding accuracy. These innovations collectively push the frontier of AI-enabled scientific reasoning, enabling more precise and interpretable pathway prediction.

• �� Data collection: Extract downstream reuse evidence from recent ML/NLP papers, filter for functional reuse via citation intent classifiers, and cluster contribution descriptions.
• �� Expert annotation: Decompose target contributions into enabling contributions based on necessity, annotate roles, rationales, and prior work groundings.
• �� Model training: Use NLP models (GPT-4, Gemini 3.1 Pro) to predict pathways, employing semantic similarity for matching.
• �� Pathway prediction: First stage generates candidate enabling contributions with roles and rationales; second stage grounds each in prior work or marks as unmapped.
• �� Evaluation: Use strict semantic F1, recall, precision, and grounding coverage metrics under various input conditions.
• �� Automation: Develop pipelines for large-scale silver pathway generation using LLMs, with validation and reranking.

The dataset comprises 262 expert-annotated pathways and 2444 silver pathways, split into development and test sets. Models are evaluated on their ability to predict enabling contributions and ground them in prior work. Metrics include F1, recall, and precision under strict semantic matching, with additional analysis on role-specific recovery. Experiments test models with different input contexts, including citation evidence and related work sections. Fine-tuning models on silver data and ablation studies on pathway components further elucidate the factors affecting performance. The evaluation emphasizes the difficulty of recovering core methodological dependencies, with current models achieving only modest success.

The top model, Gemini 3.1 Pro, achieves an F1 of 0.189, with significant gaps remaining in recovering core technical dependencies. Providing gold enabling contributions boosts prior work grounding substantially, indicating the importance of accurate pathway decomposition. Models perform better on resource and initialization dependencies than on core algorithms, highlighting the challenge of deep causal inference. The results suggest that current NLP models need further development to reliably infer complex scientific dependencies, especially for core methodological steps.

This framework can be applied to automate literature analysis, assist in research planning, and facilitate interdisciplinary collaboration by revealing underlying dependency structures. It supports the development of intelligent systems that can suggest foundational work needed for new innovations, optimize resource allocation, and improve understanding of scientific progress. Long-term, such models could enable fully automated scientific hypothesis generation and pathway discovery, transforming research workflows and accelerating discovery cycles.

Current models are limited in their ability to infer complex, deep causal dependencies, especially for core methodological contributions. Pathway annotations rely heavily on expert input, limiting scalability. The experimental scope is confined to ML/NLP literature, and cross-disciplinary validation is needed. Additionally, the models require substantial computational resources, and their reasoning transparency remains limited. Future work must focus on improving causal inference capabilities, automating annotation processes, and expanding datasets across scientific domains.