NaturalThoughts: Selecting and Distilling Reasoning Traces for General Reasoning Tasks
Selecting diverse, challenging reasoning traces from a strong teacher model enhances student model reasoning, outperforming existing datasets by 3-5% on benchmarks.
Key Findings
Methodology
This work constructs the NaturalThoughts dataset by generating reasoning traces from DeepSeek-R1 across a large pool of questions from NaturalReasoning. Multiple filtering strategies—topic diversity, semantic embedding clustering, reasoning strategy variety, and difficulty metrics—are employed to select high-quality samples. Supervised fine-tuning (SFT) is performed on Llama-3.1 and Qwen models, comparing random sampling with strategy-based filtering. The approach incorporates System-1 (answer-only) and System-2 (full reasoning) distillation, optimizing for inference efficiency. Extensive experiments on reasoning benchmarks validate the effectiveness of these methods, showing consistent performance gains.
Key Results
- Training with up to 500,000 samples from NaturalThoughts yields 3-5% improvements over baseline datasets like OpenThoughts and LIMO on GPQA-Diamond, MMLU-Pro, and SuperGPQA, with the best results achieved using reasoning strategy diversity and difficulty filtering.
- Filtering based on reasoning strategies and question difficulty significantly outperforms random sampling, especially on complex reasoning tasks, demonstrating the importance of sample quality.
- The mixed System-1/2 distillation approach reduces inference time by over 20%, while maintaining high accuracy, effectively balancing speed and reasoning depth, and mitigating overthinking issues.
Significance
This research advances the paradigm of data-centric reasoning model training by emphasizing quality, diversity, and difficulty in sample selection. It demonstrates that strategic filtering can drastically improve reasoning capabilities with fewer samples, addressing the limitations of prior datasets and manual curation. The approach enhances the generalization and robustness of large language models, with broad implications for AI applications in education, scientific research, and automated decision-making. The integration of mixed distillation techniques offers a practical pathway to deploy more efficient, reasoning-capable models in real-world scenarios.
Technical Contribution
The paper introduces a comprehensive multi-criteria filtering framework for reasoning data, combining topic diversity, semantic clustering, reasoning strategy annotation, and difficulty metrics. It innovates by integrating System-1 and System-2 distillation, allowing models to adaptively balance inference speed and reasoning depth. The methodology is validated across multiple benchmarks, showing superior performance over existing datasets and single-strategy approaches. These contributions push forward the state-of-the-art in reasoning data construction and model training efficiency, providing a scalable blueprint for future research.
Novelty
This is the first work to systematically combine multi-dimensional filtering—topic, semantic, strategic, and difficulty—to curate reasoning datasets for large models. It also pioneers the use of mixed System-1/2 distillation during training, enabling models to dynamically trade-off between speed and reasoning depth. Unlike prior datasets relying on manual curation or random sampling, this approach ensures high diversity and challenge level, significantly improving sample efficiency and reasoning robustness.
Limitations
- The filtering process depends on manual annotations and model-based disagreement metrics, which may introduce biases and increase computational costs. Fully automated, scalable filtering remains a challenge.
- While performance on benchmarks improves, the models still struggle with the most complex, multi-hop reasoning tasks, indicating room for further enhancement.
- Large-scale training demands significant computational resources, limiting accessibility and deployment in resource-constrained environments.
Future Work
Future directions include developing fully automated, scalable filtering algorithms that incorporate external knowledge bases and multi-modal data, further reducing annotation costs. Exploring adaptive inference strategies based on question complexity, integrating knowledge graphs, and extending the approach to multilingual settings are promising avenues. Additionally, optimizing training efficiency and model robustness for real-time applications will be crucial for broader industrial adoption.
AI Executive Summary
The rapid advancement of large language models (LLMs) has unlocked remarkable reasoning capabilities, yet challenges remain in efficiently training models that can handle complex, multi-step problems. Traditional approaches rely heavily on massive datasets or reinforcement learning, which are resource-intensive and often limited by data quality. This study introduces a novel framework that leverages high-quality, strategically filtered reasoning traces generated from a strong teacher model, DeepSeek-R1, across a vast pool of questions from NaturalReasoning.
By employing multi-dimensional filtering strategies—including topic diversity, semantic clustering, reasoning strategy annotation, and difficulty assessment—the authors curate the NaturalThoughts dataset. This dataset emphasizes diverse and challenging reasoning demonstrations, which are crucial for transferring complex reasoning skills to student models like Llama-3.1 and Qwen. Extensive experiments reveal that models trained on these curated samples outperform counterparts trained on existing datasets such as OpenThoughts and LIMO by 3-5% on benchmarks like GPQA-Diamond, MMLU-Pro, and SuperGPQA.
A key innovation is the integration of System-1 (answer-only) and System-2 (full reasoning) distillation, enabling models to dynamically balance inference speed and reasoning depth. This approach not only improves accuracy but also significantly reduces inference time, addressing practical deployment concerns. The results demonstrate that filtering based on reasoning strategy diversity and question difficulty is more effective than random sampling, especially for complex tasks.
Overall, this work highlights the importance of data quality over quantity, providing a scalable, effective pathway for enhancing reasoning in large models. It paves the way for future research into automated filtering, knowledge integration, and multi-modal reasoning, with broad implications for AI in education, scientific discovery, and autonomous decision-making.
Deep Analysis
Background
随着大规模语言模型（如GPT-4、PaLM）在推理任务中的表现不断提升，研究逐渐转向如何高效提升模型的推理深度和泛化能力。Chain-of-Thought（CoT）技术通过引导模型生成推理链，显著改善了复杂问题的解决能力。早期工作如OpenThoughts、LIMO强调数据质量和筛选策略，但多依赖人工标注或随机采样，难以系统性地提升推理深度。近年来，研究者开始关注推理示范的多样性和难度，试图通过筛选策略优化训练样本，推动模型在多领域、多任务中的推理能力。
Core Problem
核心问题在于，现有推理数据集多为随机采样或有限手工筛选，难以全面覆盖推理策略和难度层次，导致模型在复杂推理任务中表现不足。此外，模型过度思考（overthinking）现象严重，影响推理效率和实际应用速度。如何系统性筛选出具有代表性、多样性且难度适中的推理示范，成为提升模型推理能力的关键。此问题的难点在于推理轨迹的多样性和难度评估复杂，且需要在保证训练效率的同时，增强模型的泛化能力。
Innovation
本文提出多维筛选策略，结合推理策略多样性、主题分布、语义嵌入和难度指标，系统构建高质量推理数据集NaturalThoughts。创新点包括：1）基于推理策略多样性筛选，确保模型学习到丰富的推理路径；2）引入难度评估机制，优先训练复杂推理示范；3）结合System-1（答案）和System-2（推理链）进行混合蒸馏，提升推理速度与深度平衡。这些创新显著改善了推理训练的样本效率和模型泛化能力，为大模型推理能力的提升提供了新思路。
Methodology
- �� 从NaturalReasoning采样问题，利用DeepSeek-R1生成推理链和答案。• 采用多维筛选：
- 按主题均匀采样，确保多样性；
- 利用Llama-3.1-8B-Instruct嵌入进行密度聚类，采样分布均衡；
- 根据推理策略多样性筛选，确保样本丰富；
- 按难度指标（长度、语调、模型不一致）筛选复杂样本。
- �� 结合System-1（答案）和System-2（推理链）进行混合蒸馏，设计随机和难度导向的采样策略。• 只优化推理轨迹和答案部分，避免问句干扰。• 多轮验证筛选策略对模型推理能力的影响。
Experiments
采用Llama-3.1-8B-Instruct和Qwen-2.5-7B模型，在GPQA-Diamond、MMLU-Pro、SuperGPQA等基准上评估。训练规模从1k到50万样本，比较随机采样、策略筛选和混合蒸馏效果。指标包括准确率和推理速度，采用多随机种子确保结果稳健。验证筛选多样性和难度对性能的提升，尤其在复杂推理任务中表现优越。
Results
筛选多样且难度适中的样本显著提升模型性能，尤其在GPQA-Diamond和MMLU-Pro上，性能提升达3-5个百分点。引入System-1/2混合蒸馏后，推理速度提升20%以上，过度思考现象减少。数据规模扩大至50万后，性能持续提升，验证了数据质量优于纯数量的结论。推理策略多样性和模型不一致性指标效果最佳，验证了多样性和难度的关键作用。
Applications
该方法可广泛应用于教育、科研、自动化决策等场景，提升模型在复杂推理任务中的表现。通过高效筛选和蒸馏，减少推理时间，增强响应速度和准确性。未来结合知识图谱、多模态信息，将推动智能系统在科学探索和工业自动化中的深度应用。
Limitations & Outlook
筛选策略依赖人工标注和模型判别，存在偏差和计算成本。模型在极端复杂推理任务中的表现仍有限，需结合知识增强技术。大规模训练资源消耗高，限制了普及推广。未来需优化自动化筛选机制，降低成本，提升极端任务的适应性。
Plain Language Accessible to non-experts
想象你在一家工厂里，生产不同的商品。每个工人（模型）都要按照说明书（推理轨迹）完成任务。有些说明书简单，容易完成；有些复杂，需要多次思考和尝试。工厂里有个老师（教师模型），他写了很多详细的说明书，工人们可以学习。为了让工人更快更好地完成任务，工厂管理者（研究者）会挑选出那些既详细又难度适中的说明书，让工人反复练习。这样，工人不仅学会了基本技能，还能应对复杂问题。研究发现，挑选多样且难度适中的说明书，能让工人变得更聪明、更快，效率也更高。这就像我们训练AI模型一样，通过精心筛选的推理示范，让它学会更复杂、更准确的推理能力，最终能在各种任务中表现出色。
ELI14 Explained like you're 14
想象你在学校学数学，有时候老师会给你一些很难的问题，要你用很多步骤解答。有时候，老师会给你一些简单的问题，只需要直接回答。研究人员发现，如果你多练习那些既难又需要用不同方法解决的问题，你会变得更聪明、更擅长推理。就像你玩游戏时，挑战越大，你学得越快。为了训练AI模型，科学家们用一个“老师”模型写出很多推理过程，然后挑选出那些既复杂又有趣的例子，让AI学习。通过这样的方法，AI不仅能学会简单的推理，还能应对更难的问题。更酷的是，他们还让AI学会在回答问题时，既可以快速给出答案，也可以花时间仔细推理，找到最好的解决方案。这样，AI在未来可以更聪明、更快地帮助我们解决各种复杂问题，就像一个超级聪明的助手一样！
Abstract
Recent work has shown that distilling reasoning traces from a larger teacher model via supervised finetuning outperforms reinforcement learning with the smaller student model alone (Guo et al. 2025). However, there has not been a systematic study of what kind of reasoning demonstrations from the teacher are most effective in improving the student model's reasoning capabilities. In this work we curate high-quality "NaturalThoughts" by selecting reasoning traces from a strong teacher model based on a large pool of questions from NaturalReasoning (Yuan et al. 2025). We first conduct a systematic analysis of factors that affect distilling reasoning capabilities, in terms of sample efficiency and scalability for general reasoning tasks. We observe that simply scaling up data size with random sampling is a strong baseline with steady performance gains. Further, we find that selecting difficult examples that require more diverse reasoning strategies is more sample-efficient to transfer the teacher model's reasoning skills. Evaluated on both Llama and Qwen models, training with NaturalThoughts outperforms existing reasoning datasets such as OpenThoughts, LIMO, etc. on general STEM reasoning benchmarks including GPQA-Diamond, MMLU-Pro and SuperGPQA.