MiMo: Unlocking the Reasoning Potential of Language Model -- From Pretraining to Posttraining

The evolution of large language models (LLMs) has revolutionized natural language understanding and generation, with models like GPT-4, LLaMA, and Qwen achieving remarkable performance. However, their reliance on massive parameters (often exceeding 100B) results in prohibitively high training costs and limited accessibility. Recent research emphasizes the importance of reasoning capabilities, especially in complex tasks like mathematics, programming, and scientific question answering. Reinforcement learning (RL) has been employed to enhance reasoning, but most successful applications depend on large base models, which are resource-intensive. Small and medium models, despite their efficiency, lag behind in reasoning performance. Prior works such as DeepSeek and Qwen have shown progress but still face limitations in scalability and inference speed. Therefore, developing methods to unlock reasoning potential in parameter-efficient models remains a critical challenge.

Current state-of-the-art models excel in reasoning but at the cost of enormous computational resources, limiting their deployment. Smaller models lack sufficient reasoning capacity, especially in multi-step problems requiring extended context understanding. Existing training strategies often focus on scaling parameters rather than optimizing data and mechanisms, leading to diminishing returns. The core problem is how to effectively enhance reasoning in models with limited parameters without incurring prohibitive costs. Addressing this involves improving data quality, designing mechanisms that facilitate reasoning, and developing efficient training infrastructure. The challenge is to balance model size, training efficiency, and reasoning performance, making advanced AI accessible to broader applications.

This work introduces several key innovations: 1) a multi-source high-quality data pipeline with HTML, PDF, and synthetic data extraction, combined with a three-stage data mixing strategy to maximize reasoning pattern density; 2) the Synthetic reasoning data augmentation, generated by advanced reasoning models, to enrich training diversity; 3) the multi-token prediction (MTP) mechanism, which during inference uses multiple layers to pre-plan token generation, accelerating decoding; 4) a difficulty-based reward scheme in RL, which assigns fine-grained scores to test cases of varying difficulty, alleviating sparse reward issues; 5) a high-efficiency RL infrastructure supporting continuous rollout and asynchronous reward computation, drastically reducing training time. The architecture leverages Grouped-Query Attention, pre-RMSNorm, and long-context support, optimized for reasoning tasks. These innovations collectively enable a small model to outperform larger counterparts in reasoning benchmarks, setting a new standard for parameter-efficient reasoning systems.

• �� Data collection and preprocessing: develop specialized HTML and PDF extraction tools to preserve reasoning content; perform global deduplication and multi-dimensional filtering to ensure data quality. • Data mixing: implement a three-stage strategy—initially combining diverse sources, then increasing math and code data to 70%, and finally incorporating synthetic reasoning responses with extended context length to 32K tokens. • Model architecture: based on decoder-only Transformer, integrating Grouped-Query Attention, pre-RMSNorm, SwiGLU activation, and Rotary Positional Embedding. • Multi-Token Prediction (MTP): during pretraining, use a single MTP layer; during inference, replicate and fine-tune multiple layers for speculative decoding, accelerating long output generation. • Training: optimize with AdamW, gradually adjust learning rate, employ batch warmup, and incorporate a strategy re-sampling to stabilize RL training. • RL training: curate 130K problems, employ rule-based rewards, and utilize infrastructure supporting parallel evaluation and reward computation. • Evaluation: test on multiple benchmarks, including BBH, SuperGPQA, GSM8K, AIME, and code benchmarks, analyzing performance gains and ablation effects.

The experimental setup involves training a 36-layer, 4096-hidden-dimension Transformer on 25 trillion tokens, with synthetic data augmentation and multiToken prediction. Baselines include Llama-3.1-8B, Qwen2.5-7B, and larger models. Evaluation metrics encompass accuracy, Pass@k, and F1 scores across tasks like mathematics, coding, and reasoning. Ablation studies confirm the contributions of synthetic data, multiToken prediction, and difficulty rewards. Results show MiMo-7B-Base surpasses comparable models, achieving 75.2 on BBH, and the RL-tuned MiMo-7B-RL exceeds OpenAI o1-mini in AIME (55.4) and code benchmarks, demonstrating superior reasoning and problem-solving capabilities.

MiMo-7B-Base outperforms 32B models on reasoning benchmarks, with 75.2 on BBH and strong performance on SuperGPQA and GSM8K. The RL-finetuned MiMo-7B-RL surpasses OpenAI o1-mini, achieving 55.4 on AIME 2025 and 32.9 on LiveCodeBench v5. The model supports 32K context length, enabling advanced long-context reasoning and retrieval tasks. Ablation results highlight the importance of synthetic data, multiToken prediction, and difficulty-based rewards. Overall, the approach demonstrates that data and mechanism innovations can compensate for smaller size, achieving state-of-the-art reasoning performance.

该模型适用于数学教育、自动编程、法律分析、科研辅助等领域，尤其在需要长文本理解和多步骤推理的场景中表现优异。工业界可以利用其强大的推理能力提升自动问答、智能助手、自动编码等应用的效率。未来，结合多模态信息，模型有望实现跨领域的智能推理与决策支持，推动智能系统的普及和升级。

模型在极端复杂推理或超专业知识领域仍有限，部分任务表现不及超大模型。训练成本较高，RL阶段对硬件要求大，限制了广泛应用。模型在某些特定领域的知识掌握不足，需引入更多专业数据进行微调。未来需优化机制和架构，降低成本，提升鲁棒性和可解释性，增强模型的泛化能力。