InternLM-Math: Open Math Large Language Models Toward Verifiable Reasoning
InternLM-Math unifies chain-of-thought, reward modeling, formal proof, and code interpretation, achieving SOTA on multiple benchmarks.
Key Findings
Methodology
This work integrates chain-of-thought reasoning, reward models, formal verification, data augmentation, and code interpreters into a unified seq2seq framework, training a versatile math model. Starting from InternLM2-Base, it is further pre-trained on math-related corpora, synthetic data, and problem sets like GSM8K, MATH, and MiniF2F. Multi-task supervised fine-tuning includes problem solving, verification, proof generation, and data augmentation. The model employs a reasoning-interleaved with coding (RICO) mechanism to enhance complex calculation capabilities. Formal verification is achieved via translating reasoning paths into LEAN code, improving trustworthiness. The training strategy involves multi-stage data filtering, multi-epoch training, and mixed-precision optimization, resulting in a model capable of outperforming previous state-of-the-art models on formal and informal benchmarks.
Key Results
- The model scores 30.3 on MiniF2F without fine-tuning, surpassing many open-sourced models and approaching GPT-4's performance. It achieves 82.6% accuracy on GSM8K and 37.7% on MATH in in-context learning, outperforming models like Llemma and Minerva. Incorporating reward reranking and formal verification further boosts correctness and trustworthiness. The model's multi-task training and RICO mechanism significantly improve complex reasoning and calculation performance across benchmarks.
- In formal reasoning, the model achieves 30.3 on MiniF2F, setting a new SOTA. It demonstrates strong generalization in Chinese and multilingual math tasks, with scores exceeding 78% in zero-shot chain-of-thought reasoning. The integration of LEAN for formal proof verification enhances the reliability of generated solutions. Ablation studies confirm the importance of multi-task supervision, data augmentation, and RICO in performance gains.
- The model's multi-capability approach proves effective across diverse tasks, including informal problem solving, formal proof, and symbolic reasoning. The combination of code-interleaved reasoning and formal verification yields robust, interpretable solutions, suitable for scientific and educational applications. Its open-source release facilitates community-driven improvements and broader deployment.
Significance
This research advances the state of the art in mathematical reasoning by integrating multiple capabilities—chain-of-thought, formal proof, code execution, and reward modeling—within a single framework. It addresses longstanding issues of trustworthiness and complexity in AI-based math problem solving. The ability to verify solutions formally and generate reliable proofs opens new avenues for AI-assisted scientific discovery, automated theorem proving, and educational tools. The open-source nature accelerates community innovation, making high-performance math reasoning accessible to researchers and practitioners worldwide. This work paves the way for future AI systems that are not only powerful but also trustworthy and explainable in complex reasoning tasks.
Technical Contribution
The key technical innovation is the unification of multiple reasoning and verification modules into a single seq2seq architecture, enabling seamless multi-capability integration. The introduction of RICO allows multi-round reasoning interleaved with code execution, significantly improving complex calculation accuracy. Formal verification via translating reasoning paths into LEAN code enhances trustworthiness, bridging deep learning with formal methods. The multi-task supervised fine-tuning with reward models and data augmentation strategies results in a model that excels across diverse benchmarks. These contributions collectively push the boundaries of AI's mathematical reasoning and formal verification capabilities.
Novelty
This is the first work to unify chain-of-thought reasoning, reward modeling, formal proof, and code interpretation within a single seq2seq framework, with the innovative RICO mechanism enabling multi-round reasoning with code. The integration of LEAN-based formal verification directly into the training pipeline is novel, significantly improving solution trustworthiness. Unlike previous models focusing on single capabilities, this approach demonstrates the feasibility of a multi-capability, trustworthy mathematical AI system, setting a new benchmark in formal and informal reasoning tasks.
Limitations
- Despite improvements, the model still struggles with highly complex proofs or extremely long reasoning chains, where errors can accumulate. Formal verification depends on LEAN's expressiveness, which may limit handling of certain mathematical domains. The training process is computationally intensive, requiring large-scale resources, which may hinder deployment in resource-constrained environments. The model's performance in high-level abstract mathematics remains limited, necessitating further domain-specific data and techniques.
Future Work
Future directions include enhancing the formal verification pipeline by integrating more expressive proof systems, reducing inference costs, and expanding coverage to higher-level mathematics. Improving the robustness of reasoning paths and reducing error propagation are key goals. Incorporating more domain-specific knowledge bases and symbolic reasoning modules could further boost performance. Exploring real-world applications such as automated theorem proving, mathematical research assistance, and advanced educational tools will be prioritized. Community-driven datasets and benchmarks will also be developed to foster broader progress.
AI Executive Summary
InternLM-Math embodies a significant leap forward in AI-driven mathematical reasoning. By unifying chain-of-thought reasoning, reward modeling, formal verification, and code interpretation within a single seq2seq architecture, it achieves state-of-the-art performance across multiple benchmarks. Starting from the InternLM2-Base model, extensive pretraining on math corpora and synthetic data, combined with multi-task supervised fine-tuning, enables the model to excel in informal problem solving, formal proof, and complex calculations. The innovative reasoning-interleaved with coding (RICO) mechanism allows multi-round reasoning with code execution, greatly enhancing the model’s ability to handle intricate problems. Formal verification via translating reasoning paths into LEAN code further boosts trustworthiness, making solutions more reliable and interpretable.
Experimental results demonstrate that the model scores 30.3 on MiniF2F without fine-tuning, outperforming many open-source counterparts and approaching GPT-4’s performance. It also achieves 82.6% accuracy on GSM8K and 37.7% on MATH in in-context learning, showcasing its robustness. The integration of reward models and formal verification strategies significantly reduces errors and increases solution credibility. These advances not only push the boundaries of AI in mathematical reasoning but also open new avenues for scientific discovery, automated theorem proving, and educational applications.
The open-source release of the models and code will accelerate community-driven innovation, fostering broader adoption and further improvements. Future work aims to enhance formal verification, reduce computational costs, and expand coverage to higher mathematics. Overall, InternLM-Math paves the way for trustworthy, multi-capability AI systems capable of tackling the most challenging mathematical problems, with profound implications for science, industry, and education.
Deep Dive
Abstract
The math abilities of large language models can represent their abstract reasoning ability. In this paper, we introduce and open-source our math reasoning LLMs InternLM-Math which is continue pre-trained from InternLM2. We unify chain-of-thought reasoning, reward modeling, formal reasoning, data augmentation, and code interpreter in a unified seq2seq format and supervise our model to be a versatile math reasoner, verifier, prover, and augmenter. These abilities can be used to develop the next math LLMs or self-iteration. InternLM-Math obtains open-sourced state-of-the-art performance under the setting of in-context learning, supervised fine-tuning, and code-assisted reasoning in various informal and formal benchmarks including GSM8K, MATH, Hungary math exam, MathBench-ZH, and MiniF2F. Our pre-trained model achieves 30.3 on the MiniF2F test set without fine-tuning. We further explore how to use LEAN to solve math problems and study its performance under the setting of multi-task learning which shows the possibility of using LEAN as a unified platform for solving and proving in math. Our models, codes, and data are released at \url{https://github.com/InternLM/InternLM-Math}.