Seed-Prover: Deep and Broad Reasoning for Automated Theorem Proving
Seed-Prover achieves 78.1% proof success on IMO problems using formal verification and long chain reasoning.
Key Findings
Methodology
Seed-Prover employs a lemma-style whole-proof reasoning model, iteratively refining proofs based on Lean feedback, proved lemmas, and self-summarization. Three inference strategies enable deep and broad reasoning, with Seed-Geometry addressing geometry support gaps.
Key Results
- Seed-Prover proves 78.1% of past IMO problems, achieves 100% on MiniF2F, and over 50% on PutnamBench, significantly outperforming previous SOTA.
- Seed-Geometry solves 43 problems on the IMO-AG-50 benchmark, surpassing AlphaGeometry 2.
- In IMO 2025, Seed-Prover and Seed-Geometry fully solved 5 out of 6 problems.
Significance
This work significantly advances automated mathematical reasoning, demonstrating the effectiveness of formal verification and long chain reasoning in solving complex mathematical problems, particularly in geometry where Seed-Geometry fills Lean's support gaps.
Technical Contribution
Seed-Prover introduces lemma-style proving, combining long chain reasoning and formal verification, offering new theoretical guarantees and engineering possibilities. Seed-Geometry achieves efficient geometry problem-solving with a fast reasoning engine and extended domain-specific language.
Novelty
Seed-Prover is the first model to combine lemma-style proving with long chain reasoning, significantly enhancing the ability to solve complex mathematical problems, especially with its innovative geometry engine design.
Limitations
- Seed-Prover still faces limitations in certain complex geometry problems, especially those requiring extensive auxiliary constructions.
- Seed-Geometry's advantages are less apparent in non-geometry problems.
Future Work
Future work will focus on improving Seed-Prover's performance on non-geometry problems and exploring broader applications, particularly in the widespread use of formal verification.
AI Executive Summary
Seed-Prover is an innovative automated theorem proving system that efficiently solves complex mathematical problems through formal verification and long chain reasoning. Traditional natural language reasoning lacks clear supervision signals, but Seed-Prover overcomes this challenge by incorporating formal verification through the Lean language.
Seed-Prover employs a lemma-style whole-proof reasoning model, iteratively refining proofs based on Lean feedback, proved lemmas, and self-summarization. To tackle IMO-level contest problems, it employs three inference strategies for deep and broad reasoning. Notably, Seed-Geometry is introduced to significantly enhance geometry problem-solving capabilities where Lean lacks support.
In experiments, Seed-Prover proves 78.1% of past IMO problems, achieves 100% on MiniF2F, and over 50% on PutnamBench, significantly outperforming previous SOTA. Seed-Geometry solves 43 problems on the IMO-AG-50 benchmark, surpassing AlphaGeometry 2. This research demonstrates the immense potential of formal verification and long chain reasoning in automated mathematical reasoning.
Deep Analysis
Background
Recent advances in large language models have significantly improved mathematical reasoning capabilities, but challenges remain in theorem proving due to the lack of clear supervision signals in natural language. Works like AlphaProof have shown the potential of using formal languages for mathematical proofs, but further improvements in reasoning depth and breadth are needed.
Core Problem
The core problem is how to achieve automated proof of complex mathematical problems without clear supervision signals. Traditional natural language reasoning lacks sufficient detail and accuracy, while formal languages provide clear signals but still fall short in reasoning depth and breadth.
Innovation
Seed-Prover introduces lemma-style proving, combining long chain reasoning and formal verification to significantly enhance the ability to solve complex mathematical problems. Seed-Geometry achieves efficient geometry problem-solving with a fast reasoning engine and extended domain-specific language.
Methodology
- �� Lemma-style proving: Generates useful intermediate lemmas as shared knowledge across different inference paths.
- �� Iterative refinement: Optimizes proofs based on Lean feedback, proved lemmas, and self-summarization.
- �� Three-tiered inference strategy: Enables deep and broad reasoning, allocating thinking budget to explore interesting properties.
Experiments
Experiments were conducted on MiniF2F, PutnamBench, and IMO problems, using different inference strategies to evaluate Seed-Prover's performance. Key hyperparameters include the number of inference attempts and the use of lemma pools.
Results
Seed-Prover achieves 100% on MiniF2F, over 50% on PutnamBench, significantly outperforming previous SOTA. Seed-Geometry solves 43 problems on the IMO-AG-50 benchmark, surpassing AlphaGeometry 2.
Applications
Seed-Prover can be used for automated mathematical proofs, particularly in mathematical competitions and academic research. Its formal verification capabilities make it valuable in scenarios requiring high precision and reliability.
Limitations & Outlook
Seed-Prover still faces limitations in certain complex geometry problems, especially those requiring extensive auxiliary constructions. Seed-Geometry's advantages are less apparent in non-geometry problems. Future work will focus on improving performance in non-geometry problems.
Plain Language Accessible to non-experts
Imagine you're in a kitchen cooking a meal. Seed-Prover is like a smart cooking assistant that not only helps you find the right ingredients but also guides you through each step, ensuring the dish turns out perfectly. Traditional assistants might just give you a recipe, but Seed-Prover provides feedback at every step to ensure no mistakes are made. Seed-Geometry is like a specialist in complex pastries, quickly finding the best decoration options to make your cake both delicious and beautiful.
ELI14 Explained like you're 14
Hey there! Imagine you're playing a super complex puzzle game. Seed-Prover is like an unbeatable strategy guide that not only helps you find the key to victory but also gives you hints at every step to ensure you don't make mistakes. Seed-Geometry is like a pro at solving geometry puzzles, quickly finding the best solutions to help you breeze through the levels. Isn't that cool?
Glossary
Seed-Prover
An automated theorem proving system combining lemma-style proving and long chain reasoning.
Used to solve complex mathematical problems, especially in geometry.
Lean
A formal verification language providing clear proof signals.
Used in Seed-Prover to verify proof correctness.
Seed-Geometry
A reasoning engine specialized for geometry problems with fast reasoning capabilities.
Addresses geometry support gaps in Lean.
MiniF2F
A benchmark test for evaluating automated theorem proving systems.
Seed-Prover achieves 100% pass rate on this test.
PutnamBench
A benchmark for evaluating mathematical reasoning capabilities, containing complex math problems.
Seed-Prover achieves over 50% pass rate on this benchmark.
Open Questions Unanswered questions from this research
- 1 How to improve Seed-Prover's performance on non-geometry problems, especially those requiring extensive auxiliary constructions.
- 2 How to further optimize Seed-Geometry's reasoning speed and accuracy, particularly in complex geometry problems.
Applications
Immediate Applications
Mathematical Competitions
Seed-Prover can be used to automatically solve math competition problems, helping contestants improve problem-solving efficiency.
Academic Research
In mathematical research, Seed-Prover can be used to verify the correctness of complex theorems, enhancing research reliability.
Long-term Vision
Education
Seed-Prover can be used in math education to help students understand complex math concepts and improve learning outcomes.
Abstract
LLMs have demonstrated strong mathematical reasoning abilities by leveraging reinforcement learning with long chain-of-thought, yet they continue to struggle with theorem proving due to the lack of clear supervision signals when solely using natural language. Dedicated domain-specific languages like Lean provide clear supervision via formal verification of proofs, enabling effective training through reinforcement learning. In this work, we propose \textbf{Seed-Prover}, a lemma-style whole-proof reasoning model. Seed-Prover can iteratively refine its proof based on Lean feedback, proved lemmas, and self-summarization. To solve IMO-level contest problems, we design three test-time inference strategies that enable both deep and broad reasoning. Seed-Prover proves $78.1\%$ of formalized past IMO problems, saturates MiniF2F, and achieves over 50\% on PutnamBench, outperforming the previous state-of-the-art by a large margin. To address the lack of geometry support in Lean, we introduce a geometry reasoning engine \textbf{Seed-Geometry}, which outperforms previous formal geometry engines. We use these two systems to participate in IMO 2025 and fully prove 5 out of 6 problems. This work represents a significant advancement in automated mathematical reasoning, demonstrating the effectiveness of formal verification with long chain-of-thought reasoning.