Verifiable Geometry Problem Solving: Solver-Driven Autoformalization and Theorem Proposing
Proposes SD-GPS framework using QwenVL3-2B for autoformalization and lemma proposing, achieving 86.4% accuracy on Geometry3K.
Key Findings
Methodology
The approach integrates supervised learning and reinforcement learning, leveraging QwenVL3-2B for joint multimodal formalization. The formalizer converts raw diagram and text into executable symbolic predicates, guided by solver feedback. A lemma proposing agent suggests local auxiliary lemmas based on current proof states, filtered through symbolic verification to ensure soundness. The system is trained end-to-end, combining predicate accuracy with solver solvability, and evaluated on Geometry3K and PGPS9K datasets, outperforming existing neural and neuro-symbolic models.
Key Results
- On Geometry3K, SD-GPS achieved 86.4% completion accuracy and 90.4% multiple-choice accuracy, surpassing prior methods by 3.5 and 3.2 percentage points respectively.
- On PGPS9K, the system reached 79.8% completion and 84.5% choice accuracy, outperforming previous bests by 4.4 and 3.0 points.
- The lemma proposal mechanism, combined with symbolic verification, effectively alleviates deductive deadlocks, significantly improving reasoning success and interpretability.
Significance
This work advances the field of geometric reasoning by tightly integrating neural perception with symbolic execution, enabling verifiable and robust problem-solving. It addresses longstanding bottlenecks in autoformalization and theorem prediction, providing a scalable framework that enhances both accuracy and explainability. The method's ability to dynamically propose and verify lemmas marks a significant step toward autonomous, reliable AI systems capable of complex formal reasoning, with broad implications for scientific discovery and education.
Technical Contribution
The paper introduces a unified autoformalization module based on QwenVL3-2B, trained via supervised and solvability-guided reinforcement learning, ensuring executable formal predicates. It further develops an impasse-aware lemma proposing agent that suggests local lemmas conditioned on current proof states, verified through symbolic solvers. This tight integration of neural and symbolic components enables end-to-end training and inference, overcoming fixed rule library limitations and improving reasoning flexibility. The framework demonstrates theoretical soundness and practical robustness, setting new benchmarks.
Novelty
This is the first system to treat the symbolic solver as a continuous execution oracle during both formalization and deduction, rather than a static component. The lemma proposing mechanism, conditioned on solver states and verified symbolically, introduces a dynamic, adaptive approach to theorem generation, significantly enhancing the flexibility and scalability of neuro-symbolic reasoning systems. This paradigm shift enables neural agents to collaboratively work with symbolic engines in a closed loop, marking a novel contribution to AI reasoning architectures.
Limitations
- The system relies heavily on high-quality symbolic solvers; in complex or ambiguous problems, solver limitations may still hinder performance.
- Training involves substantial computational resources, including large datasets and iterative reinforcement learning, limiting scalability.
- Generalization to highly unstructured or real-world scenarios remains challenging, requiring further robustness improvements.
Future Work
Future directions include integrating more powerful symbolic engines, developing adaptive lemma proposal strategies, and extending the framework to other domains such as physics or chemistry. Enhancing the model's ability to handle ambiguous or incomplete inputs, reducing training costs, and improving generalization are key goals. Additionally, exploring more efficient training algorithms and broader datasets will facilitate real-world deployment and broader adoption.
AI Executive Summary
The challenge of automated geometric reasoning has persisted for decades, hindered by the difficulty of translating multimodal inputs into formal representations suitable for symbolic solvers. Traditional rule-based approaches, while mathematically rigorous, struggle with complex, ambiguous, or noisy data, limiting their practical utility. Recent advances in neural-symbolic systems have made progress, but often rely on modular pipelines that process diagrams and text separately, leading to information loss and deadlocks in reasoning.
This paper introduces SD-GPS, a novel framework that fundamentally rethinks the integration of neural perception and symbolic reasoning. Central to this approach is the use of QwenVL3-2B, a large multimodal language model, which jointly encodes raw diagrams and textual descriptions into executable formal predicates. Unlike prior systems, SD-GPS treats the symbolic solver as a continuous execution oracle, providing real-time feedback that guides both the formalization and deduction processes.
A key innovation is the impasse-aware lemma proposing agent, which observes the current proof state and suggests local auxiliary lemmas. These proposals are rigorously filtered through symbolic verification, ensuring soundness and relevance. This dynamic, closed-loop interaction enables the system to overcome deductive deadlocks, significantly improving reasoning success rates. Extensive experiments on Geometry3K and PGPS9K datasets demonstrate that SD-GPS surpasses existing neural and neuro-symbolic models, achieving over 86% accuracy in problem completion.
The significance of this work lies in its ability to unify perception, formalization, and reasoning within a single, trainable framework. It paves the way for AI systems capable of verifiable, explainable problem-solving in complex domains. Future work will focus on scaling the approach to broader scientific reasoning tasks, optimizing computational efficiency, and enhancing robustness against real-world ambiguities. Overall, SD-GPS marks a major step toward autonomous, reliable AI in formal reasoning applications.
Deep Dive
Abstract
Geometry Problem Solving have increasingly adopt the neuro-symbolic paradigm, combining neural intuition with symbolic rigor. However, current frameworks suffer from severe bottlenecks in two core stages: autoformalization, which treats multimodal translation as a static task decoupled from downstream solver compatibility, and theorem prediction, where solvers frequently hit a deductive impasse due to fixed rule libraries. To address these, we propose SD-GPS, a solver-driven framework that treats the symbolic solver as an execution oracle throughout both formalization and deduction. First, Solver-Driven Autoformalization unifies supervised formal-language adaptation and solvability-guided reinforcement learning into a single module built on QwenVL3-2B, making executability the central training signal. Second, Verified Theorem Proposing introduces an impasse-aware agent that proposes local auxiliary lemmas from current proof states, ensuring soundness by filtering all proposals through symbolic verification. Empirical evaluations on Geometry3K and PGPS9K demonstrate that SD-GPS consistently outperforms existing MLLM, neural, and neuro-symbolic methods across standard completion, multiple-choice, and cross-modal reference regimes, proving that closing the loop between multimodal perception and symbolic execution significantly improves geometric reasoning, offering profound insights into how neural agents can be grounded by formal systems to achieve verifiable problem-solving capabilities.