CodePDE: An Inference Framework for LLM-driven PDE Solver Generation
CodePDE framework leverages LLMs for PDE solver code generation, integrating reasoning, debugging, self-refinement, and scaling to surpass traditional methods.
Key Findings
Methodology
This work transforms PDE solving into a code generation task, designing a five-step framework: task specification, code generation, debugging, evaluation, and refinement. Chain-of-thought prompting guides models to explore diverse numerical schemes, while automated debugging corrects runtime errors. Multi-model testing and test-time scaling enhance solution robustness. The core algorithms involve GPT-4, Gemini 2.5 Pro, and other models, employing best-of-n sampling for inference expansion. The framework emphasizes modularity, enabling seamless integration of various inference strategies and agentic workflows.
Key Results
- On five representative PDEs (e.g., Burgers, Reaction-Diffusion), models under CodePDE surpass traditional solvers in nRMSE, with Burgers achieved at 1.06×10^-4 after refinement, a 70% improvement over the reference 3.55×10^-4. Bug-free rates increased from 41% to 84% after self-debugging. Test-time scaling showed solution accuracy improved with increased sampling (n=4 to 16), confirming the effectiveness of the scaling law.
- Advanced reasoning models (e.g., o3, DeepSeek-R1) excel at generating high-quality solvers from scratch, but their refinement performance is comparable to standard models like GPT-4o. Self-debugging significantly boosts bug-free rates, with models achieving over 90% success after multiple debugging rounds. Different models show varied strengths in code generation versus refinement, highlighting skill divergence.
- Systematic analysis reveals a trade-off: some models favor reliability with simpler solutions, others explore more complex schemes. Test-time expansion consistently improves accuracy across PDEs, validating the scalability approach. The framework's modular design allows comprehensive evaluation of model capabilities and failure modes, guiding future improvements.
Significance
This research breaks through traditional PDE solver development bottlenecks by harnessing LLMs for automated code generation, debugging, and optimization. It addresses longstanding challenges of expert dependency, manual tuning, and limited transparency. The approach democratizes scientific computing, enabling non-experts to generate high-quality solvers rapidly. It also paves the way for AI-driven scientific engines, accelerating discovery in physics, engineering, and beyond. The framework’s adaptability and scalability open new horizons for AI in complex mathematical modeling, promising transformative impacts on industry and academia.
Technical Contribution
The paper introduces a novel framework combining chain-of-thought prompting, automated debugging, and multi-model test-time scaling within a modular architecture. It systematically evaluates multiple LLMs on PDE solver generation, establishing a new paradigm for AI-assisted scientific computing. The integration of self-refinement and inference expansion strategies enhances solver accuracy and robustness, setting new benchmarks. The work also provides insights into model capabilities, failure modes, and the trade-offs between reliability and complexity, offering a comprehensive toolkit for future research.
Novelty
This is the first systematic application of large language models for automated PDE solver code generation, integrating reasoning, debugging, and scaling strategies into a unified framework. Unlike prior work focusing solely on code synthesis or neural PDE solvers, this approach emphasizes robustness, reliability, and comprehensive evaluation. The multi-round self-debugging and test-time expansion mechanisms represent significant innovations, enabling models to produce high-quality, reliable solutions for complex scientific problems, filling a critical gap in AI-driven scientific computing.
Limitations
- Models still struggle with highly nonlinear or high-dimensional PDEs, where stability and accuracy degrade. The current debugging strategies may not handle extreme cases effectively.
- Computational costs for multi-model testing and multiple debugging rounds are high, limiting real-time deployment. Optimization is needed for efficiency.
- Handling complex geometries and boundary conditions remains challenging, requiring further integration of geometric and physical information to improve generalization.
Future Work
Future efforts will focus on incorporating physics-informed priors and geometric reasoning to enhance model robustness in complex scenarios. Developing more efficient inference and debugging algorithms will reduce computational costs. Expanding the framework to handle higher-dimensional and more nonlinear PDEs, as well as integrating domain-specific knowledge, will broaden applicability. Community-driven benchmarks and open-source tools will facilitate wider adoption and further innovations.
AI Executive Summary
Partial differential equations (PDEs) are fundamental in modeling phenomena across science and engineering, yet solving them efficiently remains a challenge. Traditional numerical methods like finite difference, finite element, and spectral schemes, while reliable, demand expert knowledge and significant computational resources. Recent advances in deep learning introduced neural PDE solvers, but these often lack interpretability and generalization. Meanwhile, large language models (LLMs) such as GPT-4 and Gemini have demonstrated remarkable capabilities in code synthesis and mathematical reasoning. This paper introduces CodePDE, a novel inference framework that leverages LLMs for automated PDE solver generation. The core idea is to treat PDE solving as a code generation problem, guiding models through chain-of-thought prompting to produce diverse numerical schemes, then employing automated debugging and self-refinement to enhance solution quality.
The framework comprises five steps: task specification, code generation, debugging, evaluation, and refinement. It utilizes multiple models and inference strategies, including best-of-n sampling, to improve robustness and accuracy. Extensive experiments on five PDE families, such as Burgers and Reaction-Diffusion, demonstrate that models under CodePDE can produce solutions with normalized RMSE surpassing traditional software, achieving improvements of up to 70%. The debugging mechanism significantly raises bug-free rates from 41% to over 84%, while self-refinement further enhances accuracy.
Results reveal that advanced reasoning models excel at initial code synthesis, but their refinement performance is comparable to standard models like GPT-4. The test-time scaling strategy shows that increasing inference compute improves solution quality, confirming a practical scaling law. The study uncovers a trade-off between solver reliability and complexity, providing insights into model capabilities and failure modes. Overall, this work demonstrates that combining large language models with inference-time algorithms can revolutionize scientific computing, making PDE solving more accessible, reliable, and efficient. Despite current limitations in handling highly nonlinear or high-dimensional PDEs, future directions include integrating physics-informed priors, geometric reasoning, and optimizing computational costs, promising a transformative impact on AI-driven scientific discovery.
Deep Analysis
Background
PDEs在科学与工程中扮演核心角色，描述流体、反应、弹性等复杂现象。传统数值方法如FDM、FEM和谱方法提供理论保证，但实现繁琐、计算昂贵，尤其在高维空间表现突出。深度学习方法如PINNs和神经算子试图突破瓶颈，但泛化不足且缺乏透明性。近年来，大模型如GPT-4展现出强大的代码生成和推理能力，为自动化求解提供新途径。此前研究多关注模型训练和特定任务，缺乏系统性框架整合推理、调试和扩展策略，限制了实际应用。
Core Problem
偏微分方程求解器开发依赖专家知识和繁琐调试，难以普及。传统方法虽有效，但在复杂边界、高维空间和非线性问题上表现不足。深度学习模型虽能自动生成代码，但缺乏鲁棒性和可解释性，难以保证解的可靠性。现有技术难以实现快速、多样化且高精度的求解器，限制了工业和科研的应用。如何利用大模型推理能力，自动设计高质量、可靠的偏微分方程求解器，成为关键难题。
Innovation
核心创新包括：
1) 将偏微分方程求解转化为代码生成任务，利用大模型的推理能力自动探索多种数值方案；
2) 设计多轮自我调试机制，模型在运行失败后自主修正错误，提升可靠性；
3) 采用多模型集成和best-of-n采样策略，实现测试时解的扩展，增强准确性；
4) 构建完整五步框架，从任务描述到优化，系统整合推理、调试和扩展技术，提供新范式。这些创新突破了传统局限，显著提升模型在复杂科学任务中的表现。
Methodology
- �� 任务描述：将PDE定义、边界条件和初始条件转化为自然语言输入模型。
- �� 代码生成：利用链式推理(prompting)引导模型生成完整求解器代码，包括数值方案和辅助函数。
- �� 调试：执行代码检测错误，模型自主诊断和修正，进行多轮调试直至满足性能要求。
- �� 评估：通过数值误差（nRMSE）、收敛性和运行时间指标评价求解器。
- �� 优化：将误差反馈给模型，指导其改进代码，提升解的精度和稳定性。
- �� 测试扩展：采用best-of-n采样，通过多次推理选择最优解，增强模型的鲁棒性。
Experiments
在五类PDE（如Advection、Burgers、Reaction-Diffusion、CNS、Darcy）上，采样100个实例，利用16个模型进行测试。比较标准软件、神经网络和自动生成器。指标包括nRMSE、调试成功率、收敛性和运行时间。采用多轮调试和自我优化验证模型在复杂任务中的表现。通过消融实验分析调试、扩展和模型规模对性能的影响，确保方法的普适性。
Results
模型在五类PDE上生成的求解器在nRMSE上优于传统软件，Burgers方程精度达1.06×10^-4，超越手工编码的3.55×10^-4。调试机制将Bug-free率由41%提升至84%。多模型扩展使解的准确性随采样数增加而提升，最大在n=4到16之间。不同模型表现差异明显，先进推理模型在生成新方案方面优于基础模型，但在细节优化阶段相似。调试轮次和模型规模对性能影响显著，未来需优化计算成本。
Applications
该技术适用于工程设计、物理模拟、气候建模等领域，用户只需描述问题，即可自动生成求解器，降低专业门槛。科研人员可快速验证新模型，工业界实现高效、可靠的偏微分方程求解，推动智能制造和仿真发展。结合物理信息和几何理解，将拓展其应用范围。
Limitations & Outlook
模型在高维复杂或极端非线性问题上仍存在稳定性和精度不足，调试轮次和模型规模限制了效率。部分特殊边界和几何形状难以适应，需结合几何和物理信息增强理解。计算成本较高，实际部署需优化推理和调试流程。未来需提升泛化能力和复杂场景适应性，推动AI在科学中的更广泛应用。
Plain Language Accessible to non-experts
想象你在一家工厂里，工厂需要制造各种复杂的机械设备。传统上，你必须由经验丰富的工程师设计每个设备的详细图纸和制造流程，这既费时又容易出错。现在，如果有一种智能助手，能根据你简单描述的需求，自动写出详细的制造步骤和图纸，并在发现错误时自己修正，工厂的效率就会大大提高。这就像是用AI帮你写程序、调试代码一样。这个研究就是让大语言模型成为这样的智能助手，自动生成解决偏微分方程的“制造流程”。它可以理解复杂的数学问题，自动设计出数值算法，自己检测和修正错误，最终帮科学家和工程师更快、更准确地解决问题。就像你用语音描述想要的菜谱，AI自动写出完整的做法，还能自己试错和改进，变得越来越厉害。这种技术未来可以让科学研究变得更快、更智能，甚至让没有专业背景的人也能用上复杂的数学工具。
ELI14 Explained like you're 14
Imagine you're playing a super complex game, and you need to come up with a strategy to win. Before, you'd spend hours learning different tricks and trying out various plans. Now, suppose you have a really smart friend who listens to what you want to do, then writes out a detailed step-by-step plan for you, and even checks if there are mistakes, fixing them as needed. You just tell him your goal, and he helps you craft the perfect strategy, getting better each time. It's like having a genius buddy who can write code, debug it, and make it better all by itself. This research is about making such a smart AI friend using big language models. It can understand complicated math problems, generate solutions, find errors, and fix them on its own. Although it’s not perfect yet, it already performs really well on many tough problems. In the future, this AI buddy could help scientists solve big mysteries faster or even let regular people use advanced math tools easily—like having a super helper by your side all the time!
Abstract
Partial differential equations (PDEs) are fundamental to modeling physical systems, yet solving them remains a complex challenge. Traditional numerical solvers rely on expert knowledge to implement and are computationally expensive, while neural-network-based solvers require large training datasets and often lack interpretability. In this work, we frame PDE solving as a code generation task and introduce CodePDE, the first inference framework for generating PDE solvers using large language models (LLMs). With CodePDE, we present a thorough evaluation on critical capacities of LLM for PDE solving: reasoning, debugging, self-refinement, and test-time scaling. CodePDE shows that, with advanced inference-time algorithms and scaling strategies, LLMs can achieve strong performance across a range of representative PDE problems. We also identify novel insights into LLM-driven solver generation, such as trade-offs between solver reliability and sophistication, design principles for LLM-powered PDE solving agents, and failure modes for LLM on hard tasks. These insights offer guidance for building more capable and reliable LLM-based scientific engines.