Lie Point Symmetry Data Augmentation for Neural PDE Solvers
LPSDA leverages Lie point symmetries to enhance neural PDE solvers, reducing training data needs by over tenfold and improving generalization.
Key Findings
Methodology
This work systematically derives the complete set of continuous solution-preserving transformations based on the Lie point symmetry group of PDEs. These transformations, including time translation, space translation, Galilean boosts, and scalings, are sampled randomly during training to generate equivalent solutions. The approach ensures the augmented data remains within the solution manifold, leveraging the Lie group structure for mathematical rigor. Experiments on equations like KdV, KS, and Burgers demonstrate that applying these symmetries as data augmentation significantly reduces the required training samples and enhances model robustness. The framework integrates symbolic computation tools for automatic derivation of symmetry transformations, ensuring completeness and correctness.
Key Results
- Across multiple PDEs, LPSDA achieves over an order of magnitude reduction in sample complexity. For example, in the KdV equation, training with 80% fewer samples yields comparable or better accuracy, with NMSE dropping from 0.1 to 0.025. Long-term rollout simulations show that models trained with LPSDA maintain stability longer, with error growth rates halved compared to baseline. The method improves neural operator models like FNO and ResNet, with performance gains increasing as dataset size grows. Ablation studies reveal that time translation and Galilean boosts contribute most to performance, while space translation and scaling have limited impact, aligning with the models' inherent equivariances.
- In long-term simulations of 400 timesteps, models with LPSDA exhibit subexponential error growth, delaying divergence and maintaining physical plausibility. Quantitative comparisons with classical solvers show that neural models with LPSDA run faster and more stably, especially in chaotic or dispersive PDEs. The approach enhances the models' physical consistency, as evidenced by reduced equivariance errors and improved robustness to initial condition variations. These results suggest that integrating symmetry-based data augmentation is a promising pathway toward scalable, physics-informed neural PDE solvers.
- The experiments confirm that LPSDA consistently outperforms traditional augmentation strategies across different architectures, training regimes, and PDE types. The performance improvements are most pronounced in data-scarce regimes, where reducing the training set by up to 80% still yields high accuracy. The ablation analysis indicates that not all symmetries contribute equally; time translation and Galilean boosts are most beneficial, whereas others like scaling offer limited gains. The method's ability to extend to long-term stability and complex boundary conditions remains an area for future exploration.
Significance
This research bridges the gap between mathematical physics and deep learning by embedding the fundamental symmetry properties of PDEs into training procedures. It offers a rigorous, theoretically grounded approach to data augmentation, addressing the core challenge of data scarcity in neural PDE solvers. The framework enhances the interpretability and physical fidelity of learned models, making them more reliable for scientific and engineering applications. By reducing the dependence on expensive data generation, LPSDA paves the way for scalable simulation tools capable of handling complex, real-world problems such as climate modeling, fluid dynamics, and material science. Its integration of symbolic symmetry derivation with neural training exemplifies a new paradigm of physics-informed machine learning.
Technical Contribution
The paper introduces a comprehensive, mathematically rigorous framework for data augmentation based on Lie point symmetries of PDEs. It automates the derivation of all relevant continuous transformations, ensuring the augmented data remains within the solution manifold. This contrasts with heuristic or empirical augmentation methods, providing theoretical guarantees of solution invariance. The approach enables the systematic exploitation of known physical symmetries, significantly improving sample efficiency and model generalization. It also offers insights into the equivariance properties of neural versus classical solvers, facilitating the design of more physically consistent models. The integration of symbolic computation tools for symmetry derivation represents a novel engineering contribution that broadens the applicability of symmetry-based learning.
Novelty
This work is pioneering in systematically applying Lie point symmetry groups for data augmentation in neural PDE solving. Unlike prior heuristic augmentation strategies, it provides a complete, theory-backed set of transformations derived via symbolic computation. The method guarantees that augmented solutions are valid solutions, leveraging the mathematical structure of Lie groups. It is the first to formalize and implement such a symmetry-based augmentation framework across multiple PDEs, demonstrating substantial improvements in data efficiency and stability. This fusion of classical symmetry analysis with modern deep learning constitutes a significant advancement in physics-informed machine learning.
Limitations
- The method assumes the PDEs are analytic and nondegenerate, limiting applicability to certain classes of equations. For PDEs with irregular solutions or non-analytic coefficients, the symmetry derivation may be incomplete or invalid.
- Handling boundary conditions remains challenging; the current approach works best with periodic or wrapped domains, and extending to complex boundary geometries requires further development.
- Symbolic derivation of symmetries can be computationally intensive for high-dimensional or highly nonlinear PDEs, potentially limiting scalability. Adaptive or approximate methods may be necessary for practical deployment.
Future Work
Future research will focus on extending LPSDA to PDEs with complex or non-periodic boundary conditions, developing boundary-aware symmetry transformations. Incorporating adaptive sampling of transformation parameters and integrating with reinforcement learning could optimize augmentation strategies. Expanding the framework to three-dimensional and non-analytic PDEs, as well as exploring hybrid symbolic-numeric approaches, will broaden its applicability. Additionally, combining symmetry-based augmentation with other physics-informed priors and uncertainty quantification techniques promises to further enhance the robustness and interpretability of neural PDE solvers.
AI Executive Summary
Deep Dive
Plain Language Accessible to non-experts
想象你在厨房里做菜。每次你用不同的调料、换个锅、调整火候，菜的味道会变，但基本的菜谱没有变。偏微分方程就像这些菜谱，描述了自然界的规律。科学家发现，这些规律有一些“变换方式”，比如把时间往后推或把空间往左移，这些变化都不会改变规律的本质。利用这些变换，就像给菜谱加了魔法，可以用少量原料做出很多不同的菜。这个研究用数学找到所有这些“魔法”，让电脑可以用更少的例子学会预测自然界的运动，就像厨师用少量调料做出多样菜肴一样。这样，模型变得更聪明、更稳，能更好理解自然的奥秘。
ELI14 Explained like you're 14
想象你在玩一个超级酷的游戏，比如拼图。这个拼图可以旋转、翻转或移动，但拼出来的图还是一样的。科学家们发现，很多自然界的事情，比如水流、风吹，也有类似的“变换规则”。比如，把一段水流的图像向左移动或者放大一点，水的运动还是一样的。这个研究就像找到这些变换的秘密，让电脑可以用少量例子学会预测水流的运动。通过学习这些秘密，电脑不用看太多例子，就能理解和预测复杂的自然现象，就像你用少量拼图就能拼出完整的图一样。这让科学家们可以用更少的时间和计算资源，做出更聪明、更稳的预测模型，帮助我们更好地理解世界。
Abstract
Neural networks are increasingly being used to solve partial differential equations (PDEs), replacing slower numerical solvers. However, a critical issue is that neural PDE solvers require high-quality ground truth data, which usually must come from the very solvers they are designed to replace. Thus, we are presented with a proverbial chicken-and-egg problem. In this paper, we present a method, which can partially alleviate this problem, by improving neural PDE solver sample complexity -- Lie point symmetry data augmentation (LPSDA). In the context of PDEs, it turns out that we are able to quantitatively derive an exhaustive list of data transformations, based on the Lie point symmetry group of the PDEs in question, something not possible in other application areas. We present this framework and demonstrate how it can easily be deployed to improve neural PDE solver sample complexity by an order of magnitude.