Flowers: A Warp Drive for Neural PDE Solvers

The evolution of neural PDE solvers has seen methods like Fourier Neural Operators (FNO), DeepONets, and convolutional networks, which have improved efficiency over classical numerical schemes. FNO leverages Fourier transforms for global interactions, while CNNs impose locality priors. Transformers introduced attention mechanisms for long-range dependencies but at high computational costs. Despite progress, challenges remain in capturing complex, multiscale, and nonlocal phenomena efficiently and physically consistently. Recent efforts incorporate physics constraints or hybrid models, but scalability and interpretability are ongoing issues. The need for models that are both computationally efficient and physically interpretable motivates the development of new architectures like Flowers.

Current neural PDE solvers struggle with long-term stability, scalability to high dimensions, and physical interpretability. Fourier and attention-based models often treat long-range interactions as uniform, lacking the ability to encode characteristic transport directions. Convolutional models are limited by local receptive fields, making it difficult to model nonlocal phenomena accurately. These limitations hinder their application in complex, real-world scenarios such as turbulent flows, wave propagation in heterogeneous media, and multi-physics systems. Developing models that inherently incorporate physical structure, are computationally scalable, and provide interpretability remains a critical challenge.

Flowers introduces a pure warp-based architecture, where each layer predicts local displacements (ϱ) that warp features via sparse sampling. This design is inspired by physics principles: flow maps for conservation laws, ray-based wave propagation, and kinetic theory limits. Key innovations include:

• �� Multi-head pointwise warps that encode nonlocal interactions efficiently.
• �� Residual multiscale blocks with a U-Net scaffold for capturing complex dynamics.
• �� Theoretical grounding in physics, ensuring interpretability and physical consistency.
• �� Parameter efficiency, with a 17M model outperforming larger baselines, and a 150M version surpassing recent transformers.

This approach fundamentally shifts the paradigm from dense global mechanisms to sparse, physics-inspired pointwise operations, enabling scalable, interpretable, and physically grounded PDE solvers.

• �� Input space fields u(x) are processed by multiple heads, each predicting a feature map v(h)(x) and a displacement field ϱ(h)(x) using small MLPs.
• �� Displacements are predicted pointwise, depending only on local features, ensuring locality.
• �� Features are sampled at displaced coordinates x + ϱ(h)(x) via bilinear interpolation, enabling nonlocal interaction.
• �� Warped features from each head are concatenated, forming a multi-head output.
• �� Residual blocks embed the warp operation, with skip connections and normalization for stability.
• �� The entire model stacks these blocks in a multiscale U-Net-like architecture, capturing long-range and multiscale features.
• �� Theoretical insights relate the displacements to characteristic flows, wave rays, and kinetic transport equations, grounding the architecture in physics.

Experiments span multiple PDE benchmarks, including The Well, Rayleigh–Taylor instability, and neutron star mergers. Models with 17M, 70M, and 150M parameters are trained using Adam optimizer, evaluated with VRMSE metrics. Baselines include FNO, CNUNET, and SCOT. Ablation studies confirm the importance of learned warps and multi-head design. Results show that Flowers consistently outperforms baselines in accuracy, stability, and efficiency, especially in long-term predictions and complex geometries. The models generalize well across different physical regimes and geometries, demonstrating robustness and scalability.

Flowers achieves VRMSE below 0.02 across multiple benchmarks, outperforming FNO, CNUNET, and SCOT. The 17M model sets new state-of-the-art on The Well dataset, with superior efficiency. Larger models (70M, 150M) further improve accuracy, especially in 3D and long-horizon tasks. Ablation results highlight the critical role of learned displacements and multi-heads. The architecture scales linearly with grid size, enabling efficient high-dimensional simulations. The learned displacements align with physical flow fields, validating the physics-inspired design.

Applicable to fluid dynamics, wave propagation, climate modeling, and engineering simulations requiring fast, accurate PDE solutions. Its efficiency makes it suitable for real-time prediction and control tasks, while interpretability supports scientific understanding. The model can serve as a surrogate in multi-physics simulations, enabling rapid prototyping and parameter studies, especially in high-dimensional, complex geometries where traditional methods are computationally prohibitive.

Performance may degrade in highly turbulent or chaotic regimes where local displacements are insufficient. Long-term stability still requires larger models or additional physics constraints. The sparse sampling approach might struggle with highly nonlocal phenomena. Extending to non-conservative or strongly coupled systems remains challenging, and further work is needed to improve robustness and generalization to unseen physical regimes.