Stable Long-Horizon PDE Forecasting via Latent Structured Spectral Propagators
Introduces Structured Spectral Propagator (SSP) for long-horizon PDE forecasting, reducing errors by up to 48.9%.
Key Findings
Methodology
This paper proposes a structured spectral propagator (SSP) learned in a latent space, combining a shared spatial encoding, spectral projection, frequency-conditioned linear modal updates, and nonlinear closure. The approach employs an analysis-propagation-synthesis pipeline, utilizing Fourier transforms and modal decomposition to ensure stable recursive predictions. The model explicitly models spectral modes with a frequency-dependent linear backbone and a residual nonlinear closure, enabling robust long-term PDE forecasting. The training involves multi-step autoregressive loss functions, including reconstruction, latent rollout, and physical space errors, to enforce stability and accuracy.
Key Results
- On Navier-Stokes, shallow water, and reaction-diffusion benchmarks, SSP outperforms state-of-the-art models like FNO and U-FNO, reducing maximum errors by up to 48.9%. In long-term extrapolation beyond the training horizon, SSP maintains lower errors and exhibits superior stability, especially at 90 prediction steps, with errors significantly lower than baselines.
- Spectral fidelity is preserved during recursive rollouts, with less mode drift and dynamic instability. Ablation studies confirm that the spectral closure and frequency modulation are critical for performance, with removal leading to noticeable degradation.
- The model demonstrates strong generalization across different nonlinear PDEs, effectively capturing modal interactions and energy transfer, validating the spectral structure's role in stability.
Significance
This work addresses the fundamental challenge of error accumulation in neural PDE surrogates during long-term predictions. By embedding spectral structure into the latent space, it introduces a strong inductive bias that ensures stable modal evolution. Such a design advances both theoretical understanding and practical capabilities, enabling reliable, long-horizon simulations crucial for scientific and engineering applications like weather forecasting, fluid dynamics, and climate modeling. The approach paves the way for more interpretable and physically consistent deep learning models for PDEs.
Technical Contribution
The key technical innovation is the integration of a frequency-conditioned linear modal backbone with a nonlinear spectral closure within a structured latent space. This design explicitly encodes spectral modes, regularizes their evolution, and compensates for unresolved interactions, providing theoretical guarantees for modal coherence. The architecture separates spatial encoding from temporal propagation, enabling efficient and stable long-term predictions. The training strategy combines hierarchical losses, enforcing consistency in both latent and physical spaces, and the spectral truncation ensures computational tractability without sacrificing accuracy.
Novelty
This is the first work to embed explicit spectral structure into a neural PDE predictor for long-horizon forecasting. Unlike prior models that treat the latent space as a black box, SSP leverages modal decomposition, frequency-dependent gating, and residual closure to control mode interactions. This structured spectral approach fundamentally differs from existing latent or implicit methods, providing interpretability and stability rooted in spectral theory. It bridges classical spectral analysis with modern deep learning, offering a new paradigm for PDE modeling.
Limitations
- The model's performance depends on the spectral truncation choice; highly nonlinear or high-frequency systems may require adaptive spectral strategies. Its effectiveness on complex geometries or multi-physics problems remains to be validated.
- Training complexity and hyperparameter tuning are non-trivial, especially for spectral parameters and closure networks. Computational costs may be high for very large-scale systems.
- The approach assumes translation invariance and spectral decomposability, limiting applicability to certain classes of PDEs. Extending to irregular domains or non-stationary systems is future work.
Future Work
Future directions include extending the spectral framework to multi-scale and multi-physics PDEs, developing adaptive spectral truncation strategies, and improving model interpretability. Incorporating graph-based spectral methods could handle irregular geometries. Further research on reducing computational costs and enhancing generalization to real-world complex systems will broaden practical deployment.
AI Executive Summary
Modeling the evolution of physical systems governed by partial differential equations (PDEs) is fundamental in science and engineering. Neural operators like Fourier Neural Operator (FNO) have demonstrated efficiency in approximating PDE solutions, but their performance diminishes over long-term predictions due to error accumulation and dynamic drift. This challenge is especially critical in applications like climate modeling, fluid dynamics, and material science, where accurate long-horizon forecasts are essential.
This paper introduces a novel approach called the Structured Spectral Propagator (SSP), designed to address these issues by embedding spectral structure directly into the latent space. The core idea is to decompose the PDE evolution into spectral modes, which are then evolved using a frequency-conditioned linear backbone, complemented by a nonlinear closure that accounts for residual interactions. The architecture involves an encoder that maps physical states into a shared spatial representation, a projector that reduces this to a compact spectral propagation space, and a decoder that reconstructs the physical fields. The spectral evolution is performed in the Fourier domain, with spectral truncation ensuring computational efficiency while maintaining essential dynamics.
Experimental results on benchmark PDEs such as Navier-Stokes, shallow water, and reaction-diffusion equations demonstrate that SSP significantly outperforms existing models like FNO and U-FNO. The errors are reduced by up to 48.9%, and the model exhibits remarkable stability in long-term extrapolation, maintaining spectral fidelity and modal coherence. Ablation studies confirm that the spectral closure and frequency modulation are key to these improvements. The approach not only enhances accuracy but also provides interpretability rooted in classical spectral theory, offering new insights into modal dynamics.
Overall, SSP represents a major step forward in physics-informed deep learning, enabling reliable, long-horizon PDE forecasting. Its spectral structure offers a robust inductive bias, making it suitable for complex, nonlinear systems across scientific disciplines. Future work aims to extend this framework to multi-scale, multi-physics problems and irregular geometries, promising broad impact in computational science and engineering.
Deep Dive
Plain Language Accessible to non-experts
想象你在操控一台非常复杂的工厂机器，这台机器由许多不同的部分组成，每个部分都在以不同的速度和方式运转。传统的方法就像你每次都用手调节每个部分，容易出错，也难以保证长时间的稳定运行。而这项新技术就像给这台机器装上了一个智能调节系统，它能识别每个部分的运转频率，然后用一种特别的“频谱”方式，把这些频率拆开，单独调节每个部分的速度。这个系统还能预测未来每个部分的状态，确保整个工厂长时间平稳运行，不会因为某个部分出错而导致整体崩溃。它用一种类似于“调音器”的工具，调整每个频率，让整个机器的运转变得更稳定、更高效。这样，即使工厂运行很久，也能保持正常，不会出现突然的故障。这就像给工厂配备了一个超级智能的调节器，能提前预警和调整，让工厂一直运转得井井有条。
ELI14 Explained like you're 14
想象你在玩一个超级复杂的乐队游戏，每次你弹奏一段旋律，游戏都会记住你弹了什么，然后用这些记忆帮你下一次弹得更好。但是，有时候你的小错误会让整个乐队的节奏变乱，导致音乐不和谐。这个研究就像发明了一个超级聪明的指挥，它能把你弹的每个音拆成不同的频率，就像把音乐拆成高音和低音，然后用一种特别的“调音器”调节每个频率，让整个乐曲听起来更和谐。这个指挥会学习每个频率的节奏，然后用一种聪明的方式，把它们组合在一起，确保你每次弹奏都能保持稳定，不会出现跑调。它还能在你弹了很久之后，依然保持节奏稳定，不会突然乱掉。这个技术就像给你配备了一个超级厉害的音乐助手，帮你一直保持好节奏，让你的演奏越来越棒！
Abstract
Long-horizon forecasting of time-dependent partial differential equations (PDEs) is critical for characterizing the sustained evolution of physical systems. While neural operators have emerged as efficient surrogates, they typically learn implicit finite-time transitions from discrete observations. When deployed autoregressively, such propagators often suffer from rapid error accumulation and dynamic drift. To address this, we propose a neural forecasting framework that reformulates PDE rollout as learning a Structured Spectral Propagator (SSP) in a propagation-oriented latent space. Following an analysis-propagation-synthesis design, our framework: (i) maps physical states into a shared, time-consistent spatial representation; (ii) projects this space into a compact propagation state to isolate recurrent dynamics from fine-grained spatial details, thereby decoupling reconstruction fidelity from rollout regularity; and (iii) evolves retained spectral modes using a frequency-conditioned linear backbone complemented by a nonlinear spectral closure to account for truncated interactions. This explicit structuring endows the propagator with a strong inductive bias for coherent modal evolution. Extensive experiments demonstrate that SSP significantly outperforms state-of-the-art baselines, reducing relative $L_2$ errors by up to 48.9% and exhibiting improved stability in temporal extrapolation beyond the supervised horizon.