Rolling Diffusion Models

The evolution of generative modeling has seen diffusion models, such as Ho et al. (2020), become dominant in image and text synthesis. Extending these models to temporal data like videos and climate sequences has attracted increasing interest. Prior works, including Ho et al. (2022a) and Blattmann et al. (2023), adapted diffusion to pixel or latent space, but faced challenges in long sequence modeling due to computational costs and error propagation. These models often treat the temporal dimension as an extra spatial axis, which limits scalability and temporal understanding. Recent efforts have explored autoregressive and conditional methods, but they either suffer from slow sampling or lack global coherence. The core challenge remains: how to incorporate temporal dynamics into diffusion models effectively, especially for complex, chaotic, or long sequences.

The main issue is that existing diffusion models struggle with long sequences due to uniform noise scheduling and lack of explicit temporal bias. This results in error accumulation and loss of temporal coherence over extended periods. Additionally, treating time as a spatial dimension leads to high computational costs and limited scalability. The problem becomes more acute in scenarios with complex dynamics, such as fluid turbulence or chaotic motion, where small errors rapidly amplify. Developing a model that can adaptively handle varying temporal complexities, maintain stability, and efficiently generate long sequences remains an open challenge.

The paper introduces a local time reparameterization that maps global diffusion time to frame-dependent local times, enabling sequential denoising within a sliding window. This approach embeds a temporal inductive bias, adding more noise to future frames, reflecting increased uncertainty. It employs a boundary-aware scheduling strategy, allowing indefinite unrolling and stable long-term generation. The architecture combines convolutional and Transformer modules, capturing both local and global dependencies. The method effectively balances computational efficiency with modeling capacity, addressing the core issues of error accumulation and scalability in long sequence generation.

• �� Reparameterize global diffusion time t into per-frame local times tk, with tk ∈ [0,1], ensuring monotonicity. • Use a sliding window W to focus on a subset of frames, applying local noise schedules tWw based on frame index w. • Design a schedule tlinw = w/W + t, providing a linear local time progression. • Incorporate boundary condition schedules tinitw to handle sequence edges, enabling indefinite unrolling. • Train the model with a KL-divergence loss focusing on frames within the sliding window, optimizing local denoising. • Use a combined convolutional-Transformer architecture for capturing spatiotemporal features. • During sampling, traverse the sequence with the sliding window, progressively denoising frames based on local times, maintaining temporal coherence.

Experiments utilized the Kinetics-600 dataset for video prediction and chaotic Navier-Stokes fluid simulations. The model's performance was evaluated using the Fréchet Spectral Distance (FSD), comparing generated and real data in the frequency domain. Ablation studies tested different window sizes, scheduling functions, and boundary conditions. Hyperparameters such as noise schedule and window length were tuned for optimal results. The training employed large-scale GPU clusters, with extensive sampling to assess long-term stability. Results demonstrated that the rolling diffusion outperformed standard models, especially in long sequences, with significant improvements in structural preservation and error control.

The rolling diffusion model achieved a 15% reduction in FSD scores over baseline diffusion models on Kinetics-600, indicating better spectral fidelity. In fluid simulations, vortex structures remained coherent over longer durations, with error metrics substantially lower than traditional approaches. The boundary scheduling effectively mitigated error accumulation at sequence edges, enabling stable long-term generation. The model maintained high structural similarity and physical plausibility, validating the effectiveness of local time scheduling in complex dynamic systems.

This approach is suitable for long-term video synthesis, scientific simulations, and physical system modeling. It can be integrated into virtual reality, autonomous driving, and weather forecasting pipelines, providing more stable and realistic long sequence outputs. Its ability to handle complex, chaotic data makes it valuable for scientific research, enabling more accurate and efficient modeling of natural phenomena. The framework's flexibility allows adaptation to various modalities and scales, promising broad industrial and academic impact.

Despite its advantages, the model still faces challenges in extremely chaotic or noisy environments, where errors can propagate over very long sequences. The computational cost remains high, especially for high-resolution data. Boundary condition handling, while improved, can still introduce artifacts in some cases. Further research is needed to optimize efficiency, robustness, and generalization across diverse dynamic systems.