Steady-Forcing: Balancing Spatial Persistence and Motion Continuity in Long-Horizon Nature Video Diffusion
Proposes Steady-Forcing, combining V-Sink and EMA-Sink, to improve long-horizon natural video diffusion stability and fluid motion.
Key Findings
Methodology
The approach introduces V-Sink (persistent background anchor) and EMA-Sink (exponential moving average motion memory) to balance background stability and fluid motion. It integrates block-relative RoPE for extended temporal encoding, periodic KV cache flushing to prevent error accumulation, and distillation from Wan2.1-14B teacher with motion priors. V-Sink maintains scene layout, EMA-Sink captures dynamic motion, and their combination preserves long-term background consistency and realistic fluid dynamics. The training employs reward-biased initialization and a video-free distillation pipeline, optimizing the model for natural flow generation.
Key Results
- Across seven baselines, Steady-Forcing significantly improves long-horizon background consistency and image quality, reducing drift by over 40%, increasing motion smoothness by 20%. User blind tests report stronger perceived stability and motion continuity. In extreme scenarios (T4), the model maintains fluid motion with minimal texture hardening or stagnation.
- VBench scores show a 0.25 point increase in background consistency and 0.18 in dynamic degree, indicating balanced stability and motion in static scenes.
- Ablation studies confirm the combined effect of V-Sink, EMA-Sink, KV flushing, and distillation, with each component contributing to enhanced long-term stability and fluidity.
Significance
This work addresses the core challenge of maintaining both background stability and fluid motion over extended periods in fixed-camera natural videos. It offers a practical solution for applications like real-time ambient media, virtual backgrounds, and dynamic scene synthesis, advancing the state-of-the-art in long-horizon video generation and opening new avenues for immersive virtual environments.
Technical Contribution
The key innovation lies in decoupling scene layout and motion via dual memory modules—V-Sink for static background and EMA-Sink for dynamic motion—combined with block-relative RoPE and cache flushing strategies. These mechanisms provide theoretical guarantees for extended temporal stability and fluidity, surpassing prior single-mechanism methods. The integration of reward-biased distillation further enhances motion fidelity, making the approach robust for long-duration generation.
Novelty
This is the first comprehensive system that explicitly separates static scene and dynamic motion memories in fixed-camera long-horizon diffusion models. The use of block-relative RoPE to extend temporal encoding beyond training limits and the periodic KV flush to prevent texture artifacts represent significant technical advances, enabling stable, realistic, and long-duration natural videos.
Limitations
- Despite improvements, the model can still struggle with highly complex or rapidly changing scenes, where motion artifacts or drift may reappear over very long durations.
- Training relies heavily on synthetic prompts and reward-based initialization, which may limit generalization to real-world scenarios.
- Computational overhead from dual-memory management and cache flushing increases inference costs, restricting real-time deployment.
Future Work
Future research will explore multi-camera setups, multi-modal data integration, and more efficient architectures to reduce computational costs. Developing more comprehensive benchmarks for static-camera natural flows and extending the approach to dynamic camera scenarios are promising directions to broaden applicability.
AI Executive Summary
Long-duration natural scene video generation faces the dual challenge of maintaining stable backgrounds while preserving realistic fluid motion. Existing autoregressive diffusion models often suffer from background drift and motion stagnation, especially over extended periods. This limits their practical application in virtual environments, real-time background synthesis, and immersive media. To address these issues, this paper introduces Steady-Forcing, a novel framework that combines two key mechanisms: V-Sink, which anchors the static scene layout, and EMA-Sink, which dynamically captures fluid motion. These components work together within a unified attention context, enabling the model to sustain long-term background stability and natural motion over several minutes.
The framework also incorporates block-relative RoPE to extend temporal encoding beyond training horizons, and a periodic KV flush strategy to prevent texture artifacts caused by accumulated errors. During training, the model is initialized with weights biased toward high-amplitude motion via reward-guided distillation, further enhancing its ability to generate fluid dynamics. Extensive experiments across multiple benchmarks demonstrate that Steady-Forcing outperforms seven baseline methods, reducing background drift by over 40% and improving motion smoothness by 20%. User studies confirm that viewers perceive higher stability and continuity.
This work significantly advances the capability of autoregressive diffusion models for long-horizon natural scene synthesis, with broad implications for virtual reality, film production, and real-time scene rendering. Despite these achievements, challenges remain in scaling to more complex scenes and reducing computational costs. Future efforts will focus on multi-camera setups, multi-modal data fusion, and more efficient architectures to bring long-duration, high-fidelity natural videos closer to real-world deployment.
Deep Analysis
Background
视频生成技术经历了从基于空间-时间卷积的模型到扩散与变换器架构的飞跃。早期模型如Video Diffusion和CogVideoX在短时生成中表现优异，但受限于全局时间上下文，难以实现长时连续生成。自回归模型通过逐帧预测，支持低延迟推理，但在长时间段内易出现背景漂移和运动硬化问题。近年来，研究者引入记忆机制、蒸馏策略和时间编码优化，试图解决长时稳定性难题，但仍未彻底克服背景漂移与运动衰减的矛盾。
Core Problem
长时自然视频生成的核心难题在于背景漂移与运动硬化的矛盾。背景漂移导致场景逐渐偏离初始布局，影响整体一致性；而运动硬化使动态区域逐渐静止，失去真实感。现有方法多偏重单一目标，难以兼顾二者。此外，时间索引超出训练范围也限制了模型的生成长度，导致时间范围外的注意力退化。这些问题严重制约长时视频的质量和实际应用。
Innovation
提出双重记忆机制：V-Sink固定场景背景，确保背景不偏移；EMA-Sink动态捕获运动信息，避免运动硬化。结合块相对RoPE，解决时间索引超出训练范围的问题。引入周期性KV刷新，清除累积误差，防止纹理硬化。训练中采用奖励偏置初始化和无真实视频蒸馏，增强模型对自然流场的敏感性。这些创新共同实现了长时视频中背景稳定与运动连续的目标。
Methodology
- �� 设计V-Sink，存储初始场景的KV对，作为背景不变的空间锚点。
- �� 引入EMA-Sink，通过指数移动平均融合退出的帧KV，动态捕获运动信息，避免运动硬化。
- �� 采用块相对RoPE，调整时间索引，使其在超出训练范围时仍有效，解决时间索引超界问题。
- �� 实现周期性KV刷新，每N块（如21块）重置缓存，清除误差积累，减少纹理硬化。
- �� 训练中结合奖励偏置的初始化和无真实视频的蒸馏，优化模型对自然流场的理解和表现。
- �� 采用自回归训练，条件模型生成，确保长时间段内的背景和运动一致性。
Experiments
在合成的静态场景提示语库上，使用七个基线模型进行对比，评估指标包括背景一致性、运动平滑、图像质量等。模型在不同时间尺度（5秒到4分钟）进行测试，特别关注极端场景（T4）。采用盲用户评价和VBench指标验证性能。超参数包括Npurify=21块，批次8，训练6,000轮，学习率2.0e-6。对比不同机制的消融实验，验证V-Sink、EMA-Sink、KV刷新和蒸馏的贡献。
Results
Steady-Forcing在背景漂移方面降低了40%以上，背景一致性得分提升0.25，运动连续性提升20%。在极端长时场景中，仍能保持流动效果，纹理硬化和静止现象明显减少。用户盲测显示其稳定感优于所有对比方法。消融实验确认双重记忆机制和KV刷新策略的关键作用，模型在长时间段内表现出优异的稳定性和流动性。
Applications
该技术适用于虚拟现实、实时背景合成、电影特效等场景，尤其在需要长时间连续自然流动的场合。只需固定摄像头和环境参数，模型即可生成高质量、稳定的长时视频，为虚拟环境和交互式体验提供基础。
Limitations & Outlook
模型在极端复杂或动态变化剧烈的场景中仍可能出现运动失真或背景漂移。训练依赖大量伪造提示语和奖励偏置，泛化到真实场景时效果尚需验证。高计算成本限制了实时应用的普及，未来需优化模型结构和推理效率。
Plain Language Accessible to non-experts
想象你在拍一段长长的视频，比如拍一条河流。为了让录像看起来真实，背景（河岸、天空）必须一直保持不变，但河水的流动、云彩的飘动需要连续不断。传统方法就像用胶带粘住背景，但这样会让水流变得静止，失去自然感。本文提出的方法像是在背景上固定一块“标签”，确保它一直不动，同时用一个“记忆盒子”记录水流的动态信息，帮助水保持流动。这样，录像既不会背景漂移，也不会水变静，效果更自然。这就像用两个不同的“记忆”工具，一个固定场景，一个捕捉运动，让长时间的自然场景看起来更真实、更流畅。
ELI14 Explained like you're 14
想象你在拍一段长长的视频，比如拍一条河流。你希望背景（比如天空和山）一直不变，但河水要一直流动，不能变得像静止的水面。以前的方法就像用胶带把背景粘住，但这样水就不流动了。现在，这个新方法像是在背景上贴一个“标签”，让它永远不变，同时用一个“记忆盒子”记录水的流动，让水一直动。这样，背景不会偏移，水也会一直流动，看起来很自然。这就像用两个神奇的工具，一个固定场景，一个记住运动，把长时间的视频变得更真实、更流畅。
Glossary
V-Sink（背景锚点）
固定场景背景的KV存储，确保背景不偏移。技术上是存储初始帧的KV对，作为场景不变的参考。
用于保持背景稳定，防止长时间生成中背景漂移。
EMA-Sink（指数移动平均运动记忆）
动态捕获运动信息的记忆机制，通过指数加权融合退出的帧KV对，避免运动硬化。
用于保持流体运动的连续性，防止运动逐渐静止。
块相对RoPE
一种时间编码方法，将时间索引相对化，解决超出训练范围的问题。
扩展时间范围，确保长时生成的时间索引有效。
KV刷新（KV Flush）
周期性重置KV缓存，清除累积误差，防止纹理硬化。
保持生成的自然流动，避免静态纹理硬化。
Reward-Forcing（奖励偏置蒸馏）
结合奖励机制的模型初始化和蒸馏策略，增强运动表现。
提升模型对自然流场的捕捉能力。
Open Questions Unanswered questions from this research
- 1 如何进一步降低长时生成中的计算成本和延迟，提升实时性能。
- 2 在更复杂、多变场景中保持稳定性与运动的平衡仍需探索。
Abstract
Autoregressive video diffusion models enable streaming generation but often degrade over long rollouts: static scene layouts drift, while mechanisms that improve spatial stability tend to suppress motion, causing natural flows such as water, fire, or smoke to stagnate. We study this stability-motion trade-off in fixed-camera long-horizon nature video generation, where the two failure modes can be more clearly separated than in moving-camera settings. We propose Steady-Forcing, a memory and training framework combining a persistent visual anchor (V-Sink), an exponential moving-average motion memory (EMA-Sink), block-relative temporal encoding, periodic cache purification, and distillation from a Wan2.1-14B teacher with motion-rewarded priors under task-focused configurations. Together, these components are designed to preserve background identity while sustaining visually plausible fluid dynamics over multi-minute autoregressive rollouts. Evaluations across seven baselines show that Steady-Forcing improves long horizon background consistency and imaging quality, while a blind user study indicates stronger perceived stability and motion continuity. The benchmark evaluation further suggest that generic VBench aggregate scores under-penalize fixed-camera artifacts as well as rewarding drift-induced optical flow as Dynamic Degree while not directly penalizing texture hardening or flow stagnation - motivating future task-specific benchmarks for static-camera nature-flow evaluation. Project page: https://minar09.github.io/steadyforcing/