Mind the Generative Details: Direct Localized Detail Preference Optimization for Video Diffusion Models
LocalDPO enhances video diffusion models by optimizing localized details, outperforming on Wan2.1 and CogVideoX.
Key Findings
Methodology
LocalDPO is a post-training framework that constructs localized preference pairs from real videos and optimizes alignment at the spatio-temporal region level. It features an automated pipeline that efficiently collects preference pair data with a single inference per prompt, eliminating the need for external critic models or manual annotation. Specifically, high-quality real videos are treated as positive samples, and corresponding negatives are generated by locally corrupting them with random spatio-temporal masks and restoring only the masked regions using the frozen base model. A region-aware DPO loss is introduced to restrict preference learning to corrupted areas, enabling rapid convergence.
Key Results
- LocalDPO consistently improves video fidelity, temporal coherence, and human preference scores over other methods on Wan2.1 and CogVideoX, with a visual quality improvement of approximately 5%.
- Compared to traditional DPO, LocalDPO reduces the time cost of generating preference pairs by about 30%, enhancing efficiency.
- LocalDPO significantly reduces flickering and inconsistent motion in localized detail generation.
Significance
LocalDPO addresses common issues of localized artifacts and motion inconsistency in existing video generation models through localized detail optimization, providing a more efficient and fine-grained paradigm for video generator alignment. This method not only enhances the quality of generated videos but also offers new insights for subsequent model optimization.
Technical Contribution
LocalDPO breaks through the limitations of traditional DPO methods by constructing localized preference pairs and introducing a region-aware loss, offering more refined preference learning capabilities. Its automated pipeline reduces the need for manual annotation, improving training efficiency.
Novelty
LocalDPO is the first to introduce localized preference optimization in video generation, generating negative samples through localized corruption and restoration, significantly improving the efficiency and consistency of preference pair generation.
Limitations
- LocalDPO may experience localized detail loss when handling complex scenes, affecting generation quality.
- The method relies heavily on the base model's performance, potentially limiting its application on lower-performance models.
Future Work
Future research could explore the application of LocalDPO in different types of video generation tasks and combine it with other optimization techniques to further improve generation quality and efficiency.
AI Executive Summary
Video generation models often face issues with inconsistent localized details when generating high-quality videos. Existing preference optimization methods rely on multi-sample ranking and task-specific critic models, which are inefficient and provide ambiguous supervision signals. LocalDPO addresses this problem by constructing localized preference pairs from real videos and optimizing alignment at the spatio-temporal region level.
LocalDPO features an automated pipeline that efficiently collects preference pair data with a single inference per prompt, eliminating the need for external critic models or manual annotation. Specifically, high-quality real videos are treated as positive samples, and corresponding negatives are generated by locally corrupting them with random spatio-temporal masks and restoring only the masked regions using the frozen base model. A region-aware DPO loss is introduced to restrict preference learning to corrupted areas, enabling rapid convergence.
Experimental results show that LocalDPO consistently improves video fidelity, temporal coherence, and human preference scores over other post-training approaches, establishing a more efficient and fine-grained paradigm for video generator alignment.
Deep Analysis
Background
Recent advances in diffusion models have enabled significant progress in text-to-video generation, aiming to synthesize temporally coherent and semantically aligned videos from language prompts. However, generated videos often suffer from artifacts such as flickering objects, inconsistent motions, or implausible local details. A straightforward approach to further improve generation quality is supervised fine-tuning (SFT) on curated collections of high-quality real videos, which directly aligns the model with human-preferred outputs. However, SFT treats all training samples equally and lacks an explicit mechanism to learn from relative quality differences, making it insensitive to subtle but perceptually critical artifacts.
Core Problem
Existing video DPO approaches require generating multiple videos per prompt and ranking them using human annotations or a fine-tuned critic model, leading to heavy model-inference and high annotation cost. Preference pairs are typically based on overall scores that aggregate multiple quality dimensions, ignoring region-specific preference cues, which are critical to human subjective perception.
Innovation
LocalDPO addresses these limitations by achieving preference learning at the level of local video details. Instead of generating multiple videos and relying on human or model-based annotations, LocalDPO directly uses high-quality real videos as positive samples and corrupts local regions of these videos using the model to be optimized, thereby generating corresponding negative samples with only single inference per prompt.
Methodology
- �� Use random spatio-temporal masks to generate locally corrupted videos.
- �� Use the frozen base model to restore masked regions.
- �� Introduce a region-aware DPO loss to restrict preference learning to corrupted areas.
- �� Collect preference pair data with a single inference per prompt using an automated pipeline.
Experiments
Experiments were conducted on Wan2.1 and CogVideoX, using a 63K high-quality video dataset for fine-tuning. Comparisons were made with baseline models, SFT, and traditional DPO methods, with evaluation metrics including video fidelity, temporal coherence, and human preference scores.
Results
LocalDPO consistently improves video fidelity, temporal coherence, and human preference scores over other methods, with a visual quality improvement of approximately 5%. Compared to traditional DPO, LocalDPO reduces the time cost of generating preference pairs by about 30%.
Applications
LocalDPO can be used to enhance the quality of video generation models, particularly in tasks requiring high fidelity and temporal coherence, such as film production, virtual reality, and game development.
Limitations & Outlook
LocalDPO may experience localized detail loss when handling complex scenes, affecting generation quality. The method relies heavily on the base model's performance, potentially limiting its application on lower-performance models. Future research could explore the application of LocalDPO in different types of video generation tasks and combine it with other optimization techniques to further improve generation quality and efficiency.
Plain Language Accessible to non-experts
Imagine you're in a kitchen making a complex dish. Traditional methods would prepare all ingredients at once and cook them together, but this might lead to some ingredients being overcooked or undercooked. LocalDPO's method is like preparing all ingredients first, then handling each one according to its characteristics, like sautéing onions first, adding meat later, and finally vegetables. This ensures each ingredient is cooked to perfection, resulting in a tastier dish. Similarly, LocalDPO optimizes localized details to ensure every part of the video generation is of the highest quality.
ELI14 Explained like you're 14
Hey there! Imagine you're playing a super cool game where you need to build the perfect city. Traditional methods would build all the buildings at once, but this might make some buildings not strong enough. LocalDPO's method is like building the infrastructure first, then gradually perfecting each building, ensuring every detail is spot on. This way, your city not only looks better but can also withstand enemy attacks better!
Glossary
Diffusion Model
A generative model that creates data by progressively denoising. Used as the base model for video generation.
Used as the base model for video generation.
Direct Preference Optimization
A method for fine-tuning models using preference pairs. Used to align generative models with human preferences.
Used to align generative models with human preferences.
Spatio-temporal Mask
A mask used to select localized regions in a video for processing. Used to generate locally corrupted videos.
Used to generate locally corrupted videos.
Region-aware DPO Loss
A loss function that restricts preference learning to specific regions. Used to accelerate model convergence.
Used to accelerate model convergence.
Frozen Base Model
A pre-trained model that is not involved in training. Used to restore locally corrupted video regions.
Used to restore locally corrupted video regions.
Open Questions Unanswered questions from this research
- 1 How to maintain localized detail integrity in complex scenes? Current methods may experience detail loss in complex scenarios, requiring further research.
- 2 How to reduce reliance on base model performance? LocalDPO's dependence on base model performance limits its application on lower-performance models.
Applications
Immediate Applications
Film Production
Enhance the quality of special effects scenes in films, reducing post-production time.
Virtual Reality
Enhance visual realism and temporal coherence in VR experiences.
Long-term Vision
Game Development
Develop more immersive and interactive game worlds by improving generation quality.
Abstract
Aligning text-to-video diffusion models with human preferences is crucial for generating high-quality videos. Existing Direct Preference Otimization (DPO) methods rely on multi-sample ranking and task-specific critic models, which is inefficient and often yields ambiguous global supervision. To address these limitations, we propose LocalDPO, a novel post-training framework that constructs localized preference pairs from real videos and optimizes alignment at the spatio-temporal region level. We design an automated pipeline to efficiently collect preference pair data that generates preference pairs with a single inference per prompt, eliminating the need for external critic models or manual annotation. Specifically, we treat high-quality real videos as positive samples and generate corresponding negatives by locally corrupting them with random spatio-temporal masks and restoring only the masked regions using the frozen base model. During training, we introduce a region-aware DPO loss that restricts preference learning to corrupted areas for rapid convergence. Experiments on Wan2.1 and CogVideoX demonstrate that LocalDPO consistently improves video fidelity, temporal coherence and human preference scores over other post-training approaches, establishing a more efficient and fine-grained paradigm for video generator alignment.The code is available at https://github.com/1170300714/Local-DPO.