Collaborative Few-Step Distillation and Low-Bit Quantization for Wan2.2 Dual-Expert Video Diffusion Models
Proposes Wan2.2 video diffusion model compression via few-step distillation and low-bit quantization, achieving superior efficiency.
Key Findings
Methodology
This work integrates distribution-matching-based few-step distillation with HiF4-style low-bit floating point quantization, tailored for Wan2.2’s dual-expert architecture. The approach involves stage-wise calibration of high-noise and low-noise branches, protecting sensitive layers, and aligning activation distributions during calibration. Shortening the denoising trajectory through distillation reduces the gap between training and deployment distributions. Quantization is performed on the distilled model, calibrated under the same few-step inference setting, ensuring minimal activation mismatch. Experimental results show that the quantized model surpasses the full-precision baseline at 8 and 20 steps, with 20 steps providing the best quality-efficiency trade-off.
Key Results
- The 20-step distilled and quantized model achieves an average VBench score of 0.7074, outperforming the original full-precision model (0.6976) and the same-step full-precision model (0.7051). It maintains high subject consistency, aesthetic quality, and motion smoothness, with significant reductions in inference time and memory footprint. The approach effectively balances quality and efficiency, demonstrating superior performance in preserving details, motion continuity, and semantic alignment across various metrics.
- Across different step settings, the quantized model at 20 steps exceeds baseline scores, confirming that distribution-aligned calibration effectively mitigates quantization errors. The model demonstrates robustness in temporal and spatial fidelity, with performance gains most notable in subjective and motion metrics. Results indicate that fewer steps with quantization can outperform longer trajectories without quantization, highlighting the method’s practical value.
- Comparative analysis shows that the 20-step configuration achieves optimal trade-offs, with performance close to or better than the original model while requiring fewer denoising evaluations. The approach significantly reduces computational costs, making high-quality video generation feasible on resource-constrained devices. The experimental validation underscores the effectiveness of the combined distillation and quantization strategy for real-world deployment.
Significance
This research addresses the critical challenge of deploying large-scale video diffusion models efficiently. By combining few-step distillation with low-bit quantization, it reduces inference time, memory usage, and energy consumption without sacrificing quality. The methodology enables high-fidelity video synthesis on edge devices and real-time applications, bridging the gap between research and practical deployment. Its innovative handling of dual-expert pathways and activation distribution alignment sets a new standard for model compression in multimodal generative tasks, promising broader adoption of AI-driven video content creation.
Technical Contribution
The paper introduces a novel co-design framework integrating distribution-matching distillation with HiF4 low-bit floating point quantization, specifically tailored for multi-path MoE architectures. It develops a stage-aware calibration strategy that preserves the distinct activation distributions of high-noise and low-noise branches, protecting sensitive layers. The approach aligns the quantization process with the shortened inference trajectory, significantly reducing activation mismatch. The experimental validation demonstrates that the method maintains or surpasses baseline performance at a fraction of the computational cost, opening new avenues for deploying large diffusion models efficiently.
Novelty
This work is the first to systematically combine few-step distillation with HiF4 low-bit quantization for large dual-expert video diffusion models. Unlike prior methods focusing solely on image models or single-path architectures, it addresses the unique challenges of multi-path MoE structures, including activation distribution heterogeneity and path-specific calibration. The stage-aware calibration and entrance-layer protection strategies are novel contributions that effectively mitigate quantization-induced errors, enabling high-quality, resource-efficient video synthesis. This integrated approach sets a new benchmark in model compression for multimodal generative models.
Limitations
- The current framework relies on post-training quantization and does not include end-to-end training, which may limit performance under extreme compression scenarios.
- Calibration depends on representative prompts; mismatched calibration data could lead to performance degradation in unseen deployment scenarios.
- The approach’s effectiveness on models with even larger parameters or more complex architectures remains to be validated, and further optimization may be needed for real-time applications.
Future Work
Future directions include integrating joint training and quantization-aware training to further improve robustness. Developing adaptive, dynamic path selection and calibration strategies can enhance performance across diverse scenarios. Exploring prompt-aware calibration and mixed-precision schemes may yield better trade-offs. Additionally, hardware-aware optimization and real-time deployment on edge devices will be key to broadening practical applications.
AI Executive Summary
The rapid growth of video content demands efficient generation methods that balance quality and computational cost. Large-scale diffusion models like Wan2.2 have demonstrated remarkable visual fidelity but are hindered by their extensive sampling steps and massive parameter sizes. This paper introduces a novel solution combining few-step distillation with low-bit quantization, tailored for Wan2.2’s dual-expert architecture. The core idea is to shorten the denoising trajectory via distribution-matching distillation, which aligns the student model’s intermediate states with the teacher’s, reducing the number of required steps. Concurrently, the model undergoes HiF4-style low-bit quantization, which preserves dynamic range and detail within a minimal bit budget. A stage-aware calibration strategy ensures that activation distributions in high-noise and low-noise branches are accurately aligned, with sensitive layers protected to prevent early error propagation. The entire pipeline is designed to maintain the performance of the original model while significantly reducing inference time and memory footprint.
Experimental results on the VBench dataset show that the 20-step quantized model surpasses the original full-precision baseline in key metrics such as Subject Consistency, Aesthetic Quality, and Motion Smoothness. Notably, it achieves a VBench score of 0.7074, compared to 0.6976 for the baseline, while requiring fewer denoising steps. The approach effectively balances quality and efficiency, making high-quality video synthesis feasible on resource-constrained devices. The method’s success demonstrates that careful co-design of distillation and quantization can unlock the practical deployment of large diffusion models, opening new avenues for real-time, high-fidelity video applications. Future work will focus on end-to-end training, adaptive calibration, and hardware optimization, aiming to further enhance robustness and scalability in diverse deployment scenarios.
Deep Dive
Abstract
Large video diffusion models achieve strong visual quality but remain expensive to deploy because each sample requires many denoising steps and a large resident parameter footprint. This paper studies a deployment-oriented compression pipeline for Wan2.2-T2V-A14B by combining few-step distribution-matching distillation with low-bit quantization. The pipeline follows the model's dual-expert denoising route, calibrates the high-noise and low-noise branches separately, protects sensitive entrance layers, and uses HiF4-style low-bit representation to improve dynamic-range coverage. Quantization is calibrated on the distilled few-step student rather than on the original long-step trajectory, reducing activation-distribution mismatch during inference. The proposed co-design keeps the quantized model close to the same-step full-precision model and surpasses the original full-precision baseline at 8 and 20 steps on average. The 20-step setting gives the best quality-efficiency trade-off in the tested configurations.