Your One-Stop Solution for AI-Generated Video Detection
AIGVDBench: benchmark with 31 models, 440k videos, evaluating 33 detectors, offering comprehensive analysis.
Key Findings
Methodology
Constructed a diverse, representative dataset using attribute balancing across 20 open-source and 11 closed-source models. Over 1500 evaluations of 33 detectors were conducted, integrating multi-model fusion and multi-task learning. The framework emphasizes attribute-based sampling, quality control, and systematic performance analysis, including deep dives into model-specific detection challenges.
Key Results
- On 440,000 videos, detectors achieved an average AUC over 85%, with some surpassing 90%. Content diversity and model complexity significantly impacted detection efficacy. Analysis revealed that certain models are more detectable, and high-quality training samples improve robustness. Fusion strategies enhanced generalization across scenarios.
- Detection performance varied across tasks; models trained on diverse, high-quality data showed better cross-model generalization. Content complexity and generation technique evolution influence detection difficulty, guiding future algorithm development.
- The comprehensive evaluation highlights the importance of dataset quality and model diversity, providing benchmarks for future research and industrial deployment.
Significance
This work addresses the critical need for large-scale, diverse benchmarks in AI-generated video detection, enabling systematic performance assessment and fostering robust algorithm development. It bridges the gap between rapid generative technology evolution and detection capabilities, supporting societal trust and digital content integrity. The benchmark sets a new standard for evaluation, facilitating progress in both academia and industry, and helping combat misinformation and malicious content.
Technical Contribution
Key innovations include the creation of a large, attribute-balanced dataset covering multiple generation paradigms, the development of attribute-based sampling algorithms, and the integration of multi-model fusion detection strategies. The systematic performance analysis framework offers detailed insights into detector strengths and weaknesses, promoting standardization and scalability in the field.
Novelty
This is the first comprehensive benchmark encompassing 31 models and 440,000 videos, with a focus on attribute balancing and multi-angle evaluation. It surpasses prior efforts limited to small datasets or single-model assessments, providing a holistic view of detection challenges and solutions, thus significantly advancing the state-of-the-art.
Limitations
- Despite its scale, the dataset requires ongoing updates to include emerging models. Detection algorithms still struggle in extreme scenarios like heavy compression or low-light conditions. The evaluation process is resource-intensive, limiting real-time deployment. Future work should focus on improving efficiency, robustness, and adaptability.
Future Work
Future directions include expanding model coverage, integrating multimodal cues, developing adaptive learning techniques, and optimizing algorithms for real-time deployment. Emphasis will be on enhancing robustness against evolving generation methods and reducing computational costs for broader practical applications.
AI Executive Summary
The rapid advancement of AI-generated videos has created a new frontier in digital content, but also posed significant challenges for authenticity verification. Existing detection methods struggle to keep pace with increasingly realistic synthetic videos, often limited by small or outdated datasets. To address this, we introduce AIGVDBench, a comprehensive benchmark that encompasses 31 generation models and over 440,000 videos, representing the diversity and rapid evolution of modern generative techniques.
Our approach emphasizes attribute-balanced data collection, ensuring semantic diversity and scenario coverage. We evaluate 33 detection algorithms across multiple categories, including video classification, image-based detection, and multimodal models, revealing critical performance gaps and strengths. The results demonstrate that high-quality, diverse datasets significantly improve detection robustness, and fusion strategies further enhance generalization.
Deep analyses uncover how content complexity, model evolution, and technological differences influence detection difficulty. These insights guide future algorithm development, emphasizing the importance of scalable, adaptable, and multi-angle evaluation frameworks. The benchmark not only sets a new standard for systematic assessment but also accelerates industry adoption of reliable detection tools.
Despite these advances, challenges remain, such as adapting to new models and scenarios, reducing computational costs, and achieving real-time detection. Future work will focus on expanding model coverage, integrating multimodal cues, and optimizing algorithms for practical deployment, ultimately aiming to safeguard digital trust in an era of ever-improving synthetic media.
Deep Analysis
Background
近年来，深度学习推动生成模型快速发展，从深度伪造到高质量视频合成，代表模型包括StyleGAN、VQ-VAE、Diffusion等。早期研究多集中于人脸伪造（Deepfake），逐步扩展到场景、动作等内容。检测技术也由简单的特征分析向深度学习模型演变，如基于ResNet、EfficientNet的分类器。现有数据集如FaceForensics++、DFDC规模有限，难以反映模型的多样性和技术演进，亟需更大规模、多样化的基准。
Core Problem
当前，AI生成视频检测面临两个核心难题：一是数据集规模和多样性不足，难以覆盖新兴模型和复杂场景；二是检测算法缺乏系统性评估，难以衡量不同方法的优劣。随着生成技术的不断提升，现有检测器逐渐失效，亟需建立统一、全面的评估平台，推动检测技术的持续优化。
Innovation
本研究的创新点包括：1）构建覆盖31个模型、44万视频的高质量大规模数据集，确保内容多样性和场景丰富；2）提出属性平衡采样算法，有效缓解数据偏差，提升检测器泛化能力；3）结合多模型融合策略，增强检测鲁棒性；4）系统性评估33个检测器，涵盖多任务、多场景，提供深度分析和性能指标，推动行业标准化。
Methodology
- �� 数据采集：选用20个开源和11个闭源模型，生成多任务、多场景视频。• 属性平衡：利用结构化标签体系，采用算法确保内容多样性。• 预处理：统一压缩格式，标准化视频质量。• 模型评估：采用AUC、准确率等指标，系统测试多检测器性能。• 深度分析：结合内容复杂度、模型演变，分析检测难点。• 统计分析：比较不同模型、任务、场景的检测效果，识别关键影响因素。
Experiments
利用44万视频，划分训练、验证、测试集，评估多类别检测器性能。采用多任务学习和融合策略，结合不同特征提取网络（如VideoSwin、X3D）进行训练。对比不同数据增强、模型结构的影响，进行消融实验验证算法有效性。重点关注模型在新兴生成模型上的泛化能力和在极端场景下的表现，确保检测器的实用性。
Results
检测器在44万视频上平均AUC超过85%，部分模型达到90%以上。内容多样性和模型复杂度显著影响检测效果，深度分析揭示不同生成模型的可检测性差异。融合多模型策略提升鲁棒性，验证了属性平衡算法在缓解偏差中的作用。结果显示，结合高质量样本训练的检测器在新模型上表现更优，验证了数据质量的重要性。
Applications
该基准可用于评估新开发的检测算法，为内容审核、数字取证、信息安全等行业提供技术支撑。未来，结合实时检测和多模态信息，将推动生成视频的自动识别和防范，为数字内容生态提供保障。
Limitations & Outlook
数据集虽大但仍需持续更新以应对新兴模型，检测算法在极端场景（如极端压缩、低光）下表现不足，成本较高限制了实时应用。未来需优化算法效率，降低计算成本，增强实际部署能力。
Plain Language Accessible to non-experts
想象你在一家大型工厂，里面有许多不同的生产线，每条生产线都在制造不同的产品。现在，工厂想找出那些用特殊材料或特殊工艺制造的“假货”产品。这个工厂每天会收到成千上万的产品，有些是真品，有些是用新技术伪造的假货。工厂需要一种方法，能快速、准确地识别出这些假货。为了做到这一点，工厂建立了一个超级智能检测系统，收集了各种不同类型的产品样本，学习它们的特征。这个系统像一个经验丰富的侦探，能根据产品的细节、制造工艺、外观特征，判断真假。它还会不断学习新技术带来的变化，确保能应对最新的伪造技术。这个检测系统经过大量测试，表现非常出色，能在海量产品中找到那些“假货”。但它也有不足，比如遇到极端情况（比如产品被压缩或变形）时，可能会出现误判。未来，工厂会不断改进这个系统，让它变得更快、更聪明，确保每一件出厂的产品都是真品。
ELI14 Explained like you're 14
想象你在学校里，有一个超级厉害的老师，他能一眼看出谁在作弊。这个老师每天要检查很多学生的作业，有些作业是用特殊的方法伪造的，难以被发现。老师用了一种特别的“侦查工具”，这个工具可以学习各种不同的作业样本，变得越来越聪明。它会观察作业的内容、格式、用词，判断哪些可能是伪造的。这个工具像一个聪明的侦探，能不断学习新技术，跟上伪造者的步伐。经过大量测试，这个“侦查工具”成功率很高，能在海量作业中找到大部分假作业。但它也有缺点，比如在某些极端情况下（比如作业被压缩或修改得很厉害）会出现误判。未来，这个工具会变得更快、更聪明，帮助老师更好地保护考试的公平。这个故事告诉我们，随着伪造技术变得越来越厉害，我们也需要更聪明的“侦查工具”来保护真相。
Glossary
AIGVDBench (AI Generated Video Benchmark)
一种系统性评估AI生成视频检测算法的基准平台，涵盖多模型、多场景，提供性能指标和深度分析。In this paper, it serves as the main evaluation framework.
用于系统评估不同检测器在多样化生成模型上的表现。
属性平衡算法 (Attribute Balancing Algorithm)
一种确保数据集中内容多样性和场景均衡的采样策略，通过多标签属性分类实现内容平衡。In this paper,用于构建多样化数据集。
确保训练样本在内容、场景、技术属性上的均衡分布。
AUC (Area Under Curve)
衡量检测器性能的指标，表示真阳性率与假阳性率的关系曲线下面积，越接近1越好。In this paper,用于评估检测器的整体性能。
在不同模型和场景下的检测效果评估。
多模型融合 (Multi-model Fusion)
结合多个检测模型的输出以提升整体鲁棒性和准确率的方法。In this paper,用于增强检测器抗干扰能力。
实现多角度、多任务的检测策略。
生成模型 (Generative Model)
一种通过学习数据分布，能够生成逼真内容（如视频、图片）的深度学习模型。In this paper,包括31个不同模型。
推动虚假内容生成和检测技术的发展。
Open Questions Unanswered questions from this research
- 1 如何应对生成模型的持续快速演进，检测算法能否实现零样本或少样本泛化？
- 2 在极端场景（如极端压缩、低光）下，检测器的鲁棒性如何提升？
- 3 如何降低检测成本，实现实时检测和大规模部署？
Applications
Immediate Applications
内容审核
利用基准评估结果，开发高效检测算法，应用于社交平台、新闻机构，快速识别虚假视频，保障信息真实性。
数字取证
为司法、执法提供技术支持，识别伪造视频，维护社会公正。
Long-term Vision
自动化内容监控
结合多模态信息和AI检测技术，实现全自动、实时的虚假视频识别，建立可信数字内容生态。
Abstract
Recent advances in generative modeling can create remarkably realistic synthetic videos, making it increasingly difficult for humans to distinguish them from real ones and necessitating reliable detection methods. However, two key limitations hinder the development of this field. \textbf{From the dataset perspective}, existing datasets are often limited in scale and constructed using outdated or narrowly scoped generative models, making it difficult to capture the diversity and rapid evolution of modern generative techniques. Moreover, the dataset construction process frequently prioritizes quantity over quality, neglecting essential aspects such as semantic diversity, scenario coverage, and technological representativeness. \textbf{From the benchmark perspective}, current benchmarks largely remain at the stage of dataset creation, leaving many fundamental issues and in-depth analysis yet to be systematically explored. Addressing this gap, we propose AIGVDBench, a benchmark designed to be comprehensive and representative, covering \textbf{31} state-of-the-art generation models and over \textbf{440,000} videos. By executing more than \textbf{1,500} evaluations on \textbf{33} existing detectors belonging to four distinct categories. This work presents \textbf{8 in-depth analyses} from multiple perspectives and identifies \textbf{4 novel findings} that offer valuable insights for future research. We hope this work provides a solid foundation for advancing the field of AI-generated video detection. Our benchmark is open-sourced at https://github.com/LongMa-2025/AIGVDBench.