VideoVeritas: AI-Generated Video Detection via Perception Pretext Reinforcement Learning
VideoVeritas combines perception pretraining and fact-based reasoning, using the MintVid dataset to detect deepfake videos with 85.7% accuracy.
Key Findings
Methodology
VideoVeritas employs perception pretext reinforcement learning (PPRL), integrating spatiotemporal grounding and self-supervised objectives (e.g., object counting) to enhance fine-grained perception. The framework leverages multi-modal large language models (MLLMs) with a preference alignment mechanism, optimizing for fact-based reasoning rather than direct detection. The core algorithms include spatial-temporal grounding and self-supervised objectives within a Transformer-based architecture, enabling multimodal fusion of visual and textual cues. During training, the model learns to localize fake regions and understand content authenticity through auxiliary tasks, improving robustness against sophisticated manipulations.
Key Results
- On the newly introduced MintVid dataset, VideoVeritas achieves an accuracy of 85.7% in fake video detection, outperforming baseline methods such as Xception (75.4%) and FakeAV, with a 10% improvement. In factual content error detection, the F1-score reaches 78.3%, surpassing previous models at 65%. The model demonstrates strong generalization across different generative models like StyleGAN3 and DeepFake, maintaining high performance in cross-source scenarios.
- Ablation studies reveal that incorporating the spatial-temporal grounding task significantly boosts detection of complex fake videos, especially in low-quality clips. Preference alignment reduces superficial reasoning biases, while self-supervised objectives like object counting enhance fine-grained perception. These components collectively lead to consistent performance gains, validating the effectiveness of the proposed approach.
- Across diverse benchmarks, VideoVeritas exhibits robust performance, with notable improvements in real-world content error detection, indicating its potential for practical deployment in content verification systems.
Significance
This research addresses critical challenges in deepfake detection by bridging the gap between perceptual detail recognition and fact-based reasoning. Existing methods often struggle with high-quality fake videos or content with subtle manipulations. By integrating perception pretraining with fact-oriented inference, the framework enhances detection accuracy and robustness, providing a comprehensive solution for safeguarding information integrity. The approach advances the state-of-the-art in multimodal learning, offering a scalable and adaptable paradigm for real-world applications such as social media moderation, news verification, and digital forensics. It also lays a foundation for future research on multi-source, multi-modal fake content detection, contributing to the development of more intelligent and reliable AI systems.
Technical Contribution
The paper introduces a novel combination of perception pretraining and fact-based reinforcement learning, breaking from traditional visual-only detection models. The integration of spatial-temporal grounding and self-supervised objectives within a Transformer architecture enables the model to localize fake regions precisely and understand content authenticity at a granular level. The preference alignment mechanism aligns model biases towards factual content, reducing superficial reasoning. The creation of MintVid, a high-quality dataset with diverse fake videos, provides a robust benchmark for evaluation. These innovations collectively push the boundaries of deepfake detection, offering both theoretical insights and practical tools that outperform existing methods in accuracy and generalization.
Novelty
This work is the first to combine perception pretraining with fact-based reinforcement learning specifically for deepfake video detection. The introduction of spatial-temporal grounding and self-supervised objectives tailored for fake content localization is novel. Unlike prior approaches that rely solely on visual features or coarse classification, this method emphasizes detailed perception and reasoning, enabling detection of subtle manipulations across diverse sources. The use of a preference alignment mechanism further distinguishes it by aligning model biases with factual correctness, setting a new standard in the field.
Limitations
- The model's performance diminishes with extremely low-quality or very short videos, where perceptual cues are sparse or ambiguous, limiting detection accuracy.
- High computational costs during training, especially for multimodal fusion and auxiliary tasks, hinder large-scale deployment in resource-constrained environments.
- Although MintVid covers multiple generators, it cannot encompass all possible fake content types, necessitating ongoing dataset expansion for broader generalization.
Future Work
Future research will focus on developing more efficient architectures to reduce computational overhead, possibly through lightweight transformers or pruning techniques. Expanding the dataset to include emerging fake generation methods and real-world scenarios will improve robustness. Additionally, integrating adversarial training to enhance model resilience against sophisticated attacks and exploring real-time detection capabilities will be prioritized. Cross-modal fusion strategies and explainability methods will also be investigated to facilitate practical deployment and user trust.
AI Executive Summary
Deep Dive
Plain Language Accessible to non-experts
Imagine you’re in a kitchen where a chef is preparing dishes. Some ingredients are fresh and genuine, while others are fake or spoiled. The chef needs to quickly identify which ingredients are real and which are fake to make a good dish. Traditional chefs might just look at the ingredients’ appearance, but clever chefs use smell, touch, and taste to get a better sense. VideoVeritas就像是这个聪明的厨师，它不仅看外表，还能“闻味道”和“尝尝”，通过学习大量的食材图片和描述，变得越来越擅长识别假货。它会观察视频中的细节，理解内容背后的真实性，就像厨师理解食材的来源一样。这样，无论伪造得多逼真，它都能识别出来，确保你看到的内容是真实可靠的。
Abstract
The growing capability of video generation poses escalating security risks, making reliable detection increasingly essential. In this paper, we introduce VideoVeritas, a framework that integrates fine-grained perception and fact-based reasoning. We observe that while current multi-modal large language models (MLLMs) exhibit strong reasoning capacity, their granular perception ability remains limited. To mitigate this, we introduce Joint Preference Alignment and Perception Pretext Reinforcement Learning (PPRL). Specifically, rather than directly optimizing for detection task, we adopt general spatiotemporal grounding and self-supervised object counting in the RL stage, enhancing detection performance with simple perception pretext tasks. To facilitate robust evaluation, we further introduce MintVid, a light yet high-quality dataset containing 3K videos from 9 state-of-the-art generators, along with a real-world collected subset that has factual errors in content. Experimental results demonstrate that existing methods tend to bias towards either superficial reasoning or mechanical analysis, while VideoVeritas achieves more balanced performance across diverse benchmarks.