VidForensics-M1: Meta-Detection Reinforcement Learning with Verifiable Temporal Grounding for AI-Generated Video Forensics
Meta-detection with verifiable temporal grounding and reward redistribution boosts AI-generated video detection F1 to 75.39%, outperforming existing methods.
Key Findings
Methodology
This paper introduces VidForensics-M1, a novel framework integrating meta-detection within reinforcement learning (RL) to enhance AI-generated video detection. The core innovation lies in combining verifiable temporal grounding (VTG) with an evidence-guided reward redistribution (EGRR) mechanism. The process begins with automated data construction, where real videos from datasets like InternVid and ActivityNet are segmented randomly. Boundary frames are extracted, and a boundary-conditioned video generation model reconstructs missing segments, creating paired real-fake videos with ground-truth manipulated intervals. During training, the detector jointly optimizes label correctness and evidence trustworthiness, with the RL policy guided by a reward function that incorporates evidence quality. The EGRR mechanism dynamically reallocates rewards among responses based on the reliability of temporal evidence, measured via IoU. This approach ensures the detector learns fine-grained, verifiable localization of forgery artifacts, improving robustness against unseen generators and complex forgery scenarios. The framework employs specific algorithms such as boundary-frame conditioned generation, IoU-based temporal supervision, and evidence-aware reward calibration, validated on datasets including ViF-Bench, achieving significant performance gains.
Key Results
- On ViF-Bench, VidForensics-M1 achieved an F1 score of 75.39%, surpassing baseline methods like BusterX++ (56.43%) and DeepTraceReward (66.39%). The model demonstrated a 20% improvement in detecting complex forgeries with blurred or deformed segments, especially in challenging scenarios involving unseen generators.
- The temporal grounding component provided precise localization, with IoU scores reaching 0.81, significantly higher than previous approaches (~0.27). This precise localization enabled the detector to identify forgery boundaries with high accuracy, even in subtle manipulations.
- The evidence-guided reward redistribution effectively mitigated reward hacking, leading to more stable training and better generalization. The model maintained high detection accuracy across diverse datasets and forgery techniques, validating the robustness of the proposed approach.
Significance
This work addresses fundamental limitations in existing AI-generated video detection methods, notably the reliance on coarse supervision signals and vulnerability to unseen forgery techniques. By integrating verifiable temporal evidence into a meta-detection framework, the authors provide a scalable, interpretable, and robust solution. The approach enhances detection accuracy, especially in complex scenarios with subtle manipulations, and offers a new paradigm for trustworthy digital media verification. Its implications extend to social media moderation, legal evidence collection, and content authenticity verification, marking a significant step toward industry-standard standards for deepfake detection.
Technical Contribution
The paper's main technical contributions include the design of an automated data construction pipeline leveraging boundary-frame-conditioned generation to produce paired real-fake videos with ground-truth temporal intervals. It introduces the meta-detection framework that jointly optimizes label correctness and evidence validity within RL. The core algorithm, Evidence-Guided Reward Redistribution (EGRR), dynamically reallocates rewards based on evidence quality, preserving label supervision while emphasizing reliable forgery cues. Theoretical analysis confirms that EGRR maintains the original label-level objective, ensuring training stability. The integration of IoU-based temporal supervision with evidence calibration constitutes a novel combination, enabling fine-grained localization and robust detection across diverse scenarios.
Novelty
This research is the first to embed meta-detection into AI-generated video forensics, explicitly leveraging verifiable temporal grounding as a supervision signal. Unlike prior works that depend on textual rationales or superficial features, this method employs rule-based temporal intervals derived from controlled forgery construction, ensuring objective and scalable supervision. The introduction of evidence-guided reward redistribution within RL to enhance fine-grained localization and generalization is a novel contribution, bridging the gap between interpretability and robustness in deepfake detection. These innovations collectively push the frontier of digital media forensics, setting a new standard for verifiable and generalizable detection frameworks.
Limitations
- The effectiveness of boundary-frame conditioned generation depends heavily on the quality of the generative model; poor generation may lead to inaccurate temporal localization and supervision errors.
- Automated data construction assumes controlled manipulation scenarios; in real-world cases with complex, multi-faceted forgeries, the temporal intervals may be less precise, affecting supervision reliability.
- The training process involves complex hyperparameter tuning for reward calibration and temporal IoU thresholds, which may limit scalability and ease of deployment in diverse environments.
Future Work
Future research will explore multi-modal evidence integration, such as audio cues and motion patterns, to further improve detection robustness. Extending the framework to handle more complex, multi-region forgeries and real-world noisy data is also a priority. Additionally, developing lightweight models suitable for real-time deployment and incorporating explainability techniques to interpret detection decisions will be crucial for practical applications. The authors also suggest investigating unsupervised or semi-supervised approaches to reduce reliance on annotated data, aiming for broader applicability across different media types and forgery techniques.
AI Executive Summary
The rapid proliferation of AI-generated videos has revolutionized digital content creation, enabling highly realistic synthetic media. However, this technological leap has also introduced significant challenges in verifying content authenticity, fueling misinformation and eroding trust in digital media. Traditional detection methods, primarily based on supervised learning or superficial feature analysis, struggle to keep pace with sophisticated generative models like DeepTraceReward, BusterX++, and others. These methods often rely on coarse supervision signals, which limit their ability to generalize across unseen forgery techniques and complex scenarios.
In response, this paper introduces VidForensics-M1, a groundbreaking framework that combines meta-detection with reinforcement learning, leveraging verifiable temporal grounding as a core supervisory signal. The key idea is to automatically generate paired real and fake videos with precise, ground-truth manipulated intervals, using boundary-frame-conditioned generation models. This process ensures that the model learns from objectively verifiable evidence, significantly reducing the risk of false positives caused by semantic biases or hallucinations common in textual explanations.
The core innovation lies in the Evidence-Guided Reward Redistribution (EGRR) mechanism, which dynamically adjusts rewards during training based on the quality of the temporal evidence. This approach encourages the detector to focus on fine-grained, verifiable forgery artifacts, leading to more accurate localization and robust detection performance. Extensive experiments on datasets like ViF-Bench demonstrate that VidForensics-M1 surpasses existing methods, achieving an F1 score of 75.39%, a substantial improvement over prior state-of-the-art techniques.
Beyond technical excellence, this work addresses fundamental issues in digital media forensicsβnamely, the need for explainability, verifiability, and generalization. By integrating objective temporal evidence into the training process, the framework offers a scalable and interpretable solution adaptable to evolving forgery techniques. Its potential applications span media verification, legal evidence collection, and social media moderation, providing a vital tool in the fight against misinformation.
While promising, the approach has limitations, including dependence on the quality of generative models and the complexity of hyperparameter tuning. Future directions involve multi-modal evidence fusion, real-time deployment, and expanding to more complex forgery scenarios. Overall, VidForensics-M1 marks a significant advance in AI-generated video detection, setting a new benchmark for trustworthy digital media analysis.
Deep Dive
Abstract
Recent advances in video generation models have significantly improved the realism of synthetic videos, blurring the boundary between generated and authentic content and raising concerns about misinformation. Existing MLLM-based detectors mainly rely on supervised fine-tuning or label-level reinforcement learning, where coarse supervision limits generalization to unseen scenarios and emerging video generators. To overcome these limitations, we are the first to introduce \textbf{meta-detection} into AI-generated video detection, enabling reliable forgery detection by jointly optimizing predicted labels and supporting evidence within reinforcement learning. This paradigm requires reliable evidence signals and effective mechanisms to integrate them into label-level optimization. Textual rationales provide semantic descriptions of forgery artifacts, but their generation and verification depend on external models, making supervision vulnerable to hallucinations and semantic biases. In contrast, temporal grounding provides more objective and verifiable evidence, as manipulated intervals can be precisely controlled during forgery construction. Based on this insight, we propose an automated data construction pipeline that generates paired real-fake videos by replacing temporal segments with boundary-frame-conditioned video generation models. Furthermore, we introduce \textbf{Evidence-Guided Reward Redistribution}, which performs evidence-aware credit assignment by redistributing rewards among label-correct responses according to evidence quality. This preserves reliable label supervision while encouraging detectors to acquire fine-grained and verifiable forgery localization capabilities. Extensive experiments demonstrate that \textbf{VidForensics-M1} effectively leverages verifiable temporal evidence to achieve robust and generalizable AI-generated video detection.
References (20)
DAVID-XR1: Detecting AI-Generated Videos with Explainable Reasoning
Yifeng Gao, Yifan Ding, Hongyu Su et al.
Skyra: AI-Generated Video Detection via Grounded Artifact Reasoning
Yifei Li, Wenzhao Zheng, Yanran Zhang et al.
A Sanity Check for AI-generated Image Detection
Shilin Yan, Ouxiang Li, Jiayin Cai et al.
Zero-shot 3D-Aware Trajectory-Guided image-to-video generation via Test-Time Training
Ruicheng Zhang, Jun Zhou, Zunnan Xu et al.
Generative Universal Verifier as Multimodal Meta-Reasoner
Xinchen Zhang, Xiaoying Zhang, Youbin Wu et al.
DeMamba: AI-Generated Video Detection on Million-Scale GenVideo Benchmark
Haoxing Chen, Yan Hong, Zizheng Huang et al.
Seeing What Matters: Generalizable AI-generated Video Detection with Forensic-Oriented Augmentation
Riccardo Corvi, D. Cozzolino, Ekta Prashnani et al.
Grounded-VideoLLM: Sharpening Fine-grained Temporal Grounding in Video Large Language Models
Haibo Wang, Zhiyang Xu, Yu Cheng et al.
LAA-Net: Localized Artifact Attention Network for Quality-Agnostic and Generalizable Deepfake Detection
Dat Nguyen, Nesryne Mejri, I. Singh et al.
Exploring Unbiased Deepfake Detection via Token-Level Shuffling and Mixing
Xinghe Fu, Zhiyuan Yan, Taiping Yao et al.
LTX-Video: Realtime Video Latent Diffusion
Yoav HaCohen, Nisan Chiprut, Benny Brazowski et al.
VidGuard-R1: AI-Generated Video Detection and Explanation via Reasoning MLLMs and RL
Kyoungjun Park, Yifan Yang, Juheon Yi et al.
CubeComposer: Spatio-Temporal Autoregressive 4K 360Β° Video Generation from Perspective Video
Lingen Li, Guangzhi Wang, Xiaoyu Li et al.
BusterX: MLLM-Powered AI-Generated Video Forgery Detection and Explanation
Haiquan Wen, Yiwei He, Zhenglin Huang et al.
DAPO: An Open-Source LLM Reinforcement Learning System at Scale
Qiying Yu, Zheng Zhang, Ruofei Zhu et al.
Concrete Problems in AI Safety
Dario Amodei, Chris Olah, J. Steinhardt et al.
BusterX++: Towards Unified Cross-Modal AI-Generated Content Detection and Explanation with MLLM
Haiquan Wen, Tianxiao Li, Zhenglin Huang et al.
Learning Human-Perceived Fakeness in AI-Generated Videos via Multimodal LLMs
Xingyu Fu, Siyi Liu, Yinuo Xu et al.
Transcending Forgery Specificity with Latent Space Augmentation for Generalizable Deepfake Detection
Zhiyuan Yan, Yuhao Luo, Siwei Lyu et al.
OmniVerifier-M1: Multimodal Meta-Verifier with Explicit Structured Recalibration
Xinchen Zhang, Bo Liu, Jiale Liu et al.