MPF-Net: Exposing High-Fidelity AI-Generated Video Forgeries via Hierarchical Manifold Deviation and Micro-Temporal Fluctuations
MPF-Net detects high-fidelity AI video forgeries via hierarchical manifold deviation and micro-temporal fluctuation analysis, achieving 99.97% accuracy on VidProM.
Key Findings
Methodology
MPF-Net employs a dual-path hierarchical framework: the Static Manifold Deviation branch leverages large-scale vision foundation models (VFMs) to detect spatial anomalies and physical violations, while the Micro-Temporal Fluctuation branch analyzes residuals between consecutive frames to uncover subtle, structured patterns indicative of forgery. The core mechanism involves projecting frame residuals onto the learned data manifold, utilizing Jacobian matrices and physical noise features to reveal Manifold Projection Fluctuations (MPF). Fine-tuning with LoRA enhances sensitivity to micro-temporal signals, enabling detection even in high-fidelity, on-manifold videos. The sequential filtering process first discards obvious forgeries, then identifies subtle manipulations.
Key Results
- On VidProM, MPF-Net achieved 99.97% accuracy, outperforming traditional spatial detection methods like Gram-Net (63.64%) by a large margin. The model maintained high robustness across various video qualities and models (Veo, Wan). Ablation studies showed that micro-temporal residual analysis contributed approximately 30% to overall performance gains, validating the importance of residual structure analysis.
- Feature space visualization demonstrated that high-fidelity synthetic videos cluster tightly with real videos in MPF-based features, while real videos exhibit dispersed, chaotic patterns. The method effectively separates on-manifold from off-manifold samples, even at high resolutions.
- The framework's robustness was confirmed across multiple datasets, with consistent detection rates and low false positives, indicating strong generalization capabilities.
Significance
This work advances digital forensics by moving beyond macro-level semantic and physical violations, focusing on intrinsic micro-structural fingerprints embedded during generation. By analyzing residuals' structured homogeneity and physical noise, MPF-Net provides a robust, scalable solution for high-fidelity fake detection, addressing a critical challenge as generative models approach real-world quality. Its ability to detect subtle, deterministic artifacts enhances trustworthiness in media verification, with broad implications for security, journalism, and social media moderation.
Technical Contribution
The paper introduces a hierarchical dual-path detection framework combining VFMs for spatial anomaly detection with residual-based MPF analysis for micro-temporal artifacts. It innovatively models residuals via Jacobians and physical noise features, establishing a new paradigm for high-fidelity forgery detection. The use of LoRA fine-tuning allows efficient adaptation to subtle residual patterns without retraining entire models. The integration of manifold projection analysis and structured residual homogeneity represents a significant step forward in forensic AI research.
Novelty
This is the first comprehensive framework that combines large-scale vision foundation models with micro-temporal residual analysis to detect high-fidelity AI-generated videos. The concept of Manifold Projection Fluctuations as a fingerprint of generative artifacts is novel, providing a new theoretical basis for forensic detection beyond traditional spatial or semantic cues. The layered approach effectively addresses the limitations of prior methods that fail in high-quality content.
Limitations
- The detection performance diminishes on videos with extremely low frame rates or heavy compression, where residual signals are obscured by noise.
- In scenes with rapid motion or complex interactions, the structured residuals may become less distinguishable, requiring further model robustness improvements.
- Computational cost for processing long sequences or real-time detection remains high, necessitating optimization for practical deployment.
Future Work
Future research will focus on integrating multi-modal cues, such as audio and text, to enhance detection robustness. Efforts will also aim to optimize model architectures for real-time applications and extend the framework to other forgery types, including audio-visual manipulations. Exploring unsupervised or semi-supervised learning strategies to reduce reliance on labeled datasets and improve generalization across unseen generative models will be key directions.
AI Executive Summary
As generative models like Veo and Wan rapidly improve, synthetic videos now reach a level where macro-level semantic errors and temporal inconsistencies are hardly detectable. Traditional forensic methods, which focus on obvious spatial or physical violations, struggle against these high-fidelity forgeries. To address this challenge, this study introduces MPF-Net, a hierarchical detection framework that leverages the concept of Manifold Projection Fluctuations (MPF). The core idea is that AI-generated videos, being products of learned manifold mappings, exhibit structured residual patterns even when visually perfect.
The framework consists of two main components. The Static Manifold Deviation branch employs large-scale vision foundation models (VFMs) to identify spatial anomalies and violations of physical laws, effectively acting as a high-level filter to discard obvious forgeries. For videos that pass this filter, the Micro-Temporal Fluctuation branch analyzes residuals between consecutive frames, focusing on the structured homogeneity introduced by the generative process. By modeling residuals through Jacobians and physical noise features, the system detects subtle artifacts that are invisible to traditional detectors.
Experimental results demonstrate that MPF-Net achieves near-perfect detection accuracy of 99.97% on the VidProM benchmark, significantly outperforming existing methods. Its robustness across different datasets and high-quality videos underscores its potential as a reliable forensic tool. This approach not only enhances the detection of high-fidelity AI videos but also opens new avenues for research into micro-structural fingerprints of synthetic media. Despite some limitations in low-quality scenarios, the framework sets a new standard for future forensic detection systems, promising broader application in security, media integrity, and digital trustworthiness.
Deep Dive
Abstract
With the rapid advancement of video generation models such as Veo and Wan, the visual quality of synthetic content has reached a level where macro-level semantic errors and temporal inconsistencies are no longer prominent. However, this does not imply that the distinction between real and cutting-edge high-fidelity fake is untraceable. We argue that AI-generated videos are essentially products of a manifold-fitting process rather than a physical recording. Consequently, the pixel composition logic of consecutive adjacent frames residual in AI videos exhibits a structured and homogenous characteristic. We term this phenomenon `Manifold Projection Fluctuations' (MPF). Driven by this insight, we propose a hierarchical dual-path framework that operates as a sequential filtering process. The first, the Static Manifold Deviation Branch, leverages the refined perceptual boundaries of Large-Scale Vision Foundation Models (VFMs) to capture residual spatial anomalies or physical violations that deviate from the natural real-world manifold (off-manifold). For the remaining high-fidelity videos that successfully reside on-manifold and evade spatial detection, we introduce the Micro-Temporal Fluctuation Branch as a secondary, fine-grained filter. By analyzing the structured MPF that persists even in visually perfect sequences, our framework ensures that forgeries are exposed regardless of whether they manifest as global real-world manifold deviations or subtle computational fingerprints.