Training-free Detection of Generated Videos via Spatial-Temporal Likelihoods
Proposes STALL, a training-free video detector using spatial-temporal likelihoods, outperforming baselines with 0.85 average AUC.
Key Findings
Methodology
The approach models videos probabilistically by combining spatial likelihoods from pre-trained encoders (like DINOv3) and temporal likelihoods derived from normalized frame-to-frame differences. Spatial likelihoods assume high-dimensional features follow a Gaussian distribution, estimated from a calibration set of real videos, enabling closed-form log-likelihood computation. Temporal likelihoods normalize motion vectors to unit sphere directions, validating their Gaussianity via empirical tests, and compute transition likelihoods using covariance matrices. Fusion of percentile-normalized scores from both dimensions yields a robust, training-free detection metric. The method's core relies on statistical assumptions, requiring only a small set of real videos for calibration, making it highly scalable and adaptable.
Key Results
- On VideoFeedback and GenVideo benchmarks, STALL achieved average AUCs of 0.82 and 0.86, respectively, surpassing existing methods like D3 and ZED. On the newly introduced ComGenVid dataset, it scored an average AUC of 0.85, demonstrating strong generalization across models such as Veo3 and Sora.
- Across diverse generative models, detection accuracy remained above 80%, with robustness to image perturbations and calibration set size variations. Ablation studies confirmed that combining spatial and temporal likelihoods yields the best performance, validating the model design.
- The approach maintained high detection performance even under challenging conditions, with inference times around 0.5 seconds per video, suitable for real-time deployment.
Significance
This work addresses a critical challenge in content verification: detecting high-fidelity generated videos without retraining on new models. By leveraging statistical properties of real videos, it provides a theoretically grounded, scalable, and efficient solution. Its ability to generalize across unseen generators and robustness to perturbations makes it highly valuable for industry applications like social media moderation, news verification, and platform regulation, fostering trust in digital content.
Technical Contribution
The key innovation lies in jointly modeling spatial and temporal features within a probabilistic framework based on high-dimensional Gaussian assumptions. The method introduces a normalization step that empirically validates Gaussianity of motion vectors, enabling closed-form likelihood calculations. It departs from supervised detection paradigms, offering a simple yet powerful zero-shot approach that combines statistical rigor with computational efficiency, opening new avenues for content authenticity verification.
Novelty
This is the first method to unify spatial and temporal likelihoods in a zero-shot detection framework for videos, leveraging high-dimensional Gaussian approximations. Unlike prior approaches focusing solely on motion or content, it integrates both aspects into a single probabilistic model, providing a comprehensive and robust detection mechanism that does not require training data from generated videos.
Limitations
- The reliance on high-dimensional Gaussian assumptions may limit effectiveness in scenarios with complex, non-Gaussian content or extreme motion patterns. Its performance could degrade with highly static videos or severe noise.
- The method depends on the quality of the pre-trained encoder; if the encoder poorly captures generative artifacts, detection accuracy may decline.
- In extremely static videos, the temporal likelihood component may be degenerate, reducing detection robustness, necessitating further model refinement.
Future Work
Future directions include integrating multi-modal cues such as audio and text, developing more sophisticated statistical models beyond Gaussian assumptions, and extending the framework to handle diverse video types and compression artifacts. Enhancing robustness in static or noisy scenarios and deploying in real-world, large-scale systems are also key goals.
AI Executive Summary
The rapid advancement of deep generative models has led to highly realistic synthetic videos, posing significant challenges for content verification and misinformation prevention. Traditional detection methods often rely on supervised training with large labeled datasets, which quickly become obsolete as new models emerge. This creates an urgent need for robust, model-agnostic, zero-shot detection techniques that can adapt to unseen generators.
In response, this study introduces STALL, a novel probabilistic framework that jointly models spatial content and temporal motion likelihoods within a unified high-dimensional Gaussian approximation. The core idea is to leverage pre-trained encoders like DINOv3 to extract features from video frames, estimate Gaussian parameters from a small set of real videos, and compute likelihood scores without any training on generated data. Spatial likelihood assesses each frame independently, while temporal likelihood evaluates the consistency of frame-to-frame transitions after normalization. These scores are then combined using percentile normalization, producing a comprehensive measure of video authenticity.
Extensive experiments on multiple benchmarks, including VideoFeedback, GenVideo, and a new dataset ComGenVid, demonstrate that STALL consistently outperforms existing methods such as D3 and ZED, achieving average AUCs exceeding 0.85. The approach is computationally efficient, with inference times around half a second per video, making it suitable for real-time screening. Its robustness to image perturbations, varying calibration set sizes, and different generative models underscores its practicality.
This work marks a significant step forward in content authentication, providing a theoretically grounded, scalable, and training-free solution. By effectively capturing both content and motion anomalies, STALL offers a promising tool for industry and academia to combat misinformation and ensure the integrity of digital media. Future research will focus on integrating multi-modal cues, refining statistical assumptions, and deploying at scale to meet the evolving landscape of synthetic media generation.
Deep Dive
Plain Language Accessible to non-experts
想象你在一家工厂里，工厂每天生产各种商品。工厂的工人（模型）有自己的一套生产流程（生成视频），但有些商品（视频）是伪造的，看起来很像真的。工厂里有一个检测员（STALL），他不需要学习每个商品的详细制作过程，只需要观察商品的整体特征和生产线的流动（空间和时间的特征）。他会用一些统计方法，判断商品是否符合工厂正常的生产规律。比如，商品的外观是否符合工厂的标准（空间特征），以及生产过程中商品的变化是否自然（时间特征）。如果商品在外观或变化上都符合工厂的正常规律，检测员就会认为它是真的；如果不符合，就会判定为伪造。这个检测员不需要提前学习所有可能的伪造方式，只依赖工厂的正常商品样本，快速判断出伪造品。这就像用统计学的“规则”来识别假货一样，简单高效，适合大规模使用。
ELI14 Explained like you're 14
想象你在学校里，有个老师每天检查学生的作业。平时，老师只看作业的内容（图片），但有时候作业可能被抄袭或伪造。现在，有个聪明的助手（STALL），他不用提前学习所有作弊的方法，只需要用一些简单的统计技巧，观察每份作业的内容和变化是否符合正常的学生习惯。比如，他会看作业的风格是否一致，变化是否自然。老师只要用这个助手检查，能很快发现哪些作业是假的，哪些是真的。这个助手特别厉害，因为它不需要提前学习所有作弊技巧，只用正常学生的作业样本，就能判断出伪造的作业。这就像用数学和统计的方法，快速识别出伪造的内容，既简单又可靠。
Glossary
空间似然性 (Spatial Likelihood)
衡量每一帧图像内容符合真实数据的概率，基于高维高斯模型，反映图像的真实性。
用于评估视频每帧的内容是否符合真实分布。
时间似然性 (Temporal Likelihood)
衡量连续帧之间运动变化的自然程度，利用归一化的帧间差异构建高斯模型。
检测视频中运动是否符合真实动态。
高维高斯近似 (High-dimensional Gaussian Approximation)
在高维空间中，随机向量的投影趋向高斯分布，便于统计建模。
用于空间和时间特征的概率估计。
归一化百分位数 (Percentile Normalization)
将得分转换为相对于校准集的百分位，避免尺度差异影响。
融合空间和时间得分的关键步骤。
零样本检测 (Zero-shot Detection)
无需训练样本，直接利用统计特征判断内容真假。
本方法的核心优势。
Open Questions Unanswered questions from this research
- 1 如何进一步提升模型在极端场景（如静止或噪声干扰）下的鲁棒性仍是未解难题，未来需结合深度学习增强统计模型的表达能力。
- 2 高维高斯假设在复杂内容或极端运动条件下可能失效，探索更复杂的统计或深度模型以提升检测准确性是未来方向。
Abstract
Following major advances in text and image generation, the video domain has surged, producing highly realistic and controllable sequences. Along with this progress, these models also raise serious concerns about misinformation, making reliable detection of synthetic videos increasingly crucial. Image-based detectors are fundamentally limited because they operate per frame and ignore temporal dynamics, while supervised video detectors generalize poorly to unseen generators, a critical drawback given the rapid emergence of new models. These challenges motivate zero-shot approaches, which avoid synthetic data and instead score content against real-data statistics, enabling training-free, model-agnostic detection. We introduce STALL, a simple, training-free, theoretically justified detector that provides likelihood-based scoring for videos, jointly modeling spatial and temporal evidence within a probabilistic framework. We evaluate STALL on two public benchmarks and introduce ComGenVid, a new benchmark with state-of-the-art generative models. STALL consistently outperforms prior image- and video-based baselines. Code and data are available at https://omerbenhayun.github.io/stall-video.