V-FIND: Revealing the Intrinsic Forgery Knowledge Encoded in Video Forgery Detectors
V-FIND localizes sparse neurons to improve video forgery detection, achieving state-of-the-art results with frozen backbones and linear classifiers.
Key Findings
Methodology
V-FIND employs layer discrepancy analysis to identify critical layers exhibiting real-fake differences, then trains probes within these layers to select latent anchor neurons (LANs) that consistently carry forgery signals. These neurons form a compact subspace, enabling high detection accuracy with frozen backbones and lightweight linear classifiers. The process involves: • Computing layer-wise discrepancy signals (directional separation and centroid shift); • Training probes to evaluate neuron discriminative power; • Selecting LANs based on effect size thresholds; • Building a subspace from LAN activations for classification. This approach efficiently extracts and reuses intrinsic forgery knowledge without full model retraining.
Key Results
- On Magic Videos and MovieGen datasets, V-FIND achieves 89.37% mACC and 96.92% mAP, surpassing many SOTA methods. On DVF, AUC reaches 98.2%, demonstrating strong generalization. Ablation studies show late layers (24-32) carry more forgery-discriminative info, with threshold τd=1.5 yielding optimal LAN selection. Visualization confirms LAN responses focus on complex textures and boundaries, stable across frames.
- Neural intervention experiments validate LAN’s functional specificity, with activation maps highlighting suspicious regions in forged videos. Layer analysis indicates deeper layers encode more integrated forgery cues. Threshold sensitivity analysis confirms robustness of the LAN selection process, ensuring reliable detection performance.
- The approach significantly reduces computational costs by avoiding full retraining, while maintaining high accuracy and interpretability. The sparse subspace organization enables easy adaptation to new forgery types and enhances model transparency, addressing key limitations of black-box detectors.
Significance
This work uncovers the intrinsic sparse forgery-discriminative knowledge within video detectors, providing a new paradigm for model interpretability and efficiency. By focusing on a small set of neurons, it offers a resource-efficient, transparent, and generalizable detection mechanism. This advances both theoretical understanding and practical deployment, especially in scenarios demanding quick adaptation and low resource consumption. The insights into neuron-level organization open avenues for future research in model explainability and robustness against evolving forgery techniques.
Technical Contribution
The paper introduces a novel framework combining layer discrepancy analysis and neuron screening to locate and organize sparse forgery signals within pre-trained models. It departs from traditional end-to-end training, emphasizing internal interpretability and reusability of discriminative features. The methodology enables high detection accuracy with minimal retraining, providing a new toolset for understanding deep models' internal decision-making processes.
Novelty
This is the first systematic approach to locate and utilize sparse, functionally specialized neurons within pre-trained video forgery detectors. Unlike conventional methods that fine-tune entire models, V-FIND isolates a compact subspace of neurons responsible for forgery discrimination, offering both interpretability and efficiency. Its layer-wise discrepancy analysis and neuron effect size screening constitute innovative contributions to model interpretability in forgery detection.
Limitations
- The method depends on the architecture of the pre-trained backbone; transferring to different models may require parameter tuning. Its effectiveness in extremely low-quality or highly manipulated videos remains to be tested.
- Threshold selection for LANs, though robust, may need adjustment for different datasets or forgery types, potentially affecting generalization.
- Current focus on static forgery cues may limit performance against sophisticated, dynamic manipulations; integrating multi-modal signals could be necessary.
Future Work
Future directions include extending the framework to multi-modal forgery detection, incorporating audio and text cues, and developing adaptive thresholding techniques. Additionally, exploring dynamic neuron selection mechanisms could improve robustness against evolving forgery methods. Further research may also investigate the theoretical underpinnings of neuron sparsity and their relation to model generalization and interpretability.
AI Executive Summary
The rapid advancement of AI-generated videos has posed significant challenges to media authenticity verification. Traditional detection methods, primarily based on end-to-end deep learning models, often operate as black boxes, making it difficult to interpret their decision mechanisms and adapt efficiently to new forgery techniques. Recognizing this gap, the present study introduces V-FIND, a novel framework that leverages the internal organization of pre-trained video forgery detectors to locate sparse, highly discriminative neurons. By analyzing layer-wise discrepancies between real and forged videos, V-FIND identifies critical layers where forgery cues are most prominent. Within these layers, it trains probes to evaluate individual neuron contributions, selecting a sparse set of latent anchor neurons (LANs) that consistently encode forgery signals. These LANs are organized into a compact subspace, which, when combined with a simple linear classifier, achieves high detection accuracy across multiple benchmarks, including Magic Videos, MovieGen, and DVF. Notably, this approach does not require full model retraining, significantly reducing computational costs while enhancing interpretability. Visualization of LAN responses reveals that these neurons focus on complex textures and boundaries, aligning with human intuition about forgery cues. Layer analysis confirms that deeper layers carry more discriminative information, and threshold sensitivity tests demonstrate the robustness of LAN selection. The findings suggest that models internally contain sparse, reusable forgery knowledge, opening new avenues for explainable and resource-efficient detection. Future work aims to extend this paradigm to multi-modal scenarios and dynamic neuron selection, further strengthening the defense against sophisticated AI-generated content.
Deep Analysis
Background
随着AI生成内容（AIGC）技术的快速发展，尤其在视频合成方面，合成视频的逼真度不断提升，带来了前所未有的挑战。早期的检测方法主要依赖深度神经网络（如TimeSformer、CLIP）进行二分类，取得一定成功，但多为黑箱操作，缺乏对模型内部判别机制的理解。近年来，研究逐步关注模型内部的判别特征，试图解释神经网络的判别依据，但尚未系统性地定位到稀疏、判别性强的神经元。本研究旨在揭示模型内部潜藏的稀疏判别知识，提升检测效率和可解释性，为应对不断演变的伪造技术提供理论基础。
Core Problem
现有视频伪造检测方法多依赖全模型微调或端到端训练，资源消耗大，且难以理解模型内部的判别依据。随着伪造技术不断升级，模型的泛化能力成为瓶颈。如何在保持检测性能的同时，理解模型内部的判别机制，成为亟待解决的问题。此外，模型内部的判别知识呈稀疏分布，尚未被充分挖掘，限制了检测的效率和可解释性。
Innovation
本研究提出V-FIND框架，核心创新在于：1）通过层间差异分析定位关键层，筛选出表现出明显伪造差异的中后层；2）在关键层中训练探针，筛选出持续携带判别信号的稀疏锚定神经元（LAN）；3）组织LAN形成紧凑的伪造子空间，只需冻结原模型，训练线性分类器即可实现高效检测。此方法突破了传统端到端训练的局限，强调模型内部稀疏判别知识的提取与利用，为模型可解释性提供新途径。
Methodology
- �� 计算层间差异信号（方向性分离和归一化偏移）筛选关键层；• 在关键层中训练线性探针，评估每个神经元的判别贡献；• 根据判别效果大小筛选出LAN集合；• 构建LAN激活的子空间，训练线性分类器实现检测。整个流程在保持模型参数不变的情况下，有效提取稀疏判别信息，提升检测效率。
Experiments
采用Magic Videos、MovieGen和DVF三个公开数据集，分别评估检测性能。模型基于预训练的Qwen2.5-ViT，筛选阈值τd=1.5，得到211个LAN。对比多种SOTA方法，V-FIND在准确率、AP和AUC指标上均优于对手。通过神经元响应可视化验证LAN在复杂纹理和边界区域的激活效果，层级分析确认后期层更携带判别信息。敏感性分析确保筛选阈值的鲁棒性。
Results
V-FIND在Magic Videos上实现89.37%的平均准确率（mACC）和96.92%的平均精确率（mAP），在DVF测试集AUC达98.2%，优于多数端到端模型。神经元干预实验显示，激活响应在伪造视频中更为集中，验证了LAN的判别功能。层级分析表明，后期层（第24-32层）更富判别信息，阈值τd=1.5在神经元筛选中表现最佳。这些结果表明，模型内部稀疏判别知识可被有效提取和利用。
Applications
该方法可广泛应用于视频内容验证、数字取证和媒体真实性检测，特别适合在资源有限环境中快速部署高效检测模型。只需冻结预训练模型，训练少量线性分类器，即可实现高性能检测，降低成本，增强模型可解释性。未来还可结合多模态信息，拓展到音频、文本等多源伪造检测场景。
Limitations & Outlook
当前方法依赖预训练模型的结构特性，迁移到不同架构或新型伪造技术时，效果可能下降。神经元筛选阈值虽有一定鲁棒性，但在极端伪造或低质量视频中表现仍需验证。模型在复杂场景下的泛化能力有限，未来需结合多模态信息和动态筛选机制进行优化。
Plain Language Accessible to non-experts
想象你在一家工厂工作，工厂里有许多机器在生产不同的产品。每台机器都有很多部件，有的部件负责特定任务，比如装配、检测或包装。现在，工厂的管理者想知道哪些部件最关键，能判断出产品是否有瑕疵。你们通过观察不同产品的生产过程，发现只有少数几个关键部件在检测瑕疵时表现出明显差异。这些关键部件就像稀疏的神经元，只在特定情况下发挥作用。工厂可以只关注这些关键部件，不必重新调整所有机器，就能快速判断出瑕疵产品。这就像V-FIND在视频检测中找到少数神经元，组织成一个小的“判别空间”，用很少的调整就能实现高效检测。这个方法既节省资源，又让工厂的检测变得更透明、更容易理解。
ELI14 Explained like you're 14
想象你在学校里，有很多老师在教不同的科目。有的老师擅长数学，有的老师擅长语文。现在，学校想知道哪些老师最擅长发现学生作弊。其实，只有少数几个老师在观察学生时特别敏感，能一眼看出作弊的迹象。学校只需要找到这些“特别老师”，让他们多关注学生，就能更快发现作弊行为。这就像研究中的神经元筛选，只关注那些在判断视频真假时特别敏感的神经元。这样，检测系统变得更聪明、更快，也更容易理解它是怎么做出判断的。就像找出学校里最厉害的“作弊侦探”，用少量信息就能做出准确判断。这个方法让检测变得简单又高效，就像用少数老师的智慧解决大问题一样。
Abstract
As generated videos become increasingly realistic, reliable video forgery detection is increasingly important. Existing studies typically optimize and use video forgery detectors as black boxes, while the latent forgery-discriminative knowledge inside them remains largely unexplored. Instead of continuing to rely on resource-intensive full-model retraining to steadily improve detection performance, we ask whether video forgery detection can also be achieved by uncovering and activating sparse forensic knowledge within the detector. We find that forgery-discriminative knowledge is not uniformly distributed across the full representation space, but is concentrated in a sparse set of functionally specialized neurons. Based on this insight, we propose a video forgery-intrinsic neuron discovery (V-FIND) framework. V-FIND first localizes critical layers that exhibit pronounced discrepancies between real and forged videos, and then identifies latent anchor neurons that consistently carry forgery-discriminative signals, organizing them into a compact forensic subspace. With the original backbone frozen and only a lightweight linear classifier trained, this subspace still delivers strong detection performance across multiple external benchmarks for generated videos. Further neuron intervention experiments provide direct evidence for the functional specificity of the discovered neurons. Overall, these results suggest that video forgery detectors contain sparse, extractable, and reusable forgery-discriminative knowledge, offering a new perspective on understanding and exploiting their intrinsic forensic capability.