Deepfake Detection that Generalizes Across Benchmarks
GenD fine-tunes only LayerNorm parameters, achieving state-of-the-art cross-dataset AUROC in deepfake detection, surpassing complex models.
Key Findings
Methodology
The proposed GenD approach leverages pre-trained vision encoders (such as CLIP, PE, DINO), fine-tuning solely the Layer Normalization layers (0.03% of total parameters). It employs L2 normalization of the classification token and metric learning to form a hyperspherical feature space, which enhances generalization. Training involves paired real-fake videos, utilizing a combination of cross-entropy, uniformity, and alignment losses to promote compact and separable features. Extensive experiments across 14 benchmarks demonstrate superior cross-dataset AUROC performance compared to more complex models.
Key Results
- On average across 14 datasets, GenD outperforms recent state-of-the-art methods, with AUROC scores reaching 96.0% on CDFv2 and 92.8% on FFIW, demonstrating robust generalization. The approach maintains high performance with minimal parameter updates, confirming the effectiveness of fine-tuning only LayerNorm layers. Ablation studies show that paired training data significantly reduces shortcut learning, leading to better validation AUROC and cross-dataset robustness.
- Different pre-trained encoders (CLIP, PE, DINO) all achieve high AUROC, with CLIP variants reaching 96.0%. The method's simplicity—averaging frame-level predictions—outperforms complex temporal or multimodal methods, highlighting the power of feature space regularization. The experiments also reveal that training on older, diverse datasets yields better generalization than recent, narrow datasets.
- Ablation results confirm that only tuning LayerNorm parameters, combined with feature regularization, is sufficient for high performance. Paired data construction is crucial, as it constrains the model to focus on low-level artifacts rather than superficial cues, thus improving robustness against unseen manipulations.
Significance
This work addresses a core challenge in deepfake detection: achieving reliable generalization across unseen manipulation techniques. By demonstrating that minimal parameter tuning combined with feature regularization can surpass complex architectures, it offers a highly efficient, scalable solution. The approach reduces computational costs, facilitating deployment in real-world scenarios where models must adapt rapidly to evolving fake generation methods. Its emphasis on paired training data provides a new direction for dataset construction, emphasizing low-level artifact learning over superficial cues. Overall, this research significantly advances the field, bridging the gap between academic benchmarks and practical robustness.
Technical Contribution
The key technical innovation is the selective fine-tuning of LayerNorm parameters within a pre-trained vision encoder, combined with L2 normalization and metric learning to form a hyperspherical feature space. This design ensures that the features are compact, well-separated, and robust to domain shifts. The method leverages contrastive and regularization losses to enforce feature uniformity and alignment, leading to improved generalization. Unlike traditional full fine-tuning or architectural modifications, this approach maintains high efficiency, requiring only 0.03% of parameters to be updated, and achieves state-of-the-art results in cross-dataset evaluations.
Novelty
This study is the first to demonstrate that tuning only LayerNorm parameters, combined with feature space regularization, can significantly boost deepfake detection generalization. It departs from prior works that rely on complex architectures or full model fine-tuning, emphasizing parameter efficiency and feature normalization. The integration of paired data training to mitigate shortcut learning further distinguishes this approach, establishing a new paradigm in the field.
Limitations
- The method relies on constructing paired real-fake datasets, which may be challenging in scenarios lacking high-quality paired samples, potentially limiting applicability.
- While tuning only LayerNorm layers is efficient, it might not capture highly complex or novel fake artifacts, especially in extreme cases.
- Performance on severely compressed or low-quality videos remains to be thoroughly validated, indicating a need for further robustness testing.
Future Work
Future research will explore multi-modal extensions incorporating audio and text cues, aiming to improve detection robustness. Automating paired dataset generation could reduce manual effort, broadening real-world applicability. Additionally, investigating adaptive tuning strategies for emerging fake techniques will be crucial for maintaining model relevance.
AI Executive Summary
The rapid evolution of deepfake technology poses a significant challenge for detection systems, especially in terms of generalization to unseen manipulation methods. Traditional approaches often rely on complex architectures or extensive parameter tuning, which increase computational costs and reduce scalability. This paper introduces GenD, a parameter-efficient method that fine-tunes only the Layer Normalization layers of a pre-trained vision encoder, such as CLIP, PE, or DINO. By employing L2 normalization of the classification token and metric learning, GenD constructs a hyperspherical feature space that enforces feature compactness and separability.
Extensive evaluation across 14 benchmark datasets from 2019 to 2025 demonstrates that GenD surpasses recent state-of-the-art models in cross-dataset AUROC performance. Notably, the method achieves 96.0% AUROC on CDFv2 and 92.8% on FFIW, outperforming more complex models that require architectural modifications or full fine-tuning. The key to this success lies in training on paired real-fake videos, which constrains the model to focus on subtle low-level artifacts rather than superficial cues, thus reducing shortcut learning.
The technical innovation of tuning only the LayerNorm parameters, combined with feature regularization, offers a highly efficient and scalable solution. This approach not only reduces computational costs but also enhances robustness against domain shifts and unseen manipulation techniques. The findings suggest that effective deepfake detection can be achieved with minimal parameter updates, emphasizing the importance of dataset construction strategies.
Overall, this work pushes the frontier of deepfake detection by demonstrating that high-level generalization is attainable through targeted, minimal modifications to pre-trained models. Its implications extend to real-world deployment, where models must adapt swiftly to new threats, making it a significant step forward for both academia and industry.
Deep Dive
Abstract
The generalization of deepfake detectors to unseen manipulation techniques remains a challenge for practical deployment. Although many approaches adapt foundation models by introducing significant architectural complexity, this work demonstrates that robust generalization is achievable through a parameter-efficient adaptation of one of the foundational pre-trained vision encoders. The proposed method, GenD, fine-tunes only the Layer Normalization parameters (0.03% of the total) and enhances generalization by enforcing a hyperspherical feature manifold using L2 normalization and metric learning on it. We conducted an extensive evaluation on 14 benchmark datasets spanning from 2019 to 2025. The proposed method achieves state-of-the-art performance, outperforming more complex, recent approaches in average cross-dataset AUROC. Our analysis yields two primary findings for the field: 1) training on paired real-fake data from the same source video is essential for mitigating shortcut learning and improving generalization, and 2) detection difficulty on academic datasets has not strictly increased over time, with models trained on older, diverse datasets showing strong generalization capabilities. This work delivers a computationally efficient and reproducible method, proving that state-of-the-art generalization is attainable by making targeted, minimal changes to a pre-trained foundational image encoder model. The code is at: https://github.com/yermandy/GenD