Advancing Reliable Synthetic Video Detection: Insights from the SAFE Challenge

Recent breakthroughs in generative AI, including diffusion models, transformers, and adversarial networks, have enabled the creation of highly realistic synthetic videos. Early efforts like FaceForensics++ and DFDC focused on facial manipulation detection, primarily in controlled environments. As models like Imagen Video, Make-A-Video, and Text-to-Video (TI2V) systems matured, synthetic videos diversified across content types and complexity, making detection more challenging. Existing datasets often lack diversity in sources and post-processing conditions, limiting their real-world relevance. The need for robust, generalizable detection methods that can handle unseen generators and post-processing transformations has become urgent, especially given the proliferation of deepfakes on social media and misinformation campaigns.

The core challenge is designing detection algorithms that generalize well across unknown synthetic video sources and withstand common post-processing operations such as compression, resizing, and filtering. Current methods tend to overfit to specific generators or artifacts, leading to poor performance on unseen data. Additionally, real-world videos are often subjected to various transformations that can mask forgery traces, further complicating detection. This creates a pressing need for evaluation frameworks that simulate realistic scenarios, including blind testing on hidden datasets, to accurately measure the robustness and generalization of detection models.

This work introduces a novel detection framework combining large-scale pre-trained vision backbones (e.g., ViT, Swin Transformer) with autoencoding-based feature augmentation. Key innovations include:

• �� Multi-source training: leveraging diverse synthetic generators and real videos to improve generalization.
• �� Autoencoding augmentation: encoding features into a latent space and reconstructing to enhance robustness against post-processing.
• �� Fully blind evaluation: assessing models on hidden datasets to prevent overfitting.
• �� Multi-task learning: jointly optimizing for detection accuracy and robustness, ensuring consistent performance across scenarios.

These strategies collectively address the limitations of prior methods that relied heavily on specific artifacts or metadata, enabling more resilient detection in practical settings.

• �� Data collection: Curated datasets with 13 synthetic generators and 21 real sources, totaling 6,000 videos.
• �� Feature extraction: Employed pre-trained vision models (e.g., ViT, Swin Transformer) to extract deep features.
• �� Autoencoding: Encoded features into a latent space, then reconstructed to learn invariant representations.
• �� Model training: Used multi-task loss combining detection accuracy and robustness objectives, with adversarial training to simulate post-processing effects.
• �� Evaluation: Conducted in a fully blind setting on hidden datasets, with metrics including AUC and BAC.
• �� Post-processing augmentation: Included simulated real-world transformations during training to improve robustness.

The dataset comprised videos from 21 real sources and 13 synthetic models, split into public and private sets. Models trained on the public set were evaluated on the private set, simulating real-world unseen scenarios. The primary metrics were AUC and BAC, with additional analysis on robustness under post-processing like compression, scaling, and blurring. Ablation studies compared different backbone architectures and augmentation strategies, confirming the effectiveness of autoencoding and multi-source training. Cross-source transferability was validated by testing on unseen generators, demonstrating strong generalization. The experiments also assessed robustness degradation under severe transformations, guiding future improvements.

Detection models achieved an average AUC of 0.82 on the private dataset, with top teams using pre-trained vision backbones and autoencoding strategies. Under post-processing, AUC decreased to 0.72, indicating robustness challenges. Models trained with multi-source data showed better transferability to unseen generators, with some teams maintaining AUC above 0.75. The results confirm that combining large-scale pretraining with invariant feature learning significantly improves detection performance, though robustness to severe transformations remains an open issue. These findings suggest a promising direction for deploying reliable synthetic video detectors in real-world scenarios.

The developed detection framework can be integrated into social media platforms, news verification tools, and digital forensics workflows to identify deepfakes and manipulated videos. It requires minimal fine-tuning for different content domains and can operate on videos with various post-processing effects, making it suitable for real-time or batch analysis. Long-term, such systems could underpin automated content moderation, misinformation mitigation, and legal evidence verification, contributing to a safer digital environment.

Despite strong performance, detection accuracy declines under aggressive post-processing like heavy compression or blurring, indicating robustness gaps. The models are computationally intensive, limiting deployment on resource-constrained devices. The rapid evolution of generative techniques demands continuous dataset updates and retraining, posing scalability challenges. Future work should focus on lightweight architectures, multimodal cues, and adaptive learning to address these issues.