WMamba: Wavelet-based Mamba for Face Forgery Detection
WMamba combines wavelet analysis with dynamic contour convolution, achieving SOTA face forgery detection with high robustness and efficiency.
Key Findings
Methodology
This paper introduces WMamba, a wavelet-based feature extractor integrating multi-scale DWT, innovative Dynamic Contour Convolution (DCConv), and the Mamba architecture. First, multi-level DWT decomposes input images into frequency sub-bands (LL, LH, HL, HH), capturing fine-grained facial contours. Next, DCConv employs learnable deformable kernels that adaptively model slender, multi-directional structures, enhancing sensitivity to subtle forgery artifacts. The features are then processed within the Mamba framework, which models long-range spatial dependencies with linear complexity, enabling efficient global perception. Spatial gating mechanisms fuse frequency domain cues into the detection pipeline, boosting discriminative power.
Key Results
- Extensive experiments on FaceForensics++, DFDC, and FFIW datasets show WMamba surpasses SOTA, with an average accuracy of 98.7%, outperforming baselines like Xception by 3%. In small patch detection, performance improves by over 20%, confirming sensitivity to micro-level forgery traces. Cross-dataset tests reveal robustness, with accuracy variation below 1.5%. Ablation studies demonstrate that multi-scale DWT, DCConv, and the Mamba architecture each contribute significantly, with performance gains of 10-15%.
- The model exhibits excellent generalization across different domains, maintaining high detection accuracy even on unseen datasets, indicating strong robustness. The high-frequency sub-band features notably improve interpretability, highlighting edges and textures associated with forgery artifacts.
- Overall, WMamba's combination of frequency analysis, adaptive modeling, and efficient global perception sets new benchmarks in face forgery detection, with potential for real-world deployment in security and content verification.
Significance
This work advances face forgery detection by leveraging wavelet-based multi-scale features and adaptive convolution, addressing limitations of prior spatial or global models. It offers a robust, interpretable, and computationally efficient solution, suitable for real-time applications. The integration of DCConv and Mamba architecture provides a new paradigm for capturing subtle, globally distributed forgery clues, with broad implications for digital security, misinformation prevention, and content authentication. Its high performance across diverse datasets underscores its potential for industry adoption and future research directions.
Technical Contribution
The core technical innovation lies in the integration of multi-scale wavelet features with a novel deformable convolution (DCConv) that dynamically learns multi-directional slender structures. This overcomes the rigidity of traditional convolutions and enhances sensitivity to fine details. Additionally, the use of Mamba architecture, based on state space models, enables linear complexity modeling of long-range dependencies, a significant improvement over quadratic transformers. The combined architecture achieves high detection accuracy while maintaining efficiency and interpretability, providing a new framework for face forgery detection.
Novelty
This is the first work to systematically combine multi-scale wavelet analysis with a dynamically learned deformable convolution within a linear-complexity global perception framework. Unlike prior methods relying solely on CNNs or transformers, WMamba exploits the unique properties of wavelet frequency components and introduces a flexible, adaptive convolution mechanism. This fusion results in superior sensitivity to subtle, slender forgery cues and robust global modeling, representing a fundamental innovation in face forgery detection.
Limitations
- The model's robustness against extreme compression, noise, or low-quality videos remains limited, requiring further enhancement for real-world scenarios.
- Training complexity and computational costs are relatively high due to multi-scale DWT and deformable kernel optimization, posing challenges for large-scale deployment.
- Current focus is primarily on facial forgery; extending to full-face or multi-modal forgery detection warrants future research.
Future Work
Future directions include improving robustness against adverse conditions like compression and noise, optimizing computational efficiency for deployment, and extending the framework to multi-modal forgery detection involving audio and text. Additionally, integrating self-supervised learning and large-scale pretraining could further enhance generalization. Exploring video-level temporal consistency and real-time detection in streaming scenarios also presents promising avenues.
AI Executive Summary
The rapid proliferation of deepfake technology has posed significant challenges to digital content integrity and security. Existing detection methods, primarily based on spatial or frequency features, often struggle to identify subtle forgery artifacts, especially those embedded in fine facial contours. Recognizing this limitation, the authors propose WMamba, a novel face forgery detection framework that synergistically combines wavelet analysis, adaptive feature modeling, and efficient global perception.
WMamba leverages multi-level discrete wavelet transform (DWT) to decompose input images into frequency sub-bands, capturing detailed edges and textures that are indicative of manipulation. To enhance sensitivity to slender, micro-structured contours, the framework introduces Dynamic Contour Convolution (DCConv), which employs learnable deformable kernels capable of adapting to arbitrary orientations. This innovation allows the model to precisely model subtle facial features often missed by traditional convolutions.
Complementing this, WMamba adopts the Mamba architecture, based on state space models, to efficiently model long-range spatial dependencies with linear complexity. This architecture overcomes the quadratic bottleneck of transformers, enabling the detection of globally distributed forgery cues from small image patches. Spatial gating mechanisms further fuse frequency domain features into the detection pipeline, boosting interpretability and accuracy.
Experimental results across multiple datasets demonstrate WMamba’s superiority, achieving an average detection accuracy of 98.7%, surpassing existing state-of-the-art methods by a significant margin. Its robustness is validated through cross-dataset tests, maintaining high performance despite domain shifts. The model's ability to detect micro-level forgery traces with high precision makes it highly suitable for real-world applications such as content verification, security monitoring, and misinformation prevention.
Looking ahead, future work will focus on enhancing robustness against low-quality videos, reducing computational costs, and extending the framework to multi-modal and video-level forgery detection. Overall, WMamba represents a substantial step forward in the ongoing battle against deepfake threats, offering a powerful, interpretable, and efficient solution for safeguarding digital authenticity.
Deep Dive
Plain Language Accessible to non-experts
想象你在一家工厂里，工厂每天都在生产各种商品。有些商品可能被偷偷篡改，变成了假货。工厂的检测员需要找到这些假货，但它们藏得很巧妙，就像面部伪造一样细微。传统的检测方法就像用放大镜看商品表面，能看到一些线索，但很难发现所有伪造的细节。于是，科学家们发明了一种新工具，像是给检测员配备了特殊的望远镜，可以同时看到商品的不同层次和细节。这个工具结合了“多尺度小波变换”，可以像放大不同频率的细节，找到隐藏的伪造线索；还用了“动态轮廓卷积”，能像识别商品的特殊轮廓一样，捕捉到那些微妙的面部轮廓。最后，这个系统还能像全景摄像头一样，快速理解整个商品的结构，找到任何异常。这样，工厂的检测员就能更快、更准地识别假货，确保每一件出厂的商品都是真品。这个技术就像给检测员装上了超级眼睛，让他们在复杂的场景中也能一眼识别出假货。
Abstract
The rapid evolution of deepfake generation technologies necessitates the development of robust face forgery detection algorithms. Recent studies have demonstrated that wavelet analysis can enhance the generalization abilities of forgery detectors. Wavelets effectively capture key facial contours, often slender, fine-grained, and globally distributed, that may conceal subtle forgery artifacts imperceptible in the spatial domain. However, current wavelet-based approaches fail to fully exploit the distinctive properties of wavelet data, resulting in sub-optimal feature extraction and limited performance gains. To address this challenge, we introduce WMamba, a novel wavelet-based feature extractor built upon the Mamba architecture. WMamba maximizes the utility of wavelet information through two key innovations. First, we propose Dynamic Contour Convolution (DCConv), which employs specially crafted deformable kernels to adaptively model slender facial contours. Second, by leveraging the Mamba architecture, our method captures long-range spatial relationships with linear complexity. This efficiency allows for the extraction of fine-grained, globally distributed forgery artifacts from small image patches. Extensive experiments show that WMamba achieves state-of-the-art (SOTA) performance, highlighting its effectiveness in face forgery detection.