LAKAN: Landmark-assisted Adaptive Kolmogorov-Arnold Network for Face Forgery Detection
LAKAN integrates facial landmarks to dynamically modulate Kolmogorov-Arnold networks, boosting deepfake detection accuracy.
Key Findings
Methodology
This work leverages Kolmogorov-Arnold Networks (KAN), replacing fixed activation functions with learnable spline functions to enhance nonlinear modeling. It introduces a landmark-assisted module that encodes facial landmarks into guidance vectors via an MLP, which then dynamically generates the spline parameters—weights and scalers—for the KAN. This process enables instance-specific nonlinear feature modulation. The architecture involves inserting the LAKAN module at multiple stages of a backbone encoder, guiding the network’s focus toward critical facial regions. Extensive experiments demonstrate superior detection performance and generalization across datasets, validating the effectiveness of geometric priors in deepfake detection.
Key Results
- On multiple benchmarks, LAKAN outperforms state-of-the-art methods, achieving average AUC scores of 96.63%, 84.52%, 89.71%, and 87.32% on CDF2, DFDC, DFDCP, and FFIW, respectively. It shows notable improvements in cross-dataset and cross-manipulation scenarios, with gains of 2-3 percentage points over baselines, confirming its robustness.
- In ablation studies, replacing static fusion with gating mechanisms improved performance, and integrating LAKAN with various encoders (EfficientNet-B4, Swin-B, ConvNeXt-B) consistently enhanced detection accuracy. Visualization via Grad-CAM reveals that LAKAN directs attention to facial regions rich in forgery artifacts, supporting its interpretability.
- The method demonstrates strong generalization ability, especially when trained only on real faces, with LAKAN achieving near-perfect detection rates on unseen manipulation types, highlighting its structural prior-guided adaptability.
Significance
This research addresses key limitations of existing deepfake detectors, notably their poor generalization to unseen manipulation techniques. By integrating facial geometric priors with a flexible nonlinear function approximation framework, LAKAN significantly enhances robustness and interpretability. Its plug-and-play design allows broad applicability across different encoders, paving the way for more reliable real-world deepfake detection systems. The approach offers a promising direction for future multi-modal and multi-task forgery detection, contributing to the development of trustworthy AI content verification.
Technical Contribution
The core innovation lies in dynamically generating KAN parameters conditioned on facial landmarks, enabling instance-specific nonlinear feature modulation. This departs from traditional static CNN or Transformer models, providing a theoretically grounded, flexible approximation of complex artifacts. The landmark-guided modulation mechanism enhances focus on discriminative facial regions, improving detection accuracy and robustness. The modular design facilitates integration with various backbone architectures, expanding its practical utility. Theoretically, it bridges geometric priors with deep function approximation, offering new insights into model interpretability and generalization.
Novelty
This is the first work to incorporate facial landmarks as structural priors to dynamically regulate the activation functions of Kolmogorov-Arnold networks for face forgery detection. Unlike conventional static feature extractors, LAKAN adapts its nonlinear transformations based on individual facial geometry, enabling instance-specific focus. Its innovative combination of geometric priors with learnable spline functions marks a significant departure from existing CNN or Transformer-based methods, opening new avenues for robust, interpretable forgery detection.
Limitations
- The method relies heavily on accurate facial landmark detection; errors or occlusions can impair guidance effectiveness, especially in extreme poses or low-quality images.
- Additional computational overhead from dynamic parameter generation increases training and inference costs, potentially limiting real-time deployment.
- Current design primarily targets facial regions; it may be less effective for non-facial forgery artifacts or in scenarios with significant occlusion or distortion.
Future Work
Future directions include integrating multi-modal cues such as audio-visual signals, enhancing robustness against landmark detection errors, and optimizing the computational efficiency of the dynamic modulation process. Exploring self-supervised learning strategies could further improve adaptability in unlabeled environments. Extending the framework to handle non-facial forgeries and real-time applications will also be key to broader deployment.
AI Executive Summary
Deepfake technology has advanced rapidly, enabling the generation of highly realistic manipulated face images and videos. These synthetic media pose serious threats to social trust, security, and information integrity. Existing detection methods, primarily based on CNNs and Transformers, have demonstrated effectiveness but struggle with generalization to unseen forgery techniques and complex artifacts. This challenge stems from their reliance on fixed feature representations, which may not adapt well to diverse manipulations.
In response, this study introduces LAKAN, a landmark-assisted adaptive Kolmogorov-Arnold network designed to address these limitations. By leveraging facial landmarks as structural priors, LAKAN dynamically modulates the nonlinear activation functions within the Kolmogorov-Arnold framework. This approach allows the model to focus on the most informative facial regions, where forgery artifacts are most likely to appear. The core mechanism involves encoding facial geometry into guidance vectors via an MLP, which then generates instance-specific spline parameters to adapt the nonlinear transformations in real time.
Extensive experiments on datasets such as FaceForensics++, CDF2, DFDC, DFDCP, and FFIW demonstrate that LAKAN outperforms existing state-of-the-art methods, achieving superior AUC scores and remarkable cross-dataset generalization. Ablation studies confirm the effectiveness of the landmark-guided dynamic modulation and the plug-and-play compatibility with various backbone encoders. Visualizations further reveal that LAKAN directs attention toward key facial regions rich in forgery artifacts, enhancing interpretability.
Overall, LAKAN signifies a substantial step forward in deepfake detection technology. Its innovative integration of geometric priors with flexible nonlinear modeling offers a robust, interpretable, and adaptable framework capable of confronting evolving forgery techniques. While challenges remain—such as reliance on accurate landmark detection and computational costs—the proposed approach opens promising avenues for future research and practical deployment in security-critical applications.
Deep Dive
Abstract
The rapid development of deepfake generation techniques necessitates robust face forgery detection algorithms. While methods based on Convolutional Neural Networks (CNNs) and Transformers are effective, there is still room for improvement in modeling the highly complex and non-linear nature of forgery artifacts. To address this issue, we propose a novel detection method based on the Kolmogorov-Arnold Network (KAN). By replacing fixed activation functions with learnable splines, our KAN-based approach is better suited to this challenge. Furthermore, to guide the network's focus towards critical facial areas, we introduce a Landmark-assisted Adaptive Kolmogorov-Arnold Network (LAKAN) module. This module uses facial landmarks as a structural prior to dynamically generate the internal parameters of the KAN, creating an instance-specific signal that steers a general-purpose image encoder towards the most informative facial regions with artifacts. This core innovation creates a powerful combination between geometric priors and the network's learning process. Extensive experiments on multiple public datasets show that our proposed method achieves superior performance.