GptGet

Deep Residual Injection for Full-Spectrum Forensic Signal Perception in Multimodal Large Language Models

TL;DR

Deep-VRM achieves full-spectrum forensic signal perception via residual injection, reaching 97.42% accuracy on GenImage dataset.

cs.CV 🔴 Advanced 2026-06-15 2 views
Kaiqing Lin Zhiyuan Yan Ruoxin Chen Ke-Yue Zhang Yue Zhou Caiyong Piao Bin Li Taiping Yao Bo Wang Youchang Xiao Shouhong Ding
AI Reader Arxiv Page Download PDF
multimodal language model forensic detection deep learning residual injection

Key Findings

Methodology

The paper introduces a novel architecture called Deep Visual Residual MLLM (Deep-VRM). This approach preserves early semantic processing while injecting artifact-specific visual signals as a residual path into an intermediate layer, where they are fused with semantic token representations and propagated through subsequent trainable layers. This enables later layers to jointly model semantic reasoning and signal-level forensic cues.

Key Results

  • On the GenImage dataset, Deep-VRM achieved an average accuracy of 97.42%, significantly outperforming existing methods.
  • On the SynthBuster dataset, Deep-VRM reached 94.50% accuracy in fake image detection.
  • In wild scenarios, Deep-VRM achieved 89.51% and 90.48% accuracy on WildRF and AIGI-Bench datasets respectively.

Significance

This study addresses the conflict between semantic and low-level features in multimodal large language models for forensic detection through an innovative residual injection strategy. It not only improves detection performance but also provides new insights for future forensic detection methods.

Technical Contribution

Deep-VRM effectively separates low-level generator traces from semantic information through a residual injection strategy, avoiding damage to semantic capabilities. This method relies entirely on MLLMs without external expert detectors, showcasing new engineering possibilities.

Novelty

This is the first application of residual injection in forensic signal perception for multimodal large language models. Compared to previous methods, Deep-VRM avoids semantic interference by injecting low-level features into deep layers.

Limitations

  • The model's performance may degrade when handling extremely compressed images, as compression can eliminate low-level generator traces.
  • The model requires substantial computational resources for training, which may not be suitable for resource-constrained environments.
  • In certain scenarios, the model may fail to detect very subtle generator traces.

Future Work

Future research can explore achieving similar detection performance at lower computational costs and further improving robustness on extremely compressed images.

AI Executive Summary

The rise of AI-generated content has led to a crisis in digital trust. Existing multimodal large language models face a conflict between semantic and low-level features in forensic detection. This paper proposes a new architecture, Deep-VRM, to address this issue.

Deep-VRM preserves early semantic processing while injecting artifact-specific visual signals as a residual path into an intermediate layer, where they are fused with semantic token representations and propagated through subsequent trainable layers. This enables later layers to jointly model semantic reasoning and signal-level forensic cues. Experimental results show that Deep-VRM achieves state-of-the-art performance across multiple datasets.

While Deep-VRM makes breakthroughs in detection performance, it still faces challenges in handling extremely compressed images. Future research can explore achieving similar detection performance at lower computational costs and further improving robustness on extremely compressed images.

Deep Dive

Abstract

Multimodal large language models (MLLMs) have been increasingly adopted in forensics for their robust semantic understanding. As AI-generated images become realistic, semantic-level inconsistencies alone are often insufficient for reliable detection. This motivates a critical question: whether MLLMs can achieve full-spectrum forensic signal perception, i.e., capturing low-level generator artifacts without sacrificing pre-trained semantic knowledge. We further perform a layer-wise analysis of forensic signal perception in MLLMs, showing that semantic information is primarily formed in the early-to-middle layers, whereas direct fine-tuning for artifact learning disrupts these semantic representations. Based on this insight, we propose Deep Visual Residual MLLM (Deep-VRM) to preserve early semantic processing while injecting artifact-specific visual signals as a residual path into an intermediate layer, where they are fused with semantic token representations and propagated through subsequent trainable layers. This enables later layers to jointly model semantic reasoning and signal-level forensic cues, and surprisingly, the model learns to adaptively leverage different levels of forensic signals depending on the input, achieving robust and generalizable detection performance. Extensive experiments show that our method achieves state-of-the-art across most benchmarks. The code and data are available at https://github.com/KQL11/Deep-VRM.

cs.CV cs.AI