Detecting Backdoors in Object Detection via Pre-NMS Prediction Distribution Shift
Proposes DistScan, detecting backdoors via pre-NMS class distribution shift, achieving 97.22% accuracy.
Key Findings
Methodology
This work introduces DistScan, a novel backdoor detection framework based on the observation that backdoor injection systematically shifts the model's pre-NMS class prediction distribution away from the training class frequencies. The approach involves constructing a validation set aligned with the model's architecture—full images for single-stage detectors like YOLOv5, cropped object proposals for two-stage detectors like Faster R-CNN. From this set, the model's pre-NMS class predictions are extracted, filtered by confidence threshold δ, and aggregated into a class count vector. This vector is normalized to form the model's class distribution, which is then compared to the training class distribution R using Jensen-Shannon divergence. A divergence exceeding a threshold τ indicates a backdoor. This method requires no access to model weights or trigger information, and is architecture-agnostic, relying solely on intermediate predictions.
Key Results
- Extensive experiments on MS-COCO and PASCAL VOC datasets, using YOLOv5 and Faster R-CNN architectures, across three scene-level attack scenarios (misclassification, disappearance, insertion), demonstrate an average detection accuracy of 96.99%. The method outperforms the best baseline by 27.32 percentage points, with detection accuracy reaching nearly 100% in most scenarios. AUROC scores exceed 0.97, confirming high discriminative power. In complex datasets like COCO, detection performance remains robust, with only slight drops in some attack settings, validating the method's generalization.
- Compared to existing methods like ODScan and MIA, DistScan maintains superior robustness in scene-level attacks, where other methods fail or perform poorly. The ablation studies confirm the importance of validation set construction and threshold tuning, ensuring stability across models and data distributions.
- The results highlight that the class distribution shift is a reliable, architecture-independent signal for backdoor detection, enabling practical deployment in real-world scenarios without requiring internal model access or trigger knowledge.
Significance
This research addresses a critical security challenge in deploying object detection models in safety-critical applications. By leveraging an intrinsic property of models—the distribution of pre-NMS class predictions—DistScan provides a practical, scalable, and architecture-agnostic solution for detecting hidden backdoors. It overcomes limitations of prior methods that depend on trigger reconstruction or architecture-specific assumptions, thus broadening the scope of defenses against sophisticated scene-level attacks. The approach enhances trustworthiness in AI systems used in autonomous vehicles, surveillance, and medical diagnostics, where security breaches can have severe consequences. Moreover, it offers a new perspective on model behavior analysis, fostering further research into distribution-based security metrics.
Technical Contribution
The key technical innovation lies in identifying the pre-NMS class distribution shift as a universal indicator of backdoor presence. Unlike trigger inversion or behavioral discrepancy methods, DistScan exploits the statistical divergence between the model's intermediate predictions and the training data prior, measured via Jensen-Shannon divergence. The framework is architecture-agnostic, applicable to both one-stage and two-stage detectors, and requires only a small set of clean samples and class frequency statistics. The method's simplicity, efficiency, and theoretical grounding in distribution shift analysis distinguish it from existing defenses, offering a new paradigm for model security assessment.
Novelty
This work is the first to leverage the intrinsic distribution shift in pre-NMS predictions as a backdoor signal in object detection models. Unlike prior approaches that depend on trigger reconstruction or internal module analysis, it capitalizes on a fundamental property of models—how their intermediate class predictions deviate from learned priors when backdoored. This insight enables a universal, architecture-independent detection method that works effectively against scene-level attacks, representing a significant advancement in AI security research.
Limitations
- The detection accuracy slightly decreases in datasets with many classes (e.g., COCO) or high inter-class similarity, due to potential class confusion affecting distribution estimates.
- The method assumes accurate knowledge of training class frequencies; significant biases or incomplete data may reduce detection reliability.
- In scenarios with extremely limited samples or highly dynamic environments, the stability of pre-NMS predictions may be compromised, requiring further robustness enhancements.
Future Work
Future research will explore integrating multi-modal data (e.g., spatial, contextual features) to improve detection in complex scenes. Developing real-time, scalable tools for deployment in industry settings is also a priority. Additionally, extending the framework to other vision tasks like instance segmentation and exploring theoretical bounds of distribution shift signals will deepen understanding and applicability.
AI Executive Summary
Object detection models are fundamental to many safety-critical systems, including autonomous vehicles, surveillance, and medical diagnostics. Despite their widespread adoption, these models are vulnerable to backdoor attacks, where malicious actors embed hidden triggers during training to manipulate model outputs under specific conditions. Such attacks pose severe security risks, especially when models are sourced from untrusted parties or deployed in sensitive environments.
Traditional defenses for backdoor detection, primarily developed for image classification, rely on trigger inversion or architecture-specific signals. However, these methods often fail in the context of object detection, particularly against scene-level attacks where a single trigger can induce widespread misbehavior across all objects in a scene. Recognizing this gap, the authors propose DistScan, a novel detection framework that leverages the intrinsic property of models—the distribution of pre-NMS class predictions.
The core insight is that benign models' pre-NMS class distributions closely mirror the training data's class frequencies, owing to the optimization process. Conversely, backdoored models exhibit a systematic shift in this distribution, even on clean inputs. By constructing a validation set aligned with the model's architecture—full images for single-stage detectors like YOLOv5, cropped proposals for two-stage detectors like Faster R-CNN—the method extracts pre-NMS predictions, filters low-confidence predictions, and aggregates class counts. Comparing this distribution to the training class prior using Jensen-Shannon divergence reveals significant deviations indicative of backdoors.
Extensive experiments on MS-COCO and PASCAL VOC datasets, involving 288 models across multiple attack scenarios, demonstrate the effectiveness of DistScan. The detection accuracy averages 96.99%, surpassing existing methods by over 27 percentage points. The method maintains high AUROC scores, indicating strong discrimination ability across various settings. Notably, it performs well against scene-level attacks, where prior techniques faltered, confirming its robustness and generalizability.
This work offers a practical, architecture-agnostic solution for model security, requiring only a small set of clean samples and class frequency statistics. Its simplicity and theoretical grounding in distribution shift analysis make it suitable for real-world deployment, significantly advancing the state of AI security in object detection. By addressing the limitations of trigger-based defenses, DistScan paves the way for safer, more trustworthy AI systems in critical applications.
Deep Analysis
Background
目标检测技术经过多年的发展，已成为自动驾驶、安防监控、医疗影像等领域的基础工具。早期方法如R-CNN系列（R-CNN、Fast R-CNN、Faster R-CNN）通过区域建议和分类实现目标检测，取得了显著性能提升。近年来，一阶段检测器如YOLO系列（YOLOv3、YOLOv4、YOLOv5）通过端到端训练实现实时检测，极大推动了工业应用的普及。与此同时，目标检测模型面临的安全威胁也逐渐显现，后门攻击成为研究热点。攻击者通过在训练数据中植入触发器，使模型在特定场景下表现异常，如误判、消失或插入目标。这些攻击具有隐蔽性强、泛化能力高、难以检测的特点。现有防御方法包括触发器逆向、行为分析等，但多依赖触发器信息或模型内部访问，难以应对场景级攻击和未知触发器。随着攻击手段的不断演进，研究者亟需一种无需触发器知识、具有良好泛化能力的检测方法。
Core Problem
目标检测模型在实际部署中面临后门攻击的威胁，攻击者通过在训练数据中植入触发器，使模型在特定场景下表现异常，如误判、消失或插入目标。这些攻击具有隐蔽性强、泛化能力高、难以检测的特点。传统检测方法多依赖触发器逆向或模型内部信息，存在泛化差、易被规避的问题。如何在不依赖触发器信息和模型内部结构的情况下，有效识别后门模型，成为当前的核心难题。特别是在场景级攻击中，攻击影响范围广、表现多样，进一步增加了检测难度。解决这一问题，不仅关系到模型安全，也关系到自动驾驶、安防等行业的应用安全性。
Innovation
本研究的核心创新在于：1）发现预NMS预测类别分布偏移是后门模型的普遍标志，突破了传统依赖触发器的限制；2）提出基于统计距离（JS散度）的检测框架DistScan，简洁高效，适用多模型、多场景；3）设计了针对不同架构的验证集构建策略，确保偏移检测的鲁棒性。该方法的创新在于：1）利用模型中间预测分布的偏移作为后门信号，避免了触发器逆向的复杂性；2）实现架构无关的检测机制，适应多样化模型结构；3）通过统计偏移量的阈值判定，简化了检测流程，提升了实用性。
Methodology
- �� 构建验证集：根据模型架构（单阶段或两阶段）选择完整图像或裁剪单目标图像，确保输入分布一致。
- �� 提取预NMS预测：对验证集中的每张图片，获取模型在预NMS阶段的类别预测，过滤置信度低的预测（阈值δ=0.0005）。
- �� 统计类别频次：将剩余预测按类别统计频次，形成类别分布向量bi(fθ)，并对所有图像的向量求和归一化，得到模型的预NMS类别分布。
- �� 计算偏移：将模型的类别分布与训练类别频率（R）归一化后比较，使用JS散度衡量偏移程度。
- �� 判定阈值：若偏移值超过预设阈值τ，则判定模型为后门模型，否则为正常模型。
- �� 关键参数：训练类别频率R由训练数据统计得出，偏移阈值τ通过验证集调优，确保检测的准确性和鲁棒性。
Experiments
实验在MS-COCO和PASCAL VOC两个数据集上进行，涵盖YOLOv5和Faster R-CNN两种架构。构建了多场景攻击模型（误分类、消失、插入），每类攻击训练18个后门模型和18个正常模型，总计288个模型。评估指标包括检测准确率、TPR、FPR和AUROC。通过调优偏移阈值，确保在不同模型和攻击场景下的检测效果。还进行了消融实验验证验证集构建策略和阈值对性能的影响，确保方法的稳定性和泛化能力。
Results
在所有场景和模型中，DistScan平均检测准确率达96.99%，在多类别复杂环境（COCO）中表现尤为优越，远超对比方法如ODSCAN和MIA。特别是在场景级攻击中，DistScan几乎实现100%的检测率，FPR保持在0.00%至5.56%之间。AUROC指标也显示出极强的区分能力，平均值超过0.97。消融实验验证了验证集构建策略和偏移阈值的合理性，确保在不同模型和数据分布下的稳定性。整体结果证明，该方法具有极强的实用性和泛化能力，为目标检测安全提供了新工具。
Applications
该检测方法适用于自动驾驶、安防监控、医疗影像等需要高安全性目标检测的行业。只需少量干净样本和训练类别频率信息，即可在模型部署前快速识别潜在后门模型，有助于提升系统整体安全性。未来，可结合实时检测和多模态信息，开发自动化安全审查工具，推动行业标准制定，保障关键基础设施的安全。
Limitations & Outlook
尽管方法表现优异，但在类别数极多（如COCO）或类别相似度高的场景中，检测准确率略有下降，可能受到类别混淆影响。此外，方法依赖于训练类别频率的准确统计，若训练数据偏差严重，可能影响检测效果。对极端复杂场景或极少样本的检测效果尚未充分验证，未来需结合多模态信息和深度特征进行优化。
Plain Language Accessible to non-experts
想象你在一个工厂里工作，工厂每天生产各种不同的商品，比如玩具、衣服、电子产品等。工厂的生产线非常有序，每个环节都按照一定的流程进行，生产出的商品也符合预先设定的比例和规格。现在，有个坏人偷偷在某些商品上放了特殊的标签（就像触发器），当这些标签出现时，工厂的机器就会出错，生产出错误的商品或者不生产某些商品。这些标签很隐秘，普通人很难发现。为了检测这些隐藏的标签，工厂的管理者开始观察每个环节的商品比例，看看是否有异常。正常情况下，商品的比例应该和平时一样，但如果发现某个类别的商品比例突然偏离，就说明可能有问题。这个检测方法就像是用统计学的方法，观察每个类别的商品数量，找出偏差，判断工厂是否被“植入”了隐藏的标签。这样，即使没有直接看到标签，也能发现工厂是否被操控了。这就是本文提出的DistScan方法的核心思想：通过观察模型在预测中的类别分布偏移，检测目标检测模型是否被植入了后门。
ELI14 Explained like you're 14
想象你在学校的食堂里吃饭，平时每次吃饭，大家都点不同的菜，比如面条、米饭、炒菜、汤等。每个人点的菜的比例大致是一样的，大家都按照平时的习惯点菜。但是，有个坏人偷偷在菜单上放了特殊的标记（就像触发器），当有人点了这个标记的菜时，食堂的厨师就会突然做出奇怪的菜，或者不做某些菜。这些奇怪的变化很难被普通人发现，因为标记很隐秘。为了检测这些坏人，食堂管理者开始统计每天每种菜的点餐比例，看看是否有异常。正常情况下，比例应该和以前一样，但如果发现某个菜的点餐比例突然变了，就说明可能有问题。这个检测方法就像用数学的方法，观察每个菜的点餐比例，找出偏差，判断食堂是否被操控了。这和我们用统计学检测模型是否被植入后门的原理一样。通过观察模型预测的类别分布偏移，我们可以判断模型是否被攻击了，而不用知道触发器的具体样子。
Abstract
Object detection models deployed in safety-critical applications remain vulnerable to backdoor attacks that cause targeted misbehaviors when a hidden trigger is present. Existing detection methods either rely on trigger inversion or exploit architecture-specific assumptions, and critically, representative existing methods fail to generalize reliably to scene-level attacks, where a single trigger induces anomalous behavior across all objects in the scene simultaneously. We present DistScan, a backdoor detection framework based on a simple but previously unexploited observation: backdoor injection systematically shifts a model's pre-NMS prediction class distribution away from its training class frequencies, even on clean inputs without any trigger present. DistScan aggregates intermediate class predictions over a clean validation set and flags a model as backdoored if the resulting distribution deviates significantly from the training class frequencies, requiring no model weight access, no trigger knowledge, and no additional training. Extensive experiments on MS-COCO and PASCAL VOC across two architectures and three scene-level attack scenarios demonstrate that DistScan substantially outperforms existing methods, improving average detection accuracy over the best-performing applicable baseline by 27.32 percentage points.
References (20)
BadDet: Backdoor Attacks on Object Detection
Shih-Han Chan, Yinpeng Dong, Junyi Zhu et al.
Detector Collapse: Backdooring Object Detection to Catastrophic Overload or Blindness
Hangtao Zhang, Shengshan Hu, Yichen Wang et al.
Towards Robust Object Detection: Identifying and Removing Backdoors via Module Inconsistency Analysis
Xianda Zhang, Siyuan Liang
OdScan: Backdoor Scanning for Object Detection Models
Siyuan Cheng, Guangyu Shen, Guanhong Tao et al.
Invisible Backdoor Attack with Sample-Specific Triggers
Yuezun Li, Yiming Li, Baoyuan Wu et al.
Comprehensive study on object detection for security and surveillance: A concise review
Faisal Alamri
Evaluating Object (Mis)Detection From a Safety and Reliability Perspective: Discussion and Measures
A. Ceccarelli, Leonardo Montecchi
Detecting AI Trojans Using Meta Neural Analysis
Xiaojun Xu, Qi Wang, Huichen Li et al.
BadNets: Evaluating Backdooring Attacks on Deep Neural Networks
Tianyu Gu, Kang Liu, Brendan Dolan-Gavitt et al.
Grounding DINO: Marrying DINO with Grounded Pre-Training for Open-Set Object Detection
Shilong Liu, Zhaoyang Zeng, Tianhe Ren et al.
Deep Learning-Based Medical Object Detection: A Survey
M. Saraei, Mehrshad Lalinia, Eung-Joo Lee
Rethinking the Backdoor Attacks’ Triggers: A Frequency Perspective
Yi Zeng, Won Park, Z. Mao et al.
Targeted Backdoor Attacks on Deep Learning Systems Using Data Poisoning
Xinyun Chen, Chang Liu, Bo Li et al.
Prevalence of neural collapse during the terminal phase of deep learning training
Vardan Papyan, Xuemei Han, D. Donoho
ABS: Scanning Neural Networks for Back-doors by Artificial Brain Stimulation
Yingqi Liu, Wen-Chuan Lee, Guanhong Tao et al.
Real-time object detection, tracking, and monitoring framework for security surveillance systems
S. Abba, Ali Mohammed Bizi, Jeong-A Lee et al.
Review of Object Detection Challenges in Autonomous Driving
Shen Cao
You Only Look Once: Unified, Real-Time Object Detection
J. Redmon, S. Divvala, Ross B. Girshick et al.
Unbiased look at dataset bias
A. Torralba, Alexei A. Efros
Test-Time Backdoor Detection for Object Detection Models
Hangtao Zhang, Yichen Wang, Shihui Yan et al.