Test-Time Backdoor Detection for Object Detection Models
Proposes TRACE, a semantic-aware transformation method, achieving 30% AUROC improvement for test-time backdoor detection in object detection.
Key Findings
Methodology
TRACE employs foreground and background transformations to evaluate detection consistency across varied backgrounds and focal points. It calculates variance in object confidence scores under these transformations, exploiting the phenomenon that poisoned samples show anomalously stable detection results. The method combines background perturbation and focal object sliding to generate two metrics—contextual and focal transformation consistency—forming a unified detection score. Extensive experiments on MS-COCO, PASCAL VOC, and synthetic traffic sign datasets demonstrate a 30% AUROC gain over state-of-the-art defenses, with robustness against adaptive attacks. The approach is model-agnostic and requires only black-box access, making it practical for real-world deployment.
Key Results
- Across multiple datasets and attack types, TRACE consistently outperformed existing methods, achieving an average AUROC increase of 30%. For example, on MS-COCO, it reached AUROC scores above 0.85 against seven different backdoor attacks, surpassing Detector Cleanse by significant margins.
- The combination of background blending and focal sliding transformations effectively distinguished poisoned samples, especially in complex scenarios involving 'ghost' or 'vanishing' objects. The method maintained high detection accuracy with minimal false positives.
- The experiments confirmed TRACE’s robustness to adaptive attacks designed to mimic benign transformations, demonstrating its potential as a universal, black-box defense in diverse application contexts.
Significance
This work advances the field of object detection security by providing a practical, model-agnostic detection framework that does not rely on internal model details. By leveraging semantic-aware transformations, it addresses the limitations of pixel-level or white-box methods, effectively detecting sophisticated backdoor triggers that cause anomalous detection behaviors. The approach enhances the safety of deployed detection systems in autonomous vehicles, surveillance, and other critical applications, paving the way for more resilient AI models. Its universal applicability and robustness against adaptive attacks mark a significant step toward real-world secure AI deployment.
Technical Contribution
The core innovation lies in the semantic-aware transformation consistency metrics—contextual and focal—derived from the model’s response to background perturbations and object sliding. These metrics capture the intrinsic differences in how poisoned and clean samples react to environment changes. The framework integrates these metrics into a unified detection score using a sigmoid normalization, enabling effective black-box detection without training data or attack-specific knowledge. The theoretical analysis and extensive empirical validation establish the method’s robustness and generalization capabilities, representing a new paradigm in backdoor detection for object detection models.
Novelty
This is the first work to exploit semantic-aware transformations—background blending and focal object sliding—for backdoor detection in object detection tasks. Unlike pixel-level or white-box approaches, TRACE leverages the inherent stability differences in poisoned samples’ detection confidence across semantic transformations. Its model-agnostic, real-time detection capability fills a critical gap in the security of complex detection systems, offering a novel, generalizable solution that addresses the unique challenges posed by object detection’s output complexity.
Limitations
- The method’s effectiveness may diminish under extreme background similarity or highly context-dependent objects, leading to potential false negatives or positives.
- Computational overhead remains significant, especially when sampling multiple transformations for large-scale deployment, requiring further optimization.
- Natural benign objects with standardized shapes and colors might exhibit similar stability patterns, necessitating additional filtering strategies to reduce false alarms.
Future Work
Future research will focus on integrating semantic understanding via multi-modal data, reducing computational costs through adaptive sampling, and extending the framework to multi-task scenarios. Additionally, exploring unsupervised or semi-supervised approaches to automatically filter benign stable objects will further enhance detection accuracy and practicality in real-world applications.
AI Executive Summary
Object detection models are integral to many modern AI applications, from autonomous driving to surveillance. However, their vulnerability to backdoor attacks poses significant security risks. Attackers embed triggers into training data, causing models to behave maliciously when triggers are present, while appearing normal otherwise. Existing defenses often rely on white-box access or attack-specific features, limiting their practicality and robustness. This gap calls for a universal, black-box detection method capable of identifying poisoned samples during testing.
In response, this paper introduces TRACE, a novel detection framework based on semantic-aware transformations. By applying background blending and focal object sliding, TRACE evaluates the consistency of detection results across varied environments. The key insight is that poisoned samples exhibit anomalously stable detection confidence under these transformations, unlike benign samples that fluctuate significantly. These phenomena are quantified through two metrics—contextual and focal transformation consistency—and combined into a unified detection score.
Extensive experiments across multiple datasets and attack types demonstrate TRACE’s superior performance. It achieves an average 30% increase in AUROC over existing methods, maintains robustness against adaptive attacks, and operates effectively in a black-box setting without requiring training data or internal model details. This approach addresses core challenges in object detection security, providing a practical, scalable, and attack-agnostic solution.
The broader impact of this work lies in enhancing the safety and reliability of AI systems deployed in critical sectors. By leveraging semantic understanding and environment variability, TRACE sets a new standard for robust backdoor detection, paving the way for more secure AI applications. Future directions include integrating multi-modal data, optimizing computational efficiency, and extending to multi-task scenarios, ultimately fostering resilient and trustworthy AI systems in complex real-world environments.
Deep Dive
Plain Language Accessible to non-experts
想象你在一个工厂里，工人们每天都在生产各种商品。有些工人偷偷在商品里贴了特殊的标签（像触发器），只要有人看到，就会触发他们预设的动作。检测这些隐藏标签就像用不同的灯光和角度观察商品，看看它们在不同光线下是否表现一致。带有标签的商品在不同环境中表现得非常稳定，而正常商品会有变化。通过不断变换背景和观察商品的反应，工厂管理者可以更容易找到那些藏在商品里的“标签”，确保工厂的安全。这种方法不用知道商品的内部细节，只通过观察它们在不同条件下的表现差异，就能识别出潜在的危险商品，防止被欺骗。
Abstract
Object detection models are vulnerable to backdoor attacks, where attackers poison a small subset of training samples by embedding a predefined trigger to manipulate prediction. Detecting poisoned samples (i.e., those containing triggers) at test time can prevent backdoor activation. However, unlike image classification tasks, the unique characteristics of object detection -- particularly its output of numerous objects -- pose fresh challenges for backdoor detection. The complex attack effects (e.g., "ghost" object emergence or "vanishing" object) further render current defenses fundamentally inadequate. To this end, we design TRAnsformation Consistency Evaluation (TRACE), a brand-new method for detecting poisoned samples at test time in object detection. Our journey begins with two intriguing observations: (1) poisoned samples exhibit significantly more consistent detection results than clean ones across varied backgrounds. (2) clean samples show higher detection consistency when introduced to different focal information. Based on these phenomena, TRACE applies foreground and background transformations to each test sample, then assesses transformation consistency by calculating the variance in objects confidences. TRACE achieves black-box, universal backdoor detection, with extensive experiments showing a 30% improvement in AUROC over state-of-the-art defenses and resistance to adaptive attacks.