Off the Rails: Hijacking the Scoring Head in Generative End-to-End Driving Planners with Safety-Violating Adversarial Perturbations
Proposes DERAIL attack exploiting scoring head vulnerability in generative E2E autonomous driving, causing collision rates up to 50%.
Key Findings
Methodology
This paper analyzes the inference pattern of generative end-to-end (E2E) autonomous driving systems, identifying the scoring head as a critical vulnerability. The authors introduce DERAIL, an adversarial framework that employs behavior-level objectives—such as forward aggression, boundary push, and sudden braking—to craft perturbations via gradient-based optimization. The attack operates under both digital and physical threat models, using end-to-end differentiable pipelines that manipulate visual inputs to flip the scoring head’s decision from a safe to an unsafe candidate. Experiments across four state-of-the-art models—DiffusionDrive, GTRS-DP, GTRS-AUG, and GTRS-Dense—demonstrate that DERAIL achieves 100% attack success rate, with score drops of 39-80% and collision rates reaching 50%, outperforming baseline methods like loss maximization and feature divergence.
Key Results
- DERAIL achieves 100% success rate across all tested models, with collision rates up to 50% and score reductions between 39% and 80%. It effectively manipulates the scoring head with minimal pixel perturbations or physical patches, often imperceptible to human observers. The attack remains robust under various candidate set sizes and multi-modal scenarios, indicating a fundamental vulnerability in the scoring mechanism. Compared to traditional adversarial methods, DERAIL’s behavior-level optimization directly targets the trajectory decision, leading to more effective safety violations.
Significance
This work exposes a critical security flaw in contemporary generative autonomous driving systems, emphasizing the scoring head as a universal attack surface. The findings demonstrate that small perceptual perturbations can induce severe safety risks, such as collisions or off-road driving, without requiring internal model access. By bridging the gap between perception vulnerabilities and trajectory decision-making, the study provides a new perspective on robustness and safety in autonomous vehicles. Its implications urge the industry to incorporate safety filters and adversarial defenses, fostering more resilient autonomous driving architectures. Theoretically, it advances understanding of how decision boundaries in neural scoring modules can be exploited, influencing future research on secure AI systems.
Technical Contribution
The paper introduces DERAIL, a unified, fully differentiable adversarial attack pipeline that optimizes behavior-level objectives directly in trajectory space. It innovates by identifying the scoring head as a common vulnerability across diffusion-based and vocabulary-based planners, and designing multi-objective loss functions aligned with safety-critical behaviors. The method supports both digital pixel perturbations and physical adversarial patches, employing gradient-based optimization with stochastic geometric augmentations for robustness. This approach enables efficient, transferable attacks without model-specific tuning, representing a significant step forward in understanding and exploiting the vulnerabilities of generative AD systems.
Novelty
This is the first comprehensive study targeting the scoring head in generative E2E autonomous driving models, revealing its role as a universal attack surface. Unlike prior work focused on perception or intermediate features, DERAIL directly manipulates the trajectory decision output using behavior-level objectives. Its dual attack modes—digital and physical—along with end-to-end differentiability and multi-objective optimization, set it apart from existing adversarial strategies, establishing a new paradigm in autonomous vehicle security research.
Limitations
- The attack assumes white-box access to model weights and architecture, limiting direct applicability in black-box real-world scenarios.
- Effectiveness may diminish against models with robust safety filters or adversarial training.
- Physical patch deployment faces practical challenges such as scene variability and durability of adversarial materials.
Future Work
Future research should explore black-box attack strategies, enhance physical attack robustness, and develop effective defense mechanisms, such as safety filters or adversarial training. Extending the framework to multi-modal sensor fusion and real-world testing will further validate its applicability. Additionally, integrating formal safety guarantees and robustness certification could help mitigate such vulnerabilities in deployed systems.
AI Executive Summary
Autonomous driving has rapidly evolved with the adoption of generative models like diffusion-based and vocabulary retrieval methods, which generate trajectories conditioned on visual and spatial features. These systems typically rely on a scoring head to select the optimal trajectory from a fixed candidate set, a process that, while efficient, introduces a critical vulnerability. The scoring head acts as a decision boundary sensitive to small input perturbations, making it an attractive target for adversarial attacks.
This paper introduces DERAIL, an adversarial framework that exploits this vulnerability by optimizing behavior-level objectives—such as inducing forward aggression, boundary violations, or abrupt braking—through gradient-based methods. The attack can be executed via subtle digital pixel perturbations or physically realizable adversarial patches, both capable of flipping the scoring head’s decision from a safe to an unsafe candidate. Extensive experiments across four state-of-the-art models demonstrate that DERAIL achieves a 100% attack success rate, with collision rates soaring up to 50%, significantly higher than baseline methods.
The findings highlight a fundamental security flaw in current generative AD planners, emphasizing the need for incorporating safety filters and robust training strategies. The work bridges perception vulnerabilities with trajectory decision-making, providing a comprehensive understanding of how small perceptual changes can lead to catastrophic safety failures. Its implications extend to the design of future resilient autonomous systems, urging the community to rethink safety mechanisms and adversarial defenses.
Despite its strengths, the study assumes white-box access and faces challenges in real-world physical deployment due to scene variability. Nonetheless, it sets a new benchmark in understanding and mitigating adversarial risks in autonomous driving, paving the way for safer, more reliable intelligent transportation systems.
Deep Dive
Plain Language Accessible to non-experts
想象你在玩一款赛车游戏，你可以选择很多不同的路线。游戏中的电脑会帮你评估哪条路最安全最快，然后让你选择。现在，有个坏人偷偷在你的游戏屏幕上贴了个看不出来的贴纸，或者用特殊的光线让电脑看到的画面变得不一样。这样一来，电脑可能会误判，把危险的路线当成最好的，结果你就可能撞到障碍物或偏离路线。这就像是有人在暗中操控游戏，让你走上一条危险的路。科学家们发现，这个“评判器”其实是个漏洞，只要稍微干扰一下，就能让自动驾驶汽车变得不安全。于是，他们设计了一个叫DERAIL的办法，模拟这种攻击，帮助开发者知道怎么防止这种情况发生，让自动驾驶变得更安全、更聪明。
ELI14 Explained like you're 14
你知道吗？自动驾驶汽车其实就像一个超级聪明的机器人司机，它会看着前面的路，然后决定怎么走。它会考虑很多可能的路线，然后用一个“评分器”来判断哪条路最安全。可是，有坏人发现了这个“评分器”的漏洞，他们用一些特别的小技巧，比如偷偷在摄像头上贴个看不出来的贴纸，或者用光线让摄像头看到的画面变得不一样。这样一来，评分器就会误判，把危险的路线当成最好的，结果汽车可能会撞到东西或者偏离车道。这个研究告诉我们，这个“评分器”其实是个弱点，科学家们还设计了一个叫DERAIL的办法，模拟这种攻击，帮助大家知道怎么保护自动驾驶汽车，让它们变得更安全、更聪明。
Abstract
Generative models have recently seen rapid adoption in End-to-End (E2E) autonomous driving (AD), with diffusion-based denoising and vocabulary-based retrieval becoming the dominant trajectory-decoding paradigms. Despite their architectural diversity, current generative AD planners share a common inference pattern: a fixed set of candidate trajectories (anchors, vocabulary entries, or proposal queries) is scored by one or more learned heads conditioned on the Bird's-Eye-View (BEV) features, and the highest-scored candidate is returned as the final trajectory. Under this design, the scoring head is the only barrier between perception and the motion command, and its decision margins between competing candidates are often small. We introduce \textsc{Derail}, an adversarial framework that exploits this scoring-head attack surface. Evaluated on various generative planners, \textsc{Derail} flips the trajectory selection from a safe to an unsafe candidate, with score drops of $39$--$80\%$ and collision rates of up to $50\%$, consistently outperforming generic loss-maximization and feature-divergence attacks. Our analysis suggests that safety-violating objectives govern attack effectiveness against generative AD planners, and that the scoring-head inference pattern itself is a recurring attack surface worth explicit defensive consideration.