On Adversarial Robustness of Trajectory Prediction for Autonomous Vehicles
Proposes physics-constrained adversarial attack boosting trajectory prediction error by 150%, highlighting safety risks in autonomous driving.
Key Findings
Methodology
This paper introduces a physics-aware adversarial attack framework combining white-box PGD and black-box PSO algorithms. The attack perturbs normal vehicle trajectories within physical constraints—velocity, acceleration—to maximize prediction errors across multiple models (FQA, GRIP++, Trajectron++). The approach ensures the adversarial trajectories appear natural and feasible, mimicking real driving behavior. Experiments on three datasets (Apolloscape, NGSIM, nuScenes) demonstrate an average error increase of 167%, with some deviations exceeding half a lane width (~1.85m). Mitigation techniques such as data augmentation and trajectory smoothing reduce errors by 28%, enhancing model robustness. The comprehensive evaluation underscores the vulnerability of current models to realistic adversarial scenarios, raising critical safety concerns.
Key Results
- Across models and datasets, the attack increased average ADE/FDE by 167%, with lateral and longitudinal deviations reaching 2.03m and 3.84m respectively. Over 62% of attacks caused deviations larger than 1.85m, risking lane changes and unsafe maneuvers.
- Physical constraints maintained the naturalness of adversarial trajectories, validating their real-world plausibility. The attack’s success was consistent across white-box and black-box settings, exposing fundamental vulnerabilities.
- Data augmentation and trajectory smoothing significantly mitigated attack effects, reducing prediction errors by 28%. Different models exhibited varying robustness, with map-aware models showing better resistance.
Significance
This pioneering work systematically evaluates the adversarial robustness of trajectory prediction models, revealing their susceptibility to realistic, physically feasible attacks. As autonomous vehicles become widespread, understanding and defending against such threats is vital for safety assurance. The findings highlight the need for incorporating adversarial robustness into model design, influencing future standards and safety protocols in autonomous driving. The research bridges the gap between theoretical vulnerability analysis and practical safety implications, urging industry and academia to prioritize security evaluations alongside accuracy metrics.
Technical Contribution
The study introduces a novel adversarial attack framework that respects physical vehicle dynamics, combining PGD and PSO algorithms for diverse attack scenarios. It innovatively integrates multi-objective optimization targeting lateral and longitudinal deviations, ensuring the adversarial trajectories are both effective and realistic. The approach leverages trajectory smoothing and data augmentation to improve model resilience, offering a comprehensive methodology for safety-critical evaluations. This work advances the state-of-the-art by providing a systematic, scalable framework for testing and enhancing the robustness of deep learning-based trajectory prediction models in autonomous vehicles.
Novelty
This is the first comprehensive study to develop physically plausible adversarial attacks tailored for trajectory prediction in autonomous driving, addressing both naturalness and effectiveness. Unlike prior work focused on static image or sensor data, this research considers dynamic, real-world vehicle trajectories, introducing multi-directional, multi-objective optimization strategies. The dual-algorithm approach (PGD for white-box, PSO for black-box) broadens attack applicability. Its integration of physical constraints and multi-model validation marks a significant step forward, providing a new paradigm for安全性评估和模型防御。
Limitations
- 攻击效果依赖于未来轨迹预测，环境变化和模型不确定性可能影响实际应用中的效果。
- 生成对抗样本的计算成本较高，尤其在多目标、多模型场景下，实时防御仍具挑战性。
- 目前验证主要在模拟环境，实际复杂交通场景中的鲁棒性和可行性仍需进一步验证和优化。
Future Work
未来将探索多车辆协同攻击与防御机制，增强模型在复杂交通环境中的鲁棒性。引入多模态传感信息（如雷达、摄像头）丰富对抗样本的多样性，研究模型的安全性评估指标体系。结合强化学习实现自适应防御策略，推动自动驾驶系统的安全可信发展。
AI Executive Summary
Autonomous driving has revolutionized transportation, promising safer and more efficient roads. Yet, the safety of these systems hinges on the robustness of core modules like trajectory prediction. Despite significant advances with deep neural networks such as LSTM, GRU, and Trajectron++, these models remain vulnerable to adversarial manipulations. This paper exposes a critical security gap by developing a physics-constrained adversarial attack that can significantly distort trajectory predictions. The attack employs a combination of projected gradient descent (PGD) and particle swarm optimization (PSO) to generate subtle yet effective perturbations that maximize prediction errors by over 150%, while maintaining physical plausibility. These adversarial trajectories, crafted within vehicle dynamics constraints, can cause the autonomous vehicle to misjudge the environment, potentially leading to unsafe maneuvers like unnecessary braking or lane changes.
The experiments span three datasets—Apolloscape, NGSIM, and nuScenes—and three models, demonstrating consistent vulnerability across scenarios. Notably, over 62% of attacks produce deviations exceeding half a lane width (~1.85 meters), posing real-world safety risks. To counteract this, the authors propose mitigation strategies including data augmentation and trajectory smoothing, which reduce errors by 28%, enhancing model robustness.
This work underscores the urgent need to incorporate adversarial robustness into trajectory prediction systems. It not only reveals the fragility of current models but also provides a comprehensive framework for evaluating and improving their security. The findings have profound implications for industry standards, emphasizing that safety assessments must go beyond average accuracy to consider worst-case scenarios. Future directions include multi-vehicle attack-defense mechanisms, multi-modal sensor integration, and adaptive learning-based defenses, aiming to build safer, more reliable autonomous vehicles. Overall, this research marks a pivotal step toward understanding and mitigating security threats in autonomous driving, ensuring safer roads for everyone.
Deep Analysis
Background
自动驾驶技术经过多年的发展，逐步实现了从感知、规划到控制的完整闭环。轨迹预测作为感知模块的核心，旨在预估周围动态对象（如车辆、行人）的未来位置。早期方法多采用基于卡尔曼滤波或统计模型，近年来深度学习模型如LSTM、GRU、GMM等显著提升了预测精度。代表性模型包括FQA、GRIP++和Trajectron++，它们在Apolloscape、NGSIM和nuScenes等数据集上表现优异。然而，现有研究主要关注平均性能指标，缺乏对模型在极端或恶意扰动下的鲁棒性分析。随着自动驾驶系统逐渐商业化，模型面临来自恶意攻击者的潜在威胁，亟需系统性评估其安全性。
Core Problem
尽管深度学习模型在轨迹预测中取得了突破，但其对对抗扰动的敏感性尚未充分研究。攻击者可以操控车辆轨迹，造成预测偏差，从而误导自动驾驶系统做出危险反应，如误刹车或偏离车道。现有的安全评估多局限于正常场景，缺乏考虑自然且符合物理规律的对抗轨迹。如何在保证轨迹自然合理的前提下，最大化预测误差，成为关键难题。这关系到自动驾驶系统的安全性和可靠性，也影响其在实际道路环境中的应用。
Innovation
本研究的创新点主要体现在：1）提出结合车辆动力学约束的对抗轨迹生成框架，确保扰动轨迹的自然性和可行性；2）设计多目标、多方向的优化策略，丰富对抗样本的多样性；3）采用PGD（投影梯度下降）和PSO（粒子群优化）两种算法，适应不同攻击场景（白盒与黑盒）；4）引入轨迹平滑和数据增强技术，有效提升模型对对抗样本的鲁棒性。这些创新突破了传统静态攻击的局限，为轨迹预测模型的安全性评估提供了新思路。
Methodology
- �� 轨迹预测定义：输入历史轨迹，输出未来位置，模型包括LSTM、GRU、GMM等。
- �� 攻击目标：操控目标车辆轨迹，最大化模型预测误差，确保轨迹自然且符合车辆动力学。
- �� 扰动生成：在历史轨迹上添加微小扰动，利用PGD或PSO优化扰动，确保扰动在物理合理范围内。
- �� 物理约束：限制速度、加速度在合理范围内，确保扰动轨迹的自然性。
- �� 多目标优化：考虑横向、纵向偏差，最大化偏离距离，影响自动驾驶决策。
- �� 实验验证：在三类数据集和三种模型上测试，评估误差提升和轨迹自然性。
Experiments
采用Apolloscape、NGSIM、nuScenes三大数据集，历史轨迹长度（LI）和未来预测长度（LO）分别为6、6、4、12等。模型包括FQA、GRIP++、Trajectron++，训练细节遵循原始论文参数。攻击采用PGD和PSO算法，最大迭代100次，扰动限制在1米以内。评估指标包括ADE、FDE及偏离车道宽度的偏差，验证攻击效果和自然性。对不同模型、数据集和攻击策略进行对比分析，验证攻击成功率和模型鲁棒性差异。
Results
攻击显著提升预测误差，平均增加167%，偏差超过1.85米（车道宽度一半），部分偏差达6米以上。引入物理约束确保轨迹自然，攻击轨迹在车辆动力学范围内，验证其在实际环境中的可行性。数据增强和轨迹平滑技术有效降低模型敏感性，误差减少28%。不同模型对攻击的敏感性存在差异，结合地图信息的模型表现更鲁棒。白盒与黑盒攻击均验证了模型的脆弱性，强调安全评估的重要性。
Applications
该研究为自动驾驶系统提供了安全性评估工具，帮助开发者识别潜在漏洞。可用于设计更鲁棒的轨迹预测模型，增强系统在恶意攻击下的稳定性。未来，结合多模态信息和强化学习的防御策略，有望实现自动驾驶的全面安全保障。
Limitations & Outlook
当前方法依赖未来轨迹预测，存在环境变化和模型不确定性带来的效果不稳定性。攻击生成计算成本较高，难以实现实时应用。模型在复杂交通场景中的鲁棒性仍需验证，实际部署中还面临多重挑战。未来需优化算法效率，增强模型适应性和泛化能力。
Plain Language Accessible to non-experts
想象你在操控一辆遥控车，你可以微调它的路线，让它看起来像是在正常驾驶，但实际上是在偷偷偏离预定路线。你必须确保这条偏离的路线看起来合理，符合车辆的运动规律，比如速度不能太快或太慢。这样一来，自动驾驶系统就可能误判车辆的真实位置，做出错误反应，比如突然刹车或偏离车道。这就像在玩一场“迷惑”游戏，既要让轨迹看起来正常，又要让自动驾驶系统误以为你还在正常行驶。研究中用数学方法确保偏离轨迹合理，同时最大化误判的可能性。这个研究提醒我们，未来的自动驾驶系统必须能识别这些“伪装”的轨迹，才能保证行车安全。
ELI14 Explained like you're 14
假设你在玩一款赛车游戏，你可以偷偷改变你的赛车路线，让它看起来像是在正常跑，但实际上偏离了轨迹。这样，游戏里的AI可能会误以为你还在正常驾驶，结果做出错误的反应，比如提前刹车或转弯。研究人员发现，如果有人用类似的方法操控真实车辆轨迹，也会让自动驾驶系统误判，从而引发危险。为了测试这个问题，他们设计了一种方法，既让轨迹看起来合理，又能迷惑自动驾驶模型。就像在画一幅看似正常的画，但其实藏着一些细节，让人看不出来。这样一来，自动驾驶系统就需要变得更聪明，能识别出这些“伪装”的轨迹，保证行车安全。这项研究让我们明白，未来的自动驾驶不仅要看得清楚，还要学会识别那些故意“装扮”的轨迹，才能确保每次出行都安全无忧。
Abstract
Trajectory prediction is a critical component for autonomous vehicles (AVs) to perform safe planning and navigation. However, few studies have analyzed the adversarial robustness of trajectory prediction or investigated whether the worst-case prediction can still lead to safe planning. To bridge this gap, we study the adversarial robustness of trajectory prediction models by proposing a new adversarial attack that perturbs normal vehicle trajectories to maximize the prediction error. Our experiments on three models and three datasets show that the adversarial prediction increases the prediction error by more than 150%. Our case studies show that if an adversary drives a vehicle close to the target AV following the adversarial trajectory, the AV may make an inaccurate prediction and even make unsafe driving decisions. We also explore possible mitigation techniques via data augmentation and trajectory smoothing. The implementation is open source at https://github.com/zqzqz/AdvTrajectoryPrediction.