EvaDrive: Evolutionary Adversarial Policy Optimization for End-to-End Autonomous Driving
EvaDrive employs adversarial multi-objective reinforcement learning, achieving 94.9 PDMS on NAVSIM and diverse driving styles without external preferences.
Key Findings
Methodology
EvaDrive integrates a hierarchical generator combining autoregressive intent modeling with diffusion-based refinement, forming a multi-round adversarial co-evolution framework. The generator produces diverse candidate trajectories, guided by temporal causality and spatial flexibility, while a trainable multi-objective critic evaluates these proposals across safety, comfort, and efficiency, preserving preference structures. The Pareto frontier selection mechanism enables iterative refinement, escaping local optima. A learnable reward model replaces scalar rewards, maintaining multi-dimensional preferences. The multi-turn optimization process iteratively samples, evaluates, and refines trajectories, fostering diversity and robustness in decision-making.
Key Results
- On NAVSIM v1, EvaDrive achieves 94.9 PDMS, surpassing DiffusionDrive by 6.8, DriveSuprim by 5.0, and TrajHF by 0.9, demonstrating state-of-the-art performance.
- In Bench2Drive, it scores 64.96 in driving score, indicating strong style diversity and preference adaptation.
- The multi-round adversarial optimization significantly improves trajectory diversity, robustness, and preference preservation, outperforming baselines across multiple metrics.
Significance
This approach addresses the limitations of scalarized rewards and one-step optimization, closely mimicking human iterative decision processes. It enhances autonomous systems' ability to balance conflicting objectives, improve safety, and generate diverse, human-like driving behaviors. The explicit preservation of preference structures offers a new paradigm for multi-objective autonomous decision-making, with broad implications for industry and research.
Technical Contribution
The paper introduces a layered trajectory generator combining autoregressive and diffusion models, enabling accurate, flexible path proposals. It pioneers a multi-round adversarial framework with Pareto front exploration, maintaining multi-objective preferences without scalarization bias. The learnable reward critic explicitly models multi-dimensional metrics, facilitating stable, preference-aware optimization. This dual innovation enhances diversity, robustness, and interpretability of autonomous planning.
Novelty
This work is the first to embed multi-objective adversarial co-evolution into autonomous trajectory optimization, integrating multi-round iterative refinement with Pareto front sampling. It diverges from existing single-objective or linear scalarization methods, offering a fundamentally new way to preserve preference structures and improve decision robustness.
Limitations
- Training relies heavily on simulation data, which may not fully capture real-world complexities, leading to potential domain gaps.
- Computational costs of multi-round optimization are high, posing challenges for real-time deployment.
- Dynamic preferences and environmental changes are not fully modeled, requiring future work to incorporate adaptive preference learning.
Future Work
Future directions include integrating online learning to adapt to changing human preferences, extending multi-modal perception for better environment understanding, and optimizing computational efficiency for real-time applications. Exploring multi-agent interactions and scaling to larger environments are also promising avenues.
AI Executive Summary
Autonomous driving has rapidly advanced, yet achieving human-like iterative decision-making remains a challenge. Traditional planning methods often treat trajectory generation and evaluation as separate steps, limiting the ability to refine proposals dynamically. Recent generation-evaluation frameworks have improved diversity but lack the closed-loop interaction essential for human-like reasoning. EvaDrive introduces一种创新的多目标对抗演化强化学习框架，将轨迹生成与评估结合在一个闭环系统中，模拟人类反复试验和优化的过程。核心在于层次化生成器结合自回归意图建模和扩散模型，生成多样候选路径；同时，训练可学习的多目标critic，保持偏好结构，避免偏差。多轮对抗机制通过Pareto前沿采样，支持多样风格的轨迹优化，有效避免局部最优。大量在NAVSIM和Bench2Drive上的实验验证了EvaDrive的优越性能，PDMS达94.9，驾驶评分64.96，表现优于现有方法。该方法不仅提升了轨迹的多样性和鲁棒性，也为自主驾驶中的多目标决策提供了新思路。未来，结合在线偏好调整与多模态感知，EvaDrive有望实现更安全、更智能的自动驾驶系统，推动行业迈向更人性化的未来。
Deep Analysis
Background
自主驾驶技术经历了感知、预测、规划的逐步演进。深度学习推动感知精度提升，生成模型如DiffusionDrive增强轨迹多样性。传统模块化设计虽具解释性，但误差累积影响整体性能。端到端方法如Transfuser、UniAD实现整体优化，但在多模态、多目标场景中表现有限。生成-评估框架逐渐兴起，利用概率模型捕获轨迹不确定性，结合可微指标改善多样性。尽管如此，缺乏闭环交互，难以模拟人类反复试错的决策过程。强化学习逐渐成为解决方案，结合人类偏好微调，但多目标偏好保持与多轮优化仍是难点。
Core Problem
核心问题在于如何在复杂、多目标、多模态环境下实现轨迹的多样性、鲁棒性与偏好一致性。传统方法多采用单一标量奖励，忽略偏好结构，导致优化偏向某一目标。现有RL方法缺乏多轮反复试验机制，难以模拟人类反复试错的决策过程，容易陷入局部最优。此外，偏好偏差与多目标冲突未被充分解决，限制了自主系统的适应性与安全性。
Innovation
引入层次化轨迹生成器，结合自回归意图模型与扩散模型，提升路径生成的时间一致性与空间灵活性。提出多轮对抗式偏好优化，通过Pareto前沿采样实现偏好多样性保持，避免偏差与局部最优。采用可学习的多目标奖励模型，显式保持偏好结构，增强泛化能力。多轮优化机制支持动态环境下的连续调整，模拟人类反复试错的决策过程，突破传统单目标限制。
Methodology
- �� 轨迹生成器由两个核心部分组成：自回归意图模型捕获时间因果关系，扩散模型提供空间灵活性；
- �� 输入当前状态，利用多层感知机生成候选路径，结合视觉特征与历史轨迹信息；
- �� 通过多头交叉注意机制（MHCA）实现时间依赖建模，确保路径的时间一致性；
- �� 利用DDIM扩散模型引入随机噪声，生成多样候选路径，增强不确定性表达；
- �� 经过空间交叉注意与轻量Transformer解码器，优化路径候选，输出最终轨迹；
- �� 训练中引入多目标奖励模型，评估轨迹在安全、舒适、效率等方面的表现，保持偏好结构；
- �� 多轮优化机制：在每轮中，生成候选路径，评估多目标奖励，提取帕累托前沿，采样偏好多样的轨迹，指导下一轮生成；
- �� 采用对抗训练策略，优化生成器与奖励模型，提升轨迹质量与偏好一致性。
Experiments
在NAVSIM和Bench2Drive两个基准上进行评估，NAVSIM用于PDMS指标，模拟真实场景中的轨迹多样性与安全性；Bench2Drive通过CARLA仿真评估驾驶评分与风格多样性。模型采用ResNet34作为特征提取 backbone，训练细节包括多轮优化、Pareto前沿采样、偏好结构保持。对比基线包括DiffusionDrive、DriveSuprim、TrajHF等。指标涵盖PDMS、Driving Score等，实验验证了EvaDrive在多目标优化中的优越性，特别是在多样性、鲁棒性和偏好适应性方面。
Results
在NAVSIM v1上，EvaDrive达94.9 PDMS，超越DiffusionDrive 6.8、DriveSuprim 5.0、TrajHF 0.9，表现显著优于现有方法。在Bench2Drive中，获得64.96驾驶评分，展现多样驾驶风格与偏好适应能力。多轮对抗优化有效提升轨迹多样性，避免局部最优，保持偏好结构。实验还显示模型在不同驾驶风格下表现一致，具有良好的泛化能力。
Applications
该方法适用于自动驾驶车辆的多目标决策场景，特别是在复杂环境中实现安全、舒适与效率的平衡。可应用于城市交通、自动泊车、长途驾驶等场景，依赖高质量感知与环境建模，未来结合人类偏好动态调整，提升系统智能化水平。
Limitations & Outlook
模型训练依赖大量模拟数据，实际部署可能面临域偏差。多轮优化计算成本较高，实时性待提升。偏好结构的动态变化未充分考虑，未来需引入更复杂的偏好建模与在线学习机制。
Plain Language Accessible to non-experts
想象你在厨房做饭，准备一道复杂菜肴。你会先考虑各种食材和调料，尝试不同的搭配，然后不断品尝调整味道。这就像自主驾驶中的轨迹规划，系统会提出多个可能的路径（食材组合），评估它们的安全、舒适和效率（味道），然后不断优化，直到找到最合适的方案。EvaDrive就像一个聪明的厨师，能不断试验不同的菜谱，学会平衡各种需求，最终做出既安全又好吃的菜肴。这种反复试错、不断改进的过程，让自动驾驶变得更像人类一样聪明和细心。
ELI14 Explained like you're 14
你知道吗？开车就像玩一款需要不断调整策略的游戏。司机会试着走不同的路线，看看哪个既快又安全，还不会让乘客不舒服。这个过程就像你在玩游戏时不断尝试不同的角色技能，找到最适合的组合。EvaDrive就像一个超级聪明的游戏角色，它可以提出很多不同的路线，然后像你一样不断试验、评估，最后选出最棒的那条。它不会只看一次就决定，而是反复试验，确保每次都更好。这就像你在玩策略游戏，逐步完善你的计划，直到赢得比赛。这样，自动驾驶也能变得更聪明、更可靠，就像你变成了游戏里的高手一样！
Abstract
Autonomous driving faces significant challenges in achieving human-like iterative decision-making, which continuously generates, evaluates, and refines trajectory proposals. Current generation-evaluation frameworks isolate trajectory generation from quality assessment, preventing iterative refinement essential for planning, while reinforcement learning methods collapse multi-dimensional preferences into scalar rewards, obscuring critical trade-offs and yielding scalarization bias.To overcome these issues, we present EvaDrive, a novel multi-objective reinforcement learning framework that establishes genuine closed-loop co-evolution between trajectory generation and evaluation via adversarial optimization. EvaDrive frames trajectory planning as a multi-round adversarial game. In this game, a hierarchical generator continuously proposes candidate paths by combining autoregressive intent modeling for temporal causality with diffusion-based refinement for spatial flexibility. These proposals are then rigorously assessed by a trainable multi-objective critic that explicitly preserves diverse preference structures without collapsing them into a single scalarization bias.This adversarial interplay, guided by a Pareto frontier selection mechanism, enables iterative multi-round refinement, effectively escaping local optima while preserving trajectory diversity.Extensive experiments on NAVSIM and Bench2Drive benchmarks demonstrate SOTA performance, achieving 94.9 PDMS on NAVSIM v1 (surpassing DiffusionDrive by 6.8, DriveSuprim by 5.0, and TrajHF by 0.9) and 64.96 Driving Score on Bench2Drive. EvaDrive generates diverse driving styles via dynamic weighting without external preference data, introducing a closed-loop adversarial framework for human-like iterative decision-making, offering a novel scalarization-free trajectory optimization approach.