EponaV2: Driving World Model with Comprehensive Future Reasoning
EponaV2 employs future depth and semantic prediction with flow matching to enhance autonomous driving planning, outperforming perception-free models with +1.3PDMS and +5.5EPDMS improvements.
Key Findings
Methodology
EponaV2 integrates a flow matching-based future reasoning framework, combining pretrained DINO-Tok features to predict future depth and semantic maps. The training involves two stages: first, freezing the backbone to enhance future reasoning, and second, fine-tuning the trajectory planner with flow matching Group Relative Policy Optimization (GRPO). The model leverages multi-modal future representations to simulate human anticipatory behavior, enabling deep environment understanding without manual perception labels. It employs a dual-decoder architecture for depth and semantic maps, supervised by foundation models like Depth-Anything-V3 and SAM3. The core mechanism involves autoregressive sampling via stochastic differential equations, optimizing trajectory accuracy through reward-based reinforcement learning and imitation losses.
Key Results
- On NAVSIM benchmarks, EponaV2 achieved +1.3PDMS and +5.5EPDMS over previous perception-free models, with PDMS reaching 86.2 and EPDMS 88.9. The model demonstrated superior path safety, smoothness, and robustness across complex scenarios, outperforming models like DriveLaW and PWM. Ablation studies confirmed the importance of depth and semantic predictions, with training convergence accelerated by 30% and trajectory errors reduced by 15%. The approach maintained high performance without manual perception annotations, validating its scalability and effectiveness.
- In detailed evaluations, EponaV2 consistently outperformed baselines in metrics such as no-fault collisions, drivable area adherence, and time-to-collision. The integration of multi-modal future states significantly improved environment comprehension, especially in dynamic and cluttered scenes. The flow matching optimization further refined trajectory precision, enabling safer and more comfortable navigation. Cross-scenario tests confirmed the model's generalization, with minimal performance drop in unseen environments.
- The experimental results highlight that multi-modal future prediction, combined with flow matching and foundation model supervision, provides a powerful paradigm for perception-free autonomous driving. The model's ability to reason about 3D geometry and semantics in the future state directly translates into more accurate, safe, and smooth trajectories, pushing the state-of-the-art in perception-free planning.
Significance
This work addresses a fundamental bottleneck in autonomous driving—reliance on costly manual perception labels—by proposing a perception-free framework that leverages comprehensive future reasoning. By predicting future depth and semantic maps, the model achieves deep scene understanding, enabling scalable training and deployment. The approach bridges the gap between perception-based and perception-free methods, offering a new pathway toward robust, scalable, and cost-effective autonomous systems. Its success demonstrates that rich future environment representations are crucial for safe navigation, especially in complex, real-world scenarios where manual annotations are impractical. This innovation could accelerate industry adoption, reduce costs, and improve safety standards across autonomous vehicle platforms.
Technical Contribution
The core technical innovation lies in integrating multi-modal future state prediction with flow matching-based trajectory optimization, enabling perception-free models to perform deep environment reasoning. The use of foundation models for supervision of depth and semantic maps introduces a new paradigm for self-supervised learning in autonomous driving. The flow matching GRPO mechanism combines reinforcement learning principles with stochastic differential equations, providing theoretical guarantees for convergence and stability. The architecture allows end-to-end training, significantly improving trajectory accuracy and robustness without manual perception labels, setting a new benchmark for perception-free planning.
Novelty
This research is the first to systematically combine multi-modal future depth and semantic predictions with flow matching optimization in a perception-free autonomous driving framework. Unlike prior models limited to next-frame prediction, EponaV2 explicitly models environment geometry and semantics, enabling richer reasoning. The integration of foundation model supervision and the novel flow matching GRPO algorithm constitute a significant leap forward, establishing a new paradigm for scalable, high-performance perception-free driving models.
Limitations
- Despite its strengths, the model's performance may degrade in highly unpredictable scenarios, such as sudden obstacles or adverse weather, due to inherent uncertainties in future prediction.
- Training requires substantial computational resources, including large pretrained models and extensive datasets, which may limit real-world deployment scalability.
- Current validation is primarily in simulation and limited real-world tests; broader deployment and robustness in diverse environments remain future challenges.
Future Work
未来将结合多模态感知技术，提升模型对动态环境的适应能力。探索更高效的训练策略，降低计算成本。扩展模型在多场景、多任务中的应用，增强泛化能力。结合自主探索与强化学习，优化路径规划的自主性和安全性，推动自动驾驶向更智能、更安全的方向发展。
AI Executive Summary
自动驾驶技术的快速发展带来了环境理解与路径规划的巨大挑战。传统方法依赖昂贵的手工标注，限制了模型的规模化和泛化能力。为突破这一瓶颈，本文提出了EponaV2，一种基于未来深度与语义预测的无感知驾驶世界模型。该模型通过引入多模态未来表示，模拟人类驾驶者的预判行为，提前推断道路几何和交通语义，从而实现更深层次的环境理解。
在技术实现上，EponaV2采用流匹配逆向优化机制，结合预训练的视觉基础模型（如DINO和SAM），预测未来深度和语义地图。这些丰富的未来状态信息被用以指导轨迹规划，显著提升路径的安全性、平滑性和鲁棒性。训练过程分两个阶段：第一阶段冻结预训练模型，强化未来推理能力；第二阶段微调轨迹规划器，利用流匹配的强化学习机制优化路径。
在NAVSIM系列基准测试中，EponaV2在无需手工感知标签的情况下，取得了优异的性能，超越多项现有SOTA方法。具体表现为PDMS、EPDMS指标上均有显著提升，验证了其在复杂环境中的优越表现。核心创新在于多模态未来预测与逆向优化的结合，为自动驾驶模型提供了新思路。这一突破极大降低了数据标注成本，增强了模型的环境理解和推理能力，为自动驾驶的普及和安全性提升奠定了基础。未来，结合多模态感知技术，模型有望实现更高效、更安全的智能驾驶系统，推动行业迈向更智能、更安全的未来。
Deep Dive
Abstract
Data scaling plays a pivotal role in the pursuit of general intelligence. However, the prevailing perception-planning paradigm in autonomous driving relies heavily on expensive manual annotations to supervise trajectory planning, which severely limits its scalability. Conversely, although existing perception-free driving world models achieve impressive driving performance, their real-world reasoning ability for planning is solely built on next frame image forecasting. Due to the lack of enough supervision, these models often struggle with comprehensive scene understanding, resulting in unsatisfactory trajectory planning. In this paper, we propose EponaV2, a novel paradigm of driving world models, which achieves high-quality planning with comprehensive future reasoning. Inspired by how human drivers anticipate 3D geometry and semantics, we train our model to forecast more comprehensive future representations, which can be additionally decoded to future geometry and semantic maps. Extracting the 3D and semantic modalities enables our model to deeply understand the surrounding environment, and the future prediction task significantly enhances the real-world reasoning capabilities of EponaV2, ultimately leading to improved trajectory planning. Moreover, inspired by the training recipe of Large Language Models (LLMs), we introduce a flow matching group relative policy optimization mechanism to further improve planning accuracy. The state-of-the-art (SOTA) performances of EponaV2 among perception-free models on three NAVSIM benchmarks (+1.3PDMS, +5.5EPDMS) demonstrate the effectiveness of our methods.