DriveFuture: Future-Aware Latent World Models for Autonomous Driving
DriveFuture explicitly conditions current planning on predicted future latent states, achieving 55.5 EPDMS on NAVSIM-v2 navhard, surpassing previous SOTA.
Key Findings
Methodology
DriveFuture innovatively incorporates predicted future latent states as explicit conditions for trajectory planning. During training, it predicts future latent states from current latent representations and ego actions, then refines these predictions via cross-attention against ground-truth future latents. This process creates a planning-oriented foresight embedded in the latent space. The model employs a diffusion-based trajectory decoder conditioned on these future-aware latents. During inference, the model replaces ground-truth future states with its own predictions, enabling closed-loop planning. The core modules include a latent dynamics predictor, a future alignment adapter, and a diffusion trajectory decoder, all integrated into a unified training-inference pipeline that explicitly leverages future semantics for decision guidance.
Key Results
- On NAVSIM-v2 navhard, DriveFuture achieves 55.5 EPDMS, outperforming previous methods like DrivoR (54.6) and World4Drive (34.9), demonstrating superior safety and planning accuracy.
- In NAVSIM-v2 navtest, it scores 89.9 EPDMS, surpassing DiffVLA (86.8) and DriveWorld-VLA (85.9), confirming strong generalization across diverse scenarios.
- On NAVSIM-v1 navtest, DriveFuture attains 90.7 PDMS, outperforming many end-to-end and world model-based approaches, indicating robustness in complex environments.
Significance
This work addresses a fundamental limitation in existing latent world models, which treat future states solely as prediction targets. By explicitly conditioning current decision-making on predicted future latent states, DriveFuture enhances foresight, safety, and robustness in autonomous driving. This paradigm shift from passive future prediction to active future-conditioned planning advances the theoretical understanding and practical capabilities of autonomous systems. It enables vehicles to anticipate and adapt to future scenarios proactively, reducing errors and improving safety in complex, real-world environments. The approach opens new avenues for research in future-aware decision-making and scalable autonomous driving architectures.
Technical Contribution
DriveFuture introduces a novel framework that conditions current latent representations on predicted future states, integrating a latent dynamics predictor, a future alignment adapter, and a diffusion-based trajectory decoder. The predictor forecasts future latent states conditioned on current scene and planned trajectories, refined via cross-attention with ground-truth during training. The future alignment adapter grounds these predictions by aligning them with actual future scene embeddings, bridging the train-inference gap through a latent annealing schedule. The diffusion decoder then generates multi-modal trajectories conditioned on these future-aware latents, enabling foresight-driven planning. This end-to-end, unified pipeline significantly improves the decision quality and safety of autonomous vehicles, representing a key technical advancement over prior models that lack explicit future conditioning.
Novelty
This study is the first to explicitly incorporate predicted future latent states as conditions for current trajectory planning in a unified latent world modeling framework. Unlike prior approaches that treat future states as mere prediction targets or auxiliary signals, DriveFuture uses them as structured conditions, enabling the model to incorporate future semantics directly into decision-making. The combination of a future prediction-refinement mechanism, a grounded future alignment adapter, and a diffusion-based trajectory generator constitutes a novel paradigm that enhances foresight and robustness, setting a new standard in future-aware autonomous driving models.
Limitations
- The accuracy of future latent state prediction diminishes in highly complex or noisy environments, potentially affecting planning quality. Computational costs increase due to the added modules and training complexity. The model's performance in real-world deployment remains to be validated, as current experiments are conducted in simulated benchmarks. Handling rare or unseen scenarios still poses challenges, and further robustness improvements are needed for safety-critical applications.
Future Work
Future research will explore multi-modal and probabilistic future latent modeling to better capture environmental uncertainty. Integrating multi-sensor data and reinforcement learning could further enhance robustness and adaptability. Extending the framework to real-world vehicle systems and testing in diverse traffic conditions will be crucial. Additionally, improving model interpretability and safety guarantees will be prioritized to facilitate deployment in safety-critical scenarios. These directions aim to realize fully autonomous, foresight-enabled driving systems capable of operating reliably in complex, dynamic environments.
AI Executive Summary
Deep Dive
Plain Language Accessible to non-experts
想象你在开车，但不仅仅是看前方的路，还能提前知道前面几秒会发生什么。DriveFuture就像一个拥有超能力的司机，不仅能观察当前的道路情况，还能预测未来的路况，然后用这些预测来决定下一步怎么走。它就像你在玩一款赛车游戏，能提前看到弯道和障碍物，然后提前调整方向，避免撞车。这种提前预知未来的能力，让车子变得更聪明、更安全，也能更快到达目的地。它用一种特殊的“未来地图”帮助车辆提前规划路线，让驾驶变得更顺畅、更可靠。这样一来，自动驾驶汽车就像一个有远见的司机，总能提前准备好应对各种复杂情况。
Abstract
Existing latent world models for autonomous driving have opened a promising path toward future-aware driving intelligence. However, they typically treat future latent states as prediction targets or auxiliary signals, rather than directly conditioning trajectory planning. This can entangle current and future features in latent space. In this work, we propose DriveFuture, a future-aware latent world modeling framework for autonomous driving that explicitly learns planning-oriented foresight by conditioning the current latent state modeling process on future world states. Specifically, during training, the model first predicts future latent world states from the current latent state and ego action, and then refines the prediction against the ground-truth future latent state via cross-attention. The resulting future-aware latent serves as an explicit condition for a diffusion-based trajectory planner. During inference, DriveFuture conditions on the predicted future latent state instead of the ground-truth future state. DriveFuture achieves SOTA performance on the public NAVSIM benchmarks, reaching \textbf{55.5} EPDMS on NAVSIM-v2 {\textcolor{blue}{\textit{navhard}}}, \textbf{89.9} EPDMS on NAVSIM-v2 {\textcolor{blue}{\textit{navtest}}}, and \textbf{90.7} PDMS on NAVSIM-v1 {\textcolor{blue}{\textit{navtest}}}, respectively. These results suggest that the key to latent world modeling lies not merely in simulating future states, but more importantly in conditioning current decision-making on future states. Notably, as of April 2026, DriveFuture ranks \textbf{1st} on the \href{https://huggingface.co/spaces/AGC2025/e2e-driving-navhard}{NAVSIM-v2 {\textcolor{blue}{\textit{navhard}}}} leaderboard and achieves SOTA performance on \href{https://huggingface.co/spaces/AGC2024-P/e2e-driving-navtest}{NAVSIM-v1 {\textcolor{blue}{\textit{navtest}}}}.