DA-WAM: Decision-Aligned Future Latents for Driving World Models
DA-WAM introduces decision-aligned future latent representations for autonomous driving, achieving state-of-the-art performance with 93.7 PDMS on NAVSIM-v1, by coupling action-conditioned future prediction with trajectory scoring.
Key Findings
Methodology
The DA-WAM framework integrates predictive representation learning, action-conditioned future modeling, and trajectory scoring into a unified decision-making objective. It employs an online encoder based on V-JEPA with LoRA modules for continual adaptation, ensuring future latent representations evolve during planning. Each candidate trajectory is encoded into an action representation, which, through an action-conditioned predictor, generates a distinct future latent state. These states are evaluated by a factorized scorer conditioned on the predicted future, enabling explicit action-specific consequence modeling. The training process involves expert matching and hard negative sampling, where only the expert-matched candidate's future is supervised directly, while all candidates are scored and ranked. During inference, the model operates without future observations, relying solely on the online encoder, predictor, and scorer to evaluate candidates in real time, maintaining a one-to-one correspondence between each trajectory and its predicted future.
Key Results
- On NAVSIM-v1, DA-WAM achieved a PDMS of 93.7, outperforming previous state-of-the-art models such as DriveVLA-W0 and DiffusionDrive by over 1.5 points, demonstrating superior safety and decision quality in complex scenarios.
- Ablation studies showed that removing the future latent prediction or the hard-negative supervision resulted in performance drops of approximately 2-3 points, confirming their critical roles in model robustness.
- Across various traffic scenarios, DA-WAM consistently reduced collision and lane departure rates, indicating enhanced safety margins and decision reliability, especially at decision boundaries.
Significance
This work addresses a fundamental gap in autonomous driving: the integration of future scene prediction with decision-making. By ensuring each candidate trajectory has a dedicated, action-conditioned future latent, DA-WAM enhances the interpretability and safety of trajectory evaluation. It bridges the gap between predictive modeling and planning, enabling more precise and reliable decision-making in complex, dynamic environments. The approach advances the state of the art in world models, offering a scalable, end-to-end trainable solution that aligns future scene understanding directly with trajectory scoring. This has broad implications for deploying safer, more efficient autonomous vehicles and could influence future research directions in decision-aligned scene modeling and planning.
Technical Contribution
DA-WAM's key technical innovation lies in the continuous, joint optimization of future latent representations during planning, facilitated by an online encoder with a momentum target. The explicit generation of a distinct future latent for each candidate trajectory via an action-conditioned predictor addresses the action-specific consequence modeling gap. The factorized scorer, conditioned on these future latents, enables fine-grained trajectory evaluation aligned with safety and performance metrics. The training strategy combines expert matching with safety-critical hard negatives, providing robust supervision for boundary decision-making. This integrated, end-to-end framework surpasses previous staged or frozen representation approaches, offering a novel mechanism for decision-aligned scene understanding in autonomous driving.
Novelty
This study is the first to explicitly couple action-conditioned future latent prediction with per-trajectory scoring in an end-to-end manner, ensuring a one-to-one correspondence between each candidate and its predicted future. Unlike prior models that share or pool future states across candidates, DA-WAM generates distinct future latents for each trajectory, directly influencing the scoring process. This innovation effectively aligns the predictive scene understanding with decision-making, addressing a longstanding challenge in autonomous driving research. The integration of continual predictive supervision during planning, combined with expert and hard-negative supervision, sets a new standard for decision-aligned world modeling.
Limitations
- Despite its advances, DA-WAM's generalization to unseen or highly dynamic environments remains limited, partly due to reliance on offline datasets and scene priors. Extreme weather or rare traffic scenarios may challenge its robustness.
- The computational overhead associated with generating and evaluating multiple candidate futures can impact real-time deployment, especially in scenarios with many trajectories.
- While hard negatives improve safety boundary discrimination, their selection depends on prior knowledge and may not cover all edge cases, necessitating further integration of multi-modal data and rule-based safety constraints.
Future Work
Future research will focus on enhancing the model's generalization to diverse and unseen environments, possibly through online learning or domain adaptation techniques. Incorporating multi-modal sensor data, such as LiDAR and high-definition maps, could further improve scene understanding and safety. Efforts to optimize the computational efficiency of candidate generation and scoring will be prioritized for real-time deployment. Additionally, integrating reinforcement learning strategies to adapt decision policies dynamically and exploring scalable methods for hard-negative mining will be key directions to push this framework closer to practical autonomous driving systems.
AI Executive Summary
Autonomous driving has long grappled with the challenge of accurately predicting how scenes will evolve under different vehicle actions, and crucially, how these predictions can inform safe and efficient decision-making. Traditional world models excel at scene prediction but often treat these predictions as isolated, separate from the planning process. This separation limits their ability to directly influence trajectory selection, especially in complex traffic scenarios where safety margins are tight.
The innovative framework DA-WAM (Decision-Aligned World Action Model) addresses this core issue by tightly coupling future scene prediction with trajectory scoring. Its key insight is to generate a distinct, action-conditioned future latent representation for each candidate trajectory, ensuring that the predicted consequences are directly aligned with the decision at hand. This approach transforms the predictive model from a passive observer into an active participant in the decision-making process.
DA-WAM employs a sophisticated architecture that leverages a pretrained Video-JEPA backbone, enhanced with Low-Rank Adaptation modules, to continuously refine scene representations during planning. Each candidate trajectory is encoded into an action representation, which, through an action-conditioned predictor, generates a unique future latent state. These states are then evaluated by a factorized scorer that considers safety, comfort, and efficiency metrics, all conditioned on the predicted future. During training, expert matching ensures supervision of the most relevant trajectories, while safety-critical hard negatives sharpen the model’s ability to distinguish risky from safe options near decision boundaries.
Experimental results on NAVSIM-v1 and NAVSIM-v2 datasets demonstrate the effectiveness of DA-WAM, with PDMS scores reaching 93.7 and 94.2 respectively—significantly outperforming prior models. Ablation studies confirm that the action-conditioned future latent and hard-negative supervision are vital for these improvements. The model's ability to generate and evaluate candidate-specific futures in real time marks a substantial step forward in autonomous driving technology, promising safer and more reliable vehicle behavior.
This work has broad implications, not only advancing the technical state of the art but also providing a practical pathway toward deploying autonomous vehicles capable of nuanced, safety-aware decision-making. By aligning future scene understanding directly with trajectory evaluation, DA-WAM paves the way for more intelligent, adaptable, and trustworthy autonomous systems. Future directions include extending the framework to multi-modal sensor fusion, improving computational efficiency, and integrating reinforcement learning to further enhance decision robustness in real-world environments.
Deep Dive
Abstract
Anticipating how scenes evolve under ego actions is fundamental to safe autonomous driving, yet the full potential of world models for decision-making remains unrealized. The critical challenge lies in ensuring that future modeling is not merely predictive, but decision-informative: the predicted future must directly shape which trajectory is selected. Existing approaches decouple future representation learning from planning optimization, or share predicted states across trajectory candidates, thereby diluting the action-specific consequences that ought to guide selection. To bridge this gap, we propose DA-WAM, a framework that unifies predictive representation learning, action-conditioned future modeling, and trajectory scoring under a single decision-making objective. DA-WAM maintains predictive supervision throughout planner optimization via an online encoder and a stable momentum target, allowing future representations to co-evolve with the driving task. An action-conditioned predictor generates a distinct future latent state per trajectory candidate, which is then evaluated by a future-latent-conditioned factorized scorer. For the expert-matched trajectory, the predicted future latent is supervised by the observed future representation, while safety-critical hard negatives provide additional supervision near planning boundaries. Extensive experiments on NAVSIM-v1 and NAVSIM-v2 demonstrate state-of-the-art performance, while ablations and diagnostic analyses validate the key components.
References (20)
ReCogDrive: A Reinforced Cognitive Framework for End-to-End Autonomous Driving
Yongkang Li, Kaixin Xiong, Xiangyu Guo et al.
AutoVLA: A Vision-Language-Action Model for End-to-End Autonomous Driving with Adaptive Reasoning and Reinforcement Fine-Tuning
Zewei Zhou, Tianhui Cai, Seth Z. Zhao et al.
DriveWorld: 4D Pre-Trained Scene Understanding via World Models for Autonomous Driving
Chen Min, Dawei Zhao, Liang Xiao et al.
Enhancing End-to-End Autonomous Driving with Latent World Model
Yingyan Li, Lue Fan, Jiawei He et al.
IDOL: Inverse-Dynamics-Guided Future Prediction for End-to-End Autonomous Driving
Chenghao Zhang, Timin Li, Dongmei Li
NAVSIM: Data-Driven Non-Reactive Autonomous Vehicle Simulation and Benchmarking
Daniel Dauner, Marcel Hallgarten, Tianyu Li et al.
PRIX: Learning to Plan From Raw Pixels for End-to-End Autonomous Driving
Maciej K. Wozniak, Lian Liu, Yixi Cai et al.
V-JEPA 2: Self-Supervised Video Models Enable Understanding, Prediction and Planning
Mahmoud Assran, Adrien Bardes, David Fan et al.
DriveX: Omni Scene Modeling for Learning Generalizable World Knowledge in Autonomous Driving
Chen Shi, Shaoshuai Shi, Kehua Sheng et al.
World4Drive: End-to-End Autonomous Driving via Intention-Aware Physical Latent World Model
Yupeng Zheng, Pengxuan Yang, Zebin Xing et al.
ARTEMIS: Autoregressive End-to-End Trajectory Planning With Mixture of Experts for Autonomous Driving
Renju Feng, Ning Xi, Duanfeng Chu et al.
Driving Into the Future: Multiview Visual Forecasting and Planning with World Model for Autonomous Driving
Yu-Quan Wang, Jiawei He, Lue Fan et al.
DrivingGPT: Unifying Driving World Modeling and Planning with Multi-Modal Autoregressive Transformers
Yuntao Chen, Yu-Quan Wang, Zhaoxiang Zhang
End-to-End Driving with Online Trajectory Evaluation via BEV World Model
Yingyan Li, Yu-Quan Wang, Yang Liu et al.
VADv2: End-to-End Vectorized Autonomous Driving via Probabilistic Planning
Shaoyu Chen, Bo Jiang, Hao Gao et al.
Generalized Trajectory Scoring for End-to-end Multimodal Planning
Zhenxin Li, Wenhao Yao, Zi Wang et al.
Planning-oriented Autonomous Driving
Yi Hu, Jiazhi Yang, Li Chen et al.
iPad: Iterative Proposal-centric End-to-End Autonomous Driving
Ke Guo, Haochen Liu, Xiaojun Wu et al.
TransFuser: Imitation With Transformer-Based Sensor Fusion for Autonomous Driving
Kashyap Chitta, Aditya Prakash, Bernhard Jaeger et al.