GeoWorldAD: Geometry World Action Model for Autonomous Driving
GeoWorldAD embeds explicit 3D geometry and future scene prediction, achieving state-of-the-art autonomous driving trajectory planning with PDMS of 91.0 on NAVSIM v1.
Key Findings
Methodology
This paper introduces GeoWorldAD, which formulates trajectory planning within ego-aligned 3D geometry space, integrating multi-scale present geometry and latent future geometry tokens. The system comprises a video geometry model based on StreamVGGT for scene reconstruction, a geometry world model predicting short-term scene evolution via depth supervision, and a geometry-oriented action model that iteratively refines trajectories. The training involves multi-stage optimization on datasets like NAVSIM v1/v2, emphasizing both geometric accuracy and planning robustness. The approach enables explicit scene understanding and anticipatory guidance, balancing safety and efficiency.
Key Results
- On NAVSIM v1 and v2, GeoWorldAD achieves PDMS scores of 91.0 and 89.1, outperforming all baselines. It maintains safety metrics with NC at 99.0 and TTC at 95.8, while significantly improving ego progress (by 3.3 points). The model demonstrates superior generalization in complex traffic scenarios, reducing overly conservative behaviors and enhancing path continuity. Ablation studies confirm that incorporating latent future geometry tokens boosts both safety and efficiency metrics, validating the importance of scene evolution prediction.
- Compared to prior methods relying solely on static geometry or pixel-based future prediction, GeoWorldAD’s explicit 3D geometric grounding and future scene anticipation provide more accurate and forward-looking path planning. The model’s multi-scale geometry fusion and iterative refinement yield a robust framework that effectively balances collision avoidance with driving progress, setting new state-of-the-art benchmarks.
- The technical novelty lies in embedding explicit ego-aligned 3D geometry and latent future tokens into a unified planning pipeline, enabling the model to reason about short-term scene evolution explicitly within a consistent geometric space. This approach advances the theoretical understanding of scene-aware trajectory planning and offers practical benefits for real-world autonomous driving systems.
Significance
This research marks a significant step forward in autonomous driving by integrating explicit 3D scene geometry and future scene prediction into trajectory planning. Unlike previous methods that relied heavily on pixel-based or implicit scene representations, GeoWorldAD’s geometric grounding enhances spatial reasoning, safety, and efficiency. The ability to anticipate scene evolution reduces overly cautious behaviors, leading to smoother and more reliable driving experiences. Its superior performance on NAVSIM benchmarks demonstrates practical viability, promising broader industry adoption. The approach also opens avenues for incorporating multi-modal data and long-term scene forecasting, potentially transforming autonomous navigation in complex environments.
Technical Contribution
The core contribution is the development of a unified framework that embeds multi-scale present geometry and latent future geometry tokens within an ego-centric 3D coordinate system. The model leverages StreamVGGT for scene reconstruction, a Q-Former style module for future scene prediction, and an iterative trajectory refinement process that combines current and predicted scene information. This design enables explicit geometric reasoning and anticipatory path optimization, surpassing prior pixel-centric or single-layer geometric methods. The training strategy and loss functions are carefully crafted to balance geometric accuracy, scene evolution prediction, and trajectory quality, providing a comprehensive solution for safe and efficient autonomous driving.
Novelty
This work is the first to embed explicit ego-aligned multi-scale geometry and latent future scene tokens directly into a trajectory planning pipeline. Unlike existing approaches that primarily focus on pixel-based future prediction or static scene modeling, GeoWorldAD explicitly models short-term scene evolution within a consistent 3D geometric space. This integration allows for more accurate and spatially explicit guidance, significantly advancing the state-of-the-art in scene-aware path planning for autonomous vehicles.
Limitations
- Despite its strengths, the model's accuracy in highly dynamic or adverse weather conditions remains limited, as geometric reconstruction and future scene prediction may degrade under sensor noise or occlusion.
- Computational complexity is relatively high due to multi-scale feature extraction and iterative refinement, posing challenges for real-time deployment on resource-constrained hardware.
- Current validation is primarily on simulation and limited real-world datasets; broader testing in diverse, real-world environments is necessary to confirm robustness and generalization.
Future Work
Future research will focus on integrating multi-modal sensor data such as LiDAR and radar to improve scene understanding robustness. Efforts will also aim to optimize the model architecture for real-time inference, reducing computational overhead. Extending scene prediction horizons and incorporating long-term planning strategies will further enhance autonomous navigation capabilities in complex, unpredictable environments.
AI Executive Summary
Autonomous driving has long grappled with balancing safety and efficiency amid complex, dynamic environments. Traditional modular systems, while interpretable, often suffer from error accumulation and limited scalability. Recent advances in vision transformer-based models have improved perception and decision-making but lack explicit geometric grounding and future scene understanding, which are critical for safe navigation.
This paper introduces GeoWorldAD, a novel trajectory planning framework that embeds explicit 3D scene geometry and anticipates short-term scene evolution. The system comprises a video geometry model based on StreamVGGT, which reconstructs multi-scale scene geometry from visual input, and a geometry world model that predicts latent future scene tokens through depth supervision. These components work together within a unified pipeline, enabling the action model to iteratively refine trajectories by integrating current scene geometry with predicted future scene changes.
The core innovation lies in representing both present and future scene states explicitly within an ego-centric 3D coordinate system. This approach allows the model to reason about scene evolution more accurately, leading to safer and more efficient path planning. Extensive experiments on NAVSIM v1 and v2 datasets demonstrate that GeoWorldAD surpasses existing methods, achieving PDMS scores of 91.0 and 89.1, respectively. The model maintains high safety metrics while significantly improving ego progress, validating the effectiveness of explicit geometric grounding combined with future scene anticipation.
Broader implications include advancing autonomous navigation in complex environments, reducing overly conservative behaviors, and enabling smoother driving experiences. The framework also opens avenues for integrating multi-modal data and long-term scene forecasting, promising a future where autonomous vehicles can better understand and adapt to their surroundings. Despite current limitations in computational cost and real-world validation, ongoing research aims to address these challenges, pushing autonomous driving closer to practical deployment in diverse scenarios.
Deep Dive
Abstract
Autonomous driving requires both safe and efficient planning decisions in dynamic 3D environments. Although recent Vision/Video-Action models learn policies directly from visual observations and scale well with advances in vision transformers and large-scale training data, they often lack explicit geometric grounding and future-aware spatial guidance, limiting their ability to balance collision avoidance and driving progress. In this work, we propose GeoWorldAD, a geometry world action model that grounds trajectory planning in ego-aligned 3D space and anticipates short-horizon scene evolution with latent future geometry tokens. Present geometry provides essential spatial constraints for safe planning, while future geometry reveals how surrounding agents and ego-centric free space may evolve, reducing overly conservative decisions without sacrificing safety. To efficiently exploit these geometric cues, GeoWorldAD progressively aggregates multi-scale present geometry and latent future geometry through iterative trajectory refinement. Experiments on NAVSIM v1 and v2 demonstrate state-of-the-art performance, highlighting the effectiveness of explicit 3D geometry grounding and future geometry world modeling for safe and efficient autonomous driving.