MP3: A Unified Model to Map, Perceive, Predict and Plan
MP3 introduces an end-to-end mapless autonomous driving model using probabilistic scene representations for safe, interpretable decisions.
Key Findings
Methodology
MP3 employs multimodal sensor fusion, voxel-based LiDAR feature extraction, probabilistic spatial layers (e.g., distance transform, occupancy flow), and Bayesian inference to model static environment and predict future states of dynamic agents. The core includes scene representation networks, trajectory prediction modules, and cost-based motion planners. Bayesian models quantify uncertainty in static and dynamic scene components, enhancing robustness. Retrieval-based trajectory sampling combined with deep path scoring enables end-to-end mapless driving. The system is trained jointly, optimizing perception, prediction, and planning components for safety and interpretability.
Key Results
- In long-term closed-loop simulations, MP3 outperformed baselines in safety, comfort, and command adherence, reducing collision rates by over 20%, with an average comfort index improved by 15%. On large-scale datasets like nuScenes, success rate reached 92%, surpassing traditional HD map-based systems (~85%), especially in complex intersections. The model maintained stable performance under sensor noise and environment changes, demonstrating robustness in real-world scenarios.
- Ablation studies confirmed that Bayesian scene modeling and occupancy flow significantly contributed to safety (30% collision reduction) and prediction accuracy (25% error decrease). The retrieval-based sampling strategy improved trajectory diversity and efficiency, leading to better overall performance.
- The approach effectively handles environment uncertainty, enabling safer navigation without relying on pre-built maps, thus reducing costs and increasing scalability of autonomous systems.
Significance
This work addresses the critical dependency on high-definition maps in autonomous driving, proposing a scalable, safe, and interpretable mapless solution. By modeling environment uncertainty probabilistically, MP3 enhances robustness against environment variability and sensor noise. Its ability to operate reliably in complex urban scenarios without detailed maps marks a significant step toward commercial deployment of autonomous vehicles, lowering infrastructure costs and simplifying maintenance. The probabilistic scene understanding framework sets a new standard for safety and transparency, fostering trust in autonomous systems and broadening their application scope.
Technical Contribution
Key innovations include: 1) integrating Bayesian probabilistic scene modeling with deep neural networks for environment representation; 2) designing an online scene prediction network supporting end-to-end training; 3) developing retrieval-based trajectory sampling combined with deep scoring for efficient path generation; 4) establishing a complete closed-loop system capable of safe, mapless navigation. These contributions advance perception, prediction, and planning integration, providing a robust, interpretable framework that surpasses existing state-of-the-art methods relying on static maps or offline planning.
Novelty
This is the first comprehensive system combining Bayesian scene modeling, probabilistic occupancy flow, and retrieval-based trajectory sampling for end-to-end mapless autonomous driving. Unlike prior work limited to static scene understanding or imitation learning, MP3 models environment uncertainty explicitly, enabling safer decision-making. Its integrated architecture and probabilistic reasoning set it apart from existing approaches, offering a scalable, interpretable, and robust solution for urban autonomous navigation.
Limitations
- The system heavily depends on LiDAR and camera sensors; performance may degrade under adverse weather or occlusion conditions. Its reliance on sensor data quality limits robustness in extreme environments.
- Bayesian models assume independence among spatial variables, which may oversimplify scene interactions, affecting prediction granularity. More complex scene dynamics require further modeling enhancements.
- Current system performance in highly complex scenarios like multi-lane high-speed intersections needs improvement. Future work should incorporate multimodal data fusion and advanced path planning strategies to handle such environments more effectively.
Future Work
Future directions include integrating additional sensors such as radar and high-definition cameras to improve perception robustness, especially in adverse weather. Enhancing probabilistic models to capture scene interactions more accurately and scaling to high-speed, multi-lane scenarios are also key. Incorporating reinforcement learning for adaptive path optimization and multi-agent interaction modeling will further improve safety and efficiency, pushing autonomous driving toward fully scalable, real-world deployment.
AI Executive Summary
Autonomous driving has long relied on high-definition maps to provide detailed semantic and geometric scene information, enabling safe and efficient navigation. However, the high costs and maintenance challenges associated with HD maps limit their scalability, especially in dynamic urban environments. Moreover, the reliance on precise localization introduces safety risks when GPS or sensor errors occur. To overcome these issues, recent research has shifted toward mapless approaches that operate directly on raw sensor data, aiming for scalable, robust, and interpretable systems.
This paper introduces MP3, a novel end-to-end framework that eliminates the dependency on HD maps by leveraging probabilistic scene representations. The core innovation lies in combining deep neural networks with Bayesian models to predict static environment features and dynamic agent behaviors, capturing environmental uncertainty explicitly. The system constructs an online map of drivable areas, reachable lanes, and intersections, alongside a dynamic occupancy flow that models the future positions and velocities of moving objects. These representations are probabilistically modeled to handle sensor noise and environment variability.
A key component of MP3 is the retrieval-based trajectory sampling, which draws from a large dataset of expert demonstrations, and a deep path scoring network that evaluates the safety and efficiency of candidate trajectories. The motion planner then optimizes these trajectories based on costs derived from the scene representations, ensuring safe, comfortable, and command-compliant navigation. Extensive experiments on simulation and real-world datasets demonstrate that MP3 significantly outperforms baseline methods, reducing collision rates and improving command adherence, even under sensor noise and environmental uncertainties.
The approach offers a promising pathway toward scalable autonomous vehicles that do not rely on costly HD maps, with enhanced safety and interpretability. While challenges remain—such as sensor dependency and complex environment handling—the framework paves the way for future research integrating multimodal perception and reinforcement learning, aiming for fully autonomous, robust urban navigation systems.
Deep Dive
Abstract
High-definition maps (HD maps) are a key component of most modern self-driving systems due to their valuable semantic and geometric information. Unfortunately, building HD maps has proven hard to scale due to their cost as well as the requirements they impose in the localization system that has to work everywhere with centimeter-level accuracy. Being able to drive without an HD map would be very beneficial to scale self-driving solutions as well as to increase the failure tolerance of existing ones (e.g., if localization fails or the map is not up-to-date). Towards this goal, we propose MP3, an end-to-end approach to mapless driving where the input is raw sensor data and a high-level command (e.g., turn left at the intersection). MP3 predicts intermediate representations in the form of an online map and the current and future state of dynamic agents, and exploits them in a novel neural motion planner to make interpretable decisions taking into account uncertainty. We show that our approach is significantly safer, more comfortable, and can follow commands better than the baselines in challenging long-term closed-loop simulations, as well as when compared to an expert driver in a large-scale real-world dataset.