DriveFine: Refining-Augmented Masked Diffusion VLA for Precise and Robust Driving
DriveFine integrates masked diffusion with self-correction, achieving robust and precise autonomous driving decisions, with PDMS reaching 91.8 and EPDMS 89.7 on benchmarks.
Key Findings
Methodology
This paper introduces DriveFine, a novel VLA framework combining masked diffusion and block-wise MoE architecture. Using a pretrained multimodal masked diffusion language model (LaViDa), the model decouples generation and refinement experts. During training, the generation expert performs masked token prediction, while the refinement expert is optimized via gradient blocking, preserving core capabilities. A hybrid reinforcement learning strategy, combining online and offline rewards, encourages exploration of the refinement expert. Extensive evaluations on NAVSIM v1, v2, and Navhard demonstrate superior performance and robustness, validating the approach.
Key Results
- On NAVSIM v1, DriveFine achieves 91.8 PDMS, surpassing previous methods by over 0.5%, and reaches 89.7 EPDMS on NAVSIM v2, outperforming diffusion-based and token-based models. In Navhard scenarios, it exceeds baseline methods with a 5.5-point EPDMS increase without additional training, indicating strong generalization.
- Ablation studies show that the block MoE and reinforcement strategies contribute approximately 0.7-1.0 points each, confirming their effectiveness. The model maintains high performance across diverse scenarios, demonstrating robustness.
- Results indicate that the proposed architecture effectively balances efficiency, flexibility, and accuracy, setting new state-of-the-art benchmarks in autonomous driving trajectory planning.
Significance
This work addresses fundamental limitations of existing diffusion and token-based planners in autonomous driving—namely, modality misalignment, training inefficiency, and poor generalization. By introducing a flexible, self-correcting framework with decoupled experts, it enhances robustness and scalability. The approach offers a promising direction for deploying more reliable, data-efficient, and adaptable autonomous systems, with potential impacts spanning urban mobility, intelligent transportation, and safety-critical applications. It bridges the gap between high-fidelity generative modeling and practical deployment needs.
Technical Contribution
Technically, DriveFine pioneers the use of block-wise MoE to decouple generation and refinement, maintaining pretrained model capabilities while enabling targeted fine-tuning. The hybrid reinforcement learning strategy effectively guides exploration, balancing offline reward optimization with online adaptation. The integration of masked diffusion with multi-step iterative refinement provides a new paradigm for flexible, high-quality trajectory generation. These innovations collectively push the performance ceiling of token-based VLAs, opening avenues for future research in multimodal, self-correcting generative models.
Novelty
This is the first work to embed explicit refinement experts within a masked diffusion framework for autonomous driving, achieving a clear separation between generation and correction. Unlike prior models that suffer from irreversible decoding or modal misalignment, DriveFine's block MoE enables multi-round refinement without losing global consistency. Its hybrid RL further distinguishes it by actively exploring correction strategies, setting a new standard for robustness and efficiency in multimodal trajectory planning.
Limitations
- Despite improvements, the model still faces challenges in highly dynamic or sensor-noise-heavy environments, where local errors can accumulate. Its reliance on pretrained models limits adaptability to radically different scenarios without extensive fine-tuning.
- Computational complexity remains high during training, especially with multiple experts and reinforcement modules, which may hinder real-time deployment on resource-constrained platforms.
- The current framework assumes high-quality perception inputs; robustness under severe perception failures or adversarial conditions needs further investigation.
Future Work
Future directions include optimizing expert scheduling to reduce inference costs, integrating more adaptive perception modules, and extending the framework to multi-agent scenarios. Exploring unsupervised or self-supervised reinforcement signals could further enhance exploration efficiency. Additionally, scaling the architecture for larger, more diverse datasets and real-world deployment will be key to advancing autonomous driving safety and reliability.
AI Executive Summary
Autonomous driving systems have long grappled with the challenge of generating accurate, robust, and generalizable trajectory plans. Traditional deterministic planners, such as MLP-based or anchor-based classifiers, struggle with multimodal uncertainties inherent in real-world driving. Recent advances leverage generative models like diffusion-based planners and token-based approaches, each exhibiting distinct strengths and weaknesses. Diffusion models excel in capturing multimodal distributions but suffer from modality misalignment, slow training, and limited generalization. Token-based models offer flexible decoding but are hampered by irreversible token generation and error accumulation, risking trajectory failures.
This paper introduces DriveFine, an innovative framework that combines the advantages of both paradigms through a masked diffusion model integrated with a block-wise Mixture-of-Experts (MoE) architecture. The core idea is to decouple the generation and refinement processes, preserving the foundational capabilities of pretrained models while enabling targeted improvements. During training, the generation expert predicts masked tokens, while the refinement expert, activated via a block MoE, iteratively refines trajectories with gradient blocking to prevent interference. This design maintains efficiency and scalability.
A hybrid reinforcement learning strategy further enhances exploration and stability. Offline rewards guide initial refinement, while online exploration encourages the refinement expert to discover better trajectories. Extensive experiments on NAVSIM v1, v2, and Navhard datasets demonstrate that DriveFine achieves state-of-the-art results, with PDMS reaching 91.8 and EPDMS 89.7, outperforming existing models significantly. Ablation studies confirm the contributions of the block MoE and reinforcement strategies, validating their effectiveness.
The significance of this work lies in its ability to address key limitations of current generative planners, notably modal misalignment and poor generalization. By enabling multi-step, flexible trajectory refinement, DriveFine offers a more robust, scalable solution for autonomous driving. Its promising results pave the way for safer, more reliable autonomous vehicles capable of navigating complex environments. Future work will focus on reducing computational costs, enhancing robustness under adverse conditions, and extending the framework to broader applications in intelligent transportation systems.
Deep Dive
Plain Language Accessible to non-experts
想象你在厨房里做菜。传统的厨师就像用菜谱一步步照做的机器人，遇到突发情况就不知道怎么办。而现在，有个聪明的助手会不断观察厨师的操作，及时帮忙调整火候、加调料，确保菜肴完美。这就像DriveFine，它在自动驾驶中扮演这个“聪明助手”的角色。它会先画出一条大致的路线，就像厨师按照菜谱做菜，然后不断用“调味料”——修正机制——调整路线，确保车辆安全平稳地行驶。这个系统由两个“厨师”组成：一个负责“做菜”——生成轨迹，另一个负责“调味”——修正偏差。通过不断试错和学习，它变得越来越聪明，能应对各种复杂的交通环境。这样，车辆就像一个经验丰富的司机，能在各种路况下自信前行，避免碰撞或偏离路线。这个方法让自动驾驶变得更可靠、更智能，就像厨房里的顶级厨师一样，能应对任何突发状况。
Abstract
Vision-Language-Action (VLA) models for autonomous driving increasingly adopt generative planners trained with imitation learning followed by reinforcement learning. Diffusion-based planners suffer from modality alignment difficulties, low training efficiency, and limited generalization. Token-based planners are plagued by cumulative causal errors and irreversible decoding. In summary, the two dominant paradigms exhibit complementary strengths and weaknesses. In this paper, we propose DriveFine, a masked diffusion VLA model that combines flexible decoding with self-correction capabilities. In particular, we design a novel plug-and-play block-MoE, which seamlessly injects a refinement expert on top of the generation expert. By enabling explicit expert selection during inference and gradient blocking during training, the two experts are fully decoupled, preserving the foundational capabilities and generic patterns of the pretrained weights, which highlights the flexibility and extensibility of the block-MoE design. Furthermore, we design a hybrid reinforcement learning strategy that encourages effective exploration of refinement expert while maintaining training stability. Extensive experiments on NAVSIM v1, v2, and Navhard benchmarks demonstrate that DriveFine exhibits strong efficacy and robustness. The code will be released at https://github.com/MSunDYY/DriveFine.