NoRD: A Data-Efficient Vision-Language-Action Model that Drives without Reasoning
NoRD achieves efficient vision-language-action learning on Waymo and NAVSIM with <60% data, no reasoning.
Key Findings
Methodology
NoRD employs a reasoning-free vision-language-action model optimized with the Dr. GRPO algorithm, reducing data requirements. The model undergoes supervised fine-tuning on limited data followed by reinforcement learning post-training using Dr. GRPO to overcome traditional GRPO's challenges with high-variance samples.
Key Results
- On the Waymo dataset, NoRD achieved an RFS score of 7.709 using less than 12,000 samples, outperforming most reasoning-based models.
- On the NAVSIM dataset, NoRD reached a PDM score of 85.6 with less than 90,000 samples, nearing state-of-the-art models.
- Dr. GRPO optimization led to an 11.68% performance improvement on high-variance samples.
Significance
NoRD significantly reduces the reliance on large datasets and reasoning annotations in autonomous driving models, enhancing data efficiency. This research offers a more economical solution for autonomous driving, particularly in data-scarce scenarios.
Technical Contribution
NoRD introduces the Dr. GRPO algorithm to address traditional GRPO's optimization challenges with high-variance samples, achieving efficient learning without reasoning. This method maintains performance comparable to state-of-the-art models while reducing data requirements.
Novelty
NoRD is the first to achieve performance comparable to reasoning models without reasoning, effectively optimizing weak SFT policies with Dr. GRPO, overcoming limitations of traditional methods.
Limitations
- NoRD's performance in extreme driving scenarios requires further validation, especially in highly data-scarce conditions.
- The model still requires significant computational resources during training, which may limit its application in resource-constrained environments.
Future Work
Future research could explore applying NoRD in more complex driving scenarios and optimizing its performance in resource-constrained environments. Further reducing training data requirements is also a key direction.
AI Executive Summary
The NoRD model addresses the reliance on large datasets and reasoning annotations in traditional vision-language-action models by introducing the Dr. GRPO algorithm. It performs exceptionally on the Waymo and NAVSIM datasets using only 60% of the data required by traditional methods.
NoRD's core technology lies in its data efficiency and reasoning-free design. By undergoing supervised fine-tuning on limited data followed by reinforcement learning post-training with Dr. GRPO, NoRD overcomes traditional GRPO's optimization challenges with high-variance samples.
Experimental results show that NoRD achieved an RFS score of 7.709 on the Waymo dataset and a PDM score of 85.6 on the NAVSIM dataset, both nearing state-of-the-art reasoning models. This demonstrates NoRD's balance between data efficiency and performance.
Deep Analysis
Background
Vision-language-action models have made significant progress in autonomous driving, but their reliance on large datasets and reasoning annotations limits their application. Traditional methods like AutoVLA and RecogDrive require extensive data and reasoning annotations to achieve high performance.
Core Problem
Existing vision-language-action models' high demand for data and reasoning annotations leads to high costs and complex training processes, limiting their application in resource-constrained environments.
Innovation
NoRD introduces the Dr. GRPO algorithm, reducing data requirements and eliminating the need for reasoning annotations. Compared to traditional methods, NoRD achieves breakthroughs in data efficiency and performance.
Methodology
- �� Supervised fine-tuning with limited data
- �� Reinforcement learning post-training with Dr. GRPO
- �� Use of k-disc tokenization for data efficiency
- �� Testing on Waymo and NAVSIM datasets
Experiments
Experiments were conducted on the Waymo and NAVSIM datasets, with evaluation metrics including RFS and PDM scores. Baseline models included AutoVLA and RecogDrive.
Results
NoRD achieved an RFS score of 7.709 on the Waymo dataset and a PDM score of 85.6 on the NAVSIM dataset, both nearing state-of-the-art reasoning models.
Applications
NoRD can be used in autonomous driving systems, especially in data-scarce scenarios. Its efficient data usage and reasoning-free design offer advantages in resource-constrained environments.
Limitations & Outlook
NoRD's performance in extreme driving scenarios requires further validation, especially in highly data-scarce conditions. Future research could explore applying NoRD in more complex driving scenarios.
Plain Language Accessible to non-experts
Imagine a factory where traditional autonomous driving models are like production lines requiring vast raw materials and complex processes, while NoRD is like an efficient automated workshop that produces high-quality products with minimal raw materials. By reducing reliance on data and reasoning, NoRD achieves greater production efficiency.
ELI14 Explained like you're 14
Imagine you're playing a racing game, and traditional autonomous driving models are like cars that need lots of coins and complex controls, while NoRD is like a car that's cheap and easy to control, winning races with fewer coins. NoRD achieves higher efficiency and performance by reducing reliance on data and reasoning.
Glossary
Vision-Language-Action Model
A model combining vision, language, and action information for complex tasks like autonomous driving.
Used in NoRD to achieve efficient autonomous driving.
Dr. GRPO
An optimization algorithm designed to address traditional GRPO's challenges with high-variance samples.
Used in NoRD for reinforcement learning post-training.
PDM Score
A metric evaluating autonomous driving model performance, considering factors like safety and comfort.
Used to evaluate NoRD's performance on the NAVSIM dataset.
RFS Score
A metric evaluating autonomous driving model performance on the Waymo dataset.
Used to evaluate NoRD's performance on the Waymo dataset.
k-disc tokenization
A technique for improving data efficiency by segmenting and clustering trajectories to reduce data volume.
Used in NoRD to enhance data efficiency.
Open Questions Unanswered questions from this research
- 1 How to further improve NoRD's performance in extreme driving scenarios? Existing methods perform poorly in highly data-scarce conditions.
- 2 How to further optimize NoRD's computational efficiency in resource-constrained environments?
Applications
Immediate Applications
Autonomous Driving Systems
NoRD can be used in autonomous driving systems, especially in data-scarce scenarios. Its efficient data usage and reasoning-free design offer advantages in resource-constrained environments.
Long-term Vision
Intelligent Transportation Systems
NoRD's efficient design can be used to build smarter transportation systems, reducing traffic accidents and improving traffic efficiency.
Abstract
Vision-Language-Action (VLA) models are advancing autonomous driving by replacing modular pipelines with unified end-to-end architectures. However, current VLAs face two expensive requirements: (1) massive dataset collection, and (2) dense reasoning annotations. In this work, we address both challenges with NORD (No Reasoning for Driving). Compared to existing VLAs, NORD achieves competitive performance while being fine-tuned on <60% of the data and no reasoning annotations, resulting in 3x fewer tokens. We identify that standard Group Relative Policy Optimization (GRPO) fails to yield significant improvements when applied to policies trained on such small, reasoning-free datasets. We show that this limitation stems from difficulty bias, which disproportionately penalizes reward signals from scenarios that produce high-variance rollouts within GRPO. NORD overcomes this by incorporating Dr. GRPO, a recent algorithm designed to mitigate difficulty bias in LLMs. As a result, NORD achieves competitive performance on Waymo and NAVSIM with a fraction of the training data and no reasoning overhead, enabling more efficient autonomous systems. Website: https://nord-vla-ai.github.io/