CLEAR: Cognition and Latent Evaluation for Adaptive Routing in End-to-End Autonomous Driving
CLEAR integrates Drive-JEPA and single-step latent drift, achieving 93.7 PDMS at 99FPS for efficient multi-modal autonomous driving path prediction.
Key Findings
Methodology
The framework employs a frozen Drive-JEPA backbone for geometric features and fine-tuned Qwen 3.5 0.8B for scene semantic states. It replaces traditional multi-step diffusion with a single-step conditional drift in a VAE latent space, guided by a conditioning coefficient α that balances diversity and precision. An adaptive scheduler dynamically adjusts sampling parameters N and α based on scene complexity, inferred from LLM hidden states. A cross-attention scorer evaluates candidate trajectories against scene semantics. This design enables high-fidelity, multi-modal path prediction at 99 FPS without dense geometric annotations or iterative sampling, achieving a PDMS of 93.7 on NAVSIM v1.
Key Results
- On NAVSIM v1, CLEAR surpasses prior methods with a PDMS of 93.7, outperforming DriveSuprim (93.5) and Drive-JEPA (93.3). It significantly improves safety metrics such as collision avoidance and lane adherence. In the v2 protocol, EPDMS reaches 88.6, demonstrating robustness across scenarios. Ablation studies confirm that the combination of semantic-aware scoring and adaptive sampling yields the best performance, with the model effectively balancing diversity and accuracy across complex traffic scenes.
- The ablation results show that replacing heuristic scoring with LLM-driven cross-attention improves PDMS by 0.2 points, and adaptive scheduling further boosts it by 0.4 points, reaching 93.7. The model maintains multi-modality in complex intersections and high-speed scenarios, with path diversity and safety metrics outperforming baselines. The experiments validate the efficiency and effectiveness of the single-step latent drift approach, enabling real-time deployment.
- In diverse traffic environments, CLEAR demonstrates superior multi-modal path generation, especially in challenging intersections and highways. Its ability to generate scene-appropriate, safe trajectories rapidly makes it promising for real-world autonomous driving, reducing computational costs while enhancing safety and robustness.
Significance
This work addresses the bottleneck of slow, multi-step diffusion-based path planning by introducing a highly efficient single-step latent drift mechanism. It leverages deep semantic understanding from large models to guide trajectory generation, significantly reducing inference latency while maintaining multi-modality and safety. The approach bridges the gap between high-fidelity planning and real-time deployment, marking a major step toward practical autonomous vehicles. Its ability to operate without dense geometric annotations or iterative sampling reduces costs and complexity, making advanced path planning accessible for industry applications. The integration of semantic reasoning and generative modeling sets a new standard for safe, efficient autonomous navigation.
Technical Contribution
The core innovation lies in replacing the multi-step diffusion process with a single-step conditional drift in a VAE latent space, guided by scene semantics extracted from a fine-tuned LLM. The adaptive scheduler dynamically adjusts the drift coefficient α and sample count N based on scene complexity, inferred from LLM hidden states, enabling scene-aware multi-modality. The cross-attention scorer evaluates candidate trajectories against deep semantic features, replacing heuristic metrics. This combination allows high-speed, multi-modal path prediction with minimal overhead, providing theoretical guarantees of diversity and accuracy. The architecture effectively decouples geometric diversity from physical precision, offering a flexible, scalable solution for real-time autonomous planning.
Novelty
This is the first work to implement a single-step latent space drift mechanism for multi-modal trajectory prediction in autonomous driving, integrating deep semantic features from large language models. Unlike traditional diffusion models requiring multiple denoising steps, this approach achieves comparable diversity and fidelity with a single forward pass, drastically reducing latency. The scene-adaptive scheduling based on LLM hidden states introduces a new paradigm for scene-aware generative modeling, enabling dynamic trade-offs between diversity and accuracy. These innovations collectively push the boundary of real-time, high-fidelity path planning, representing a significant departure from prior multi-step diffusion and heuristic-based methods.
Limitations
- The discrete set of predefined (α, N) schemes may limit optimal parameter selection, especially in highly dynamic or unseen scenarios, potentially reducing adaptability.
- The multi-stage training pipeline, involving separate pretraining of VAE, PCA, LLM fine-tuning, and downstream modules, increases system complexity and training time.
- In extremely complex or high-speed scenarios, the model might still face challenges in path diversity and safety, requiring further robustness enhancements.
Future Work
Future efforts will focus on developing continuous or differentiable scheduling mechanisms for finer parameter control, enabling more precise scene adaptation. End-to-end joint training of all modules could simplify the pipeline and improve coherence. Additionally, integrating multi-modal perception inputs and reinforcement learning strategies may further enhance safety and robustness in diverse, real-world traffic conditions.
AI Executive Summary
Autonomous driving path planning has long been challenged by the need for real-time, multi-modal trajectory generation in complex traffic environments. Traditional diffusion models, while capable of capturing diverse behaviors, suffer from slow iterative denoising processes that hinder deployment in safety-critical systems. This paper introduces CLEAR, a novel framework that leverages a single-step latent space drift mechanism guided by deep scene semantics extracted from large language models. By replacing multi-step diffusion with a fast, conditional drift in a variational autoencoder's latent space, the system achieves high-speed generation of multiple candidate trajectories—up to 99 frames per second—without sacrificing diversity. The architecture integrates a frozen Drive-JEPA visual encoder for geometric features and a fine-tuned Qwen 3.5 0.8B model for scene understanding, which provides the semantic context for path generation and evaluation.
A key innovation is the adaptive scheduler, which dynamically adjusts the drift coefficient α and the number of samples N based on scene complexity inferred from LLM hidden states. This allows the system to allocate computational resources efficiently—generating more diverse paths in complex scenarios like intersections, and focusing on precision in simpler cases such as highway cruising. The cross-attention scorer further refines candidate selection by evaluating trajectories against deep semantic features, replacing heuristic scoring functions.
Experimental results on NAVSIM v1 demonstrate that CLEAR surpasses existing methods, achieving a PDMS of 93.7, outperforming baselines like DriveSuprim and Drive-JEPA. It maintains high efficiency, running at 99FPS, and exhibits robust multi-modal coverage across diverse traffic scenarios. The approach significantly reduces computational overhead, eliminates the need for dense geometric annotations, and demonstrates the potential for real-time, safe autonomous navigation.
Looking ahead, the authors plan to explore continuous parameter optimization and joint end-to-end training to further enhance adaptability and performance. Overall, CLEAR represents a major step forward in integrating deep semantic reasoning with generative path planning, promising safer and more efficient autonomous vehicles in the future.
Deep Analysis
Background
Autonomous driving技术经历了从模块化到端到端的演变，深度学习推动了感知、路径规划和控制的融合。早期方法依赖激光雷达和高精度地图，计算成本高且依赖繁琐的几何重建。近年来，视觉感知和深度学习模型如Drive-JEPA、MAE等逐步简化感知流程，提升了系统鲁棒性。路径规划方面，传统基于规则和优化的算法难以应对复杂多变的交通场景。扩散模型等生成方法逐渐被引入，展现出多模态路径的潜力，但多步采样带来的延迟成为瓶颈。现有研究多集中在提升生成质量，缺乏高效、实时的解决方案。本文在此背景下，结合深度语义理解和潜在空间单步漂移，提出了新颖的路径预测框架，旨在解决实时性和多模态性之间的矛盾。
Core Problem
自主驾驶在复杂交通环境中面临多模态路径生成的挑战。传统方法多依赖多步采样和几何重建，计算成本高，难以满足实时控制需求。同时，单一路径预测无法充分表达交通场景的多样性，影响安全性。如何在保证多模态覆盖的同时，提升推理速度和路径质量，成为核心难题。此外，场景理解的深度和语义信息的融入不足，限制了路径的合理性和安全性。解决这些问题，需要创新的模型架构和高效的生成机制。
Innovation
主要创新在于引入单步潜在空间漂移机制，替代传统多步去噪，显著提升推理速度。结合Drive-JEPA提供的几何特征和Qwen大模型的深层语义状态，实现多模态路径的高效生成。提出自适应调度器，根据场景复杂度动态调节漂移系数α和采样数量N，增强模型的场景适应性。利用交叉注意力评分器，将深层语义信息融入路径评估，超越传统启发式方法。这些创新共同推动了自主路径规划的效率和安全性，为端到端自主驾驶提供了新思路。
Methodology
- �� 视觉编码：采用Drive-JEPA作为视觉特征提取器，输出几何结构信息；
- �� 语义特征：利用微调的Qwen 3.5 0.8B模型提取场景语义状态，作为路径生成的条件信息；
- �� 潜在漂移：在VAE潜在空间中实现单步条件漂移，结合正向吸引和排斥机制，生成多模态路径候选；
- �� 条件调节：引入系数α调控多样性与精度，利用PCA确保路径的运动学合理性；
- �� 自适应调度：训练调度器，根据场景复杂度动态选择α和N，优化计算资源分配；
- �� 路径评分：用交叉注意力机制，将深层语义状态与候选路径匹配，评估安全性和合理性；
- �� 训练流程：多阶段预训练VAE、微调LLM、训练调度器和评分器，确保模型稳定性和性能。
Experiments
在NAVSIM数据集上进行评估，采用PDMS和EPDMS作为主要指标。训练过程中，利用130,000条轨迹预训练VAE，17,000场景的驾驶问答对微调LLM，合成对比数据训练调度器和评分器。模型在不同复杂度场景中进行多样性调节，验证α的动态调控效果。对比基线包括Drive-JEPA、DiffusionDrive等，进行消融实验验证模块贡献。测试中，模型以99FPS速度生成路径，显著优于传统多步扩散模型，PDMS达93.7，安全性指标优异。模型在复杂交叉口和高速公路场景中表现出良好的多模态覆盖和路径合理性。
Results
在NAVSIM v1中，CLEAR实现PDMS 93.7，优于DriveSuprim（93.5）和Drive-JEPA（93.3），安全指标如碰撞和车道偏离显著改善。 在v2协议中，EPDMS达88.6，表现优异。消融实验显示，交叉注意力评分器和调度器的引入分别提升PDMS 0.2和0.4点，验证其有效性。路径多样性和安全性在复杂交通场景中得到充分体现，模型在高速公路和交叉口场景中表现出强鲁棒性。整体结果表明，该方法在保持高效率的同时，显著提升路径规划的安全性和多模态表达能力。
Applications
该模型适用于自动驾驶车辆的路径规划系统，尤其在复杂交叉口和高速公路场景中。只需视觉感知和交通问答数据，无需繁琐几何重建，便可实现高效多模态路径生成。可应用于智能交通管理、自动驾驶辅助系统，提升车辆安全性和行驶效率。未来，结合感知融合和强化学习，有望实现更复杂环境下的自主决策，推动自动驾驶产业的商业化落地。
Limitations & Outlook
模型调度器采用离散预定义方案，可能无法捕捉场景中最优的连续调节参数，影响模型的细粒度调控能力。模型训练流程复杂，涉及多阶段预训练（VAE、PCA、LLM微调）和多任务优化，增加了系统复杂性和调试难度。在极端复杂或高速场景下，模型仍存在路径多样性不足或安全性不足的风险，需进一步优化场景理解和多模态融合机制。
Plain Language Accessible to non-experts
想象你在玩一款超级复杂的游戏，你需要在很多不同的场景中找到最好的行动路线。有时候场景很简单，比如在家门口走路，就只需要直走；但有时候场景很复杂，比如在繁忙的街道交叉口，你可能要考虑很多因素，比如车流、行人、交通信号灯。以前的方法就像用一把尺子测量每一条可能的路线，然后反复试验，既慢又不够智能。而这篇论文介绍的系统，就像有个非常聪明的机器人助手，它可以在一瞬间根据场景的复杂程度，快速生成多种可能的路线方案，然后帮你挑选出最安全、最合适的那一条。它还会根据交通情况自动调节生成的方案多样性，确保在简单场景中快速完成，在复杂场景中考虑更多可能。这样，车辆就能像你有个聪明的助手一样，快速做出安全的决定，避免碰撞和迷路。这个技术就像你在游戏中拥有一个超级智能的伙伴，帮你轻松应对各种挑战，既快又安全！
ELI14 Explained like you're 14
Imagine you're playing a super complex game where you need to find the best route in many different scenarios. Sometimes it's simple, like walking out of your house—just go straight. But other times, it's complicated, like crossing a busy street with cars, pedestrians, and traffic lights. Old methods are like measuring every possible route with a ruler and trying them one by one—slow and not very smart. This paper introduces a clever helper—like a super-smart robot—that can instantly generate many route options based on how complicated the scene is. It then picks the safest and best one for you. Plus, it can adjust how many options it makes depending on whether you're on a quiet road or a busy intersection. So, your car gets a smart assistant that quickly makes safe decisions, avoiding crashes and getting lost. It’s like having a gaming buddy who’s always ready to help you win, fast and safely!
Abstract
End-to-end autonomous driving models often struggle to balance multi-modal maneuver generation with real-time inference constraints. While diffusion models successfully capture diverse driving behaviors, their iterative denoising process incurs unacceptable latency for safety-critical deployment. To address this, we propose CLEAR (Cognition and Latent Evaluation for Adaptive Routing), a framework that combines ultra-fast generative planning with deep semantic reasoning. CLEAR employs Drive-JEPA as the visual encoder and replaces the multi-step denoising chain with a single-step conditional drift in a VAE latent space, introducing a conditioning coefficient to balance diversity and expert precision. Meanwhile, we fully fine-tune Qwen~3.5~0.8B on driving QA pairs to extract scene-aware hidden states. These states guide both an Adaptive Scheduler, which selects the conditioning coefficient $α$ and sample count $N$ from a discrete set of predefined schemes, and a cross-attention scorer that selects the optimal trajectory from candidates. On the NAVSIM v1 benchmark, CLEAR achieves a state-of-the-art PDMS of 93.7. Our results demonstrate that high-fidelity, multi-modal planning can be executed efficiently without dense geometric annotations or iterative sampling.