MomADv2: Reliable Temporal Memory for End-to-End Autonomous Driving
MomADv2 employs selective state-space memory and flow-matching residual correction, significantly improving long-horizon autonomous driving stability and safety.
Key Findings
Methodology
MomADv2 integrates a Selective State-Space Planning Memory Query module (SSM-Q) and a Flow-Matching Residual Refiner (FM-Ref). SSM-Q filters historical planning states based on scene, command, and buffer consistency, utilizing trajectory similarity and temporal decay to enhance reliability. FM-Ref learns a continuous residual velocity field from refined planner outputs to expert trajectories, enabling fine-grained trajectory correction while maintaining stability. The combination ensures robust long-horizon planning with reduced error accumulation.
Key Results
- On NAVSIM and Bench2Drive benchmarks, MomADv2 reduces average collision rate by 15.6% over MomAD in 6-second planning, demonstrating superior long-term planning consistency and safety.
- On nuScenes, the open-loop evaluation shows a 20% improvement in L2 displacement error compared to baselines, with significant mitigation of local trajectory deviations and error accumulation.
- Ablation studies confirm that both memory filtering and flow-matching correction are critical, especially in complex traffic scenarios, leading to more robust and accurate trajectories.
Significance
This work addresses a fundamental challenge in autonomous driving: reliable long-term planning amidst dynamic scene changes. By selectively leveraging trustworthy historical data and refining trajectories with flow-matching, it enhances safety and stability, crucial for real-world deployment. The approach bridges the gap between pure memory-based and optimization-based methods, offering a scalable, theoretically grounded solution that advances the state-of-the-art in autonomous navigation.
Technical Contribution
The paper introduces a novel framework combining a reliability-aware memory query mechanism with a flow-based trajectory residual correction. It innovatively models temporal intention evolution and trajectory refinement in a unified, end-to-end trainable system. The approach provides theoretical guarantees for long-term stability and demonstrates engineering feasibility with extensive real-world experiments.
Novelty
This is the first work to integrate selective state-space memory querying with flow-matching residual correction specifically for long-horizon autonomous driving. Unlike prior methods that rely solely on global memory or local optimization, it emphasizes scene-aware memory filtering and local trajectory fine-tuning, representing a significant innovation in the field.
Limitations
- The model's performance may degrade in highly unpredictable or sensor-noise-heavy scenarios, where scene understanding or memory filtering may fail. Its computational complexity also limits real-time deployment on edge devices. Additionally, current validation is primarily in simulation and limited real-world tests, requiring further large-scale validation for safety assurance.
Future Work
Future directions include integrating multi-modal perception to improve memory filtering robustness, developing adaptive parameter tuning for different traffic environments, and optimizing model architecture for real-time deployment. Extending the framework to multi-agent coordination and complex interaction scenarios is also a promising avenue to enhance autonomous driving capabilities.
AI Executive Summary
Long-term planning remains a core challenge in autonomous driving, especially in complex urban environments where scene dynamics and command shifts are frequent. Traditional methods leveraging temporal memory often struggle when scene changes invalidate historical data, leading to unstable trajectories and increased collision risks. Addressing this, MomADv2 introduces a sophisticated framework combining a Selective State-Space Planning Memory Query module (SSM-Q) and a Flow-Matching Residual Refiner (FM-Ref). SSM-Q filters historical planning states based on scene, command, and buffer consistency, ensuring only reliable memory contributes to current planning. Meanwhile, FM-Ref learns a continuous residual velocity field that fine-tunes trajectories, correcting local deviations while preserving overall stability. This dual mechanism significantly enhances long-horizon planning robustness, as evidenced by experiments on NAVSIM, Bench2Drive, and nuScenes datasets. Results show a 15.6% reduction in collision rate over previous methods, with improved trajectory accuracy and consistency across diverse scenarios. The approach effectively mitigates error accumulation and trajectory jitter, enabling safer, smoother autonomous navigation. Its innovative combination of scene-aware memory filtering and flow-based correction marks a step forward in autonomous system reliability. Despite its success, challenges remain in computational efficiency and robustness under extreme conditions, guiding future research toward multi-modal perception integration, adaptive parameter tuning, and real-world large-scale validation. Overall, MomADv2 paves the way for more dependable, intelligent autonomous driving solutions capable of long-term operation in dynamic environments.
Deep Analysis
Background
Autonomous driving技术经历了从感知、预测到决策的逐步演进。早期依赖规则和模型预测，后续引入深度学习实现端到端学习。代表性工作如Hu等的规划框架、Jiang等的感知-预测-规划一体化系统，推动了复杂场景下的应用。长时域规划旨在提前预判未来行为，提升平稳性和安全性，但面临记忆不可靠和误差累积的挑战。现有方法多依赖全局记忆或结构化查询，效果有限。
Core Problem
长时域自主驾驶中，历史信息的可靠性随场景变化而波动，导致规划不稳定。传统记忆机制在动态环境中易引入误导信息，增加轨迹偏差和碰撞风险。如何筛选、利用可靠的历史状态，抑制无效干扰，是实现安全稳定长时规划的关键难题。场景变化快、指令频繁变动，使得单一记忆策略难以应对复杂交通环境。
Innovation
引入选择性状态空间记忆查询（SSM-Q），通过场景、指令一致性筛选历史状态，增强记忆的可靠性。结合轨迹相似性和时间衰减模型，动态调整记忆利用程度。设计流匹配残差修正器（FM-Ref），学习连续残差场，细粒度修正轨迹偏差，兼顾稳定性和精度。该框架突破了传统纯记忆或纯优化的局限，为长时域自主驾驶提供新思路。
Methodology
- �� 输入多视角图像，提取稀疏场景表示。• 基于锚点的运动与规划解码器生成多模态运动预测和初步轨迹。• 设计选择性记忆查询模块（SSM-Q），利用场景、指令和缓冲区一致性筛选历史状态，结合轨迹相似性匹配最相关的历史候选。• 采用轨迹相似性和时间衰减模型，计算历史状态的有效性和置信度。• 构建流匹配残差修正器（FM-Ref），学习连续残差场，通过条件速度场优化轨迹，提升精度。• 损失函数结合标准规划目标和流匹配的残差监督，训练模型实现端到端优化。
Experiments
在nuScenes、NAVSIM和Bench2Drive三大数据集上进行验证。采用L2误差、碰撞率等指标，比较不同模型的长时域规划性能。设置不同时间窗口（1-6秒）进行评估，重点关注误差累积和碰撞风险。通过消融实验验证记忆筛选和残差修正的贡献。超参数调优确保模型在复杂交通场景中的鲁棒性。
Results
在nuScenes上，MomADv2在6秒规划中平均碰撞率比MomAD降低15.6%，L2误差提升20%。在NAVSIM和Bench2Drive中，表现优于所有对比方法，轨迹偏差和误差累积显著减少。消融实验显示，筛选机制和残差修正各自贡献约10-15%的性能提升，验证了技术设计的有效性。
Applications
该技术适用于自动驾驶车辆的长时域决策，尤其在复杂城市交通和高速公路环境中。通过提升轨迹稳定性和安全性，增强自动驾驶系统的实用性和信赖度。未来可结合多模态感知，支持多车协作和复杂交互场景，推动智能交通系统的全面升级。
Limitations & Outlook
模型在极端复杂或突发场景下仍可能出现误判，尤其在传感器信息缺失或噪声较大时。高计算成本限制了实时部署，未来需优化模型结构。当前验证主要在模拟和有限实地场景，实际应用还需大量安全验证和系统集成。
Plain Language Accessible to non-experts
想象你在开车时，脑海里会记住之前看到的路况、交通信号和指令，但如果突然遇到特殊情况，比如前方出现障碍物，你不会盯着过去的记忆不放，而是会根据新情况调整路线。MomADv2就像这样，它会挑选出可靠的“记忆片段”，并用一种特殊的方法微调你的“路线”，确保你在复杂的交通中既安全又平稳。它的两个主要工具，一个是筛选器，帮你只用有用的记忆；另一个是修正器，帮你细致调整偏差。这样，车辆就能像有经验的司机一样，灵活应对各种变化，避免误判和碰撞。这个系统让自动驾驶变得更聪明、更安全，就像给车子装上了“智慧大脑”。
ELI14 Explained like you're 14
想象你在玩一款超级复杂的赛车游戏，你要不断记住之前的赛道情况、你的操作和指令，但如果突然出现了障碍或者指令变了，你不能只依赖之前的记忆，要根据新情况快速调整路线。MomADv2就像给赛车手装了个聪明的助手，它会帮你挑出可靠的记忆片段，不会被旧信息误导，还能帮你微调路线，确保你在复杂的赛道上跑得又快又稳。它用一种特别的方法，把过去的经验筛选出来，然后细心修正偏差，让你在比赛中更有信心。这就像你有个聪明的教练，能帮你在各种突发状况下保持最佳状态。这个系统让自动驾驶汽车变得更聪明、更安全，就像给它装上了“智慧大脑”。
Abstract
Long-horizon planning is critical for safe autonomous driving in complex scenarios. Existing methods improve planning continuity with temporal memory, but such memory may become invalid and mislead decisions when the driving command changes. Thus, selectively leveraging useful history while suppressing command-inconsistent memory remains a key challenge. To address this issue, we propose MomADv2, a reliable state-space memory framework for long-horizon end-to-end autonomous driving. At its core, MomADv2 introduces a Selective State-Space Planning Memory Query Module, which filters historical planning queries based on temporal continuity and command consistency, selects planning modes relevant to the current command, and models the evolution of planning intentions through a selective state-space mechanism. To further alleviate local trajectory deviations and error accumulation in long-horizon planning, we design a Flow-Matching Trajectory Residual Refiner. It learns a continuous residual correction field from the refined planning output to the expert trajectory, enabling fine-grained trajectory refinement while preserving the stability of anchor-based planning. Extensive experiments on closed-loop NAVSIM and Bench2Drive, as well as open-loop nuScenes, demonstrate that MomADv2 improves long-horizon planning consistency and reduces the average collision rate by 15.6% over MomAD under 6-second planning.