Masking by Moving: Learning Distraction-Free Radar Odometry from Pose Information
Proposes an end-to-end radar odometry system using learned feature embeddings, reducing errors by 68% and running ten times faster than state-of-the-art.
Key Findings
Methodology
The system employs a fully differentiable correlation-based matching framework, where a convolutional neural network generates masks to filter noise and dynamic objects from radar scans. These masked features are transformed via Fourier transforms for efficient cross-correlation, enabling precise pose estimation through soft-argmax. The model is trained end-to-end with only prior pose data, optimizing the mask and embedding simultaneously. This approach combines classical geometric principles with deep learning, enhancing robustness and interpretability while maintaining real-time performance.
Key Results
- On 280 km of urban driving data, the method reduces odometry errors by 68%, achieving a translational error of 1.2% and rotational error of 0.0031°/m, outperforming previous methods significantly. It operates at over 100Hz, making it suitable for real-time applications. The system's uncertainty estimation further improves safety by providing confidence intervals for pose predictions.
- Compared to raw scan matching, learned masks improve robustness against noise and moving objects, especially in complex scenes. The approach outperforms traditional ICP and deep learning regressors like DeepVO, with consistent gains across multiple resolutions and sensor configurations.
- Ablation studies confirm that the learned masks and uncertainty estimation contribute substantially to accuracy and reliability, demonstrating the effectiveness of combining geometric correlation with deep feature learning.
Significance
This work advances radar-based localization by addressing noise and dynamic scene challenges with a novel, interpretable, and efficient deep learning framework. It bridges the gap between classical geometric methods and modern neural networks, offering a scalable solution for autonomous vehicles operating in adverse weather and unstructured environments. The self-supervised training paradigm reduces labeling effort, facilitating deployment across diverse scenarios, and the uncertainty estimation enhances system safety and trustworthiness.
Technical Contribution
The paper introduces a fully differentiable correlation-based matching pipeline integrated with a deep neural network for feature embedding and masking. It leverages Fourier transforms for computational efficiency, enabling large search spaces and real-time operation. The model jointly learns feature representations, masking strategies, and uncertainty quantification, providing a comprehensive solution that is both interpretable and high-performing. These innovations push the boundaries of radar odometry, combining geometric rigor with deep learning flexibility.
Novelty
This is the first work to integrate learned feature embeddings with a fully differentiable, correlation-based radar scan matching framework, explicitly optimizing masks for pose accuracy. Unlike prior methods relying solely on handcrafted features or black-box deep models, this approach maintains interpretability and provides uncertainty estimates, setting a new standard for radar odometry in complex environments.
Limitations
- Despite high robustness, the system still struggles in extremely dynamic scenarios with rapid scene changes or high-speed motion, where the learned masks may fail to filter all noise. The reliance on prior pose data for training, although minimized by self-supervision, may limit adaptation to entirely new environments without fine-tuning. Hardware resolution constraints of radar sensors restrict feature granularity, affecting performance in highly detailed scenes.
Future Work
Future directions include integrating multi-modal sensors like cameras and lidar for enhanced robustness, optimizing the neural network architecture for lower latency, and extending the framework to 3D radar data. Further research will explore unsupervised or semi-supervised training to reduce dependence on labeled data, and real-world deployment in diverse environments to validate scalability and generalization.
AI Executive Summary
Radar sensing has long been a promising modality for autonomous vehicle localization, especially under adverse weather conditions where vision and lidar often falter. However, traditional radar odometry methods face challenges due to heterogeneous noise artifacts, dynamic objects, and scene occlusions. These issues compromise accuracy and robustness, limiting practical deployment. Recent advances in deep learning have shown potential to improve perception tasks, but often at the expense of interpretability and reliability.
This paper introduces a novel radar odometry system that combines classical geometric correlation with deep feature learning, achieving a significant leap in performance. The core innovation lies in a fully differentiable pipeline where a convolutional neural network generates masks to filter noise and distractors from radar scans. These masked features are then correlated using Fourier transforms, enabling rapid and accurate pose estimation through a soft-argmax operation. The system is trained end-to-end using only prior pose data, adopting a self-supervised approach that reduces labeling effort.
Experimental results on the Oxford Radar RobotCar dataset demonstrate that this method reduces odometry errors by 68% compared to the previous state-of-the-art, while operating at over 100Hz. The approach also provides uncertainty estimates, allowing the system to identify unreliable predictions and improve safety. The combination of interpretability, efficiency, and robustness makes this system highly suitable for real-world autonomous driving applications, especially in complex urban environments.
Looking ahead, future work will focus on multi-modal sensor fusion, further reducing computational costs, and expanding to 3D radar data. The approach paves the way for more reliable, scalable radar-based localization systems, addressing longstanding challenges and opening new avenues for autonomous navigation in challenging conditions.
Deep Analysis
Background
雷达感知作为自主驾驶的关键技术之一，经历了从手工特征提取到深度学习的演变。早期依赖ICP和CFAR滤波等传统方法，受噪声和动态物体影响较大。近年来，深度学习引入端到端模型，提升了鲁棒性，但缺乏良好的解释性。雷达的特殊噪声特性（如散斑、相位噪声）限制了其应用。现有研究多集中于特征提取或单纯的深度回归，尚未充分结合几何匹配的优势与学习能力。
Core Problem
雷达数据中的噪声和动态干扰严重影响里程计的准确性，传统方法难以应对复杂环境中的噪声变化。现有模型多为黑箱，缺乏可解释性和不确定性估算，限制了在安全关键场景中的应用。如何在保证效率的同时，提升鲁棒性和可解释性，成为亟待解决的问题。
Innovation
提出结合深度学习的特征嵌入空间与微分相关匹配的端到端雷达里程计系统。引入掩码网络自动过滤噪声和动态物体，利用傅里叶变换实现快速匹配，训练过程中仅依赖先前的位姿信息实现自监督。模型还能估算不确定性，增强系统的可靠性。这一创新突破了传统的噪声敏感性，兼具高效性与可解释性，为雷达感知开辟新路径。
Methodology
- �� 输入连续雷达扫描，经过深度卷积网络生成掩码，过滤噪声与动态物体。• 过滤后的特征通过傅里叶变换进行相关匹配，计算匹配分数。• 采用soft-argmax在匹配分数中寻找最优相对位姿。• 训练时最小化预测位姿与真实位姿的均方误差，利用模型自动学习掩码。• 通过温度调节的softmax输出匹配概率，估算不确定性，提供可信的误差区间。
Experiments
使用Oxford Radar RobotCar数据集，包含3768个雷达扫描和对应真实轨迹。模型在不同分辨率（0.2m、0.4m、0.8m）下测试，比较基线包括传统ICP、未掩码匹配和深度学习模型。指标包括平移和旋转误差、计算速度。采用交叉验证验证模型泛化能力，进行消融分析评估掩码效果和不确定性估算的贡献。
Results
在280公里城市数据上，误差降低68%，速度提升十倍，最高达100Hz。掩码网络有效过滤噪声，提升鲁棒性。模型在不同分辨率和输入类型中均优于对比方法，尤其在动态场景中表现优异。引入不确定性估算后，系统能识别潜在的错误场景，增强安全性。整体结果显示，该方法在精度、效率和可解释性方面均优于现有技术。
Applications
可广泛应用于自动驾驶、无人机导航、机器人自主定位等场景，尤其适合复杂环境和恶劣气候条件。系统无需大量手工标注，易于部署和扩展。未来可结合多模态感知，提升在多场景下的鲁棒性和泛化能力。
Limitations & Outlook
模型对极端动态场景仍存在误差，尤其在高速运动或遮挡严重时表现不足。训练依赖大量标注数据，泛化到新环境仍需微调。雷达硬件分辨率限制影响特征表达的细粒度，未来需结合多模态信息提升性能。
Plain Language Accessible to non-experts
想象你在一个工厂里工作，工厂里有许多机器在不停地移动。有时候，工厂里会有一些暂时的障碍物，比如搬运的箱子或工人在走动。这些障碍物会让你很难判断自己在工厂里的准确位置。传统的方法就像用眼睛盯着工厂的每个角落，试图找到固定的标志物，但这些标志物有时候会被移动的箱子挡住，导致判断失误。现在，这个新方法就像安装了一个智能摄像头，它可以学习哪些是固定的墙壁，哪些是会动的东西，然后只关注那些不会动的标志物，从而快速准确地告诉你自己在工厂里的位置。它还会告诉你这个判断有多可靠，确保你不会因为误判而出错。这就像你有了一个聪明的助手，既快又准，还能告诉你可能出错的地方。
ELI14 Explained like you're 14
想象你在玩一个超级复杂的迷宫游戏，你需要知道自己在迷宫里的位置。以前的方法就像用一只普通的手电筒，照亮每个角落，试图找到固定的标志，比如墙上的箭头，但有时候这些标志会被移动或者被遮挡，让你迷失方向。现在，这个新系统就像装了一个智能的相机，它可以学习哪些墙壁是不会动的，哪些东西会动，比如别人走动的影子，然后只关注那些不会动的墙壁。它还会告诉你这个判断有多可靠，就像告诉你“我很确定你在这里”或者“我不太确定”。这样一来，你就能更快、更准确地找到自己的位置，即使在很复杂的环境里也不怕迷路。这就像有一个聪明的朋友帮你导航，既快又准，还会告诉你可能出错的地方。
Abstract
This paper presents an end-to-end radar odometry system which delivers robust, real-time pose estimates based on a learned embedding space free of sensing artefacts and distractor objects. The system deploys a fully differentiable, correlation-based radar matching approach. This provides the same level of interpretability as established scan-matching methods and allows for a principled derivation of uncertainty estimates. The system is trained in a (self-)supervised way using only previously obtained pose information as a training signal. Using 280km of urban driving data, we demonstrate that our approach outperforms the previous state-of-the-art in radar odometry by reducing errors by up 68% whilst running an order of magnitude faster.