The Finer Points: A Systematic Comparison of Point-Cloud Extractors for Radar Odometry
This study systematically compares 13 radar point-cloud extraction algorithms for ICP odometry in autonomous driving, with K-strongest outperforming others by ~14-25%.
Key Findings
Methodology
Using two FMCW radar datasets totaling 176 km, 13 extraction algorithms—including CFAR variants, K-strongest, C18, C19, and CFEAR—were tuned and evaluated within a standardized ICP odometry pipeline. The pipeline relied solely on raw point clouds, avoiding descriptors, to ensure fair comparison. Parameters such as thresholds, window sizes, and scaling factors were optimized for each method. Performance was measured by translational and rotational errors, with statistical analysis confirming the robustness of the results across different noise environments. The study demonstrated that simpler algorithms like K-strongest achieved the best balance of accuracy and speed, outperforming more complex methods.
Key Results
- K-strongest achieved average translation errors of 1.03% and 1.02% on F1 and F2 datasets, respectively, outperforming CFAR variants by approximately 14-25%. Its runtime was around 20 ms per frame, suitable for real-time applications.
- CFAR-based algorithms showed variable performance depending on noise conditions; tuning thresholds improved accuracy but did not surpass K-strongest overall.
- Optimizing front-end radar signal processing (thresholds, filtering) yielded larger performance gains than backend matching adjustments, emphasizing the importance of initial signal refinement.
Significance
This work addresses the lack of comprehensive benchmarking for radar point-cloud extraction algorithms in autonomous driving. By providing quantitative comparisons within a common pipeline, it guides practitioners toward effective choices, reducing trial-and-error. The findings highlight that simple, well-tuned algorithms can outperform more complex ones, challenging assumptions in the field. The insights into parameter tuning and noise robustness are valuable for designing reliable radar perception systems, crucial for safe autonomous navigation in diverse environments.
Technical Contribution
The study introduces a standardized evaluation framework, combining systematic parameter tuning with a pure ICP matching pipeline. It emphasizes the critical role of front-end signal processing, demonstrating that simple algorithms like K-strongest can achieve state-of-the-art accuracy. The work also provides detailed tuning strategies and performance metrics, establishing a benchmark for future algorithm development. The approach bridges the gap between theoretical signal processing and practical odometry performance, offering a scalable methodology for real-world deployment.
Novelty
This is the first comprehensive, head-to-head comparison of multiple radar point-cloud extraction algorithms within a unified ICP odometry pipeline, focusing on real-world autonomous driving data. Unlike prior studies that evaluate algorithms in isolation or under probabilistic models, this work directly correlates extraction quality with odometry accuracy. It highlights the importance of parameter tuning and demonstrates that simplicity can outperform complexity in noisy urban environments, challenging conventional wisdom.
Limitations
- The evaluation is limited to two datasets, which may not encompass all environmental variables such as extreme weather or high dynamic scenarios. Generalization to other radar platforms remains to be validated.
- The pipeline relies solely on point-cloud data, excluding descriptor-based or deep learning methods that could further improve robustness.
- ICP performance depends heavily on point cloud quality; in cases of severe multi-path or clutter, accuracy may degrade. Future work should incorporate multi-sensor fusion and advanced filtering.
Future Work
Future directions include extending the benchmarking to off-road and high-dynamic environments, integrating deep learning-based feature extraction, and exploring multi-sensor fusion with LiDAR and cameras. Developing adaptive parameter tuning methods and real-time implementation on embedded systems will further enhance practical deployment. Additionally, investigating robustness under adverse weather and multi-path interference remains a priority.
AI Executive Summary
Radar sensing has become a vital component in autonomous vehicle navigation due to its robustness under adverse weather and lighting conditions. However, the quality of the point clouds extracted from radar signals critically influences the accuracy of odometry algorithms like ICP. Despite numerous proposed extraction methods—ranging from CFAR variants to simple strongest return algorithms—there has been no systematic comparison to guide practical choices. This study fills that gap by evaluating 13 common algorithms on two publicly available FMCW radar datasets, totaling 176 km of driving data.
The experimental setup involved tuning each algorithm’s parameters within a standardized ICP pipeline, focusing solely on raw point clouds to ensure fairness. Results revealed that the simplest method, K-strongest, consistently outperformed more complex algorithms, reducing average translation errors by around 14-25%. This counterintuitive finding underscores the importance of front-end signal processing and parameter optimization, often more impactful than backend matching strategies.
The significance of these findings extends beyond academic interest. They provide clear guidance for practitioners aiming to deploy radar-based odometry in real-world autonomous systems. The demonstrated robustness and efficiency of the K-strongest algorithm suggest that simple, well-tuned methods can meet the demanding requirements of urban navigation, paving the way for more reliable and cost-effective radar perception solutions.
While the study offers valuable insights, limitations remain. The datasets used are limited in environmental diversity, and the pipeline relies solely on point clouds without descriptor-based enhancements. Future work will explore multi-sensor fusion, deep learning integration, and robustness under challenging conditions, aiming to further elevate radar perception capabilities in autonomous driving.
Deep Dive
Abstract
A key element of many odometry pipelines using spinning frequency-modulated continuous-wave (FMCW) radar is the extraction of a point-cloud from the raw signal. This extraction greatly impacts the overall performance of point-cloud-based odometry. This paper provides a first-of-its-kind, comprehensive comparison of 13 common radar point-cloud extractors for the task of iterative closest point based odometry in autonomous driving environments. Each extractor's parameters are tuned and tested on two FMCW radar datasets using approximately 176km of data from public roads. We find that the simplest, and fastest extractor, K-strongest, is the best overall extractor, consistently outperforming the average by 13.59% and 24.94% on each dataset, respectively. Additionally, we highlight the significance of tuning an extractor and the substantial improvement in odometry accuracy that it yields.