mmPlace: Robust Place Recognition with Intermediate Frequency Signal of Low-cost Single-chip Millimeter Wave Radar
mmPlace transforms intermediate frequency signals into range-azimuth heatmaps, employs rotation-based concatenation, achieving 87.37% recall@1 in challenging scenarios.
Key Findings
Methodology
mmPlace converts single-chip radar's intermediate frequency (IF) signals into range-azimuth heatmaps, then uses a deep convolutional encoder to extract robust features. It employs a rotating platform to capture multi-angle data, concatenating heatmaps to expand the field of view (FOV). Key algorithms include distance estimation (Eq.1), angle estimation (Eq.2), and heatmap generation (Eq.3). The convolutional encoder (4 layers, with max-pooling) produces a descriptor optimized with triplet margin loss (Eq.4). Heatmap concatenation leverages cosine similarity (Eq.5) and sign consistency (Eq.6) to align frames, enhancing robustness against rotational and lateral variations.
Key Results
- On the milliSonic dataset, mmPlace achieves 87.37% recall@1, outperforming point cloud methods (~42%) and image-based methods (~49%). The system maintains high accuracy across rotations from 0° to 40° and lateral shifts up to 3 meters. Heatmap size optimization (1024×512) yields a max F1 of 0.943 with latency ~20ms, suitable for real-time applications.
- The concatenation process extends the FOV from 120° to 300°, significantly improving recognition robustness in dynamic scenarios. Ablation studies confirm that larger heatmaps and concatenation improve accuracy and stability.
- Experiments demonstrate the system's effectiveness in diverse environments, including campus, city roads, and underground parking, with consistent performance under rotational and lateral disturbances.
Significance
This work addresses the longstanding challenge of low-cost radar-based place recognition under rotational and environmental variations. By transforming raw IF signals into rich spatial heatmaps and employing rotation-aware concatenation, mmPlace offers a scalable, robust solution for autonomous navigation. Its low-cost hardware makes it feasible for widespread deployment in autonomous vehicles, robotics, and security systems, especially in adverse weather or low-light conditions where optical sensors fail. The approach bridges the gap between affordability and robustness, pushing radar perception closer to mainstream use.
Technical Contribution
The paper introduces a novel pipeline converting IF signals into range-azimuth heatmaps, combined with a deep convolutional encoder trained with triplet loss for discriminative features. The rotation-based heatmap concatenation method effectively enlarges the FOV and mitigates limited overlap issues. The integration of cosine similarity-based alignment and sign consistency ensures precise frame stitching, significantly boosting recognition robustness. The system's design balances low computational cost with high accuracy, outperforming existing point cloud and image-based methods.
Novelty
This is the first work to utilize intermediate frequency signals from low-cost single-chip radar for robust place recognition via heatmap representation. The innovative rotation-based heatmap concatenation method effectively expands the FOV and enhances robustness against orientation changes. Unlike prior methods relying on dense point clouds or optical features, this approach leverages rich spatial information from raw RF signals, offering a new paradigm for low-cost, high-robustness localization.
Limitations
- The system's performance may degrade under extremely high rotation speeds or severe signal interference, which can affect heatmap alignment accuracy.
- Accurate relative pose estimation during concatenation is crucial; errors can accumulate, impacting recognition reliability.
- Environmental factors like heavy rain or dense fog may still influence RF signal quality, limiting robustness in extreme weather.
Future Work
Future efforts will focus on integrating multi-sensor data, such as visual and LiDAR inputs, to further improve robustness. Developing end-to-end deep learning models for joint feature extraction and pose estimation could reduce reliance on precise pose alignment. Additionally, optimizing hardware and algorithms for real-time deployment in autonomous systems remains a key direction.
AI Executive Summary
Place recognition is fundamental for autonomous navigation, enabling robots and vehicles to identify previously visited locations. Traditional methods rely heavily on optical sensors like cameras and LiDAR, which perform poorly in fog, rain, or darkness. Radar, especially low-cost single-chip millimeter-wave radar, offers a promising alternative due to its insensitivity to adverse weather. However, its sparse point cloud data and limited FOV pose significant challenges for robust place recognition.
This paper introduces mmPlace, a novel system that transforms the raw intermediate frequency (IF) signals from single-chip radar into rich range-azimuth heatmaps. These heatmaps capture spatial information about surrounding objects, akin to a detailed thermal image. To address the limited FOV and orientation variations, the authors employ a rotating platform to capture multiple angles, then concatenate these heatmaps to create an expanded, more comprehensive representation.
The core technical innovation lies in the deep convolutional encoder trained with triplet margin loss, which extracts discriminative features from the heatmaps. The system uses cosine similarity and sign consistency algorithms to precisely align and stitch heatmaps across rotation cycles, effectively enlarging the field of view up to 300°. Extensive experiments on the newly collected milliSonic dataset, covering campus, urban, and underground scenarios, demonstrate that mmPlace achieves 87.37% recall@1, outperforming existing point cloud and image-based methods.
The approach offers a low-cost, robust solution for real-time place recognition, with broad implications for autonomous vehicles, robotics, and security systems. Its ability to operate reliably in challenging environments marks a significant step forward in radar-based localization. Future work will explore multi-sensor fusion, end-to-end learning, and hardware optimization to further enhance performance and deployment feasibility.
Deep Dive
Abstract
Place recognition is crucial for tasks like loop-closure detection and re-localization. Single-chip millimeter wave radar (single-chip radar in short) emerges as a low-cost sensor option for place recognition, with the advantage of insensitivity to degraded visual environments. However, it encounters two challenges. Firstly, sparse point cloud from single-chip radar leads to poor performance when using current place recognition methods, which assume much denser data. Secondly, its performance significantly declines in scenarios involving rotational and lateral variations, due to limited overlap in its field of view (FOV). We propose mmPlace, a robust place recognition system to address these challenges. Specifically, mmPlace transforms intermediate frequency (IF) signal into range azimuth heatmap and employs a spatial encoder to extract features. Additionally, to improve the performance in scenarios involving rotational and lateral variations, mmPlace employs a rotating platform and concatenates heatmaps in a rotation cycle, effectively expanding the system's FOV. We evaluate mmPlace's performance on the milliSonic dataset, which is collected on the University of Science and Technology of China (USTC) campus, the city roads surrounding the campus, and an underground parking garage. The results demonstrate that mmPlace outperforms point cloud-based methods and achieves 87.37% recall@1 in scenarios involving rotational and lateral variations.