Framework and Multi-modal Dataset for Roadwork Zone Detection and Geo-localization

Autonomous driving increasingly depends on HD maps for precise navigation. However, static maps cannot capture transient changes like roadwork zones, leading to safety and efficiency issues. Prior research has explored camera and LiDAR-based detection, but these often operate in local coordinates and lack global contextualization. Datasets such as KITTI and nuScenes support object detection but lack detailed annotations for temporary dynamic objects and their geolocations. Recent efforts incorporate multi-sensor fusion but are limited in providing comprehensive benchmarks for dynamic environment mapping. This work addresses these gaps by creating a multi-task, multi-scenario dataset and a scalable geo-localization pipeline, advancing the state-of-the-art in dynamic environment perception.

The core challenge is to detect temporary roadwork zones accurately and determine their positions in the global coordinate system in real time. Existing methods struggle with boundary ambiguity, environmental variability, and the need for continuous, consistent tracking across frames. Achieving high-precision geo-localization of dynamic objects is crucial for updating HD maps and ensuring safe autonomous navigation. The difficulty lies in integrating multi-modal sensor data, maintaining target identity over time, and transforming local detections into accurate global positions despite sensor noise and environmental disturbances.

This work introduces three key innovations: 1) a comprehensive RZDG dataset with real and simulated multimodal data supporting multiple perception tasks; 2) an extension of AB3DMOT for robust multi-sensor, multi-object tracking that fuses detections across frames; 3) a coordinate transformation framework leveraging GPS/IMU data to convert local detections into accurate global geolocations. These innovations enable continuous, precise localization of transient roadwork zones, facilitating dynamic map updates and safer autonomous driving. The dataset’s detailed annotations and the pipeline’s flexibility represent significant advancements over prior static or single-sensor approaches.

• �� Data collection: Using a sensor-equipped vehicle (camera, LiDAR, GPS/IMU) in real-world scenarios and CARLA simulation for diverse environments.
• �� Object detection: Applying models like SMOKE, PointPillars, MVXNet to generate 3D bounding boxes.
• �� Tracking: Employing Kalman filters and Hungarian algorithm to associate detections across frames, forming tracklets.
• �� Coordinate transformation: Using GPS/IMU data to map local 3D positions to global latitude-longitude coordinates, based on Haversine formula.
• �� Fusion: Combining last-frame positions or weighted averages of tracklets for robust global localization.
• �� Evaluation: Using metrics like Precision, Recall, F1-score, and geodesic distance thresholds to assess detection and localization accuracy.

Experiments involved evaluating detection models on RZDG-Real and RZDG-Sim datasets, measuring AP scores across difficulty levels. Tracking accuracy was validated through multi-frame consistency, with the entire pipeline tested under various environmental conditions. Hyperparameters were tuned on NVIDIA RTX 4090 GPUs, with hyperparameters such as learning rates and batch sizes optimized for each detector. The system’s ability to maintain target identity and achieve sub-meter localization errors was confirmed through extensive cross-scene testing, demonstrating robustness and scalability. Ablation studies compared single-sensor versus multi-sensor fusion, confirming the latter’s superiority.

The pipeline achieved detection precision of 0.565 on real data and 0.615 on simulated data, with recall rates of 0.898 and 0.809 respectively. F1-scores were 0.597 and 0.665. Over 85% of true positives had localization errors within 1 meter, verified through Haversine distance calculations. Fusion strategies effectively reduced false positives and improved target consistency. The results validate the pipeline’s capability to accurately detect and geo-localize transient roadwork zones in complex urban environments, outperforming baseline methods.

This technology can be integrated into autonomous vehicle perception systems for real-time detection of temporary road changes, enabling dynamic map updates and safer navigation. It also benefits traffic management agencies by providing accurate, up-to-date information on construction zones. Long-term, the framework supports the development of adaptive, intelligent transportation systems capable of responding to urban dynamics, reducing accidents and congestion.

The system’s performance declines under adverse weather or low-light conditions, as sensor data quality deteriorates. Dependence on high-precision GPS/IMU makes it vulnerable to signal loss or errors, especially in urban canyons. Computational complexity limits real-time deployment in high-density scenarios, requiring further optimization. Future work should focus on enhancing robustness, reducing hardware reliance, and expanding dataset diversity to improve generalization.