LoD-Loc v3: Generalized Aerial Localization in Dense Cities using Instance Silhouette Alignment

Urban visual localization has evolved from feature-based methods like SIFT and SuperPoint to 3D model matching approaches such as SfM and textured meshes. While high-fidelity models provide accurate localization, their creation and maintenance costs are prohibitive for large-scale deployment. LoD models, standardized by CityGML, offer a scalable alternative, enabling low-detail representations focused on architectural structures. Early works like LoD-Loc aligned wireframes from high-detail models, but faced challenges in dense scenes due to semantic ambiguity. LoD-Loc v2 extended this to low-detail building silhouettes, yet still suffered from poor generalization and ambiguity in complex environments. The need for scalable, robust solutions remains critical as urban environments grow more complex.

The core challenge lies in achieving accurate, real-time localization in densely built urban scenes where semantic silhouette ambiguity and scene variability hinder performance. Existing methods struggle with generalization across different cities and environmental conditions, leading to high failure rates. The reliance on detailed 3D reconstructions is costly and impractical for widespread use. Therefore, developing a scalable, robust approach that can operate effectively with low-detail models and synthetic data is essential to enable autonomous navigation and city-scale mapping.

The paper introduces two main innovations: first, a synthetic data pipeline leveraging Unreal Engine 5 to generate over 100,000 photorealistic images with precise building instance annotations, addressing data scarcity and improving cross-scene generalization; second, a novel instance silhouette alignment framework that assigns unique IDs to buildings, extracts instance masks with SAM, and optimally matches these masks using a Dice-based cost function. This approach reduces ambiguity inherent in semantic contours, especially in dense urban scenes. The integration of synthetic data with instance-level matching constitutes a new paradigm in urban localization, enabling high accuracy and robustness.

• �� Data generation: Use UE5 and Cesium for Unreal to render photorealistic urban scenes, employing OpenSceneGraph for precise instance mask rendering aligned with camera parameters. • Model training: Fine-tune SAM for building instance segmentation on the synthetic dataset, leveraging prompt learning and LoRA for efficiency. • Pose estimation: Render the instanced LoD model at various hypothesized poses, extract instance masks, and compute the Dice similarity with query masks. • Cost function: Use confidence or area-based weights to compute a matching score (cins), optimizing pose hypotheses via coarse-to-fine search. • Optimization: Employ particle filtering with iterative refinement, guided by the matching cost, to converge on the accurate pose. • Evaluation: Test on multiple datasets, measuring success rates at predefined thresholds, validating cross-scene generalization.

The evaluation involves three datasets: UAVD4L-LoDv2, Swiss-EPFLv2, and Tokyo-LoDv3, covering diverse urban environments. Baselines include feature matching (SIFT, SuperPoint, LoFTR), feature alignment (DINOv2, RoMa), and previous LoD methods. Metrics focus on success rates within (2m, 2°), (3m, 3°), and (5m, 5°) thresholds. Training uses synthetic data with AdamW optimizer, 20 epochs, and SAM fine-tuning via LoRA. Testing involves coarse-to-fine pose optimization with 40 iterations, beam search, and angular perturbations. Results demonstrate that LoD-Loc v3 surpasses all baselines, especially in dense scenes, with success rates over 97%, validating the approach’s robustness and generalization.

LoD-Loc v3 achieves 97.6% success at (2m, 2°) on UAVD4L-LoDv2, outperforming LoD-Loc v2 by over 30%. On Swiss-EPFLv2, success reaches 58.6%, a significant improvement over prior methods. In dense urban scenes of Tokyo-LoDv3, success rates reach 39.3%, with median errors reduced to 2.29 meters and 0.27°. The model trained solely on synthetic data demonstrates excellent cross-scene generalization, validating the synthetic dataset's effectiveness. Ablation studies confirm that instance silhouette alignment significantly outperforms semantic contour methods, especially in complex dense environments.

This approach is directly applicable to autonomous urban UAV navigation, city mapping, and emergency response. It requires only synthetic data for training, reducing data collection costs. The method’s robustness in dense, cluttered environments enables deployment in real-time systems for urban planning, infrastructure monitoring, and disaster management. Future integration with multi-sensor data can further enhance performance under adverse conditions, broadening industry adoption.

Current limitations include sensitivity to extreme weather, lighting variations, and occlusions not represented in synthetic data. Computational complexity during rendering and matching stages limits real-time deployment on resource-constrained platforms. Architectural deviations in some real-world buildings from synthetic models can reduce accuracy. Further research is needed to improve efficiency, robustness, and domain adaptation for broader practical use.