Dual Pose-Graph Semantic Localization for Vision-Based Autonomous Drone Racing

Autonomous drone racing has emerged as a challenging benchmark, pushing the limits of onboard sensing and computation. Competitions require drones to navigate through sequences of racing gates at high speed, often relying on a single monocular camera for perception. Under these conditions, accurate and robust localization is critical for trajectory planning and gate traversal. However, existing visual SLAM systems often struggle under these conditions due to motion blur and feature instability. Feature-based SLAM systems like ORB-SLAM3 can mitigate drift through loop closure, but they rely on stable visual features that degrade in fast motion. Visual-inertial odometry (VIO) approaches like VINS-Mono offer improved robustness through IMU fusion, but IMU data may not always be available or reliable in the extreme dynamics encountered in racing. Direct methods are sensitive to photometric changes, which may affect performance under varying lighting conditions.

The core problem in drone racing is achieving precise real-time localization under high-speed and aggressive maneuvering conditions. Existing visual SLAM systems often perform poorly under these conditions due to motion blur and feature instability. Additionally, these systems do not exploit the structured nature of racing environments. Racing tracks provide a strong semantic prior: gates are distinctive, repeated landmarks whose positions define the track layout. Incorporating gate detections into the localization pipeline can provide drift-correcting constraints analogous to loop closures. However, naively adding every gate observation as a new edge in a pose graph rapidly inflates the graph, increasing optimization time and undermining real-time performance.

The core innovations of this study include the introduction of a dual pose-graph architecture to address the limitations of existing methods in drone racing. • This architecture accumulates multiple gate observations between keyframes and optimizes them into a single refined constraint per landmark, which is then promoted to a persistent main graph, preserving the information richness of frequent detections. • By doing so, the system improves localization accuracy without increasing computational cost. • Additionally, the sensor-agnostic design of the system allows for broad applicability, enabling integration with different odometry and detection sources.

Method details: • Input: The system takes as input odometry estimates from any source (visual odometry, VIO, or other) providing relative pose constraints and detections of semantic objects, providing bearing and range measurements to landmarks. • Graph representation: The method is formulated as a factor graph composed of two types of nodes and two types of edges. • Temporary graph: Between main graph keyframes, a temporary graph accumulates detection edges at high frequency. • Main graph: The main graph maintains a compact, long-lived representation suitable for incremental optimization and loop closure. • Optimization: The optimal trajectory and landmark estimates are obtained by minimizing the sum of squared Mahalanobis distance errors over all edges.

Experimental design: The researchers validated the effectiveness of the method using the TII-RATM dataset, which provides high-resolution images, IMU data, and motion-capture ground-truth poses. The experiments followed a standard protocol, using a downsampled image stream and the built-in online calibration system for camera calibration. The researchers also validated the system on sequences collected during the A2RL drone racing competition, each consisting of two laps through 11 gates. The experiments evaluated Absolute Trajectory Error (ATE) and graph optimization time to assess the trade-off between accuracy and computational cost.

Results analysis: The experimental results show that the method reduces Absolute Trajectory Error (ATE) by 56% to 74% compared to standalone visual-inertial odometry (VIO) on the TII-RATM dataset. An ablation study indicates that the dual-graph architecture achieves 10% to 12% higher accuracy than a single-graph baseline at identical computational cost. Deployment in the A2RL competition demonstrated that the system performs real-time onboard localization during flight, reducing the drift of the odometry baseline by up to 4.2 meters. These findings highlight the potential of this method in autonomous drone racing.

Application scenarios: The method can be directly applied in autonomous drone racing to improve localization accuracy and reduce drift. Additionally, the sensor-agnostic design of the system allows for broad applicability, enabling integration with different odometry and detection sources. In the future, the method can also be extended to other robotics applications, such as autonomous driving and indoor navigation, to verify its applicability and robustness in different environments.

Limitations & outlook: Despite the method's excellent performance in drone racing, the dual-graph architecture's optimization frequency is lower due to keyframe-based integration, which may reduce correction responsiveness under very high update rates. Additionally, direct methods are sensitive to photometric changes, which may affect the system's performance under extreme lighting conditions. Future research directions include validating the sensor-agnostic capabilities of the framework and exploring integration with additional odometry and detection sources.