Perception-Aware Time-Optimal Planning for Quadrotor Waypoint Flight

The evolution of autonomous aerial vehicles has seen significant progress from classical control algorithms to deep learning-based perception systems. Path planning methods such as A*, RRT, and their variants have achieved success in static environments but struggle with real-time dynamic scenarios. Recent advances include time-optimal control and polynomial trajectory optimization, exemplified by Mellinger’s minimum-snap and Qin’s multi-segment polynomial methods. However, these approaches often neglect perception constraints, leading to visual information degradation during high-speed maneuvers. The emerging concept of perception-aware planning aims to address this gap by integrating visual robustness into trajectory optimization, yet existing solutions are limited in handling complex environments with diverse gate geometries and dynamic constraints.

The core challenge is to enable quadrotors to fly at high speeds while maintaining reliable visual perception for localization and navigation. Traditional methods optimize for minimal time without considering visual constraints, resulting in trajectories that may cause the camera to lose targets or have poor feature visibility. Incorporating perception objectives into the nonlinear, kinodynamic planning problem introduces high complexity, especially when balancing conflicting goals of speed and visual robustness. Achieving real-time, scalable solutions that can handle complex geometries and diverse environmental conditions remains an open problem, critical for advancing autonomous flight in real-world scenarios.

This work introduces a comprehensive framework that unifies nonlinear dynamics, perception constraints, and geometric gate models into a single optimization problem. Key innovations include: 1) an information-theoretic metric based on Fisher information to quantify visual position uncertainty; 2) multi-objective optimization combining speed, view continuity, and look-ahead alignment; 3) a scalable second-order cone programming (SOCP) formulation ensuring numerical stability; 4) a novel model predictive contouring controller (MPCTC) that separates lateral and longitudinal errors for robust high-speed tracking. These innovations collectively enable a new level of perception-aware, time-optimal planning for quadrotors.

• �� Formulate full nonlinear quadrotor dynamics considering aerodynamic effects and actuation limits.
• �� Define perception objectives: position uncertainty minimization via Fisher information, continuous view constraints, and look-ahead gaze alignment.
• �� Incorporate these objectives into a unified nonlinear optimization problem, solved via SOCP for stability.
• �� Use a multi-objective weighting scheme to balance speed and perception robustness.
• �� Implement a model predictive contouring controller (MPCTC) that decouples lateral and longitudinal errors, reducing corner-cutting.
• �� Validate the approach through extensive real-world experiments on diverse race tracks, measuring success rate, speed, and tracking error.

Experiments involve real quadrotor flights on complex race tracks with diverse gate geometries, including convex polygons and polytopes. The system is tested at speeds up to 9.8 m/s, with success rate, tracking error, and robustness as key metrics. Baseline comparisons include traditional time-optimal methods without perception constraints. Ablation studies evaluate the impact of each perception objective. The experiments demonstrate the system’s ability to maintain visual target visibility, achieve high speeds, and adapt to complex environments with a success rate of 100%. Data collected includes flight speed, positional accuracy, and visual target visibility metrics, confirming the effectiveness of the integrated perception-aware optimization.

The system achieves a maximum speed of 9.8 m/s with an average positional error of 0.07 m, outperforming previous methods by significant margins. The success rate on challenging Split-S courses improves from 55% to 100%, demonstrating robustness. Incorporating perception objectives reduces visual target loss and improves localization stability. The MPCTC controller effectively manages high-speed maneuvers, with errors below 0.23 m during aggressive turns. These results validate the hypothesis that perception-aware optimization enhances both speed and visual robustness in complex environments.

This framework is directly applicable to autonomous drone racing, inspection in cluttered environments, and search-and-rescue missions requiring rapid navigation with reliable perception. It can be integrated into existing UAV control systems with modifications to perception modules and optimization pipelines. The approach also paves the way for future multi-robot coordination and adaptive perception strategies, broadening the scope of autonomous aerial systems in industry and research.

The current approach assumes prior knowledge of environment geometry, limiting adaptability to unknown or dynamic settings. Real-time computational demands, although optimized, may still challenge deployment on embedded hardware in highly dynamic scenarios. Visual perception at very high speeds can be affected by sensor latency and noise, necessitating further robustness enhancements. Extending the framework to handle dynamic obstacles and unknown environments remains an open challenge for future work.