Decentralized End-to-End Multi-AAV Pursuit Using Predictive Spatio-Temporal Observation via Deep Reinforcement Learning
Proposes a PSTO-based decentralized end-to-end MARL framework enabling LiDAR-driven control for multi-UAV pursuit, achieving high success and scalability.
Key Findings
Methodology
The core is the Predictive Spatio-Temporal Observation (PSTO), a unified egocentric grid encoding obstacle geometry, adversarial intent, and teammate motion via fixed-resolution spherical projection. A dual-stream CNN processes PSTO, mapping raw LiDAR data directly to continuous control commands. The training employs MAPPO with a multi-objective reward balancing pursuit efficiency, safety, and formation. The system enables static obstacle avoidance, dynamic target interception, and team cooperation without relying on privileged ground-truth states, validated through extensive simulations and real-world outdoor tests.
Key Results
- In simulation, the method achieved 100% success rate in 2v1 pursuit, with an average capture time of 125 steps, outperforming heuristic baselines. It demonstrated robust scalability across team sizes (3, 6, 9) with success rates above 94%, without retraining. The approach maintained high performance under dense obstacles and high target speeds, validated by ablation studies showing the importance of PSTO components.
- Real outdoor experiments with quadrotor swarms relying solely on onboard LiDAR confirmed zero-shot pursuit capabilities, successfully capturing targets in unstructured environments. The system's robustness and scalability highlight its potential for practical deployment in complex scenarios.
- Comparative analysis indicates that PSTO's dense geometric and predictive encoding significantly improves environment understanding and coordination, surpassing traditional geometric or feature-based methods, especially under perception noise and occlusion.
Significance
This work advances autonomous multi-UAV pursuit by integrating perception and control end-to-end, overcoming limitations of reliance on ground-truth states and geometric abstractions. It addresses critical challenges in perception uncertainty, dynamic environment adaptation, and multi-agent coordination, providing a scalable, real-time solution. The demonstrated success in simulation and outdoor tests paves the way for practical applications in surveillance, search and rescue, and defense. Its ability to generalize across team sizes without retraining marks a significant step toward flexible, robust autonomous swarm systems, with broad implications for robotics and AI research.
Technical Contribution
The study introduces PSTO, a dense, predictive egocentric grid that aligns obstacle geometry with adversarial and teammate motion through fixed-resolution spherical projection. It integrates this representation into a dual-stream CNN, enabling end-to-end learning from raw LiDAR data. The policy leverages MAPPO with a multi-objective reward, balancing pursuit speed, safety, and formation. The approach achieves scalable multi-agent coordination without reliance on privileged information or explicit geometric features, supported by a curriculum training strategy that ensures robustness and generalization. The method's ability to transfer from simulation to real outdoor environments with onboard sensing and computation demonstrates a significant engineering breakthrough.
Novelty
This is the first framework to utilize PSTO for dense environment encoding in decentralized multi-UAV pursuit, directly mapping raw LiDAR to control commands. Unlike prior works relying on simplified states or geometric features, this approach fuses predictive intent and obstacle geometry into a unified, scalable representation, enabling end-to-end learning. Its ability to generalize across team sizes and operate solely on onboard sensing distinguishes it from existing methods, marking a new paradigm in autonomous swarm pursuit.
Limitations
- 依赖队员间的通信共享状态信息，通信中断可能影响协作效果。
- 在极端复杂或高速目标环境中，感知误差和预测偏差可能导致策略失效。
- 训练依赖大量模拟数据，实际部署时需考虑环境差异和迁移问题。
Future Work
未来将优化感知融合机制，减少对通信的依赖，增强系统鲁棒性。计划引入多模态感知和自适应预测模型，提升环境理解能力。还将扩展多目标、多任务场景，推动端到端深度强化学习在实际无人机系统中的应用，促进自主无人机在复杂环境中的广泛部署。
AI Executive Summary
Autonomous pursuit and interception by multiple UAVs have long been constrained by reliance on accurate state estimation and geometric features. Traditional methods such as differential games or Voronoi partitions depend heavily on privileged ground-truth data, making them fragile under sensing noise and occlusion. Recent advances in deep reinforcement learning have shown promise for single-agent navigation, but extending these to multi-agent pursuit remains challenging due to perception uncertainty and coordination complexity.
This paper introduces a novel decentralized end-to-end framework leveraging Predictive Spatio-Temporal Observation (PSTO), a dense egocentric grid that fuses obstacle geometry, adversarial intent, and teammate motion via fixed-resolution spherical projection. The core innovation is integrating this representation into a dual-stream CNN policy trained with MAPPO, enabling UAVs to directly map raw LiDAR data to continuous control commands. The system is designed to handle static obstacles, dynamic targets, and multi-agent cooperation without relying on privileged information or explicit geometric abstractions.
Simulation results demonstrate that the proposed method achieves 100% success in 2v1 pursuit tasks, with an average capture time of 125 steps, outperforming heuristic baselines. It scales seamlessly across team sizes (3, 6, 9) with success rates above 94%, showing excellent generalization. Outdoor experiments with quadrotor swarms relying solely on onboard LiDAR confirmed the system’s robustness and real-world applicability, marking a significant step toward practical autonomous swarm pursuit.
The approach addresses key challenges in perception uncertainty, scalability, and real-time control, offering a scalable, robust solution for complex environments. Future work will focus on reducing communication dependency, enhancing perception robustness, and extending to multi-task scenarios, paving the way for autonomous UAV swarms in diverse applications such as surveillance, search and rescue, and defense.
Deep Dive
Plain Language Accessible to non-experts
想象你和朋友在玩一场追逐游戏，你们在一个大房子里追捕藏起来的小偷。每个人都只能用望远镜看到自己面前的一小块区域，不能看到全局。你们需要根据自己看到的线索，猜测小偷可能的位置和运动方向，然后一起行动。你们不断调整位置，避免撞到家具或彼此，还要合作把小偷包围起来。这个过程就像用一张特殊的地图，把所有的线索融合在一起，帮助你们更聪明地追捕。系统通过学习如何根据有限信息做出最优决策，就像训练一只聪明的猎犬，能在复杂的环境中找到目标。
ELI14 Explained like you're 14
想象你和朋友在玩捉迷藏，但你们只能看到自己面前的一小块房间。你们要找到藏起来的小偷，但不能直接看到他，只能猜测他可能在哪个房间，然后一起行动。你们每个人都用望远镜观察环境，然后用一张特殊的地图，把所有线索放在一起，帮你们判断小偷可能的运动轨迹。你们还要避免撞到家具或彼此，保持队形。通过不断练习，你们变得越来越聪明，知道怎么合作追捕，即使环境很复杂，也能成功。这就像让无人机用感知和预测，学会在复杂环境中合作追捕目标。
Abstract
Decentralized cooperative pursuit in cluttered environments is challenging for autonomous aerial swarms, especially under partial and noisy perception. Existing methods often rely on abstracted geometric features or privileged ground-truth states, and therefore sidestep perceptual uncertainty in real-world settings. We propose a decentralized end-to-end multi-agent reinforcement learning (MARL) framework that maps raw LiDAR observations directly to continuous control commands. Central to the framework is the Predictive Spatio-Temporal Observation (PSTO), an egocentric grid representation that aligns obstacle geometry with predictive adversarial intent and teammate motion in a unified, fixed-resolution projection. Built on PSTO, a single decentralized policy enables agents to navigate static obstacles, intercept dynamic targets, and maintain cooperative encirclement. Simulations demonstrate that the proposed method achieves superior capture efficiency and competitive success rates compared to state-of-the-art learning-based approaches relying on privileged obstacle information. Furthermore, the unified policy scales seamlessly across different team sizes without retraining. Finally, fully autonomous outdoor experiments validate the framework on a quadrotor swarm relying on only onboard sensing and computing.