SRL-MPC: Shape-Aware Reinforcement Learned Model Predictive Control
SRL-MPC integrates high-order control barrier functions with reinforcement learning for shape-aware multi-robot navigation, achieving 100% success in dense, complex scenarios.
Key Findings
Methodology
The approach employs support function transformation to formulate high-order control barrier functions (HOCBF) based on geometric separation features (GSF), capturing complex shapes without simplification. A neural policy, trained via reinforcement learning (RL), reads GSFs and dynamically updates MPC parameters such as safety distance and tracking weights, enabling real-time adaptation to neighboring geometries. The framework decomposes the optimization into two subproblems: one updating GSFs using geometric shortest-line algorithms, and the other solving a convex HOCBF-constrained MPC with soft residual penalties. This iterative process ensures safety and adaptability. The integration preserves MPC’s safety guarantees while leveraging RL’s flexibility, resulting in robust, scalable navigation in dense, heterogeneous environments.
Key Results
- In randomized dense crowd scenarios with arbitrarily shaped robots, SRL-MPC achieved 100% success rate, zero collisions, and reduced average navigation time by 15% compared to baseline methods like ORCA and SARL. Path lengths were shorter, and the system maintained high robustness across densities from 10 to 25 robots.
- Ablation studies confirmed that geometric feature updates and RL-based parameter tuning significantly enhance safety and efficiency. The method outperformed traditional optimization and RL approaches, especially in high-density, non-convex environments.
- Experiments demonstrated real-time performance, with each robot successfully navigating complex shapes, including non-convex unions and through-traffic scenarios, validating the method’s scalability and robustness.
Significance
This work addresses long-standing challenges in shape-aware collision avoidance for dense, heterogeneous robot fleets. By combining geometric encoding with adaptive control, it overcomes limitations of simplified models and static constraints. The framework enhances the safety, scalability, and generalization of multi-robot navigation, paving the way for practical deployment in logistics, autonomous warehouses, and crowded public spaces. Its ability to handle arbitrary shapes without geometric approximation marks a significant advancement in autonomous systems, bridging the gap between theoretical safety guarantees and real-world complexity.
Technical Contribution
The paper’s core contributions include: 1) applying support function transformation to formulate high-order safety constraints for complex shapes, 2) integrating RL to adapt MPC parameters based on local geometric features, 3) employing geometric shortest-line algorithms for efficient neighbor feature extraction, and 4) designing a decomposed optimization scheme that ensures real-time solvability and safety guarantees. These innovations extend the applicability of MPC to non-convex, multi-shape environments and introduce a flexible, learning-enhanced safety layer that maintains formal guarantees while adapting to dynamic scenarios.
Novelty
This is the first work to embed shape-aware support function-based HOCBF constraints into a reinforcement learning-guided MPC framework for multi-robot navigation. Unlike prior approaches relying on circular or convex approximations, this method directly models arbitrary shapes, enabling precise safety guarantees. The combination of geometric feature encoding, high-order barrier functions, and learned parameter tuning represents a novel integration, significantly advancing the state-of-the-art in safe, adaptive multi-robot control.
Limitations
- The approach relies on convex decomposition for non-convex shapes, which may introduce approximation errors affecting safety margins. Handling highly complex or non-convex geometries directly remains challenging.
- Real-time geometric feature computation and RL parameter updates may face scalability issues as the number of neighbors or shape complexity increases, requiring further optimization.
- Training depends heavily on simulated data; transferring to real-world scenarios with perception noise, sensor errors, or unmodeled dynamics may require additional robustness measures.
Future Work
Future directions include extending the framework to directly handle non-convex shapes without decomposition, improving computational efficiency for large-scale environments, and integrating perception modules for online geometric recognition. Exploring online learning to adapt to changing environments and extending the safety guarantees to more complex dynamic scenarios are also promising avenues.
AI Executive Summary
Navigating densely populated environments with robots of arbitrary shapes remains a critical challenge in autonomous systems. Traditional collision avoidance methods often simplify shapes to circles or convex polygons, sacrificing accuracy and safety in complex scenarios. Existing optimization-based approaches provide formal safety guarantees but struggle with non-convex geometries and real-time adaptability. Reinforcement learning offers adaptability but lacks explicit safety constraints, risking collisions in dense, unpredictable environments.
This paper introduces SRL-MPC, a novel framework that combines the geometric precision of support function transformations with the adaptability of reinforcement learning. The core idea involves formulating high-order control barrier functions (HOCBF) based on geometric separation features (GSF), which encode complex shapes without approximation. A neural policy trained via PPO observes these features and dynamically adjusts MPC parameters such as safety margins and control weights, enabling real-time, shape-aware navigation.
The technical innovation lies in decomposing the optimization into a geometric feature update subproblem and a convex MPC subproblem with soft residual penalties. This iterative process ensures safety, efficiency, and adaptability even in highly dynamic, dense crowds. Extensive experiments in randomized scenarios with up to 25 arbitrarily shaped robots demonstrate the method’s robustness, achieving 100% success rate, zero collisions, and outperforming baseline methods in safety and speed.
The significance of this work extends beyond simulation. It provides a scalable, generalizable solution for autonomous navigation in complex environments, with potential applications in logistics, public safety, and autonomous vehicles. While current limitations include geometric decomposition and computational costs, ongoing research aims to address these challenges, promising a future where multi-robot systems can operate safely and efficiently in any environment.
Deep Dive
Abstract
Safe and efficient shape-aware navigation in heterogeneous crowds and robot fleets remains challenging. Traditional approaches often assume homogeneous robots, sparse workspaces, simplified geometry, offline computation, or handcrafted parameters to make the problem tractable, which limits their deployment in dense crowd scenarios. Toward this end, we propose Shape-Aware Reinforcement Learned Model Predictive Control (SRL-MPC), a method for safe, efficient, and adaptive navigation in crowds with heterogeneous shapes without geometry simplification. To encode shape-aware safety, we formulate high-order control barrier function (HOCBF) constraints from geometric separation features (GSFs) based on support function transformation. A reinforcement learning (RL) framework then learns a neural policy that reads GSFs and outputs real-time MPC parameter updates, enabling the MPC solver to adapt to neighboring crowd geometries. The key advantage of SRL-MPC is that it preserves the safety structure and generalizability of MPC while integrating the adaptability and intelligence of RL. Experiments in randomized crowd scenarios with arbitrary shaped robot fleets demonstrate the effectiveness, scalability, and robustness of SRL-MPC. The results show that SRL-MPC substantially outperforms representative baselines in safety and adaptability. Project website: https://hanruihua.github.io/srl_mpc_project/