Chasing Autonomy: Dynamic Retargeting and Control Guided RL for Performant and Controllable Humanoid Running
Proposes dynamic retargeting and control-guided RL for humanoid running at 3.3m/s, enabling autonomous, controllable, high-speed locomotion.
Key Findings
Methodology
This work introduces a hybrid systems-based dynamic optimization framework using multiple-shooting methods with hard constraints to generate a library of periodic, dynamically feasible human-inspired motions. The approach models the robot's gait phases—single support and flight—and enforces periodicity via impact reset maps. The generated trajectories are embedded into a reinforcement learning environment where a control Lyapunov function (CLF) guides reward shaping, promoting stability and velocity tracking. The RL policy is trained with goal conditioning and control-guided rewards, enabling precise velocity and heading control. The trained policy is transferred zero-shot to hardware (Unitree G1), demonstrating high-speed running and obstacle avoidance in real-world outdoor environments.
Key Results
- The optimized motion library achieves accurate periodicity and dynamic feasibility across speeds from 1.2m/s to 3.6m/s, with velocity tracking errors reduced to as low as 0.157m/s at 1.4m/s, outperforming purely kinematic retargeting methods (error 0.438m/s).
- In hardware tests, the RL policy maintains stable running at speeds up to 3.3m/s, covering hundreds of meters, and successfully performs obstacle avoidance and path following, significantly surpassing prior RL-based approaches.
- Ablation studies show that combining dynamic optimization with goal-conditioned CLF rewards yields the best velocity tracking and stability, demonstrating the importance of integrated trajectory optimization and reward design.
Significance
This research addresses the longstanding challenge of enabling humanoid robots to perform fast, stable, and controllable running in complex environments. By integrating dynamic trajectory optimization with reinforcement learning, it overcomes limitations of purely kinematic or static motion libraries. The approach enhances robustness, adaptability, and real-world applicability, paving the way for autonomous humanoids in search-and-rescue, logistics, and outdoor exploration. The ability to generate and control dynamic motions at high speeds marks a significant step toward practical, autonomous humanoid robots capable of operating in unstructured environments.
Technical Contribution
The paper's key innovations include the use of multiple-shooting dynamic optimization with hybrid system modeling to generate a library of feasible, periodic motions from a single human demonstration. It introduces a control Lyapunov function-based reward shaping mechanism in RL, ensuring trajectory stability and velocity accuracy. Additionally, the integration of goal conditioning and control-guided rewards improves command tracking and maneuverability. These contributions collectively advance the state-of-the-art in dynamic humanoid locomotion, combining rigorous optimization with flexible RL-based control.
Novelty
This work is the first to combine multiple-shooting dynamic optimization with hard constraints for motion retargeting, producing a library of multi-speed, periodic motions that are dynamically feasible. It uniquely employs CLF-based reward shaping within RL to guarantee stability, addressing the challenge of high-speed, agile humanoid running. Unlike prior methods limited to single motion playback or low-speed gaits, this approach enables continuous, high-speed, and controllable locomotion in real-world environments.
Limitations
- The optimization process is computationally intensive, limiting real-time updates or adaptation in highly dynamic scenarios. Further acceleration or approximation methods are needed for real-time deployment.
- Current strategies depend heavily on high-quality human demonstrations and may struggle with noisy or limited data sources, affecting generalization.
- Hardware robustness under extreme conditions, such as rough terrains or high-speed impacts, remains to be validated. Long-term endurance and energy efficiency also require further investigation.
Future Work
Future research will focus on reducing optimization computational costs for real-time adaptation, integrating multi-source data for more robust motion libraries, and extending the framework to more complex behaviors like jumping or multi-directional running. Incorporating visual perception and autonomous planning will further enhance the robot’s environmental adaptability. Exploring end-to-end learning approaches that unify trajectory generation and control could lead to even more flexible and resilient humanoid locomotion systems.
AI Executive Summary
Achieving fast, stable, and controllable humanoid running remains a core challenge in robotics. Traditional methods relying on predefined trajectories or purely kinematic retargeting often fall short in dynamic, real-world scenarios. This study introduces a novel framework that combines dynamic trajectory optimization with reinforcement learning to generate and control high-speed locomotion. The core idea involves using a hybrid systems model to formulate a multiple-shooting optimization problem, which enforces physical and periodic constraints, producing a library of feasible motions across a range of speeds. These motions are embedded into a reinforcement learning environment where a control Lyapunov function (CLF) guides reward shaping, promoting stability and precise velocity tracking.
The training process incorporates goal conditioning and control-guided rewards, enabling the policy to adapt to commanded velocities and heading directions. Once trained, the policy is transferred directly to a hardware humanoid robot (Unitree G1), where it demonstrates the ability to run at speeds up to 3.3 meters per second, covering hundreds of meters in outdoor environments. The robot also exhibits obstacle avoidance and path following capabilities, validating the approach’s robustness and practical relevance.
Experimental results show that the optimized motion library significantly outperforms purely kinematic retargeting methods, with velocity errors reduced by nearly 50%. The integration of dynamic optimization and CLF-based reward shaping ensures stable, agile, and controllable locomotion, addressing key limitations of prior work. This research opens new avenues for autonomous humanoid robots in complex, unstructured environments, with potential applications in rescue, logistics, and outdoor exploration.
Despite these advances, challenges remain in computational efficiency, data quality, and robustness under extreme conditions. Future work aims to accelerate optimization algorithms, incorporate multi-source data for better generalization, and extend the framework to more complex behaviors. Overall, this work marks a significant step toward realizing autonomous humanoids capable of high-speed, stable, and controllable outdoor locomotion, bridging the gap between laboratory prototypes and real-world deployment.
Deep Dive
Plain Language Accessible to non-experts
想象你在操控一辆遥控车，要让它跑得快、稳，还能避开障碍。以前的方法就像用手调整方向盘，跑得不够自然，也不够快。现在，这个研究像是给遥控车装上了一个聪明的程序，它可以根据不同速度自动调整跑步轨迹，确保车子既快又稳。这个程序还可以让车子在复杂的街道上自主避障，就像人一样灵活。它通过学习和优化，变得越来越聪明，能在真实世界里跑几百米，速度最高可以到3.3米每秒！这就像给机器人装上了“聪明的运动大脑”，让它能在各种环境中跑得更快、更远、更安全。
ELI14 Explained like you're 14
想象你在玩一款赛车游戏，你可以让你的车子跑得又快又稳，还能自己避开障碍。可是，真正的机器人要做到这些就难多了。以前的方法就像是给机器人画一条线，让它沿着跑，但它跑得不够自然，也不够快。这个新方法就像是给机器人装上了一个聪明的“运动大脑”，它可以学习怎么跑得像人一样快，还能在真实的街道上跑几百米，速度最高可以到3.3米每秒！它还会根据不同的速度调整动作，确保运动既流畅又稳定。这样一来，机器人就能在复杂的环境中自主跑步，像人一样灵活自如，未来在救援、巡逻等方面都能帮上大忙！
Abstract
Humanoid robots have the promise of locomoting like humans, including fast and dynamic running. Recently, reinforcement learning (RL) controllers that can mimic human motions have become popular as they can generate very dynamic behaviors, but they are often restricted to single motion play-back which hinders their deployment in long duration and autonomous locomotion. In this paper, we present a pipeline to dynamically retarget human motions through an optimization routine with hard constraints to generate improved periodic reference libraries from a single human demonstration. We then study the effect of both the reference motion and the reward structure on the reference and commanded velocity tracking, concluding that a goal-conditioned and control-guided reward which tracks dynamically optimized human data results in the best performance. We deploy the policy on hardware, demonstrating its speed and endurance by achieving running speeds of up to 3.3 m/s on a Unitree G1 robot and traversing hundreds of meters in real-world environments. Additionally, to demonstrate the controllability of the locomotion, we use the controller in a full perception and planning autonomy stack for obstacle avoidance while running outdoors.