Blind Bipedal Stair Traversal via Sim-to-Real Reinforcement Learning
Using sim-to-real RL with environment randomization, a bipedal robot learns proprioceptive stair traversal without external perception.
Key Findings
Methodology
This work employs Proximal Policy Optimization (PPO) with environment randomization to train a control policy for bipedal locomotion. The policy, modeled by a recurrent neural network with LSTM layers, receives proprioceptive inputs—joint states, pelvis orientation, and command signals—and outputs joint PD targets. During training, stochastic variations in terrain parameters (height, width, slope) and dynamics (mass, friction) are introduced to enhance robustness. The approach requires no specialized reward modifications; the environment variability alone drives the policy to generalize to unseen stair-like terrains. The training process involves simulating diverse scenarios with multiple random seeds, then transferring the learned policy to the real robot Cassie.
Key Results
- Simulation results show that the trained policies achieve over 85% success rate in climbing and descending stairs with heights between 10-21cm and lengths of 24-30cm across various approach speeds. The policies demonstrate high robustness to terrain variations and transfer effectively to hardware, maintaining success rates above 80%. Energy efficiency, measured by cost of transport (CoT), remains competitive (~0.46) despite the added complexity. Ablation studies confirm the importance of memory (LSTM) over feedforward networks, with the former outperforming in generalization and stability.
- Compared to vision-dependent methods, the proprioceptive approach exhibits superior robustness under occlusion, poor lighting, and environmental disturbances. The policies adapt to non-uniform stairs and complex terrains, showing strong generalization. Quantitative analysis indicates that the approach reduces reliance on external sensors, simplifying hardware requirements and increasing reliability in real-world scenarios.
- Experimental validation on Cassie confirms that the policies can handle a variety of stair geometries and disturbances, including uneven surfaces, logs, and curbs. The success rate in real-world tests exceeds 80%, with smooth, stable transitions. The approach's simplicity and robustness suggest significant potential for deployment in autonomous navigation tasks where perception may be unreliable or unavailable.
Significance
This research advances the field of legged robotics by demonstrating that proprioception alone can support reliable stair traversal in human-scale robots. It addresses key limitations of vision-based systems, such as susceptibility to occlusion and lighting conditions, offering a more robust and cost-effective solution. The ability to generalize across diverse terrains without environment-specific tuning paves the way for autonomous robots operating safely in unstructured, real-world environments. This work also highlights the power of environment randomization combined with reinforcement learning to achieve high robustness, inspiring future research in perception-free control strategies. Its implications extend to rescue, service, and exploration robots, where sensor reliability is often compromised.
Technical Contribution
The core technical innovation lies in integrating environment randomization into a standard RL training pipeline, enabling the policy to generalize to unseen stair-like terrains without additional reward engineering. The use of a recurrent neural network with LSTM layers allows the policy to leverage temporal information, crucial for handling partial observability and dynamic disturbances. The approach simplifies control design by avoiding reliance on external perception modules, instead relying solely on proprioceptive feedback. The successful sim-to-real transfer demonstrates the effectiveness of combined environment randomization and robust policy architecture, opening new avenues for perception-independent locomotion control.
Novelty
This is the first demonstration of a human-scale bipedal robot reliably traversing a variety of real-world stairs using only proprioception, without any external perception or terrain models. Unlike prior work that depends heavily on vision or precise environment modeling, this approach leverages environment randomization and reinforcement learning to achieve generalization. The method’s novelty also includes training policies in simulation with randomized stair parameters, then successfully deploying them on hardware, marking a significant step forward in perception-free legged locomotion.
Limitations
- The current policies are tested primarily on static, moderate-sized stairs; performance on irregular, moving, or highly complex terrains remains unverified. The approach assumes accurate proprioceptive sensing and may struggle with sensor noise or failure.
- Transfer to different hardware platforms may require fine-tuning due to variations in actuator dynamics and control frequency. The method’s robustness in extreme environmental conditions, such as wet or icy surfaces, is yet to be validated.
- While environment randomization enhances generalization, it does not fully address the challenge of dynamic or unpredictable obstacles beyond stairs, such as moving objects or deformable terrains. Future work should explore these scenarios.
Future Work
Future research will focus on integrating limited external perception to further improve robustness and efficiency, especially in highly complex environments. Extending the method to handle dynamic obstacles, irregular stairs, and deformable terrains is a priority. Additionally, optimizing energy consumption and control latency will be pursued to enable longer operational durations. Exploring multi-modal sensory fusion and adaptive environment randomization strategies could further enhance generalization, paving the way for truly autonomous legged robots capable of navigating unstructured, unpredictable terrains.
AI Executive Summary
The quest for autonomous legged robots capable of navigating complex, unstructured environments remains a central challenge in robotics. Traditional approaches heavily rely on external perception systems such as cameras and lidar, which, despite their success in controlled settings, often falter under real-world conditions like occlusion, poor lighting, or environmental disturbances. This reliance limits the robustness and reliability of such systems, especially in critical applications like search and rescue or disaster response.
In response, recent advances in reinforcement learning (RL) have demonstrated promising results in enabling robots to learn locomotion behaviors directly from interaction data. However, most RL-based controllers still depend on rich sensory inputs, making them vulnerable to perception failures. The present work breaks new ground by showing that a bipedal robot can learn to traverse stairs reliably using only proprioceptive feedback—information about joint angles, velocities, and body orientation—without any external perception.
The key innovation lies in training the control policy within a simulated environment where stair parameters—height, width, slope—are randomly varied. This environment randomization exposes the policy to a broad distribution of terrains, fostering robust generalization. The policy itself is implemented as a recurrent neural network with LSTM layers, enabling it to remember past states and adapt to changing conditions. Training employs the PPO algorithm, which optimizes the policy to maximize locomotion stability and robustness without modifying the reward function, simplifying the process.
Experimental results show that the trained policy successfully navigates a variety of real-world stairs on the Cassie robot, achieving success rates above 80%. It maintains stable gait patterns across different approach speeds and terrain variations, demonstrating strong transferability from simulation to hardware. The approach also exhibits resilience to uneven surfaces, logs, and curbs, highlighting its robustness.
This work significantly advances autonomous locomotion by reducing dependence on perception sensors, thus increasing reliability and reducing costs. It opens new avenues for deploying legged robots in disaster zones, unstructured environments, and other scenarios where perception might be unreliable. Future directions include integrating limited external sensing, handling more complex terrains, and optimizing energy efficiency, paving the way toward truly perception-independent autonomous robots.
Deep Dive
Abstract
Accurate and precise terrain estimation is a difficult problem for robot locomotion in real-world environments. Thus, it is useful to have systems that do not depend on accurate estimation to the point of fragility. In this paper, we explore the limits of such an approach by investigating the problem of traversing stair-like terrain without any external perception or terrain models on a bipedal robot. For such blind bipedal platforms, the problem appears difficult (even for humans) due to the surprise elevation changes. Our main contribution is to show that sim-to-real reinforcement learning (RL) can achieve robust locomotion over stair-like terrain on the bipedal robot Cassie using only proprioceptive feedback. Importantly, this only requires modifying an existing flat-terrain training RL framework to include stair-like terrain randomization, without any changes in reward function. To our knowledge, this is the first controller for a bipedal, human-scale robot capable of reliably traversing a variety of real-world stairs and other stair-like disturbances using only proprioception.