Blind Bipedal Stair Traversal via Sim-to-Real Reinforcement Learning
Open accessAbstract
Accurate and precise terrain estimation is a difficult problem for robot locomotion in real-world environments.Thus, it is useful to have systems that do not depend on accurate estimation to the point of fragility.In this paper, we explore the limits of such an approach by investigating the problem of traversing stair-like terrain without any external perception or terrain models on a bipedal robot.For such blind bipedal platforms, the problem appears difficult (even for humans) due to the surprise elevation changes.Our main contribution is to show that sim-to-real reinforcement learning (RL) can achieve robust locomotion over stair-like terrain on the bipedal robot Cassie using only proprioceptive feedback.Importantly, this only requires modifying an existing flat-terrain training RL framework to include stair-like terrain randomization, without any changes in reward function.To our knowledge, this is the first controller for a bipedal, human-scale robot capable of reliably traversing a variety of real-world stairs and other stair-like disturbances using only proprioception.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Siekmann et al. [61] proposes a simple terrain randomization to learn robust proprioceptive controllers for bipedal robots involved in the task of climbing and descending stairs.
“Siekmann et al. [61] proposes a simple terrain randomization to learn robust proprioceptive controllers for bipedal robots involved in the task of climbing and descending stairs.”
From Crossing the Reality Gap: A Survey on Sim-to-Real Transferability of Robot Controllers in Reinforcement Learning (2021) · cites this paper as [61]
Cites
Adam: A Method for Stochastic Optimization
Kingma, D. P., & Ba, J. (2014) · arXiv · doi:10.48550/arxiv.1412.6980 · Cited by 98,946 · see citing works
Proximal Policy Optimization Algorithms
Schulman, J., Wolski, F., Dhariwal, P., Radford, A., & Klimov, O. (2017) · arXiv · doi:10.48550/arxiv.1707.06347 · Cited by 10,625 · see citing works
Learning quadrupedal locomotion over challenging terrain
Lee, J., Hwangbo, J., Wellhausen, L., Koltun, V., & Hutter, M. (2020) · Science Robotics · doi:10.1126/scirobotics.abc5986 · Cited by 1,156 · see citing works
Memory-based control with recurrent neural networks
Heess, N., Hunt, J. J., Lillicrap, T. P., & Silver, D. (2015) · arXiv · doi:10.48550/arxiv.1512.04455 · Cited by 211 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
Salvato, E., Fenu, G., Medvet, E., & Pellegrino, F. A. (2021) · IEEE Access · doi:10.1109/access.2021.3126658 · Cited by 200 · see citing works
“[61] proposes a simple terrain randomization to learn robust proprioceptive controllers for bipedal”
BackgroundDayDreamer: World Models for Physical Robot Learning
Wu, P., Escontrela, A., Hafner, D., Goldberg, K., & Abbeel, P. (2022) · arXiv · doi:10.48550/arxiv.2206.14176 · Cited by 76 · see citing works
Li, G., Zhang, G., & Qin, C. (2021) · Sensors · doi:10.3390/s21186100 · Cited by 2 · see citing works
“In recent years, DRL has also found applications in the field of computer vision [27], robotics [28], wireless transmission [29], and other complex problems [30].”
BackgroundTowards Embodiment Scaling Laws in Robot Locomotion
Ai, B., Dai, L., Bohlinger, N., Li, D., Mu, T., Wu, Z., Fay, K., Christensen, H. I., Peters, J., & Su, H. (2025) · arXiv · doi:10.48550/arxiv.2505.05753
“The combination of scalable on-policy RL algorithms, such as Proximal Policy Optimization (PPO) [81], with fast and highly parallelizable simulators has enabled the training of powerful locomotion policies for quadruped [11, 16, 19, 64, 67, 90, 108] and humanoid robots [52, 55, 78, 86, 109].”
- Authors
- Siekmann, J., Green, K., Warila, J., Fern, A., & Hurst, J.
- Affiliations
- Oregon State University (US)
- ORCID
- Fern, Alan
- Venue
- Robotics: Science and Systems XVII
- Published
- 2021-07-12
- OpenAlex
- W3162157701
- Cited by
- 193
- References
- 4
- Citations by year
- 2026: 19 · 2025: 42 · 2024: 45 · 2023: 40 · 2022: 42 · 2021: 5
- FWCI
- 62.51
- Type
- conference paper
- Language
- en
- Open access
- bronze
- doi.org
- Locations
- Robotics: Science and Systems (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Funders
- National Science Foundation (1849343)
- Defense Advanced Research Projects Agency (W911NF-16-1-0002, -16-1-0002)
- Topics
- Physical Sciences › Engineering › Biomedical Engineering › Robotic Locomotion and Control
- Physical Sciences › Computer Science › Artificial Intelligence › Reinforcement Learning in Robotics
- Physical Sciences › Engineering › Biomedical Engineering › Prosthetics and Rehabilitation Robotics
- Keywords
- bipedalism, sim-to-real transfer, reinforcement learning, Cassie robot, proprioception, terrain estimation, legged robot control
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
