Bridging the Reality Gap: Analyzing Sim-to-Real Transfer Techniques for Reinforcement Learning in Humanoid Bipedal Locomotion
The DOI matches its Crossref record; 17 of the 17 references it cites were found in a registry or an index.
Abstract
Reinforcement learning (RL) offers a promising solution for controlling humanoid robots, particularly for bipedal locomotion, by learning adaptive and flexible control strategies. However, direct RL application is hindered by time-consuming trial-and-error processes, necessitating training in simulation before real-world transfer. This introduces a reality gap that degrades performance. Although various methods have been proposed for sim-to-real transfer, they have not been validated on a consistent hardware platform, making it difficult to determine which components are key to overcoming the reality gap. In contrast, we systematically evaluate techniques to enhance RL policy robustness during sim-to-real transfer by controlling variables and comparing them on a single robot to isolate and analyze the impact of each technique. These techniques include dynamics randomization, state history usage, noise/bias/delay modeling, state selection, perturbations, and network size. We quantitatively assess the reality gap by simulating diverse conditions and conducting experiments on real hardware. Our findings provide insights into bridging the reality gap, advancing robust RL-trained humanoid robots for real-world applications.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Citations
Cites
Sim-to-Real Transfer of Robotic Control with Dynamics Randomization
Peng, X. B., Andrychowicz, M., Zaremba, W., & Abbeel, P. (2018) · 2018 IEEE International Conference on Robotics and Automation (ICRA) · doi:10.1109/icra.2018.8460528 · Cited by 1,309 · cited 1×
“Action spaces that naturally lead to smooth and feasible actions reduce the risk of violating physical constraints.”
Sim-to-Real: Learning Agile Locomotion For Quadruped Robots
Tan, J., Zhang, T., Coumans, E., Iscen, A., Bai, Y., Hafner, D., Bohez, S., & Vanhoucke, V. (2018) · Robotics: Science and Systems XIV · doi:10.15607/rss.2018.xiv.010 · Cited by 735
Sim-to-Real: Learning Agile Locomotion For Quadruped Robots
Tan, J., Zhang, T., Coumans, E., Iscen, A., Bai, Y., Hafner, D., Bohez, S., & Vanhoucke, V. (2018) · arXiv · doi:10.48550/arxiv.1804.10332 · Cited by 734
Closing the Sim-to-Real Loop: Adapting Simulation Randomization with Real World Experience
Chebotar, Y., Handa, A., Makoviychuk, V., Macklin, M., Issac, J., Ratliff, N., & Fox, D. (2019) · 2019 International Conference on Robotics and Automation (ICRA) · doi:10.1109/icra.2019.8793789 · Cited by 457
Reinforcement learning for versatile, dynamic, and robust bipedal locomotion control
Li, Z., Peng, X. B., Abbeel, P., Levine, S., Berseth, G., & Sreenath, K. (2024) · The International Journal of Robotics Research · doi:10.1177/02783649241285161 · Cited by 287
Rapid locomotion via reinforcement learning
Margolis, G. B., Yang, G., Paigwar, K., Chen, T., & Agrawal, P. (2024) · The International Journal of Robotics Research · doi:10.1177/02783649231224053 · Cited by 226
Reinforcement Learning for Robust Parameterized Locomotion Control of Bipedal Robots
Li, Z., Cheng, X., Peng, X. B., Abbeel, P., Levine, S., Berseth, G., & Sreenath, K. (2021) · 2021 IEEE International Conference on Robotics and Automation (ICRA) · doi:10.1109/icra48506.2021.9560769 · Cited by 219
Ji, G., Mun, J., Kim, H., & Hwangbo, J. (2022) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2022.3151396 · Cited by 203
Learning Locomotion Skills for Cassie: Iterative Design and Sim-to-Real
Xie, Z., Clary, P., Dao, J., Morais, P., HURST, J. W., & van de Panne, M. (2019) · Conference on Robot Learning · Cited by 195
Legged Robots that Keep on Learning: Fine-Tuning Locomotion Policies in the Real World
Smith, L., Kew, J. C., Peng, X. B., Ha, S., Tan, J., & Levine, S. (2022) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra46639.2022.9812166 · Cited by 169
Cited by
Zhang, J., Liu, J., Zong, H., Ji, P., Fang, L., Li, Y., Yang, H., & Bing Hong, X. U. (2025) · Cyborg and Bionic Systems · doi:10.34133/cbsystems.0365 · Cited by 29
“While virtual environments enable efficient training with diverse terrains and idealized models, the transition to real-world deployment is often hindered by discrepancies including sensor noise, actuator delays, and contact uncertainties [ 226 , 227 ].”
Route Optimization for UGVs: A Systematic Analysis of Applications, Algorithms and Challenges
Ponce, D. F. Y., Montalvo, W., Guamán-Gavilanes, X. A., & Echeverría-Cadena, M. D. (2025) · Applied Sciences · doi:10.3390/app15126477 · Cited by 8
“The inconsistent reporting of evaluation metrics across studies further complicates comparative benchmarking [58].”
Shahid, A. A., Narang, Y., Petrone, V., Ferrentino, E., Handa, A., Fox, D., Pavone, M., & Roveda, L. (2025) · IEEE International Conference on Automation Science and Engineering (CASE) · doi:10.1109/case58245.2025.11163870 · Cited by 9
“This suggests that a better exploration across the agents, favored by PBRL, not only leads to higher rewards, but is also one of the aspects that could lead to a successful real-world transfer [31].”
Gadekallu, T. R., Zhao, Y., Wen, Z., Bhattacharya, P., Xia, Y., Cai, J., Zeydan, E., Fang, K., & Feng, H. (2026) · IEEE Internet of Things Journal · doi:10.1109/jiot.2026.3703862 · Cited by 3
“High-fidelity simulation and sim-to-real transfer techniques help bridge the gap between simulated and real-world environments, addressing the challenge of policy transfer across domains [53].”
Virtual-to-Real Motion Adaptation Using Genetic Algorithm for Multi-DOF Disaster Response Robots
Kamezaki, M., Katano, T., & Sugano, S. (2026) · IEEE Access · doi:10.1109/access.2026.3663368
“To mitigate this issue, simulation-based learning and sim-to-real transfer approaches have been introduced [14], enabling robots to efficiently acquire motion skills in virtual environments and transfer them to the physical world.”
Metadata
- Authors
- Kim, D., Lee, H., Cha, J., & Park, J.
- Affiliations
- Seoul National University (KR)
- Venue
- IEEE Robotics & Automation Magazine
- Published
- 2025-03
- Volume
- 32
- Issue
- 1
- Pages
- 49-58
- ISSN
- 1070-9932 (ISSN-L)
- 1558-223X
- OpenAlex
- W4405304880
- Semantic Scholar
- 1c10b20ac71752944c9f0c3f973e385ed1185a6a
- Cited by
- 12
- References
- 17
- Citations by year
- 2026: 8 · 2025: 4
- FWCI
- 1.78
- Type
- journal article
- Language
- en
- Open access
- closed
- Licence
- Locations
- IEEE Robotics & Automation Magazine (publishedVersion)
- Funders
- National Research Foundation of Korea (2021R1A2C3005914)
- NRF (RS-2024-00461583)
- Seoul National University (2021-0-01343)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Reinforcement Learning in Robotics
- Physical Sciences › Engineering › Control and Systems Engineering › Robotic Locomotion and Control
- Physical Sciences › Engineering › Biomedical Engineering › Prosthetics and Rehabilitation Robotics
- Keywords
- sim-to-real transfer, reality gap, humanoid robots, bipedalism, reinforcement learning, policy robustness, state history, noise modeling, delay modeling, state selection, perturbations, network size, real-world evaluation
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Bridging the Reality Gap: Analyzing Sim-to-Real Transfer Techniques for Reinforcement Learning in Humanoid Bipedal Locomotion. https://app.cochat.ai/papers/10.1109/mra.2024.3505784
- The paper
- Kim, D., Lee, H., Cha, J., & Park, J. (2025). Bridging the Reality Gap: Analyzing Sim-to-Real Transfer Techniques for Reinforcement Learning in Humanoid Bipedal Locomotion. IEEE Robotics & Automation Magazine. https://doi.org/10.1109/mra.2024.3505784
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.