The DOI matches its Crossref record; 39 of the 39 references it cites were found in a registry or an index.
Learning robust perceptive locomotion for quadrupedal robots in the wild
Open accessAbstract
Legged robots that can operate autonomously in remote and hazardous environments will greatly increase opportunities for exploration into underexplored areas. Exteroceptive perception is crucial for fast and energy-efficient locomotion: Perceiving the terrain before making contact with it enables planning and adaptation of the gait ahead of time to maintain speed and stability. However, using exteroceptive perception robustly for locomotion has remained a grand challenge in robotics. Snow, vegetation, and water visually appear as obstacles on which the robot cannot step or are missing altogether due to high reflectance. In addition, depth perception can degrade due to difficult lighting, dust, fog, reflective or transparent surfaces, sensor occlusion, and more. For this reason, the most robust and general solutions to legged locomotion to date rely solely on proprioception. This severely limits locomotion speed because the robot has to physically feel out the terrain before adapting its gait accordingly. Here, we present a robust and general solution to integrating exteroceptive and proprioceptive perception for legged locomotion. We leverage an attention-based recurrent encoder that integrates proprioceptive and exteroceptive input. The encoder is trained end to end and learns to seamlessly combine the different perception modalities without resorting to heuristics. The result is a legged locomotion controller with high robustness and speed. The controller was tested in a variety of challenging natural and urban environments over multiple seasons and completed an hour-long hike in the Alps in the time recommended for human hikers.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Hochreiter, S., & Schmidhuber, J. (1997) · Neural Computation · doi:10.1162/neco.1997.9.8.1735 · Cited by 102,318
Learning Phrase Representations using RNN Encoder–Decoder for Statistical Machine Translation
Cho, K., van Merrienboer, B., Gulcehre, C., Bahdanau, D., Bougares, F., Schwenk, H., & Bengio, Y. (2014) · Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.3115/v1/d14-1179 · Cited by 25,545
Proximal Policy Optimization Algorithms
Schulman, J., Wolski, F., Dhariwal, P., Radford, A., & Klimov, O. (2017) · arXiv · doi:10.48550/arxiv.1707.06347 · Cited by 10,625
Grandmaster level in StarCraft II using multi-agent reinforcement learning
Vinyals, O., Babuschkin, I., Czarnecki, W. M., Mathieu, M., Dudzik, A., Chung, J., Choi, D. H., Powell, R., Ewalds, T., Georgiev, P., et al. (2019) · Nature · doi:10.1038/s41586-019-1724-z · Cited by 3,455
BigDog, the Rough-Terrain Quadruped Robot
Raibert, M., Blankespoor, K., Nelson, G., & Playter, R. (2008) · IFAC Proceedings Volumes · doi:10.3182/20080706-5-kr-1001.01833 · Cited by 1,495
Learning agile and dynamic motor skills for legged robots
Hwangbo, J., Lee, J., Dosovitskiy, A., Bellicoso, D., Tsounis, V., Koltun, V., & Hutter, M. (2019) · Science Robotics · doi:10.1126/scirobotics.aau5872 · Cited by 1,482
Self-citationLearning quadrupedal locomotion over challenging terrain
Lee, J., Hwangbo, J., Wellhausen, L., Koltun, V., & Hutter, M. (2020) · Science Robotics · doi:10.1126/scirobotics.abc5986 · Cited by 1,156
Self-citationDeepMimic: example-guided deep reinforcement learning of physics-based character skills
Peng, X. B., Abbeel, P., Levine, S., & van de Panne, M. (2018) · ACM Transactions on Graphics · doi:10.1145/3197517.3201311 · Cited by 880
Learning quadrupedal locomotion over challenging terrain
Lee, J., Hwangbo, J., Wellhausen, L., Koltun, V., & Hutter, M. (2020) · Repository for Publications and Research Data (ETH Zurich) · doi:10.3929/ethz-b-000448343 · Cited by 752
Self-citationSim-to-Real: Learning Agile Locomotion For Quadruped Robots
Tan, J., Zhang, T., Coumans, E., Iscen, A., Bai, Y., Hafner, D., Bohez, S., & Vanhoucke, V. (2018) · Robotics: Science and Systems XIV · doi:10.15607/rss.2018.xiv.010 · Cited by 735
Mini Cheetah: A Platform for Pushing the Limits of Dynamic Quadruped Control
Katz, B., Carlo, J. D., & Kim, S. (2019) · 2019 International Conference on Robotics and Automation (ICRA) · doi:10.1109/icra.2019.8793865 · Cited by 544
RMA: Rapid Motor Adaptation for Legged Robots
Kumar, A., Fu, Z., Pathak, D., & Malik, J. (2021) · Robotics: Science and Systems XVII · doi:10.15607/rss.2021.xvii.011 · Cited by 513
peng (2017) · doi:10.1145/3072959.3073602 · Cited by 502
Gaze and the Control of Foot Placement When Walking in Natural Terrain
Matthis, J. S., Yates, J. L., & Hayhoe, M. M. (2018) · Current Biology · doi:10.1016/j.cub.2018.03.008 · Cited by 424
Learning Agile Robotic Locomotion Skills by Imitating Animals
Bin Peng, X., Coumans, E., Zhang, T., Lee, T.-W., Tan, J., & Levine, S. (2020) · Robotics: Science and Systems XVI · doi:10.15607/rss.2020.xvi.064 · Cited by 357
Kinect v2 for mobile robot navigation: Evaluation and modeling
Fankhauser, P., Bloesch, M., Rodriguez, D., Kaestner, R., Hutter, M., & Siegwart, R. (2015) · 2015 International Conference on Advanced Robotics (ICAR) · doi:10.1109/icar.2015.7251485 · Cited by 314
Self-citationProbabilistic Terrain Mapping for Mobile Robots With Uncertain Localization
Fankhauser, P., Bloesch, M., & Hutter, M. (2018) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2018.2849506 · Cited by 311
Self-citationState Estimation for Legged Robots: Consistent Fusion of Leg Kinematics and IMU
Bloesch, M., Hutter, M., Hoepflinger, M. A., Leutenegger, S., Gehring, C., Remy, C. D., & Siegwart, R. (2013) · Robotics · doi:10.7551/mitpress/9816.003.0008 · Cited by 259
Self-citationPer-Contact Iteration Method for Solving Contact Dynamics
Hwangbo, J., Lee, J., & Hutter, M. (2018) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2018.2792536 · Cited by 243 · cited 2×
“We use RaiSim [58] as our simulator to build the training environment.”
MethodSelf-citationDeepGait: Planning and Control of Quadrupedal Gaits Using Deep Reinforcement Learning
Tsounis, V., Alge, M., Lee, J., Farshidian, F., & Hutter, M. (2020) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2020.2979660 · Cited by 223
Self-citationRobust Rough-Terrain Locomotion with a Quadrupedal Robot
Fankhauser, P., Bjelonic, M., Dario Bellicoso, C., Miki, T., & Hutter, M. (2018) · 2018 IEEE International Conference on Robotics and Automation (ICRA) · doi:10.1109/icra.2018.8460731 · Cited by 205
Self-citationMulti-expert learning of adaptive legged locomotion
Yang, C., Yuan, K., Zhu, Q., Yu, W., & Li, Z. (2020) · Science Robotics · doi:10.1126/scirobotics.abb2174 · Cited by 201
Blind Bipedal Stair Traversal via Sim-to-Real Reinforcement Learning
Siekmann, J., Green, K., Warila, J., Fern, A., & Hurst, J. (2021) · arXiv · doi:10.48550/arxiv.2105.08328 · Cited by 193
Blind Bipedal Stair Traversal via Sim-to-Real Reinforcement Learning
Siekmann, J., Green, K., Warila, J., Fern, A., & Hurst, J. (2021) · Robotics: Science and Systems XVII · doi:10.15607/rss.2021.xvii.061 · Cited by 193
ROBOT-CENTRIC ELEVATION MAPPING WITH UNCERTAINTY ESTIMATES
FANKHAUSER, P., BLOESCH, M., GEHRING, C., HUTTER, M., & SIEGWART, R. (2014) · Mobile Service Robotics · doi:10.1142/9789814623353_0051 · Cited by 192
Self-citationTranzatto, M., Mascarich, F., Bernreiter, L., Godinho, C., Camurri, M., Khattak, S., Dang, T., Reijgwart, V., Loeje, J., Wisth, D., et al. (2022) · Field Robotics · doi:10.55417/fr.2022011 · Cited by 138
Self-citationPerceptive Locomotion in Rough Terrain – Online Foothold Optimization
Jenelten, F., Miki, T., Vijayan, A. E., Bjelonic, M., & Hutter, M. (2020) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2020.3007427 · Cited by 122
Self-citationALLSTEPS: Curriculum‐driven Learning of Stepping Stone Skills
Xie, Z., Ling, H. Y., Kim, N. H., & van de Panne, M. (2020) · Computer Graphics Forum · doi:10.1111/cgf.14115 · Cited by 122
The diffractive achromat full spectrum computational imaging with diffractive optics
Peng, Y., Fu, Q., Heide, F., & Heidrich, W. (2016) · ACM Transactions on Graphics · doi:10.1145/2897824.2925941 · Cited by 98
Trajectory and foothold optimization using low-dimensional models for rough terrain locomotion
Mastalli, C., Focchi, M., Havoutis, I., Radulescu, A., Calinon, S., Buchli, J., Caldwell, D. G., & Semini, C. (2017) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra.2017.7989131 · Cited by 92
Robust Deep Multi-modal Learning Based on Gated Information Fusion Network
Kim, J., Koh, J., Kim, Y., Choi, J., Hwang, Y., & Choi, J. W. (2019) · Lecture notes in computer science · doi:10.1007/978-3-030-20870-7_6 · Cited by 84
Jumping over obstacles with MIT Cheetah 2
Park, H.-W., Wensing, P. M., & Kim, S. (2020) · Robotics and Autonomous Systems · doi:10.1016/j.robot.2020.103703 · Cited by 83
Fast and Continuous Foothold Adaptation for Dynamic Locomotion Through CNNs
Villarreal, O., Barasuol, V., Camurri, M., Franceschi, L., Focchi, M., Pontil, M., Caldwell, D. G., & Semini, C. (2019) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2019.2899434 · Cited by 78
Stereo vision and terrain modeling for quadruped robots
Kolter, J. Z., Kim, Y., & Ng, A. Y. (2009) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/robot.2009.5152795 · Cited by 76
An optimization approach to rough terrain locomotion
Zucker, M., Bagnell, J. A., Atkeson, C. G., & Kuffner, J. (2010) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/robot.2010.5509176 · Cited by 74
Adaptive Motion Planning for Autonomous Rough Terrain Traversal with a Walking Robot
Belter, D., Łabęcki, P., & Skrzypczyński, P. (2015) · Journal of Field Robotics · doi:10.1002/rob.21610 · Cited by 72
Focchi, M., Orsolino, R., Camurri, M., Barasuol, V., Mastalli, C., Caldwell, D. G., & Semini, C. (2018) · HAL · Cited by 42
Deep Gated Multi-modal Learning: In-hand Object Pose Changes Estimation using Tactile and Image Data
Anzai, T., & Takahashi, K. (2020) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros45743.2020.9341799 · Cited by 39
Focchi, M., Orsolino, R., Camurri, M., Barasuol, V., Mastalli, C., Caldwell, D. G., & Semini, C. (2020) · Springer Tracts in Advanced Robotics · doi:10.1007/978-3-030-22327-4_9 · Cited by 28
Cited by
Multi-environment robotic transitions through adaptive morphogenesis
Baines, R., Patiballa, S. K., Booth, J., Ramirez, L., Sipple, T., Garcia, A., Fish, F., & Kramer-Bottiglio, R. (2022) · Nature · doi:10.1038/s41586-022-05188-w · Cited by 324
Teleoperation of Humanoid Robots: A Survey
Darvish, K., Penco, L., Ramos, J., Cisneros, R., Pratt, J., Yoshida, E., Ivaldi, S., & Pucci, D. (2023) · IEEE Transactions on Robotics · doi:10.1109/tro.2023.3236952 · Cited by 278
Perceptive Locomotion Through Nonlinear Model-Predictive Control
Grandia, R., Jenelten, F., Yang, S., Farshidian, F., & Hutter, M. (2023) · IEEE Transactions on Robotics · doi:10.1109/tro.2023.3275384 · Cited by 265
Self-citationANYmal parkour: Learning agile navigation for quadrupedal robots
Hoeller, D., Rudin, N., Sako, D., & Hutter, M. (2024) · Science Robotics · doi:10.1126/scirobotics.adi7566 · Cited by 263
Self-citationNavigating to objects in the real world
Gervet, T., Chintala, S., Batra, D., Malik, J., & Chaplot, D. S. (2023) · Science Robotics · doi:10.1126/scirobotics.adf6991 · Cited by 125
Metadata
- Authors
- Miki, T., Lee, J., Hwangbo, J., Wellhausen, L., Koltun, V., & Hutter, M.
- Venue
- Science Robotics
- Published
- 2022-01-26
- Volume
- 7
- Issue
- 62
- Pages
- eabk2822-eabk2822
- ISSN
- 2470-9476 (ISSN-L)
- OpenAlex
- W4205430897
- Cited by
- 813
- References
- 39
- FWCI
- 44.66
- Type
- journal article
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.