Where Should I Walk? Predicting Terrain Properties From Images Via Self-Supervised Learning
Open accessThe DOI matches its Crossref record; 5 of the 5 references it cites were found in a registry or an index.
Abstract
Legged robots have the potential to traverse diverse and rugged terrain. To find a safe and efficient navigation path and to carefully select individual footholds, it is useful to be able to predict properties of the terrain ahead of the robot. In this letter, we propose a method to collect data from robot-terrain interaction and associate it to images. Using sparse data acquired in teleoperation experiments with a quadrupedal robot, we train a neural network to generate a dense prediction of the terrain properties in front of the robot. To generate training data, we project the foothold positions from the robot trajectory into on-board camera images. We then attach labels to these footholds by identifying the dominant features of the force-torque signal measured with sensorized feet. We show that data collected in this fashion can be used to train a convolutional network for terrain property prediction as well as weakly supervised semantic segmentation. Finally, we show that the predicted terrain properties can be used for autonomous navigation of the ANYmal quadruped robot.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
ORB-SLAM2: An Open-Source SLAM System for Monocular, Stereo, and RGB-D Cameras
Mur-Artal, R., & Tardos, J. D. (2017) · IEEE Transactions on Robotics · doi:10.1109/tro.2017.2705103 · Cited by 6,898 · cited 1×
“For tracking the camera pose we use a visual SLAM system ORB-SLAM2 [19].”
MethodWhat Uncertainties Do We Need in Bayesian Deep Learning for Computer Vision?
Kendall, A., & Gal, Y. (2017) · arXiv · doi:10.48550/arxiv.1703.04977 · Cited by 3,088 · cited 1×
“For the regression task, we predict both the mean mi of the ground reaction score as well as its variance σi for each pixel i and employ a negative Gaussian log-likelihood loss [22], averaged over valid pixels i ∈ V for which ground truth m i is available:”
MethodANYmal - a highly mobile and dynamic quadrupedal robot
Hutter, M., Gehring, C., Jud, D., Lauber, A., Bellicoso, C. D., Tsounis, V., Hwangbo, J., Bodie, K., Fankhauser, P., Bloesch, M., et al. (2016) · 2016 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros.2016.7758092 · Cited by 969 · cited 2×
“We deploy and test the proposed system on the ANYmal quadruped [6].”
MethodSelf-citationAdapNet: Adaptive semantic segmentation in adverse environmental conditions
Valada, A., Vertens, J., Dhall, A., & Burgard, W. (2017) · 2017 IEEE International Conference on Robotics and Automation (ICRA) · doi:10.1109/icra.2017.7989540 · Cited by 209 · cited 2×
“Most camera-based approaches either classify different terrain types and then attach a value to every class [4], [5], [9], [12] or directly predict a manually defined”
BackgroundMethodSPOC: Deep Learning-based Terrain Classification for Mars Rover Missions
Rothrock, B., Kennedy, R., Cunningham, C., Papon, J., Heverly, M., & Ono, M. (2016) · AIAA SPACE 2016 · doi:10.2514/6.2016-5539 · Cited by 161
Cited by
BADGR: An Autonomous Self-Supervised Learning-Based Navigation System
Kahn, G., Abbeel, P., & Levine, S. (2021) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2021.3057023 · Cited by 284
Object detection recognition and robot grasping based on machine learning: a survey
Bai, Q., Li, S., Yang, J., Song, Q., Li, Z., & Zhang, X. (2020) · IEEE Access · doi:10.1109/access.2020.3028740 · Cited by 154
Morales, E. F., Murrieta-Cid, R., Becerra, I., & Esquivel-Basaldua, M. A. (2021) · Intelligent Service Robotics · doi:10.1007/s11370-021-00398-z · Cited by 101
Tapered whisker reservoir computing for real-time terrain identification-based navigation
Yu, Z., Sadati, S. M. H., Perera, S., Hauser, H., Childs, P. R. N., & Nanayakkara, T. (2023) · Scientific Reports · doi:10.1038/s41598-023-31994-x · Cited by 21
Yu, Z., Sadati, S. M. H., Hauser, H., Childs, P. R. N., & Nanayakkara, T. (2022) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2022.3159859 · Cited by 15
Metadata
- Authors
- Wellhausen, L., Dosovitskiy, A., Ranftl, R., Walas, K., Cadena, C., & Hutter, M.
- Venue
- IEEE Robotics and Automation Letters
- Published
- 2019-04
- Volume
- 4
- Issue
- 2
- Pages
- 1509-1516
- ISSN
- 2377-3766 (ISSN-L)
- 2377-3774
- OpenAlex
- W2913921102
- Cited by
- 195
- References
- 5
- FWCI
- 7.13
- Type
- journal article
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Where Should I Walk? Predicting Terrain Properties From Images Via Self-Supervised Learning. https://app.cochat.ai/papers/10.1109/lra.2019.2895390
- The paper
- Wellhausen, L., Dosovitskiy, A., Ranftl, R., Walas, K., Cadena, C., & Hutter, M. (2019). Where Should I Walk? Predicting Terrain Properties From Images Via Self-Supervised Learning. IEEE Robotics and Automation Letters. https://doi.org/10.1109/lra.2019.2895390
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.