Unsupervised body scheme learning through self-perception
The DOI matches its Crossref record; 19 of the 21 references it cites were found in a registry or an index.
Abstract
In this paper, we present an approach allowing a robot to learn a generative model of its own physical body from scratch using self-perception with a single monocular camera. Our approach yields a compact Bayesian network for the robot's kinematic structure including the forward and inverse models relating action signals and body pose. We propose to simultaneously learn local action models for all pairs of perceivable body parts from data generated through random "motor babbling." From this repertoire of local models, we construct a Bayesian network for the full system using the pose prediction accuracy on a separate cross validation data set as the criterion for model selection. The resulting model can be used to predict the body pose when no perception is available and allows for gradient-based posture control. In experiments with real and simulated manipulator arms, we show that our system is able to quickly learn compact and accurate models and to robustly deal with noisy observations.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Citations
Cites
Gaussian Processes for Machine Learning
Rasmussen, C. E., & Williams, C. K. I. (2005) · doi:10.7551/mitpress/3206.001.0001 · Cited by 11,705
Introduction to Robotics mechanics and Control
Craig, J. (1986) · Cited by 5,024
Introduction to robotics: Mechanics and control
Merat, F. (1987) · IEEE Journal on Robotics and Automation · doi:10.1109/jra.1987.1087086 · Cited by 2,398
Principles of Robot Motion: Theory, Algorithms, and Implementations
Choset, H., & Latombe, J. (2005) · Medical Entomology and Zoology · Cited by 2,149
Maravita, A., & Iriki, A. (2004) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2003.12.008 · Cited by 1,214 · cited 1×
“Brain scan studies on monkeys that have been trained to use tools revealed that the tool itself even gets integrated into their body schemes over time [5].”
The body schema and multisensory representation(s) of peripersonal space
Holmes, N. P., & Spence, C. (2004) · Cognitive Processing · doi:10.1007/s10339-004-0013-3 · Cited by 623
Online self-calibration for mobile robots
Roy, N., & Thrun, S. (2003) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/robot.1999.770447 · Cited by 142
Self-Organizing Body Schema for Motor Planning
Morasso, P., & Sanguineti, V. (1995) · Journal of Motor Behavior · doi:10.1080/00222895.1995.9941699 · Cited by 126
Learning forward models for robots
Dearden, A., & Demiris, Y. (2005) · Spiral (Imperial College London) · Cited by 115
Dynamic Imitation in a Humanoid Robot through Nonparametric Probabilistic Inference
Grimes, D., Chalodhorn, R., & Rao, R. (2006) · Robotics: Science and Systems II · doi:10.15607/rss.2006.ii.026 · Cited by 97
Research works printed but not linked: 4
- ware, 1915, Approximating the distribution for sum of product of normal variables
- fiala, 2004, Artag revision 1 a fiducial marker system using digital techniques
- 10.1002/(SICI)1099-0917(199709/12)6:3/4<179::AID-EDP157>3.0.CO;2-R · doi:10.1002/(sici)1099-0917(199709/12)6:3/4<179::aid-edp157>3.0.co;2-r
- bongard, 2006, automated synthesis of body schema using multiple sensor modalities, Proceedings of the 10th International Conference on the Simulation and Synthesis of Living Systems (ALIFE X), 220
Cited by
Model learning for robot control: A survey
Nguyen-Tuong, D., & Peters, J. (2011) · Cognitive Processing · doi:10.1007/s10339-011-0404-1 · Cited by 503
On-line regression algorithms for learning mechanical models of robots: A survey
Sigaud, O., Salaün, C., & Padois, V. (2011) · Robotics and Autonomous Systems · doi:10.1016/j.robot.2011.07.006 · Cited by 150
Ditto: Building Digital Twins of Articulated Objects from Interaction
Jiang, Z., Hsu, C.-C., & Zhu, Y. (2022) · 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52688.2022.00553 · Cited by 92
Body schema learning for robotic manipulators from visual self-perception
Sturm, J., Plagemann, C., & Burgard, W. (2009) · Journal of Physiology-Paris · doi:10.1016/j.jphysparis.2009.08.005 · Cited by 58
Self-citationRoggen, D., Magnenat, S., Waibel, M., & Tröster, G. (2011) · IEEE Robotics & Automation Magazine · doi:10.1109/mra.2011.940992 · Cited by 51
Learning Kinematic Models of Articulated Objects
Sturm, J. (2013) · Springer tracts in advanced robotics · doi:10.1007/978-3-642-37160-8_4 · Cited by 51
Self-citationAUTONOMOUS ONLINE LEARNING OF REACHING BEHAVIOR IN A HUMANOID ROBOT
Jamone, L., Natale, L., Nori, F., Metta, G., & Sandini, G. (2012) · International Journal of Humanoid Robotics · doi:10.1142/s021984361250017x · Cited by 52
Tool-body assimilation model considering grasping motion through deep learning
Takahashi, K., Kim, K., Ogata, T., & Sugano, S. (2017) · Robotics and Autonomous Systems · doi:10.1016/j.robot.2017.01.002 · Cited by 43
Roggen, D., Magnenat, S., Waibel, M., & Tröster, G. (2011) · Repository for Publications and Research Data (ETH Zurich) · doi:10.3929/ethz-a-010001702 · Cited by 34
Learning predictive models of a depth camera & manipulator from raw execution traces
Boots, B., Byravan, A., & Fox, D. (2014) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra.2014.6907443 · Cited by 32
Learning task error models for manipulation
Pástor, P., Kalakrishnan, M., Binney, J., Kelly, J. W., Righetti, L., Sukhatme, G. S., & Stefan Schaal, A. (2013) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra.2013.6630935 · Cited by 31
Takahashi, K., Ogata, T., Nakanishi, J., Cheng, G., & Sugano, S. (2017) · Advanced Robotics · doi:10.1080/01691864.2017.1383939 · Cited by 26
Adaptive Body Scheme Models for Robust Robotic Manipulation
Sturm, J., Plagemann, C., & Burgard, W. (2008) · Robotics: Science and Systems · doi:10.15607/rss.2008.iv.015 · Cited by 29
Self-citationDitto in the House: Building Articulation Models of Indoor Scenes through Interactive Perception
Hsu, C.-C., Jiang, Z., & Zhu, Y. (2023) · 2023 IEEE International Conference on Robotics and Automation (ICRA) · doi:10.1109/icra48891.2023.10161431 · Cited by 21
Autonomous Learning of the Robot Kinematic Model
Libera, A. D., Castaman, N., Ghidoni, S., & Carli, R. (2020) · IEEE Transactions on Robotics · doi:10.1109/tro.2020.3038690 · Cited by 15
Chang, H. J., & Demiris, Y. (2015) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2015.7298933 · Cited by 14
Learning and Acting in Peripersonal Space: Moving, Reaching, and Grasping
Juett, J., & Kuipers, B. (2019) · Frontiers in Neurorobotics · doi:10.3389/fnbot.2019.00004 · Cited by 13
“Robotic Modeling Some robotics researchers (e.g., Hersch et al., 2008; Sturm et al., 2008) focus on learning the kind of precise model of the robot that is used for traditional forward and inverse kinematics-based motion planning.”
BackgroundTool-Body Assimilation Model Based on Body Babbling and Neurodynamical System
Takahashi, K., Ogata, T., Tjandra, H., Yamaguchi, Y., & Sugano, S. (2015) · Mathematical Problems in Engineering · doi:10.1155/2015/837540 · Cited by 12
Highly Articulated Kinematic Structure Estimation Combining Motion and Skeleton Information
Chang, H. J., & Demiris, Y. (2017) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2017.2748579 · Cited by 13
Kobayashi, Y., Harada, K., & Takagi, K. (2019) · Robotics and Autonomous Systems · doi:10.1016/j.robot.2019.04.010 · Cited by 10
Gaussian processes for flexible robot learning.
Plagemann, C. (2008) · FreiDok plus (Universitätsbibliothek Freiburg) · Cited by 10
Self-citationRoboception and adaptation in a cognitive robot
Augello, A., Gaglio, S., Infantino, I., Maniscalco, U., Pilato, G., & Vella, F. (2023) · Robotics and Autonomous Systems · doi:10.1016/j.robot.2023.104400 · Cited by 7
Learning to Grasp by Extending the Peri-Personal Space Graph
Juett, J., & Kuipers, B. J. (2018) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros.2018.8593938 · Cited by 6
Forward Models Applied in Visual Servoing for a Reaching Task in the iCub Humanoid Robot
Gamarra, D. F. T., Pinpin, L. K., Laschi, C., & Dario, P. (2009) · Applied Bionics and Biomechanics · doi:10.1080/11762320903180849 · Cited by 5
Learning probabilistic models for mobile manipulation robots
Sturm, J., & Burgard, W. (2013) · International Joint Conference on Artificial Intelligence · Cited by 5
Self-citationLearning symbolic forward models for robotic motion planning and control
Moriguchi, H., & Lipson, H. (2011) · European Conference on Artificial Life · doi:10.7551/978-0-262-29714-1-ch086 · Cited by 6
Towards improving the absolute accuracy of lightweight robots by nonparametric calibration
Sabsch, J., Hanses, M., Zug, S., & Elkmann, N. (2017) · IEEE International Conference on Emerging Technologies and Factory Automation (ETFA) · doi:10.1109/etfa.2017.8247764 · Cited by 4
Learning Kinematics from Direct Self-Observation Using Nearest-Neighbor Methods
Schulz, H., Ott, L., Sturm, J., & Burgard, W. (2009) · Springer eBooks · doi:10.1007/978-3-642-01213-6_2 · Cited by 3
Self-citationAn evolutionary cognitive architecture made of a bag of networks
Churchill, A. W., & Fernando, C. (2014) · Evolutionary Intelligence · doi:10.1007/s12065-014-0121-7 · Cited by 3
Simultaneous Localization, Mapping and Self-body Shape Estimation by a Mobile Robot
Taniguchi, A., WanPeng, L., Taniguchi, T., Takano, T., Hagiwara, Y., & Yano, S. (2017) · Advances in intelligent systems and computing · doi:10.1007/978-3-319-48036-7_5 · Cited by 2
Mimura, T., Hagiwara, Y., Taniguchi, T., & Inamura, T. (2017) · Advanced Robotics · doi:10.1080/01691864.2016.1270854 · Cited by 2
Robot kinematic structure classification from time series of visual data
Libera, A. D., Terzi, M., Rossi, A., Susto, G. A., & Ruggero Carli, A. (2019) · European Control Conference (ECC) · doi:10.23919/ecc.2019.8795900 · Cited by 2
Towards Understanding Articulated Objects
Sturm, J., Stachniss, C., Pradeep, V., Plagemann, C., Konolige, K., & Burgard, W. (2009) · Cited by 2
Self-citationHigh performance visual servoing for controlled µm-positioning
Pieters, R., Jonker, P. P., & Nijmeijer, H. (2010) · World Congress on Intelligent Control and Automation · doi:10.1109/wcica.2010.5554983 · Cited by 2
Body Model Transition by Tool Grasping During Motor Babbling Using Deep Learning and RNN
Takahashi, K., Tjandra, H., Ogata, T., & Sugano, S. (2016) · Lecture notes in computer science · doi:10.1007/978-3-319-44778-0_20 · Cited by 1
Metadata
- Authors
- Sturm, J., Plagemann, C., & Burgard, W.
- Affiliations
- University of Freiburg (DE)
- ORCID
- Burgard, Wolfram
- Venue
- 2008 IEEE International Conference on Robotics and Automation
- Publisher
- IEEE
- Published
- 2008-05
- Pages
- 3328-3333
- ISSN
- 1050-4729 (ISSN-L)
- OpenAlex
- W1967265872
- Cited by
- 35
- References
- 21
- Citations by year
- 2023: 2 · 2022: 1 · 2020: 1 · 2019: 3 · 2018: 1 · 2017: 6 · 2016: 1 · 2015: 2 · 2014: 2 · 2013: 3 · 2012: 1
- FWCI
- 2.97
- Type
- conference paper
- Language
- en
- Open access
- closed
- Locations
- Proceedings - IEEE International Conference on Robotics and Automation/Proceedings (publishedVersion)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Reinforcement Learning in Robotics
- Physical Sciences › Engineering › Control and Systems Engineering › Robot Manipulation and Learning
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Robotics and Sensor-Based Localization
- Keywords
- self-perception, unsupervised learning, Bayesian networks, forward kinematics, inverse kinematics, motor babbling, postural control, robot manipulators, monocular camera, model selection
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.