Model learning for robot control: A survey
Abstract
Models are among the most essential tools in robotics, such as kinematics and dynamics models of the robot's own body and controllable external objects. It is widely believed that intelligent mammals also rely on internal models in order to generate their actions. However, while classical robotics relies on manually generated models that are based on human insights into physics, future autonomous, cognitive robots need to be able to automatically generate models that are based on information which is extracted from the data streams accessible to the robot. In this paper, we survey the progress in model learning with a strong focus on robot control on a kinematic as well as dynamical level. Here, a model describes essential information about the behavior of the environment and the influence of an agent on this environment. In the context of model-based learning control, we view the model from three different perspectives. First, we need to study the different possible model learning architectures for robotics. Second, we discuss what kind of problems these architecture and the domain of robotics imply for the applicable learning methods. From this discussion, we deduce future directions of real-time learning algorithms. Third, we show where these scenarios have been used successfully in several case studies.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Fairness is another emerging property of ML applications.
“Fairness is another emerging property [26,118].”
From Software engineering challenges for machine learning applications: A literature review (2020) · cites this paper as [26]
Design issues on data collection, pre-processing, cleaning, labeling and augmentation, including big data challenges, exist for ML applications.
“-Data Design: Design issues on data collection, pre-processing, cleaning, labeling and augmentation, including big data challenges [17,23,25-27, 31,33,41,44,74-76,105,114-116].”
From Software engineering challenges for machine learning applications: A literature review (2020) · cites this paper as [26]
Cites
Neural Networks: A Comprehensive Foundation
Haykin, S. (1998) · Cited by 30,070 · see citing works
The Elements of Statistical Learning
Hastie, T., Friedman, J., & Tibshirani, R. (2001) · Springer Series in Statistics · doi:10.1007/978-0-387-21606-5 · Cited by 24,162 · see citing works
Slotine, J.-J. E., & Li, W. (1991) · CERN Document Server (European Organization for Nuclear Research) · Cited by 18,937 · see citing works
System Identification: Theory for the User
Lennart Ljung, A. (1987) · Cited by 17,690 · see citing works
Nonlinear Dimensionality Reduction by Locally Linear Embedding
Roweis, S. T., & Saul, L. K. (2000) · Science · doi:10.1126/science.290.5500.2323 · Cited by 14,965 · see citing works
Bellman, R. E. (2010) · doi:10.1515/9781400835386 · Cited by 14,588 · see citing works
A Global Geometric Framework for Nonlinear Dimensionality Reduction
Tenenbaum, J. B., Silva, V. D., & Langford, J. C. (2000) · Science · doi:10.1126/science.290.5500.2319 · Cited by 13,585 · see citing works
A tutorial on support vector regression
Smola, A. J., & Schölkopf, B. (2004) · Statistics and Computing · doi:10.1023/b:stco.0000035301.49549.88 · Cited by 13,482 · see citing works
Conditional Random Fields: Probabilistic Models for Segmenting and Labeling Sequence Data
Lafferty, J., McCallum, A., & Pereira, F. C. N. (2001) · ScholarlyCommons (University of Pennsylvania) · Cited by 12,953 · see citing works
Schölkopf, B., & Smola, A. J. (2001) · The MIT Press eBooks · doi:10.7551/mitpress/4175.001.0001 · Cited by 11,934 · see citing works
Proceedings of the 24th international conference on Machine learning
(2007) · doi:10.1145/1273496 · Cited by 11,869 · see citing works
Gaussian Processes for Machine Learning
Rasmussen, C. E., & Williams, C. K. I. (2005) · doi:10.7551/mitpress/3206.001.0001 · Cited by 11,705 · see citing works
System identification—Theory for the user
Sinha, N. K. (1989) · Automatica · doi:10.1016/0005-1098(89)90019-8 · Cited by 9,215 · see citing works
Proceedings of the 19th International Joint Conference on Artificial Intelligence
Pujol, J. M., Delgado, J., Ramon Sangüesa, A., & Flache, A. (2005) · International Joint Conference on Artificial Intelligence · Cited by 5,771 · see citing works
Adaptive Mixtures of Local Experts
Jacobs, R. A., Jordan, M. I., Nowlan, S. J., & Hinton, G. E. (1991) · Neural Computation · doi:10.1162/neco.1991.3.1.79 · Cited by 5,384 · see citing works
Introduction to Robotics mechanics and Control
Craig, J. (1986) · Cited by 5,021 · see citing works
Åström, K. J., & Wittenmark, B. (1989) · Cited by 5,009 · see citing works
Chapelle, O., Schölkopf, B., & Zien, A. (2006) · The MIT Press eBooks · doi:10.7551/mitpress/9780262033589.001.0001 · Cited by 4,560 · see citing works
Learning With Kernels: Support Vector Machines, Regularization, Optimization, and Beyond
Williams, C. K. I. (2003) · Journal of the American Statistical Association · doi:10.1198/jasa.2003.s269 · Cited by 4,390 · see citing works
Predictive Control With Constraints
Maciejowski, J. M. (2001) · Cambridge University Engineering Department Publications Database · Cited by 4,305 · see citing works
Bayesian Learning for Neural Networks
Neal, R. M. (1996) · Lecture Notes in Statistics · doi:10.1007/978-1-4612-0745-0 · Cited by 4,276 · see citing works
Narendra, K. S., & Annaswamy, A. M. (1989) · Cited by 3,966 · see citing works
Spong, M. W. (1989) · Andalas University Repository (Andalas University) · Cited by 3,811 · see citing works
Proceedings. 1985 IEEE International Conference on Robotics and Automation
(1985) · doi:10.1109/robot.1985 · Cited by 3,692 · see citing works
Predictive control with constraints
Camacho, E. F. (2003) · Automatica · doi:10.1016/s0005-1098(03)00056-6 · Cited by 3,546 · see citing works
Khatib, O. (1987) · IEEE J. Robotics Autom. · doi:10.1109/jra.1987.1087068 · Cited by 3,233 · see citing works
Reservoir computing approaches to recurrent neural network training
Lukoševičius, M., & Jaeger, H. (2009) · Computer Science Review · doi:10.1016/j.cosrev.2009.03.005 · Cited by 2,956 · see citing works
A practical Bayesian framework for backpropagation networks
MacKay, D. J. C. (1992) · Neural Computation · doi:10.1162/neco.1992.4.3.448 · Cited by 2,917 · see citing works
Local Polynomial Modelling and its Applications
Fan, J., & Gijbels, I. (1994) · Springer eBooks · doi:10.1007/978-1-4899-3150-4 · Cited by 2,897 · see citing works
(1999) · doi:10.1162/089976600300015565 · Cited by 2,799 · see citing works
Internal models for motor control and trajectory planning.
Kawato, M. (1999) · Current Opinion in Neurobiology · doi:10.1016/s0959-4388(99)00028-8 · Cited by 2,685 · see citing works
Internal models in the cerebellum
Wolpert, D. M., Miall, R. C., & Kawato, M. (1998) · Trends in Cognitive Sciences · doi:10.1016/s1364-6613(98)01221-2 · Cited by 2,469 · see citing works
Multiple paired forward and inverse models for motor control
Wolpert, D. M., & Kawato, M. (1998) · Neural Networks · doi:10.1016/s0893-6080(98)00066-5 · Cited by 2,152 · see citing works
Unsupervised learning of finite mixture models
Figueiredo, M. A. T., & Jain, A. K. (2002) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/34.990138 · Cited by 2,119 · see citing works
Active Learning with Statistical Models
Cohn, D. A., Zoubin Ghahramani, A., & Jordan, M. I. (1996) · Journal of Artificial Intelligence Research · doi:10.1613/jair.295 · Cited by 2,054 · see citing works
Large Margin Methods for Structured and Interdependent Output Variables
Tsochantaridis, I., Joachims, T., Hofmann, T., & Altün, Y. (2005) · MPG.PuRe (Max Planck Society) · Cited by 1,879 · see citing works
A Unifying View of Sparse Approximate Gaussian Process Regression
Quiñonero-Candela, J., & Rasmussen, C. E. (2005) · doi:10.5555/1046920.1194909 · Cited by 1,739 · see citing works
Atkeson, C. G., Moore, A., & Schaal, S. (1997) · Artificial Intelligence Review · doi:10.1023/a:1006559212014 · Cited by 1,684 · see citing works
Proceedings of the 20th conference on Uncertainty in artificial intelligence
Meek, C., Chickering, M., & Halpern, J. Y. (2004) · Cited by 1,548 · see citing works
GAUSSIAN PROCESSES FOR MACHINE LEARNING
Seeger, M. (2004) · International Journal of Neural Systems · doi:10.1142/s0129065704001899 · Cited by 1,518 · see citing works
Optimal unsupervised learning in a single-layer linear feedforward neural network
Sanger, T. D. (1989) · Neural Networks · doi:10.1016/0893-6080(89)90044-0 · Cited by 1,506 · see citing works
Forward models: supervised learning with a distal teacher
Jordan, M. I., & Rumelhart, D. E. (1992) · Cognitive Science · doi:10.1207/s15516709cog1603_1 · Cited by 1,451 · see citing works
Is imitation learning the route to humanoid robots?
Schaal, S. (1999) · Trends in Cognitive Sciences · doi:10.1016/s1364-6613(99)01327-3 · Cited by 1,325 · see citing works
Adaptive control using multiple models
Narendra, K. S., & Balakrishnan, J. (1997) · IEEE Transactions on Automatic Control · doi:10.1109/9.554398 · Cited by 1,279 · see citing works
A controller to overcome dead time
Smith, O. J. M. (1959) · Medical Entomology and Zoology · Cited by 1,263 · see citing works
Input space versus feature space in kernel-based methods
Schölkopf, B., Mika, S., Burges, C. J. C., Knirsch, P., Müller, K. R., Rätsch, G., & Smola, A. J. (1999) · IEEE Transactions on Neural Networks · doi:10.1109/72.788641 · Cited by 1,159 · see citing works
Introduction to ROBOTICS mechanics and control
Baillieul, J. (1987) · IEEE Transactions on Automatic Control · doi:10.1109/tac.1987.1104613 · Cited by 1,076 · see citing works
Nonparametric and Semiparametric Models
Härdle, W. K., Werwatz, A., Müller, M., et al. (2004) · Springer series in statistics · doi:10.1007/978-3-642-17146-8 · Cited by 1,061 · see citing works
Modeling and Control of Robot Manipulators
(2000) · Industrial Robot the international journal of robotics research and application · doi:10.1108/ir.2000.04927aae.003 · Cited by 989 · see citing works
Müller, P. C. (1992) · Automatica · doi:10.1016/0005-1098(92)90197-n · Cited by 837 · see citing works
Dyna, an Integrated Architecture for Learning, Planning, and Reacting
Sutton, R. S. (1991) · ACM SIGART Bulletin · doi:10.1145/122344.122377 · Cited by 777 · cited 2× · see citing works
“Forward models have also been essential in model based reinforcement learning approaches, which relate to the problem of optimal control [156,9, 103].”
BackgroundMethodSparse On-Line Gaussian Processes
Csató, L., & Opper, M. (2002) · Neural Computation · doi:10.1162/089976602317250933 · Cited by 762 · see citing works
Modeling, Identification and Control of Robots
Khalil, W., Dombre, E., & Nagurka, M. (2003) · Applied Mechanics Reviews · doi:10.1115/1.1566397 · Cited by 754 · see citing works
MOSAIC Model for Sensorimotor Learning and Control
Haruno, M., Wolpert, D. M., & Kawato, M. (2001) · Neural Computation · doi:10.1162/089976601750541778 · Cited by 710 · see citing works
Resilient Machines Through Continuous Self-Modeling
Bongard, J., Zykov, V., & Lipson, H. (2006) · Science · doi:10.1126/science.1133687 · Cited by 709 · see citing works
Optimal robot excitation and identification
Swevers, J., Ganseman, C., Tükel, D. B., et al. (1997) · IEEE Transactions on Robotics and Automation · doi:10.1109/70.631234 · Cited by 662 · see citing works
Adaptive Approximation Based Control
Farrell, J. A., & Polycarpou, M. M. (2006) · Wiley eBooks · doi:10.1002/0471781819 · Cited by 622 · see citing works
Unsupervised Learning of Models for Recognition
Weber, M., Welling, M., & Perona, P. (2000) · Lecture notes in computer science · doi:10.1007/3-540-45054-8_2 · Cited by 605 · see citing works
Modeling, Identification and Control of Robots
(2002) · Elsevier eBooks · doi:10.1016/b978-1-903996-66-9.x5000-3 · Cited by 592 · see citing works
Estimation of Inertial Parameters of Manipulator Loads and Links
Atkeson, C. G., An, C. H., & Hollerbach, J. M. (1986) · The International Journal of Robotics Research · doi:10.1177/027836498600500306 · Cited by 590 · see citing works
B, J. F. A., & Gijbels, I. (1995) · Journal of the Royal Statistical Society Series B (Statistical Methodology) · doi:10.1111/j.2517-6161.1995.tb02034.x · Cited by 566 · see citing works
Autonomous Inverted Helicopter Flight via Reinforcement Learning
Ng, A. Y., Coates, A., Diel, M., Ganapathi, V., Schulte, J., Tse, B., Berger, E., & Liang, E. (2006) · Springer tracts in advanced robotics · doi:10.1007/11552246_35 · Cited by 557 · see citing works
Locally Weighted Learning for Control
Atkeson, C. G., Moore, A., & Schaal, S. (1997) · Artificial Intelligence Review · doi:10.1023/a:1006511328852 · Cited by 551 · see citing works
An Application of Reinforcement Learning to Aerobatic Helicopter Flight
Abbeel, P., Coates, A., Quigley, M., & Ng, A. Y. (2007) · The MIT Press eBooks · doi:10.7551/mitpress/7503.003.0006 · Cited by 551 · see citing works
(2007) · The MIT Press eBooks · doi:10.7551/mitpress/7496.001.0001 · Cited by 539 · see citing works
Robot Learning from Demonstration
Atkeson, C. G., & Stefan Schaal, A. (2012) · SpringerReference · doi:10.1007/springerreference_302487 · Cited by 532 · see citing works
Incremental Online Learning in High Dimensions
Vijayakumar, S., D’Souza, A., & Stefan Schaal, A. (2005) · Neural Computation · doi:10.1162/089976605774320557 · Cited by 527 · see citing works
Thrun, S., & Mitchell, T. M. (1995) · Robotics and Autonomous Systems · doi:10.1016/0921-8890(95)00004-y · Cited by 523 · see citing works
Differential dynamic programming
Jacobson, D. H., & Mayne, D. Q. (1970) · CERN Document Server (European Organization for Nuclear Research) · Cited by 510 · see citing works
Identification of MIMO non-linear systems using a forward-regression orthogonal estimator
Billings, S. A., Chen, S., & Korenberg, M. J. (1989) · International Journal of Control · doi:10.1080/00207178908559767 · Cited by 507 · see citing works
Learning and Reproduction of Gestures by Imitation
Calinon, S., D'halluin, F., Sauser, E. L., Caldwell, D. G., & Billard, A. (2010) · IEEE Robotics & Automation Magazine · doi:10.1109/mra.2010.936947 · Cited by 461 · see citing works
Smoothing by Local Regression: Principles and Methods
Cleveland, W. S., & Loader, C. (1996) · Contributions to statistics · doi:10.1007/978-3-642-48425-4_2 · Cited by 460 · see citing works
Feedback-error-learning neural network for trajectory control of a robotic manipulator
Miyamoto, H., Kawato, M., Setoyama, T., et al. (1988) · Neural Networks · doi:10.1016/0893-6080(88)90030-5 · Cited by 447 · see citing works
Adaptation and learning using multiple models, switching, and tuning
Narendra, K. S., Balakrishnan, J., & Cılız, M. K. (1995) · IEEE Control Systems · doi:10.1109/37.387616 · Cited by 443 · see citing works
Exploiting Task Relatedness for Multiple Task Learning
Ben-David, S., & Schuller, R. (2003) · Lecture notes in computer science · doi:10.1007/978-3-540-45167-9_41 · Cited by 432 · see citing works
Evaluation of gaussian processes and other methods for non-linear regression
Hinton, G. E., & Rasmussen, C. E. (1997) · Library and Archives Canada (Government of Canada) · Cited by 430 · see citing works
SYNTHESIS OF WHOLE-BODY BEHAVIORS THROUGH HIERARCHICAL CONTROL OF BEHAVIORAL PRIMITIVES
Sentis, L., & Khatib, O. (2005) · International Journal of Humanoid Robotics · doi:10.1142/s0219843605000594 · Cited by 429 · see citing works
Predictive Representations of State
Littman, M. L., & Sutton, R. S. (2001) · Cited by 424 · see citing works
Tresp, V. (2000) · Neural Computation · doi:10.1162/089976600300014908 · Cited by 411 · see citing works
Accurate On-line Support Vector Regression
Ma, J., Theiler, J. P., & Perkins, S. J. (2003) · Neural Computation · doi:10.1162/089976603322385117 · Cited by 408 · see citing works
Girard, A., Rasmussen, C. E., Candela, J. Q., & Murray‐Smith, R. (2002) · Cited by 384 · see citing works
Operational Space Control: A Theoretical and Empirical Comparison
Nakanishi, J., Cory, R., Mistry, M., Peters, J., & Stefan Schaal, A. (2008) · The International Journal of Robotics Research · doi:10.1177/0278364908091463 · Cited by 381 · see citing works
Self-citationBayesian Gaussian Process Latent Variable Model
Titsias, M. K., & Lawrence, N. D. (2010) · International Conference on Artificial Intelligence and Statistics · Cited by 378 · see citing works
Application of a General Learning Algorithm to the Control of Robotic Manipulators
Miller, W. T., Glanz, F. H., & Kraft, L. G. (1987) · The International Journal of Robotics Research · doi:10.1177/027836498700600207 · Cited by 373 · see citing works
An overview of reservoir computing: theory, applications and implementations
Schrauwen, B., Verstraeten, D., & Campenhout, J. V. (2007) · Ghent University Academic Bibliography (Ghent University) · Cited by 369 · see citing works
PEGASUS: A Policy Search Method for Large MDPs and POMDPsPreprint
Ng, A. Y., & Jordan, M. I. (2013) · arXiv (Cornell University) · doi:10.48550/arxiv.1301.3878 · Cited by 364 · see citing works
Infinite Mixtures of Gaussian Process Experts
Rasmussen, C. E., & Ghahramani, Z. (2001) · Cited by 357 · see citing works
Feedback-Error-Learning Neural Network for Supervised Motor Learning
Kawato, M. (1990) · Elsevier eBooks · doi:10.1016/b978-0-444-88400-8.50047-9 · Cited by 355 · see citing works
Autoregressive model fitting for control
Akaike, H. (1971) · Annals of the Institute of Statistical Mathematics · doi:10.1007/bf02479221 · Cited by 344 · see citing works
GP-BayesFilters: Bayesian filtering using Gaussian process prediction and observation models
Ko, J., & Fox, D. (2009) · Autonomous Robots · doi:10.1007/s10514-009-9119-x · Cited by 342 · see citing works
Persistent excitation in adaptive systems
Narendra, K. S., & Annaswamy, A. M. (1987) · International Journal of Control · doi:10.1080/00207178708933715 · Cited by 333 · see citing works
Autoregressive Model Fitting for Control
Akaike, H. (1998) · Springer series in statistics · doi:10.1007/978-1-4612-1694-0_12 · Cited by 324 · see citing works
Most likely heteroscedastic Gaussian process regression
Kristian Kersting, B., Plagemann, C., Pfaff, P., & Burgard, W. (2007) · International Conference on Machine Learning · doi:10.1145/1273496.1273546 · Cited by 321 · see citing works
Model Learning with Local Gaussian Process Regression
Nguyen-Tuong, D., Seeger, M., & Peters, J. (2009) · Advanced Robotics · doi:10.1163/016918609x12529286896877 · Cited by 316 · see citing works
Self-citationExciting Trajectories for the Identification of Base Inertial Parameters of Robots
Gautier, M., & Khalil, W. (1992) · The International Journal of Robotics Research · doi:10.1177/027836499201100408 · Cited by 310 · see citing works
Prior Knowledge in Support Vector Kernels
Schölkopf, B., Simard, P., Smola, A. J., & Vapnik, V. N. (1998) · Cited by 298 · see citing works
(2000) · Cited by 296 · see citing works
D'Souza, A., Vijayakumar, S., & Schaal, S. (2002) · doi:10.1109/iros.2001.973374 · Cited by 287 · see citing works
Gaussian process model based predictive control
Kocijan, J., Murray‐Smith, R., Rasmussen, C. E., et al. (2004) · Proceedings of the ... American Control Conference/Proceedings of the American Control Conference · doi:10.23919/acc.2004.1383790 · Cited by 279 · see citing works
Analysis of a greedy active learning strategy
Dasgupta, S. (2004) · Cited by 251 · see citing works
Body Schema in Robotics: A Review
Hoffmann, M., Marques, H. G., Arieta, A. H., Sumioka, H., Lungarella, M., & Pfeifer, R. (2010) · IEEE Transactions on Autonomous Mental Development · doi:10.1109/tamd.2010.2086454 · Cited by 242 · see citing works
Scalable Techniques from Nonparametric Statistics for Real Time Robot Learning
Schaal, S., Atkeson, C. G., & Vijayakumar, S. (2002) · Applied Intelligence · doi:10.1023/a:1015727715131 · Cited by 236 · see citing works
Efficient Algorithms for Minimizing Cross Validation Error
Moore, A., & Lee, M. S. (1994) · Elsevier eBooks · doi:10.1016/b978-1-55860-335-6.50031-3 · Cited by 233 · see citing works
Real-time application of neural networks for sensor-based control of robots with vision
Miller, W. T. (1989) · IEEE Transactions on Systems Man and Cybernetics · doi:10.1109/21.35345 · Cited by 232 · see citing works
Likelilood ratio gradient estimation
Glynn, P. W. (1987) · Winter Simulation Conference · doi:10.1145/318371.318612 · Cited by 225 · see citing works
An SVM learning approach to robotic grasping
Pelossof, R., Miller, A. T., Allen, P., & Jebara, T. (2004) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/robot.2004.1308797 · Cited by 219 · see citing works
A neural network model predictive controller
Åkesson, B. M., & Toivonen, H. T. T. (2006) · Journal of Process Control · doi:10.1016/j.jprocont.2006.06.001 · Cited by 215 · see citing works
Modeling and control of robot manipulators
de Oliveira, M., & Jamshidi, M. M. (2001) · Automatica · doi:10.1016/s0005-1098(01)00110-8 · Cited by 211 · see citing works
Sparse probabilistic regression for activity-independent human pose inference
Urtasun, R., & Darrell, T. J. (2008) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2008.4587360 · Cited by 206 · see citing works
Backpropagation-decorrelation: online recurrent learning with O(N) complexity
Steil, J. J. (2005) · IEEE International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn.2004.1380039 · Cited by 206 · see citing works
Proceedings of the European Symposium on Artificial Neural Networks
Tang, T. B., & Murray, A. T. (2006) · Cited by 202 · see citing works
Model reference adaptive control algorithms for industrial robots
NICOSIA, S., & Tomei, P. (1984) · Automatica · doi:10.1016/0005-1098(84)90013-x · Cited by 201 · see citing works
A general algorithm for dynamic feedback linearization of robots with elastic joints
Luca, A. D., & Lucibello, P. (2002) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/robot.1998.677024 · Cited by 194 · see citing works
Goal Babbling Permits Direct Learning of Inverse Kinematics
Rolf, M., Steil, J. J., & Gienger, M. (2010) · IEEE Transactions on Autonomous Mental Development · doi:10.1109/tamd.2010.2062511 · Cited by 179 · see citing works
Using model knowledge for learning inverse dynamics
Nguyen-Tuong, D., & Peters, J. (2010) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/robot.2010.5509858 · Cited by 172 · see citing works
Self-citationSteil, J. J. (2007) · Neural Networks · doi:10.1016/j.neunet.2007.04.011 · Cited by 166 · see citing works
Learning to Control in Operational Space
Peters, J., & Schaal, S. (2008) · The International Journal of Robotics Research · doi:10.1177/0278364907087548 · Cited by 162 · see citing works
Self-citationSchaal, S., & Atkeson, C. G. (2010) · IEEE Robotics & Automation Magazine · doi:10.1109/mra.2010.936957 · Cited by 158 · see citing works
Neural predictive control for a car-like mobile robot
Gu, D., & Hu, H. (2002) · Robotics and Autonomous Systems · doi:10.1016/s0921-8890(02)00172-0 · Cited by 151 · see citing works
A probabilistic model for appearance-based robot localization
Kröse, B., Vlassis, N. A., Bunschoten, R., & Motomura, Y. (2001) · Image and Vision Computing · doi:10.1016/s0262-8856(00)00086-x · Cited by 148 · see citing works
A unifying framework for robot control with redundant DOFs
Peters, J., Mistry, M., Udwadia, F. E., et al. (2007) · Autonomous Robots · doi:10.1007/s10514-007-9051-x · Cited by 141 · see citing works
Self-citationActive Policy Learning for Robot Planning and Exploration under Uncertainty
Martínez-Cantín, R., de Freitas, N., Arnaud Doucet, B., & Castellanos, J. A. (2007) · Robotics: Science and Systems · doi:10.15607/rss.2007.iii.041 · Cited by 141 · see citing works
Feedback error learning and nonlinear adaptive control
Nakanishi, J., & Stefan Schaal, A. (2004) · Neural Networks · doi:10.1016/j.neunet.2004.05.003 · Cited by 138 · see citing works
Neural networks for advanced control of robot manipulators
Patiño, H. D., Carelli, R., & Kuchen, B. (2002) · IEEE Transactions on Neural Networks · doi:10.1109/72.991420 · Cited by 137 · see citing works
A self-tuning multistep predictor application
Keyser, R. M. C. D., & Cauwenberghe, A. R. V. (1981) · Automatica · doi:10.1016/0005-1098(81)90092-3 · Cited by 123 · see citing works
Planning Among Movable Obstacles with Artificial Constraints
Stilman, M., & Kuffner, J. (2008) · The International Journal of Robotics Research · doi:10.1177/0278364908098457 · Cited by 121 · see citing works
Minimax differential dynamic programming: application to a biped walking robot
Morimoto, J., Zeglin, G. J., & Atkeson, C. G. (2003) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros.2003.1248926 · Cited by 115 · see citing works
Shibata, T., & Stefan Schaal, A. (2001) · Neural Networks · doi:10.1016/s0893-6080(00)00084-8 · Cited by 113 · see citing works
Recurrent cerebellar architecture solves the motor-error problem
Porrill, J., Dean, P., & Stone, J. V. (2004) · Proceedings of the Royal Society B Biological Sciences · doi:10.1098/rspb.2003.2658 · Cited by 108 · see citing works
Composite adaptive control with locally weighted statistical learning
Nakanishi, J., Farrell, J. A., & Schaal, S. (2004) · Neural Networks · doi:10.1016/j.neunet.2004.08.009 · Cited by 108 · see citing works
Thrun, S., & Mitchell, T. M. (1995) · Springer eBooks · doi:10.1007/978-3-642-79629-6_7 · Cited by 102 · see citing works
Multi-objective Model Predictive Control
Nakayama, H., Yun, Y., & SHIRAKAWA, M. (2009) · Lecture notes in economics and mathematical systems · doi:10.1007/978-3-642-04045-0_24 · Cited by 100 · see citing works
Planning Among Movable Obstacles with Artificial Constraints
Stilman, M., & Kuffner, J. (2008) · Springer tracts in advanced robotics · doi:10.1007/978-3-540-68405-3_8 · Cited by 94 · see citing works
Evaluation of Adaptive Mixtures of Competing Experts
Nowlan, S. J., & Hinton, G. E. (1990) · neural information processing systems · Cited by 94 · see citing works
Fuzzy Model Reference Learning Control
Layne, J. R., & Passino, K. M. (1996) · Journal of Intelligent & Fuzzy Systems · doi:10.3233/ifs-1996-4103 · Cited by 94 · see citing works
Learning locomotion over rough terrain using terrain templates
Kalakrishnan, M., Buchli, J., Pástor, P., & Stefan Schaal, A. (2009) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros.2009.5354701 · Cited by 87 · see citing works
Semiparametric Support Vector and Linear Programming Machines
Smola, A. J., Frieß, T.-T., & Schölkopf, B. (1998) · Cited by 80 · see citing works
Learning predictive terrain models for legged robot locomotion
Plagemann, C., Mischke, S., Prentice, S., Kristian Kersting, B., Roy, N., & Burgard, W. (2008) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros.2008.4651026 · Cited by 78 · see citing works
Incremental online sparsification for model learning in real-time robot control
Nguyen-Tuong, D., & Peters, J. (2011) · Neurocomputing · doi:10.1016/j.neucom.2010.06.033 · Cited by 75 · see citing works
Self-citationMovement Generation with Circuits of Spiking Neurons
Joshi, P., & Maass, W. (2005) · Neural Computation · doi:10.1162/0899766054026684 · Cited by 75 · see citing works
Robustness of multipredictor adaptive regulators: MUSMAR
Mosca, E., Zappa, G., & Lemos, J. M. (1989) · Automatica · doi:10.1016/0005-1098(89)90095-2 · Cited by 73 · see citing works
Recognition of manipulated objects by motor learning with modular architecture networks
Gomi, H., & Kawato, M. (1993) · Neural Networks · doi:10.1016/s0893-6080(05)80053-x · Cited by 69 · see citing works
Butz, M. V., Herbort, O., & Hoffmann, J. (2007) · Psychological Review · doi:10.1037/0033-295x.114.4.1015 · Cited by 69 · see citing works
Nonparametric Representation of Policies and Value Functions: A Trajectory-Based Approach
Atkeson, C. G., & Morimoto, J. (2018) · doi:10.1184/r1/6556976 · Cited by 68 · see citing works
Likelihood Ratio Gradient Estimation: An Overview
Glynn, P. W. (1987) · doi:10.21236/ada197085 · Cited by 68 · see citing works
Learning to predict how rigid objects behave under simple manipulation
Kopicki, M., Zurek, S., Stolkin, R., et al. (2011) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra.2011.5980295 · Cited by 63 · see citing works
A Bayesian Approach to Nonlinear Parameter Identification for Rigid Body Dynamics
Ting, J.-A., Mistry, M., Peters, J., et al. (2006) · doi:10.15607/rss.2006.ii.032 · Cited by 62 · see citing works
Self-citationLEARNING-BASED MODEL PREDICTIVE CONTROL FOR MARKOV DECISION PROCESSES
Negenborn, R. R., De Schutter, B., Wiering, M. A., & Hellendoorn, H. (2005) · IFAC Proceedings Volumes · doi:10.3182/20050703-6-cz-1902.00280 · Cited by 59 · see citing works
Sparse incremental learning for interactive robot control policy estimation
Grollman, D. H., & Jenkins, O. C. (2008) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/robot.2008.4543716 · Cited by 59 · see citing works
Slip Prediction Using Visual Information
Angelova, A., Matthies, L. H., Helmick, D. M., et al. (2006) · doi:10.15607/rss.2006.ii.014 · Cited by 55 · see citing works
Fast, Robust Adaptive Control by Learning only Forward Models
Moore, A. W. (1991) · neural information processing systems · Cited by 53 · see citing works
Sparse Online Greedy Support Vector Regression
Engel, Y., Mannor, S., & Meir, R. (2002) · Lecture notes in computer science · doi:10.1007/3-540-36755-1_8 · Cited by 52 · see citing works
Model predictive neural control with applications to a 6 DOF helicopter model
Wan, E. A., & Bogdanov, A. A. (2001) · Proceedings of the ... American Control Conference/Proceedings of the American Control Conference · doi:10.1109/acc.2001.945592 · Cited by 51 · see citing works
Body schema acquisition through active learning
Martínez-Cantín, R., Lopes, M. C., & Montesano, L. (2010) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/robot.2010.5509406 · Cited by 47 · see citing works
Incorporating Prior Knowledge on Features into Learning
Krupka, E., & Tishby, N. (2007) · Cited by 43 · see citing works
Closing the learning-planning loop with predictive state representations
Boots, B., Siddiqi, S. M., & Gordon, G. J. (2010) · Adaptive Agents and Multi-Agents Systems · doi:10.5555/1838206.1838386 · Cited by 42 · see citing works
Reinhart, F., & Steil, J. J. (2009) · IEEE-RAS International Conference on Humanoid Robots (Humanoids) · doi:10.1109/ichr.2009.5379558 · Cited by 42 · see citing works
Local Dimensionality Reduction for Non-Parametric Regression
Hoffmann, H., Schaal, S., & Vijayakumar, S. (2009) · Neural Processing Letters · doi:10.1007/s11063-009-9098-0 · Cited by 39 · see citing works
Real-Time Local GP Model Learning
Nguyen-Tuong, D., Seeger, M., & Peters, J. (2009) · Studies in computational intelligence · doi:10.1007/978-3-642-05181-4_9 · Cited by 37 · see citing works
Self-citationUnsupervised body scheme learning through self-perception
Sturm, J., Plagemann, C., & Burgard, W. (2008) · 2008 IEEE International Conference on Robotics and Automation · doi:10.1109/robot.2008.4543718 · Cited by 35 · see citing works
Efficient exploration and learning of whole body kinematics
Rolf, M., Steil, J. J., & Gienger, M. (2009) · IEEE International Conference on Development and Learning (ICDL) · doi:10.1109/devlrn.2009.5175522 · Cited by 34 · see citing works
Dynamic Path Planning for a 7-DOF Robot Arm
Klanke, S., Lebedev, D., Haschke, R., et al. (2006) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros.2006.281798 · Cited by 34 · see citing works
Control of redundant robots using learned models: An operational space control approach
Salaün, C., Padois, V., & Sigaud, O. (2009) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros.2009.5354438 · Cited by 29 · see citing works
Learning discontinuities with products-of-sigmoids for switching between local models
Toussaint, M., & Vijayakumar, S. (2005) · International Conference on Machine Learning · doi:10.1145/1102351.1102465 · Cited by 27 · see citing works
Evidence for a Forward Dynamics Model in Human Adaptive Motor Control
Bhushan, N., & Shadmehr, R. (1998) · neural information processing systems · Cited by 27 · see citing works
SILICON SUPPORT VECTOR MACHINE WITH ON-LINE LEARNING
Genov, R., Chakrabartty, S., & Cauwenberghs, G. (2003) · International Journal of Pattern Recognition and Artificial Intelligence · doi:10.1142/s0218001403002472 · Cited by 26 · see citing works
Reinhart, F., & Steil, J. J. (2008) · ECSIS Symposium on Learning and Adaptive Behaviors for Robotic Systems (LAB-RS) · doi:10.1109/lab-rs.2008.17 · Cited by 26 · see citing works
Simulation control of a biped robot with Support Vector Regression
Ferreira, J. P., Crisostomo, M. M., Coimbra, A. P., et al. (2007) · IEEE International Symposium on Intelligent Signal Processing, · doi:10.1109/wisp.2007.4447538 · Cited by 25 · see citing works
Learning Global Direct Inverse Kinematics
DeMers, D., & Kreutz-Delgado, K. (1991) · Cited by 25 · see citing works
Active learning using mean shift optimization for robot grasping
Kroemer, O., Detry, R. J., Piater, J., & Peters, J. (2009) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros.2009.5354345 · Cited by 25 · see citing works
Self-citationRapid learning of humanoid body schemas with Kinematic Bézier Maps
Ulbrich, S., de Angulo, V. R., Asfour, T., Carme Torras, A., & Dillmann, R. (2009) · IEEE-RAS International Conference on Humanoid Robots (Humanoids) · doi:10.1109/ichr.2009.5379543 · Cited by 24 · see citing works
Local Online Support Vector Regression for Learning Control
Choi, Y., Cheong, S.-Y., & Schweighofer, N. (2007) · IEEE International Symposium on Computational Intelligence in Robotics and Automation (CIRA) · doi:10.1109/cira.2007.382883 · Cited by 24 · see citing works
Adaptive autonomous control using online value iteration with gaussian processes
Rottmann, A., & Burgard, W. (2009) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/robot.2009.5152660 · Cited by 23 · see citing works
Learning Multiple Models of Non-linear Dynamics for Control Under Varying Contexts
Petkos, G., Toussaint, M., & Vijayakumar, S. (2006) · Lecture notes in computer science · doi:10.1007/11840817_93 · Cited by 22 · see citing works
A Unifying Methodology for Robot Control with Redundant DOFs
Peters, J., Mistry, M., Udwadia, F. E., et al. (2008) · Cited by 19 · see citing works
Self-citationHerbort, O., Butz, M. V., & Pedersen, G. K. M. (2009) · Studies in computational intelligence · doi:10.1007/978-3-642-05181-4_5 · Cited by 18 · see citing works
Adaptive neural network control of flexible joint robots based on feedback linearization
Ge, S. S., Lee, T. H., & Tan, E. G. (1998) · International Journal of Systems Science · doi:10.1080/00207729808929555 · Cited by 17 · see citing works
Experiments in nonlinear adaptive control
Burdet, E., Sprenger, B., & Codourey, A. (2002) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/robot.1997.620092 · Cited by 17 · see citing works
Otani, K. (1993) · Medical Entomology and Zoology · Cited by 17 · see citing works
Prediction learning in robotic manipulation
Kopicki, M. (2010) · University of Birmingham Institutional Research Archive (University of Birmingham) · Cited by 16 · see citing works
Incremental Sparsification for Real-time Online Model Learning
Nguyen-Tuong, D., & Peters, J. (2010) · Cited by 16 · see citing works
Self-citationAttractor-based computation with reservoirs for online learning of inverse kinematics
Reinhart, F., & Steil, J. J. (2009) · Cited by 15 · see citing works
Bayesian Kernel Shaping for Learning Control
Ting, J.-A., Kalakrishnan, M., Vijayakumar, S., et al. (2008) · ERA · Cited by 14 · see citing works
A learning framework for generic sensory-motor maps
Lopes, M. C., & Damas, B. (2007) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros.2007.4399566 · Cited by 13 · see citing works
Closing the Learning-Planning Loop with Predictive State Representations
Boots, B., Siddiqi, S. M., & Gordon, G. (2010) · doi:10.15607/rss.2010.vi.036 · Cited by 13 · see citing works
Kernel carpentry for online regression using randomly varying coefficient model
Edakunni, N. U., Schaal, S., & Vijayakumar, S. (2007) · ERA · Cited by 12 · see citing works
Successive one-step-ahead predictions in multiple model predictive control
Chow, C.-M., Kuznetsov, A. G., & CLARKE, D. W. (1998) · International Journal of Systems Science · doi:10.1080/00207729808929589 · Cited by 11 · see citing works
Neural-Network Inverse Dynamic Online Learning Control on Physical Exoskeleton
Cao, H., Yin, Y., Du, D., et al. (2006) · Lecture notes in computer science · doi:10.1007/11893295_77 · Cited by 9 · see citing works
Quiñonero-Candela, J., Rasmussen, C. E., & Williams, C. (2007) · Cited by 8 · see citing works
Steffen, J., Klanke, S., Vijayakumar, S., et al. (2009) · Cited by 7 · see citing works
Model Reference Adaptive Control for Actuators of a Biped Robot Locomotion
Vempaty, P. K., Cheok, K. C., & Loh, R. N. K. (2009) · Cited by 7 · see citing works
Heteroscedastic Gaussian Process Regression for Modeling Range Sensors in Mobile Robotics
Plagemann, C., Kersting, K., Pfaff, P., et al. (2005) · Cited by 6 · see citing works
Differential Dynamic Programming
Mays, L. W. (2018) · CRC Press eBooks · doi:10.1201/9781351075206-9 · Cited by 6 · see citing works
Skočaj, D., Kristan, M., Vrečko, A., et al. (2010) · Cognitive systems monographs · doi:10.1007/978-3-642-11694-0_7 · Cited by 4 · see citing works
Closing the Learning-Planning Loop with Predictive State Representations (Extended Abstract)
Boots, B., Siddiqi, S. M., & Gordon, G. J. (2010) · Cited by 4 · see citing works
A Long-Range Adaptive Controller for Robot Manipulators
Coito, F. J., & Lemos, J. M. (1991) · The International Journal of Robotics Research · doi:10.1177/027836499101000607 · Cited by 3 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
Domain randomization for transferring deep neural networks from simulation to the real world
Tobin, J., Fong, R., Ray, A., Schneider, J., Zaremba, W., & Abbeel, P. (2017) · 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros.2017.8202133 · Cited by 3,124 · see citing works
“Even with strong system identification, the real world has unmodeled physical effects like nonrigidity, gear backlash, wear-and-tear, and fluid dynamics that are not captured by current physics simulators (though learning techniques may help bridge this gap [33]).”
BackgroundAn algorithmic perspective on imitation learning
Osa, T., Pajarinen, J., Neumann, G., Bagnell, J. A., Abbeel, P., & Peters, J. (2018) · Foundations and Trends in Robotics · doi:10.1561/2300000053 · Cited by 405 · see citing works
Self-citationSoftware engineering challenges for machine learning applications: A literature review
Kumeno, F. (2020) · Intelligent Decision Technologies · doi:10.3233/idt-190160 · Cited by 81 · see citing works
Cascaded Gaussian Processes for Data-efficient Robot Dynamics Learning
Rezaei-Shoshtari, S., Meger, D., & Sharf, I. (2019) · 2019 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros40897.2019.8968107 · Cited by 19 · see citing works
- Authors
- Nguyen-Tuong, D., & Peters, J.
- Affiliations
- Max Planck Institute for Biological Cybernetics (DE)
- Max Planck Society (DE)
- Corresponding author
- Nguyen-Tuong, Duy
- Peters, Jan
- ORCID
- Peters, Jan
- Venue
- Cognitive Processing
- Published
- 2011-04-13
- Volume
- 12
- Issue
- 4
- Pages
- 319-340
- ISSN
- 1612-4782 (ISSN-L)
- 1612-4790
- OpenAlex
- W1997543377
- Cited by
- 503
- References
- 196
- Citations by year
- 2026: 18 · 2025: 25 · 2024: 30 · 2023: 40 · 2022: 32 · 2021: 35 · 2020: 58 · 2019: 49 · 2018: 45 · 2017: 51 · 2016: 31 · 2015: 27 · 2014: 31 · 2013: 18 · 2012: 11
- FWCI
- 3.41
- Type
- journal article
- Language
- en
- Open access
- closed
- Licence
- www.springer.com/tdm (tdm)
- Locations
- Cognitive Processing (publishedVersion)
- PubMed (publishedVersion)
- Max Planck Institute for Plasma Physics (submittedVersion)
- Max Planck Digital Library (submittedVersion)
- MPG.PuRe (Max Planck Society) (submittedVersion)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Reinforcement Learning in Robotics
- Physical Sciences › Engineering › Control and Systems Engineering › Robot Manipulation and Learning
- Physical Sciences › Engineering › Biomedical Engineering › Robotic Locomotion and Control
- Keywords
- model learning, robot control, robot kinematics, robot dynamics, internal models, learning architecture
- MeSH terms
- Algorithms, Artificial Intelligence, Humans, Learning, Models, Psychological, Problem Solving, Robotics
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
