The DOI matches its Crossref record; 51 of the 51 references it cites were found in a registry or an index.
Gaussian Processes for Data-Efficient Learning in Robotics and Control
Open accessAbstract
Autonomous learning has been a promising direction in control and robotics for more than a decade since data-driven learning allows to reduce the amount of engineering knowledge, which is otherwise required. However, autonomous reinforcement learning (RL) approaches typically require many interactions with the system to learn controllers, which is a practical limitation in real systems, such as robots, where many interactions can be impractical and time consuming. To address this problem, current learning approaches typically require task-specific knowledge in form of expert demonstrations, realistic simulators, pre-shaped policies, or specific knowledge about the underlying dynamics. In this paper, we follow a different approach and speed up learning by extracting more information from data. In particular, we learn a probabilistic, non-parametric Gaussian process transition model of the system. By explicitly incorporating model uncertainty into long-term planning and controller learning our approach reduces the effects of model errors, a key problem in model-based learning. Compared to state-of-the art RL our model-based policy search method achieves an unprecedented speed of learning. We demonstrate its applicability to autonomous learning in real robot and control tasks.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Reinforcement Learning: An Introduction
Sutton, R. S., & Barto, A. G. (1998) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.1998.712192 · Cited by 27,225
Reinforcement Learning: An Introduction
(2005) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.2004.842673 · Cited by 24,496
Table of Integrals, Series, and Products
(1980) · Elsevier eBooks · doi:10.1016/c2013-0-10754-4 · Cited by 18,782
Gaussian Processes for Machine Learning
Rasmussen, C. E., & Williams, C. K. I. (2005) · doi:10.7551/mitpress/3206.001.0001 · Cited by 11,705
Self-citationTable of Integrals, Series and Products.
L., Y., Gradshteyn, I. S., & Ryzhik, I. M. (1981) · Mathematics of Computation · doi:10.2307/2007757 · Cited by 8,436
Information theory, inference, and learning algorithms
(2004) · Choice Reviews Online · doi:10.5860/choice.41-5949 · Cited by 6,362
Watkins, C. (1989) · OpenGrey (Institut de l'Information Scientifique et Technique) · doi:10.17863/cam.132164 · Cited by 5,297
Åström, K. J., & Wittenmark, B. (1989) · Cited by 5,020
Information Theory, Inference, and Learning Algorithms
(2004) · Kybernetes · doi:10.1108/03684920410534506 · Cited by 4,937
Brochu, E., Cora, V. M., & de Freitas, N. (2010) · arXiv (Cornell University) · doi:10.48550/arxiv.1012.2599 · Cited by 2,011
Information-Theoretic Regret Bounds for Gaussian Process Optimization in the Bandit Setting
Srinivas, N., Krause, A., Kakade, S. M., & Seeger, M. W. (2012) · IEEE Transactions on Information Theory · doi:10.1109/tit.2011.2182033 · Cited by 1,850
Bayesian Experimental Design: A Review
Chaloner, K., & Verdinelli, I. (1995) · Statistical Science · doi:10.1214/ss/1177009939 · Cited by 1,839
Petersen, K. B., & Pedersen, M. S. (2012) · doi:10.5281/zenodo.8002926 · Cited by 1,434
Sparse Gaussian Processes using Pseudo-inputs
Snelson, E., & Ghahramani, Z. (2005) · Cited by 1,415
Using confidence bounds for exploitation-exploration trade-offs
Auer, P. L. (2003) · Journal of Machine Learning Research · doi:10.5555/944919.944941 · Cited by 1,229
PILCO: A Model-Based and Data-Efficient Approach to Policy Search
Deisenroth, M. P., & Rasmussen, C. E. (2011) · Cambridge University Engineering Department Publications Database · Cited by 1,136
Self-citationRobust Controller Design Using Normalized Coprime Factor Plant Descriptions
(1990) · Lecture Notes in Control and Information Sciences · doi:10.1007/bfb0043199 · Cited by 1,126
Gaussian Process Dynamical Models for Human Motion
Wang, J. M., Fleet, D. J., & Hertzmann, A. (2008) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2007.1167 · Cited by 1,020
Reinforcement Learning in Continuous Time and Space
Doya, K. (2000) · Neural Computation · doi:10.1162/089976600300015961 · Cited by 960
Reinforcement learning of motor skills with policy gradients
Peters, J., & Schaal, S. (2008) · Neural Networks · doi:10.1016/j.neunet.2008.02.003 · Cited by 831
A Survey on Policy Search for Robotics
Deisenroth, M. P. (2011) · Foundations and Trends in Robotics · doi:10.1561/2300000021 · Cited by 798
Self-citationPolicy Gradient Methods for Robotics
Peters, J., & Schaal, S. (2006) · 2006 IEEE/RSJ International Conference on Intelligent Robots and Systems · doi:10.1109/iros.2006.282564 · Cited by 498
(2011) · Encyclopedia of Machine Learning · doi:10.1007/978-0-387-30164-8_447 · Cited by 400
PEGASUS: A Policy Search Method for Large MDPs and POMDPs
Ng, A. Y., & Jordan, M. I. (2013) · arXiv (Cornell University) · doi:10.48550/arxiv.1301.3878 · Cited by 360
Autonomous Helicopter Flight via Reinforcement Learning
Kim, H. J., Jordan, M. I., Sastry, S., & Ng, A. Y. (2003) · neural information processing systems · Cited by 287
Autonomous helicopter control using reinforcement learning policy search methods
Bagnell, J. A., & Schneider, J. G. (2002) · Proceedings 2001 ICRA. IEEE International Conference on Robotics and Automation (Cat. No.01CH37164) · doi:10.1109/robot.2001.932842 · Cited by 276
A comparison of direct and model-based reinforcement learning
Atkeson, C. G., & Santamaria, J. C. (2002) · Proceedings of International Conference on Robotics and Automation · doi:10.1109/robot.1997.606886 · Cited by 239
Practical Bayesian optimization
Lizotte, D. J. (2008) · ERA: Education and Research Archive (University of Alberta) · doi:10.7939/r3-379f-gw15 · Cited by 215
Using inaccurate models in reinforcement learning
Abbeel, P., Quigley, M., & Ng, A. Y. (2006) · Proceedings of the 23rd international conference on Machine learning - ICML '06 · doi:10.1145/1143844.1143845 · Cited by 207
Learning to Control a Low-Cost Manipulator using Data-Efficient Reinforcement Learning
Deisenroth, M., Rasmussen, C., & Fox, D. (2011) · Robotics: Science and Systems VII · doi:10.15607/rss.2011.vii.008 · Cited by 205
Self-citationGaussian Process Training with Input Noise
McHutchon, A., & Rasmussen, C. E. (2011) · Cited by 205
Self-citationGaussian Processes and Reinforcement Learning for Identification and Control of an Autonomous Blimp
Ko, J., Klein, D. J., Fox, D., & Haehnel, D. (2007) · Proceedings 2007 IEEE International Conference on Robotics and Automation · doi:10.1109/robot.2007.363075 · Cited by 196
Self-citationGaussian process dynamic programming
Deisenroth, M. P., Rasmussen, C. E., & Peters, J. (2009) · Neurocomputing · doi:10.1016/j.neucom.2008.12.019 · Cited by 192
Self-citationThe loss function of sensorimotor learning
Körding, K. P., & Wolpert, D. M. (2004) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.0308394101 · Cited by 191
Adaptive Dual Control Methods: An Overview
Wittenmark, B. (1995) · IFAC Proceedings Volumes · doi:10.1016/s1474-6670(17)45327-4 · Cited by 180
Bayes meets bellman: the Gaussian process approach to temporal difference learning
Engel, Y., Mannor, S., & Meir, R. (2003) · Cited by 174
Efficient Reinforcement Learning using Gaussian Processes
Deisenroth, M. P. (2010) · Repository KITopen (Karlsruhe Institute of Technology) · doi:10.5445/ksp/1000019799 · Cited by 163
Self-citationCandela, J. Q., Girard, A., Larsen, J. O., & Rasmussen, C. E. (2003) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp.2003.1202463 · Cited by 135
Self-citationData-Efficient Generalization of Robot Skills with Contextual Policy Search
Kupcsik, A., Deisenroth, M., Peters, J., & Neumann, G. (2013) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v27i1.8546 · Cited by 92
Self-citationState-Space Inference and Learning with Gaussian Processes
Turner, R. D., Deisenroth, M. P., & Rasmussen, C. E. (2010) · MPG.PuRe (Max Planck Society) · Cited by 81
Self-citationReinforcement Learning Using Neural Networks, with Applications to Motor Control
Coulom, R. (2002) · HAL (Le Centre pour la Communication Scientifique Directe) · Cited by 80
Exploiting Model Uncertainty Estimates for Safe Dynamic Control Learning
Schneider, J. (1996) · Cited by 75
GP-BayesFilters: Bayesian filtering using Gaussian process prediction and observation models
Ko, J., & Fox, D. (2008) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros.2008.4651188 · Cited by 69
Self-citationLearning to Control a Low-Cost Manipulator Using Data-Efficient Reinforcement Learning
Deisenroth, M. P., Rasmussen, C. E., & Fox, D. (2012) · The MIT Press eBooks · doi:10.7551/mitpress/9481.003.0013 · Cited by 60
Self-citationGaussian processes for global optimization
Osborne, M. A., Garnett, R., & Roberts, S. J. (2009) · Oxford University Research Archive (ORA) (University of Oxford) · Cited by 58
Model-based imitation learning by probabilistic trajectory matching
Englert, P. A. J., Paraschos, A., Peters, J., & Deisenroth, M. P. (2013) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra.2013.6630832 · Cited by 52
Self-citationProbabilistic model-based imitation learning
Englert, P. A. J., Paraschos, A., Deisenroth, M. P., & Peters, J. (2013) · Adaptive Behavior · doi:10.1177/1059712313491614 · Cited by 40
Self-citationGaussian Processes in Reinforcement Learning
Rasmussen, C. E., Kuß, M., Thrun, S., Saul, L. K., & Schölkopf, B. (2003) · Cited by 39
Self-citationVariational Bayesian learning of nonlinear hidden state-space models for model predictive control
Raiko, T., & Tornio, M. (2009) · Neurocomputing · doi:10.1016/j.neucom.2009.06.009 · Cited by 35
(2012) · Harvard University Press eBooks · doi:10.2307/j.ctt2jbq07.6 · Cited by 35
Model-free off-policy reinforcement learning in continuous environment
Wawrzyński, P., & Pacut, A. (2005) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn.2004.1380086 · Cited by 13
Cited by
Mathematics for Machine Learning
Deisenroth, M. P., Faisal, A. A., & Ong, C. S. (2020) · Cambridge University Press eBooks · doi:10.1017/9781108679930 · Cited by 573
Self-citationDeep reinforcement learning in a handful of trials using probabilistic dynamics models
Chua, K., Calandra, R., McAllister, R., & Levine, S. (2018) · arXiv · doi:10.48550/arxiv.1805.12114 · Cited by 525
An algorithmic perspective on imitation learning
Osa, T., Pajarinen, J., Neumann, G., Bagnell, J. A., Abbeel, P., & Peters, J. (2018) · Foundations and Trends in Robotics · doi:10.1561/2300000053 · Cited by 405
Reinforcement learning in robotic applications: a comprehensive survey
Singh, B., Kumar, R., & Singh, V. P. (2022) · Artificial Intelligence Review · doi:10.1007/s10462-021-09997-9 · Cited by 392
An Algorithmic Perspective on Imitation Learning
Osa, T., Pajarinen, J., Neumann, G., Bagnell, J. A., Abbeel, P., & Peters, J. (2018) · now publishers, Inc. eBooks · doi:10.1561/9781680834116 · Cited by 368
Metadata
- Authors
- Deisenroth, M. P., Fox, D., & Rasmussen, C. E.
- Venue
- IEEE Transactions on Pattern Analysis and Machine Intelligence
- Published
- 2015-02
- Volume
- 37
- Issue
- 2
- Pages
- 408-423
- ISSN
- 0162-8828 (ISSN-L)
- 2160-9292
- OpenAlex
- W2018705428
- Cited by
- 659
- References
- 51
- FWCI
- 26.87
- Type
- journal article
- Language
- en
- Open access
- bronze
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.