The DOI matches its Crossref record; 51 of the 51 references it cites were found in a registry or an index.
Gaussian Processes for Data-Efficient Learning in Robotics and Control
Open accessAbstract
Autonomous learning has been a promising direction in control and robotics for more than a decade since data-driven learning allows to reduce the amount of engineering knowledge, which is otherwise required. However, autonomous reinforcement learning (RL) approaches typically require many interactions with the system to learn controllers, which is a practical limitation in real systems, such as robots, where many interactions can be impractical and time consuming. To address this problem, current learning approaches typically require task-specific knowledge in form of expert demonstrations, realistic simulators, pre-shaped policies, or specific knowledge about the underlying dynamics. In this paper, we follow a different approach and speed up learning by extracting more information from data. In particular, we learn a probabilistic, non-parametric Gaussian process transition model of the system. By explicitly incorporating model uncertainty into long-term planning and controller learning our approach reduces the effects of model errors, a key problem in model-based learning. Compared to state-of-the art RL our model-based policy search method achieves an unprecedented speed of learning. We demonstrate its applicability to autonomous learning in real robot and control tasks.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Reinforcement Learning: An Introduction
Sutton, R. S., & Barto, A. G. (1998) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.1998.712192 · Cited by 27,225
Reinforcement Learning: An Introduction
(2005) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.2004.842673 · Cited by 24,496
Table of Integrals, Series, and Products
(1980) · Elsevier eBooks · doi:10.1016/c2013-0-10754-4 · Cited by 18,782
Gaussian Processes for Machine Learning
Rasmussen, C. E., & Williams, C. K. I. (2005) · doi:10.7551/mitpress/3206.001.0001 · Cited by 11,705
Self-citationTable of Integrals, Series and Products.
L., Y., Gradshteyn, I. S., & Ryzhik, I. M. (1981) · Mathematics of Computation · doi:10.2307/2007757 · Cited by 8,436
Information theory, inference, and learning algorithms
(2004) · Choice Reviews Online · doi:10.5860/choice.41-5949 · Cited by 6,362
Watkins, C. (1989) · OpenGrey (Institut de l'Information Scientifique et Technique) · doi:10.17863/cam.132164 · Cited by 5,297
Åström, K. J., & Wittenmark, B. (1989) · Cited by 5,020
Information Theory, Inference, and Learning Algorithms
(2004) · Kybernetes · doi:10.1108/03684920410534506 · Cited by 4,937
Brochu, E., Cora, V. M., & de Freitas, N. (2010) · arXiv (Cornell University) · doi:10.48550/arxiv.1012.2599 · Cited by 2,011
Cited by
Mathematics for Machine Learning
Deisenroth, M. P., Faisal, A. A., & Ong, C. S. (2020) · Cambridge University Press eBooks · doi:10.1017/9781108679930 · Cited by 573
Self-citationDeep reinforcement learning in a handful of trials using probabilistic dynamics models
Chua, K., Calandra, R., McAllister, R., & Levine, S. (2018) · arXiv · doi:10.48550/arxiv.1805.12114 · Cited by 525
An algorithmic perspective on imitation learning
Osa, T., Pajarinen, J., Neumann, G., Bagnell, J. A., Abbeel, P., & Peters, J. (2018) · Foundations and Trends in Robotics · doi:10.1561/2300000053 · Cited by 405
“The marginal distribution p(xt+1|µt, Σt) in (3.75) can be approximated by a Gaussian distribution by following the results from [Deisenroth and Rasmussen, 2011, Deisenroth et al., 2013a].”
BackgroundMethodInfluentialReinforcement learning in robotic applications: a comprehensive survey
Singh, B., Kumar, R., & Singh, V. P. (2022) · Artificial Intelligence Review · doi:10.1007/s10462-021-09997-9 · Cited by 392
An Algorithmic Perspective on Imitation Learning
Osa, T., Pajarinen, J., Neumann, G., Bagnell, J. A., Abbeel, P., & Peters, J. (2018) · now publishers, Inc. eBooks · doi:10.1561/9781680834116 · Cited by 368
Benchmarking Model-Based Reinforcement Learning
Wang, T., Bao, X., Clavera, I., Hoang, J., Wen, Y., Langlois, E., Zhang, S., Zhang, G., Abbeel, P., & Ba, J. (2019) · arXiv · doi:10.48550/arxiv.1907.02057 · Cited by 226
Salvato, E., Fenu, G., Medvet, E., & Pellegrino, F. A. (2021) · IEEE Access · doi:10.1109/access.2021.3126658 · Cited by 200
“…although the potential usefulness of PILCO and Meta-RL to cope with the RG, the former underestimates state uncertainty at future time steps [119], thus possibly leading to a decrease in performance; the latter, on the other hand, is computational demanding and needs an high number of…”
BackgroundA Survey on Policy Search Algorithms for Learning Robot Controllers in a Handful of Trials
Chatzilygeroudis, K., Vassiliades, V., Stulp, F., Calinon, S., & Mouret, J.-B. (2020) · IEEE Transactions on Robotics · doi:10.1109/tro.2019.2958211 · Cited by 160
Silver, T., Allen, K., Tenenbaum, J., & Kaelbling, L. (2018) · arXiv · doi:10.48550/arxiv.1812.06298 · Cited by 113
Meta Reinforcement Learning with Latent Variable Gaussian Processes
Sæmundsson, S., Hofmann, K., & Deisenroth, M. P. (2018) · arXiv · doi:10.48550/arxiv.1803.07551 · Cited by 106
Self-citationAVID: Learning Multi-Stage Tasks via Pixel-Level Translation of Human Videos
Smith, L., Dhawan, N., Zhang, M., Abbeel, P., & Levine, S. (2020) · Robotics: Science and Systems XVI · doi:10.15607/rss.2020.xvi.024 · Cited by 102
(2020) · RWTH Publications (RWTH Aachen) · Cited by 68
Model-Based Policy Search Using Monte Carlo Gradient Estimation With Real Systems Application
Amadio, F., Dalla Libera, A., Antonello, R., Nikovski, D., Carli, R., & Romeres, D. (2022) · IEEE Transactions on Robotics · doi:10.1109/tro.2022.3184837 · Cited by 29
Cascaded Gaussian Processes for Data-efficient Robot Dynamics Learning
Rezaei-Shoshtari, S., Meger, D., & Sharf, I. (2019) · 2019 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros40897.2019.8968107 · Cited by 19
Metadata
- Authors
- Deisenroth, M. P., Fox, D., & Rasmussen, C. E.
- Venue
- IEEE Transactions on Pattern Analysis and Machine Intelligence
- Published
- 2015-02
- Volume
- 37
- Issue
- 2
- Pages
- 408-423
- ISSN
- 0162-8828 (ISSN-L)
- 2160-9292
- OpenAlex
- W2018705428
- Cited by
- 659
- References
- 51
- FWCI
- 26.87
- Type
- journal article
- Language
- en
- Open access
- bronze
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.