The DOI matches its Crossref record; 43 of the 43 references it cites were found in a registry or an index.
Policy Gradient Methods for Robotics
Abstract
The acquisition and improvement of motor skills and control policies for robotics from trial and error is of essential importance if robots should ever leave precisely pre-structured environments. However, to date only few existing reinforcement learning methods have been scaled into the domains of high-dimensional robots such as manipulator, legged or humanoid robots. Policy gradient methods remain one of the few exceptions and have found a variety of applications. Nevertheless, the application of such methods is not without peril if done in an uninformed manner. In this paper, we give an overview on learning with policy gradient methods for robotics with a strong focus on recent advances in the field. We outline previous applications to robotics and show how the most recently developed methods can significantly improve learning performance. Finally, we evaluate our most promising algorithm in the application of hitting a baseball with an anthropomorphic arm
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Practical Methods of Optimization
Fletcher, R. (2000) · Wiley eBooks · doi:10.1002/9781118723203 · Cited by 7,605
(1999) · Cited by 5,147
Simple Statistical Gradient-Following Algorithms for Connectionist Reinforcement Learning
Williams, R. J. (1992) · Machine Learning · doi:10.1007/bf00992696 · Cited by 3,416
Natural Gradient Works Efficiently in Learning
Amari, S.-I. (1998) · Neural Computation · doi:10.1162/089976698300017746 · Cited by 2,681
Introduction to Stochastic Search and Optimization: Estimation, Simulation, and Control
Spall, J. C. (2003) · Wiley eBooks · doi:10.1002/0471722138 · Cited by 1,890
Modelling and Control of Robot Manipulators
Sciavicco, L., & Siciliano, B. (2000) · Advanced Textbooks in Control and Signal Processing · doi:10.1007/978-1-4471-0449-0 · Cited by 1,701
On Choosing and Bounding Probability Metrics
Gibbs, A. L., & Su, F. E. (2002) · International Statistical Review · doi:10.1111/j.1751-5823.2002.tb00178.x · Cited by 1,163
INTRODUCTION TO STOCHASTIC SEARCH AND OPTIMIZATION: ESTIMATION, SIMULATION, AND CONTROL
Nowak, W. (2012) · Medical Entomology and Zoology · Cited by 1,033
Simple Statistical Gradient-Following Algorithms for Connectionist Reinforcement Learning
Williams, R. J. (1992) · Reinforcement Learning · doi:10.1007/978-1-4615-3618-5_2 · Cited by 903
Kakade, S. M. (2001) · ScholarlyCommons (University of Pennsylvania) · Cited by 775
Cited by
Reinforcement learning of motor skills with policy gradients
Peters, J., & Schaal, S. (2008) · Neural Networks · doi:10.1016/j.neunet.2008.02.003 · Cited by 831
Self-citationA Survey on Policy Search for Robotics
Deisenroth, M. P. (2011) · Foundations and Trends in Robotics · doi:10.1561/2300000021 · Cited by 798
Peters, J., & Schaal, S. (2008) · Neurocomputing · doi:10.1016/j.neucom.2007.11.026 · Cited by 738
Self-citationGaussian Processes for Data-Efficient Learning in Robotics and Control
Deisenroth, M. P., Fox, D., & Rasmussen, C. E. (2015) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2013.218 · Cited by 659
A Survey on Machine-Learning Techniques in Cognitive Radios
Bkassiny, M., Li, Y., & Jayaweera, S. K. (2013) · IEEE Communications Surveys & Tutorials · doi:10.1109/surv.2012.100412.00017 · Cited by 580
How to train your robot with deep reinforcement learning: Lessons we have learned
Ibarz, J., Tan, J., Finn, C., Kalakrishnan, M., Pastor, P., & Levine, S. (2021) · The International Journal of Robotics Research · doi:10.1177/0278364920987859 · Cited by 570
Chen, Z., & Liu, B. (2018) · Synthesis Lectures on Artificial Intelligence and Machine Learning · doi:10.1007/978-3-031-01581-6 · Cited by 466
Information theoretic MPC for model-based reinforcement learning
Williams, G., Wagener, N., Goldfain, B., Drews, P., Rehg, J. M., Boots, B., & Theodorou, E. A. (2017) · 2017 IEEE International Conference on Robotics and Automation (ICRA) · doi:10.1109/icra.2017.7989202 · Cited by 460
Learning to Walk Via Deep Reinforcement Learning
Haarnoja, T., Ha, S., Zhou, A., Tan, J., Tucker, G., & Levine, S. (2019) · Robotics: Science and Systems XV · doi:10.15607/rss.2019.xv.011 · Cited by 430
Policy search for motor primitives in robotics
Kober, J., & Peters, J. (2011) · Machine Learning · doi:10.1007/s10994-010-5223-6 · Cited by 427
Self-citationLearning to select and generalize striking movements in robot table tennis
Mülling, K., Kober, J., Kroemer, O., & Peters, J. (2013) · The International Journal of Robotics Research · doi:10.1177/0278364912472380 · Cited by 396
Self-citationCooperative Adaptive Cruise Control: A Reinforcement Learning Approach
Desjardins, C., & Chaib-draa, B. (2011) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2011.2157145 · Cited by 344
Information-Theoretic Model Predictive Control: Theory and Applications to Autonomous Driving
Williams, G., Drews, P., Goldfain, B., Rehg, J. M., & Theodorou, E. A. (2018) · IEEE Transactions on Robotics · doi:10.1109/tro.2018.2865891 · Cited by 334
Continuous Deep Q-Learning with Model-based Acceleration
Gu, S., Lillicrap, T., Sutskever, I., & Levine, S. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1603.00748 · Cited by 304
Bayesian optimization with safety constraints: safe and automatic parameter tuning in robotics
Berkenkamp, F., Krause, A., & Schoellig, A. P. (2023) · Machine Learning · doi:10.1007/s10994-021-06019-1 · Cited by 286
Robot motor skill coordination with EM-based Reinforcement Learning
Kormushev, P., Calinon, S., & Caldwell, D. G. (2010) · 2010 IEEE/RSJ International Conference on Intelligent Robots and Systems · doi:10.1109/iros.2010.5649089 · Cited by 258
Reinforcement Learning in Robotics: Applications and Real-World Challenges
Kormushev, P., Calinon, S., & Caldwell, D. (2013) · Robotics · doi:10.3390/robotics2030122 · Cited by 258
Reinforcement learning for robot soccer
Riedmiller, M., Gabel, T., Hafner, R., & Lange, S. (2009) · Autonomous Robots · doi:10.1007/s10514-009-9120-4 · Cited by 255
Reinforcement learning of motor skills in high dimensions: A path integral approach
Theodorou, E., Buchli, J., & Schaal, S. (2010) · 2010 IEEE International Conference on Robotics and Automation · doi:10.1109/robot.2010.5509336 · Cited by 252
Self-citationMartinez-Cantin, R., de Freitas, N., Brochu, E., Castellanos, J., & Doucet, A. (2009) · Autonomous Robots · doi:10.1007/s10514-009-9130-2 · Cited by 217
Yao, S., Zhao, Y., Zhang, A., Su, L., & Abdelzaher, T. (2017) · Proceedings of the 15th ACM Conference on Embedded Network Sensor Systems · doi:10.1145/3131672.3131675 · Cited by 188
Chen, Z., & Liu, B. (2018) · Synthesis Lectures on Artificial Intelligence and Machine Learning · doi:10.2200/s00737ed1v01y201610aim033 · Cited by 154
Movement templates for learning of hitting and batting
Kober, J., Mülling, K., Krömer, O., Lampert, C. H., Schölkopf, B., & Peters, J. (2010) · 2010 IEEE International Conference on Robotics and Automation · doi:10.1109/robot.2010.5509672 · Cited by 152
Self-citationDynamic Resource Allocation During Natural Disasters Using Multi-agent Environment
Vereshchaka, A., & Dong, W. (2019) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-21741-9_13 · Cited by 10
Metadata
- Authors
- Peters, J., & Schaal, S.
- Venue
- 2006 IEEE/RSJ International Conference on Intelligent Robots and Systems
- Publisher
- IEEE
- Published
- 2006-10
- Pages
- 2219-2225
- OpenAlex
- W2127107099
- Cited by
- 498
- References
- 43
- FWCI
- 10.32
- Type
- conference paper
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.