Reinforcement learning of motor skills with policy gradients
Abstract
Autonomous learning is one of the hallmarks of human and animal behavior, and understanding the principles of learning will be crucial in order to achieve true autonomy in advanced machines like humanoid robots. In this paper, we examine learning of complex motor skills with human-like limbs. While supervised learning can offer useful tools for bootstrapping behavior, e.g., by learning from demonstration, it is only reinforcement learning that offers a general approach to the final trial-and-error improvement that is needed by each individual acquiring a skill. Neither neurobiological nor machine learning studies have, so far, offered compelling results on how reinforcement learning can be scaled to the high-dimensional continuous state and action spaces of humans or humanoids. Here, we combine two recent research developments on learning motor control in order to achieve this scaling. First, we interpret the idea of modular motor control by means of motor primitives as a suitable way to generate parameterized control policies for reinforcement learning. Second, we combine motor primitives with the theory of stochastic policy gradient learning, which currently seems to be the only feasible framework for reinforcement learning for humanoids. We evaluate different policy gradient methods with a focus on their applicability to parameterized motor primitives. We compare these algorithms in the context of motor primitive learning, and show that our most modern algorithm, the Episodic Natural Actor-Critic outperforms previous algorithms by at least an order of magnitude. We demonstrate the efficiency of this reinforcement learning method in the application of learning to hit a baseball with an anthropomorphic robot arm.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Comparator models are agnostic regarding the specific mathematical computation underlying their function, which can be explained by a variety of Bayesian or non-Bayesian mechanisms.
“Crucially, comparator models are agnostic regarding the specific mathematical computation underlying their function, which can be explained by a variety of Bayesian (e.g., variational Bayes, Kalman filtering) or non-Bayesian mechanisms (e.g., reinforcement learning) (Friston 2011; Peters and Schaal 2008; Wolpert et al. 1995).”
From From cognitivism to autopoiesis: towards a computational framework for the embodied mind (2018) · cites this paper as Peters and Schaal 2008
Cites
Advances in Neural Information Processing Systems 14
(2002) · The MIT Press eBooks · doi:10.7551/mitpress/1120.001.0001 · Cited by 9,184 · see citing works
Simple Statistical Gradient-Following Algorithms for Connectionist Reinforcement Learning
Williams, R. J. (1992) · Machine Learning · doi:10.1007/bf00992696 · Cited by 7,936 · see citing works
Practical Methods of Optimization
Fletcher, R. (2000) · Wiley eBooks · doi:10.1002/9781118723203 · Cited by 7,649 · see citing works
(1999) · Cited by 5,147 · see citing works
Neuronlike adaptive elements that can solve difficult learning control problems
Barto, A. G., Sutton, R. S., & Anderson, C. W. (1983) · IEEE Transactions on Systems Man and Cybernetics · doi:10.1109/tsmc.1983.6313077 · Cited by 3,238 · see citing works
Natural Gradient Works Efficiently in Learning
Амари, Ш.-И. (1998) · Neural Computation · doi:10.1162/089976698300017746 · Cited by 2,745 · see citing works
Introduction to Stochastic Search and Optimization
Spall, J. C. (2003) · Wiley eBooks · doi:10.1002/0471722138 · Cited by 1,912 · see citing works
Konda, V. R., & Tsitsiklis, J. N. (2002) · Cited by 1,764 · see citing works
Mathematical Methods and Algorithms for Signal Processing
Moon, T. K., & Stirling, W. C. (1999) · Cited by 1,502 · see citing works
On Choosing and Bounding Probability Metrics
Gibbs, A. L., & Su, F. E. (2002) · International Statistical Review · doi:10.1111/j.1751-5823.2002.tb00178.x · Cited by 1,199 · see citing works
INTRODUCTION TO STOCHASTIC SEARCH AND OPTIMIZATION: ESTIMATION, SIMULATION, AND CONTROL
Nowak, W. (2012) · Medical Entomology and Zoology · Cited by 1,030 · see citing works
Modeling and Control of Robot Manipulators
(2000) · Industrial Robot the international journal of robotics research and application · doi:10.1108/ir.2000.04927aae.003 · Cited by 989 · see citing works
Matrix Algebra from a Statistician's Perspective
Ziegel, E. R., & Harville, D. A. (1998) · Technometrics · doi:10.2307/1270666 · Cited by 898 · see citing works
Advances in Neural Information Processing Systems 7
Tesauro, G., Touretzky, D. S., & Leen, T. K. (1995) · Cited by 869 · see citing works
Movement imitation with nonlinear dynamical systems in humanoid robots
Ijspeert, A. J., Nakanishi, J., & Stefan Schaal, A. (2003) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/robot.2002.1014739 · Cited by 862 · see citing works
Feature Article: Optimization for simulation: Theory vs. Practice
Fu, M. C. (2002) · INFORMS journal on computing · doi:10.1287/ijoc.14.3.192.113 · Cited by 777 · see citing works
Kakade, S. M. (2001) · ScholarlyCommons (University of Pennsylvania) · Cited by 691 · see citing works
Policy gradient reinforcement learning for fast quadrupedal locomotion
Kohl, N., & Stone, P. (2004) · IEEE International Conference on Robotics and Automation, 2004. Proceedings. ICRA '04. 2004 · doi:10.1109/robot.2004.1307456 · Cited by 676 · see citing works
Learning Attractor Landscapes for Learning Motor Primitives
Ijspeert, A. J., Nakanishi, J., Stefan Schaal, A., Becker, S., Thrun, S., & Obermayer, K. (2002) · Infoscience (Ecole Polytechnique Fédérale de Lausanne) · Cited by 572 · see citing works
Proceedings of the European Conference on Machine Learning
Brazdil, P. (1993) · Cited by 548 · see citing works
Matrix Algebra From a Statistician's Perspective
Harville, D. A. (1998) · Technometrics · doi:10.1080/00401706.1998.10485214 · Cited by 538 · see citing works
Differential dynamic programming
DENHAM, W. F. (1971) · IEEE Transactions on Automatic Control · doi:10.1109/tac.1971.1099760 · Cited by 530 · see citing works
Infinite-Horizon Policy-Gradient Estimation
Baxter, J., & Bartlett, P. L. (2001) · Cited by 525 · see citing works
Policy Gradient Methods for Robotics
Peters, J., & Stefan Schaal, A. (2006) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros.2006.282564 · Cited by 511 · see citing works
Self-citationOn the Sample Complexity of Reinforcement Learning
Kakade, S. M. (2003) · OpenGrey (Institut de l'Information Scientifique et Technique) · Cited by 510 · see citing works
Differential dynamic programming
Jacobson, D. H., & Mayne, D. Q. (1970) · CERN Document Server (European Organization for Nuclear Research) · Cited by 510 · see citing works
Motor primitives in vertebrates and invertebrates
Flash, T., & Hochner, B. (2005) · Current Opinion in Neurobiology · doi:10.1016/j.conb.2005.10.011 · Cited by 508 · see citing works
Likelihood ratio gradient estimation for stochastic systems
Glynn, P. W. (1990) · Communications of the ACM · doi:10.1145/84537.84552 · Cited by 402 · see citing works
Sammut, C., & Webb, G. I. (2010) · Encyclopedia of Machine Learning · doi:10.1007/978-0-387-30164-8_447 · Cited by 400 · see citing works
Learning from demonstration and adaptation of biped locomotion
Nakanishi, J., Morimoto, J., Endo, G., Cheng, G., Stefan Schaal, A., & Kawato, M. (2004) · Robotics and Autonomous Systems · doi:10.1016/j.robot.2004.03.003 · Cited by 399 · see citing works
Introduction to Stochastic Search and Optimization: Estimation, Simulation, and Control
Frey, J. C. (2004) · Journal of the American Statistical Association · doi:10.1198/jasa.2004.s362 · Cited by 383 · see citing works
PEGASUS: A Policy Search Method for Large MDPs and POMDPsPreprint
Ng, A. Y., & Jordan, M. I. (2013) · arXiv (Cornell University) · doi:10.48550/arxiv.1301.3878 · Cited by 364 · see citing works
Schaal, S., Peters, J., Nakanishi, J., & Ijspeert, A. J. (2005) · Springer tracts in advanced robotics · doi:10.1007/11008941_60 · Cited by 349 · see citing works
Self-citationInfinite-Horizon Policy-Gradient Estimation
Baxter, J., & Bartlett, P. L. (2001) · Journal of Artificial Intelligence Research · doi:10.1613/jair.806 · Cited by 343 · see citing works
Variance Reduction Techniques for Gradient Estimates in Reinforcement Learning
Greensmith, E., Bartlett, P. L., & Baxter, J. (2004) · doi:10.5555/1005332.1044710 · Cited by 314 · see citing works
Peters, J., Vijayakumar, S., & Schaal, S. (2005) · Lecture notes in computer science · doi:10.1007/11564096_29 · Cited by 311 · see citing works
Self-citationReinforcement Learning for Humanoid Robotics
Peters, J., Vijayakumar, S., & Schaal, S. (2003) · Cited by 309 · see citing works
Self-citationA stochastic reinforcement learning algorithm for learning real-valued functions
Gullapalli, V. (1990) · Neural Networks · doi:10.1016/0893-6080(90)90056-q · Cited by 292 · see citing works
Balasubramanian, V. (1997) · Neural Computation · doi:10.1162/neco.1997.9.2.349 · Cited by 236 · see citing works
Advances in Neural Information Processing Systems 11 (NIPS 1998)
Ferrari‐Trecate, G., Williams, C. K. I., & Opper, M. (1999) · Cited by 233 · see citing works
Likelilood ratio gradient estimation
Glynn, P. W. (1987) · Winter Simulation Conference · doi:10.1145/318371.318612 · Cited by 225 · see citing works
Acquiring robot skills via reinforcement learning
(1994) · IEEE Control Systems · doi:10.1109/37.257890 · Cited by 206 · see citing works
Biped dynamic walking using reinforcement learning
Benbrahim, H., & Franklin, J. A. (1997) · Robotics and Autonomous Systems · doi:10.1016/s0921-8890(97)00043-2 · Cited by 198 · see citing works
Proceedings International Conference on Artificial Neural Networks ICANN'95
Bishop, C. (1995) · Cited by 189 · see citing works
The Optimal Reward Baseline for Gradient-Based Reinforcement LearningPreprint
Weaver, L., & Tao, N. (2013) · arXiv (Cornell University) · doi:10.48550/arxiv.1301.2315 · Cited by 158 · see citing works
The Computation and Theory of Optimal Control
Dyer, P., McReynolds, S. R., & Eggleston, D. M. (1972) · Journal of Dynamic Systems Measurement and Control · doi:10.1115/1.3426573 · Cited by 152 · see citing works
Bagnell, J. A., & Schneider, J. (2018) · Figshare · doi:10.1184/r1/6552458 · Cited by 152 · see citing works
Reinforcement learning for imitating constrained reaching movements
Guenter, F., Hersch, M., Calinon, S., & Billard, A. (2007) · Advanced Robotics · doi:10.1163/156855307782148550 · Cited by 144 · see citing works
Experiments with Infinite-Horizon, Policy-Gradient Estimation
Baxter, J., Bartlett, P. L., & Weaver, L. (2001) · Journal of Artificial Intelligence Research · doi:10.1613/jair.807 · Cited by 139 · see citing works
Gradient Optimization and Nonlinear Control
Hasdorff, L. (1976) · Medical Entomology and Zoology · Cited by 137 · see citing works
The Computation and Theory of Optimal Control
(1970) · Mathematics in Science and Engineering/Mathematics in science and engineering · doi:10.1016/s0076-5392(08)x6199-5 · Cited by 136 · see citing works
Simulation-Based Optimization with Stochastic Approximation Using Common Random Numbers
Kleinman, N. L., Spall, J. C., & Naiman, D. Q. (1999) · Management Science · doi:10.1287/mnsc.45.11.1570 · Cited by 102 · see citing works
Ben-Itzhak, S., & Karniel, A. (2007) · Neural Computation · doi:10.1162/neco.2007.12-05-077 · Cited by 95 · see citing works
Natural Actor-Critic for Road Traffic Optimisation
Richter, S., Aberdeen, D., & Yu, J. (2007) · The MIT Press eBooks · doi:10.7551/mitpress/7503.003.0151 · Cited by 94 · see citing works
Using Local Trajectory Optimizers to Speed Up Global Optimization in Dynamic Programming
Atkeson, C. G. (1993) · Cited by 78 · see citing works
Minimax Differential Dynamic Programming: An Application to Robust Biped Walking
Morimoto, J., & Atkeson, C. G. (2018) · doi:10.1184/r1/6555545 · Cited by 77 · see citing works
Reinforcement learning for a CPG-driven biped robot
Mori, T., Nakamura, Y., Sato, M.-A., et al. (2004) · National Conference on Artificial Intelligence · Cited by 66 · see citing works
Mitsunaga, N., Smith, C., Kanda, T., Ishiguro, H., & Hagita, N. (2005) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros.2005.1545206 · Cited by 65 · see citing works
Machine learning of motor skills for robotics
Stefan Schaal, A., & Peters, J. (2007) · Cited by 55 · see citing works
Self-citationBaird, I., & Leemon, C. (1993) · doi:10.21236/ada280862 · Cited by 55 · see citing works
Rapid synchronization and accurate phase-locking of rhythmic motor primitives
Pongas, D., Billard, A. G., & Schaal, S. (2005) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros.2005.1545257 · Cited by 43 · see citing works
Self-citationOptimizing Average Reward Using Discounted Rewards
Kakade, S. M. (2001) · Lecture notes in computer science · doi:10.1007/3-540-44581-1_40 · Cited by 40 · see citing works
Learning CPG sensory feedback with policy gradient for biped locomotion for a full-body humanoid
Endo, G., Morimoto, J., Matsubara, T., Nakanishi, J., & Cheng, G. (2005) · National Conference on Artificial Intelligence · Cited by 35 · see citing works
Reinforcement Learning for Biped Locomotion
Sato, M.-A., Nakamura, Y., & Ishii, S. (2002) · Lecture notes in computer science · doi:10.1007/3-540-46084-5_126 · Cited by 35 · see citing works
(2012) · Harvard University Press eBooks · doi:10.2307/j.ctt2jbq07.6 · Cited by 32 · see citing works
Sadegh, P., & Spall, J. C. (1997) · Proceedings of the ... American Control Conference/Proceedings of the American Control Conference · doi:10.1109/acc.1997.609490 · Cited by 29 · see citing works
Real-time learning: a ball on a beam
Benbrahim, H., Doleac, J., Franklin, J. A., & Selfridge, O. G. (2003) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn.1992.287219 · Cited by 29 · see citing works
Efficient Gradient Estimation for Motor Control LearningPreprint
Lawrence, G., Cowan, N. J., & Russell, S. (2012) · arXiv (Cornell University) · doi:10.48550/arxiv.1212.2475 · Cited by 28 · see citing works
Statistical Machine Learning and Combinatorial Optimization
Berny, A. (2001) · Natural computing series · doi:10.1007/978-3-662-04448-3_14 · Cited by 25 · see citing works
Learning Control Under Extreme Uncertainty
Gullapalli, V. (1992) · Cited by 25 · see citing works
An RLS-Based Natural Actor-Critic Algorithm for Locomotion of a Two-Linked Robot Arm
Park, J., Kim, J.-H., & Kang, D. (2005) · Lecture notes in computer science · doi:10.1007/11596448_9 · Cited by 21 · see citing works
Ueno, T., Nakamura, Y., Takuma, T., et al. (2006) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros.2006.281663 · Cited by 16 · see citing works
Natural Policy Gradient Reinforcement Learning for a CPG Control of a Biped Robot
Nakamura, Y., Mori, T., & Ishii, S. (2004) · Lecture notes in computer science · doi:10.1007/978-3-540-30217-9_98 · Cited by 14 · see citing works
A Kendama learning robot based on a dynamic optimization theory
Miyamoto, H., Gandolfo, F., Gomi, H., Stefan Schaal, A., Koike, Y., Osu, R., Nakano, E., Wada, Y., & Kawato, M. (2002) · IEEE RO-MAN · doi:10.1109/roman.1995.531981 · Cited by 8 · see citing works
Fukushima-Kimura, B. H., Kamijima, Y., Kawamura, K., & Sakai, A. (2023) · Transactions of the Institute of Systems Control and Information Engineers · doi:10.5687/iscie.36.9 · Cited by 6 · see citing works
Natural Actor-Critic for Road Traffic Optimisation
Schölkopf, B., Platt, J. C., & Hofmann, T. F. (2007) · Cited by 3 · see citing works
Meiden, W. E., & van der (1998) · Mededelingen van het Wiskundig Genootschap · Cited by 2 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
Deep learning in neural networks: an overview
Schmidhuber, J. (2015) · Neural Networks · doi:10.1016/j.neunet.2014.09.003 · Cited by 18,570 · see citing works
Reinforcement learning in robotics: A survey
Kober, J., Bagnell, J. A., & Peters, J. (2013) · The International Journal of Robotics Research · doi:10.1177/0278364913495721 · Cited by 3,160 · see citing works
Self-citationSoft Actor-Critic Algorithms and Applications
Haarnoja, T., Zhou, A., Hartikainen, K., Tucker, G., Ha, S., Tan, J., Kumar, V., Zhu, H., Gupta, A., Abbeel, P., et al. (2018) · arXiv · doi:10.48550/arxiv.1812.05905 · Cited by 1,994 · see citing works
Target-driven visual navigation in indoor scenes using deep reinforcement learning
Zhu, Y., Mottaghi, R., Kolve, E., Lim, J. J., Gupta, A., Fei-Fei, L., & Farhadi, A. (2017) · 2017 IEEE International Conference on Robotics and Automation (ICRA) · doi:10.1109/icra.2017.7989381 · Cited by 1,533 · see citing works
Learning Complex Dexterous Manipulation with Deep Reinforcement Learning and Demonstrations
Rajeswaran, A., Kumar, V., Gupta, A., Vezzani, G., Schulman, J., Todorov, E., & Levine, S. (2018) · Robotics: Science and Systems XIV · doi:10.15607/rss.2018.xiv.049 · Cited by 847 · see citing works
Overcoming Exploration in Reinforcement Learning with Demonstrations
Nair, A., McGrew, B., Andrychowicz, M., Zaremba, W., & Abbeel, P. (2018) · 2018 IEEE International Conference on Robotics and Automation (ICRA) · doi:10.1109/icra.2018.8463162 · Cited by 712 · see citing works
On-Line Building Energy Optimization Using Deep Reinforcement Learning
Mocanu, E., Mocanu, D. C., Nguyen, P. H., Liotta, A., Webber, M. E., Gibescu, M., & Slootweg, J. G. (2019) · IEEE Transactions on Smart Grid · doi:10.1109/tsg.2018.2834219 · Cited by 649 · see citing works
Cheng, R., Orosz, G., Murray, R. M., & Burdick, J. W. (2019) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v33i01.33013387 · Cited by 611 · see citing works
Deep Imitation Learning for Complex Manipulation Tasks from Virtual Reality Teleoperation
Zhang, T., McCarthy, Z., Jow, O., Lee, D., Chen, X., Goldberg, K., & Abbeel, P. (2018) · 2018 IEEE International Conference on Robotics and Automation (ICRA) · doi:10.1109/icra.2018.8461249 · Cited by 609 · see citing works
How to train your robot with deep reinforcement learning: Lessons we have learned
Ibarz, J., Tan, J., Finn, C., Kalakrishnan, M., Pastor, P., & Levine, S. (2021) · The International Journal of Robotics Research · doi:10.1177/0278364920987859 · Cited by 570 · see citing works
Learning deep control policies for autonomous aerial vehicles with MPC-guided policy search
Zhang, T., Kahn, G., Levine, S., & Abbeel, P. (2016) · 2016 IEEE International Conference on Robotics and Automation (ICRA) · doi:10.1109/icra.2016.7487175 · Cited by 452 · see citing works
“While this method is not as general as policy gradient RL algorithms, which can optimize arbitrary parameterized policies with arbitrary observations under unknown dynamics [32], it is fast, simple, and often effective [39].”
MethodFrom cognitivism to autopoiesis: towards a computational framework for the embodied mind
Allen, M., & Friston, K. J. (2018) · Synthese · doi:10.1007/s11229-016-1288-5 · Cited by 386 · see citing works
Learning to Adapt in Dynamic, Real-World Environments through Meta-Reinforcement Learning
Nagabandi, A., Clavera, I., Liu, S., Fearing, R. S., Abbeel, P., Levine, S., & Finn, C. (2018) · arXiv · doi:10.48550/arxiv.1803.11347 · Cited by 370 · see citing works
Reinforcement learning for intelligent healthcare applications: A survey
Coronato, A., Naeem, M., De Pietro, G., & Paragliola, G. (2020) · Artificial Intelligence in Medicine · doi:10.1016/j.artmed.2020.101964 · Cited by 320 · see citing works
Dynamic movement primitives in robotics: A tutorial survey
Saveriano, M., Abu-Dakka, F. J., Kramberger, A., & Peternel, L. (2023) · The International Journal of Robotics Research · doi:10.1177/02783649231201196 · Cited by 267 · see citing works
Martinez-Cantin, R., de Freitas, N., Brochu, E., Castellanos, J., & Doucet, A. (2009) · Autonomous Robots · doi:10.1007/s10514-009-9130-2 · Cited by 217 · see citing works
AWAC: Accelerating Online Reinforcement Learning with Offline Datasets
Nair, A., Gupta, A., Dalal, M., & Levine, S. (2020) · arXiv · doi:10.48550/arxiv.2006.09359 · Cited by 175 · see citing works
“One class of algorithms initializes the policy via behavioral cloning from demonstrations, and then fine-tunes with reinforcement learning [33, 15, 40, 18, 35, 10, 50].”
BackgroundLearning force control policies for compliant manipulation
Kalakrishnan, M., Righetti, L., Pastor, P., & Schaal, S. (2011) · 2011 IEEE/RSJ International Conference on Intelligent Robots and Systems · doi:10.1109/iros.2011.6095096 · Cited by 161 · see citing works
Self-citationDynamic Cloth Manipulation with Deep Reinforcement Learning
Jangir, R., Alenya, G., & Torras, C. (2020) · 2020 IEEE International Conference on Robotics and Automation (ICRA) · doi:10.1109/icra40945.2020.9196659 · Cited by 114 · see citing works
Reinforcement learning of clothing assistance with a dual-arm robot
Tamei, T., Matsubara, T., Rai, A., & Shibata, T. (2011) · 2011 11th IEEE-RAS International Conference on Humanoid Robots · doi:10.1109/humanoids.2011.6100915 · Cited by 103 · see citing works
APRIL: Active Preference Learning-Based Reinforcement Learning
Akrour, R., Schoenauer, M., & Sebag, M. (2012) · Lecture Notes in Computer Science · doi:10.1007/978-3-642-33486-3_8 · Cited by 95 · see citing works
Reinforcement learning of a motor skill for wearing a T-shirt using topology coordinates
Matsubara, T., Shinohara, D., & Kidode, M. (2013) · Advanced Robotics · doi:10.1080/01691864.2013.777012 · Cited by 41 · see citing works
Kadi, H. A., & Terzić, K. (2023) · Sensors · doi:10.3390/s23052389 · Cited by 19 · see citing works
“Natural Policy Gradient (NPG) [131,132] resolves the issue by adding a constraint on the policy space: Solving this objective is difficult in practice, so TRPO approximates the surrogate advantage and the mean of the KL-divergence using the Taylor series up to the first order and second order of the…”
Method
- Authors
- Peters, J., & Schaal, S.
- Affiliations
- University of Southern California (US)
- Corresponding author
- Peters, Jan
- ORCID
- Peters, Jan
- Venue
- Neural Networks
- Publisher
- Elsevier BV
- Published
- 2008-05
- Volume
- 21
- Issue
- 4
- Pages
- 682-697
- ISSN
- 0893-6080 (ISSN-L)
- OpenAlex
- W2125612430
- Cited by
- 831
- References
- 77
- Citations by year
- 2026: 19 · 2025: 27 · 2024: 43 · 2023: 50 · 2022: 49 · 2021: 62 · 2020: 75 · 2019: 77 · 2018: 80 · 2017: 71 · 2016: 39 · 2015: 31 · 2014: 45 · 2013: 43 · 2012: 44
- FWCI
- 39.57
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Neural Networks (publishedVersion)
- PubMed (publishedVersion)
- TUbilio (Technical University of Darmstadt) (acceptedVersion)
- Max Planck Institute for Plasma Physics (submittedVersion)
- MPG.PuRe (Max Planck Society) (submittedVersion)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Reinforcement Learning in Robotics
- Life Sciences › Neuroscience › Cognitive Neuroscience › Motor Control and Adaptation
- Physical Sciences › Computer Science › Computational Theory and Mathematics › Adaptive Dynamic Programming Control
- Keywords
- reinforcement learning, motor skill learning, policy gradient, motor primitives, humanoid robots, anthropomorphic robot arm
- MeSH terms
- Algorithms, Animals, Artificial Intelligence, Computer Simulation, Extremities, Feedback, Forelimb, Humans, Locomotion, Motor Skills, Movement, Reinforcement, Psychology, Robotics, Stochastic Processes
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
