Natural Actor-Critic
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Cites
Advances in Neural Information Processing Systems 19
(2007) · The MIT Press eBooks · doi:10.52202/085713 · Cited by 16,037 · see citing works
(1999) · Cited by 5,147 · see citing works
(2012) · Springer eBooks · doi:10.1007/978-1-4419-1428-6_5086 · Cited by 3,601 · see citing works
2009 International Joint Conference on Artificial Intelligence
(2009) · doi:10.1109/jcai15434.2009 · Cited by 2,846 · see citing works
Natural Gradient Works Efficiently in Learning
Амари, Ш.-И. (1998) · Neural Computation · doi:10.1162/089976698300017746 · Cited by 2,745 · see citing works
Barto, A. (1998) · IFAC Proceedings Volumes · doi:10.1016/s1474-6670(17)38315-5 · Cited by 2,172 · see citing works
Zhang, B., Li, C., & Lin, N. (2020) · De Gruyter eBooks · doi:10.1515/9783110595567-011 · Cited by 1,939 · see citing works
Konda, V. R., & Tsitsiklis, J. N. (2002) · Cited by 1,764 · see citing works
Mathematical Methods and Algorithms for Signal Processing
Moon, T. K., & Stirling, W. C. (1999) · Cited by 1,502 · see citing works
Bertsekas, D. P. (2001) · Encyclopedia of Optimization · doi:10.1007/0-306-48332-7_333 · Cited by 1,486 · see citing works
Kakade, S. M. (2001) · ScholarlyCommons (University of Pennsylvania) · Cited by 691 · see citing works
Proceedings of IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS'94)
(1994) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros.1994.407417 · Cited by 551 · see citing works
Policy Gradient Methods for Robotics
Peters, J., & Stefan Schaal, A. (2006) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros.2006.282564 · Cited by 511 · see citing works
Self-citationAdaptive linear quadratic control using policy iteration
Bradtke, S. J., Ydstie, B. E., & Barto, A. G. (2005) · Proceedings of the ... American Control Conference/Proceedings of the American Control Conference · doi:10.1109/acc.1994.735224 · Cited by 416 · see citing works
Peters, J., Vijayakumar, S., & Schaal, S. (2005) · Lecture notes in computer science · doi:10.1007/11564096_29 · Cited by 311 · see citing works
Self-citationReinforcement Learning for Humanoid Robotics
Peters, J., Vijayakumar, S., & Schaal, S. (2003) · Cited by 309 · see citing works
Self-citationLeast-Squares Temporal Difference Learning
Boyan, J. A. (1999) · Cited by 250 · see citing works
Gradient Descent for General Reinforcement Learning
Baird, L. C., & Moore, A. (1998) · Cited by 236 · see citing works
Learning rhythmic movements by demonstration using nonlinear oscillators
Ijspeort, A. J., Nakanishi, J., & Schaal, S. (2003) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/irds.2002.1041514 · Cited by 183 · see citing works
Self-citationLocal minima and plateaus in hierarchical structures of multilayer perceptrons
Fukumizu, K., & Амари, Ш.-И. (2000) · Neural Networks · doi:10.1016/s0893-6080(00)00009-5 · Cited by 178 · see citing works
Barto, A. G. (1997) · Elsevier eBooks · doi:10.1016/b978-012526430-3/50003-9 · Cited by 168 · see citing works
Bagnell, J. A., & Schneider, J. (2018) · Figshare · doi:10.1184/r1/6552458.v1 · Cited by 152 · see citing works
Reinforcement learning for imitating constrained reaching movements
Guenter, F., Hersch, M., Calinon, S., & Billard, A. (2007) · Advanced Robotics · doi:10.1163/156855307782148550 · Cited by 144 · see citing works
Natural Actor-Critic for Road Traffic Optimisation
Richter, S., Aberdeen, D., & Yu, J. (2007) · The MIT Press eBooks · doi:10.7551/mitpress/7503.003.0151 · Cited by 94 · see citing works
Policy-Gradient Algorithms for Partially Observable Markov Decision Processes
Aberdeen, D. (2003) · ANU Open Research (Australian National University) · doi:10.25911/5d7a2b73cbb88 · Cited by 74 · see citing works
Baird, I., & Leemon, C. (1993) · doi:10.21236/ada280862 · Cited by 55 · see citing works
Shaping multi-agent systems with gradient reinforcement learning
Buffet, O., Dutech, A., & Charpillet, F. (2007) · Autonomous Agents and Multi-Agent Systems · doi:10.1007/s10458-006-9010-5 · Cited by 39 · see citing works
Applying the Episodic Natural Actor-Critic Architecture to Motor Primitive Learning
Peters, J., & Schaal, S. (2007) · Cited by 27 · see citing works
Self-citationAn RLS-Based Natural Actor-Critic Algorithm for Locomotion of a Two-Linked Robot Arm
Park, J., Kim, J.-H., & Kang, D. (2005) · Lecture notes in computer science · doi:10.1007/11596448_9 · Cited by 21 · see citing works
Ueno, T., Nakamura, Y., Takuma, T., et al. (2006) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros.2006.281663 · Cited by 16 · see citing works
Conditional random fields for multi-agent reinforcement learning
Zhang, X., Aberdeen, D., & Vishwanathan, S. V. N. (2007) · International Conference on Machine Learning · doi:10.1145/1273496.1273640 · Cited by 15 · see citing works
An Introduction to Reinforcement Learning Theory: Value Function Methods
Bartlett, P. L. (2003) · Lecture notes in computer science · doi:10.1007/3-540-36434-x_5 · Cited by 11 · see citing works
Scaling Reinforcement Learning Paradigms for Motor Learning
Peters, J., Vijayakumar, S., & Schaal, S. (2003) · Cited by 3 · see citing works
Self-citation
Flagged first, then by citations
Cited by
Sorted by citations
Deep learning in neural networks: an overview
Schmidhuber, J. (2015) · Neural Networks · doi:10.1016/j.neunet.2014.09.003 · Cited by 18,570 · see citing works
A survey of robot learning from demonstration
Argall, B. D., Chernova, S., Veloso, M., & Browning, B. (2009) · Robotics and Autonomous Systems · doi:10.1016/j.robot.2008.10.024 · Cited by 3,227 · see citing works
Reinforcement learning in robotics: A survey
Kober, J., Bagnell, J. A., & Peters, J. (2013) · The International Journal of Robotics Research · doi:10.1177/0278364913495721 · Cited by 3,160 · see citing works
Self-citationDeep Imitation Learning for Complex Manipulation Tasks from Virtual Reality Teleoperation
Zhang, T., McCarthy, Z., Jow, O., Lee, D., Chen, X., Goldberg, K., & Abbeel, P. (2018) · 2018 IEEE International Conference on Robotics and Automation (ICRA) · doi:10.1109/icra.2018.8461249 · Cited by 609 · see citing works
Serotonin Selectively Modulates Reward Value in Human Decision-Making
Seymour, B., Daw, N. D., Roiser, J. P., Dayan, P., & Dolan, R. (2012) · The Journal of Neuroscience · doi:10.1523/jneurosci.0053-12.2012 · Cited by 252 · see citing works
Model-Free reinforcement learning with continuous action in practice
Degris, T., Pilarski, P. M., & Sutton, R. S. (2012) · 2012 American Control Conference (ACC) · doi:10.1109/acc.2012.6315022 · Cited by 239 · see citing works
Martinez-Cantin, R., de Freitas, N., Brochu, E., Castellanos, J., & Doucet, A. (2009) · Autonomous Robots · doi:10.1007/s10514-009-9130-2 · Cited by 217 · see citing works
Salvato, E., Fenu, G., Medvet, E., & Pellegrino, F. A. (2021) · IEEE Access · doi:10.1109/access.2021.3126658 · Cited by 200 · see citing works
“In addition, policy search approaches result in more efficient trainings in terms of time needed for convergence [33, 34, 35, 36, 37, 38].”
BackgroundA Deep Actor-Critic Reinforcement Learning Framework for Dynamic Multichannel Access
Zhong, C., Lu, Z., Gursoy, M. C., & Velipasalar, S. (2019) · IEEE Transactions on Cognitive Communications and Networking · doi:10.1109/tccn.2019.2952909 · Cited by 123 · see citing works
Kadi, H. A., & Terzić, K. (2023) · Sensors · doi:10.3390/s23052389 · Cited by 19 · see citing works
“Natural Policy Gradient (NPG) [131,132] resolves the issue by adding a constraint on the policy space: Solving this objective is difficult in practice, so TRPO approximates the surrogate advantage and the mean of the KL-divergence using the Taylor series up to the first order and second order of the…”
Method
- Authors
- Peters, J., & Schaal, S.
- Affiliations
- University of Southern California (US)
- Corresponding author
- Peters, Jan
- ORCID
- Peters, Jan
- Venue
- Neurocomputing
- Publisher
- Elsevier BV
- Published
- 2008-03
- Volume
- 71
- Issue
- 7-9
- Pages
- 1180-1190
- ISSN
- 0925-2312 (ISSN-L)
- OpenAlex
- W2172968643
- Cited by
- 738
- References
- 33
- Citations by year
- 2026: 15 · 2025: 37 · 2024: 37 · 2023: 54 · 2022: 41 · 2021: 65 · 2020: 56 · 2019: 52 · 2018: 41 · 2017: 33 · 2016: 22 · 2015: 28 · 2014: 42 · 2013: 43 · 2012: 55
- FWCI
- 60.27
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Neurocomputing (publishedVersion)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Reinforcement Learning in Robotics
- Physical Sciences › Computer Science › Computational Theory and Mathematics › Adaptive Dynamic Programming Control
- Physical Sciences › Computer Science › Artificial Intelligence › Neural Networks and Applications
- Keywords
- natural actor-critic, natural policy gradient, actor-critic algorithms, value function approximation, linear regression, learning control, robotic arm control
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review