Natural actor–critic algorithms

Reinforcement Learning: An Introduction

Sutton, R. S., & Barto, A. G. (1998) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.1998.712192 · Cited by 27,225

Reinforcement Learning: An Introduction

(2005) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.2004.842673 · Cited by 24,496

Nonlinear Programming

Bertsekas, D. P. (1997) · Journal of the Operational Research Society · doi:10.1038/sj.jors.2600425 · Cited by 11,726

Untitled work

(1995) · Cited by 10,558

Markov decision processes: Discrete stochastic dynamic programming

Hazeghi, K., & Puterman, M. L. (1995) · Journal of the American Statistical Association · doi:10.2307/2291177 · Cited by 8,037

Learning to predict by the methods of temporal differences

Sutton, R. S. (1988) · Machine Learning · doi:10.1007/bf00115009 · Cited by 5,148

Untitled work

(1999) · Cited by 5,147

Adaptive Signal Processing

(1991) · Springer eBooks · doi:10.1007/978-3-7091-2840-4 · Cited by 5,095

Neuro-dynamic Programming

(2012) · Encyclopedia of the Sciences of Learning · doi:10.1007/978-1-4419-1428-6_5086 · Cited by 3,515

Simple Statistical Gradient-Following Algorithms for Connectionist Reinforcement Learning

Williams, R. J. (1992) · Machine Learning · doi:10.1007/bf00992696 · Cited by 3,416

Neuronlike adaptive elements that can solve difficult learning control problems

Barto, A. G., Sutton, R. S., & Anderson, C. W. (1983) · IEEE Transactions on Systems, Man, and Cybernetics · doi:10.1109/tsmc.1983.6313077 · Cited by 3,233

Natural Gradient Works Efficiently in Learning

Amari, S.-I. (1998) · Neural Computation · doi:10.1162/089976698300017746 · Cited by 2,681

Adaptive Algorithms and Stochastic Approximations

Benveniste, A., Métivier, M., & Priouret, P. (1990) · Springer eBooks · doi:10.1007/978-3-642-75894-2 · Cited by 2,006

Temporal difference learning and TD-Gammon

Tesauro, G. (1995) · Communications of the ACM · doi:10.1145/203330.203343 · Cited by 1,618 · cited 2×

Adaptive signal processing

(1986) · Signal Processing · doi:10.1016/0165-1684(86)90025-3 · Cited by 1,594

On-line Q-learning Using Connectionist Systems

Rummery, G. A. (1994) · Cited by 1,477

Neuro-Dynamic Programming: NDP

Bertsekas, D. P. (2001) · Encyclopedia of Optimization · doi:10.1007/0-306-48332-7_333 · Cited by 1,466

Stochastic Approximation Methods for Constrained and Unconstrained Systems

Kushner, H. J., & Clark, D. S. (1978) · Applied Mathematical Sciences · doi:10.1007/978-1-4684-9352-8 · Cited by 1,459

An analysis of temporal-difference learning with function approximation

Tsitsiklis, J. N., & Van Roy, B. (1997) · IEEE Transactions on Automatic Control · doi:10.1109/9.580874 · Cited by 1,194

Parallel and distributed computation

Bertsekas, D. P., Tsitsiklis, J. N., & Sennikov, A. N. (1989) · Cited by 1,187

Generalization in Reinforcement Learning: Successful Examples Using Sparse Coarse Coding

Sutton, R. S. (1995) · Cited by 1,072

Stochastic Approximation Algorithms and Applications

Puhalskii, A., Kushner, H. J., & Yin, G. G. (1998) · Journal of the American Statistical Association · doi:10.2307/2669885 · Cited by 994

Residual Algorithms: Reinforcement Learning with Function Approximation

Baird, L. (1995) · Machine Learning Proceedings 1995 · doi:10.1016/b978-1-55860-377-6.50013-x · Cited by 918

Reinforcement learning of motor skills with policy gradients

Peters, J., & Schaal, S. (2008) · Neural Networks · doi:10.1016/j.neunet.2008.02.003 · Cited by 831

A Natural Policy Gradient

Kakade, S. M. (2001) · ScholarlyCommons (University of Pennsylvania) · Cited by 775

Temporal Difference Learning and TD-Gammon

Tesauro, G. (1995) · ICGA Journal · doi:10.3233/icg-1995-18207 · Cited by 764

Temporal credit assignment in reinforcement learning

Sutton, R. S. (1984) · ScholarWorks@UMassAmherst (University of Massachusetts Amherst) · Cited by 749

Natural Actor-Critic

Peters, J., & Schaal, S. (2008) · Neurocomputing · doi:10.1016/j.neucom.2007.11.026 · Cited by 738

OnActor-Critic Algorithms

Konda, V. R., & Tsitsiklis, J. N. (2003) · SIAM Journal on Control and Optimization · doi:10.1137/s0363012901385691 · Cited by 709

Linear Least-Squares algorithms for temporal difference learning

Bradtke, S. J., & Barto, A. G. (1996) · Machine Learning · doi:10.1007/bf00114723 · Cited by 599

Convergent activation dynamics in continuous time networks

Hirsch, M. W. (1989) · Neural Networks · doi:10.1016/0893-6080(89)90018-x · Cited by 587

Policy gradient reinforcement learning for fast quadrupedal locomotion

Kohl, N., & Stone, P. (2004) · IEEE International Conference on Robotics and Automation, 2004. Proceedings. ICRA '04. 2004 · doi:10.1109/robot.2004.1307456 · Cited by 564

Autonomous Inverted Helicopter Flight via Reinforcement Learning

Ng, A. Y., Coates, A., Diel, M., Ganapathi, V., Schulte, J., Tse, B., Berger, E., & Liang, E. (2006) · Springer Tracts in Advanced Robotics · doi:10.1007/11552246_35 · Cited by 553

Nonlinear Programming

Bard, J. F. (1998) · Nonconvex optimization and its applications · doi:10.1007/978-1-4757-2836-1_4 · Cited by 550

Control Techniques for Complex Networks

Meyn, S. (2007) · Cambridge University Press eBooks · doi:10.1017/cbo9780511804410 · Cited by 528

Infinite-Horizon Policy-Gradient Estimation

Baxter, J., & Bartlett, P. L. (2001) · Cited by 525

Generalization in Reinforcement Learning: Safely Approximating the Value Function

Boyan, J. A., & Moore, A. (1994) · Cited by 508

The O.D.E. Method for Convergence of Stochastic Approximation and Reinforcement Learning

Borkar, V. S., & Meyn, S. P. (2000) · SIAM Journal on Control and Optimization · doi:10.1137/s0363012997331639 · Cited by 499

Asynchronous stochastic approximation and Q-learning

Tsitsiklis, J. N. (1994) · Machine Learning · doi:10.1007/bf00993306 · Cited by 453

Stable Function Approximation in Dynamic Programming

Gordon, G. J. (1995) · Machine Learning Proceedings 1995 · doi:10.1016/b978-1-55860-377-6.50040-2 · Cited by 446

Stochastic approximation with two time scales

Borkar, V. S. (1997) · Systems & Control Letters · doi:10.1016/s0167-6911(97)90015-3 · Cited by 389

Likelihood ratio gradient estimation for stochastic systems

Glynn, P. W. (1990) · Communications of the ACM · doi:10.1145/84537.84552 · Cited by 381

FOR LEAST-SQUARES POLICY ITERATION

Schwab, D. (2016) · Cited by 372

Infinite-Horizon Policy-Gradient Estimation

Baxter, J., & Bartlett, P. L. (2001) · Journal of Artificial Intelligence Research · doi:10.1613/jair.806 · Cited by 343

Reinforcement Learning for Humanoid Robotics

Peters, J., Vijayakumar, S., & Schaal, S. (2003) · Cited by 309

Simulation-based optimization of Markov reward processes

Marbach, P., & Tsitsiklis, J. N. (2001) · IEEE Transactions on Automatic Control · doi:10.1109/9.905687 · Cited by 306

Variance Reduction Techniques for Gradient Estimates in Reinforcement Learning

Greensmith, E., Bartlett, P. L., & Baxter, J. (2004) · doi:10.5555/1005332.1044710 · Cited by 300

Elevator Group Control Using Multiple Reinforcement Learning Agents

Crites, R. H., & Barto, A. G. (1998) · Machine Learning · doi:10.1023/a:1007518724497 · Cited by 275

Nonconvergence to Unstable Points in Urn Models and Stochastic Approximations

Pemantle, R. (1990) · The Annals of Probability · doi:10.1214/aop/1176990853 · Cited by 263

Least-Squares Temporal Difference Learning

Boyan, J. A. (1999) · Cited by 259

Numerical dynamic programming in economicsPreprint

Rust, J. P. (1996) · RePEc: Research Papers in Economics · Cited by 237

Actor-Critic--Type Learning Algorithms for Markov Decision Processes

Konda, V. R., & Borkar, V. S. (1999) · SIAM Journal on Control and Optimization · doi:10.1137/s036301299731669x · Cited by 234

A Survey of Applications of Markov Decision Processes

White, D. J. (1993) · Journal of the Operational Research Society · doi:10.1057/jors.1993.181 · Cited by 208

Perturbation realization, potentials, and sensitivity analysis of Markov processes

Cao, X., & Chen, H.-F. (1997) · IEEE Transactions on Automatic Control · doi:10.1109/9.633827 · Cited by 208

Learning Algorithms for Markov Decision Processes with Average Cost

Abounadi, J., Bertsekas, D., & Borkar, V. S. (2001) · SIAM Journal on Control and Optimization · doi:10.1137/s0363012999361974 · Cited by 204

Incremental Natural Actor-Critic Algorithms

Bhatnagar, S., Ghavamzadeh, M., Lee, M., & Sutton, R. S. (2007) · Cited by 155

Chapter 14 Numerical dynamic programming in economics

Rust, J. P. (1996) · Handbook of computational economics · doi:10.1016/s1574-0021(96)01016-7 · Cited by 148

Covariant Policy Search

Bagnell, J. A., & Schneider, J. (2018) · Figshare · doi:10.1184/r1/6552458 · Cited by 148

Average cost temporal-difference learning

Tsitsiklis, J. N., & Roy, B. V. (1999) · Automatica · doi:10.1016/s0005-1098(99)00099-0 · Cited by 140

Functional approximations and dynamic programming

Bellman, R., & Dreyfus, S. E. (1959) · Mathematics of Computation · doi:10.1090/s0025-5718-1959-0107376-8 · Cited by 131

Regularized Policy Iteration

Farahmand, A. M., Ghavamzadeh, M., Mannor, S., & Szepesvári, C. (2008) · Cited by 110

Natural Actor-Critic for Road Traffic Optimisation

Richter, S., Aberdeen, D., & Yu, J. (2007) · The MIT Press eBooks · doi:10.7551/mitpress/7503.003.0151 · Cited by 96

Bayesian Policy Gradient Algorithms

Ghavamzadeh, M., & Engel, Y. (2007) · The MIT Press eBooks · doi:10.7551/mitpress/7503.003.0062 · Cited by 74

Adaptive multivariate three-timescale stochastic approximation algorithms for simulation based optimization

Bhatnagar, S. (2005) · ACM Transactions on Modeling and Computer Simulation · doi:10.1145/1044322.1044326 · Cited by 68

KnightCap: A Chess Programm That Learns by Combining TD(lambda) with Game-Tree Search

Baxter, J., Tridgell, A., & Weaver, L. (1998) · International Conference on Machine Learning · Cited by 65

Improved Temporal Difference Methods with Linear Function Approximation

Si, J., Barto, A. G., Powell, W. B., & Wunsch, D. (2009) · Institute of Electrical and Electronics Engineers eBooks · doi:10.1109/9780470544785.ch9 · Cited by 61

Advantage Updating

Baird, I., & Leemon, C. (1993) · doi:10.21236/ada280862 · Cited by 55

Adaptive Newton-based multivariate smoothed functional algorithms for simulation optimization

Bhatnagar, S. (2007) · ACM Transactions on Modeling and Computer Simulation · doi:10.1145/1315575.1315577 · Cited by 52

Hierarchical policy gradient algorithms

Ghavamzadeh, M., & Mahadevan, S. (2003) · ScholarWorks@UMassAmherst (University of Massachusetts Amherst) · Cited by 44

Some Pathological Traps for Stochastic Approximation

Brandière, O. (1998) · SIAM Journal on Control and Optimization · doi:10.1137/s036301299630759x · Cited by 40

A Simultaneous Perturbation Stochastic Approximation-Based Actor–Critic Algorithm for Markov Decision Processes

Bhatnagar, S., & Kumar, S. S. (2004) · IEEE Transactions on Automatic Control · doi:10.1109/tac.2004.825622 · Cited by 34

Average cost temporal-difference learning

Tsitsiklis, J. N., & Roy, B. V. (2002) · IEEE Conference on Decision and Control (CDC) · doi:10.1109/cdc.1997.650675 · Cited by 29

Reinforcement Learning Based Algorithms for Average Cost Markov Decision Processes

Abdulla, M. S., & Bhatnagar, S. (2007) · Discrete Event Dynamic Systems · doi:10.1007/s10626-006-0003-y · Cited by 17

Information theoretic justification of Boltzmann selection and its generalization to Tsallis case

Dukkipati, A., Murty, M. N., & Bhatnagar, S. (2005) · IEEE Congress on Evolutionary Computation (CEC) · doi:10.1109/cec.2005.1554889 · Cited by 7

Algorithms for Reinforcement Learning

Szepesvári, C. (2010) · Synthesis Lectures on Artificial Intelligence and Machine Learning · doi:10.1007/978-3-031-01551-9 · Cited by 1,049

A Survey of Actor-Critic Reinforcement Learning: Standard and Natural Policy Gradients

Grondman, I., Busoniu, L., Lopes, G. A. D., & Babuska, R. (2012) · IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews) · doi:10.1109/tsmcc.2012.2218595 · Cited by 1,042

Reinforcement learning algorithms: A brief survey

Shakya, A. K., Pillai, G., & Chakrabarty, S. (2023) · Expert Systems with Applications · doi:10.1016/j.eswa.2023.120495 · Cited by 656

Fully Decentralized Multi-Agent Reinforcement Learning with Networked Agents

Zhang, K., Yang, Z., Liu, H., Zhang, T., & Başar, T. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1802.08757 · Cited by 293

Model-Free reinforcement learning with continuous action in practice

Degris, T., Pilarski, P. M., & Sutton, R. S. (2012) · 2012 American Control Conference (ACC) · doi:10.1109/acc.2012.6315022 · Cited by 239