Cited by 8,972Reinforcement Learning in RoboticsAdvanced Bandit Algorithms ResearchMachine Learning and Algorithms

The DOI matches its Crossref record; 149 of the 162 references it cites were found in a registry or an index.

Reinforcement Learning: A Survey

Open access

Kaelbling, L. P., Littman, M. L., Moore, A. W.·1996·Journal of Artificial Intelligence Research

Open this paper in CoChat — free account

Abstract

This paper surveys the field of reinforcement learning from a computer-science perspective. It is written to be accessible to researchers familiar with machine learning. Both the historical basis of the field and a broad selection of current work are summarized. Reinforcement learning is the problem faced by an agent that learns behavior through trial-and-error interactions with a dynamic environment. The work described here has a resemblance to work in psychology, but differs considerably in the details and in the use of the word ``reinforcement.'' The paper discusses central issues of reinforcement learning, including trading off exploration and exploitation, establishing the foundations of the field via Markov decision theory, learning from delayed reinforcement, constructing empirical models to accelerate learning, making use of generalization and hierarchy, and coping with hidden state. It concludes with a survey of some implemented systems and an assessment of the practical utility of current methods for reinforcement learning.

Integrity signals

Registry

Crossref · 10.1613/jair.301

Cites

149 of 149 references verified0 citation errors · 0 not checkable

Statistics

Recomputing p-values needs the paper’s full text.

Claims the paper makes

We couldn’t retrieve this paper’s full text, so its claims and self-check weren’t run. The registry and citation checks still apply.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Genetic algorithms in search, optimization, and machine learning
(1989) · Choice Reviews Online · doi:10.5860/choice.27-0936 · Cited by 49,178
Adaptation in Natural and Artificial Systems
Holland, J. H. (1992) · The MIT Press eBooks · doi:10.7551/mitpress/1090.001.0001 · Cited by 47,340
Genetic Algorithms in Search, Optimization and Machine Learning
Goldberg, D. E., & Robson, D. (1988) · Cited by 18,391
Dynamic Programming
Bellman, R. E. (2010) · doi:10.1515/9781400835386 · Cited by 14,588
Untitled work
(1995) · Cited by 10,558
Q-learning
Watkins, C. J. C. H., & Dayan, P. (1992) · Machine Learning · doi:10.1007/bf00992698 · Cited by 9,083
Simple Statistical Gradient-Following Algorithms for Connectionist Reinforcement Learning
Williams, R. J. (1992) · Machine Learning · doi:10.1007/bf00992696 · Cited by 8,608
Markov decision processes: Discrete stochastic dynamic programming
Hazeghi, K., & Puterman, M. L. (1995) · Journal of the American Statistical Association · doi:10.2307/2291177 · Cited by 8,037
Locally Weighted Regression: An Approach to Regression Analysis by Local Fitting
Cleveland, W. S., & Devlin, S. J. (1988) · Journal of the American Statistical Association · doi:10.1080/01621459.1988.10478639 · Cited by 5,893
Learning from Delayed Rewards
Watkins, C. (1989) · OpenGrey (Institut de l'Information Scientifique et Technique) · doi:10.17863/cam.132164 · Cited by 5,674
Theory of Linear and Integer Programming
Letchford, A. N., & Schrijver, A. (2000) · Journal of the Operational Research Society · doi:10.2307/253980 · Cited by 5,573
Parallel and Distributed Computation: Numerical Methods
Bertsekas, D. P., & Tsitsiklis, J. N. (1989) · DSpace@MIT (Massachusetts Institute of Technology) · Cited by 5,293
Empirical Model Building and Response Surfaces
Freeny, A. E., Box, G. E. P., & Draper, N. R. (1988) · Technometrics · doi:10.2307/1270169 · Cited by 5,238
Learning to predict by the methods of temporal differences
Sutton, R. S. (1988) · Machine Learning · doi:10.1007/bf00115009 · Cited by 5,148
Some Studies in Machine Learning Using the Game of Checkers
Samuel, A. L. (1959) · IBM Journal of Research and Development · doi:10.1147/rd.33.0210 · Cited by 4,443
A Theory of the Learnable
Valiant, L. G. (1984) · ACM eBooks · doi:10.1145/3749740.3749746 · Cited by 4,211
Dynamic Programming and Markov Processes.
Freimer, M. L., & Howard, R. A. (1961) · Journal of the American Statistical Association · doi:10.2307/2282268 · Cited by 3,388
Neuronlike adaptive elements that can solve difficult learning control problems
Barto, A. G., Sutton, R. S., & Anderson, C. W. (1983) · IEEE Transactions on Systems Man and Cybernetics · doi:10.1109/tsmc.1983.6313077 · Cited by 3,325
Theory of Linear and Integer Programming
Smith, D. K., & Schrijver, A. (1987) · Journal of the Operational Research Society · doi:10.2307/2582770 · Cited by 2,840
Markov games as a framework for multi-agent reinforcement learning
Littman, M. L. (1994) · Elsevier eBooks · doi:10.1016/b978-1-55860-335-6.50027-1 · Cited by 2,789
Self-citation
A New Approach to Manipulator Control: The Cerebellar Model Articulation Controller (CMAC)
Albus, J. S. (1975) · Journal of Dynamic Systems, Measurement, and Control · doi:10.1115/1.3426922 · Cited by 2,177 · cited 1×
“A very di(cid:11)erent kind of function approximator (CMAC (Albus, 1975)) that has weak generalization.”
Background
Empirical Model-Building and Response Surfaces.
Kshirsagar, A. M., Box, G. E. P., & Draper, N. R. (1988) · Journal of the American Statistical Association · doi:10.2307/2288890 · Cited by 1,894
Markov Decision Processes: Discrete Stochastic Dynamic Programming
Baxter, L. A., & Puterman, M. L. (1995) · Technometrics · doi:10.2307/1269932 · Cited by 1,889
Learning Automata: An Introduction
Narendra, K. S., & Thathachar, M. A. L. (1989) · CERN Document Server (European Organization for Nuclear Research) · Cited by 1,556
On-line Q-learning Using Connectionist Systems
Rummery, G. A. (1994) · Cited by 1,477
Some studies in machine learning using the game of checkers
Samuel, A. L. (2000) · IBM Journal of Research and Development · doi:10.1147/rd.441.0206 · Cited by 1,439
Classifier Fitness Based on Accuracy
Wilson, S. W. (1995) · Evolutionary Computation · doi:10.1162/evco.1995.3.2.149 · Cited by 1,396
Integrated Architectures for Learning, Planning, and Reacting Based on Approximating Dynamic Programming
Sutton, R. S. (1990) · Elsevier eBooks · doi:10.1016/b978-1-55860-141-3.50030-4 · Cited by 1,381
Dynamic Programming: Deterministic and Stochastic Models
Bertsekas, D. P. (1987) · Medical Entomology and Zoology · Cited by 1,348
Learning to act using real-time dynamic programming
Barto, A. G., Bradtke, S. J., & Singh, S. P. (1995) · Artificial Intelligence · doi:10.1016/0004-3702(94)00011-o · Cited by 1,129
Generalization in Reinforcement Learning: Successful Examples Using Sparse Coarse Coding
Sutton, R. S. (1995) · Cited by 1,072
Simple Statistical Gradient-Following Algorithms for Connectionist Reinforcement Learning
Williams, R. J. (1992) · Springer eBooks · doi:10.1007/978-1-4615-3618-5_2 · Cited by 1,024
Residual Algorithms: Reinforcement Learning with Function Approximation
Baird, L. C. (1995) · Elsevier eBooks · doi:10.1016/b978-1-55860-377-6.50013-x · Cited by 976
Finite State Markovian Decision Processes
Derman, C. (1970) · Cited by 853
Asynchronous Stochastic Approximation and Q-Learning
Tsitsiklis, J. N. (1994) · Machine Learning · doi:10.1023/a:1022689125041 · Cited by 844
Prioritized Sweeping: Reinforcement Learning with Less Data and Less Time
Moore, A., & Atkeson, C. G. (1993) · Machine Learning · doi:10.1023/a:1022635613229 · Cited by 842
Self-citation
Reinforcement learning for robots using neural networks
Lin, L.-J. (1992) · Defense Technical Information Center (DTIC) · Cited by 841
TD-Gammon, a Self-Teaching Backgammon Program, Achieves Master-Level Play
Tesauro, G. J. (1994) · Neural Computation · doi:10.1162/neco.1994.6.2.215 · Cited by 821
Practical Issues in Temporal Difference Learning
Tesauro, G. J. (1992) · Machine Learning · doi:10.1007/bf00992697 · Cited by 818
On the Convergence of Stochastic Iterative Dynamic Programming Algorithms
Jaakkola, T., Jordan, M. I., & Singh, S. P. (1994) · Neural Computation · doi:10.1162/neco.1994.6.6.1185 · Cited by 801
Temporal Difference Learning and TD-Gammon
Tesauro, G. (1995) · ICGA Journal · doi:10.3233/icg-1995-18207 · Cited by 784
Temporal credit assignment in reinforcement learning
Sutton, R. S. (1984) · ScholarWorks@UMassAmherst (University of Massachusetts Amherst) · Cited by 749
Parallel Distributed Processing: Explorations in the microstructures of cognition. By David E. Rumelhart, James L. McClelland, and the PDP Research Group. Vol. I, Foundations; Vol. II, Psychological and biological models. Cambridge, MA: MIT Press, 1986. Pp. xx, 547; xii, 611.
Sampson, G. R. (1987) · Language · doi:10.2307/415721 · Cited by 742
Optimum systems control
Eslami, M. (1978) · Journal of the Franklin Institute · doi:10.1016/0016-0032(78)90124-2 · Cited by 707
Learning Automata - A Survey
Narendra, K. S., & Thathachar, M. A. L. (1974) · IEEE Transactions on Systems Man and Cybernetics · doi:10.1109/tsmc.1974.5408453 · Cited by 672
Learning policies for partially observable environments: Scaling up
Littman, M. L., Cassandra, A. R., & Kaelbling, L. P. (1995) · Elsevier eBooks · doi:10.1016/b978-1-55860-377-6.50052-9 · Cited by 659
Self-citation
Reinforcement Learning with Replacing Eligibility Traces
Singh, S. P., & Sutton, R. S. (1996) · Machine Learning · doi:10.1007/bf00114726 · Cited by 624
Curious model-building control systems
Schmidhuber, J. (1991) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn.1991.170605 · Cited by 601
Learning in Embedded Systems
Kaelbling, L. P. (1993) · doi:10.7551/mitpress/4168.001.0001 · Cited by 599
Self-citation
Markov Decision Processes: Discrete Stochastic Dynamic Programming
Puterman, M. L. (2005) · Cited by 588
Automatic programming of behavior-based robots using reinforcement learning
Mahadevan, S., & Connell, J. H. (1992) · Artificial Intelligence · doi:10.1016/0004-3702(92)90058-6 · Cited by 583
Bandit Problems, Sequential Allocation of Experiments.
Jones, P. W., Berry, D. A., & Fristedt, B. (1987) · Journal of the Operational Research Society · doi:10.2307/2582852 · Cited by 583
Acting Optimally in Partially Observable Stochastic Domains
Cassandra, A. R., Kaelbling, L. P., & Littman, M. L. (1994) · Cited by 570
Self-citation
Feudal Reinforcement Learning
Dayan, P., & Hinton, G. E. (1992) · Cited by 540
A survey of algorithmic methods for partially observed Markov decision processes
Lovejoy, W. S. (1991) · Annals of Operations Research · doi:10.1007/bf02055574 · Cited by 527
The complexity of stochastic games
Condon, A. (1992) · Information and Computation · doi:10.1016/0890-5401(92)90048-k · Cited by 524
Generalization in Reinforcement Learning: Safely Approximating the Value Function
Boyan, J. A., & Moore, A. (1994) · Cited by 508
Self-citation
Improving Elevator Performance Using Reinforcement Learning
Crites, R. H., & Barto, A. G. (1995) · Cited by 499
Empirical model building and response surfaces
Box, G. E. P., & Draper, N. R. (1987) · Cited by 483
Neural Network Perception for Mobile Robot Guidance
Pomerleau, D. (1993) · Springer eBooks · doi:10.1007/978-1-4615-3192-0 · Cited by 481
Stable Function Approximation in Dynamic Programming
Gordon, G. J. (1995) · Elsevier eBooks · doi:10.1016/b978-1-55860-377-6.50040-2 · Cited by 458
Feature-Based Methods for Large Scale Dynamic Programming
Tsitsiklis, J. N., & Roy, B. V. (1996) · Machine Learning · doi:10.1007/bf00114724 · Cited by 452
Brains, Behavior and Robotics
Albus, J. S. (1981) · DigitalGeorgetown (Georgetown University Library) · Cited by 448
On the Complexity of Solving Markov Decision ProblemsPreprint
Littman, M. L., Dean, T., & Kaelbling, L. P. (2013) · arXiv (Cornell University) · doi:10.48550/arxiv.1302.4971 · Cited by 434
Self-citation
Average Reward Reinforcement Learning: Foundations, Algorithms, and Empirical Results
Mahadevan, S. (1996) · Machine Learning · doi:10.1007/bf00114727 · Cited by 419
Reward Functions for Accelerated Learning
Matarić, M. J. (1994) · Elsevier eBooks · doi:10.1016/b978-1-55860-335-6.50030-1 · Cited by 416
Reinforcement learning with selective perception and hidden state
McCallum, A. K., & Ballard, D. H. (1996) · Cited by 415
Stochastic Optimal Control: Theory and Application
Stengel, R. F. (1986) · Medical Entomology and Zoology · Cited by 408
A reinforcement learning approach to job-shop scheduling
Zhang, W., & Dietterich, T. G. (1995) · Cited by 378
Learning to coordinate behaviors
Maes, P., & Brooks, R. A. (1990) · Cited by 376
A Survey of Partially Observable Markov Decision Processes: Theory, Models, and Algorithms
Monahan, G. E., & MONAHANt, G. E. (2007) · Cited by 371
Bandit Problems: Sequential Allocation of Experiments.
Bather, J., Berry, D., & Fristedt, B. E. (1986) · Journal of the Royal Statistical Society Series A (General) · doi:10.2307/2981558 · Cited by 336
The Convergence of TD(λ) for General λ
Dayan, P. (1992) · Machine Learning · doi:10.1007/bf00992701 · Cited by 328
Reinforcement learning with perceptual aliasing: the perceptual distinctions approach
Chrisman, L. (1992) · Cited by 316
Issues in Using Function Approximation for Reinforcement Learning
Thrun, S., & Schwartz, A. (1999) · Cited by 311
A Reinforcement Learning Method for Maximizing Undiscounted Rewards
Schwartz, A. (1993) · Elsevier eBooks · doi:10.1016/b978-1-55860-307-3.50045-9 · Cited by 310
Transfer of Learning by Composing Solutions of Elemental Sequential Tasks
Singh, S. P. (1992) · Machine Learning · doi:10.1007/bf00992700 · Cited by 301
A stochastic reinforcement learning algorithm for learning real-valued functions
Gullapalli, V. (1990) · Neural Networks · doi:10.1016/0893-6080(90)90056-q · Cited by 292
Optimum systems control
Sage, A. P., & White, C. C. (1977) · Defense Technical Information Center (DTIC) · Cited by 271
The Parti-game Algorithm for Variable Resolution Reinforcement Learning in Multidimensional State-spaces
Moore, A., & Atkeson, C. G. (1995) · Machine Learning · doi:10.1023/a:1022656217772 · Cited by 261
Self-citation
On Nonterminating Stochastic Games
Hoffman, A. J., & Karp, R. M. (1966) · Management Science · doi:10.1287/mnsc.12.5.359 · Cited by 260
Theories of Learning, 4th Edition. By Ernest R. Hilgard and Gordon H. Bower. Englewood Cliffs, N.J.: Prentice-Hall, Inc., 1975
Bandlow, R. J. (1976) · NASSP Bulletin · doi:10.1177/019263657606040024 · Cited by 257
Input generalization in delayed reinforcement learning: an algorithm and performance comparisons
Chapman, D., & Kaelbling, L. P. (1991) · Cited by 245
Self-citation
Robot juggling: implementation of memory-based learning
Schaal, S., & Atkeson, C. G. (1994) · IEEE Control Systems · doi:10.1109/37.257895 · Cited by 243
Modified Policy Iteration Algorithms for Discounted Markov Decision Problems
Puterman, M. L., & Shin, M. C. (1978) · Management Science · doi:10.1287/mnsc.24.11.1127 · Cited by 240
Robot shaping: developing autonomous agents through learning
Dorigo, M., & Colombetti, M. (1994) · Artificial Intelligence · doi:10.1016/0004-3702(94)90047-7 · Cited by 238
Numerical dynamic programming in economicsPreprint
Rust, J. P. (1996) · RePEc: Research Papers in Economics · Cited by 237
The role of exploration in learning control
Thrun, S. (1992) · Cited by 226
Efficient Learning and Planning Within the Dyna Framework
Peng, J., & Williams, R. J. (1993) · Adaptive Behavior · doi:10.1177/105971239300100403 · Cited by 208
A Probabilistic Production and Inventory Problem
d'Epenoux, F. (1963) · Management Science · doi:10.1287/mnsc.10.1.98 · Cited by 202
Dynamic Programming and Markov Processes
Beranek, W., & Howard, R. A. (1961) · Technometrics · doi:10.2307/1266484 · Cited by 201
Stochastic Optimal Control
Malliaris, A. G. (1987) · The New Palgrave Dictionary of Economics · doi:10.1057/978-1-349-95121-5_1601-1 · Cited by 185
Planning with deadlines in stochastic domains
Dean, T., Kaelbling, L. P., Kirman, J., & Nicholson, A. E. (1993) · Cited by 183
Self-citation
Instance-Based Utile Distinctions for Reinforcement Learning with Hidden State
McCallum, R. A. (1995) · Elsevier eBooks · doi:10.1016/b978-1-55860-377-6.50055-4 · Cited by 182
TD(?) converges with probability 1
Dayan, P., & B, T. J. S. A. (1994) · Machine Learning · doi:10.1023/a:1022657612745 · Cited by 174
Adaptive aggregation methods for infinite horizon dynamic programming
Bertsekas, D. P., & Castañón, D. A. (1989) · IEEE Transactions on Automatic Control · doi:10.1109/9.24227 · Cited by 174
Hierarchical Learning in Stochastic Domains: Preliminary Results
Kaelbling, L. P. (1993) · Elsevier eBooks · doi:10.1016/b978-1-55860-307-3.50028-9 · Cited by 171
Self-citation
Variable Resolution Dynamic Programming: Efficiently Learning Action Maps in Multivariate Real-valued State-spaces
Moore, A. (1991) · Elsevier eBooks · doi:10.1016/b978-1-55860-200-7.50069-6 · Cited by 168
Self-citation
Memoryless Policies: Theoretical Limitations and Practical Results
Littman, M. L. (1994) · The MIT Press eBooks · doi:10.7551/mitpress/3117.003.0041 · Cited by 167
Self-citation
Multi-Armed Bandit Allocation Indices
Jones, P. W., & Gittins, J. C. (1989) · Journal of the Operational Research Society · doi:10.2307/2582928 · Cited by 165
On the Convergence of Stochastic Iterative Dynamic Programming Algorithms
Jaakkola, T., Jordan, M. I., & Singh, S. P. (1993) · doi:10.21236/ada276517 · Cited by 161
Overcoming Incomplete Perception with Utile Distinction Memory
McCallum, R. A. (1993) · Elsevier eBooks · doi:10.1016/b978-1-55860-307-3.50031-9 · Cited by 157
Planning by Incremental Dynamic Programming
Sutton, R. S. (1991) · Elsevier eBooks · doi:10.1016/b978-1-55860-200-7.50073-8 · Cited by 153
Learning to Play the Game of Chess
Thrun, S. (1994) · Cited by 145
Complexity analysis of real-time reinforcement learning
Koenig, S., & Simmons, R. G. (1993) · Cited by 143
Programming robots using reinforcement learning and teaching
Lin, L.-J. (1991) · National Conference on Artificial Intelligence · Cited by 142
The parti-game algorithm for variable resolution reinforcement learning in multidimensional state-spaces
Moore, A., & Atkeson, C. G. (1995) · Machine Learning · doi:10.1007/bf00993591 · Cited by 138
Self-citation
Incremental Multi-Step Q-Learning
Peng, J., & Williams, R. J. (1994) · Elsevier eBooks · doi:10.1016/b978-1-55860-335-6.50035-0 · Cited by 131
Introduction to Control Theory, Including Optimal Control
Graham, A., & Burghes, D. N. (1980) · Medical Entomology and Zoology · Cited by 121
A self-learning rule-based controller employing approximate reasoning and neural net concepts
Lee, C.-C. (1991) · International Journal of Intelligent Systems · doi:10.1002/int.4550060105 · Cited by 119
Temporal Difference Learning of Position Evaluation in the Game of Go
Schraudolph, N. N., Dayan, P., & B, T. J. S. A. (1993) · MPG.PuRe (Max Planck Society) · Cited by 118
Asynchronous stochastic approximation and Q-learning
Tsitsiklis, J. N. (2002) · IEEE Conference on Decision and Control (CDC) · doi:10.1109/cdc.1993.325119 · Cited by 116
Adding Temporary Memory to ZCS
Cliff, D., & Ross, S. (1994) · Adaptive Behavior · doi:10.1177/105971239400300201 · Cited by 115
Learning and Problem Solving with Multilayer Connectionist Systems
Anderson, C. W. (1986) · Cited by 110
Mathematical and Computational Techniques for Multilevel Adaptive Methods
Rüde, U. (1993) · Society for Industrial and Applied Mathematics eBooks · doi:10.1137/1.9781611970968 · Cited by 110
Reinforcement learning with a hierarchy of abstract models
Singh, S. P. (1992) · Cited by 108
Rapid, safe, and incremental learning of navigation strategies
Millán, J. D. R. (1996) · IEEE Transactions on Systems Man and Cybernetics Part B (Cybernetics) · doi:10.1109/3477.499792 · Cited by 104
Learning in embedded systems
(1993) · Choice Reviews Online · doi:10.5860/choice.31-1560 · Cited by 101
Memory Approaches to Reinforcement Learning in Non-Markovian Domains
Lin, L., & Mitchell, T. M. (1992) · Cited by 100
Learning to act using real-time dynamic programming
Barto, A. G., Bradtke, S. J., & Singh, S. P. (1996) · The MIT Press eBooks · doi:10.7551/mitpress/2026.003.0005 · Cited by 92
Feature-Based Methods for Large Scale Dynamic Programming
Tsitsiklis, J. N., & Roy, B. V. (1996) · Springer eBooks · doi:10.1007/978-0-585-33656-5_5 · Cited by 90
Reinforcement Learning in Markovian and Non-Markovian Environments
Schmidhuber, J. (1990) · Cited by 89
Reinforcement Learning With High-Dimensional, Continuous Actions
Baird, I., Klopf Leemon, C., & Anderson Sam, H. (1993) · Defense Technical Information Center (DTIC) · doi:10.21236/ada280844 · Cited by 88
Efficient reinforcement learning
Fiechter, C.-N. (1994) · Conference on Learning Theory · doi:10.1145/180139.181019 · Cited by 79
Reinforcement learning and its application to control
Gullapalli, V. (1992) · Cited by 77
Complexity and Cooperation in Q-Learning
Whitehead, S. D. (1991) · Elsevier eBooks · doi:10.1016/b978-1-55860-200-7.50075-1 · Cited by 67
ALECSYS and the AutonoMouse: Learning to Control a Real Robot by Distributed Classifier Systems
Dorigo, M. (1995) · Machine Learning · doi:10.1023/a:1022649410928 · Cited by 67
Reinforcement Learning with Replacing Eligibility Traces
Singh, S. P., & Sutton, R. S. (2007) · Springer eBooks · doi:10.1007/978-0-585-33656-5_7 · Cited by 66
Scaling Reinforcement Learning to Robotics by Exploiting the Subsumption Architecture
Mahadevan, S., & Connell, J. H. (1991) · Elsevier eBooks · doi:10.1016/b978-1-55860-200-7.50068-4 · Cited by 64
Learning to Solve Markovian Decision Processes
Singh, S. P. (1993) · Cited by 63
Rapid Task Learning for Real Robots
Connell, J. H., & Mahadevan, S. (1993) · Springer eBooks · doi:10.1007/978-1-4615-3184-5_5 · Cited by 53
Generalization and Scaling in Reinforcement Learning
Ackley, D. H., & Littman, M. L. (1989) · Cited by 53
Self-citation
Continual Learning In Reinforcement Environments
Ring, M. B. (1994) · Cited by 50
Robust Reinforcement Learning in Motion Planning
Singh, S. P., Barto, A. G., Grupen, R. A., & Connolly, C. I. (1993) · Cited by 48
Associative Reinforcement Learning: Functions in k-DNF
Kaelbling, L. P. (1994) · Machine Learning · doi:10.1023/a:1022689909846 · Cited by 46
Self-citation
A Modular Q-Learning Architecture for Manipulator Task Decomposition
Tham, C., & Prager, R. W. (1994) · Elsevier eBooks · doi:10.1016/b978-1-55860-335-6.50045-3 · Cited by 41
Feature-based methods for large scale dynamic programming
Tsitsiklis, J. N., & Roy, B. V. (2002) · Proceedings of the IEEE Conference on Decision & Control, including the Symposium on Adaptive Processes · doi:10.1109/cdc.1995.478954 · Cited by 38
To Discount or not to Discount in Reinforcement Learning: A Case Study Comparing R Learning and Q Learning
Mahadevan, S. (1994) · Elsevier eBooks · doi:10.1016/b978-1-55860-335-6.50028-3 · Cited by 37
Associative Reinforcement Learning: A Generate and Test Algorithm
Kaelbling, L. P. (1994) · Machine Learning · doi:10.1023/a:1022642026684 · Cited by 29
Self-citation
Emergent Control and Planning in an Autonomous Vehicle
Meeden, L., McGraw, G. A., & Blank, D. (1993) · Scholarship, Research, and Creative Work at Bryn Mawr College (Bryn Mawr College) · Cited by 25
Memory-Based Learning for Control
Moore, A. W., Atkeson, C. G., & Schaal, S. A. (2018) · doi:10.1184/r1/6555530 · Cited by 24
Self-citation
Learning and problem-solving with multilayer connectionist systems (adaptive, strategy learning, neural networks, reinforcement learning)
Anderson, C. W. (1986) · ScholarWorks@UMassAmherst (University of Massachusetts Amherst) · Cited by 21
Associative reinforcement learning: A generate and test algorithm
Kaelbling, L. P. (1994) · Machine Learning · doi:10.1007/bf00993348 · Cited by 21
Self-citation
Transfer of Learning by Composing Solutions of Elemental Sequential Tasks
Singh, S. P. (1992) · Springer eBooks · doi:10.1007/978-1-4615-3618-5_6 · Cited by 20
Active Exploration and Learning in Real-Valued Spaces using Multi-Armed Bandit Allocation Indices
Salganicoff, M., & Ungar, L. (1995) · Elsevier eBooks · doi:10.1016/b978-1-55860-377-6.50066-9 · Cited by 16
Fast and Efficient Reinforcement Learning with Truncated Temporal Differences
Cichosz, P., & MULAWKA, J. J. (1995) · Elsevier eBooks · doi:10.1016/b978-1-55860-377-6.50021-9 · Cited by 14
Predicting Real-Time Planner Preformance by Domain Characterization
Kirman, J. (1994) · Cited by 9
Artificial Neural Networks and Approximate Reasoning for Intelligent Control in Space
Berenji (1991) · doi:10.1109/acc.1991.4174548 · Cited by 6
Introduction to control theory, including optimal control
Burghes, D. N., & Graham, A. (1980) · CERN Document Server (European Organization for Nuclear Research) · Cited by 4

Sorted by citations

Deep learning in neural networks: an overview
Schmidhuber, J. (2015) · Neural Networks · doi:10.1016/j.neunet.2014.09.003 · Cited by 18,570
Ant colony system: a cooperative learning approach to the traveling salesman problem
Dorigo, M., & Gambardella, L. M. (1997) · IEEE Transactions on Evolutionary Computation · doi:10.1109/4235.585892 · Cited by 8,081
Machine learning: Algorithms, real-world applications and research directions
Sarker, I. H. (2021) · SN Computer Science · doi:10.1007/s42979-021-00592-x · Cited by 5,583
An integrative theory of locus coeruleus-norepinephrine function: adaptive gain and optimal performance
Aston-Jones, G., & Cohen, J. D. (2005) · Annual Review of Neuroscience · doi:10.1146/annurev.neuro.28.061604.135709 · Cited by 4,472
Efficient Processing of Deep Neural Networks: A Tutorial and Survey
Sze, V., Chen, Y.-H., Yang, T.-J., & Emer, J. S. (2017) · Proceedings of the IEEE · doi:10.1109/jproc.2017.2761740 · Cited by 4,215
“y Recently, many of the grand AI challenges involving game play have been overcome using DNNs. These successes also required innovations in training techniques and many rely on reinforcement learning [30]. DNNs have surpassed human level accuracy in playing Atari [31] as well as Go [6], where an exhaustive search of a…”
Background

See all 8,972

Authors: Kaelbling, L. P., Littman, M. L., & Moore, A. W.
Affiliations: Brown University (US)
Carnegie Mellon University (US)
ORCID: Kaelbling, L. P.
Littman, M. L.
Moore, A. W.
Venue: Journal of Artificial Intelligence Research
Publisher: AI Access Foundation
Published: 1996-05-01
Volume: 4
Pages: 237-285
ISSN: 1076-9757 (ISSN-L)
DOI: 10.1613/jair.301
OpenAlex: W2107726111
Cited by: 8,972
References: 162
Citations by year: 2026: 367 · 2025: 703 · 2024: 626 · 2023: 597 · 2022: 498 · 2021: 549 · 2020: 583 · 2019: 416 · 2018: 351 · 2017: 266 · 2016: 201 · 2015: 218 · 2014: 233 · 2013: 272 · 2012: 225
FWCI: 94.14
Type: journal article
Language: en
Open access: diamond
doi.org
Licence: cc-by
Locations: Journal of Artificial Intelligence Research (publishedVersion, cc-by)
arXiv (Cornell University) (submittedVersion)
arXiv (Cornell University)
Funders: National Science Foundation (9453383, 9312395)
Topics: Physical Sciences › Computer Science › Artificial Intelligence › Reinforcement Learning in Robotics
Social Sciences › Decision Sciences › Management Science and Operations Research › Advanced Bandit Algorithms Research
Physical Sciences › Computer Science › Artificial Intelligence › Machine Learning and Algorithms
Keywords: reinforcement learning, Markov decision theory, exploration-exploitation balance, delayed reinforcement, empirical model, generalization, hierarchical learning, hidden states, learning from interaction, agent-environment interaction

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

Reinforcement Learning: A Survey

Abstract

Overview

Integrity signals

Verification

Claims the paper makes

Claims other papers make about this one

Citations

Cites

Cited by

Related

Metadata

Record

About these checks