The DOI matches its Crossref record; 32 of the 32 references it cites were found in a registry or an index.
Reinforcement learning is direct adaptive optimal control
Abstract
Neural network reinforcement learning methods are described and considered as a direct approach to adaptive optimal control of nonlinear systems. These methods have their roots in studies of animal learning and in early learning control work. An emerging deeper understanding of these methods is summarized that is obtained by viewing them as a synthesis of dynamic programming and stochastic approximation methods. The focus is on Q-learning systems, which maintain estimates of utilities for all state-action pairs and make use of these estimates to select actions. The use of hybrid direct/indirect methods is briefly discussed.>
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Watkins, C. (1989) · OpenGrey (Institut de l'Information Scientifique et Technique) · doi:10.17863/cam.132164 · Cited by 5,297
Learning to predict by the methods of temporal differences
Sutton, R. S. (1988) · Machine Learning · doi:10.1007/bf00115009 · Cited by 5,148
Self-citationSome Studies in Machine Learning Using the Game of Checkers
Samuel, A. L. (1959) · IBM Journal of Research and Development · doi:10.1147/rd.33.0210 · Cited by 4,443
Neuronlike adaptive elements that can solve difficult learning control problems
Barto, A. G., Sutton, R. S., & Anderson, C. W. (1983) · IEEE Transactions on Systems, Man, and Cybernetics · doi:10.1109/tsmc.1983.6313077 · Cited by 3,233
Pattern Classification and Scene Analysis.
Clarke, M. R. B., Duda, R. O., & Hart, P. E. (1974) · Journal of the Royal Statistical Society. Series A (General) · doi:10.2307/2344977 · Cited by 3,222
Learning Automata: An Introduction
Narendra, K. S., & Thathachar, M. A. L. (1989) · CERN Document Server (European Organization for Nuclear Research) · Cited by 1,540
Korf, R. E. (1990) · Artificial Intelligence · doi:10.1016/0004-3702(90)90054-4 · Cited by 917
Temporal credit assignment in reinforcement learning
Sutton, R. S. (1984) · ScholarWorks@UMassAmherst (University of Massachusetts Amherst) · Cited by 749
Berry, D. A., & Fristedt, B. (1985) · Springer eBooks · doi:10.1007/978-94-015-3711-7 · Cited by 633
Automatic programming of behavior-based robots using reinforcement learning
Mahadevan, S., & Connell, J. (1992) · Artificial Intelligence · doi:10.1016/0004-3702(92)90058-6 · Cited by 591
Bandit Problems, Sequential Allocation of Experiments.
Jones, P. W., Berry, D. A., & Fristedt, B. (1987) · The Journal of the Operational Research Society · doi:10.2307/2582852 · Cited by 566
Learning to control an inverted pendulum using neural networks
Anderson, C. W. (1989) · IEEE Control Systems Magazine · doi:10.1109/37.24809 · Cited by 540
Time-Derivative Models of Pavlovian Reinforcement
Sutton, R. S., & Barto, A. G. (1990) · Cited by 499
Learning and Sequential Decision Making
Barto, A. G., Sutton, R. S., & Watkins, C. (1989) · Cited by 357
Neural networks for control and system identification
Werbos, P. J. (2003) · Proceedings of the 28th IEEE Conference on Decision and Control · doi:10.1109/cdc.1989.70114 · Cited by 319
Pattern-recognizing stochastic learning automata
Barto, A. G., & Anandan, P. (1985) · IEEE Transactions on Systems, Man, and Cybernetics · doi:10.1109/tsmc.1985.6313371 · Cited by 305
Learning to perceive and act by trial and error
Whitehead, S. D., & Ballard, D. H. (1991) · Machine Learning · doi:10.1007/bf00058926 · Cited by 300
Efficient memory-based learning for robot control
Moore, A. W. (2021) · doi:10.48456/tr-209 · Cited by 282
Learning sequential decision rules using simulation models and competition
Grefenstette, J. J., Ramsey, C. L., & Schultz, A. C. (1990) · Machine Learning · doi:10.1007/bf00116876 · Cited by 271
Werbos, P. (1987) · IEEE Transactions on Systems, Man, and Cybernetics · doi:10.1109/tsmc.1987.289329 · Cited by 251
Associative search network: A reinforcement learning associative memory
Barto, A. G., Sutton, R. S., & Brouwer, P. S. (1981) · Biological Cybernetics · doi:10.1007/bf00453370 · Cited by 242
Hagan, M. T., & Demuth, H. B. (1999) · Proceedings of the ... American Control Conference/Proceedings of the American Control Conference · doi:10.1109/acc.1999.786109 · Cited by 225
Connectionist Learning for Control: An Overview
Barto, A. G. (1991) · Neural Networks for Control · doi:10.7551/mitpress/4939.003.0005 · Cited by 200
An adaptive optimal controller for discrete-time Markov environments
Witten, I. H. (1977) · Information and Control · doi:10.1016/s0019-9958(77)90354-0 · Cited by 180
Estimation and control in Markov chains
Mandl, P. (1974) · Advances in Applied Probability · doi:10.1017/s0001867800039707 · Cited by 177
Training and tracking in robotics
Selfridge, O. G., Sutton, R. S., & Barto, A. G. (1985) · Cited by 116
Decentralized learning in finite Markov chains
Wheeler, R. M., & Narendra, K. S. (1986) · IEEE Transactions on Automatic Control · doi:10.1109/tac.1986.1104342 · Cited by 98
First Results with Dyna, an Integrated Architecture for Learning, Planning and Reacting
Sutton, R. S. (1991) · The MIT Press eBooks · doi:10.7551/mitpress/4939.003.0012 · Cited by 65
Self-improving Reactive Agents: Case Studies of Reinforcement Learning Frameworks
Lin, L.-J. (1991) · The MIT Press eBooks · doi:10.7551/mitpress/3115.003.0041 · Cited by 49
On the Computational Economics of Reinforcement Learning
Barto, A. G., & Singh, S. P. (1991) · Elsevier eBooks · doi:10.1016/b978-1-4832-1448-1.50010-x · Cited by 39
REINFORCEMENT LEARNING AND DYNAMIC PROGRAMMING
Barto, A. G. (1995) · IFAC Proceedings Volumes · doi:10.1016/s1474-6670(17)45266-9 · Cited by 36
Learning control of finite Markov chains with an explicit trade-off between estimation and control
Sato, M., Abe, K., & Takeda, H. (1988) · IEEE Transactions on Systems Man and Cybernetics · doi:10.1109/21.21595 · Cited by 32
Cited by
Learning to act using real-time dynamic programming
Barto, A. G., Bradtke, S. J., & Singh, S. P. (1995) · Artificial Intelligence · doi:10.1016/0004-3702(94)00011-o · Cited by 1,104
A review On reinforcement learning: Introduction and applications in industrial process control
Nian, R., Liu, J., & Huang, B. (2020) · Computers & Chemical Engineering · doi:10.1016/j.compchemeng.2020.106886 · Cited by 711
Bhasin, S., Kamalapurkar, R., Johnson, M., Vamvoudakis, K. G., Lewis, F. L., & Dixon, W. E. (2013) · Automatica · doi:10.1016/j.automatica.2012.09.019 · Cited by 642
Vrabie, D., & Lewis, F. (2009) · Neural Networks · doi:10.1016/j.neunet.2009.03.008 · Cited by 634
Advances and opportunities in machine learning for process data analytics
Qin, S. J., & Chiang, L. H. (2019) · Computers & Chemical Engineering · doi:10.1016/j.compchemeng.2019.04.003 · Cited by 351
Metadata
- Venue
- IEEE Control Systems
- Published
- 1992-04
- Volume
- 12
- Issue
- 2
- Pages
- 19-22
- ISSN
- 1066-033X (ISSN-L)
- 1941-000X
- OpenAlex
- W1990005421
- Cited by
- 524
- References
- 32
- FWCI
- 9.31
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.