The DOI matches its Crossref record; 86 of the 86 references it cites were found in a registry or an index.
Reinforcement learning and adaptive dynamic programming for feedback control
Abstract
Living organisms learn by acting on their environment, observing the resulting reward stimulus, and adjusting their actions accordingly to improve the reward. This action-based or reinforcement learning can capture notions of optimal behavior occurring in natural systems. We describe mathematical formulations for reinforcement learning and a practical implementation method known as adaptive dynamic programming. These give us insight into the design of controllers for man-made engineered systems that both learn and exhibit optimal behavior.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Reinforcement Learning: An Introduction
Sutton, R. S., & Barto, A. G. (1998) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.1998.712192 · Cited by 27,225
Reinforcement Learning: An Introduction
(2005) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.2004.842673 · Cited by 24,496
Bellman, R. E. (2010) · doi:10.1515/9781400835386 · Cited by 14,588
A neural substrate of prediction and reward
Schultz, W., Dayan, P., & Montague, P. R. (1997) · Science · doi:10.1126/science.275.5306.1593 · Cited by 9,388
Watkins, C. J. C. H., & Dayan, P. (1992) · Machine Learning · doi:10.1007/bf00992698 · Cited by 9,083
Introduction to Reinforcement Learning
Sutton, R. S., & Barto, A. G. (1998) · The MIT Press eBooks · Cited by 6,855
Watkins, C. (1989) · OpenGrey (Institut de l'Information Scientifique et Technique) · doi:10.17863/cam.132164 · Cited by 5,297
Learning to predict by the methods of temporal differences
Sutton, R. S. (1988) · Machine Learning · doi:10.1007/bf00115009 · Cited by 5,148
(2012) · Encyclopedia of the Sciences of Learning · doi:10.1007/978-1-4419-1428-6_5086 · Cited by 3,515
Dynamic Programming and Markov Processes.
Freimer, M., & Howard, R. A. (1961) · Journal of the American Statistical Association · doi:10.2307/2282268 · Cited by 3,378
Cited by
(2012) · IEEE Control Systems · doi:10.1109/mcs.2012.2214134 · Cited by 1,105
Jiang, Y., & Jiang, Z.-P. (2012) · Automatica · doi:10.1016/j.automatica.2012.06.096 · Cited by 1,072
Optimal and Autonomous Control Using Reinforcement Learning: A Survey
Kiumarsi, B., Vamvoudakis, K. G., Modares, H., & Lewis, F. L. (2018) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2017.2773458 · Cited by 891
Self-citationPolicy Iteration Adaptive Dynamic Programming Algorithm for Discrete-Time Nonlinear Systems
Liu, D., & Wei, Q. (2014) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2013.2281663 · Cited by 753
Adaptive Dynamic Programming for Control: A Survey and Recent Advances
Liu, D., Xue, S., Zhao, B., Luo, B., & Wei, Q. (2021) · IEEE Transactions on Systems, Man, and Cybernetics: Systems · doi:10.1109/tsmc.2020.3042876 · Cited by 708
Zhang, H., Cui, L., Zhang, X., & Luo, Y. (2011) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.2011.2168538 · Cited by 643
Kiumarsi, B., Lewis, F. L., Modares, H., Karimpour, A., & Naghibi-Sistani, M.-B. (2014) · Automatica · doi:10.1016/j.automatica.2014.02.015 · Cited by 574
Self-citationModares, H., Lewis, F. L., & Naghibi-Sistani, M.-B. (2014) · Automatica · doi:10.1016/j.automatica.2013.09.043 · Cited by 570
Self-citationMachine learning in medical applications: a review of state-of-the-art methods
Shehab, M., Abualigah, L., Shambour, Q., Abu-Hashem, M. A., Shambour, M. K. Y., Alsalibi, A. I., & Gandomi, A. H. (2022) · Computers in Biology and Medicine · doi:10.1016/j.compbiomed.2022.105458 · Cited by 542
Lewis, F. L., & Vamvoudakis, K. G. (2011) · IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics) · doi:10.1109/tsmcb.2010.2043839 · Cited by 544
“Recent surveys are given by Lewis and Vrabie [19] and Wang et al. [38].”
MethodSelf-citationVamvoudakis, K. G., & Lewis, F. L. (2011) · Automatica · doi:10.1016/j.automatica.2011.03.005 · Cited by 525
“Reinforcement learning (RL) is a sub-area of machine learning concerned with how to methodically modify the actions of an agent (player) based on observed responses from its environment (Lewis & Vrabie, 2009, Powell, 2007, Sutton & Barto, 1998).”
BackgroundSelf-citationZhang, H., Zhang, J., Yang, G.-H., & Luo, Y. (2015) · IEEE Transactions on Fuzzy Systems · doi:10.1109/tfuzz.2014.2310238 · Cited by 519
Value Iteration Adaptive Dynamic Programming for Optimal Control of Discrete-Time Nonlinear Systems
Wei, Q., Liu, D., & Lin, H. (2016) · IEEE Transactions on Cybernetics · doi:10.1109/tcyb.2015.2492242 · Cited by 465
Reinforcement learning for control: Performance, stability, and deep approximators
Buşoniu, L., de Bruin, T., Tolić, D., Kober, J., & Palunko, I. (2018) · Annual Reviews in Control · doi:10.1016/j.arcontrol.2018.09.005 · Cited by 463
Modares, H., Lewis, F. L., & Naghibi-Sistani, M.-B. (2013) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2013.2276571 · Cited by 462
Self-citationZhang, H., Jiang, H., Luo, Y., & Xiao, G. (2017) · IEEE Transactions on Industrial Electronics · doi:10.1109/tie.2016.2542134 · Cited by 424
Robust Adaptive Dynamic Programming and Feedback Stabilization of Nonlinear Systems
Jiang, Y., & Jiang, Z.-P. (2014) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2013.2294968 · Cited by 414
Adaptive Dynamic Programming and Adaptive Optimal Output Regulation of Linear Systems
Gao, W., & Jiang, Z.-P. (2016) · IEEE Transactions on Automatic Control · doi:10.1109/tac.2016.2548662 · Cited by 413
Wang, D., Liu, D., Wei, Q., Zhao, D., & Jin, N. (2012) · Automatica · doi:10.1016/j.automatica.2012.05.049 · Cited by 394
Lee, J. H., Shin, J., & Realff, M. J. (2018) · Computers & Chemical Engineering · doi:10.1016/j.compchemeng.2017.10.008 · Cited by 387
Observer-Based Adaptive Fuzzy Fault-Tolerant Optimal Control for SISO Nonlinear Systems
Li, Y., Sun, K., & Tong, S. (2019) · IEEE Transactions on Cybernetics · doi:10.1109/tcyb.2017.2785801 · Cited by 378
Adaptive Critic Nonlinear Robust Control: A Survey
Wang, D., He, H., & Liu, D. (2017) · IEEE Transactions on Cybernetics · doi:10.1109/tcyb.2017.2712188 · Cited by 374
Liu, D., Wang, D., & Li, H. (2014) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2013.2280013 · Cited by 352
Value iteration and adaptive dynamic programming for data-driven adaptive optimal control design
Bian, T., & Jiang, Z.-P. (2016) · Automatica · doi:10.1016/j.automatica.2016.05.003 · Cited by 348
Cooperative Adaptive Cruise Control: A Reinforcement Learning Approach
Desjardins, C., & Chaib-draa, B. (2011) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2011.2157145 · Cited by 344
Safe Reinforcement Learning Using Robust MPC
Zanon, M., & Gros, S. (2021) · IEEE Transactions on Automatic Control · doi:10.1109/tac.2020.3024161 · Cited by 338
Model-Free Optimal Tracking Control via Critic-Only Q-Learning
Luo, B., Liu, D., Huang, T., & Wang, D. (2016) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2016.2585520 · Cited by 332
Actor–Critic-Based Optimal Tracking for Partially Unknown Nonlinear Discrete-Time Systems
Kiumarsi, B., & Lewis, F. L. (2015) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2014.2358227 · Cited by 326
Self-citationWang, J., Wu, J., Shen, H., Cao, J., & Rutkowski, L. (2023) · IEEE Transactions on Cybernetics · doi:10.1109/tcyb.2022.3220537 · Cited by 321
Liu, D., Wang, D., Wang, F.-Y., Li, H., & Yang, X. (2014) · IEEE Transactions on Cybernetics · doi:10.1109/tcyb.2014.2357896 · Cited by 310
Finite-Approximation-Error-Based Optimal Control Approach for Discrete-Time Nonlinear Systems
Liu, D., & Wei, Q. (2013) · IEEE Transactions on Cybernetics · doi:10.1109/tsmcb.2012.2216523 · Cited by 304
Wang, F.-Y., Jin, N., Liu, D., & Wei, Q. (2011) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.2010.2076370 · Cited by 294
Approximate policy iteration: a survey and some new methods
Bertsekas, D. P. (2011) · Journal of Control Theory and Applications · doi:10.1007/s11768-011-1005-3 · Cited by 273
Liu, D., Li, H., & Wang, D. (2014) · IEEE Transactions on Systems, Man, and Cybernetics: Systems · doi:10.1109/tsmc.2013.2295351 · Cited by 263
Safe reinforcement learning: A control barrier function optimization approach
Marvi, Z., & Kiumarsi, B. (2021) · International Journal of Robust and Nonlinear Control · doi:10.1002/rnc.5132 · Cited by 254
Gao, W., Jiang, Y., Jiang, Z.-P., & Chai, T. (2016) · Automatica · doi:10.1016/j.automatica.2016.05.008 · Cited by 234
“All rights reserved. feedback control systems (Lewis & Vrabie, 2009; Lewis, Vrabie, & Vamvoudakis, 2012; Liu & Wei, 2014; Wang, Zhang, & Liu, 2009).”
Reinforcement learning for iot security: A comprehensive survey
Uprety, A., & Rawat, D. B. (2021) · IEEE Internet of Things Journal · doi:10.1109/jiot.2020.3040957 · Cited by 214
Memory-based control with recurrent neural networks
Heess, N., Hunt, J. J., Lillicrap, T. P., & Silver, D. (2015) · arXiv · doi:10.48550/arxiv.1512.04455 · Cited by 211
“Similar ideas had previously been exploited in NFQCA [4] and in the ADP [13] community.”
ResultLearning-Based Adaptive Optimal Tracking Control of Strict-Feedback Nonlinear Systems
Gao, W., & Jiang, Z.-P. (2018) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2017.2761718 · Cited by 176
Embedded Adaptive Fuzzy Controller Based on Reinforcement Learning for DC Motor with Flexible Shaft
Aziz Khater, A., El-Bardini, M., & El-Rabaie, N. M. (2015) · Arabian Journal for Science and Engineering · doi:10.1007/s13369-015-1752-4 · Cited by 20
Metadata
- Authors
- Lewis, F. L., & Vrabie, D.
- Venue
- IEEE Circuits and Systems Magazine
- Published
- 2009
- Volume
- 9
- Issue
- 3
- Pages
- 32-50
- ISSN
- 0163-6812 (ISSN-L)
- 1531-636X
- 1558-0830
- OpenAlex
- W2165726932
- Cited by
- 1,558
- References
- 86
- FWCI
- 26.72
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.