The DOI matches its Crossref record; 206 of the 206 references it cites were found in a registry or an index.
Adaptive Dynamic Programming for Control: A Survey and Recent Advances
Abstract
This article reviews the recent development of adaptive dynamic programming (ADP) with applications in control. First, its applications in optimal regulation are introduced, and some skilled and efficient algorithms are presented. Next, the use of ADP to solve game problems, mainly nonzero-sum game problems, is elaborated. It is followed by applications in large-scale systems. Note that although the functions presented in this article are based on continuous-time systems, various applications of ADP in discrete-time systems are also analyzed. Moreover, in each section, not only some existing techniques are discussed, but also possible directions for future work are pointed out. Finally, some overall prospects for the future are given, followed by conclusions of this article. Through a comprehensive and complete investigation of its applications in many existing fields, this article fully demonstrates that the ADP intelligent control method is promising in today's artificial intelligence era. Furthermore, it also plays a significant role in promoting economic and social development.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Human-level control through deep reinforcement learning
Mnih, V., Kavukcuoglu, K., Silver, D., Rusu, A. A., Veness, J., Bellemare, M. G., Graves, A., Riedmiller, M., Fidjeland, A. K., Ostrovski, G., et al. (2015) · Nature · doi:10.1038/nature14236 · Cited by 30,357
Reinforcement Learning: An Introduction
Sutton, R. S., & Barto, A. G. (1998) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.1998.712192 · Cited by 27,225
Reinforcement Learning: An Introduction
(2005) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.2004.842673 · Cited by 24,496
Mastering the game of Go with deep neural networks and tree search
Silver, D., Huang, A., Maddison, C. J., Guez, A., Sifre, L., van den Driessche, G., Schrittwieser, J., Antonoglou, I., Panneershelvam, V., Lanctot, M., et al. (2016) · Nature · doi:10.1038/nature16961 · Cited by 15,612
Watkins, C. J. C. H., & Dayan, P. (1992) · Machine Learning · doi:10.1007/bf00992698 · Cited by 9,083
Reinforcement Learning: A Survey
Kaelbling, L. P., Littman, M. L., & Moore, A. W. (1996) · Journal of Artificial Intelligence Research · doi:10.1613/jair.301 · Cited by 8,972
Watkins, C. (1989) · OpenGrey (Institut de l'Information Scientifique et Technique) · doi:10.17863/cam.132164 · Cited by 5,297
Learning to predict by the methods of temporal differences
Sutton, R. S. (1988) · Machine Learning · doi:10.1007/bf00115009 · Cited by 5,148
Åström, K. J., & Wittenmark, B. (1989) · Cited by 5,020
Event-Triggered Real-Time Scheduling of Stabilizing Control Tasks
Tabuada, P. (2007) · IEEE Transactions on Automatic Control · doi:10.1109/tac.2007.904277 · Cited by 4,702
Dynamic Noncooperative Game Theory
(1982) · Mathematics in Science and Engineering · doi:10.1016/s0076-5392(08)x6169-7 · Cited by 4,057
Neuronlike adaptive elements that can solve difficult learning control problems
Barto, A. G., Sutton, R. S., & Anderson, C. W. (1983) · IEEE Transactions on Systems, Man, and Cybernetics · doi:10.1109/tsmc.1983.6313077 · Cited by 3,233
Approximate Dynamic Programming: Solving the Curses of Dimensionality
Powell, W. B. (2011) · Wiley Series in Probability and Statistics · doi:10.1002/9781118029176 · Cited by 2,297
Hornik, K., Stinchcombe, M., & White, H. (1990) · Neural Networks · doi:10.1016/0893-6080(90)90005-6 · Cited by 2,111
L/sub 2/-gain analysis of nonlinear systems and nonlinear state-feedback H/sub infinity / control
van der Schaft, A. J. (1992) · IEEE Transactions on Automatic Control · doi:10.1109/9.256331 · Cited by 1,587
Approximate Dynamic Programming: Solving the Curses of Dimensionality
Powell, W. B. (2007) · Wiley Series in Probability and Statistics · doi:10.1002/9780470182963 · Cited by 1,566
Reinforcement learning and adaptive dynamic programming for feedback control
Lewis, F. L., & Vrabie, D. (2009) · IEEE Circuits and Systems Magazine · doi:10.1109/mcas.2009.933854 · Cited by 1,558
End-to-End Training of Deep Visuomotor Policies
Levine, S., Finn, C., Darrell, T. J., & Abbeel, P. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1504.00702 · Cited by 1,344
Prokhorov, D. V., & Wunsch, D. C. (1997) · IEEE Transactions on Neural Networks · doi:10.1109/72.623201 · Cited by 1,186
(2012) · IEEE Control Systems · doi:10.1109/mcs.2012.2214134 · Cited by 1,105
Learning to act using real-time dynamic programming
Barto, A. G., Bradtke, S. J., & Singh, S. P. (1995) · Artificial Intelligence · doi:10.1016/0004-3702(94)00011-o · Cited by 1,104
Discrete-Time Nonlinear HJB Solution Using Approximate Dynamic Programming: Convergence Proof
Al-Tamimi, A., Lewis, F. L., & Abu-Khalaf, M. (2008) · IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics) · doi:10.1109/tsmcb.2008.926614 · Cited by 1,075
Jiang, Y., & Jiang, Z.-P. (2012) · Automatica · doi:10.1016/j.automatica.2012.06.096 · Cited by 1,072
H∞-Optimal Control and Related Minimax Design Problems
Başar, T., & Bernhard, P. (2008) · Birkhäuser Boston eBooks · doi:10.1007/978-0-8176-4757-5 · Cited by 1,043
A Survey of Actor-Critic Reinforcement Learning: Standard and Natural Policy Gradients
Grondman, I., Busoniu, L., Lopes, G. A. D., & Babuska, R. (2012) · IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews) · doi:10.1109/tsmcc.2012.2218595 · Cited by 1,042
Approximate dynamic programming for real-time control and neural modeling
Werbos, P. J. (1992) · Medical Entomology and Zoology · Cited by 971
(2011) · Encyclopedia of Machine Learning · doi:10.1007/978-0-387-30164-8_588 · Cited by 950
Adaptive optimal control for continuous-time linear systems based on policy iteration
Vrabie, D., Pastravanu, O., Abu-Khalaf, M., & Lewis, F. L. (2009) · Automatica · doi:10.1016/j.automatica.2008.08.017 · Cited by 949
Adaptive Dynamic Programming: An Introduction
Wang, F.-Y., Zhang, H., & Liu, D. (2009) · IEEE Computational Intelligence Magazine · doi:10.1109/mci.2009.932261 · Cited by 898
Self-citationOptimal and Autonomous Control Using Reinforcement Learning: A Survey
Kiumarsi, B., Vamvoudakis, K. G., Modares, H., & Lewis, F. L. (2018) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2017.2773458 · Cited by 891
Disturbance attenuation and H/sub infinity /-control via measurement feedback in nonlinear systems
Isidori, A., & Astolfi, A. (1992) · IEEE Transactions on Automatic Control · doi:10.1109/9.159566 · Cited by 885
Handbook of Learning and Approximate Dynamic Programming
Si, J., Barto, A. G., Powell, W. B., & Wunsch, D. (2004) · Institute of Electrical and Electronics Engineers eBooks · doi:10.1109/9780470544785 · Cited by 856
Online learning control by association and reinforcement
Si, J., & Wang, Y.-T. (2001) · IEEE Transactions on Neural Networks · doi:10.1109/72.914523 · Cited by 773
Policy Iteration Adaptive Dynamic Programming Algorithm for Discrete-Time Nonlinear Systems
Liu, D., & Wei, Q. (2014) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2013.2281663 · Cited by 753
Self-citationModares, H., & Lewis, F. L. (2014) · Automatica · doi:10.1016/j.automatica.2014.05.011 · Cited by 701
Murray, J. J., Cox, C. J., Lendaris, G. G., & Saeks, R. (2002) · IEEE Transactions on Systems, Man and Cybernetics, Part C (Applications and Reviews) · doi:10.1109/tsmcc.2002.801727 · Cited by 677
Bhasin, S., Kamalapurkar, R., Johnson, M., Vamvoudakis, K. G., Lewis, F. L., & Dixon, W. E. (2013) · Automatica · doi:10.1016/j.automatica.2012.09.019 · Cited by 642
A Menu of Designs for Reinforcement Learning Over Time
Werbos, P. J. (1991) · Neural Networks for Control · doi:10.7551/mitpress/4939.003.0007 · Cited by 605
Kiumarsi, B., Lewis, F. L., Modares, H., Karimpour, A., & Naghibi-Sistani, M.-B. (2014) · Automatica · doi:10.1016/j.automatica.2014.02.015 · Cited by 574
Modares, H., Lewis, F. L., & Jiang, Z. (2015) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2015.2441749 · Cited by 570
Modares, H., Lewis, F. L., & Naghibi-Sistani, M.-B. (2014) · Automatica · doi:10.1016/j.automatica.2013.09.043 · Cited by 570
Vamvoudakis, K. G., & Lewis, F. L. (2011) · Automatica · doi:10.1016/j.automatica.2011.03.005 · Cited by 525
Adaptive Dynamic Programming with Applications in Optimal Control
Liu, D., Wei, Q., Wang, D., Yang, X., & Li, H. (2017) · Advances in industrial control · doi:10.1007/978-3-319-50815-3 · Cited by 491
Self-citationValue Iteration Adaptive Dynamic Programming for Optimal Control of Discrete-Time Nonlinear Systems
Wei, Q., Liu, D., & Lin, H. (2016) · IEEE Transactions on Cybernetics · doi:10.1109/tcyb.2015.2492242 · Cited by 465
Self-citationModares, H., Lewis, F. L., & Naghibi-Sistani, M.-B. (2013) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2013.2276571 · Cited by 462
Modares, H., & Lewis, F. L. (2014) · IEEE Transactions on Automatic Control · doi:10.1109/tac.2014.2317301 · Cited by 449
Concurrent learning for convergence in adaptive control without persistency of excitation
Chowdhary, G., & Johnson, E. (2010) · 49th IEEE Conference on Decision and Control (CDC) · doi:10.1109/cdc.2010.5717148 · Cited by 419
Adaptive Dynamic Programming and Adaptive Optimal Output Regulation of Linear Systems
Gao, W., & Jiang, Z.-P. (2016) · IEEE Transactions on Automatic Control · doi:10.1109/tac.2016.2548662 · Cited by 413
Neuro-dynamic programming: an overview
Bertsekas, D. P., & Tsitsiklis, J. N. (2002) · Proceedings of 1995 34th IEEE Conference on Decision and Control · doi:10.1109/cdc.1995.478953 · Cited by 411
Wang, D., Liu, D., Wei, Q., Zhao, D., & Jin, N. (2012) · Automatica · doi:10.1016/j.automatica.2012.05.049 · Cited by 394
Self-citationAdaptive Critic Nonlinear Robust Control: A Survey
Wang, D., He, H., & Liu, D. (2017) · IEEE Transactions on Cybernetics · doi:10.1109/tcyb.2017.2712188 · Cited by 374
Self-citationReinforcement learning improves behaviour from evaluative feedback
Littman, M. L. (2015) · Nature · doi:10.1038/nature14540 · Cited by 368
Off-Policy Reinforcement Learning for $ H_\infty $ Control Design
Luo, B., Wu, H.-N., & Huang, T. (2015) · IEEE Transactions on Cybernetics · doi:10.1109/tcyb.2014.2319577 · Cited by 362
Self-citationLiu, D., Wang, D., & Li, H. (2014) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2013.2280013 · Cited by 352
Self-citationLincoln, B., & Rantzer, A. (2006) · IEEE Transactions on Automatic Control · doi:10.1109/tac.2006.878720 · Cited by 332
Model-Free Optimal Tracking Control via Critic-Only Q-Learning
Luo, B., Liu, D., Huang, T., & Wang, D. (2016) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2016.2585520 · Cited by 332
Self-citationActor–Critic-Based Optimal Tracking for Partially Unknown Nonlinear Discrete-Time Systems
Kiumarsi, B., & Lewis, F. L. (2015) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2014.2358227 · Cited by 326
Reinforcement Learning: A Tutorial Survey and Recent Advances
Gosavi, A. (2009) · INFORMS Journal on Computing · doi:10.1287/ijoc.1080.0305 · Cited by 326
Event-triggered optimal adaptive control algorithm for continuous-time nonlinear systems
Vamvoudakis, K. G. (2014) · IEEE/CAA Journal of Automatica Sinica · doi:10.1109/jas.2014.7004686 · Cited by 313
Liu, D., Wang, D., Wang, F.-Y., Li, H., & Yang, X. (2014) · IEEE Transactions on Cybernetics · doi:10.1109/tcyb.2014.2357896 · Cited by 310
Self-citationFinite-Approximation-Error-Based Optimal Control Approach for Discrete-Time Nonlinear Systems
Liu, D., & Wei, Q. (2013) · IEEE Transactions on Cybernetics · doi:10.1109/tsmcb.2012.2216523 · Cited by 304
Self-citationWang, F.-Y., Jin, N., Liu, D., & Wei, Q. (2011) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.2010.2076370 · Cited by 294
Self-citationData-based approximate policy iteration for affine nonlinear continuous-time optimal control design
Luo, B., Wu, H.-N., Huang, T., & Liu, D. (2014) · Automatica · doi:10.1016/j.automatica.2014.10.056 · Cited by 280
Self-citationLiu, D., Li, H., & Wang, D. (2014) · IEEE Transactions on Systems, Man, and Cybernetics: Systems · doi:10.1109/tsmc.2013.2295351 · Cited by 263
Self-citationAn Event-Triggered ADP Control Approach for Continuous-Time System With Unknown Internal States
Zhong, X., & He, H. (2017) · IEEE Transactions on Cybernetics · doi:10.1109/tcyb.2016.2523878 · Cited by 263
Deep Reinforcement Learning-Based Automatic Exploration for Navigation in Unknown Environment
Li, H., Zhang, Q., & Zhao, D. (2020) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2019.2927869 · Cited by 261
Werbos, P. (1987) · IEEE Transactions on Systems, Man, and Cybernetics · doi:10.1109/tsmc.1987.289329 · Cited by 251
Adaptive Dynamic Programming for Control: Algorithms and Stability
Zhang, H., Liu, D., Luo, Y., & Wang, D. (2013) · Communications and Control Engineering · doi:10.1007/978-1-4471-4757-2 · Cited by 250
Self-citationOptimal Tracking Control of Unknown Discrete-Time Linear Systems Using Input-Output Measured Data
Kiumarsi, B., Lewis, F. L., Naghibi-Sistani, M.-B., & Karimpour, A. (2015) · IEEE Transactions on Cybernetics · doi:10.1109/tcyb.2014.2384016 · Cited by 249
Chen, C., Modares, H., Xie, K., Lewis, F. L., Wan, Y., & Xie, S. (2019) · IEEE Transactions on Automatic Control · doi:10.1109/tac.2019.2905215 · Cited by 243
Global Adaptive Dynamic Programming for Continuous-Time Nonlinear Systems
Jiang, Y., & Jiang, Z.-P. (2015) · IEEE Transactions on Automatic Control · doi:10.1109/tac.2015.2414811 · Cited by 241
Off-Policy Actor-Critic Structure for Optimal Control of Unknown Systems With Disturbances
Song, R., Lewis, F. L., Wei, Q., & Zhang, H. (2016) · IEEE Transactions on Cybernetics · doi:10.1109/tcyb.2015.2421338 · Cited by 239
Self-citationOnline solution of nonlinear two‐player zero‐sum games using synchronous policy iteration
Vamvoudakis, K. G., & Lewis, F. L. (2011) · International Journal of Robust and Nonlinear Control · doi:10.1002/rnc.1760 · Cited by 238
1 Global Adaptive Dynamic Programming for Continuous-Time Nonlinear SystemsPreprint
Jiang, Y., Member, S., & Jiang, Z. (2016) · Cited by 235
Reinforcement learning and optimal adaptive control: An overview and implementation examples
Khan, S. G., Herrmann, G., Lewis, F. L., Pipe, T., & Melhuish, C. (2012) · Annual Reviews in Control · doi:10.1016/j.arcontrol.2012.03.004 · Cited by 234
Song, R., Lewis, F. L., & Wei, Q. (2017) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2016.2582849 · Cited by 233
Self-citationXu, H., Jagannathan, S., & Lewis, F. L. (2012) · Automatica · doi:10.1016/j.automatica.2012.03.007 · Cited by 230
Wei, Q., Liu, D., & Shi, G. (2014) · IEEE Transactions on Industrial Electronics · doi:10.1109/tie.2014.2361485 · Cited by 230
Self-citationExperience Replay for Optimal Control of Nonzero-Sum Game Systems With Unknown Dynamics
Zhao, D., Zhang, Q., Wang, D., & Zhu, Y. (2016) · IEEE Transactions on Cybernetics · doi:10.1109/tcyb.2015.2488680 · Cited by 229
Zhu, Y., Zhao, D., He, H., & Ji, J. (2017) · IEEE Transactions on Industrial Electronics · doi:10.1109/tie.2016.2597763 · Cited by 225
Wang, D., Liu, D., & Wei, Q. (2012) · Neurocomputing · doi:10.1016/j.neucom.2011.03.058 · Cited by 224
Self-citationWei, Q., & Liu, D. (2014) · IEEE Transactions on Automation Science and Engineering · doi:10.1109/tase.2013.2284545 · Cited by 224
Self-citationPolicy Gradient Adaptive Dynamic Programming for Data-Based Optimal Control
Luo, B., Liu, D., Wu, H.-N., Wang, D., & Lewis, F. L. (2017) · IEEE Transactions on Cybernetics · doi:10.1109/tcyb.2016.2623859 · Cited by 222
Self-citationHe, H., Ni, Z., & Fu, J. (2012) · Neurocomputing · doi:10.1016/j.neucom.2011.05.031 · Cited by 221
Liu, D., Wang, D., Zhao, D., Wei, Q., & Jin, N. (2012) · IEEE Transactions on Automation Science and Engineering · doi:10.1109/tase.2012.2198057 · Cited by 220
Self-citationData-Based Adaptive Critic Designs for Nonlinear Robust Optimal Control With Uncertain Dynamics
Wang, D., Liu, D., Zhang, Q., & Zhao, D. (2016) · IEEE Transactions on Systems, Man, and Cybernetics: Systems · doi:10.1109/tsmc.2015.2492941 · Cited by 218
Self-citationPadhi, R., Unnikrishnan, N., Wang, X., & Balakrishnan, S. N. (2006) · Neural Networks · doi:10.1016/j.neunet.2006.08.010 · Cited by 216
Yang, X., Liu, D., & Wang, D. (2014) · International Journal of Control · doi:10.1080/00207179.2013.848292 · Cited by 203
Self-citationDiscrete-Time Deterministic $Q$ -Learning: A Novel Convergence Analysis
Wei, Q., Lewis, F. L., Sun, Q., Yan, P., & Song, R. (2017) · IEEE Transactions on Cybernetics · doi:10.1109/tcyb.2016.2542923 · Cited by 200
Self-citationAdaptive critic designs: A case study for neurocontrol
Prokhorov, D. V., Santiago, R. A., & Wunsch, D. C. (1995) · Neural Networks · doi:10.1016/0893-6080(95)00042-9 · Cited by 199
Modares, H., Nageshrao, S. P., Lopes, G. A. D., Babuška, R., & Lewis, F. L. (2016) · Automatica · doi:10.1016/j.automatica.2016.05.017 · Cited by 198
Deep Reinforcement Learning With Visual Attention for Vehicle Classification
Zhao, D., Chen, Y., & Lv, L. (2017) · IEEE Transactions on Cognitive and Developmental Systems · doi:10.1109/tcds.2016.2614675 · Cited by 198
Consistency of HDP applied to a simple reinforcement learning problem
Werbos, P. J. (1990) · Neural Networks · doi:10.1016/0893-6080(90)90088-3 · Cited by 194
Novel iterative neural dynamic programming for data-based approximate optimal control design
Mu, C., Wang, D., & He, H. (2017) · Automatica · doi:10.1016/j.automatica.2017.03.022 · Cited by 193
(2017) · IEEE Control Systems · doi:10.1109/mcs.2016.2621461 · Cited by 192
Zhang, H., Song, R., Wei, Q., & Zhang, T. (2011) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.2011.2172628 · Cited by 191
Self-citationYang, Q., & Jagannathan, S. (2012) · IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics) · doi:10.1109/tsmcb.2011.2166384 · Cited by 190
Adaptive Learning in Tracking Control Based on the Dual Critic Network Design
Ni, Z., He, H., & Wen, J. (2013) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2013.2247627 · Cited by 189
Finite-Approximation-Error-Based Discrete-Time Iterative Adaptive Dynamic Programming
Wei, Q., Wang, F.-Y., Liu, D., & Yang, X. (2014) · IEEE Transactions on Cybernetics · doi:10.1109/tcyb.2014.2354377 · Cited by 188
Self-citationZhao, B., Liu, D., & Luo, C. (2020) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2019.2954983 · Cited by 188
Self-citationDong, L., Zhong, X., Sun, C., & He, H. (2017) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2016.2541020 · Cited by 186
Mu, C., Ni, Z., Sun, C., & He, H. (2017) · IEEE Transactions on Cybernetics · doi:10.1109/tcyb.2016.2548941 · Cited by 181
Modares, H., Lewis, F. L., & Jiang, Z.-P. (2016) · IEEE Transactions on Cybernetics · doi:10.1109/tcyb.2015.2477810 · Cited by 180
Online adaptive algorithm for optimal control with integral reinforcement learning
Vamvoudakis, K. G., Vrabie, D., & Lewis, F. L. (2014) · International Journal of Robust and Nonlinear Control · doi:10.1002/rnc.3018 · Cited by 179
Event-Triggered Optimal Control With Performance Guarantees Using Adaptive Dynamic Programming
Luo, B., Yang, Y., Liu, D., & Wu, H.-N. (2020) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2019.2899594 · Cited by 177
Self-citationAdaptive Dynamic Programming for Discrete-Time Zero-Sum Games
Wei, Q., Liu, D., Lin, Q., & Song, R. (2017) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2016.2638863 · Cited by 176
Self-citationZhao, B., Wang, D., Shi, G., Liu, D., & Li, Y. (2018) · IEEE Transactions on Systems, Man, and Cybernetics: Systems · doi:10.1109/tsmc.2017.2690665 · Cited by 176
Self-citationLearning-Based Adaptive Optimal Tracking Control of Strict-Feedback Nonlinear Systems
Gao, W., & Jiang, Z.-P. (2018) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2017.2761718 · Cited by 176
Discrete-Time Local Value Iteration Adaptive Dynamic Programming: Convergence Analysis
Wei, Q., Lewis, F. L., Liu, D., Song, R., & Lin, H. (2018) · IEEE Transactions on Systems, Man, and Cybernetics: Systems · doi:10.1109/tsmc.2016.2623766 · Cited by 173
Self-citationQin, J., Li, M., Shi, Y., Ma, Q., & Zheng, W. X. (2018) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2018.2832025 · Cited by 172
Xue, S., Luo, B., & Liu, D. (2020) · IEEE Transactions on Systems, Man, and Cybernetics: Systems · doi:10.1109/tsmc.2018.2852810 · Cited by 171
Self-citationOn Mixed Data and Event Driven Design for Adaptive-Critic-Based Nonlinear $H_{\infty}$ Control
Wang, D., Mu, C., Liu, D., & Ma, H. (2018) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2016.2642128 · Cited by 168
Self-citationGeneralized Policy Iteration Adaptive Dynamic Programming for Discrete-Time Nonlinear Systems
Liu, D., Wei, Q., & Yan, P. (2015) · IEEE Transactions on Systems Man and Cybernetics Systems · doi:10.1109/tsmc.2015.2417510 · Cited by 158
Self-citationEvent-Driven Adaptive Robust Control of Nonlinear Systems With Uncertainties Through NDP Strategy
Wang, D., Mu, C., He, H., & Liu, D. (2016) · IEEE Transactions on Systems Man and Cybernetics Systems · doi:10.1109/tsmc.2016.2592682 · Cited by 156
Self-citationLin, H., Zhao, B., Liu, D., & Alippi, C. (2020) · IEEE/CAA Journal of Automatica Sinica · doi:10.1109/jas.2020.1003225 · Cited by 155
Self-citationZhong, X., He, H., Zhang, H., & Wang, Z. (2014) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2014.2305841 · Cited by 153
The Optimal Consumption of Depletable Natural Resources
Weinstein, M. C., & Zeckhauser, R. (1975) · The Quarterly Journal of Economics · doi:10.2307/1885258 · Cited by 144
Sahoo, A., Xu, H., & Jagannathan, S. (2015) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2015.2453320 · Cited by 144
Adaptive $Q$ -Learning for Data-Based Optimal Output Regulation With Experience Replay
Luo, B., Yang, Y., & Liu, D. (2018) · IEEE Transactions on Cybernetics · doi:10.1109/tcyb.2018.2821369 · Cited by 143
Self-citationJiang, Y., Kiumarsi, B., Fan, J., Chai, T., Li, J., & Lewis, F. L. (2019) · IEEE Transactions on Cybernetics · doi:10.1109/tcyb.2018.2890046 · Cited by 143
Infinite Horizon Self-Learning Optimal Control of Nonaffine Discrete-Time Nonlinear Systems
Wei, Q., Liu, D., & Yang, X. (2015) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2015.2401334 · Cited by 142
Self-citationA Novel Iterative $\theta $ -Adaptive Dynamic Programming for Discrete-Time Nonlinear Systems
Wei, Q., & Liu, D. (2013) · IEEE Transactions on Automation Science and Engineering · doi:10.1109/tase.2013.2280974 · Cited by 138
Self-citationLiu, D., Huang, Y., Wang, D., & Wei, Q. (2013) · International Journal of Control · doi:10.1080/00207179.2013.790562 · Cited by 138
Self-citationA boundedness result for the direct heuristic dynamic programming
Liu, F., Sun, J. Q., Si, J., Guo, W., & Mei, S. (2012) · Neural Networks · doi:10.1016/j.neunet.2012.02.005 · Cited by 134
Yang, X., Liu, D., & Huang, Y. (2013) · IET Control Theory and Applications · doi:10.1049/iet-cta.2013.0472 · Cited by 134
Self-citationCall admission control and routing in integrated services networks using neuro-dynamic programming
Marbach, P., Mihatsch, O., & Tsitsiklis, J. N. (2000) · IEEE Journal on Selected Areas in Communications · doi:10.1109/49.824797 · Cited by 134
Luo, B., Liu, D., & Wu, H. (2017) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2017.2751018 · Cited by 134
Self-citationOptimal control for discrete-time affine non-linear systems using general value iteration
Li, H., & Liu, D. (2012) · IET Control Theory and Applications · doi:10.1049/iet-cta.2011.0783 · Cited by 133
Self-citationEvent-triggered optimal tracking control of nonlinear systems
Vamvoudakis, K. G., Mojoodi, A., & Ferraz, H. (2016) · International Journal of Robust and Nonlinear Control · doi:10.1002/rnc.3587 · Cited by 133
Zhao, B., Liu, D., & Li, Y. (2016) · Information Sciences · doi:10.1016/j.ins.2016.12.016 · Cited by 132
Self-citationReinforcement learning solution for HJB equation arising in constrained optimal control problem
Luo, B., Wu, H., Huang, T., & Liu, D. (2015) · Neural Networks · doi:10.1016/j.neunet.2015.08.007 · Cited by 131
Self-citationLiu, D., Wang, D., & Yang, X. (2012) · Information Sciences · doi:10.1016/j.ins.2012.07.006 · Cited by 129
Self-citationYang, X., He, H., & Liu, D. (2017) · IEEE Transactions on Systems Man and Cybernetics Systems · doi:10.1109/tsmc.2017.2774602 · Cited by 129
Self-citationRelaxed dynamic programming in switching systems
Rantzer, A. (2006) · IEE Proceedings - Control Theory and Applications · doi:10.1049/ip-cta:20050094 · Cited by 128
Online Learning Control Using Adaptive Critic Designs With Sparse Kernel Machines
Xu, X., Hou, Z., Lian, C., & He, H. (2013) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2012.2236354 · Cited by 126
Xue, S., Luo, B., Liu, D., & Yang, Y. (2020) · IEEE Transactions on Systems Man and Cybernetics Systems · doi:10.1109/tsmc.2020.2997559 · Cited by 122
Self-citationXue, S., Luo, B., & Liu, D. (2020) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2020.3009015 · Cited by 119
Self-citationData-Driven Distributed Optimal Consensus Control for Unknown Multiagent Systems With Input-Delay
Zhang, H., Yue, D., Dou, C., Zhao, W., & Xie, X. (2018) · IEEE Transactions on Cybernetics · doi:10.1109/tcyb.2018.2819695 · Cited by 117
Yang, X., & He, H. (2019) · IEEE Transactions on Systems Man and Cybernetics Systems · doi:10.1109/tsmc.2019.2898370 · Cited by 117
Adaptive Suboptimal Output-Feedback Control for Linear Systems Using Integral Reinforcement Learning
Zhu, L. M., Modares, H., Gan, O. P., Lewis, F. L., & Yue, B. (2014) · IEEE Transactions on Control Systems Technology · doi:10.1109/tcst.2014.2322778 · Cited by 108
Construction of Suboptimal Control Sequences
Leake, R. J., & Liu, R.-W. (1967) · SIAM Journal on Control · doi:10.1137/0305004 · Cited by 107
Discrete-Time Optimal Control via Local Policy Iteration Adaptive Dynamic Programming
Wei, Q., Liu, D., Lin, Q., & Song, R. (2016) · IEEE Transactions on Cybernetics · doi:10.1109/tcyb.2016.2586082 · Cited by 107
Self-citationGoal Representation Heuristic Dynamic Programming on Maze Navigation
Ni, Z., He, H., Wen, J., & Xu, X. (2013) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2013.2271454 · Cited by 106
Missile defense and interceptor allocation by neuro-dynamic programming
Bertsekas, D. P., Homer, M. L., Logan, D. A., Patek, S. D., & SANDELL, N. R. (2000) · IEEE Transactions on Systems Man and Cybernetics - Part A Systems and Humans · doi:10.1109/3468.823480 · Cited by 106
Balancing Value Iteration and Policy Iteration for Discrete-Time Control
Luo, B., Yang, Y., Wu, H., & Huang, T. (2019) · IEEE Transactions on Systems Man and Cybernetics Systems · doi:10.1109/tsmc.2019.2898389 · Cited by 106
Self-citationOptimal Output Regulation for Model-Free Quanser Helicopter With Multistep Q-Learning
Luo, B., Wu, H., & Huang, T. (2017) · IEEE Transactions on Industrial Electronics · doi:10.1109/tie.2017.2772162 · Cited by 104
Self-citationNon-zero sum Nash Q-learning for unknown deterministic continuous-time linear systems
Vamvoudakis, K. G. (2015) · Automatica · doi:10.1016/j.automatica.2015.08.017 · Cited by 103
Huang, Y., & Liu, D. (2013) · Neurocomputing · doi:10.1016/j.neucom.2012.07.047 · Cited by 98
Self-citationContinuous-Time Q-Learning for Infinite-Horizon Discounted Cost Linear Quadratic Regulator Problems
Muthukumar, P., Modares, H., Lewis, F. L., & Aurangzeb, M. (2015) · IEEE Transactions on Cybernetics · doi:10.1109/tcyb.2014.2322116 · Cited by 97
Werbos, P. J. (2008) · IEEE Transactions on Systems Man and Cybernetics Part B (Cybernetics) · doi:10.1109/tsmcb.2008.924139 · Cited by 94
Event-Driven Nonlinear Discounted Optimal Regulation Involving a Power System Application
Wang, D., He, H., Zhong, X., & Liu, D. (2017) · IEEE Transactions on Industrial Electronics · doi:10.1109/tie.2017.2698377 · Cited by 94
Self-citationPeng, Z., Zhao, Y., Hu, J., & Ghosh, B. K. (2018) · Information Sciences · doi:10.1016/j.ins.2018.12.079 · Cited by 91
Yang, X., Liu, D., & Wei, Q. (2014) · IET Control Theory and Applications · doi:10.1049/iet-cta.2014.0186 · Cited by 89
Self-citationWei, Q., & Liu, D. (2012) · Neural Networks · doi:10.1016/j.neunet.2012.02.027 · Cited by 87
Self-citationKamalapurkar, R., Klotz, J. R., & Dixon, W. E. (2014) · IEEE/CAA Journal of Automatica Sinica · doi:10.1109/jas.2014.7004681 · Cited by 85
Comprehensive comparison of online ADP algorithms for continuous-time optimal control
Zhu, Y., & Zhao, D. (2017) · Artificial Intelligence Review · doi:10.1007/s10462-017-9548-4 · Cited by 83
Jiang, H., Zhang, H., Luo, Y., & Han, J. (2018) · IEEE Transactions on Systems Man and Cybernetics Systems · doi:10.1109/tsmc.2018.2810117 · Cited by 82
Wei, Q., & Liu, D. (2014) · Neurocomputing · doi:10.1016/j.neucom.2013.09.069 · Cited by 79
Self-citationOptimal Elevator Group Control via Deep Asynchronous Actor–Critic Learning
Wei, Q., Wang, L., Liu, Y., & Polycarpou, M. M. (2020) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2020.2965208 · Cited by 79
Self-citationUsing ADP to Understand and Replicate Brain Intelligence: The Next Level Design?
Werbos, P. J. (2007) · Springer eBooks · doi:10.1007/978-3-540-73267-9_6 · Cited by 78
Adaptive Optimal Output Regulation of Time-Delay Systems via Measurement Feedback
Gao, W., & Jiang, Z. (2018) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2018.2850520 · Cited by 78
Tang, Y., Yang, J. B., Yan, J., & He, H. (2015) · Neurocomputing · doi:10.1016/j.neucom.2015.04.092 · Cited by 72
Liu, D., Li, H., & Wang, D. (2015) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2015.2402203 · Cited by 72
Self-citationA Theoretical Foundation of Goal Representation Heuristic Dynamic Programming
Zhong, X., Ni, Z., & He, H. (2015) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2015.2490698 · Cited by 70
Deterministic policy gradient adaptive dynamic programming for model-free optimal control
Zhang, Y., Zhao, B., & Liu, D. (2020) · Neurocomputing · doi:10.1016/j.neucom.2019.11.032 · Cited by 68
Self-citationZhao, B., Liu, D., & Li, Y. (2016) · IET Control Theory and Applications · doi:10.1049/iet-cta.2015.1105 · Cited by 66
Self-citationMulti-step heuristic dynamic programming for optimal control of nonlinear discrete-time systems
Luo, B., Liu, D., Huang, T., Yang, X., & Ma, H. (2017) · Information Sciences · doi:10.1016/j.ins.2017.05.005 · Cited by 64
Self-citationData-Driven Distributed Output Consensus Control for Partially Observable Multiagent Systems
Jiang, H., & He, H. (2018) · IEEE Transactions on Cybernetics · doi:10.1109/tcyb.2017.2788819 · Cited by 64
Yang, X., Liu, D., Wang, D., & Wei, Q. (2014) · Neural Networks · doi:10.1016/j.neunet.2014.03.008 · Cited by 61
Self-citationQu, Q., Zhang, H., Feng, T., & Jiang, H. (2016) · Neurocomputing · doi:10.1016/j.neucom.2016.10.058 · Cited by 61
Xue, S., Luo, B., Liu, D., & Li, Y. (2019) · Neurocomputing · doi:10.1016/j.neucom.2018.09.097 · Cited by 60
Self-citationWang, D., Liu, D., Mu, C., & Ma, H. (2016) · Neurocomputing · doi:10.1016/j.neucom.2016.06.020 · Cited by 59
Self-citationActor-Critic Off-Policy Learning for Optimal Control of Multiple-Model Discrete-Time Systems
Škach, J., Kiumarsi, B., Lewis, F. L., & Straka, O. (2016) · IEEE Transactions on Cybernetics · doi:10.1109/tcyb.2016.2618926 · Cited by 59
Zhao, B., & Li, Y. (2018) · International Journal of Control Automation and Systems · doi:10.1007/s12555-016-0711-5 · Cited by 59
Self-citationLiu, D., Li, C., Li, H., Wang, D., & Ma, H. (2015) · Neurocomputing · doi:10.1016/j.neucom.2014.07.082 · Cited by 58
Self-citationNeuro-optimal control for a class of unknown nonlinear dynamic systems using SN-DHP technique
Wang, D., & Liu, D. (2013) · Neurocomputing · doi:10.1016/j.neucom.2013.04.006 · Cited by 54
Self-citationDiscrete-Time Stable Generalized Self-Learning Optimal Control With Approximation Errors
Wei, Q., Li, B., & Song, R. (2017) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2017.2661865 · Cited by 52
Self-citationYasini, S., Sistani, M. B. N., & Kirampor, A. (2014) · International Journal of Machine Learning and Cybernetics · doi:10.1007/s13042-014-0300-y · Cited by 51
Mu, C., Sun, C., Song, A., & Yu, H. (2016) · Neurocomputing · doi:10.1016/j.neucom.2016.06.059 · Cited by 50
Yang, X., & He, H. (2018) · IEEE Transactions on Systems Man and Cybernetics Systems · doi:10.1109/tsmc.2018.2837899 · Cited by 47
Fuzzy-Based Goal Representation Adaptive Dynamic Programming
Tang, Y., He, H., Ni, Z., Zhong, X., Zhao, D., & Xu, X. (2015) · IEEE Transactions on Fuzzy Systems · doi:10.1109/tfuzz.2015.2505327 · Cited by 46
Zhang, K., Zhang, H., Xiao, G., & Su, H. (2017) · Neurocomputing · doi:10.1016/j.neucom.2017.04.008 · Cited by 46
Guo, W., Si, J., Liu, F., & Mei, S. (2017) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2017.2702566 · Cited by 45
On integral generalized policy iteration for continuous-time linear quadratic regulations
Lee, J. Y., Park, J. B., & Choi, Y. H. (2014) · Automatica · doi:10.1016/j.automatica.2013.12.009 · Cited by 44
Zhao, B., Liu, D., Yang, X., & Li, Y. (2017) · International Journal of Systems Science · doi:10.1080/00207721.2017.1296982 · Cited by 44
Self-citationADP: Goals, Opportunities and Principles
Si, J., Barto, A. G., Powell, W. B., & Wunsch, D. (2009) · Institute of Electrical and Electronics Engineers eBooks · doi:10.1109/9780470544785.ch1 · Cited by 43
Zhao, B., Jia, L., Xia, H., & Li, Y. (2018) · Nonlinear Dynamics · doi:10.1007/s11071-018-4309-8 · Cited by 42
Self-citationDynamic Multiobjective Control for Continuous-Time Systems Using Reinforcement Learning
Lopez, V. G., & Lewis, F. L. (2018) · IEEE Transactions on Automatic Control · doi:10.1109/tac.2018.2869462 · Cited by 42
Wei, Q., Liu, D., & Xu, Y. (2014) · Soft Computing · doi:10.1007/s00500-014-1533-0 · Cited by 40
Self-citationEvent-Triggered Suboptimal Tracking Controller Design for a Class of Nonlinear Discrete-Time Systems
Batmani, Y., Davoodi, M., & Meskin, N. (2017) · IEEE Transactions on Industrial Electronics · doi:10.1109/tie.2017.2698413 · Cited by 40
Zhang, H., Xiao, G., Liu, Y., & Liu, L. (2018) · IEEE Transactions on Systems Man and Cybernetics Systems · doi:10.1109/tsmc.2018.2853091 · Cited by 39
Yang, X., & He, H. (2018) · Neural Networks · doi:10.1016/j.neunet.2018.05.005 · Cited by 37
Zhao, B., Li, Y., & Liu, D. (2017) · Neurocomputing · doi:10.1016/j.neucom.2016.12.063 · Cited by 36
Self-citationGao, W., Jiang, Y., Jiang, Z.-P., & Chai, T. (2014) · Proceeding of the 11th World Congress on Intelligent Control and Automation · doi:10.1109/wcica.2014.7053043 · Cited by 35
Data-Based Optimal Control for Weakly Coupled Nonlinear Systems Using Policy Iteration
Li, C., Liu, D., & Wang, D. (2016) · IEEE Transactions on Systems Man and Cybernetics Systems · doi:10.1109/tsmc.2016.2606479 · Cited by 31
Self-citationWang, W., Chen, X., Fu, H., & Wu, M. (2019) · International Journal of Systems Science · doi:10.1080/00207721.2019.1599463 · Cited by 29
Policy iteration based Q-learning for linear nonzero-sum quadratic differential games
Li, X., Peng, Z., Liang, L., & Zha, W. (2019) · Science China Information Sciences · doi:10.1007/s11432-018-9602-1 · Cited by 25
Mu, C., & Wang, K. (2019) · ISA Transactions · doi:10.1016/j.isatra.2019.01.025 · Cited by 20
Li, C., Liu, D., & Li, H. (2015) · IET Control Theory and Applications · doi:10.1049/iet-cta.2014.1325 · Cited by 19
Self-citation
Cited by
Wang, D., Gao, N., Liu, D., Li, J., & Lewis, F. L. (2024) · IEEE/CAA Journal of Automatica Sinica · doi:10.1109/jas.2023.123843 · Cited by 364
Self-citationRay, S., Kasturi, K., Patnaik, S., & Nayak, M. R. (2023) · Journal of Energy Storage · doi:10.1016/j.est.2023.108672 · Cited by 118
Bai, W., Chen, D., Zhao, B., & D'Ariano, A. (2025) · IEEE Transactions on Intelligent Vehicles · doi:10.1109/tiv.2024.3458894 · Cited by 31
Self-citation
Metadata
- Authors
- Liu, D., Xue, S., Zhao, B., Luo, B., & Wei, Q.
- Venue
- IEEE Transactions on Systems, Man, and Cybernetics: Systems
- Published
- 2021-01
- Volume
- 51
- Issue
- 1
- Pages
- 142-160
- ISSN
- 2168-2216 (ISSN-L)
- 2168-2232
- OpenAlex
- W3114263192
- Cited by
- 708
- References
- 206
- FWCI
- 33.83
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.