Reinforcement Learning and Approximate Dynamic Programming for Feedback Control
The DOI matches its Crossref record; 33 of the 33 references it cites were found in a registry or an index.
Abstract
In this chapter, we extend the ADP algorithm, dual heuristic programming (DHP), to include a “bootstrapping” parameter λ, analogous to that used in the reinforcement learning algorithm TD(λ). The resulting algorithm, which we call VGL(λ) for value-gradient learning, is proven to produce a weight update that can be equivalent to backpropagation through time (BPTT) applied to a greedy policy on a critic function. This provides a surprising connection between the two alternate methods of BPTT and DHP. Under certain smoothness conditions, VGL(λ=1) with a greedy policy acquires the strong convergence conditions of BPTT, while using a general function approximator for the critic. We show that this can lead to increased stability in the learning of control problems by a neural network
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“ADP represents a kind of data-based control technique which can guarantee the stability of the feedback control system [9].”
From Data-driven control of hydraulic servo actuator: An event-triggered adaptive dynamic programming approach (2023) · cites this paper as [9]
In the citing paper
“Even if an approximated model of the HSA can be developed, the dynamic uncertainty, produced by the mismatch between the approximated model and the true HSA model, will degrade the control performance of the traditionally designed optimal controller [9,12].”
From Data-driven control of hydraulic servo actuator: An event-triggered adaptive dynamic programming approach (2023) · cites this paper as [9]
In the citing paper
“Adaptive dynamic programming (ADP) ensures an e ective way to achieve high performance of the optimal controller which relies on adaptive control, optimal control and reinforcement learning [9, 23-26].”
From Data-driven control of hydraulic servo actuator: An event-triggered adaptive dynamic programming approach (2023) · cites this paper as [9]
Citations
Cites
Neural Networks for Pattern Recognition
Bishop, C. M. (1995) · doi:10.1093/oso/9780198538493.001.0001 · Cited by 12,287
Watkins, C. (1989) · OpenGrey (Institut de l'Information Scientifique et Technique) · doi:10.17863/cam.132164 · Cited by 5,297
Learning to predict by the methods of temporal differences
Sutton, R. S. (1988) · Machine Learning · doi:10.1007/bf00115009 · Cited by 5,148
Backpropagation through time: what it does and how to do it
Werbos, P. J. (1990) · Proceedings of the IEEE · doi:10.1109/5.58337 · Cited by 4,879
Online actor–critic algorithm to solve the continuous-time infinite horizon optimal control problem
Vamvoudakis, K. G., & Lewis, F. L. (2010) · Automatica · doi:10.1016/j.automatica.2010.02.018 · Cited by 1,703
Approximate dynamic programming for real-time control and neural modeling
Werbos, P. J. (1992) · Cited by 1,427
An analysis of temporal-difference learning with function approximation
Tsitsiklis, J. N., & Van Roy, B. (1997) · IEEE Transactions on Automatic Control · doi:10.1109/9.580874 · Cited by 1,194
Prokhorov, D. V., & Wunsch, D. C. (1997) · IEEE Transactions on Neural Networks · doi:10.1109/72.623201 · Cited by 1,186
Self-citationDiscrete-Time Nonlinear HJB Solution Using Approximate Dynamic Programming: Convergence Proof
Al-Tamimi, A., Lewis, F. L., & Abu-Khalaf, M. (2008) · IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics) · doi:10.1109/tsmcb.2008.926614 · Cited by 1,075
Approximate dynamic programming for real-time control and neural modeling
Werbos, P. J. (1992) · Medical Entomology and Zoology · Cited by 971
Cited by
Items 101–200 of 584
System Stability of Learning-Based Linear Optimal Control With General Discounted Value Iteration
Wang, D., Ren, J., Ha, M., & Qiao, J. (2022) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2021.3137524 · Cited by 79
Advanced value iteration for discrete-time intelligent critic control: A survey
Zhao, M., Wang, D., Qiao, J., Ha, M., & Ren, J. (2023) · Artificial Intelligence Review · doi:10.1007/s10462-023-10497-1 · Cited by 79
“As an indispensable branch of artificial intelligence, reinforcement learning (RL) has achieved rather marvelous successes in the fields of computer science (Silver et al. 2016), autonomous driving (Kiran et al. 2022), intelligent control (Lewis et al. 2012; Lewis and Liu 2013), and so on.”
BackgroundZhu, Y., Zhao, D., Yang, X., & Zhang, Q. (2017) · IEEE Transactions on Cybernetics · doi:10.1109/tcyb.2016.2643687 · Cited by 76
“Adaptive dynamic programming (ADP) exhibits promising performance in solving many control problems, including H∞ control (see [10]–[12] and references quoted therein).”
BackgroundMa, H., Xu, L., & Yang, G. (2019) · IEEE Transactions on Cybernetics · doi:10.1109/tcyb.2018.2889679 · Cited by 75
Reinforcement Learning Tracking Control for Robotic Manipulator With Kernel-Based Dynamic Model
Hu, Y., Wang, W., Liu, H., & Liu, L. (2019) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2019.2945019 · Cited by 81
Wang, D., Li, C., Liu, D., & Mu, C. (2016) · Information Sciences · doi:10.1016/j.ins.2016.05.034 · Cited by 72
Zhao, F., Gao, W., Jiang, Z., & Liu, T. (2020) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2020.3027301 · Cited by 74
Liu, D., Li, H., & Wang, D. (2015) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2015.2402203 · Cited by 72
“Adaptive dynamic programming (ADP) [3], also known as approximate dynamic programming [4]–[6] or neurodynamic programming [7], has received significantly increasing attention as a learning method for optimizing the policy when interacting with the environment.”
BackgroundYang, X., Liu, D., Ma, H., & Xu, Y. (2015) · Information Sciences · doi:10.1016/j.ins.2015.09.001 · Cited by 71
“These assumptions are common techniques, which have been used in [13,14,28,42,44].”
MethodAdaptive Optimal Control of Linear Periodic Systems: An Off-Policy Value Iteration Approach
Pang, B., & Jiang, Z. (2020) · IEEE Transactions on Automatic Control · doi:10.1109/tac.2020.2987313 · Cited by 71
A Theoretical Foundation of Goal Representation Heuristic Dynamic Programming
Zhong, X., Ni, Z., & He, H. (2015) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2015.2490698 · Cited by 70
Liu, M., Wan, Y., Lewis, F. L., & Lopez, V. G. (2020) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2020.2969215 · Cited by 70
Data-Driven Coordinated Attack Policy Design Based on Adaptive $\mathcal {L}_2$ -Gain Optimal Theory
An, L., & Yang, G. (2017) · IEEE Transactions on Automatic Control · doi:10.1109/tac.2017.2753466 · Cited by 70
“Based on the robust ADP technique [29], this is an offline design scheme.”
MethodStability Analysis of Optimal Adaptive Control Using Value Iteration With Approximation Errors
Heydari, A. (2018) · IEEE Transactions on Automatic Control · doi:10.1109/tac.2018.2790260 · Cited by 69
Bayesian statistics: relevant for the brain?
Kording, K. P. (2014) · Current Opinion in Neurobiology · doi:10.1016/j.conb.2014.01.003 · Cited by 58
Offline and Online Adaptive Critic Control Designs With Stability Guarantee Through Value Iteration
Ha, M., Wang, D., & Liu, D. (2021) · IEEE Transactions on Cybernetics · doi:10.1109/tcyb.2021.3107801 · Cited by 65
“RECENTLY, reinforcement learning (RL) [1], [2] has enjoyed rather remarkable successes for a wide range of fields in computer games [3]–[5], power systems [6]–[9], intelligent control [10]–[14], and so forth.”
BackgroundEfficient rare event simulation by optimal nonequilibrium forcing
Hartmann, C., & Schütte, C. (2012) · Journal of Statistical Mechanics Theory and Experiment · doi:10.1088/1742-5468/2012/11/p11004 · Cited by 98
Deep Neural Network-Based Approximate Optimal Tracking for Unknown Nonlinear Systems
Greene, M. L., Bell, Z. I., Nivison, S., & Dixon, W. E. (2023) · IEEE Transactions on Automatic Control · doi:10.1109/tac.2023.3246761 · Cited by 64
Multi-step heuristic dynamic programming for optimal control of nonlinear discrete-time systems
Luo, B., Liu, D., Huang, T., Yang, X., & Ma, H. (2017) · Information Sciences · doi:10.1016/j.ins.2017.05.005 · Cited by 64
Exarchos, I., & Theodorou, E. A. (2017) · Automatica · doi:10.1016/j.automatica.2017.09.004 · Cited by 73
“…Differential Dynamic Programming [6, 7], which is based on linearization of the dynamics and a quadratic approximation of the cost functions around nominal trajectories, as well as sampling-based methods, which are distinguished for their ability to accommodate scalable iterative schemes.”
Online optimal solutions for multi-player nonzero-sum game with completely unknown dynamics
Lv, Y., Ren, X., & Na, J. (2017) · Neurocomputing · doi:10.1016/j.neucom.2017.12.045 · Cited by 61
Dual Heuristic Dynamic Programming Based Energy Management Control for Hybrid Electric Vehicles
Wang, Y., & Jiao, X. (2022) · Energies · doi:10.3390/en15093235 · Cited by 60
Generalized value iteration for discounted optimal control with stability analysis
Ha, M., Wang, D., & Liu, D. (2020) · Systems & Control Letters · doi:10.1016/j.sysconle.2020.104847 · Cited by 60
Actor-Critic Off-Policy Learning for Optimal Control of Multiple-Model Discrete-Time Systems
Škach, J., Kiumarsi, B., Lewis, F. L., & Straka, O. (2016) · IEEE Transactions on Cybernetics · doi:10.1109/tcyb.2016.2618926 · Cited by 64
Heydari, A. (2020) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2020.3021037 · Cited by 59
Event-triggered reinforcement learning approach for unknown nonlinear continuous-time system
Zhong, X., Ni, Z., He, H., Xu, X., & Zhao, D. (2014) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn.2014.6889787 · Cited by 61
“Adaptive dynamic programming (ADP) have been studied and adopted for the solution seeking for the HamiltonJacobi-Bellman (HJB) equation in recent years [11], [12], [13].”
MethodReinforcement Learning Control of Robotic Knee With Human-in-the-Loop by Flexible Policy Iteration
Gao, X. H., Si, J., Wen, Y., Li, M., & Huang, H. (2021) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2021.3071727 · Cited by 64
“RL-based adaptive optimal control approaches, or approximate/adaptive dynamic programming (ADP) [22], [23], are a promising alternative as they have demonstrated their capability of learning from data measurements in an online or offline manner in several realistic application problems, including large-scale control p…”
BackgroundA Novel Value Iteration Scheme With Adjustable Convergence Rate
Ha, M., Wang, D., & Liu, D. (2022) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2022.3143527 · Cited by 55
A survey of modeling and control in ball screw feed-drive system
Huang, T., Kang, Y., Du, S., Zhang, Q., Luo, Z., Tang, Q., & Yang, K. (2022) · The International Journal of Advanced Manufacturing Technology · doi:10.1007/s00170-022-09506-4 · Cited by 56
Gao, W., Zhou, X., Pan, M., Zhou, W., Lü, F., & Huang, J. (2021) · Aerospace Science and Technology · doi:10.1016/j.ast.2021.107248 · Cited by 51
Luo, X., Lv, Y., Zhou, M., Wang, W., & Zhao, W. (2016) · Personal and Ubiquitous Computing · doi:10.1007/s00779-016-0916-x · Cited by 50
“Meanwhile, the applications of ADP are focused on some nonlinear and complex systems, such as aircraft systems, power systems, and many others related to IoT [16].”
BackgroundInfluentialPrioritizing Useful Experience Replay for Heuristic Dynamic Programming-Based Learning Systems
Ni, Z., Malla, N., & Zhong, X. (2019) · IEEE Transactions on Cybernetics · doi:10.1109/tcyb.2018.2853582 · Cited by 50
Learning-Based Adaptive Optimal Control of Linear Time-Delay Systems: A Policy Iteration Approach
Cui, L., Pang, B., & Jiang, Z. (2023) · IEEE Transactions on Automatic Control · doi:10.1109/tac.2023.3273786 · Cited by 49
“Therefore, for these safetycritical engineering systems, conventional RL is not directly applicable to learning stable optimal controllers from data, which has motivated the development of adaptive dynamic programming [27], [28].”
BackgroundContinuous-Time Robust Dynamic Programming
Bian, T., & Jiang, Z. (2019) · SIAM Journal on Control and Optimization · doi:10.1137/18m1214147 · Cited by 49
“Indeed, the majority of existing adaptive optimal control and DP methods [BT96, SBPW04, Pow07, LL13, JJ17] falls into this category, and RL is considered as a machine learning reinterpretation of direct adaptive control [SBW92].”
BackgroundDeptula, P., Bell, Z. I., Doucette, E. A., Curtis, J. W., & Dixon, W. E. (2020) · Automatica · doi:10.1016/j.automatica.2020.108922 · Cited by 49
Shi, X., Zhou, D., Chen, X., & Zhou, Z. (2021) · Aerospace Science and Technology · doi:10.1016/j.ast.2021.106898 · Cited by 49
Kiumarsi, B., Lewis, F. L., & Levine, D. S. (2015) · Neurocomputing · doi:10.1016/j.neucom.2014.12.067 · Cited by 50
Gr-GDHP: A New Architecture for Globalized Dual Heuristic Dynamic Programming
Zhong, X., Ni, Z., & He, H. (2016) · IEEE Transactions on Cybernetics · doi:10.1109/tcyb.2016.2598282 · Cited by 47
“The recent survey papers [7], [12] also provided the overview of the ADP techniques and its possible applications on real industry complex systems.”
BackgroundZhao, F., Gao, W., Liu, T., & Jiang, Z. (2021) · Automatica · doi:10.1016/j.automatica.2021.110103 · Cited by 46
Optimal Critic Learning for Robot Control in Time-Varying Environments
Wang, C., Li, Y., Ge, S. S., & Lee, T. H. (2015) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2014.2378812 · Cited by 45
Feature selection and feature learning for high-dimensional batch reinforcement learning: A survey
Liu, D., Li, H., & Wang, D. (2015) · Machine Intelligence Research · doi:10.1007/s11633-015-0893-y · Cited by 53
Wang, D., Mu, C., Zhang, Q., & Liu, D. (2016) · Neurocomputing · doi:10.1016/j.neucom.2016.07.002 · Cited by 45
Yi, J., Shi, C., Zhong, X., Zhou, W., & He, H. (2018) · IEEE Transactions on Industrial Informatics · doi:10.1109/tii.2018.2850001 · Cited by 45
“For decades, adaptive dynamic programming (ADP) [8]–[10], as a branch of optimal control theory, has been ap-”
BackgroundApproximate neural optimal control with reinforcement learning for a torsional pendulum device
Wang, D., & Qiao, J. (2019) · Neural Networks · doi:10.1016/j.neunet.2019.04.026 · Cited by 45
Hybrid online learning control in networked multiagent systems: A survey
Poveda, J. I., Benosman, M., & Teel, A. R. (2018) · International Journal of Adaptive Control and Signal Processing · doi:10.1002/acs.2866 · Cited by 44
“Using this feedback law and Assumption 1, semiglobal practical asymptotic stability of the containment error zi defined in (11) can be achieved.”
BackgroundDevelopment of Blast Furnace Burden Distribution Process Modeling and Control
Yang, Y., Yin, Y., Wunsch, D. C., Zhang, S., Chen, X., Li, X., Cheng, S., Wu, M., & Liu, K. (2017) · ISIJ International · doi:10.2355/isijinternational.isijint-2017-002 · Cited by 43
Lv, Y., Ren, X., & Na, J. (2019) · ISA Transactions · doi:10.1016/j.isatra.2019.08.025 · Cited by 45
Zhang, H., Zhao, C., & Ding, J. (2022) · Control Engineering Practice · doi:10.1016/j.conengprac.2022.105302 · Cited by 46
Fu, Z., Xie, W., Rakheja, S., & Na, J. (2017) · IEEE/CAA Journal of Automatica Sinica · doi:10.1109/jas.2017.7510322 · Cited by 42
Zhao, J. (2020) · Neurocomputing · doi:10.1016/j.neucom.2020.06.083 · Cited by 42
Multiagent Adversarial Collaborative Learning via Mean-Field Theory
Luo, G., Zhang, H., He, H., Li, J., & Wang, F. (2020) · IEEE Transactions on Cybernetics · doi:10.1109/tcyb.2020.3025491 · Cited by 48
Event-Triggered Robust Adaptive Dynamic Programming With Output Feedback for Large-Scale Systems
Zhao, F., Gao, W., Liu, T., & Jiang, Z. (2022) · IEEE Transactions on Control of Network Systems · doi:10.1109/tcns.2022.3186623 · Cited by 42
Particle Swarm Optimization for Adaptive-Critic Feedback Control with Power System Applications
Wang, D., Ren, J., Huang, H., & Qiao, J. (2025) · Chinese Journal of Electronics · doi:10.23919/cje.2024.00.287 · Cited by 41
A neural network based online learning and control approach for Markov jump systems
Zhong, X., He, H., Zhang, H., & Wang, Z. (2014) · Neurocomputing · doi:10.1016/j.neucom.2014.01.060 · Cited by 41
“Neural networks are applied to implement this ADP approach and online learning method is used to tune the weights of the critic and the action networks.”
MethodSong, J. L., He, S., Liu, F., Niu, Y., & Ding, Z. (2016) · IET Control Theory and Applications · doi:10.1049/iet-cta.2015.0973 · Cited by 41
“Moreover, RL shares some essential features with adaptive dynamic programming (ADP), which is originally inspired by learning mechanisms observed in biological systems [19, 20, 21, 22].”
BackgroundMu, C., & Wang, D. (2017) · Neurocomputing · doi:10.1016/j.neucom.2017.03.047 · Cited by 41
Optimal Adaptive Control of Uncertain Nonlinear Continuous-Time Systems With Input and State Delays
Moghadam, R., & Jagannathan, S. (2021) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2021.3112566 · Cited by 41
Data-Driven Design and Optimization of Feedback Control Systems for Industrial Applications
Zhang, Y., Yang, Y. Z., Ding, S. X., & Li, L. (2014) · IEEE Transactions on Industrial Electronics · doi:10.1109/tie.2014.2301757 · Cited by 39
Al-Dabooni, S., & Wunsch, D. C. (2019) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2019.2919338 · Cited by 39
“We use the vector critic method because it is a smooth and stable vector output [43], [44], [47].”
MethodOnline learning based on adaptive learning rate for a class of recurrent fuzzy neural network
Khater, A. A., El-Nagar, A. M., El-Bardini, M. A., & El-Rabaie, N. M. (2019) · Neural Computing and Applications · doi:10.1007/s00521-019-04372-w · Cited by 39
“This method employs Bellman’s principle of optimality, but it has a well-recognized problem, namely curse of dimensionality [16].”
MethodA Novel Structure of Actor-Critic Learning Based on an Interval Type-2 TSK Fuzzy Neural Network
Khater, A. A., El-Nagar, A. M., El-Bardini, M. A., & El-Rabaie, N. M. (2019) · IEEE Transactions on Fuzzy Systems · doi:10.1109/tfuzz.2019.2949554 · Cited by 40
An Equivalence Between Adaptive Dynamic Programming With a Critic and Backpropagation Through Time
Fairbank, M., Alonso, E., & Prokhorov, D. V. (2013) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2013.2271778 · Cited by 39
“By comparing (11) with (5), we note that”
BackgroundSelf-citationPang, B., Bian, T., & Jiang, Z. (2019) · Control Theory and Technology · doi:10.1007/s11768-019-8168-8 · Cited by 38
Jiang, H., Zhang, H., Liu, Y., & Han, J. (2017) · Neurocomputing · doi:10.1016/j.neucom.2017.01.047 · Cited by 37
Wu, Q., Zhao, B., Liu, D., & Polycarpou, M. M. (2022) · Neural Networks · doi:10.1016/j.neunet.2022.10.025 · Cited by 37
Robust controller design of continuous-time nonlinear system using neural network
Zhong, X., He, H., & Prokhorov, D. V. (2013) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn.2013.6707098 · Cited by 36
“The authors are also grateful to Toyota Research Institute NA for the support of this work.”
BackgroundSelf-citationActive hybrid journal bearings with lubrication control: Towards machine learning
Li, S., Babin, A., Shutin, D., Kazakov, Y., Liu, Y., Chen, Z., & Алексеевич, С. Л. (2022) · Tribology International · doi:10.1016/j.triboint.2022.107805 · Cited by 36
Model-Free Q-Learning for the Tracking Problem of Linear Discrete-Time Systems
Li, C., Ding, J., Lewis, F. L., & Chai, T. (2022) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2022.3195357 · Cited by 36
“Hence, the common way to deal with the tracking problem is to transform it into a regulator one [26]–[30].”
BackgroundGao, W., Jiang, Y., Jiang, Z.-P., & Chai, T. (2014) · Proceeding of the 11th World Congress on Intelligent Control and Automation · doi:10.1109/wcica.2014.7053043 · Cited by 35
“Approximate/adaptive dynamic programming (ADP), inspired by biological systems, is a non-model based approach and has become a hot research topic recently [1], [2], [3].”
BackgroundYang, G. (2025) · IEEE Transactions on Automation Science and Engineering · doi:10.1109/tase.2025.3639494 · Cited by 36
“Specifically, based on the final control results, the CMNNs will generate performance evaluation signals to excite the AMNNs to acquire improved uncertainty learning e ff ect [28], [29], [30].”
Transient Optimization of Parallel Connected Inverters in Islanded AC Microgrids
Dissanayake, A. M., & Ekneligoda, N. C. (2018) · IEEE Transactions on Smart Grid · doi:10.1109/tsg.2018.2871413 · Cited by 35
“For any set of admissible control policies [34] ui ∈ i; i ∈ N , the infinitesimal version of the cost function (23) which is the”
BackgroundInfluentialLoad Shedding Scheme with Deep Reinforcement Learning to Improve Short-term Voltage Stability
Zhang, J., Lü, C., Fang, C., Ling, X., & Zhang, Y. (2018) · IEEE Innovative Smart Grid Technologies-Asia · doi:10.1109/isgt-asia.2018.8467877 · Cited by 35
“Reinforcement learning (RL) [9], another research hotspot in the field of ML, has also been widely used in power system optimization and control [10][11][12].”
BackgroundModel-Free Robust Optimal Feedback Mechanisms of Biological Motor Control
Bian, T., Wolpert, D. M., & Jiang, Z. (2020) · Neural Computation · doi:10.1162/neco_a_01260 · Cited by 39
Reinforcement Learning-Based Structural Control of Floating Wind Turbines
Zhang, J., Zhao, X., & Wei, X. (2020) · IEEE Transactions on Systems Man and Cybernetics Systems · doi:10.1109/tsmc.2020.3032622 · Cited by 37
“Originally proposed by Webos [22], [23], ADP has caught extensive attention recently on the optimal control of both Continuous-Time (CT) and Discrete-Time (DT) systems [24], [25], [26], [27], [28], [29], [30], [31].”
BackgroundOptimal Switching of DC–DC Power Converters Using Approximate Dynamic Programming
Heydari, A. (2016) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2016.2635586 · Cited by 34
Moghadam, R., Pappa, N., & Jagannathan, S. (2021) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2021.3061414 · Cited by 32
“Remark 2: Note that Assumption 1 is a standard assumption in the literature [2], [12], [20] by using the target NN weights.”
Reinforcement learning based PID controller design for LFC in a microgrid
Esmaeili, M., Shayeghi, H., Nejad, H. M., & Younesi, A. (2017) · COMPEL The International Journal for Computation and Mathematics in Electrical and Electronic Engineering · doi:10.1108/compel-09-2016-0408 · Cited by 32
Reinforcement learning for linear continuous-time systems: an incremental learning approach
Bian, T., & Jiang, Z. (2019) · IEEE/CAA Journal of Automatica Sinica · doi:10.1109/jas.2019.1911390 · Cited by 31
Sparse online kernelized actor-critic Learning in reproducing kernel Hilbert space
Yang, Y., Zhu, H., Zhang, Q., Zhao, B., Li, Z., & Wunsch, D. C. (2021) · Artificial Intelligence Review · doi:10.1007/s10462-021-10045-9 · Cited by 31
Manifold Regularized Reinforcement Learning
Li, H., Liu, D., & Wang, D. (2017) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2017.2650943 · Cited by 32
Tian, Y., Shi, Y., Shao, X., Hu, Q., Yang, H., & Zhu, Z. H. (2024) · IEEE Transactions on Aerospace and Electronic Systems · doi:10.1109/taes.2024.3419763 · Cited by 30
Adaptive near‐optimal controllers for non‐linear decentralised feedback stabilisation problems
Wang, D., He, H., Zhao, B., & Liu, D. (2017) · IET Control Theory and Applications · doi:10.1049/iet-cta.2016.1383 · Cited by 30
Zhao, J., Gan, M., & Zhang, C. (2019) · Neurocomputing · doi:10.1016/j.neucom.2019.06.090 · Cited by 30
Data‐driven design of two‐degree‐of‐freedom controllers using reinforcement learning techniques
Zhang, Y., Ding, S. X., Yang, Y. Z., & Li, L. (2014) · IET Control Theory and Applications · doi:10.1049/iet-cta.2014.0156 · Cited by 30
“In the context of RL, it can be viewed as the exploration component of the control input, which guarantees the persistent excitation (PE) condition required by the RLS algorithm [20, 22].”
BackgroundWu, H., & Liu, Z. (2019) · IEEE Transactions on Systems Man and Cybernetics Systems · doi:10.1109/tsmc.2019.2931332 · Cited by 29
“More recently, reinforcement learning (RL) or approximate dynamic programming (ADP) has became very popular to obtain the numerical solution of Riccati/HJB equations and great success has been achieved in both theories [12]–[18] and applications [19]–[21].”
BackgroundAdaptive Observation-Based Efficient Reinforcement Learning for Uncertain Systems
Ran, M., & Xie, L. (2021) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2021.3070852 · Cited by 28
Heuristic dynamic programming with internal goal representation
Ni, Z., & He, H. (2013) · Soft Computing · doi:10.1007/s00500-013-1112-9 · Cited by 27
Optimal Output Regulation for General Linear Systems via Adaptive Dynamic Programming
Wu, Y., Liang, Q., & Hu, J. (2021) · IEEE Transactions on Cybernetics · doi:10.1109/tcyb.2021.3086223 · Cited by 28
Reinforcement learning of ball screw feed drive controllers
Fernandez‐Gauna, B., Ansoategui, I., Etxeberria‐Agiriano, I., & Graña, M. (2014) · Engineering Applications of Artificial Intelligence · doi:10.1016/j.engappai.2014.01.015 · Cited by 27
Data-driven Finite-horizon Optimal Control for Linear Time-varying Discrete-time Systems
Pang, B., Bian, T., & Jiang, Z. (2018) · IEEE Conference on Decision and Control (CDC) · doi:10.1109/cdc.2018.8619347 · Cited by 27
SENSITIVITY ANALYSIS OF NONLINEAR BEHAVIOR WITH DISTORTED PROBABILITY
Cao, X., & Wan, X. (2013) · Mathematical Finance · doi:10.1111/mafi.12076 · Cited by 27
Shi, Z., & Wang, Z. (2021) · Neurocomputing · doi:10.1016/j.neucom.2021.01.070 · Cited by 27
Li, D., & Dong, J. (2023) · IEEE Transactions on Systems Man and Cybernetics Systems · doi:10.1109/tsmc.2023.3320653 · Cited by 26
Comparative Analysis of Current Control Techniques to Support Virtual Inertia Applications
Tamrakar, U., Shrestha, D., Malla, N., Ni, Z., Hansen, T. M., Tamrakar, I., & Tonkoski, R. (2018) · Applied Sciences · doi:10.3390/app8122695 · Cited by 26
Wang, J., & Yang, G. (2019) · IEEE/ASME Transactions on Mechatronics · doi:10.1109/tmech.2019.2906666 · Cited by 26
“[17] presented a data-driven FTC strategy, which minimized a quadratic performance index in the context of approximate dynamic programming (ADP) [22], [23].”
MethodQing-tao, Z., Si, J., & Sun, J. Q. (2021) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2021.3053037 · Cited by 28
Data-Based Optimal Microgrid Management for Energy Trading With Integral Q -Learning Scheme
Lv, Y., Wu, Z., & Zhao, X. (2023) · IEEE Internet of Things Journal · doi:10.1109/jiot.2023.3267428 · Cited by 26
“As a major class of machine learning techniques, reinforcement learning (RL) has attracted much attention [32], [33].”
BackgroundAn adaptive neuro-control approach for multi-machine power systems
Ni, Z., Tang, Y., Sui, X., He, H., & Wen, J. (2015) · International Journal of Electrical Power & Energy Systems · doi:10.1016/j.ijepes.2015.08.012 · Cited by 25
Yin, H., Mehta, P. G., Meyn, S., & Shanbhag, U. V. (2013) · IEEE Transactions on Automatic Control · doi:10.1109/tac.2013.2287733 · Cited by 25
Chai, Y., Luo, J., & Ma, W. (2021) · ISA Transactions · doi:10.1016/j.isatra.2021.02.037 · Cited by 25
Metadata
- Authors
- Fairbank, M., Prokhorov, D., & Alonso, E.
- Affiliations
- The University of Texas at Arlington (US)
- University of Illinois Chicago (US)
- Venue
- Wiley eBooks
- Publisher
- Wiley
- Published
- 2012-12-17
- OpenAlex
- W560518094
- Cited by
- 577
- References
- 33
- Citations by year
- 2026: 15 · 2025: 27 · 2024: 38 · 2023: 48 · 2022: 38 · 2021: 50 · 2020: 44 · 2019: 64 · 2018: 35 · 2017: 54 · 2016: 43 · 2015: 39 · 2014: 48 · 2013: 27 · 2012: 7
- FWCI
- 6.42
- Type
- book
- Language
- en
- Open access
- closed
- Licence
- doi.wiley.com/10.1002/tdm_license_1.1 (tdm)
- Locations
- Wiley eBooks (publishedVersion)
- City Research Online (City University London) (acceptedVersion)
- Topics
- Physical Sciences › Engineering › Control and Systems Engineering › Adaptive Dynamic Programming Control
- Physical Sciences › Computer Science › Artificial Intelligence › Reinforcement Learning in Robotics
- Physical Sciences › Engineering › Control and Systems Engineering › Advanced Control Systems Optimization
- Keywords
- reinforcement learning, approximate dynamic programming, feedback control, adaptive dynamic programming, multiplayer games, decision-making, robot control, industrial process control, economic decision-making
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.