Reinforcement Learning and Approximate Dynamic Programming for Feedback Control
The DOI matches its Crossref record; 33 of the 33 references it cites were found in a registry or an index.
Abstract
In this chapter, we extend the ADP algorithm, dual heuristic programming (DHP), to include a “bootstrapping” parameter λ, analogous to that used in the reinforcement learning algorithm TD(λ). The resulting algorithm, which we call VGL(λ) for value-gradient learning, is proven to produce a weight update that can be equivalent to backpropagation through time (BPTT) applied to a greedy policy on a critic function. This provides a surprising connection between the two alternate methods of BPTT and DHP. Under certain smoothness conditions, VGL(λ=1) with a greedy policy acquires the strong convergence conditions of BPTT, while using a general function approximator for the critic. We show that this can lead to increased stability in the learning of control problems by a neural network
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“ADP represents a kind of data-based control technique which can guarantee the stability of the feedback control system [9].”
From Data-driven control of hydraulic servo actuator: An event-triggered adaptive dynamic programming approach (2023) · cites this paper as [9]
In the citing paper
“Even if an approximated model of the HSA can be developed, the dynamic uncertainty, produced by the mismatch between the approximated model and the true HSA model, will degrade the control performance of the traditionally designed optimal controller [9,12].”
From Data-driven control of hydraulic servo actuator: An event-triggered adaptive dynamic programming approach (2023) · cites this paper as [9]
In the citing paper
“Adaptive dynamic programming (ADP) ensures an e ective way to achieve high performance of the optimal controller which relies on adaptive control, optimal control and reinforcement learning [9, 23-26].”
From Data-driven control of hydraulic servo actuator: An event-triggered adaptive dynamic programming approach (2023) · cites this paper as [9]
Citations
Cites
Neural Networks for Pattern Recognition
Bishop, C. M. (1995) · doi:10.1093/oso/9780198538493.001.0001 · Cited by 12,287
Watkins, C. (1989) · OpenGrey (Institut de l'Information Scientifique et Technique) · doi:10.17863/cam.132164 · Cited by 5,297
Learning to predict by the methods of temporal differences
Sutton, R. S. (1988) · Machine Learning · doi:10.1007/bf00115009 · Cited by 5,148
Backpropagation through time: what it does and how to do it
Werbos, P. J. (1990) · Proceedings of the IEEE · doi:10.1109/5.58337 · Cited by 4,879
Online actor–critic algorithm to solve the continuous-time infinite horizon optimal control problem
Vamvoudakis, K. G., & Lewis, F. L. (2010) · Automatica · doi:10.1016/j.automatica.2010.02.018 · Cited by 1,703
Approximate dynamic programming for real-time control and neural modeling
Werbos, P. J. (1992) · Cited by 1,427
An analysis of temporal-difference learning with function approximation
Tsitsiklis, J. N., & Van Roy, B. (1997) · IEEE Transactions on Automatic Control · doi:10.1109/9.580874 · Cited by 1,194
Prokhorov, D. V., & Wunsch, D. C. (1997) · IEEE Transactions on Neural Networks · doi:10.1109/72.623201 · Cited by 1,186
Self-citationDiscrete-Time Nonlinear HJB Solution Using Approximate Dynamic Programming: Convergence Proof
Al-Tamimi, A., Lewis, F. L., & Abu-Khalaf, M. (2008) · IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics) · doi:10.1109/tsmcb.2008.926614 · Cited by 1,075
Approximate dynamic programming for real-time control and neural modeling
Werbos, P. J. (1992) · Medical Entomology and Zoology · Cited by 971
Cited by
Items 401–500 of 584
Luo, C., Ning, X., & Tang, R. (2025) · IEEE Transactions on Aerospace and Electronic Systems · doi:10.1109/taes.2025.3564579 · Cited by 3
A novel approach to feedback control with deep reinforcement learning
Bidi, K. A., Coron, J., Hayat, A., & Lichtlé, N. (2025) · Systems & Control Letters · doi:10.1016/j.sysconle.2025.106102 · Cited by 3
“One can cite for instance optimal control-based feedback, dynamical programming [2,3], backstepping [4–6], and many others.”
Yang, X., Liu, D., & Wei, Q. (2014) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn.2014.6889462 · Cited by 3
“Uptonow, while RL has been widely employed to derive the optimal control for nonlinear systems [11], [12], [13], [14], [15], [16], [17], [18], most of these applications depend on an initial stabilizing control.”
MethodStochastic Adaptive Dynamic Programming for Robust Optimal Control Design
Bian, T., & Jiang, Z. (2016) · Elsevier eBooks · doi:10.1016/b978-0-12-805246-4.00007-0 · Cited by 3
SOLITONS FOR DESCRIBING 3-D PHYSICAL REALITY: THE CURRENT FRONTIER
Werbos, P. J. (2013) · WORLD SCIENTIFIC eBooks · doi:10.1142/9789814434805_0035 · Cited by 3
Towards enabling deep learning techniques for adaptive dynamic programming
Ni, Z., Malla, N., & Zhong, X. (2017) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn.2017.7966205 · Cited by 3
“the instant control action, while the action network produces an online learning based control signal to improve the control performance [19]–[26].”
BackgroundModel Predictive Control for Skydiver Fall-Rate Adjustment
Clarke, A., & Gutman, P. (2019) · Mediterranean Conference on Control and Automation (MED) · doi:10.1109/med.2019.8798553 · Cited by 3
“Reinforcement learning, neural networks, and similar methods [9] can be used to control plants with unknown or hard to model dynamics.”
MethodArmesto, L., & Sala, A. (2021) · Revista Iberoamericana de Automática e Informática Industrial RIAI · doi:10.4995/riai.2021.15698 · Cited by 3
“…2011) de sistemas no lineales es, desde sus inicios, muy activa en este tipo de problemas, dando lugar a las disciplinas que se conocen como programación dinámica aproximada (Busoniu et al., 2010; Lewis and Vrabie, 2009; Lewis and Liu, 2013) o aprendizaje por refuerzo (Sutton and Barto, 1998).”
BackgroundPolicy Iteration and Event-Triggered Robust Adaptive Dynamic Programming for Large-Scale Systems
Zhao, F., Gao, W., Liu, T., & Jiang, Z. (2021) · IFAC-PapersOnLine · doi:10.1016/j.ifacol.2021.10.383 · Cited by 3
Model-Based Approximate Optimal Feedback Control of a Hypersonic Vehicle
Greene, M. L., Deptula, P., Bialy, B., & Dixon, W. E. (2022) · AIAA SciTech Forum · doi:10.2514/6.2022-0613 · Cited by 3
Structured Online Learning for Low-Level Control of Quadrotors
Farsi, M., & Liu, J. (2022) · Proceedings of the ... American Control Conference/Proceedings of the American Control Conference · doi:10.23919/acc53348.2022.9867729 · Cited by 3
“In reinforcement learning, it is useful for determining the time horizon considered for minimizing the objective [19].”
BackgroundAl-Mahasneh, A. J., Anavatti, S. G., & Garratt, M. (2022) · IEEE Access · doi:10.1109/access.2022.3210136 · Cited by 3
“Model-free RL methods have been widely utilized for feedback control, mainly because they can be utilized as Data-Driven Controllers (DDC) for dynamic systems without explicitly relying on the mathematical models of these systems [4], [5].”
MethodOutput‐feedback stochastic model predictive control of chance‐constrained nonlinear systems
Zhang, J., & Ohtsuka, T. (2023) · IET Control Theory and Applications · doi:10.1049/cth2.12456 · Cited by 3
Markolf, L., & Stursberg, O. (2023) · Proceedings of the ... American Control Conference/Proceedings of the American Control Conference · doi:10.23919/acc55779.2023.10156344 · Cited by 3
“[6], [7], [8], [9], [10], [11] and the references cited therein.”
SNAC Approach to Aircraft Motion Control
Tiumentsev, Y. V., & Tshay, R. A. (2023) · Studies in computational intelligence · doi:10.1007/978-3-031-44865-2_45 · Cited by 3
Information Relaxation and a Duality-Driven Algorithm for Stochastic Dynamic Programs
Chen, N., Ma, X., Liu, Y., & Yu, W. (2024) · Operations Research · doi:10.1287/opre.2020.0464 · Cited by 7
An Optimized Position Control via Reinforcement-Learning-Based Hybrid Structure Strategy
Amare, N. D., Yang, S. J., & Son, Y. I. (2025) · Actuators · doi:10.3390/act14040199 · Cited by 3
Approximate policy iteration with unsupervised feature learning based on manifold regularization
Li, H., Liu, D., & Wang, D. (2015) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn.2015.7280311 · Cited by 3
“A closely related topic is adaptive or approximate dynamic programming [4]-[9] which adopts a control-theoretic point of view and terminology.”
BackgroundRL-based optimal networked control considering network delay of discrete-time linear systems
Fujita, T., & Ushio, T. (2015) · European Control Conference (ECC) · doi:10.1109/ecc.2015.7330910 · Cited by 3
Learning Algorithms for Differential Games of Continuous-Time Systems
Liu, D., Wei, Q., Wang, D., Yang, X., & Li, H. (2017) · Advances in industrial control · doi:10.1007/978-3-319-50815-3_11 · Cited by 3
Luo, X., Luo, H., & Chang, X. (2014) · International Conference on Identification, Information and Knowledge in the Internet of Things (IIKI) · doi:10.1109/iiki.2014.24 · Cited by 3
Improving LMI controllers for discrete nonlinear systems using policy iteration
Díaz, H., Sala, A., & Armesto, L. (2017) · International Conference on System Theory, Control and Computing (ICSTCC) · doi:10.1109/icstcc.2017.8107140 · Cited by 3
“The Q-learning algorithm for q = 2 is initialized with the one-step controller and a dataset D with N = 1000 samples for Hx = diag([100, 10]), Hu = 1, γ = 1.”
MethodOptimal Reagents Control for Flotation Processes: An Adaptive Dynamic Programming Approach
Li, Z., Huang, M., Gui, W., Hua, Y., & Zhu, J. (2019) · Chinese Automation Congress · doi:10.1109/cac48633.2019.8997264 · Cited by 3
Networked Control of Nonlinear Systems under Partial Observation Using Continuous Deep Q-Learning
Ikemoto, J., & Ushio, T. (2019) · IEEE Conference on Decision and Control (CDC) · doi:10.1109/cdc40024.2019.9029214 · Cited by 3
Reinforcement Learning for Adaptive Periodic Linear Quadratic Control
Pang, B., Jiang, Z., & Mareels, I. (2019) · IEEE Conference on Decision and Control (CDC) · doi:10.1109/cdc40024.2019.9030252 · Cited by 3
“Despite these significant progresses over the past decade [3], [4], [5], [6], little research has been devoted to the adaptive optimal control problem for time-varying systems.”
BackgroundReinforcement Learning: An Industrial Perspective
Surana, A. (2021) · Studies in systems, decision and control · doi:10.1007/978-3-030-60990-0_21 · Cited by 3
Neural Network Tracking Controls of SCARA Manipulator System
Lv, Y., Huang, Y., Wu, X., & Jian, L. (2021) · Chinese Control and Decision Conference (CCDC) · doi:10.1109/ccdc52312.2021.9602275 · Cited by 3
Fast Online Reinforcement Learning of Distributed Optimal Controller for Large-Scale Network Systems
Hoshiya, T., & Sadamoto, T. (2021) · IEEE Conference on Control Technology and Applications (CCTA) · doi:10.1109/ccta48906.2021.9659050 · Cited by 3
Autonomous Sensor Path Planning and Control for Active Information Gathering
Lu, W. (2014) · DukeSpace (Duke University) · Cited by 3
Integral Policy Iteration for Zero-Sum Games with Completely Unknown Nonlinear Dynamics
Li, H., Liu, D., & Wang, D. (2013) · Lecture notes in computer science · doi:10.1007/978-3-642-42054-2_29 · Cited by 2
Predictive Control with Terminal Costs Based on Online Learning Using Value Iteration
Moreno-Mora, F., & Streif, S. (2024) · European Control Conference (ECC) · doi:10.23919/ecc64448.2024.10591271 · Cited by 2
“Two important methods can be distinguished in the ADP literature, namely Value Iteration (VI) and Policy Iteration (PI) (see [10], [16] for an overview).”
MethodKernel-Based Adaptive Critic Designs for Optimal Control of Nonlinear Discrete-Time System
Tan, F., & Guan, X. (2018) · Chinese Control Conference (CCC) · doi:10.23919/chicc.2018.8482778 · Cited by 2
“This means that we need to use computers and choose the optimal numerical algorithm to get the optimal solution [6], [7].”
MethodReinforcement Learning for Distributed Control and Multi-player Games
Kiumarsi, B., Modares, H., & Lewis, F. L. (2021) · Studies in systems, decision and control · doi:10.1007/978-3-030-60990-0_2 · Cited by 2
Luo, W., Liang, M., Liu, D., & Zhao, B. (2021) · International Conference on Intelligent Control and Information Processing · doi:10.1109/icicip53388.2021.9642185 · Cited by 2
A Value Iteration Approach to Adaptive Optimal Control of Linear Time-Delay Systems
Cui, L., Pang, B., & Jiang, Z. (2023) · IFAC-PapersOnLine · doi:10.1016/j.ifacol.2023.10.524 · Cited by 2
Convergence Rates of Online Critic Value Function Approximation in Native Spaces
Niu, S., Bouland, A., Wang, H., Fotiadis, F., Kurdila, A. J., L’Afflitto, A., Paruchuri, S. T., & Vamvoudakis, K. G. (2024) · IEEE Control Systems Letters · doi:10.1109/lcsys.2024.3417178 · Cited by 2
“The works referenced in [8] and [9], for example, provide comprehensive insights into the modern theory underlying many recent online and offline methods.”
BackgroundNi, Y., Yang, H., Xu, Y., Jiang, B., & Lu, N. (2024) · IFAC-PapersOnLine · doi:10.1016/j.ifacol.2024.07.232 · Cited by 2
Xin, P., Wang, D., Liu, A., & Qiao, J. (2025) · Neurocomputing · doi:10.1016/j.neucom.2024.129311 · Cited by 2
Xia, L., Li, Q., Song, R., & Yang, G. (2025) · IEEE Access · doi:10.1109/access.2025.3526871 · Cited by 2
Convergence and Robustness of Value and Policy Iteration for the Linear Quadratic Regulator
Song, B., Wu, C., & Iannelli, A. (2025) · European Control Conference (ECC) · doi:10.23919/ecc65951.2025.11187181 · Cited by 2
“Approximate dynamic programming (ADP) [1]–[3] is a powerful algorithmic approach designed to solve sequential decision-making problems across a broad range of applications.”
Boundary control of linear one-dimensional parabolic PDE using neuro-dynamic programming
Talaei, B., Jagannathan, S., & Singler, J. R. (2015) · The proceedings of the IEEE Conference on Control Applications/The proceedings of the ... IEEE Conference on Control Applications · doi:10.1109/cca.2015.7320691 · Cited by 2
“However, due to the difficulty of designing the controller in infinite dimensional domain and with PDE dynamics, the DPS system was discretized into an approximate finite dimensional state space and subsequently the welldeveloped NDP algorithms [8] were utilized for this reduced order model.”
MethodConvergence analysis of GrDHP-based optimal control for discrete-time nonlinear system
Zhong, X., Ni, Z., & He, H. (2016) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn.2016.7727797 · Cited by 2
Back to optimality: a formal framework to express the dynamics of learning optimal behavior
Alonso, E., Fairbank, M., & Mondragón, E. (2015) · Adaptive Behavior · doi:10.1177/1059712315589355 · Cited by 2
Self-citationApproximate dynamic programming, local or global optimal solution?
Heydari, A., & Balakrishnan, S. N. (2014) · Proceedings of the ... American Control Conference/Proceedings of the American Control Conference · doi:10.1109/acc.2014.6859117 · Cited by 2
“For an extensive literature on adaptive critic based problems, the reader is referred to [16] and the”
BackgroundWang, D., Liu, D., Li, H., & Yang, X. (2014) · Chinese Control Conference (CCC) · doi:10.1109/chicc.2014.6896464 · Cited by 2
Werbos, P. J. (2015) · Studies in systems, decision and control · doi:10.1007/978-3-319-24406-8_19 · Cited by 2
Value function approximation for the control of multiscale dynamical systems
Zhu, P., Morelli, J., & Ferrari, S. (2016) · IEEE Conference on Decision and Control (CDC) · doi:10.1109/cdc.2016.7799109 · Cited by 2
“Several authors have demonstrated that ADP can be used to solve complex, nonlinear control problems in the absence of an accurate system model, and optimize performance in the face of unforeseen changes and uncertainties [10], [26]–[29].”
BackgroundStabilizing Dynamic State Feedback Controller Synthesis: A Reinforcement Learning Approach
Solís, M. A., Olivares, M., & Allende, H. (2016) · Studies in Informatics and Control · doi:10.24846/v25i2y201612 · Cited by 2
“Applications of RL, as a research field inside of Machine Learning, has got extended to areas such as Robotics [10] or Control Theory [7, 12, 16], by means of choosing a suitable representation of the problem to be solved.”
BackgroundEvent-Triggered Adaptive Dynamic Programming for Uncertain Nonlinear Systems
Zhang, Q., Zhao, D., & Wang, D. (2017) · Communications in computer and information science · doi:10.1007/978-981-10-5230-9_2 · Cited by 2
“As is known, it is intractable to give an analytic solution to the HJB equation for the nonlinear systems [3].”
BackgroundAn Improved Adaptive Optimal Regulation Framework with Robust Control Synthesis
Wang, D., & Mu, C. (2018) · Studies in systems, decision and control · doi:10.1007/978-981-13-1253-3_6 · Cited by 2
Dynamic Neural Networks: Structures and Training Methods
Tiumentsev, Y. V., & Вячеславович, Е. М. (2019) · Elsevier eBooks · doi:10.1016/b978-0-12-815254-6.00012-5 · Cited by 2
Approximate Optimal Orbit Transfer of Non-cooperative Debris
Greene, M. L., Riano-Rios, C., Bevilacqua, R., Fitz-Coy, N. G., & Dixon, W. E. (2020) · AIAA SciTech Forum · doi:10.2514/6.2020-1823 · Cited by 2
Wang, A., & Wang, H. (2021) · Studies in systems, decision and control · doi:10.1007/978-3-030-60990-0_23 · Cited by 2
Cooperative Model-Based Reinforcement Learning for Approximate Optimal Tracking
Greene, M. L., Bell, Z. I., Nivison, S., How, J. P., & Dixon, W. E. (2021) · Proceedings of the ... American Control Conference/Proceedings of the American Control Conference · doi:10.23919/acc50511.2021.9483047 · Cited by 2
Hierarchical RL for Load Balancing and QoS Management in Multi-Access Networks
Ornatelli, A., Tortorelli, A., Giuseppi, A., & Priscoli, F. D. (2021) · Mediterranean Conference on Control and Automation (MED) · doi:10.1109/med51440.2021.9480246 · Cited by 2
Zhong, X., & He, H. (2021) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn52387.2021.9534429 · Cited by 2
Yang, X., Liu, D., & Wang, D. (2013) · Lecture notes in computer science · doi:10.1007/978-3-642-42054-2_79 · Cited by 2
“It has been extensively employed to obtain optimal control for nonlinear systems [5–7].”
MethodZhang, H., Liu, D., Luo, Y., & Wang, D. (2012) · Communications and control engineering/Communications and control engineering series · doi:10.1007/978-1-4471-4757-2_1 · Cited by 2
Learning an Improved LMI Controller Based on Takagi-Sugeno Models via Value Iteration
Díaz, H., Negrete, K. P., & Yépez, J. (2022) · Lecture notes in electrical engineering · doi:10.1007/978-3-031-08942-8_7 · Cited by 2
Somers, L. A., Haddad, W. M., Kokolakis, N.-M. T., & Vamvoudakis, K. G. (2023) · International Journal of Adaptive Control and Signal Processing · doi:10.1002/acs.3729 · Cited by 2
Rates of Convergence in a Class of Native Spaces for Reinforcement Learning and Control
Bouland, A., Niu, S., Paruchuri, S. T., Kurdila, A. J., Burns, J. A., & Schuster, E. (2023) · IEEE Control Systems Letters · doi:10.1109/lcsys.2023.3343439 · Cited by 2
“Standard optimal control analysis [1]–[3] shows that the value function satisfies the Hamilton-Jacobi-Bellman (HJB) equation which is equivalent to 0 = H ( x ∗ , µ ∗ , ∇ V µ ∗ ) , where µ ∗ is given by µ ∗ = − R − 1 g T ∇ V µ ∗ , and x ∗ is the optimal trajectory generated by µ ∗ .”
BackgroundInfluentialModel-free intelligent critic design with error analysis for neural tracking control
Gao, N., Wang, D., Zhao, M., & Hu, L. (2023) · Neurocomputing · doi:10.1016/j.neucom.2023.127198 · Cited by 2
Convex Q-Learning in Continuous Time with Application to Dispatch of Distributed Energy Resources
Lu, F., Mathias, J., Meyn, S., & Kalsi, K. (2023) · IEEE Conference on Decision and Control (CDC) · doi:10.1109/cdc49753.2023.10383620 · Cited by 2
Lateral Motion Control of a Maneuverable Aircraft Using Reinforcement Learning
Tiumentsev, Y. V., & Zarubin, R. A. (2024) · Optical Memory and Neural Networks · doi:10.3103/s1060992x2401003x · Cited by 2
Convexity and monotonicity in nonlinear optimal control under uncertaintyPreprint
Kircher, K. J., & Zhang, K. M. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1904.00987 · Cited by 2
“Some examples are classical linear feedback control design, reinforcement learning [23]–[26] and approximate dynamic programming [4], [27]–[29], approximation methods for multistage stochastic programming [6], [30]–[33] and robust optimization with recourse [33]–[36], the Q-design procedure [37]–[42], and model predic…”
BackgroundReinforcement Learning for Approximate Optimal Control
Dixon, W. E. (2021) · Springer eBooks · doi:10.1007/978-3-030-44184-5_100063 · Cited by 2
Reinforcement-Learning-Based Controller Design for Nonaffine Nonlinear Systems
Yang, X., Liu, D., & Wei, Q. (2014) · Lecture notes in computer science · doi:10.1007/978-3-319-12436-0_7 · Cited by 1
Liu, J., & Farsi, M. (2022) · Wiley eBooks · doi:10.1002/9781119808602.ch3 · Cited by 1
Error-based adaptive optimal tracking control of nonlinear discrete-time systems
Li, C., Ding, J., Lewis, F. L., & Chai, T. (2023) · Science China Information Sciences · doi:10.1007/s11432-023-3818-3 · Cited by 1
Social Capital in the Performance on Born Global: Systematic Literature Review
Marjan, Y., Sukoco, B. M., Sabar, S., & Usman, I. (2024) · SAGE Open · doi:10.1177/21582440241257356 · Cited by 1
Delay-aware chemotherapy dosing via online critic learning
Rahimi, F., & Samadi, M. (2025) · Scientific Reports · doi:10.1038/s41598-025-25060-x · Cited by 1
Policy-Adjustable Q-Learning for Data-Driven Nonlinear Optimal Tracking Control
Chen, J., Gong, D., Zhao, Y., Song, S., & Zhu, M. H. (2026) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2026.3672136 · Cited by 1
“…the framework of ADP, the action-dependent heuristic dynamic programming (ADHDP), also known as Q-learning, has attracted substantial interest because it directly approximates the action–value function, thereby facilitating the determination of the optimal control policy [10], [11], [12], [13].”
Aksikas, I. (2026) · Mediterranean Conference on Control and Automation (MED) · doi:10.1109/med70602.2026.11598385 · Cited by 1
Adaptive dynamic programming for nonlinear nonaffine systems
Bian, T., Jiang, Y., & Jiang, Z. (2014) · IEEE Conference on Decision and Control (CDC) · doi:10.1109/cdc.2014.7039949 · Cited by 1
“See also [20], [21] for two review papers, and [22] for a recent book.”
BackgroundAdaptive dynamic programming for H∞ control of constrained-input nonlinear systems
Xiong, Y., Derong, L., Qinglai, W., & Ding, W. (2015) · Chinese Control Conference (CCC) · doi:10.1109/chicc.2015.7260105 · Cited by 1
Adaptive dynamic programming boundary control of uncertain coupled semi-linear parabolic PDE
Talaei, B., Jagannathan, S., & Singler, J. R. (2015) · The proceedings of the IEEE Conference on Control Applications/The proceedings of the ... IEEE Conference on Control Applications · doi:10.1109/isic.2015.7307299 · Cited by 1
Robust adaptive dynamic programming for continuous-time linear and nonlinear systems
Jiang, Y. (2014) · Cited by 2
Topology-preserving flocking of nonlinear agents using optimistic planning
Buşoniu, L., & Morărescu, I. (2015) · Control Theory and Technology · doi:10.1007/s11768-015-4107-5 · Cited by 1
“Note that reinforcement learning [19] and adaptive dynamic programming [20] also aim to solve this type of optimal control problem.”
BackgroundFuzziness, Probability, Uncertainty and the Foundations of Knowledge
Werbos, P. J. (2012) · Studies in fuzziness and soft computing · doi:10.1007/978-3-642-35644-5_52 · Cited by 1
Data-Based Optimal Tracking Control of Nonaffine Nonlinear Discrete-Time Systems
Luo, B., Liu, D., Huang, T., & Li, C. (2016) · Lecture notes in computer science · doi:10.1007/978-3-319-46681-1_68 · Cited by 1
Data-Based Neuro-Optimal Temperature Control of Water Gas Shift Reaction
Liu, D., Wei, Q., Wang, D., Yang, X., & Li, H. (2017) · Advances in industrial control · doi:10.1007/978-3-319-50815-3_14 · Cited by 1
Analog quantum computing (AQC) by revisiting the underlying physics
Werbos, P. J. (2015) · Proceedings of SPIE, the International Society for Optical Engineering/Proceedings of SPIE · doi:10.1117/12.2192741 · Cited by 1
Overview of Robust Adaptive Critic Control Design
Wang, D., & Mu, C. (2018) · Studies in systems, decision and control · doi:10.1007/978-981-13-1253-3_1 · Cited by 1
Intelligent Critic Control with Disturbance Attenuation for a Micro-Grid System
Wang, D., & Mu, C. (2018) · Studies in systems, decision and control · doi:10.1007/978-981-13-1253-3_9 · Cited by 1
Data‐Driven Approximated Optimal Control of Sulfur Flotation Process
He, M. (2019) · Complexity · doi:10.1155/2019/4754508 · Cited by 1
“The industrial applications of ADP include oil production [18], coal gasification [19], and energy scheduling [20], to name a few.”
BackgroundData-driven Optimal Preview Output Tracking of Linear Discrete-time Systems
Liu, Z., & Wu, H. (2019) · Chinese Control Conference (CCC) · doi:10.23919/chicc.2019.8865154 · Cited by 1
“Recently, reinforcement learning (RL) [6, 7], also known as approximate/adaptive dynamic programming (ADP) [8, 9] has been a hot topic and received considerable attention.”
BackgroundAn adaptive dynamic programming method for torque ripple minimization of PMSM
Yan, T., Liu, Q., Dou, B., Li, Q., & Li, B. (2019) · Journal of Industrial and Management Optimization · doi:10.3934/jimo.2019136 · Cited by 1
“Lewis[9] has proposed that ADP could obtain the optimal control strategy by numerical calculation under inaccurate conditions, solving HJB equation and algebraic curve (ARE) with online information, and proving the stability of ADP.”
BackgroundA Novel On-line VI-ADP for Nonlinear Discrete-time Systems
Li, C., Ding, J., Liu, C., & Lewis, F. L. (2019) · International Conference on Control and Automation · doi:10.1109/icca.2019.8899631 · Cited by 1
“It has many different combining algorithms between on-line, off-line, on-policy and off-policy for discrete-time and continuous-time linear or non-linear systems [4]-[7].”
BackgroundAdaptive Critic Control For Aircraft Lateral-Directional Dynamics
Ashraf, I. K., & Kampen, E.-J. V. (2020) · AIAA SciTech Forum · doi:10.2514/6.2020-1845 · Cited by 1
Value-iteration-based Adaptive Optimal Reagents Control for Antimony Flotation Process
Li, Z., Huang, M., Gui, W., & Jiang, Z. (2020) · Chinese Control Conference (CCC) · doi:10.23919/ccc50068.2020.9188349 · Cited by 1
Model-Based Switched Approximate Dynamic Programming for Functional Electrical Stimulation Cycling
Makumi, W. A., Greene, M. L., Stubbs, K. J., & Dixon, W. E. (2022) · Proceedings of the ... American Control Conference/Proceedings of the American Control Conference · doi:10.23919/acc53348.2022.9867819 · Cited by 1
“Specficially, ADP uses a reinforcement learning (RL)-based actor-critic framework to approximate the value function in real-time [9].”
MethodZhang, K., Xue, W., Zhang, R., & Wei, C. (2022) · Chinese Control Conference (CCC) · doi:10.23919/ccc55666.2022.9901845 · Cited by 1
“As one class of RL method, adaptive dynamic programming (ADP) maps the relationship between control and performance, and is applied in many optimal control designs [3–5].”
MethodAdaptive Optimal Control for Large-Scale Systems based on Robust Policy Iteration
Zhao, F., & Zhao, L. (2022) · Chinese Control and Decision Conference (CCDC) · doi:10.1109/ccdc55256.2022.10033503 · Cited by 1
Adaptive Optimal Control of UAV Formation Based on Policy Iteration
Xu, G., Zhang, S., & Liu, H. (2022) · Chinese Control and Decision Conference (CCDC) · doi:10.1109/ccdc55256.2022.10033911 · Cited by 1
Online adaptive data-driven control for unknown nonlinear systems with constrained-input
Liu, C., Chu, Z., & Li, Y. (2023) · International Conference on Cyber-Energy Systems and Intelligent Energy (ICCSIE) · doi:10.1109/iccsie55183.2023.10175236 · Cited by 1
“Although ADP and RL were proposed out of different contexts, they have much in common when dealing with OCPs [5].”
BackgroundRL-Based Robust Controller Design of Disturbed Servo System
Lv, Y., Tang, Y. L., Zhao, J., & Huang, Y. (2023) · International Annual Conference on Complex Systems and Intelligent Science (CSIS-IAC) · doi:10.1109/csis-iac60628.2023.10363938 · Cited by 1
“…the optimal control problem of nonlinear systems, we must first construct a Hamilton-Jacobi-Bellman (HJB) equation, and for disturbed nonlinear systems, we must obtain the optimal control strategy by constructing Hamilton-Jacobi-Isaacs (HJI) equation, but the equation is difficult to solve [10].”
MethodAdaptive Optimal Output-Feedback Control of Discrete-Time Systems based on Hybrid Iteration
Liu, S., Gao, W., & Jiang, Z. (2024) · IEEE Conference on Decision and Control (CDC) · doi:10.1109/cdc56724.2024.10886087 · Cited by 1
“Adaptive dynamic programming (ADP) represents a data-driven approach that does not require accurate knowledge of the system model. see, e.g., [2]–[8].”
Mixed Reinforcement Learning with Additive Stochastic UncertaintyPreprint
Mu, Y., Li, S. E., Liu, C., Sun, Q., Nie, B., Cheng, B., & Peng, B. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2003.00848 · Cited by 1
“Thus, gradient calculation can be easier and more accurate than merely using data samples [9].”
BackgroundApplication of Artificial Intelligence In Smart City
Law, J. W. (2021) · Cited by 1
Neural Control and Approximate Dynamic Programming
Lewis, F. L., & Vamvoudakis, K. G. (2014) · Springer eBooks · doi:10.1007/978-1-4471-5102-9_224-2 · Cited by 1
“The current status of work in approximate dynamic programming (ADP) for feedback control is given in Lewis and Liu (2012). ADP is a form of reinforcement learning based on an actor/critic structure.”
Background
Metadata
- Authors
- Fairbank, M., Prokhorov, D., & Alonso, E.
- Affiliations
- The University of Texas at Arlington (US)
- University of Illinois Chicago (US)
- Venue
- Wiley eBooks
- Publisher
- Wiley
- Published
- 2012-12-17
- OpenAlex
- W560518094
- Cited by
- 577
- References
- 33
- Citations by year
- 2026: 15 · 2025: 27 · 2024: 38 · 2023: 48 · 2022: 38 · 2021: 50 · 2020: 44 · 2019: 64 · 2018: 35 · 2017: 54 · 2016: 43 · 2015: 39 · 2014: 48 · 2013: 27 · 2012: 7
- FWCI
- 6.42
- Type
- book
- Language
- en
- Open access
- closed
- Licence
- doi.wiley.com/10.1002/tdm_license_1.1 (tdm)
- Locations
- Wiley eBooks (publishedVersion)
- City Research Online (City University London) (acceptedVersion)
- Topics
- Physical Sciences › Engineering › Control and Systems Engineering › Adaptive Dynamic Programming Control
- Physical Sciences › Computer Science › Artificial Intelligence › Reinforcement Learning in Robotics
- Physical Sciences › Engineering › Control and Systems Engineering › Advanced Control Systems Optimization
- Keywords
- reinforcement learning, approximate dynamic programming, feedback control, adaptive dynamic programming, multiplayer games, decision-making, robot control, industrial process control, economic decision-making
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.