Cited by 677Adaptive Dynamic Programming ControlAdvanced Control Systems OptimizationReinforcement Learning in Robotics

The DOI matches its Crossref record; 32 of the 32 references it cites were found in a registry or an index.

Adaptive dynamic programming

Murray, J.J., Cox, C.J., Lendaris, G.G., Saeks, R.·2002·IEEE Transactions on Systems, Man and Cybernetics, Part C (Applications and Reviews)

DOI 10.1109/tsmcc.2002.801727

Open this paper in CoChat — free account

Abstract

Unlike the many soft computing applications where it suffices to achieve a "good approximation most of the time," a control system must be stable all of the time. As such, if one desires to learn a control law in real-time, a fusion of soft computing techniques to learn the appropriate control law with hard computing techniques to maintain the stability constraint and guarantee convergence is required. The objective of the paper is to describe an adaptive dynamic programming algorithm (ADPA) which fuses soft computing techniques to learn the optimal cost (or return) functional for a stabilizable nonlinear system with unknown dynamics and hard computing techniques to verify the stability and convergence of the algorithm. Specifically, the algorithm is initialized with a (stabilizing) cost functional and the system is run with the corresponding control law (defined by the Hamilton-Jacobi-Bellman equation), with the resultant state trajectories used to update the cost functional in a soft computing mode. Hard computing techniques are then used to show that this process is globally convergent with stepwise stability to the optimal cost functional/control law pair for an (unknown) input affine system with an input quadratic performance measure (modulo the appropriate technical conditions). Three specific implementations of the ADPA are developed for 1) the linear case, 2) for the nonlinear case using a locally quadratic approximation to the cost functional, and 3) the nonlinear case using a radial basis function approximation of the cost functional; illustrated by applications to flight control.

Integrity signals

Registry

Crossref · 10.1109/tsmcc.2002.801727

Cites

32 of 32 references verified0 citation errors · 0 not checkable

Statistics

Recomputing p-values needs the paper’s full text.

Claims the paper makes

This paper’s full text isn’t openly available, so its claims and self-check weren’t run. The registry and citation checks still apply.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Dynamic Programming
Bellman, R. E. (2010) · doi:10.1515/9781400835386 · Cited by 14,588
Linear Optimal Control Systems
Kwakernaak, H., Sivan, R., & Tyreus, B. N. D. (1974) · Journal of Dynamic Systems Measurement and Control · doi:10.1115/1.3426828 · Cited by 5,010
Neuronlike adaptive elements that can solve difficult learning control problems
Barto, A. G., Sutton, R. S., & Anderson, C. W. (1983) · IEEE Transactions on Systems Man and Cybernetics · doi:10.1109/tsmc.1983.6313077 · Cited by 3,325
Stable adaptive fuzzy control of nonlinear systems
Wang, L.-X. (1993) · IEEE Transactions on Fuzzy Systems · doi:10.1109/91.227383 · Cited by 1,622
Dynamic Programming: Deterministic and Stochastic Models
Bertsekas, D. P. (1987) · Medical Entomology and Zoology · Cited by 1,348
Introduction to dynamic systems: theory, models and applications
(1981) · Mathematics and Computers in Simulation · doi:10.1016/0378-4754(81)90069-0 · Cited by 1,136
Gaussian Networks for Direct Adaptive Control
Sanner, R. M., & Slotine, J.-J. E. (1991) · Proceedings of the ... American Control Conference/Proceedings of the American Control Conference · doi:10.23919/acc.1991.4791778 · Cited by 1,079
Approximate dynamic programming for real-time control and neural modeling
Werbos, P. J. (1992) · Medical Entomology and Zoology · Cited by 971
Introduction to dynamic systems
Williams, M. (2016) · Structural Dynamics · doi:10.1201/9781315375311-5 · Cited by 568
Adaptive neural network control of nonlinear systems by state and output feedback
Ge, S. S., Hang, C. C., & Zhang, T. (1999) · IEEE Transactions on Systems Man and Cybernetics Part B (Cybernetics) · doi:10.1109/3477.809035 · Cited by 419

Flagged first, then by citations

Show all 32 references

Research works printed but not linked: 4

2001, Jour of Spacecraft and Rockets Special Section on Hyper-X, 38, 801
sitz, 1998, HYPER-X Hypersonic experimental research vehicle
wang, 1996, stable adaptive fuzzy control of nonlinear systems, IEEE Trans Syst Man Cybern B Cybern, 26, 677
barnett, 1971, The Matrices of Control Theory

Sorted by citations

Online actor–critic algorithm to solve the continuous-time infinite horizon optimal control problem
Vamvoudakis, K. G., & Lewis, F. L. (2010) · Automatica · doi:10.1016/j.automatica.2010.02.018 · Cited by 1,722
Reinforcement learning and adaptive dynamic programming for feedback control
Lewis, F. L., & Vrabie, D. (2009) · IEEE Circuits and Systems Magazine · doi:10.1109/mcas.2009.933854 · Cited by 1,591
Nearly optimal control laws for nonlinear systems with saturating actuators using a neural network HJB approach
Abu-Khalaf, M., & Lewis, F. L. (2005) · Automatica · doi:10.1016/j.automatica.2004.11.034 · Cited by 1,350
Reinforcement Learning and Feedback Control: Using Natural Decision Methods to Design Optimal Adaptive Controllers
Lewis, F. L., Vrabie, D., & Vamvoudakis, K. G. (2012) · IEEE Control Systems · doi:10.1109/mcs.2012.2214134 · Cited by 1,122
Computational adaptive optimal control for continuous-time linear systems with completely unknown dynamics
Jiang, Y., & Jiang, Z. (2012) · Automatica · doi:10.1016/j.automatica.2012.06.096 · Cited by 1,079
Discrete-Time Nonlinear HJB Solution Using Approximate Dynamic Programming: Convergence Proof
Al-Tamimi, A., Lewis, F. L., & Abu-Khalaf, M. (2008) · IEEE Transactions on Systems Man and Cybernetics Part B (Cybernetics) · doi:10.1109/tsmcb.2008.926614 · Cited by 1,073
Adaptive optimal control for continuous-time linear systems based on policy iteration
Vrabie, D., Pastravanu, O. C., Abu-Khalaf, M., & Lewis, F. L. (2008) · Automatica · doi:10.1016/j.automatica.2008.08.017 · Cited by 953
Adaptive Dynamic Programming: An Introduction
Wang, F., Zhang, H., & Liu, D. (2009) · IEEE Computational Intelligence Magazine · doi:10.1109/mci.2009.932261 · Cited by 901
Handbook of Learning and Approximate Dynamic Programming
Si, J., Barto, A. G., Powell, W. B., & Wunsch, D. C. (2004) · Institute of Electrical and Electronics Engineers eBooks · doi:10.1109/9780470544785 · Cited by 863
Policy Iteration Adaptive Dynamic Programming Algorithm for Discrete-Time Nonlinear Systems
Liu, D., & Wei, Q. (2013) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2013.2281663 · Cited by 753
Adaptive Dynamic Programming for Control: A Survey and Recent Advances
Liu, D., Xue, S., Zhao, B., Luo, B., & Wei, Q. (2020) · IEEE Transactions on Systems Man and Cybernetics Systems · doi:10.1109/tsmc.2020.3042876 · Cited by 725
Neural-Network-Based Near-Optimal Control for a Class of Discrete-Time Affine Nonlinear Systems With Control Constraints
Zhang, H., Luo, Y., & Liu, D. (2009) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.2009.2027233 · Cited by 652
A novel actor–critic–identifier architecture for approximate optimal control of uncertain nonlinear systems
Bhasin, S., Kamalapurkar, R., Johnson, M. L., Vamvoudakis, K. G., Lewis, F. L., & Dixon, W. E. (2012) · Automatica · doi:10.1016/j.automatica.2012.09.019 · Cited by 654
Data-Driven Robust Approximate Optimal Tracking Control for Unknown General Nonlinear Systems Using Adaptive Dynamic Programming Method
Zhang, H., Cui, L., Zhang, X., & Luo, Y. (2011) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.2011.2168538 · Cited by 645
Neural network approach to continuous-time direct adaptive optimal control for partially unknown nonlinear systems
Vrabie, D., & Lewis, F. L. (2009) · Neural Networks · doi:10.1016/j.neunet.2009.03.008 · Cited by 636
Model-free Q-learning designs for linear discrete-time zero-sum games with application to H-infinity control
Al-Tamimi, A., Lewis, F. L., & Abu-Khalaf, M. (2007) · Automatica · doi:10.1016/j.automatica.2006.09.019 · Cited by 611
Integral reinforcement learning and experience replay for adaptive optimal control of partially-unknown constrained-input continuous-time systems
Modares, H., Lewis, F. L., & Sistani, M. B. N. (2013) · Automatica · doi:10.1016/j.automatica.2013.09.043 · Cited by 574
A Novel Infinite-Time Optimal Tracking Control Scheme for a Class of Discrete-Time Nonlinear Systems via the Greedy HDP Iteration Algorithm
Zhang, H., Wei, Q., & Luo, Y. (2008) · IEEE Transactions on Systems Man and Cybernetics Part B (Cybernetics) · doi:10.1109/tsmcb.2008.920269 · Cited by 489
Value Iteration Adaptive Dynamic Programming for Optimal Control of Discrete-Time Nonlinear Systems
Wei, Q., Liu, D., & Lin, H. (2015) · IEEE Transactions on Cybernetics · doi:10.1109/tcyb.2015.2492242 · Cited by 468
Near-Optimal Control for Nonzero-Sum Differential Games of Continuous-Time Nonlinear Systems Using Single-Network ADP
Zhang, H., Cui, L., & Luo, Y. (2012) · IEEE Transactions on Cybernetics · doi:10.1109/tsmcb.2012.2203336 · Cited by 465
Robust Adaptive Dynamic Programming and Feedback Stabilization of Nonlinear Systems
Jiang, Y., & Jiang, Z. (2014) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2013.2294968 · Cited by 419
Optimal control of unknown nonaffine nonlinear discrete-time systems based on adaptive dynamic programming
Wang, D., Liu, D., Wei, Q., Zhao, D., & Jin, N. (2012) · Automatica · doi:10.1016/j.automatica.2012.05.049 · Cited by 394
Adaptive Critic Nonlinear Robust Control: A Survey
Wang, D., He, H., & Liu, D. (2017) · IEEE Transactions on Cybernetics · doi:10.1109/tcyb.2017.2712188 · Cited by 384
Off-Policy Reinforcement Learning for $ H_\infty $ Control Design
Luo, B., Wu, H., & Huang, T. (2014) · IEEE Transactions on Cybernetics · doi:10.1109/tcyb.2014.2319577 · Cited by 363
Simplified Optimized Backstepping Control for a Class of Nonlinear Strict-Feedback Systems With Unknown Dynamic Functions
Wen, G., Chen, C. L. P., & Ge, S. S. (2020) · IEEE Transactions on Cybernetics · doi:10.1109/tcyb.2020.3002108 · Cited by 347
Value iteration and adaptive dynamic programming for data-driven adaptive optimal control design
Bian, T., & Jiang, Z.-P. (2016) · Automatica · doi:10.1016/j.automatica.2016.05.003 · Cited by 348
Dynamic Energy Management System for a Smart Microgrid
Venayagamoorthy, G. K., Sharma, R. K., Gautam, P. K., & Ahmadi, A. (2016) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2016.2514358 · Cited by 329
Computational Intelligence in Urban Traffic Signal Control: A Survey
Zhao, D., Dai, Y., & Zhang, Z. (2011) · IEEE Transactions on Systems Man and Cybernetics Part C (Applications and Reviews) · doi:10.1109/tsmcc.2011.2161577 · Cited by 315
Finite-Approximation-Error-Based Optimal Control Approach for Discrete-Time Nonlinear Systems
Liu, D., & Wei, Q. (2012) · IEEE Transactions on Cybernetics · doi:10.1109/tsmcb.2012.2216523 · Cited by 303
A novel optimal bipartite consensus control scheme for unknown multi-agent systems via model-free reinforcement learning
Peng, Z., Hu, J., Shi, K., Luo, R., Huang, R., Ghosh, B. K., & Huang, J. (2019) · Applied Mathematics and Computation · doi:10.1016/j.amc.2019.124821 · Cited by 302
Adaptive Dynamic Programming for Finite-Horizon Optimal Control of Discrete-Time Nonlinear Systems With $\varepsilon$-Error Bound
Wang, F., Jin, N., Liu, D., & Wei, Q. (2010) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.2010.2076370 · Cited by 294
Cyber-physical-social system in intelligent transportation
Xiong, G., Zhu, F., Liu, X., Dong, X., Huang, W., Chen, S., & Zhao, K. (2015) · IEEE/CAA Journal of Automatica Sinica · doi:10.1109/jas.2015.7152667 · Cited by 266
Online Synchronous Approximate Optimal Learning Algorithm for Multi-Player Non-Zero-Sum Games With Unknown Dynamics
Liu, D., Li, H., & Wang, D. (2014) · IEEE Transactions on Systems Man and Cybernetics Systems · doi:10.1109/tsmc.2013.2295351 · Cited by 261
Data-Driven Zero-Sum Neuro-Optimal Control for a Class of Continuous-Time Unknown Nonlinear Systems With Disturbance Using ADP
Wei, Q., Song, R., & Yan, P. (2015) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2015.2464080 · Cited by 256
Reinforcement Learning Neural-Network-Based Controller for Nonlinear Discrete-Time Systems With Input Constraints
He, P., & Jagannathan, S. (2007) · IEEE Transactions on Systems Man and Cybernetics Part B (Cybernetics) · doi:10.1109/tsmcb.2006.883869 · Cited by 236
Residential energy scheduling for variable weather solar energy based on adaptive dynamic programming
Liu, D., Xu, Y., Wei, Q., & Liu, X. (2017) · IEEE/CAA Journal of Automatica Sinica · doi:10.1109/jas.2017.7510739 · Cited by 235
Neural-Network-Based Constrained Optimal Control Scheme for Discrete-Time Switched Nonlinear System Using Dual Heuristic Programming
Zhang, H., Qin, C., & Luo, Y. (2014) · IEEE Transactions on Automation Science and Engineering · doi:10.1109/tase.2014.2303139 · Cited by 234
Online Adaptive Policy Learning Algorithm for $H_{\infty }$ State Feedback Control of Unknown Affine Nonlinear Discrete-Time Systems
Zhang, H., Qin, C., Jiang, B., & Luo, Y. (2014) · IEEE Transactions on Cybernetics · doi:10.1109/tcyb.2014.2313915 · Cited by 234
Adaptive Fault-Tolerant Tracking Control for MIMO Discrete-Time Systems via Reinforcement Learning Algorithm With Less Learning Parameters
Liu, L., Wang, Z., & Zhang, H. (2016) · IEEE Transactions on Automation Science and Engineering · doi:10.1109/tase.2016.2517155 · Cited by 232
Experience Replay for Optimal Control of Nonzero-Sum Game Systems With Unknown Dynamics
Zhao, D., Zhang, Q., Wang, D., & Zhu, Y. (2015) · IEEE Transactions on Cybernetics · doi:10.1109/tcyb.2015.2488680 · Cited by 229
Finite-horizon neuro-optimal tracking control for a class of discrete-time nonlinear systems using adaptive dynamic programming approach
Wang, D., Liu, D., & Wei, Q. (2011) · Neurocomputing · doi:10.1016/j.neucom.2011.03.058 · Cited by 225
Adaptive Dynamic Programming for Optimal Tracking Control of Unknown Nonlinear Systems With Application to Coal Gasification
Wei, Q., & Liu, D. (2013) · IEEE Transactions on Automation Science and Engineering · doi:10.1109/tase.2013.2284545 · Cited by 223
Adaptive dynamic programming and optimal control of nonlinear nonaffine systems
Bian, T., Jiang, Y., & Jiang, Z. (2014) · Automatica · doi:10.1016/j.automatica.2014.08.023 · Cited by 223
A single network adaptive critic (SNAC) architecture for optimal control synthesis for a class of nonlinear systems
Padhi, R., Unnikrishnan, N., Wang, X., & Balakrishnan, S. N. (2006) · Neural Networks · doi:10.1016/j.neunet.2006.08.010 · Cited by 216
Asymptotically Stable Adaptive–Optimal Control Algorithm With Saturating Actuators and Relaxed Persistence of Excitation
Vamvoudakis, K. G., Miranda, M. F., & Hespanha, J. P. (2015) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2015.2487972 · Cited by 213
Distributed Optimal Consensus Control for Nonlinear Multiagent System With Unknown Dynamic
Zhang, J., Zhang, H., & Feng, T. (2017) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2017.2728622 · Cited by 205
Reinforcement learning for adaptive optimal control of unknown continuous-time nonlinear systems with input constraints
Yang, X., Liu, D., & Wang, D. (2013) · International Journal of Control · doi:10.1080/00207179.2013.848292 · Cited by 203
Optimal Tracking Control for a Class of Nonlinear Discrete-Time Systems With Time Delays Based on Heuristic Dynamic Programming
Zhang, H., Song, R., Wei, Q., & Zhang, T. (2011) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.2011.2172628 · Cited by 193
The intelligent critic framework for advanced optimal control
Wang, D., Ha, M., & Zhao, M. (2022) · Artificial Intelligence Review · doi:10.1007/s10462-021-10118-9 · Cited by 193
Integral Q-learning and explorized policy iteration for adaptive optimal control of continuous-time linear systems
Lee, J. Y., Park, J. B., & Choi, Y. H. (2012) · Automatica · doi:10.1016/j.automatica.2012.06.008 · Cited by 191
Reinforcement Learning in Continuous State and Action Spaces
van Hasselt, H. P. (2012) · Adaptation, learning, and optimization · doi:10.1007/978-3-642-27645-3_7 · Cited by 197
Reinforcement Learning Controller Design for Affine Nonlinear Discrete-Time Systems using Online Approximators
Yang, Q., & Jagannathan, S. (2011) · IEEE Transactions on Systems Man and Cybernetics Part B (Cybernetics) · doi:10.1109/tsmcb.2011.2166384 · Cited by 190
Adaptive Learning in Tracking Control Based on the Dual Critic Network Design
Ni, Z., He, H., & Wen, J. (2013) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2013.2247627 · Cited by 189
Finite-Approximation-Error-Based Discrete-Time Iterative Adaptive Dynamic Programming
Wei, Q., Wang, F., Liu, D., & Yang, X. (2014) · IEEE Transactions on Cybernetics · doi:10.1109/tcyb.2014.2354377 · Cited by 188
Neural-network-based zero-sum game for discrete-time nonlinear systems via iterative adaptive dynamic programming algorithm
Liu, D., Li, H., & Wang, D. (2013) · Neurocomputing · doi:10.1016/j.neucom.2012.11.021 · Cited by 188
Optimal control of unknown affine nonlinear discrete-time systems using offline-trained neural networks with proof of convergence
Dierks, T., Thumati, B. T., & Jagannathan, S. (2009) · Neural Networks · doi:10.1016/j.neunet.2009.06.014 · Cited by 186
Adaptive Critic Designs for Discrete-Time Zero-Sum Games With Application to $H_{\infty}$ Control
Al-Tamimi, A., Abu-Khalaf, M., & Lewis, F. L. (2007) · IEEE Transactions on Systems Man and Cybernetics Part B (Cybernetics) · doi:10.1109/tsmcb.2006.880135 · Cited by 184
Online adaptive algorithm for optimal control with integral reinforcement learning
Vamvoudakis, K. G., Vrabie, D., & Lewis, F. L. (2013) · International Journal of Robust and Nonlinear Control · doi:10.1002/rnc.3018 · Cited by 180
Robust adaptive dynamic programming for linear and nonlinear systems: An overview
Jiang, Z., & Jiang, Y. (2013) · European Journal of Control · doi:10.1016/j.ejcon.2013.05.017 · Cited by 171
Discrete-Time Local Value Iteration Adaptive Dynamic Programming: Convergence Analysis
Wei, Q., Lewis, F. L., Liu, D., Song, R., & Lin, H. (2016) · IEEE Transactions on Systems Man and Cybernetics Systems · doi:10.1109/tsmc.2016.2623766 · Cited by 173
A Self-Learning Call Admission Control Scheme for CDMA Cellular Networks
Liu, D., Zhang, Y., & Zhang, H. (2005) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.2005.853408 · Cited by 162
Generalized Policy Iteration Adaptive Dynamic Programming for Discrete-Time Nonlinear Systems
Liu, D., Wei, Q., & Yan, P. (2015) · IEEE Transactions on Systems Man and Cybernetics Systems · doi:10.1109/tsmc.2015.2417510 · Cited by 158
Robust Adaptive Dynamic Programming for Large-Scale Systems With an Application to Multimachine Power Systems
Jiang, Y., & Jiang, Z. (2012) · IEEE Transactions on Circuits & Systems II Express Briefs · doi:10.1109/tcsii.2012.2213353 · Cited by 155
Mixed Iterative Adaptive Dynamic Programming for Optimal Battery Energy Control in Smart Residential Microgrids
Wei, Q., Liu, D., Lewis, F. L., Liu, Y., & Zhang, J. (2017) · IEEE Transactions on Industrial Electronics · doi:10.1109/tie.2017.2650872 · Cited by 155
Data-based fault tolerant control for affine nonlinear systems through particle swarm optimized neural networks
Lin, H., Zhao, B., Liu, D., & Alippi, C. (2020) · IEEE/CAA Journal of Automatica Sinica · doi:10.1109/jas.2020.1003225 · Cited by 155
Dynamic Event-Triggering Neural Learning Control for Partially Unknown Nonlinear Systems
Mu, C., Wang, K., & Qiu, T. (2020) · IEEE Transactions on Cybernetics · doi:10.1109/tcyb.2020.3004493 · Cited by 150
Action dependent heuristic dynamic programming for home energy resource scheduling
Fuselli, D., Angelis, F. D., Boaro, M., Squartini, S., Wei, Q., Liu, D., & Piazza, F. (2013) · International Journal of Electrical Power & Energy Systems · doi:10.1016/j.ijepes.2012.11.023 · Cited by 153
Infinite Horizon Self-Learning Optimal Control of Nonaffine Discrete-Time Nonlinear Systems
Wei, Q., Liu, D., & Yang, X. (2015) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2015.2401334 · Cited by 142
Multiple UAVs path planning algorithms: a comparative study
Sathyaraj, B. M., Jain, L. C., Finn, A., & Drake, S. P. (2008) · Fuzzy Optimization and Decision Making · doi:10.1007/s10700-008-9035-0 · Cited by 141
Adaptive Optimal Control of Highly Dissipative Nonlinear Spatially Distributed Processes With Neuro-Dynamic Programming
Luo, B., Wu, H., & Li, H. (2014) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2014.2320744 · Cited by 139
A Novel Iterative $\theta $ -Adaptive Dynamic Programming for Discrete-Time Nonlinear Systems
Wei, Q., & Liu, D. (2013) · IEEE Transactions on Automation Science and Engineering · doi:10.1109/tase.2013.2280974 · Cited by 138
Neural-network-observer-based optimal control for unknown nonlinear systems using adaptive dynamic programming
Liu, D., Huang, Y., Wang, D., & Wei, Q. (2013) · International Journal of Control · doi:10.1080/00207179.2013.790562 · Cited by 138
Neural‐network‐based online optimal control for uncertain non‐linear continuous‐time systems with control constraints
Yang, X., Liu, D., & Huang, Y. (2013) · IET Control Theory and Applications · doi:10.1049/iet-cta.2013.0472 · Cited by 134
Discrete-Time Nonzero-Sum Games for Multiplayer Using Policy-Iteration-Based Adaptive Dynamic Programming Algorithms
Zhang, H., Jiang, H., Luo, C., & Xiao, G. (2016) · IEEE Transactions on Cybernetics · doi:10.1109/tcyb.2016.2611613 · Cited by 134
Optimal Consensus Control Design for Multiagent Systems With Multiple Time Delay Using Adaptive Dynamic Programming
Zhang, H., Ren, H., Mu, Y., & Han, J. (2021) · IEEE Transactions on Cybernetics · doi:10.1109/tcyb.2021.3090067 · Cited by 134
Fault-tolerant control of a hydraulic servo actuator via adaptive dynamic programming
Stojanović, V. (2023) · Mathematical Modelling and Control · doi:10.3934/mmc.2023016 · Cited by 136
Reinforcement learning solution for HJB equation arising in constrained optimal control problem
Luo, B., Wu, H., Huang, T., & Liu, D. (2015) · Neural Networks · doi:10.1016/j.neunet.2015.08.007 · Cited by 131
A novel adaptive dynamic programming based on tracking error for nonlinear discrete-time systems
Li, C., Ding, J., Lewis, F. L., & Chai, T. (2021) · Automatica · doi:10.1016/j.automatica.2021.109687 · Cited by 128
Integral Reinforcement Learning for Continuous-Time Input-Affine Nonlinear Systems With Simultaneous Invariant Explorations
Lee, J. Y., Park, J. B., & Choi, Y. H. (2014) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2014.2328590 · Cited by 127
Data-Based Optimal Control of Multiagent Systems: A Reinforcement Learning Design Approach
Zhang, J., Wang, Z., & Zhang, H. (2018) · IEEE Transactions on Cybernetics · doi:10.1109/tcyb.2018.2868715 · Cited by 125
From inverse optimal control to inverse reinforcement learning: A historical review
Azar, N. A., Shahmansoorian, A., & Davoudi, M. (2020) · Annual Reviews in Control · doi:10.1016/j.arcontrol.2020.06.001 · Cited by 127
Reinforcement Learning-Based Output Feedback Control of Nonlinear Systems With Input Constraints
He, P., & Jagannathan, S. (2005) · IEEE Transactions on Systems Man and Cybernetics Part B (Cybernetics) · doi:10.1109/tsmcb.2004.840124 · Cited by 124
Optimal distributed synchronization control for continuous-time heterogeneous multi-agent differential graphical games
Wei, Q., Liu, D., & Lewis, F. L. (2015) · Information Sciences · doi:10.1016/j.ins.2015.04.044 · Cited by 122
Optimized Leader-Follower Consensus Control Using Reinforcement Learning for a Class of Second-Order Nonlinear Multiagent Systems
Wen, G., & Li, B. (2021) · IEEE Transactions on Systems Man and Cybernetics Systems · doi:10.1109/tsmc.2021.3130070 · Cited by 115
Decentralized Tracking Optimization Control for Partially Unknown Fuzzy Interconnected Systems via Reinforcement Learning Method
Zhang, K., Zhang, H., Mu, Y., & Liu, C. (2020) · IEEE Transactions on Fuzzy Systems · doi:10.1109/tfuzz.2020.2966418 · Cited by 112
Learning-Based Control: A Tutorial and Some Recent Results
Jiang, Z., Bian, T., & Weinan, G. (2020) · Foundations and Trends® in Systems and Control · doi:10.1561/2600000023 · Cited by 138
Online optimal tracking control of continuous-time linear systems with unknown dynamics by using adaptive dynamic programming
Qin, C., Zhang, H., & Luo, Y. (2013) · International Journal of Control · doi:10.1080/00207179.2013.863432 · Cited by 111
Issues on Stability of ADP Feedback Controllers for Dynamical Systems
Balakrishnan, S. N., Ding, J., & Lewis, F. L. (2008) · IEEE Transactions on Systems Man and Cybernetics Part B (Cybernetics) · doi:10.1109/tsmcb.2008.926599 · Cited by 108
A self-learning scheme for residential energy system control and management
Huang, T., & Liu, D. (2011) · Neural Computing and Applications · doi:10.1007/s00521-011-0711-6 · Cited by 108
Discrete-Time Optimal Control via Local Policy Iteration Adaptive Dynamic Programming
Wei, Q., Liu, D., Lin, Q., & Song, R. (2016) · IEEE Transactions on Cybernetics · doi:10.1109/tcyb.2016.2586082 · Cited by 107
Data-driven control of hydraulic servo actuator: An event-triggered adaptive dynamic programming approach
Djordjevic, V., Tao, H., Song, X., He, S., Gao, W., & Stojanovic, V. (2023) · Mathematical Biosciences and Engineering · doi:10.3934/mbe.2023376 · Cited by 107
Event-Triggered Adaptive Critic Control Design for Discrete-Time Constrained Nonlinear Systems
Ha, M., Wang, D., & Liu, D. (2018) · IEEE Transactions on Systems Man and Cybernetics Systems · doi:10.1109/tsmc.2018.2868510 · Cited by 102
Adaptive optimal control for a class of continuous-time affine nonlinear systems with unknown internal dynamics
Liu, D., Yang, X., & Li, H. (2012) · Neural Computing and Applications · doi:10.1007/s00521-012-1249-y · Cited by 102
Continuous-Time Distributed Policy Iteration for Multicontroller Nonlinear Systems
Wei, Q., Li, H., Yang, X., & He, H. (2020) · IEEE Transactions on Cybernetics · doi:10.1109/tcyb.2020.2979614 · Cited by 98
Nonlinear System Control Using Adaptive Neural Fuzzy Networks Based on a Modified Differential Evolution
Chen, C.-H., Lin, C.-J., & Lin, C. (2009) · IEEE Transactions on Systems Man and Cybernetics Part C (Applications and Reviews) · doi:10.1109/tsmcc.2009.2016572 · Cited by 97
Neural-network-based optimal tracking control scheme for a class of unknown discrete-time nonlinear systems using iterative ADP algorithm
Huang, Y., & Liu, D. (2013) · Neurocomputing · doi:10.1016/j.neucom.2012.07.047 · Cited by 98
An Overview of Research on Adaptive Dynamic Programming
Zhang, H., Zhang, X., Luo, Y., & Jun, Y. A. N. G. (2013) · Acta Automatica Sinica · doi:10.1016/s1874-1029(13)60031-2 · Cited by 136
Adaptive Learning and Control for MIMO System Based on Adaptive Dynamic Programming
Fu, J. Y., He, H., & Zhou, X. (2011) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.2011.2147797 · Cited by 96
Continuous-Time Q-Learning for Infinite-Horizon Discounted Cost Linear Quadratic Regulator Problems
Muthukumar, P., Modares, H., Lewis, F. L., & Aurangzeb, M. (2015) · IEEE Transactions on Cybernetics · doi:10.1109/tcyb.2014.2322116 · Cited by 97
Approximate $N$ -Player Nonzero-Sum Game Solution for an Uncertain Continuous Nonlinear System
Johnson, M., Kamalapurkar, R., Bhasin, S., & Dixon, W. E. (2014) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2014.2350835 · Cited by 95

Authors: Murray, J. J., Cox, C. J., Lendaris, G. G., & Saeks, R.
Affiliations: Stony Brook University (US)
Accurate Automation (United States) (US)
Portland State University (US)
ORCID: Murray, J.J.
Venue: IEEE Transactions on Systems, Man and Cybernetics, Part C (Applications and Reviews)
Publisher: Institute of Electrical and Electronics Engineers (IEEE)
Published: 2002-05
Volume: 32
Issue: 2
Pages: 140-153
ISSN: 1094-6977 (ISSN-L)
DOI: 10.1109/tsmcc.2002.801727
OpenAlex: W2160561608
Cited by: 677
References: 32
Citations by year: 2026: 16 · 2025: 36 · 2024: 31 · 2023: 26 · 2022: 34 · 2021: 34 · 2020: 39 · 2019: 28 · 2018: 27 · 2017: 42 · 2016: 36 · 2015: 40 · 2014: 42 · 2013: 43 · 2012: 45
FWCI: 4.99
Type: journal article
Language: en
Open access: closed
Licence: ieeexplore.ieee.org/Xplorehelp/downloads/license-information/IEEE.html (vor)
Locations: IEEE Transactions on Systems Man and Cybernetics Part C (Applications and Reviews) (publishedVersion)
PDXScholar (Portland State University) (submittedVersion)
Topics: Physical Sciences › Engineering › Control and Systems Engineering › Adaptive Dynamic Programming Control
Physical Sciences › Engineering › Control and Systems Engineering › Advanced Control Systems Optimization
Physical Sciences › Computer Science › Artificial Intelligence › Reinforcement Learning in Robotics
Keywords: adaptive dynamic programming, optimal control, Hamilton-Jacobi-Bellman equation, stability analysis, convergence analysis, unknown dynamics, nonlinear systems, input-affine systems, radial basis function approximation, flight control, soft computing

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

Adaptive dynamic programming

Abstract

Overview

Integrity signals

Verification

Claims the paper makes

Claims other papers make about this one

Citations

Cites

Cited by

Related

Metadata

Record

About these checks