Adaptive dynamic programming
Abstract
Unlike the many soft computing applications where it suffices to achieve a "good approximation most of the time," a control system must be stable all of the time. As such, if one desires to learn a control law in real-time, a fusion of soft computing techniques to learn the appropriate control law with hard computing techniques to maintain the stability constraint and guarantee convergence is required. The objective of the paper is to describe an adaptive dynamic programming algorithm (ADPA) which fuses soft computing techniques to learn the optimal cost (or return) functional for a stabilizable nonlinear system with unknown dynamics and hard computing techniques to verify the stability and convergence of the algorithm. Specifically, the algorithm is initialized with a (stabilizing) cost functional and the system is run with the corresponding control law (defined by the Hamilton-Jacobi-Bellman equation), with the resultant state trajectories used to update the cost functional in a soft computing mode. Hard computing techniques are then used to show that this process is globally convergent with stepwise stability to the optimal cost functional/control law pair for an (unknown) input affine system with an input quadratic performance measure (modulo the appropriate technical conditions). Three specific implementations of the ADPA are developed for 1) the linear case, 2) for the nonlinear case using a locally quadratic approximation to the cost functional, and 3) the nonlinear case using a radial basis function approximation of the cost functional; illustrated by applications to flight control.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
ADP is an effective way to achieve high performance of the optimal controller and relies on adaptive control, optimal control and reinforcement learning.
“Adaptive dynamic programming (ADP) ensures an e ective way to achieve high performance of the optimal controller which relies on adaptive control, optimal control and reinforcement learning [9, 23-26].”
From Data-driven control of hydraulic servo actuator: An event-triggered adaptive dynamic programming approach (2023) · cites this paper as [23]
Cites
Bellman, R. E. (2010) · doi:10.1515/9781400835386 · Cited by 14,588 · see citing works
Linear Optimal Control Systems
Kwakernaak, H., Sivan, R., & Tyreus, B. N. D. (1974) · Journal of Dynamic Systems Measurement and Control · doi:10.1115/1.3426828 · Cited by 5,010 · see citing works
Neuronlike adaptive elements that can solve difficult learning control problems
Barto, A. G., Sutton, R. S., & Anderson, C. W. (1983) · IEEE Transactions on Systems Man and Cybernetics · doi:10.1109/tsmc.1983.6313077 · Cited by 3,238 · see citing works
Stable adaptive fuzzy control of nonlinear systems
Wang, L.-X. (1993) · IEEE Transactions on Fuzzy Systems · doi:10.1109/91.227383 · Cited by 1,622 · see citing works
Dynamic Programming: Deterministic and Stochastic Models
Bertsekas, D. P. (1987) · Medical Entomology and Zoology · Cited by 1,348 · see citing works
Introduction to dynamic systems: theory, models and applications
(1981) · Mathematics and Computers in Simulation · doi:10.1016/0378-4754(81)90069-0 · Cited by 1,136 · see citing works
Gaussian Networks for Direct Adaptive Control
Sanner, R. M., & Slotine, J.-J. E. (1991) · Proceedings of the ... American Control Conference/Proceedings of the American Control Conference · doi:10.23919/acc.1991.4791778 · Cited by 1,074 · see citing works
Approximate dynamic programming for real-time control and neural modeling
Werbos, P. J. (1992) · Medical Entomology and Zoology · Cited by 971 · see citing works
Introduction to dynamic systems
Williams, M. (2016) · Structural Dynamics · doi:10.1201/9781315375311-5 · Cited by 568 · see citing works
Adaptive neural network control of nonlinear systems by state and output feedback
Ge, S. S., Hang, C. C., & Zhang, T. (1999) · IEEE Transactions on Systems Man and Cybernetics Part B (Cybernetics) · doi:10.1109/3477.809035 · Cited by 419 · see citing works
Ray, W. D., & Barnett, S. M. (1972) · Operational Research Quarterly (1970-1977) · doi:10.2307/3007980 · Cited by 312 · see citing works
Stable adaptive fuzzy controllers with application to inverted pendulum tracking
Wang, L.-X. (1996) · IEEE Transactions on Systems Man and Cybernetics Part B (Cybernetics) · doi:10.1109/3477.537311 · Cited by 263 · see citing works
Maybeck, P. S. (1999) · International Journal of Robust and Nonlinear Control · doi:10.1002/(sici)1099-1239(19991215)9:14 · Cited by 195 · see citing works
Self-citationDirect Adaptive and Neural Control of Wing-Rock Motion of Slender Delta Wings
Singh, S. N., Yirn, W., & Wells, W. R. (1995) · Journal of Guidance Control and Dynamics · doi:10.2514/3.56652 · Cited by 144 · see citing works
Stability of fuzzy linguistic control systems
Langari, G., & Tomizuka, M. (1990) · Proceedings of the IEEE Conference on Decision & Control, including the Symposium on Adaptive Processes · doi:10.1109/cdc.1990.204014 · Cited by 142 · see citing works
Variable neural networks for adaptive control of nonlinear systems
Liu, S., Kadirkamanathan, V., & Billings, S. A. (1999) · IEEE Transactions on Systems Man and Cybernetics Part C (Applications and Reviews) · doi:10.1109/5326.740668 · Cited by 108 · see citing works
Foundations of Mathematical Analysis
Ponnusamy, S. (2011) · Birkhäuser Boston eBooks · doi:10.1007/978-0-8176-8292-7 · Cited by 92 · see citing works
Coaker, P. B. (1964) · Journal of the Operational Research Society · doi:10.1057/jors.1964.34 · Cited by 77 · see citing works
Foundations of mathematical analysis
(2012) · Choice Reviews Online · doi:10.5860/choice.49-6937 · Cited by 71 · see citing works
Applications of Liapunov Methods in Stability
Halanay, A., & Rǎsvan, V. B. (1993) · Springer eBooks · doi:10.1007/978-94-011-1600-8 · Cited by 53 · see citing works
Adaptive critic design for intelligent steering and speed control of a 2-axle vehicle
Lendaris, G. G., Schultz, L., & Shannon, T. T. (2000) · doi:10.1109/ijcnn.2000.861283 · Cited by 43 · see citing works
Self-citationAdaptive critic design in learning to play game of Go
Zaman, R., Prokhorov, D. V., & Wunsch, D. C. (2002) · doi:10.1109/icnn.1997.611623 · Cited by 20 · see citing works
Development and Flight Test of the X-43A-LS Hypersonic Configuration UAV
Gibson, C., Neidhoefer, J. C., Cooper, S., Carlton, L. A., Cox, C. J., & Jorgensen, C. C. (2002) · 1st UAV Conference · doi:10.2514/6.2002-3462 · Cited by 20 · see citing works
Self-citationFuzzy control stabilization with applications to motorcycle control
Wu, J. C., & Liu, T. S. (1996) · IEEE Transactions on Systems Man and Cybernetics Part B (Cybernetics) · doi:10.1109/3477.544297 · Cited by 15 · see citing works
Adaptive critic control and functional link neural networks
Cox, C. J., & Saeks, R. E. (2002) · Conference proceedings/Conference proceedings - IEEE International Conference on Systems, Man, and Cybernetics · doi:10.1109/icsmc.1998.728126 · Cited by 14 · see citing works
Self-citationLoFLYTE - A neurocontrols testbed
Saeks, R. E., Cox, C. J., & Pap, R. M. (1997) · AIAA Aerospace Sciences Meeting and Exhibit · doi:10.2514/6.1997-85 · Cited by 11 · see citing works
Self-citationProkhorov, D. V., & Feldkamp, L. A. (2002) · doi:10.1109/icnn.1997.614396 · Cited by 10 · see citing works
Neural adaptive control of LoFLYTE(R)
Cox, C. J., Neidhoefer, J. C., Saeks, R. E., & Lendaris, G. C. (2001) · Proceedings of the ... American Control Conference/Proceedings of the American Control Conference · doi:10.1109/acc.2001.946345 · Cited by 8 · see citing works
Self-citationPap, R. M., Saeks, R. E., Lewis, C., Carlton, A. J., Cox, C. J., Kocher, M., & Kersting, T. (1998) · AIAA International Space Planes and Hypersonic Systems and Technologies Conference · doi:10.2514/6.1998-1519 · Cited by 7 · see citing works
Self-citationA neurocontroller for robotic applications
Cox, C. J., Saeks, R. E., Lothers, M., Pap, R. M., & Thomas, C. (2003) · Conference proceedings/Conference proceedings - IEEE International Conference on Systems, Man, and Cybernetics · doi:10.1109/icsmc.1992.271544 · Cited by 6 · see citing works
Self-citationDevinatz, A., & Kaplan, J. L. (1972) · Indiana University Mathematics Journal · doi:10.1512/iumj.1973.22.22029 · Cited by 6 · see citing works
An improvement in the MacFarlane-Potter method for solving the algebraic Riccati equation
Holley, W. E., & Wei, S. Y. (1979) · IEEE Transactions on Automatic Control · doi:10.1109/jacc.1979.4170724 · Cited by 3 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
Online actor–critic algorithm to solve the continuous-time infinite horizon optimal control problem
Vamvoudakis, K. G., & Lewis, F. L. (2010) · Automatica · doi:10.1016/j.automatica.2010.02.018 · Cited by 1,722 · see citing works
Reinforcement learning and adaptive dynamic programming for feedback control
Lewis, F. L., & Vrabie, D. (2009) · IEEE Circuits and Systems Magazine · doi:10.1109/mcas.2009.933854 · Cited by 1,569 · see citing works
Abu-Khalaf, M., & Lewis, F. L. (2005) · Automatica · doi:10.1016/j.automatica.2004.11.034 · Cited by 1,350 · see citing works
Lewis, F. L., Vrabie, D., & Vamvoudakis, K. G. (2012) · IEEE Control Systems · doi:10.1109/mcs.2012.2214134 · Cited by 1,107 · see citing works
Jiang, Y., & Jiang, Z. (2012) · Automatica · doi:10.1016/j.automatica.2012.06.096 · Cited by 1,079 · see citing works
Discrete-Time Nonlinear HJB Solution Using Approximate Dynamic Programming: Convergence Proof
Al-Tamimi, A., Lewis, F. L., & Abu-Khalaf, M. (2008) · IEEE Transactions on Systems Man and Cybernetics Part B (Cybernetics) · doi:10.1109/tsmcb.2008.926614 · Cited by 1,073 · see citing works
Adaptive optimal control for continuous-time linear systems based on policy iteration
Vrabie, D., Pastravanu, O. C., Abu-Khalaf, M., & Lewis, F. L. (2008) · Automatica · doi:10.1016/j.automatica.2008.08.017 · Cited by 953 · see citing works
Adaptive Dynamic Programming: An Introduction
Wang, F., Zhang, H., & Liu, D. (2009) · IEEE Computational Intelligence Magazine · doi:10.1109/mci.2009.932261 · Cited by 901 · see citing works
Handbook of Learning and Approximate Dynamic Programming
Si, J., Barto, A. G., Powell, W. B., & Wunsch, D. C. (2004) · Institute of Electrical and Electronics Engineers eBooks · doi:10.1109/9780470544785 · Cited by 863 · see citing works
Policy Iteration Adaptive Dynamic Programming Algorithm for Discrete-Time Nonlinear Systems
Liu, D., & Wei, Q. (2013) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2013.2281663 · Cited by 749 · see citing works
Adaptive Dynamic Programming for Control: A Survey and Recent Advances
Liu, D., Xue, S., Zhao, B., Luo, B., & Wei, Q. (2020) · IEEE Transactions on Systems Man and Cybernetics Systems · doi:10.1109/tsmc.2020.3042876 · Cited by 714 · see citing works
Zhang, H., Luo, Y., & Liu, D. (2009) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.2009.2027233 · Cited by 653 · see citing works
Bhasin, S., Kamalapurkar, R., Johnson, M. L., Vamvoudakis, K. G., Lewis, F. L., & Dixon, W. E. (2012) · Automatica · doi:10.1016/j.automatica.2012.09.019 · Cited by 646 · see citing works
Zhang, H., Cui, L., Zhang, X., & Luo, Y. (2011) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.2011.2168538 · Cited by 645 · see citing works
Vrabie, D., & Lewis, F. L. (2009) · Neural Networks · doi:10.1016/j.neunet.2009.03.008 · Cited by 636 · see citing works
Al-Tamimi, A., Lewis, F. L., & Abu-Khalaf, M. (2007) · Automatica · doi:10.1016/j.automatica.2006.09.019 · Cited by 611 · see citing works
Modares, H., Lewis, F. L., & Sistani, M. B. N. (2013) · Automatica · doi:10.1016/j.automatica.2013.09.043 · Cited by 574 · see citing works
Zhang, H., Wei, Q., & Luo, Y. (2008) · IEEE Transactions on Systems Man and Cybernetics Part B (Cybernetics) · doi:10.1109/tsmcb.2008.920269 · Cited by 488 · see citing works
Value Iteration Adaptive Dynamic Programming for Optimal Control of Discrete-Time Nonlinear Systems
Wei, Q., Liu, D., & Lin, H. (2015) · IEEE Transactions on Cybernetics · doi:10.1109/tcyb.2015.2492242 · Cited by 468 · see citing works
Zhang, H., Cui, L., & Luo, Y. (2012) · IEEE Transactions on Cybernetics · doi:10.1109/tsmcb.2012.2203336 · Cited by 465 · see citing works
Robust Adaptive Dynamic Programming and Feedback Stabilization of Nonlinear Systems
Jiang, Y., & Jiang, Z. (2014) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2013.2294968 · Cited by 415 · see citing works
Wang, D., Liu, D., Wei, Q., Zhao, D., & Jin, N. (2012) · Automatica · doi:10.1016/j.automatica.2012.05.049 · Cited by 395 · see citing works
Adaptive Critic Nonlinear Robust Control: A Survey
Wang, D., He, H., & Liu, D. (2017) · IEEE Transactions on Cybernetics · doi:10.1109/tcyb.2017.2712188 · Cited by 378 · see citing works
Off-Policy Reinforcement Learning for $ H_\infty $ Control Design
Luo, B., Wu, H., & Huang, T. (2014) · IEEE Transactions on Cybernetics · doi:10.1109/tcyb.2014.2319577 · Cited by 363 · see citing works
Wen, G., Chen, C. L. P., & Ge, S. S. (2020) · IEEE Transactions on Cybernetics · doi:10.1109/tcyb.2020.3002108 · Cited by 347 · see citing works
Value iteration and adaptive dynamic programming for data-driven adaptive optimal control design
Bian, T., & Jiang, Z.-P. (2016) · Automatica · doi:10.1016/j.automatica.2016.05.003 · Cited by 348 · see citing works
Dynamic Energy Management System for a Smart Microgrid
Venayagamoorthy, G. K., Sharma, R. K., Gautam, P. K., & Ahmadi, A. (2016) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2016.2514358 · Cited by 329 · see citing works
Computational Intelligence in Urban Traffic Signal Control: A Survey
Zhao, D., Dai, Y., & Zhang, Z. (2011) · IEEE Transactions on Systems Man and Cybernetics Part C (Applications and Reviews) · doi:10.1109/tsmcc.2011.2161577 · Cited by 315 · see citing works
Finite-Approximation-Error-Based Optimal Control Approach for Discrete-Time Nonlinear Systems
Liu, D., & Wei, Q. (2012) · IEEE Transactions on Cybernetics · doi:10.1109/tsmcb.2012.2216523 · Cited by 304 · see citing works
Peng, Z., Hu, J., Shi, K., Luo, R., Huang, R., Ghosh, B. K., & Huang, J. (2019) · Applied Mathematics and Computation · doi:10.1016/j.amc.2019.124821 · Cited by 302 · see citing works
Wang, F., Jin, N., Liu, D., & Wei, Q. (2010) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.2010.2076370 · Cited by 294 · see citing works
Cyber-physical-social system in intelligent transportation
Xiong, G., Zhu, F., Liu, X., Dong, X., Huang, W., Chen, S., & Zhao, K. (2015) · IEEE/CAA Journal of Automatica Sinica · doi:10.1109/jas.2015.7152667 · Cited by 266 · see citing works
Liu, D., Li, H., & Wang, D. (2014) · IEEE Transactions on Systems Man and Cybernetics Systems · doi:10.1109/tsmc.2013.2295351 · Cited by 263 · see citing works
Wei, Q., Song, R., & Yan, P. (2015) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2015.2464080 · Cited by 253 · see citing works
He, P., & Jagannathan, S. (2007) · IEEE Transactions on Systems Man and Cybernetics Part B (Cybernetics) · doi:10.1109/tsmcb.2006.883869 · Cited by 236 · see citing works
Liu, D., Xu, Y., Wei, Q., & Liu, X. (2017) · IEEE/CAA Journal of Automatica Sinica · doi:10.1109/jas.2017.7510739 · Cited by 235 · see citing works
Zhang, H., Qin, C., & Luo, Y. (2014) · IEEE Transactions on Automation Science and Engineering · doi:10.1109/tase.2014.2303139 · Cited by 234 · see citing works
Zhang, H., Qin, C., Jiang, B., & Luo, Y. (2014) · IEEE Transactions on Cybernetics · doi:10.1109/tcyb.2014.2313915 · Cited by 234 · see citing works
Liu, L., Wang, Z., & Zhang, H. (2016) · IEEE Transactions on Automation Science and Engineering · doi:10.1109/tase.2016.2517155 · Cited by 232 · see citing works
Experience Replay for Optimal Control of Nonzero-Sum Game Systems With Unknown Dynamics
Zhao, D., Zhang, Q., Wang, D., & Zhu, Y. (2015) · IEEE Transactions on Cybernetics · doi:10.1109/tcyb.2015.2488680 · Cited by 229 · see citing works
Wang, D., Liu, D., & Wei, Q. (2011) · Neurocomputing · doi:10.1016/j.neucom.2011.03.058 · Cited by 224 · see citing works
Wei, Q., & Liu, D. (2013) · IEEE Transactions on Automation Science and Engineering · doi:10.1109/tase.2013.2284545 · Cited by 224 · see citing works
Adaptive dynamic programming and optimal control of nonlinear nonaffine systems
Bian, T., Jiang, Y., & Jiang, Z. (2014) · Automatica · doi:10.1016/j.automatica.2014.08.023 · Cited by 220 · see citing works
Padhi, R., Unnikrishnan, N., Wang, X., & Balakrishnan, S. N. (2006) · Neural Networks · doi:10.1016/j.neunet.2006.08.010 · Cited by 216 · see citing works
Vamvoudakis, K. G., Miranda, M. F., & Hespanha, J. P. (2015) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2015.2487972 · Cited by 210 · see citing works
Distributed Optimal Consensus Control for Nonlinear Multiagent System With Unknown Dynamic
Zhang, J., Zhang, H., & Feng, T. (2017) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2017.2728622 · Cited by 205 · see citing works
Yang, X., Liu, D., & Wang, D. (2013) · International Journal of Control · doi:10.1080/00207179.2013.848292 · Cited by 204 · see citing works
Zhang, H., Song, R., Wei, Q., & Zhang, T. (2011) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.2011.2172628 · Cited by 193 · see citing works
The intelligent critic framework for advanced optimal control
Wang, D., Ha, M., & Zhao, M. (2022) · Artificial Intelligence Review · doi:10.1007/s10462-021-10118-9 · Cited by 192 · see citing works
Lee, J. Y., Park, J. B., & Choi, Y. H. (2012) · Automatica · doi:10.1016/j.automatica.2012.06.008 · Cited by 191 · see citing works
Reinforcement Learning in Continuous State and Action Spaces
van Hasselt, H. P. (2012) · Adaptation, learning, and optimization · doi:10.1007/978-3-642-27645-3_7 · Cited by 193 · see citing works
Yang, Q., & Jagannathan, S. (2011) · IEEE Transactions on Systems Man and Cybernetics Part B (Cybernetics) · doi:10.1109/tsmcb.2011.2166384 · Cited by 190 · see citing works
Adaptive Learning in Tracking Control Based on the Dual Critic Network Design
Ni, Z., He, H., & Wen, J. (2013) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2013.2247627 · Cited by 189 · see citing works
Finite-Approximation-Error-Based Discrete-Time Iterative Adaptive Dynamic Programming
Wei, Q., Wang, F., Liu, D., & Yang, X. (2014) · IEEE Transactions on Cybernetics · doi:10.1109/tcyb.2014.2354377 · Cited by 188 · see citing works
Liu, D., Li, H., & Wang, D. (2013) · Neurocomputing · doi:10.1016/j.neucom.2012.11.021 · Cited by 188 · see citing works
Dierks, T., Thumati, B. T., & Jagannathan, S. (2009) · Neural Networks · doi:10.1016/j.neunet.2009.06.014 · Cited by 186 · see citing works
Adaptive Critic Designs for Discrete-Time Zero-Sum Games With Application to $H_{\infty}$ Control
Al-Tamimi, A., Abu-Khalaf, M., & Lewis, F. L. (2007) · IEEE Transactions on Systems Man and Cybernetics Part B (Cybernetics) · doi:10.1109/tsmcb.2006.880135 · Cited by 184 · see citing works
Online adaptive algorithm for optimal control with integral reinforcement learning
Vamvoudakis, K. G., Vrabie, D., & Lewis, F. L. (2013) · International Journal of Robust and Nonlinear Control · doi:10.1002/rnc.3018 · Cited by 180 · see citing works
Robust adaptive dynamic programming for linear and nonlinear systems: An overview
Jiang, Z., & Jiang, Y. (2013) · European Journal of Control · doi:10.1016/j.ejcon.2013.05.017 · Cited by 173 · see citing works
Discrete-Time Local Value Iteration Adaptive Dynamic Programming: Convergence Analysis
Wei, Q., Lewis, F. L., Liu, D., Song, R., & Lin, H. (2016) · IEEE Transactions on Systems Man and Cybernetics Systems · doi:10.1109/tsmc.2016.2623766 · Cited by 173 · see citing works
A Self-Learning Call Admission Control Scheme for CDMA Cellular Networks
Liu, D., Zhang, Y., & Zhang, H. (2005) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.2005.853408 · Cited by 162 · see citing works
Generalized Policy Iteration Adaptive Dynamic Programming for Discrete-Time Nonlinear Systems
Liu, D., Wei, Q., & Yan, P. (2015) · IEEE Transactions on Systems Man and Cybernetics Systems · doi:10.1109/tsmc.2015.2417510 · Cited by 157 · see citing works
Jiang, Y., & Jiang, Z. (2012) · IEEE Transactions on Circuits & Systems II Express Briefs · doi:10.1109/tcsii.2012.2213353 · Cited by 155 · see citing works
Wei, Q., Liu, D., Lewis, F. L., Liu, Y., & Zhang, J. (2017) · IEEE Transactions on Industrial Electronics · doi:10.1109/tie.2017.2650872 · Cited by 155 · see citing works
Lin, H., Zhao, B., Liu, D., & Alippi, C. (2020) · IEEE/CAA Journal of Automatica Sinica · doi:10.1109/jas.2020.1003225 · Cited by 155 · see citing works
Dynamic Event-Triggering Neural Learning Control for Partially Unknown Nonlinear Systems
Mu, C., Wang, K., & Qiu, T. (2020) · IEEE Transactions on Cybernetics · doi:10.1109/tcyb.2020.3004493 · Cited by 150 · see citing works
Action dependent heuristic dynamic programming for home energy resource scheduling
Fuselli, D., Angelis, F. D., Boaro, M., Squartini, S., Wei, Q., Liu, D., & Piazza, F. (2013) · International Journal of Electrical Power & Energy Systems · doi:10.1016/j.ijepes.2012.11.023 · Cited by 149 · see citing works
Infinite Horizon Self-Learning Optimal Control of Nonaffine Discrete-Time Nonlinear Systems
Wei, Q., Liu, D., & Yang, X. (2015) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2015.2401334 · Cited by 143 · see citing works
Multiple UAVs path planning algorithms: a comparative study
Sathyaraj, B. M., Jain, L. C., Finn, A., & Drake, S. P. (2008) · Fuzzy Optimization and Decision Making · doi:10.1007/s10700-008-9035-0 · Cited by 141 · see citing works
Luo, B., Wu, H., & Li, H. (2014) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2014.2320744 · Cited by 139 · see citing works
A Novel Iterative $\theta $ -Adaptive Dynamic Programming for Discrete-Time Nonlinear Systems
Wei, Q., & Liu, D. (2013) · IEEE Transactions on Automation Science and Engineering · doi:10.1109/tase.2013.2280974 · Cited by 138 · see citing works
Liu, D., Huang, Y., Wang, D., & Wei, Q. (2013) · International Journal of Control · doi:10.1080/00207179.2013.790562 · Cited by 138 · see citing works
Yang, X., Liu, D., & Huang, Y. (2013) · IET Control Theory and Applications · doi:10.1049/iet-cta.2013.0472 · Cited by 134 · see citing works
Zhang, H., Jiang, H., Luo, C., & Xiao, G. (2016) · IEEE Transactions on Cybernetics · doi:10.1109/tcyb.2016.2611613 · Cited by 134 · see citing works
Zhang, H., Ren, H., Mu, Y., & Han, J. (2021) · IEEE Transactions on Cybernetics · doi:10.1109/tcyb.2021.3090067 · Cited by 134 · see citing works
Fault-tolerant control of a hydraulic servo actuator via adaptive dynamic programming
Stojanović, V. (2023) · Mathematical Modelling and Control · doi:10.3934/mmc.2023016 · Cited by 136 · see citing works
Reinforcement learning solution for HJB equation arising in constrained optimal control problem
Luo, B., Wu, H., Huang, T., & Liu, D. (2015) · Neural Networks · doi:10.1016/j.neunet.2015.08.007 · Cited by 131 · see citing works
A novel adaptive dynamic programming based on tracking error for nonlinear discrete-time systems
Li, C., Ding, J., Lewis, F. L., & Chai, T. (2021) · Automatica · doi:10.1016/j.automatica.2021.109687 · Cited by 128 · see citing works
Lee, J. Y., Park, J. B., & Choi, Y. H. (2014) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2014.2328590 · Cited by 126 · see citing works
Data-Based Optimal Control of Multiagent Systems: A Reinforcement Learning Design Approach
Zhang, J., Wang, Z., & Zhang, H. (2018) · IEEE Transactions on Cybernetics · doi:10.1109/tcyb.2018.2868715 · Cited by 125 · see citing works
From inverse optimal control to inverse reinforcement learning: A historical review
Azar, N. A., Shahmansoorian, A., & Davoudi, M. (2020) · Annual Reviews in Control · doi:10.1016/j.arcontrol.2020.06.001 · Cited by 127 · see citing works
Reinforcement Learning-Based Output Feedback Control of Nonlinear Systems With Input Constraints
He, P., & Jagannathan, S. (2005) · IEEE Transactions on Systems Man and Cybernetics Part B (Cybernetics) · doi:10.1109/tsmcb.2004.840124 · Cited by 124 · see citing works
Wei, Q., Liu, D., & Lewis, F. L. (2015) · Information Sciences · doi:10.1016/j.ins.2015.04.044 · Cited by 122 · see citing works
Wen, G., & Li, B. (2021) · IEEE Transactions on Systems Man and Cybernetics Systems · doi:10.1109/tsmc.2021.3130070 · Cited by 115 · see citing works
Zhang, K., Zhang, H., Mu, Y., & Liu, C. (2020) · IEEE Transactions on Fuzzy Systems · doi:10.1109/tfuzz.2020.2966418 · Cited by 113 · see citing works
Learning-Based Control: A Tutorial and Some Recent Results
Jiang, Z., Bian, T., & Weinan, G. (2020) · Foundations and Trends® in Systems and Control · doi:10.1561/2600000023 · Cited by 136 · see citing works
Qin, C., Zhang, H., & Luo, Y. (2013) · International Journal of Control · doi:10.1080/00207179.2013.863432 · Cited by 111 · see citing works
Issues on Stability of ADP Feedback Controllers for Dynamical Systems
Balakrishnan, S. N., Ding, J., & Lewis, F. L. (2008) · IEEE Transactions on Systems Man and Cybernetics Part B (Cybernetics) · doi:10.1109/tsmcb.2008.926599 · Cited by 109 · see citing works
A self-learning scheme for residential energy system control and management
Huang, T., & Liu, D. (2011) · Neural Computing and Applications · doi:10.1007/s00521-011-0711-6 · Cited by 108 · see citing works
Discrete-Time Optimal Control via Local Policy Iteration Adaptive Dynamic Programming
Wei, Q., Liu, D., Lin, Q., & Song, R. (2016) · IEEE Transactions on Cybernetics · doi:10.1109/tcyb.2016.2586082 · Cited by 107 · see citing works
Djordjevic, V., Tao, H., Song, X., He, S., Gao, W., & Stojanovic, V. (2023) · Mathematical Biosciences and Engineering · doi:10.3934/mbe.2023376 · Cited by 107 · see citing works
Event-Triggered Adaptive Critic Control Design for Discrete-Time Constrained Nonlinear Systems
Ha, M., Wang, D., & Liu, D. (2018) · IEEE Transactions on Systems Man and Cybernetics Systems · doi:10.1109/tsmc.2018.2868510 · Cited by 102 · see citing works
Liu, D., Yang, X., & Li, H. (2012) · Neural Computing and Applications · doi:10.1007/s00521-012-1249-y · Cited by 102 · see citing works
Continuous-Time Distributed Policy Iteration for Multicontroller Nonlinear Systems
Wei, Q., Li, H., Yang, X., & He, H. (2020) · IEEE Transactions on Cybernetics · doi:10.1109/tcyb.2020.2979614 · Cited by 98 · see citing works
Chen, C.-H., Lin, C.-J., & Lin, C. (2009) · IEEE Transactions on Systems Man and Cybernetics Part C (Applications and Reviews) · doi:10.1109/tsmcc.2009.2016572 · Cited by 97 · see citing works
Huang, Y., & Liu, D. (2013) · Neurocomputing · doi:10.1016/j.neucom.2012.07.047 · Cited by 97 · see citing works
An Overview of Research on Adaptive Dynamic Programming
Zhang, H., Zhang, X., Luo, Y., & Jun, Y. A. N. G. (2013) · Acta Automatica Sinica · doi:10.1016/s1874-1029(13)60031-2 · Cited by 96 · see citing works
Adaptive Learning and Control for MIMO System Based on Adaptive Dynamic Programming
Fu, J. Y., He, H., & Zhou, X. (2011) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.2011.2147797 · Cited by 96 · see citing works
Continuous-Time Q-Learning for Infinite-Horizon Discounted Cost Linear Quadratic Regulator Problems
Muthukumar, P., Modares, H., Lewis, F. L., & Aurangzeb, M. (2015) · IEEE Transactions on Cybernetics · doi:10.1109/tcyb.2014.2322116 · Cited by 97 · see citing works
Approximate $N$ -Player Nonzero-Sum Game Solution for an Uncertain Continuous Nonlinear System
Johnson, M., Kamalapurkar, R., Bhasin, S., & Dixon, W. E. (2014) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2014.2350835 · Cited by 95 · see citing works
- Authors
- Murray, J. J., Cox, C. J., Lendaris, G. G., & Saeks, R.
- Affiliations
- Stony Brook University (US)
- Accurate Automation (United States) (US)
- Portland State University (US)
- ORCID
- Murray, J.J.
- Venue
- IEEE Transactions on Systems, Man and Cybernetics, Part C (Applications and Reviews)
- Published
- 2002-05
- Volume
- 32
- Issue
- 2
- Pages
- 140-153
- ISSN
- 1094-6977 (ISSN-L)
- OpenAlex
- W2160561608
- Cited by
- 677
- References
- 32
- Citations by year
- 2026: 16 · 2025: 36 · 2024: 31 · 2023: 26 · 2022: 34 · 2021: 34 · 2020: 39 · 2019: 28 · 2018: 27 · 2017: 42 · 2016: 36 · 2015: 40 · 2014: 42 · 2013: 43 · 2012: 45
- FWCI
- 4.99
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- IEEE Transactions on Systems Man and Cybernetics Part C (Applications and Reviews) (publishedVersion)
- PDXScholar (Portland State University) (submittedVersion)
- Topics
- Physical Sciences › Computer Science › Computational Theory and Mathematics › Adaptive Dynamic Programming Control
- Physical Sciences › Engineering › Control and Systems Engineering › Advanced Control Systems Optimization
- Physical Sciences › Computer Science › Artificial Intelligence › Reinforcement Learning in Robotics
- Keywords
- adaptive dynamic programming, optimal control, Hamilton-Jacobi-Bellman equation, stability analysis, convergence analysis, unknown dynamics, nonlinear systems, input-affine systems, radial basis function approximation, flight control, soft computing
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
