The DOI matches its Crossref record; 32 of the 32 references it cites were found in a registry or an index.
Adaptive dynamic programming
Abstract
Unlike the many soft computing applications where it suffices to achieve a "good approximation most of the time," a control system must be stable all of the time. As such, if one desires to learn a control law in real-time, a fusion of soft computing techniques to learn the appropriate control law with hard computing techniques to maintain the stability constraint and guarantee convergence is required. The objective of the paper is to describe an adaptive dynamic programming algorithm (ADPA) which fuses soft computing techniques to learn the optimal cost (or return) functional for a stabilizable nonlinear system with unknown dynamics and hard computing techniques to verify the stability and convergence of the algorithm. Specifically, the algorithm is initialized with a (stabilizing) cost functional and the system is run with the corresponding control law (defined by the Hamilton-Jacobi-Bellman equation), with the resultant state trajectories used to update the cost functional in a soft computing mode. Hard computing techniques are then used to show that this process is globally convergent with stepwise stability to the optimal cost functional/control law pair for an (unknown) input affine system with an input quadratic performance measure (modulo the appropriate technical conditions). Three specific implementations of the ADPA are developed for 1) the linear case, 2) for the nonlinear case using a locally quadratic approximation to the cost functional, and 3) the nonlinear case using a radial basis function approximation of the cost functional; illustrated by applications to flight control.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Adaptive dynamic programming (ADP) ensures an e ective way to achieve high performance of the optimal controller which relies on adaptive control, optimal control and reinforcement learning [9, 23-26].”
From Data-driven control of hydraulic servo actuator: An event-triggered adaptive dynamic programming approach (2023) · cites this paper as [23]
Citations
Cites
Bellman, R. E. (2010) · doi:10.1515/9781400835386 · Cited by 14,588
Linear Optimal Control Systems
Kwakernaak, H., Sivan, R., & Tyreus, B. N. D. (1974) · Journal of Dynamic Systems Measurement and Control · doi:10.1115/1.3426828 · Cited by 5,010
Neuronlike adaptive elements that can solve difficult learning control problems
Barto, A. G., Sutton, R. S., & Anderson, C. W. (1983) · IEEE Transactions on Systems Man and Cybernetics · doi:10.1109/tsmc.1983.6313077 · Cited by 3,325
Stable adaptive fuzzy control of nonlinear systems
Wang, L.-X. (1993) · IEEE Transactions on Fuzzy Systems · doi:10.1109/91.227383 · Cited by 1,622
Dynamic Programming: Deterministic and Stochastic Models
Bertsekas, D. P. (1987) · Medical Entomology and Zoology · Cited by 1,348
Introduction to dynamic systems: theory, models and applications
(1981) · Mathematics and Computers in Simulation · doi:10.1016/0378-4754(81)90069-0 · Cited by 1,136
Gaussian Networks for Direct Adaptive Control
Sanner, R. M., & Slotine, J.-J. E. (1991) · Proceedings of the ... American Control Conference/Proceedings of the American Control Conference · doi:10.23919/acc.1991.4791778 · Cited by 1,079
Approximate dynamic programming for real-time control and neural modeling
Werbos, P. J. (1992) · Medical Entomology and Zoology · Cited by 971
Introduction to dynamic systems
Williams, M. (2016) · Structural Dynamics · doi:10.1201/9781315375311-5 · Cited by 568
Adaptive neural network control of nonlinear systems by state and output feedback
Ge, S. S., Hang, C. C., & Zhang, T. (1999) · IEEE Transactions on Systems Man and Cybernetics Part B (Cybernetics) · doi:10.1109/3477.809035 · Cited by 419
Ray, W. D., & Barnett, S. M. (1972) · Operational Research Quarterly (1970-1977) · doi:10.2307/3007980 · Cited by 312
Stable adaptive fuzzy controllers with application to inverted pendulum tracking
Wang, L.-X. (1996) · IEEE Transactions on Systems Man and Cybernetics Part B (Cybernetics) · doi:10.1109/3477.537311 · Cited by 263
Direct Adaptive and Neural Control of Wing-Rock Motion of Slender Delta Wings
Singh, S. N., Yirn, W., & Wells, W. R. (1995) · Journal of Guidance Control and Dynamics · doi:10.2514/3.56652 · Cited by 144
Stability of fuzzy linguistic control systems
Langari, G., & Tomizuka, M. (1990) · Proceedings of the IEEE Conference on Decision & Control, including the Symposium on Adaptive Processes · doi:10.1109/cdc.1990.204014 · Cited by 142
Coaker, P. B., Bellman, R., & Dreyfus, S. E. (1964) · Journal of the Operational Research Society · doi:10.1057/jors.1964.34 · Cited by 126
Variable neural networks for adaptive control of nonlinear systems
Liu, S., Kadirkamanathan, V., & Billings, S. A. (1999) · IEEE Transactions on Systems Man and Cybernetics Part C (Applications and Reviews) · doi:10.1109/5326.740668 · Cited by 108
Foundations of Mathematical Analysis
Ponnusamy, S. (2011) · Birkhäuser Boston eBooks · doi:10.1007/978-0-8176-8292-7 · Cited by 92
Foundations of mathematical analysis
(2012) · Choice Reviews Online · doi:10.5860/choice.49-6937 · Cited by 71
Applications of Liapunov Methods in Stability
Halanay, A., & Rǎsvan, V. B. (1993) · Springer eBooks · doi:10.1007/978-94-011-1600-8 · Cited by 53
On the design of a neural network autolander
Cox, C. J., Stepniewski, S. W., Jorgensen, C. C., Saeks, R. E., & Lewis, C. (1999) · International Journal of Robust and Nonlinear Control · doi:10.1002/(sici)1099-1239(19991215)9:14 · Cited by 43
Self-citationAdaptive critic design for intelligent steering and speed control of a 2-axle vehicle
Lendaris, G. G., Schultz, L., & Shannon, T. T. (2000) · doi:10.1109/ijcnn.2000.861283 · Cited by 43
Self-citationAdaptive critic design in learning to play game of Go
Zaman, R., Prokhorov, D. V., & Wunsch, D. C. (2002) · doi:10.1109/icnn.1997.611623 · Cited by 20
Development and Flight Test of the X-43A-LS Hypersonic Configuration UAV
Gibson, C., Neidhoefer, J. C., Cooper, S., Carlton, L. A., Cox, C. J., & Jorgensen, C. C. (2002) · 1st UAV Conference · doi:10.2514/6.2002-3462 · Cited by 20
Self-citationFuzzy control stabilization with applications to motorcycle control
Wu, J. C., & Liu, T. S. (1996) · IEEE Transactions on Systems Man and Cybernetics Part B (Cybernetics) · doi:10.1109/3477.544297 · Cited by 15
Adaptive critic control and functional link neural networks
Cox, C. J., & Saeks, R. E. (2002) · Conference proceedings/Conference proceedings - IEEE International Conference on Systems, Man, and Cybernetics · doi:10.1109/icsmc.1998.728126 · Cited by 14
Self-citationLoFLYTE - A neurocontrols testbed
Saeks, R. E., Cox, C. J., & Pap, R. M. (1997) · AIAA Aerospace Sciences Meeting and Exhibit · doi:10.2514/6.1997-85 · Cited by 11
Self-citationProkhorov, D. V., & Feldkamp, L. A. (2002) · doi:10.1109/icnn.1997.614396 · Cited by 10
Neural adaptive control of LoFLYTE(R)
Cox, C. J., Neidhoefer, J. C., Saeks, R. E., & Lendaris, G. C. (2001) · Proceedings of the ... American Control Conference/Proceedings of the American Control Conference · doi:10.1109/acc.2001.946345 · Cited by 8
Self-citationPap, R. M., Saeks, R. E., Lewis, C., Carlton, A. J., Cox, C. J., Kocher, M., & Kersting, T. (1998) · AIAA International Space Planes and Hypersonic Systems and Technologies Conference · doi:10.2514/6.1998-1519 · Cited by 7
Self-citationA neurocontroller for robotic applications
Cox, C. J., Saeks, R. E., Lothers, M., Pap, R. M., & Thomas, C. (2003) · Conference proceedings/Conference proceedings - IEEE International Conference on Systems, Man, and Cybernetics · doi:10.1109/icsmc.1992.271544 · Cited by 6
Self-citationDevinatz, A., & Kaplan, J. L. (1972) · Indiana University Mathematics Journal · doi:10.1512/iumj.1973.22.22029 · Cited by 6
An improvement in the MacFarlane-Potter method for solving the algebraic Riccati equation
Holley, W. E., & Wei, S. Y. (1979) · IEEE Transactions on Automatic Control · doi:10.1109/jacc.1979.4170724 · Cited by 3
Research works printed but not linked: 4
- 2001, Jour of Spacecraft and Rockets Special Section on Hyper-X, 38, 801
- sitz, 1998, HYPER-X Hypersonic experimental research vehicle
- wang, 1996, stable adaptive fuzzy control of nonlinear systems, IEEE Trans Syst Man Cybern B Cybern, 26, 677
- barnett, 1971, The Matrices of Control Theory
Cited by
Sorted by citations
Online actor–critic algorithm to solve the continuous-time infinite horizon optimal control problem
Vamvoudakis, K. G., & Lewis, F. L. (2010) · Automatica · doi:10.1016/j.automatica.2010.02.018 · Cited by 1,722
Reinforcement learning and adaptive dynamic programming for feedback control
Lewis, F. L., & Vrabie, D. (2009) · IEEE Circuits and Systems Magazine · doi:10.1109/mcas.2009.933854 · Cited by 1,591
Abu-Khalaf, M., & Lewis, F. L. (2005) · Automatica · doi:10.1016/j.automatica.2004.11.034 · Cited by 1,350
Lewis, F. L., Vrabie, D., & Vamvoudakis, K. G. (2012) · IEEE Control Systems · doi:10.1109/mcs.2012.2214134 · Cited by 1,122
Jiang, Y., & Jiang, Z. (2012) · Automatica · doi:10.1016/j.automatica.2012.06.096 · Cited by 1,079
Metadata
- Authors
- Murray, J. J., Cox, C. J., Lendaris, G. G., & Saeks, R.
- Affiliations
- Stony Brook University (US)
- Accurate Automation (United States) (US)
- Portland State University (US)
- ORCID
- Murray, J.J.
- Venue
- IEEE Transactions on Systems, Man and Cybernetics, Part C (Applications and Reviews)
- Published
- 2002-05
- Volume
- 32
- Issue
- 2
- Pages
- 140-153
- ISSN
- 1094-6977 (ISSN-L)
- OpenAlex
- W2160561608
- Cited by
- 677
- References
- 32
- Citations by year
- 2026: 16 · 2025: 36 · 2024: 31 · 2023: 26 · 2022: 34 · 2021: 34 · 2020: 39 · 2019: 28 · 2018: 27 · 2017: 42 · 2016: 36 · 2015: 40 · 2014: 42 · 2013: 43 · 2012: 45
- FWCI
- 4.99
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- IEEE Transactions on Systems Man and Cybernetics Part C (Applications and Reviews) (publishedVersion)
- PDXScholar (Portland State University) (submittedVersion)
- Topics
- Physical Sciences › Engineering › Control and Systems Engineering › Adaptive Dynamic Programming Control
- Physical Sciences › Engineering › Control and Systems Engineering › Advanced Control Systems Optimization
- Physical Sciences › Computer Science › Artificial Intelligence › Reinforcement Learning in Robotics
- Keywords
- adaptive dynamic programming, optimal control, Hamilton-Jacobi-Bellman equation, stability analysis, convergence analysis, unknown dynamics, nonlinear systems, input-affine systems, radial basis function approximation, flight control, soft computing
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
