Learning-Based Adaptive Optimal Tracking Control of Strict-Feedback Nonlinear Systems
Abstract
This paper proposes a novel data-driven control approach to address the problem of adaptive optimal tracking for a class of nonlinear systems taking the strict-feedback form. Adaptive dynamic programming (ADP) and nonlinear output regulation theories are integrated for the first time to compute an adaptive near-optimal tracker without any a priori knowledge of the system dynamics. Fundamentally different from adaptive optimal stabilization problems, the solution to a Hamilton-Jacobi-Bellman (HJB) equation, not necessarily a positive definite function, cannot be approximated through the existing iterative methods. This paper proposes a novel policy iteration technique for solving positive semidefinite HJB equations with rigorous convergence analysis. A two-phase data-driven learning method is developed and implemented online by ADP. The efficacy of the proposed adaptive optimal tracking control methodology is demonstrated via a Van der Pol oscillator with time-varying exogenous signals.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
ADP is an effective way to achieve high performance of the optimal controller and relies on adaptive control, optimal control and reinforcement learning.
“Adaptive dynamic programming (ADP) ensures an e ective way to achieve high performance of the optimal controller which relies on adaptive control, optimal control and reinforcement learning [9, 23-26].”
From Data-driven control of hydraulic servo actuator: An event-triggered adaptive dynamic programming approach (2023) · cites this paper as [25]
Cites
Nonlinear and adaptive control design
Krstić, M., Kokotović, P. V., & Kanellakopoulos, I. (1995) · Cited by 10,429 · see citing works
Lewis, F. L., Vrabie, D. L., & Syrmos, V. L. (2012) · doi:10.1002/9781118122631 · Cited by 2,020 · see citing works
Online actor–critic algorithm to solve the continuous-time infinite horizon optimal control problem
Vamvoudakis, K. G., & Lewis, F. L. (2010) · Automatica · doi:10.1016/j.automatica.2010.02.018 · Cited by 1,722 · see citing works
Approximate Dynamic Programming
Powell, W. B. (2007) · Wiley series in probability and statistics · doi:10.1002/9780470182963 · Cited by 1,582 · see citing works
Output regulation of nonlinear systems
Isidori, A., & Byrnes, C. I. (1990) · IEEE Transactions on Automatic Control · doi:10.1109/9.45168 · Cited by 1,575 · see citing works
Reinforcement learning and adaptive dynamic programming for feedback control
Lewis, F. L., & Vrabie, D. (2009) · IEEE Circuits and Systems Magazine · doi:10.1109/mcas.2009.933854 · Cited by 1,569 · see citing works
Huang, J. (2004) · Society for Industrial and Applied Mathematics eBooks · doi:10.1137/1.9780898718683 · Cited by 1,422 · see citing works
Lavretsky, E., & Wise, K. A. (2012) · Advanced textbooks in control and signal processing · doi:10.1007/978-1-4471-4396-3_11 · Cited by 1,002 · see citing works
(2010) · Springer eBooks · doi:10.1007/978-0-387-30164-8_588 · Cited by 978 · see citing works
Adaptive optimal control for continuous-time linear systems based on policy iteration
Vrabie, D., Pastravanu, O. C., Abu-Khalaf, M., & Lewis, F. L. (2008) · Automatica · doi:10.1016/j.automatica.2008.08.017 · Cited by 953 · see citing works
Approximation Theory and Methods
Powell, M. J. D. (1981) · Cambridge University Press eBooks · doi:10.1017/cbo9781139171502 · Cited by 776 · see citing works
Zhang, H., Cui, L., Zhang, X., & Luo, Y. (2011) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.2011.2168538 · Cited by 645 · see citing works
Modares, H., & Lewis, F. L. (2014) · IEEE Transactions on Automatic Control · doi:10.1109/tac.2014.2317301 · Cited by 451 · see citing works
Adaptive Non-linear Tracking with Complete Compensation of Unknown Disturbances
Олегович, Н. В. (1998) · European Journal of Control · doi:10.1016/s0947-3580(98)70107-4 · Cited by 436 · see citing works
Adaptive Dynamic Programming and Adaptive Optimal Output Regulation of Linear Systems
Gao, W., & Jiang, Z. (2016) · IEEE Transactions on Automatic Control · doi:10.1109/tac.2016.2548662 · Cited by 418 · see citing works
Self-citationAn Approximation Theory of Optimal Control for Trainable Manipulators
Saridis, G. N., & Lee, C.-S. G. (1979) · IEEE Transactions on Systems Man and Cybernetics · doi:10.1109/tsmc.1979.4310171 · Cited by 349 · see citing works
Value iteration and adaptive dynamic programming for data-driven adaptive optimal control design
Bian, T., & Jiang, Z.-P. (2016) · Automatica · doi:10.1016/j.automatica.2016.05.003 · Cited by 348 · see citing works
Self-citationInverse optimal design of input-to-state stabilizing nonlinear controllers
Krstić, M., & Li, Z.-H. (1998) · IEEE Transactions on Automatic Control · doi:10.1109/9.661589 · Cited by 344 · see citing works
Model-Free Optimal Tracking Control via Critic-Only Q-Learning
Luo, B., Liu, D., Huang, T., & Wang, D. (2016) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2016.2585520 · Cited by 332 · see citing works
Structurally stable output regulation of nonlinear systems
Byrnes, C. I., Priscoli, F. D., Isidori, A., et al. (1997) · Automatica · doi:10.1016/s0005-1098(96)00184-7 · Cited by 317 · see citing works
Robust Adaptive Dynamic Programming
Jiang, Y., & Jiang, Z. (2017) · Wiley eBooks · doi:10.1002/9781119132677 · Cited by 250 · see citing works
Self-citationGlobal Adaptive Dynamic Programming for Continuous-Time Nonlinear Systems
Jiang, Y., & Jiang, Z.-P. (2015) · IEEE Transactions on Automatic Control · doi:10.1109/tac.2015.2414811 · Cited by 244 · see citing works
Self-citation1 Global Adaptive Dynamic Programming for Continuous-Time Nonlinear SystemsPreprint
Jiang, Y., Member, S., & Jiang, Z. (2016) · Cited by 235 · see citing works
Self-citationGao, W., Jiang, Y., Jiang, Z.-P., & Chai, T. (2016) · Automatica · doi:10.1016/j.automatica.2016.05.008 · Cited by 234 · see citing works
Self-citationApproximate optimal trajectory tracking for continuous-time nonlinear systems
Kamalapurkar, R., Dinh, H. T., Bhasin, S., et al. (2014) · Automatica · doi:10.1016/j.automatica.2014.10.103 · Cited by 233 · see citing works
Fan, Q., & Yang, G. (2015) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2015.2472974 · Cited by 229 · see citing works
Adaptive Learning in Tracking Control Based on the Dual Critic Network Design
Ni, Z., He, H., & Wen, J. (2013) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2013.2247627 · Cited by 189 · see citing works
Data-Driven Adaptive Optimal Control of Connected Vehicles
Gao, W., Jiang, Z., & Özbay, K. M. A. (2016) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2016.2597279 · Cited by 175 · see citing works
Self-citationRobust adaptive dynamic programming for linear and nonlinear systems: An overview
Jiang, Z., & Jiang, Y. (2013) · European Journal of Control · doi:10.1016/j.ejcon.2013.05.017 · Cited by 173 · see citing works
Self-citationOptimal Control of Nonlinear Continuous-Time Systems in Strict-Feedback Form
Zargarzadeh, H., Dierks, T., & Jagannathan, S. (2015) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2015.2441712 · Cited by 168 · see citing works
Optimal Nonlinear Tracking of Spacecraft Attitude Maneuvers
Sharma, R., & Tewari, A. (2004) · IEEE Transactions on Control Systems Technology · doi:10.1109/tcst.2004.825060 · Cited by 162 · see citing works
Wang, D., Liu, D., Li, H., et al. (2015) · IEEE Transactions on Systems Man and Cybernetics Systems · doi:10.1109/tsmc.2015.2466191 · Cited by 160 · see citing works
Wang, L., Li, H., Zhou, Q. F., & Lu, R. (2017) · IEEE Transactions on Cybernetics · doi:10.1109/tcyb.2017.2684131 · Cited by 154 · see citing works
Adaptive continuous-time linear quadratic Gaussian control
Duncan, T. E., Guo, L., & Pasik-Duncan, B. (1999) · IEEE Transactions on Automatic Control · doi:10.1109/9.788532 · Cited by 124 · see citing works
The Construction of Optimal Linear and Nonlinear Regulators
Krener, A. J. (1992) · Birkhäuser Boston eBooks · doi:10.1007/978-1-4757-2204-8_21 · Cited by 119 · see citing works
Zhu, Y., Zhao, D., & Li, X. (2016) · IET Control Theory and Applications · doi:10.1049/iet-cta.2015.0769 · Cited by 88 · see citing works
Optimal Tracking Control of Motion Systems
Mannava, A., Balakrishnan, S. N., Tang, L. X., et al. (2011) · IEEE Transactions on Control Systems Technology · doi:10.1109/tcst.2011.2168608 · Cited by 56 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
Djordjevic, V., Tao, H., Song, X., He, S., Gao, W., & Stojanovic, V. (2023) · Mathematical Biosciences and Engineering · doi:10.3934/mbe.2023376 · Cited by 107 · see citing works
Self-citation
- Authors
- Gao, W., & Jiang, Z.-P.
- Affiliations
- Georgia Southern University (US)
- New York University (US)
- Venue
- IEEE Transactions on Neural Networks and Learning Systems
- Published
- 2018-06
- Volume
- 29
- Issue
- 6
- Pages
- 2614-2624
- ISSN
- 2162-237X (ISSN-L)
- 2162-2388
- OpenAlex
- W2767307332
- Cited by
- 176
- References
- 37
- Citations by year
- 2026: 16 · 2025: 15 · 2024: 21 · 2023: 24 · 2022: 22 · 2021: 19 · 2020: 21 · 2019: 21 · 2018: 16 · 2017: 1
- FWCI
- 12.12
- Type
- journal article
- Language
- en
- Open access
- closed
- Licence
- Locations
- IEEE Transactions on Neural Networks and Learning Systems (publishedVersion)
- PubMed (publishedVersion)
- Funders
- U.S. National Science Foundation (ECCS-1501044)
- Topics
- Physical Sciences › Computer Science › Computational Theory and Mathematics › Adaptive Dynamic Programming Control
- Physical Sciences › Engineering › Control and Systems Engineering › Adaptive Control of Nonlinear Systems
- Physical Sciences › Engineering › Control and Systems Engineering › Iterative Learning Control Systems
- Keywords
- strict-feedback nonlinear systems, adaptive dynamic programming, nonlinear output regulation, Hamilton-Jacobi-Bellman equation, policy iteration, data-driven control, Van der Pol oscillator, online learning
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
