The DOI matches its Crossref record; 62 of the 62 references it cites were found in a registry or an index.
Value Iteration Adaptive Dynamic Programming for Optimal Control of Discrete-Time Nonlinear Systems
Abstract
In this paper, a value iteration adaptive dynamic programming (ADP) algorithm is developed to solve infinite horizon undiscounted optimal control problems for discrete-time nonlinear systems. The present value iteration ADP algorithm permits an arbitrary positive semi-definite function to initialize the algorithm. A novel convergence analysis is developed to guarantee that the iterative value function converges to the optimal performance index function. Initialized by different initial functions, it is proven that the iterative value function will be monotonically nonincreasing, monotonically nondecreasing, or nonmonotonic and will converge to the optimum. In this paper, for the first time, the admissibility properties of the iterative control laws are developed for value iteration algorithms. It is emphasized that new termination criteria are established to guarantee the effectiveness of the iterative control laws. Neural networks are used to approximate the iterative value function and compute the iterative control law, respectively, for facilitating the implementation of the iterative ADP algorithm. Finally, two simulation examples are given to illustrate the performance of the present method.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Reinforcement Learning: An Introduction
Sutton, R. S., & Barto, A. G. (1998) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.1998.712192 · Cited by 27,225
(1995) · Cited by 10,558
A Review on Basic Data-Driven Approaches for Industrial Process Monitoring
Yin, S., Ding, S. X., Xie, X., & Luo, H. (2014) · IEEE Transactions on Industrial Electronics · doi:10.1109/tie.2014.2301773 · Cited by 1,716
Approximate Dynamic Programming: Solving the Curses of Dimensionality
Powell, W. B. (2007) · Wiley Series in Probability and Statistics · doi:10.1002/9780470182963 · Cited by 1,566
Reinforcement learning and adaptive dynamic programming for feedback control
Lewis, F. L., & Vrabie, D. (2009) · IEEE Circuits and Systems Magazine · doi:10.1109/mcas.2009.933854 · Cited by 1,558
Prokhorov, D. V., & Wunsch, D. C. (1997) · IEEE Transactions on Neural Networks · doi:10.1109/72.623201 · Cited by 1,186
(2012) · IEEE Control Systems · doi:10.1109/mcs.2012.2214134 · Cited by 1,105
Discrete-Time Nonlinear HJB Solution Using Approximate Dynamic Programming: Convergence Proof
Al-Tamimi, A., Lewis, F. L., & Abu-Khalaf, M. (2008) · IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics) · doi:10.1109/tsmcb.2008.926614 · Cited by 1,075
Data-based techniques focused on modern industry: An overview
Yin, S., Li, X., Gao, H., & Kaynak, O. (2015) · IEEE Transactions on Industrial Electronics · doi:10.1109/tie.2014.2308133 · Cited by 1,025
(2011) · Encyclopedia of Machine Learning · doi:10.1007/978-0-387-30164-8_588 · Cited by 950
Reinforcement Learning and Dynamic Programming Using Function Approximators
Busoniu, L., Babuska, R., De Schutter, B., & Ernst, D. (2017) · CRC Press eBooks · doi:10.1201/9781439821091 · Cited by 919
Online learning control by association and reinforcement
Si, J., & Wang, Y.-T. (2001) · IEEE Transactions on Neural Networks · doi:10.1109/72.914523 · Cited by 773
Policy Iteration Adaptive Dynamic Programming Algorithm for Discrete-Time Nonlinear Systems
Liu, D., & Wei, Q. (2014) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2013.2281663 · Cited by 753
Self-citationModares, H., & Lewis, F. L. (2014) · Automatica · doi:10.1016/j.automatica.2014.05.011 · Cited by 701
Murray, J. J., Cox, C. J., Lendaris, G. G., & Saeks, R. (2002) · IEEE Transactions on Systems, Man and Cybernetics, Part C (Applications and Reviews) · doi:10.1109/tsmcc.2002.801727 · Cited by 677
Zhang, H., Luo, Y., & Liu, D. (2009) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.2009.2027233 · Cited by 649
Self-citationBhasin, S., Kamalapurkar, R., Johnson, M., Vamvoudakis, K. G., Lewis, F. L., & Dixon, W. E. (2013) · Automatica · doi:10.1016/j.automatica.2012.09.019 · Cited by 642
A Menu of Designs for Reinforcement Learning Over Time
Werbos, P. J. (1991) · Neural Networks for Control · doi:10.7551/mitpress/4939.003.0007 · Cited by 605
Modares, H., Lewis, F. L., & Naghibi-Sistani, M.-B. (2014) · Automatica · doi:10.1016/j.automatica.2013.09.043 · Cited by 570
Improved PLS Focused on Key-Performance-Indicator-Related Fault Diagnosis
Yin, S., Zhu, X., & Kaynak, O. (2015) · IEEE Transactions on Industrial Electronics · doi:10.1109/tie.2014.2345331 · Cited by 537
Zhang, H., Wei, Q., & Luo, Y. (2008) · IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics) · doi:10.1109/tsmcb.2008.920269 · Cited by 489
Self-citationLiu, D., Yang, X., Wang, D., & Wei, Q. (2015) · IEEE Transactions on Cybernetics · doi:10.1109/tcyb.2015.2417170 · Cited by 440
Self-citationRobust Adaptive Dynamic Programming and Feedback Stabilization of Nonlinear Systems
Jiang, Y., & Jiang, Z.-P. (2014) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2013.2294968 · Cited by 414
Zhang, H., Wei, Q., & Liu, D. (2011) · Automatica · doi:10.1016/j.automatica.2010.10.033 · Cited by 409
Self-citationLincoln, B., & Rantzer, A. (2006) · IEEE Transactions on Automatic Control · doi:10.1109/tac.2006.878720 · Cited by 358
Liu, D., Wang, D., Wang, F.-Y., Li, H., & Yang, X. (2014) · IEEE Transactions on Cybernetics · doi:10.1109/tcyb.2014.2357896 · Cited by 310
Self-citationYin, S., & Huang, Z. (2015) · IEEE/ASME Transactions on Mechatronics · doi:10.1109/tmech.2014.2358674 · Cited by 308
Finite-Approximation-Error-Based Optimal Control Approach for Discrete-Time Nonlinear Systems
Liu, D., & Wei, Q. (2013) · IEEE Transactions on Cybernetics · doi:10.1109/tsmcb.2012.2216523 · Cited by 304
Self-citationWei, Q., Song, R., & Yan, P. (2016) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2015.2464080 · Cited by 252
Self-citationOff-Policy Actor-Critic Structure for Optimal Control of Unknown Systems With Disturbances
Song, R., Lewis, F. L., Wei, Q., & Zhang, H. (2016) · IEEE Transactions on Cybernetics · doi:10.1109/tcyb.2015.2421338 · Cited by 239
Self-citationZhang, H., Qin, C., Jiang, B., & Luo, Y. (2014) · IEEE Transactions on Cybernetics · doi:10.1109/tcyb.2014.2313915 · Cited by 234
Wei, Q., Liu, D., & Shi, G. (2014) · IEEE Transactions on Industrial Electronics · doi:10.1109/tie.2014.2361485 · Cited by 230
Self-citationWei, Q., & Liu, D. (2014) · IEEE Transactions on Automation Science and Engineering · doi:10.1109/tase.2013.2284545 · Cited by 224
Self-citationZhang, H., Song, R., Wei, Q., & Zhang, T. (2011) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.2011.2172628 · Cited by 191
Self-citationFinite-Approximation-Error-Based Discrete-Time Iterative Adaptive Dynamic Programming
Wei, Q., Wang, F.-Y., Liu, D., & Yang, X. (2014) · IEEE Transactions on Cybernetics · doi:10.1109/tcyb.2014.2354377 · Cited by 188
Self-citationWei, Q., & Liu, D. (2014) · IEEE Transactions on Industrial Electronics · doi:10.1109/tie.2014.2301770 · Cited by 179
Self-citationRobust PLS approach for KPI-related prediction and diagnosis against outliers and missing data
Yin, S., Wang, G., & Yang, X. (2014) · International Journal of Systems Science · doi:10.1080/00207721.2014.886136 · Cited by 176
Power System Stability Control for a Wind Farm Based on Adaptive Dynamic Programming
Tang, Y., He, H., Wen, J., & Liu, J. (2015) · IEEE Transactions on Smart Grid · doi:10.1109/tsg.2014.2346740 · Cited by 168
Distributed Robust Synchronization of Dynamical Networks With Stochastic Coupling
Tang, Y., Gao, H., & Kurths, J. (2014) · IEEE Transactions on Circuits and Systems I: Regular Papers · doi:10.1109/tcsi.2013.2285699 · Cited by 168
Zhong, X., He, H., Zhang, H., & Wang, Z. (2014) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2014.2305841 · Cited by 153
Stability of Dynamical Systems
Liao, X., Wang, L., & Yu, P. (2007) · Monograph series on nonlinear science and complexity · doi:10.1016/s1574-6917(07)05001-5 · Cited by 153
Adaptive Dynamic Programming for a Class of Complex-Valued Nonlinear Systems
Song, R., Xiao, W., Zhang, H., & Sun, C. (2014) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2014.2306201 · Cited by 143
Infinite Horizon Self-Learning Optimal Control of Nonaffine Discrete-Time Nonlinear Systems
Wei, Q., Liu, D., & Yang, X. (2015) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2015.2401334 · Cited by 142
Self-citationMultiple Actor-Critic Structures for Continuous-Time Optimal Control Using Input-Output Data
Song, R., Lewis, F. L., Wei, Q., Zhang, H., Jiang, Z., & Levine, D. S. (2015) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2015.2399020 · Cited by 140
Self-citationLee, J. M., & Lee, J. H. (2005) · Automatica · doi:10.1016/j.automatica.2005.02.006 · Cited by 139
A Novel Iterative $\theta $ -Adaptive Dynamic Programming for Discrete-Time Nonlinear Systems
Wei, Q., & Liu, D. (2013) · IEEE Transactions on Automation Science and Engineering · doi:10.1109/tase.2013.2280974 · Cited by 138
Self-citationRevisiting Approximate Dynamic Programming and its Convergence
Heydari, A. (2014) · IEEE Transactions on Cybernetics · doi:10.1109/tcyb.2014.2314612 · Cited by 133
Wei, Q., Liu, D., Shi, G., & Liu, Y. (2015) · IEEE Transactions on Industrial Electronics · doi:10.1109/tie.2014.2388198 · Cited by 128
Self-citationWei, Q., Liu, D., & Lewis, F. L. (2015) · Information Sciences · doi:10.1016/j.ins.2015.04.044 · Cited by 122
Self-citationOptimal Control of Epidemic Information Dissemination Over Networks
Chen, P., Cheng, S., & Chen, K. (2014) · IEEE Transactions on Cybernetics · doi:10.1109/tcyb.2014.2306781 · Cited by 102
Wei, Q., Zhang, H., & Dai, J. (2008) · Neurocomputing · doi:10.1016/j.neucom.2008.05.012 · Cited by 101
Self-citationContinuous-Time Q-Learning for Infinite-Horizon Discounted Cost Linear Quadratic Regulator Problems
Muthukumar, P., Modares, H., Lewis, F. L., & Aurangzeb, M. (2015) · IEEE Transactions on Cybernetics · doi:10.1109/tcyb.2014.2322116 · Cited by 97
Wei, Q., & Liu, D. (2012) · Neural Networks · doi:10.1016/j.neunet.2012.02.027 · Cited by 87
Self-citationNumerical adaptive learning control scheme for discrete‐time non‐linear systems
Wei, Q., & Liu, D. (2013) · IET Control Theory and Applications · doi:10.1049/iet-cta.2012.0486 · Cited by 73
Self-citationLiu, D., Li, H., & Wang, D. (2015) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2015.2402203 · Cited by 72
Self-citationMehraeen, S., Dierks, T., Jagannathan, S., & Crow, M. L. (2012) · IEEE Transactions on Cybernetics · doi:10.1109/tsmcb.2012.2227253 · Cited by 61
Efficient Video Stitching Based on Fast Structure Deformation
Li, J., Xu, W., Zhang, J., Zhang, M., Wang, Z., & Li, X. (2014) · IEEE Transactions on Cybernetics · doi:10.1109/tcyb.2014.2381774 · Cited by 41
Wei, Q., Wang, D., & Zhang, D. (2012) · Neural Computing and Applications · doi:10.1007/s00521-012-1188-7 · Cited by 39
Self-citationNonlinear neuro-optimal tracking control via stable iterative Q-learning algorithm
Wei, Q., Song, R., & Sun, Q. (2015) · Neurocomputing · doi:10.1016/j.neucom.2015.05.075 · Cited by 34
Self-citationHuman Body Segmentation via Data-Driven Graph Cut
Li, S., Lu, H., & Shao, X. (2014) · IEEE Transactions on Cybernetics · doi:10.1109/tcyb.2014.2301193 · Cited by 25
Clipping in Neurocontrol by Adaptive Dynamic Programming
Fairbank, M., Prokhorov, D. V., & Alonso, E. (2014) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2014.2297991 · Cited by 11
Off-Policy Actor-Critic Structure for Optimal Control of Unknown Systems with Disturbances
Song, R., Wei, Q., & Li, Q. (2018) · Studies in systems, decision and control · doi:10.1007/978-981-13-1712-5_9 · Cited by 2
Self-citation
Cited by
Adaptive Dynamic Programming for Control: A Survey and Recent Advances
Liu, D., Xue, S., Zhao, B., Luo, B., & Wei, Q. (2021) · IEEE Transactions on Systems, Man, and Cybernetics: Systems · doi:10.1109/tsmc.2020.3042876 · Cited by 708
Self-citationWang, D., Gao, N., Liu, D., Li, J., & Lewis, F. L. (2024) · IEEE/CAA Journal of Automatica Sinica · doi:10.1109/jas.2023.123843 · Cited by 364
Self-citationAdaptive Fuzzy Inverse Optimal Control for Uncertain Strict-Feedback Nonlinear Systems
Li, Y.-M., Min, X., & Tong, S. (2020) · IEEE Transactions on Fuzzy Systems · doi:10.1109/tfuzz.2019.2935693 · Cited by 314
Adaptive Controller Design-Based ABLF for a Class of Nonlinear Time-Varying State Constraint Systems
Liu, Y.-J., Lu, S., Li, D., & Tong, S. (2017) · IEEE Transactions on Systems, Man, and Cybernetics: Systems · doi:10.1109/tsmc.2016.2633007 · Cited by 284
Metadata
- Authors
- Wei, Q., Liu, D., & Lin, H.
- Venue
- IEEE Transactions on Cybernetics
- Published
- 2016-03
- Volume
- 46
- Issue
- 3
- Pages
- 840-853
- ISSN
- 2168-2267 (ISSN-L)
- 2168-2275
- OpenAlex
- W2189990206
- Cited by
- 465
- References
- 62
- FWCI
- 36.27
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.