The DOI matches its Crossref record; 38 of the 38 references it cites were found in a registry or an index.
Adaptive Optimal Control of Unknown Constrained-Input Systems Using Policy Iteration and Neural Networks
Abstract
This paper presents an online policy iteration (PI) algorithm to learn the continuous-time optimal control solution for unknown constrained-input systems. The proposed PI algorithm is implemented on an actor-critic structure where two neural networks (NNs) are tuned online and simultaneously to generate the optimal bounded control policy. The requirement of complete knowledge of the system dynamics is obviated by employing a novel NN identifier in conjunction with the actor and critic NNs. It is shown how the identifier weights estimation error affects the convergence of the critic NN. A novel learning rule is developed to guarantee that the identifier weights converge to small neighborhoods of their ideal values exponentially fast. To provide an easy-to-check persistence of excitation condition, the experience replay technique is used. That is, recorded past experiences are used simultaneously with current data for the adaptation of the identifier weights. Stability of the whole system consisting of the actor, critic, system state, and system identifier is guaranteed while all three networks undergo adaptation. Convergence to a near-optimal control law is also shown. The effectiveness of the proposed method is illustrated with a simulation example.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Reinforcement Learning: An Introduction
Sutton, R. S., & Barto, A. G. (1998) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.1998.712192 · Cited by 27,225
Multilayer feedforward networks are universal approximators
Hornik, K., Stinchcombe, M., & White, H. (1989) · Neural Networks · doi:10.1016/0893-6080(89)90020-8 · Cited by 21,728
Multilayer feedforward networks are universal approximators
HornikK., StinchcombeM., & WhiteH. (1989) · Neural Networks · doi:10.5555/70405.70408 · Cited by 9,081
Ioannou, P., & Sun, J. (1995) · Medical Entomology and Zoology · doi:10.1109/acc.1984.4171550 · Cited by 5,663
Neuronlike adaptive elements that can solve difficult learning control problems
Barto, A. G., Sutton, R. S., & Anderson, C. W. (1983) · IEEE Transactions on Systems, Man, and Cybernetics · doi:10.1109/tsmc.1983.6313077 · Cited by 3,233
Adaptive control stability, convergence, and robustness
Egardt, B. (1993) · Automatica · doi:10.1016/0005-1098(93)90081-4 · Cited by 2,712
Lewis, F. L., Vrabie, D. L., & Syrmos, V. L. (2012) · doi:10.1002/9781118122631 · Cited by 2,020
Self-citationThe Method of Weighted Residuals and Variational Principles
Finlayson, B. A. (2013) · Society for Industrial and Applied Mathematics eBooks · doi:10.1137/1.9781611973242 · Cited by 1,945
Neural Network Control Of Robot Manipulators And Non-Linear Systems
Lewis, F. W., Jagannathan, S., & Yesildirak, A. (2020) · CRC Press eBooks · doi:10.1201/9781003062714 · Cited by 1,867
Self-citationOnline actor–critic algorithm to solve the continuous-time infinite horizon optimal control problem
Vamvoudakis, K. G., & Lewis, F. L. (2010) · Automatica · doi:10.1016/j.automatica.2010.02.018 · Cited by 1,703
Self-citationApproximate Dynamic Programming: Solving the Curses of Dimensionality
Powell, W. B. (2007) · Wiley Series in Probability and Statistics · doi:10.1002/9780470182963 · Cited by 1,566
Self-improving reactive agents based on reinforcement learning, planning and teaching
Lin, L.-J. (1992) · Machine Learning · doi:10.1007/bf00992699 · Cited by 1,564
Reinforcement learning and adaptive dynamic programming for feedback control
Lewis, F. L., & Vrabie, D. (2009) · IEEE Circuits and Systems Magazine · doi:10.1109/mcas.2009.933854 · Cited by 1,558
Self-citationAbu-Khalaf, M., & Lewis, F. L. (2005) · Automatica · doi:10.1016/j.automatica.2004.11.034 · Cited by 1,349
Self-citationProkhorov, D. V., & Wunsch, D. C. (1997) · IEEE Transactions on Neural Networks · doi:10.1109/72.623201 · Cited by 1,186
Jiang, Y., & Jiang, Z.-P. (2012) · Automatica · doi:10.1016/j.automatica.2012.06.096 · Cited by 1,072
Ioannou, P., & Fidan, B. (2006) · Society for Industrial and Applied Mathematics eBooks · doi:10.1137/1.9780898718652 · Cited by 972
Reinforcement Learning in Continuous Time and Space
Doya, K. (2000) · Neural Computation · doi:10.1162/089976600300015961 · Cited by 960
Adaptive optimal control for continuous-time linear systems based on policy iteration
Vrabie, D., Pastravanu, O., Abu-Khalaf, M., & Lewis, F. L. (2009) · Automatica · doi:10.1016/j.automatica.2008.08.017 · Cited by 949
Self-citationAdaptive Dynamic Programming: An Introduction
Wang, F.-Y., Zhang, H., & Liu, D. (2009) · IEEE Computational Intelligence Magazine · doi:10.1109/mci.2009.932261 · Cited by 898
High-order neural network structures for identification of dynamical systems
Kosmatopoulos, E. B., Polycarpou, M. M., Christodoulou, M. A., & Ioannou, P. A. (1995) · IEEE Transactions on Neural Networks · doi:10.1109/72.363477 · Cited by 802
Zhang, H., Luo, Y., & Liu, D. (2009) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.2009.2027233 · Cited by 649
Bhasin, S., Kamalapurkar, R., Johnson, M., Vamvoudakis, K. G., Lewis, F. L., & Dixon, W. E. (2013) · Automatica · doi:10.1016/j.automatica.2012.09.019 · Cited by 642
Self-citationVrabie, D., & Lewis, F. (2009) · Neural Networks · doi:10.1016/j.neunet.2009.03.008 · Cited by 634
Self-citationA Menu of Designs for Reinforcement Learning Over Time
Werbos, P. J. (1991) · Neural Networks for Control · doi:10.7551/mitpress/4939.003.0007 · Cited by 605
Concurrent learning for convergence in adaptive control without persistency of excitation
Chowdhary, G., & Johnson, E. (2010) · 49th IEEE Conference on Decision and Control (CDC) · doi:10.1109/cdc.2010.5717148 · Cited by 419
Neural networks for control and system identification
Werbos, P. J. (2003) · Proceedings of the 28th IEEE Conference on Decision and Control · doi:10.1109/cdc.1989.70114 · Cited by 319
Finite-Approximation-Error-Based Optimal Control Approach for Discrete-Time Nonlinear Systems
Liu, D., & Wei, Q. (2013) · IEEE Transactions on Cybernetics · doi:10.1109/tsmcb.2012.2216523 · Cited by 304
Experience Replay for Real-Time Reinforcement Learning Control
Adam, S., Busoniu, L., & Babuska, R. (2012) · IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews) · doi:10.1109/tsmcc.2011.2106494 · Cited by 255
Venayagamoorthy, G. K., Harley, R. G., & Wunsch, D. C. (2002) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.2002.1000146 · Cited by 220
Lee, J. Y., Park, J. B., & Choi, Y. H. (2012) · Automatica · doi:10.1016/j.automatica.2012.06.008 · Cited by 188
Reinforcement learning in continuous time: advantage updating
Baird, L. C. (1994) · Proceedings of 1994 IEEE International Conference on Neural Networks (ICNN'94) · doi:10.1109/icnn.1994.374604 · Cited by 177
A Self-Learning Call Admission Control Scheme for CDMA Cellular Networks
Liu, D., Zhang, Y., & Zhang, H. (2005) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.2005.853408 · Cited by 162
Adaptive Critic Learning Techniques for Engine Torque and Air–Fuel Ratio Control
Liu, D., Javaherian, H., Kovalenko, O., & Huang, T. (2008) · IEEE Transactions on Systems Man and Cybernetics Part B (Cybernetics) · doi:10.1109/tsmcb.2008.922019 · Cited by 154
Continuous-Time Adaptive Critics
Hanselmann, T., Noakes, L., & Zaknich, A. (2007) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.2006.889499 · Cited by 124
Real-time reinforcement learning by sequential Actor-Critics and experience replay
Wawrzyński, P. (2009) · Neural Networks · doi:10.1016/j.neunet.2009.05.011 · Cited by 118
Issues on Stability of ADP Feedback Controllers for Dynamical Systems
Balakrishnan, S. N., Ding, J., & Lewis, F. L. (2008) · IEEE Transactions on Systems Man and Cybernetics Part B (Cybernetics) · doi:10.1109/tsmcb.2008.926599 · Cited by 108
Self-citationDierks, T., & Jagannathan, S. (2010) · Proceedings of the IEEE Conference on Decision & Control, including the Symposium on Adaptive Processes · doi:10.1109/cdc.2010.5717676 · Cited by 88
Cited by
Policy Iteration Adaptive Dynamic Programming Algorithm for Discrete-Time Nonlinear Systems
Liu, D., & Wei, Q. (2014) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2013.2281663 · Cited by 753
Adaptive Dynamic Programming for Control: A Survey and Recent Advances
Liu, D., Xue, S., Zhao, B., Luo, B., & Wei, Q. (2021) · IEEE Transactions on Systems, Man, and Cybernetics: Systems · doi:10.1109/tsmc.2020.3042876 · Cited by 708
Li, Y., Liu, Y., & Tong, S. (2022) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2021.3051030 · Cited by 663
Adaptive Fuzzy Neural Network Control for a Constrained Robot Using Impedance Learning
He, W., & Dong, Y. (2018) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2017.2665581 · Cited by 642
Zhang, H., Jiang, H., Luo, Y., & Xiao, G. (2017) · IEEE Transactions on Industrial Electronics · doi:10.1109/tie.2016.2542134 · Cited by 424
Metadata
- Authors
- Modares, H., Lewis, F. L., & Naghibi-Sistani, M.-B.
- Venue
- IEEE Transactions on Neural Networks and Learning Systems
- Published
- 2013-10
- Volume
- 24
- Issue
- 10
- Pages
- 1513-1525
- ISSN
- 2162-237X (ISSN-L)
- 2162-2388
- OpenAlex
- W2010152647
- Cited by
- 462
- References
- 38
- FWCI
- 33.28
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.