The DOI matches its Crossref record; 49 of the 49 references it cites were found in a registry or an index.
Reinforcement-Learning-Based Robust Controller Design for Continuous-Time Uncertain Nonlinear Systems Subject to Input Constraints
Abstract
The design of stabilizing controller for uncertain nonlinear systems with control constraints is a challenging problem. The constrained-input coupled with the inability to identify accurately the uncertainties motivates the design of stabilizing controller based on reinforcement-learning (RL) methods. In this paper, a novel RL-based robust adaptive control algorithm is developed for a class of continuous-time uncertain nonlinear systems subject to input constraints. The robust control problem is converted to the constrained optimal control problem with appropriately selecting value functions for the nominal system. Distinct from typical action-critic dual networks employed in RL, only one critic neural network (NN) is constructed to derive the approximate optimal control. Meanwhile, unlike initial stabilizing control often indispensable in RL, there is no special requirement imposed on the initial control. By utilizing Lyapunov's direct method, the closed-loop optimal control system and the estimated weights of the critic NN are proved to be uniformly ultimately bounded. In addition, the derived approximate optimal control is verified to guarantee the uncertain nonlinear system to be stable in the sense of uniform ultimate boundedness. Two simulation examples are provided to illustrate the effectiveness and applicability of the present approach.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Reinforcement Learning: An Introduction
(2005) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.2004.842673 · Cited by 24,496
Multilayer feedforward networks are universal approximators
Hornik, K., Stinchcombe, M., & White, H. (1989) · Neural Networks · doi:10.1016/0893-6080(89)90020-8 · Cited by 21,728
Bellman, R. E. (2010) · doi:10.1515/9781400835386 · Cited by 14,588
Principles of mathematical analysis
Rudin, W. (1964) · UCLA - Biblioteca de Ciencias y Tecnologia Felix Morales Bueno · Cited by 5,300
Principles of mathematical analysis
Hammer, C. (1953) · Journal of the Franklin Institute · doi:10.1016/0016-0032(53)90617-6 · Cited by 3,985
Beyond Regression : "New Tools for Prediction and Analysis in the Behavioral Sciences
Werbos, P. J. (1974) · Medical Entomology and Zoology · Cited by 2,789
Beyond regression : new fools for prediction and analysis in the behavioral sciences
Werbos, P. J. (1974) · Medical Entomology and Zoology · Cited by 2,739
Applied Optimal Control: Optimization, Estimation, and Control
Bryson, A. E., Ho, Y.-C., & Siouris, G. M. (1979) · IEEE Transactions on Systems, Man, and Cybernetics · doi:10.1109/tsmc.1979.4310229 · Cited by 2,582
Lewis, F. L., Vrabie, D. L., & Syrmos, V. L. (2012) · doi:10.1002/9781118122631 · Cited by 2,020
The Method of Weighted Residuals and Variational Principles
Finlayson, B. A. (2013) · Society for Industrial and Applied Mathematics eBooks · doi:10.1137/1.9781611973242 · Cited by 1,945
Neural Network Control Of Robot Manipulators And Non-Linear Systems
Lewis, F. W., Jagannathan, S., & Yesildirak, A. (2020) · CRC Press eBooks · doi:10.1201/9781003062714 · Cited by 1,867
Online actor–critic algorithm to solve the continuous-time infinite horizon optimal control problem
Vamvoudakis, K. G., & Lewis, F. L. (2010) · Automatica · doi:10.1016/j.automatica.2010.02.018 · Cited by 1,703
Abu-Khalaf, M., & Lewis, F. L. (2005) · Automatica · doi:10.1016/j.automatica.2004.11.034 · Cited by 1,349
(2012) · IEEE Control Systems · doi:10.1109/mcs.2012.2214134 · Cited by 1,105
Control Systems with Actuator Saturation
Hu, T., & Lin, Z. (2001) · Birkhäuser Boston eBooks · doi:10.1007/978-1-4612-0205-9 · Cited by 1,089
Jiang, Y., & Jiang, Z.-P. (2012) · Automatica · doi:10.1016/j.automatica.2012.06.096 · Cited by 1,072
Policy Iteration Adaptive Dynamic Programming Algorithm for Discrete-Time Nonlinear Systems
Liu, D., & Wei, Q. (2014) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2013.2281663 · Cited by 753
Self-citationModares, H., & Lewis, F. L. (2014) · Automatica · doi:10.1016/j.automatica.2014.05.011 · Cited by 701
Galerkin approximations of the generalized Hamilton-Jacobi-Bellman equation
Beard, R. W., Saridis, G. N., & Wen, J. T.-Y. (1997) · Automatica · doi:10.1016/s0005-1098(97)00128-3 · Cited by 667
Zhang, H., Cui, L., Zhang, X., & Luo, Y. (2011) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.2011.2168538 · Cited by 643
Bhasin, S., Kamalapurkar, R., Johnson, M., Vamvoudakis, K. G., Lewis, F. L., & Dixon, W. E. (2013) · Automatica · doi:10.1016/j.automatica.2012.09.019 · Cited by 642
Modares, H., Lewis, F. L., & Naghibi-Sistani, M.-B. (2014) · Automatica · doi:10.1016/j.automatica.2013.09.043 · Cited by 570
Lewis, F. L., & Vamvoudakis, K. G. (2011) · IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics) · doi:10.1109/tsmcb.2010.2043839 · Cited by 544 · cited 1×
“Consequently, RL is often applied to adaptive optimal controller designs [24]–[37].”
MethodZhang, H., Cui, L., & Luo, Y. (2013) · IEEE Transactions on Cybernetics · doi:10.1109/tsmcb.2012.2203336 · Cited by 464
Robust Adaptive Dynamic Programming and Feedback Stabilization of Nonlinear Systems
Jiang, Y., & Jiang, Z.-P. (2014) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2013.2294968 · Cited by 414
Off-Policy Reinforcement Learning for $ H_\infty $ Control Design
Luo, B., Wu, H.-N., & Huang, T. (2015) · IEEE Transactions on Cybernetics · doi:10.1109/tcyb.2014.2319577 · Cited by 362
Li, H., & Shi, Y. (2014) · IEEE Transactions on Automatic Control · doi:10.1109/tac.2013.2294618 · Cited by 260
Neural Network Control of a Class of Nonlinear Systems With Actuator Saturation
Gao, W., & Selmic, R. R. (2006) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.2005.863416 · Cited by 254
Adaptive Dynamic Programming for Control: Algorithms and Stability
Zhang, H., Liu, D., Luo, Y., & Wang, D. (2013) · Communications and Control Engineering · doi:10.1007/978-1-4471-4757-2 · Cited by 250
Self-citationWang, D., Liu, D., & Li, H. (2014) · IEEE Transactions on Automation Science and Engineering · doi:10.1109/tase.2013.2296206 · Cited by 238
Self-citationZhang, H., Qin, C., & Luo, Y. (2014) · IEEE Transactions on Automation Science and Engineering · doi:10.1109/tase.2014.2303139 · Cited by 234
Liu, Y.-J., Tang, L., Tong, S., Chen, C. L. P., & Li, D.-J. (2015) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2014.2360724 · Cited by 225
Padhi, R., Unnikrishnan, N., Wang, X., & Balakrishnan, S. N. (2006) · Neural Networks · doi:10.1016/j.neunet.2006.08.010 · Cited by 216
Yang, X., Liu, D., & Wang, D. (2014) · International Journal of Control · doi:10.1080/00207179.2013.848292 · Cited by 203
Self-citationAdaptive Learning in Tracking Control Based on the Dual Critic Network Design
Ni, Z., He, H., & Wen, J. (2013) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2013.2247627 · Cited by 189
Finite-Approximation-Error-Based Discrete-Time Iterative Adaptive Dynamic Programming
Wei, Q., Wang, F.-Y., Liu, D., & Yang, X. (2014) · IEEE Transactions on Cybernetics · doi:10.1109/tcyb.2014.2354377 · Cited by 188
Self-citationZhong, X., He, H., Zhang, H., & Wang, Z. (2014) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2014.2305841 · Cited by 153
Infinite Horizon Self-Learning Optimal Control of Nonaffine Discrete-Time Nonlinear Systems
Wei, Q., Liu, D., & Yang, X. (2015) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2015.2401334 · Cited by 142
Self-citationYang, X., Liu, D., & Huang, Y. (2013) · IET Control Theory and Applications · doi:10.1049/iet-cta.2013.0472 · Cited by 134
Self-citationRevisiting Approximate Dynamic Programming and its Convergence
Heydari, A. (2014) · IEEE Transactions on Cybernetics · doi:10.1109/tcyb.2014.2314612 · Cited by 133
Liu, D., Wang, D., & Yang, X. (2012) · Information Sciences · doi:10.1016/j.ins.2012.07.006 · Cited by 129
Self-citationLee, J. Y., Park, J. B., & Choi, Y. H. (2014) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2014.2328590 · Cited by 127
Goal Representation Heuristic Dynamic Programming on Maze Navigation
Ni, Z., He, H., Wen, J., & Xu, X. (2013) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2013.2271454 · Cited by 106
Continuous-Time Q-Learning for Infinite-Horizon Discounted Cost Linear Quadratic Regulator Problems
Muthukumar, P., Modares, H., Lewis, F. L., & Aurangzeb, M. (2015) · IEEE Transactions on Cybernetics · doi:10.1109/tcyb.2014.2322116 · Cited by 97
Kusy, M., & Zajdel, R. (2014) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2014.2376703 · Cited by 86
Yang, X., Liu, D., Wang, D., & Wei, Q. (2014) · Neural Networks · doi:10.1016/j.neunet.2014.03.008 · Cited by 61
Self-citationMehraeen, S., Dierks, T., Jagannathan, S., & Crow, M. L. (2012) · IEEE Transactions on Cybernetics · doi:10.1109/tsmcb.2012.2227253 · Cited by 61
Yang, X., Liu, D., & Wei, Q. (2014) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn.2014.6889462 · Cited by 3
Self-citation
Cited by
Value Iteration Adaptive Dynamic Programming for Optimal Control of Discrete-Time Nonlinear Systems
Wei, Q., Liu, D., & Lin, H. (2016) · IEEE Transactions on Cybernetics · doi:10.1109/tcyb.2015.2492242 · Cited by 465
Self-citationLi, H., Wu, Y., & Chen, M. (2021) · IEEE Transactions on Cybernetics · doi:10.1109/tcyb.2020.2982168 · Cited by 422
Observer-Based Adaptive Fuzzy Fault-Tolerant Optimal Control for SISO Nonlinear Systems
Li, Y., Sun, K., & Tong, S. (2019) · IEEE Transactions on Cybernetics · doi:10.1109/tcyb.2017.2785801 · Cited by 378
Adaptive Critic Nonlinear Robust Control: A Survey
Wang, D., He, H., & Liu, D. (2017) · IEEE Transactions on Cybernetics · doi:10.1109/tcyb.2017.2712188 · Cited by 374
Self-citationShang, C., & You, F. (2019) · Engineering · doi:10.1016/j.eng.2019.01.019 · Cited by 332
Metadata
- Authors
- Liu, D., Yang, X., Wang, D., & Wei, Q.
- Venue
- IEEE Transactions on Cybernetics
- Published
- 2015-07
- Volume
- 45
- Issue
- 7
- Pages
- 1372-1385
- ISSN
- 2168-2267 (ISSN-L)
- 2168-2275
- OpenAlex
- W2035003264
- Cited by
- 440
- References
- 49
- FWCI
- 31.61
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.