Data-Driven Performance-Prescribed Reinforcement Learning Control of an Unmanned Surface Vehicle
The DOI matches its Crossref record; 40 of the 40 references it cites were found in a registry or an index.
Abstract
An unmanned surface vehicle (USV) under complicated marine environments can hardly be modeled well such that model-based optimal control approaches become infeasible. In this article, a self-learning-based model-free solution only using input-output signals of the USV is innovatively provided. To this end, a data-driven performance-prescribed reinforcement learning control (DPRLC) scheme is created to pursue control optimality and prescribed tracking accuracy simultaneously. By devising state transformation with prescribed performance, constrained tracking errors are substantially converted into constraint-free stabilization of tracking errors with unknown dynamics. Reinforcement learning paradigm using neural network-based actor-critic learning framework is further deployed to directly optimize controller synthesis deduced from the Bellman error formulation such that transformed tracking errors evolve a data-driven optimal controller. Theoretical analysis eventually ensures that the entire DPRLC scheme can guarantee prescribed tracking accuracy, subject to optimal cost. Both simulations and virtual-reality experiments demonstrate the remarkable effectiveness and superiority of the proposed DPRLC scheme.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Beyond Regression : "New Tools for Prediction and Analysis in the Behavioral Sciences
Werbos, P. J. (1974) · Medical Entomology and Zoology · Cited by 2,789
Robust Adaptive Control of Feedback Linearizable MIMO Nonlinear Systems With Prescribed Performance
Bechlioulis, C. P., & Rovithakis, G. A. (2008) · IEEE Transactions on Automatic Control · doi:10.1109/tac.2008.929402 · Cited by 2,756
Online actor–critic algorithm to solve the continuous-time infinite horizon optimal control problem
Vamvoudakis, K. G., & Lewis, F. L. (2010) · Automatica · doi:10.1016/j.automatica.2010.02.018 · Cited by 1,703
Bechlioulis, C. P., & Rovithakis, G. A. (2009) · Automatica · doi:10.1016/j.automatica.2008.08.012 · Cited by 1,054
Approximate dynamic programming for real-time control and neural modeling
Werbos, P. J. (1992) · Medical Entomology and Zoology · Cited by 971
Adaptive maneuvering, with experiments, for a model ship in a marine control laboratory
Skjetne, R., Fossen, T. I., & Kokotović, P. V. (2005) · Automatica · doi:10.1016/j.automatica.2004.10.006 · Cited by 870
Observer-Based Adaptive Fuzzy Fault-Tolerant Optimal Control for SISO Nonlinear Systems
Li, Y., Sun, K., & Tong, S. (2019) · IEEE Transactions on Cybernetics · doi:10.1109/tcyb.2017.2785801 · Cited by 378
Off-Policy Reinforcement Learning for $ H_\infty $ Control Design
Luo, B., Wu, H.-N., & Huang, T. (2015) · IEEE Transactions on Cybernetics · doi:10.1109/tcyb.2014.2319577 · Cited by 362
Adaptive Robust Finite-Time Trajectory Tracking Control of Fully Actuated Marine Surface Vehicles
Wang, N., Qian, C., Sun, J.-C., & Liu, Y.-C. (2016) · IEEE Transactions on Control Systems Technology · doi:10.1109/tcst.2015.2496585 · Cited by 346
Self-citationIterative Learning Control for a Flapping Wing Micro Aerial Vehicle Under Distributed Disturbances
He, W., Meng, T., He, X., & Sun, C. (2019) · IEEE Transactions on Cybernetics · doi:10.1109/tcyb.2018.2808321 · Cited by 329
Platoon Formation Control With Prescribed Performance Guarantees for USVs
Dai, S.-L., He, S., Lin, H., & Wang, C. (2018) · IEEE Transactions on Industrial Electronics · doi:10.1109/tie.2017.2758743 · Cited by 327
Reinforcement learning-based optimal tracking control of an unknown unmanned surface vehicle
Wang, N., Gao, Y., Zhao, H., & Ahn, C. K. (2021) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2020.3009214 · Cited by 322
Self-citationAdaptive Neural Control of Underactuated Surface Vessels With Prescribed Performance Guarantees
Dai, S.-L., He, S., Wang, M., & Yuan, C. (2019) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2018.2876685 · Cited by 320
Dynamic Learning From Adaptive Neural Control of Robot Manipulators With Prescribed Performance
Wang, M., & Yang, A. (2017) · IEEE Transactions on Systems, Man, and Cybernetics: Systems · doi:10.1109/tsmc.2016.2645942 · Cited by 268
Adaptive Tracking Control of Surface Vessel Using Optimized Backstepping Technique
Wen, G., Ge, S. S., Chen, C. L. P., Tu, F., & Wang, S. (2019) · IEEE Transactions on Cybernetics · doi:10.1109/tcyb.2018.2844177 · Cited by 262
Accurate Trajectory Tracking of Disturbed Surface Vehicles: A Finite-Time Control Approach
Wang, N., Karimi, H. R., Li, H., & Su, S.-F. (2019) · IEEE/ASME Transactions on Mechatronics · doi:10.1109/tmech.2019.2906395 · Cited by 258
Self-citationHou, Z., & Zhu, Y. (2013) · IEEE Transactions on Industrial Informatics · doi:10.1109/tii.2013.2257806 · Cited by 242
Value and Policy Iterations in Optimal Control and Adaptive Dynamic Programming
Bertsekas, D. P. (2017) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2015.2503980 · Cited by 238
Wang, D., Liu, D., & Li, H. (2014) · IEEE Transactions on Automation Science and Engineering · doi:10.1109/tase.2013.2296206 · Cited by 238
Wang, Y., Jiang, B., Wu, Z.-G., Xie, S., & Peng, Y. (2021) · IEEE Transactions on Systems, Man, and Cybernetics: Systems · doi:10.1109/tsmc.2020.2964808 · Cited by 223
He, H., Ni, Z., & Fu, J. (2012) · Neurocomputing · doi:10.1016/j.neucom.2011.05.031 · Cited by 221
Data-Based Adaptive Critic Designs for Nonlinear Robust Optimal Control With Uncertain Dynamics
Wang, D., Liu, D., Zhang, Q., & Zhao, D. (2016) · IEEE Transactions on Systems, Man, and Cybernetics: Systems · doi:10.1109/tsmc.2015.2492941 · Cited by 218
Bechlioulis, C. P., Karras, G. C., Heshmati-Alamdari, S., & Kyriakopoulos, K. J. (2017) · IEEE Transactions on Control Systems Technology · doi:10.1109/tcst.2016.2555247 · Cited by 217
Deep Reinforcement Learning With Visual Attention for Vehicle Classification
Zhao, D., Chen, Y., & Lv, L. (2017) · IEEE Transactions on Cognitive and Developmental Systems · doi:10.1109/tcds.2016.2614675 · Cited by 198
Zhang, H., Cai, Y., Wang, Y., & Su, H. (2020) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2019.2958107 · Cited by 190
Event-Triggered Optimal Control With Performance Guarantees Using Adaptive Dynamic Programming
Luo, B., Yang, Y., Liu, D., & Wu, H.-N. (2020) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2019.2899594 · Cited by 177
Data-Driven MFAC for a Class of Discrete-Time Nonlinear Systems With RBFNN
Zhu, Y., & Hou, Z. (2014) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2013.2291792 · Cited by 130
Wang, M., Wang, C., Shi, P. J., & Liu, X. (2015) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2015.2496622 · Cited by 121
Hyperbolic-Tangent LOS Guidance-Based Finite-Time Path Following of Underactuated Marine Vehicles
Wang, N., & Ahn, C. K. (2019) · IEEE Transactions on Industrial Electronics · doi:10.1109/tie.2019.2947845 · Cited by 118
Self-citationSuccessive Waypoints Tracking of an Underactuated Surface Vehicle
Wang, N., & Karimi, H. R. (2019) · IEEE Transactions on Industrial Informatics · doi:10.1109/tii.2019.2922823 · Cited by 115
Self-citationOptimal Elevator Group Control via Deep Asynchronous Actor–Critic Learning
Wei, Q., Wang, L., Liu, Y., & Polycarpou, M. M. (2020) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2020.2965208 · Cited by 87
Zuo, S., Song, Y., Lewis, F. L., & Davoudi, A. A. (2017) · IEEE Transactions on Cybernetics · doi:10.1109/tcyb.2017.2761878 · Cited by 83
Adaptive Near-Optimal Control of Uncertain Systems With Application to Underactuated Surface Vessels
Zhang, Y., Li, S., & Liu, X. (2017) · IEEE Transactions on Control Systems Technology · doi:10.1109/tcst.2017.2705057 · Cited by 81
Tutsoy, Ö., Barkana, D. E., & Tugal, H. (2018) · ISA Transactions · doi:10.1016/j.isatra.2018.03.002 · Cited by 75
Stability Analysis of Optimal Adaptive Control Using Value Iteration With Approximation Errors
Heydari, A. (2018) · IEEE Transactions on Automatic Control · doi:10.1109/tac.2018.2790260 · Cited by 69
Tutsoy, Ö., & Çolak, Ş. (2015) · Proceedings of the Institution of Mechanical Engineers Part I Journal of Systems and Control Engineering · doi:10.1177/0959651815603910 · Cited by 50
Vamvoudakis, K., Vrabie, D., & Lewis, F. (2009) · 2009 IEEE Symposium on Adaptive Dynamic Programming and Reinforcement Learning · doi:10.1109/adprl.2009.4927523 · Cited by 45
Wang, H., Liu, X., & Liu, K. (2015) · IET Control Theory and Applications · doi:10.1049/iet-cta.2014.0709 · Cited by 42
Control Design of a Marine Vessel System Using Reinforcement Learning
Yin, Z. J., He, W., Yang, C., & Sun, C. (2018) · Neurocomputing · doi:10.1016/j.neucom.2018.05.061 · Cited by 40
Wang, T., Sui, S., & Tong, S. (2017) · Neurocomputing · doi:10.1016/j.neucom.2017.03.053 · Cited by 36
Cited by
Survey of deep learning for autonomous surface vehicles in marine environments
Qiao, Y., Yin, J., Wang, W., Duarte, F., Yang, J., & Ratti, C. (2023) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2023.3235911 · Cited by 192
Metadata
- Authors
- Wang, N., Gao, Y., & Zhang, X.
- Venue
- IEEE Transactions on Neural Networks and Learning Systems
- Published
- 2021-12
- Volume
- 32
- Issue
- 12
- Pages
- 5456-5467
- ISSN
- 2162-237X (ISSN-L)
- 2162-2388
- OpenAlex
- W3131678030
- Cited by
- 282
- References
- 40
- FWCI
- 21.87
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.