The DOI matches its Crossref record; 31 of the 31 references it cites were found in a registry or an index.
Deep reinforcement learning-based controller for path following of an unmanned surface vehicle
Abstract
In this paper, a deep reinforcement learning (DRL)-based controller for path following of an unmanned surface vehicle (USV) is proposed. The proposed controller can self-develop a vehicle’s path following capability by interacting with the nearby environment. A deep deterministic policy gradient (DDPG) algorithm, which is an actor-critic-based reinforcement learning algorithm, was adapted to capture the USV’s experience during the path-following trials. A Markov decision process model, which includes the state, action, and reward formulation, specially designed for the USV path-following problem is suggested. The control policy was trained with repeated trials of path-following simulation. The proposed method’s path-following and self-learning capabilities were validated through USV simulation and a free-running test of the full-scale USV.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“It was able to learn different policies from traditional controllers during training because it learned how to map the system output to control actions [13].”
From Deep reinforcement learning-based adaptive fuzzy control for electro-hydraulic servo system (2025) · cites this paper as [13]
Citations
Cites
Human-level control through deep reinforcement learning
Mnih, V., Kavukcuoglu, K., Silver, D., Rusu, A. A., Veness, J., Bellemare, M. G., Graves, A., Riedmiller, M., Fidjeland, A. K., Ostrovski, G., et al. (2015) · Nature · doi:10.1038/nature14236 · Cited by 30,357
Continuous control with deep reinforcement learning
Lillicrap, T. P., Hunt, J. J., Pritzel, A., Heess, N., Erez, T., Tassa, Y., Silver, D., & Wierstra, D. (2015) · arXiv · doi:10.48550/arxiv.1509.02971 · Cited by 10,782
Introduction to Reinforcement Learning
Sutton, R. S., & Barto, A. G. (1998) · The MIT Press eBooks · Cited by 6,855
Marine Control Systems Guidance, Navigation, and Control of Ships, Rigs and Underwater Vehicles
Fossen, T. I. (2002) · Cited by 2,040
Vector Field Path Following for Miniature Air Vehicles
Nelson, D. R., Barber, D. B., McLain, T. W., et al. (2007) · IEEE Transactions on Robotics · doi:10.1109/tro.2007.898976 · Cited by 650
Cui, R., Yang, C., Li, Y., & Sharma, S. (2017) · IEEE Transactions on Systems, Man, and Cybernetics: Systems · doi:10.1109/tsmc.2016.2645699 · Cited by 538
Collision avoidance for an unmanned surface vehicle using deep reinforcement learning
Woo, J., & Kim, N. (2020) · Ocean Engineering · doi:10.1016/j.oceaneng.2020.107001 · Cited by 264
Self-citationConcise deep reinforcement learning obstacle avoidance for underactuated unmanned marine vessels
Cheng, Y., & Zhang, W. (2018) · Neurocomputing · doi:10.1016/j.neucom.2017.06.066 · Cited by 247
A Deep Reinforcement Learning Strategy for UAV Autonomous Landing on a Moving Platform
Rodriguez-Ramos, A., Sampedro, C., Bavle, H., de la Puente, P., & Campoy, P. (2019) · Journal of Intelligent & Robotic Systems · doi:10.1007/s10846-018-0891-8 · Cited by 229
OCEANS 2015 - MTS/IEEE Washington
(2015) · doi:10.23919/oceans20940.2015 · Cited by 189
Modeling, Identification, and Control of an Unmanned Surface Vehicle
Sonnenburg, C. R., & Woolsey, C. A. (2013) · Journal of Field Robotics · doi:10.1002/rob.21452 · Cited by 187
Dynamic model identification of unmanned surface vehicles using deep learning network
Woo, J., Park, J., Yu, C.-W., & Kim, N. (2018) · Applied Ocean Research · doi:10.1016/j.apor.2018.06.011 · Cited by 147
Self-citationPath‐following algorithms and experiments for an unmanned surface vehicle
Bibuli, M., Bruzzone, G., Caccia, M., et al. (2009) · Journal of Field Robotics · doi:10.1002/rob.20303 · Cited by 132
Adaptive Path-Following Control for an Unmanned Surface Vessel Using an Identified Dynamic Model
Shin, J., Kwak, D. J., & Lee, Y.-I. (2017) · IEEE/ASME Transactions on Mechatronics · doi:10.1109/tmech.2017.2651057 · Cited by 129
Robust adaptive formation control of underactuated autonomous surface vessels based on MLP and DOB
Lu, Y., Zhang, G., Sun, Z., & Zhang, W. (2018) · Nonlinear Dynamics · doi:10.1007/s11071-018-4374-z · Cited by 127
A Time-Varying Lookahead Distance Guidance Law for Path Following
Lekkas, A. M., & Fossen, T. I. (2012) · IFAC Proceedings Volumes · doi:10.3182/20120919-3-it-2046.00068 · Cited by 126
Path optimization for marine vehicles in ocean currents using reinforcement learning
Yoo, B., & Kim, J. (2015) · Journal of Marine Science and Technology · doi:10.1007/s00773-015-0355-9 · Cited by 118
Zhang, G., Deng, Y., Zhang, W., & Huang, C. (2018) · Ocean Engineering · doi:10.1016/j.oceaneng.2018.10.009 · Cited by 72
Unmanned surface vehicles - game changing technology for naval operations
Corfield, S., & Young, J. M. (2006) · Institution of Engineering and Technology eBooks · doi:10.1049/pbce069e_ch15 · Cited by 64
Majohr, J., & Buch, T. (2006) · Institution of Engineering and Technology eBooks · doi:10.1049/pbce069e_ch16 · Cited by 61
Mu, D., Wang, G., Fan, Y., et al. (2017) · Applied Sciences · doi:10.3390/app7121232 · Cited by 54
Zhang, R., Tang, P., Su, Y., et al. (2014) · IEEE/CAA Journal of Automatica Sinica · doi:10.1109/jas.2014.7004666 · Cited by 53
Trajectory tracking algorithm for autonomous vehicles using adaptive reinforcement learning
Paula, M. D., & Acosta, G. G. (2015) · MTS/IEEE OCEANS Conference (OCEANS) · doi:10.23919/oceans.2015.7401861 · Cited by 41
Experimental Evaluation of Supervisory Switching Control for Unmanned Surface Vehicles
Bertaska, I. R., & von Ellenrieder, K. D. (2018) · IEEE Journal of Oceanic Engineering · doi:10.1109/joe.2018.2802019 · Cited by 34
Efficient path following algorithm for unmanned surface vehicle
Niu, H., Lu, Y., Savvaris, A., et al. (2016) · OCEANS 2016 - Shanghai · doi:10.1109/oceansap.2016.7485430 · Cited by 27
Semi-online neural-q-learning for real-time robot learning
Carreras, M., Ridao, P., & El-Fakdi, A. (2004) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros.2003.1250705 · Cited by 21
Using of soft computing techniques to control of underwater robot
Garus, J., & Żak, B. (2010) · International Conference on Methods and Models in Automation and Robotics · doi:10.1109/mmar.2010.5587198 · Cited by 17
Straight path following of unmanned surface vehicle under flow disturbance
Zhu, J., Wang, J., Zheng, T., et al. (2016) · OCEANS 2016 - Shanghai · doi:10.1109/oceansap.2016.7485486 · Cited by 15
Reinforcement learning: The application to autonomous biomimetic underwater vehicles control
Magalhães, J., Damas, B., & Lobo, V. J. A. S. (2018) · IOP Conference Series Earth and Environmental Science · doi:10.1088/1755-1315/172/1/012019 · Cited by 11
Vector Field based Guidance Method for Docking of an Unmanned Surface Vehicle
Woo, J., & Kim, N. (2016) · Cited by 10
Self-citationMeng, W., Guo, C. G., Liu, Y., Yang, Y., & Lei, Z. (2012) · International Conference on Intelligent Control and Information Processing · doi:10.1109/icicip.2012.6391415 · Cited by 9
Research works printed but not linked: 4
- Bertram, 2008, Unmanned surface vehicles-a survey, Skibsteknisk Selskab, Copenhagen, Denmark, 1, 1
- Naeem, W., Sutton, R., Chudley, J., Modelling and control of an unmanned surface vehicle for environmental monitoring, in: UKACC International Control Conference, August, Glasgow, Scotland, 2006.
- Silver, D., Lever, G., Heess, N., Degris, T., Wierstra, D., Riedmiller, M., Deterministic policy gradient algorithms, in: ICML, 2014.
- Yiannis, P., Mitchel, W., Operations architecture and vector field guidance for the river scout subscale unmanned surface vehicle, in: Defence and Homeland Security Symposium, 2013, pp. 55–60.
Cited by
Survey of deep learning for autonomous surface vehicles in marine environments
Qiao, Y., Yin, J., Wang, W., Duarte, F., Yang, J., & Ratti, C. (2023) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2023.3235911 · Cited by 192
Dynamic robotic tracking of underwater targets using reinforcement learning
Masmitja, I., Martin, M., O’Reilly, T., Kieft, B., Palomeras, N., Navarro, J., & Katija, K. (2023) · Science Robotics · doi:10.1126/scirobotics.ade7811 · Cited by 96
Deep reinforcement learning-based adaptive fuzzy control for electro-hydraulic servo system
Khater, A. A., Fekry, M., El-Bardini, M., & El-Nagar, A. M. (2025) · Neural Computing and Applications · doi:10.1007/s00521-024-10741-x · Cited by 17
“It was able to learn different policies from traditional controllers during training because it learned how to map the system output to control actions [13].”
Metadata
- Authors
- Woo, J., Yu, C., & Kim, N.
- Affiliations
- Seoul National University (KR)
- Corresponding author
- Kim, Nakwan
- Venue
- Ocean Engineering
- Publisher
- Elsevier BV
- Published
- 2019-07
- Volume
- 183
- Pages
- 155-166
- ISSN
- 0029-8018 (ISSN-L)
- OpenAlex
- W2963917788
- Cited by
- 246
- References
- 31
- Citations by year
- 2026: 20 · 2025: 31 · 2024: 50 · 2023: 57 · 2022: 46 · 2021: 25 · 2020: 15 · 2019: 2
- FWCI
- 10.15
- Type
- journal article
- Language
- en
- Open access
- closed
- Licence
- www.elsevier.com/tdm/userlicense/1.0/ (tdm)
- Locations
- Ocean Engineering (publishedVersion)
- Funders
- Seoul National University, Republic of Korea
- Agency for Defense Development (UD160009DD)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Reinforcement Learning in Robotics
- Physical Sciences › Engineering › Ocean Engineering › Maritime Navigation and Safety
- Physical Sciences › Engineering › Control and Systems Engineering › Adaptive Dynamic Programming Control
- Keywords
- unmanned surface vehicles, path following, deep reinforcement learning, deep deterministic policy gradient, actor-critic, Markov decision process, autonomous navigation, self-learning control
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.