The DOI matches its Crossref record; 31 of the 31 references it cites were found in a registry or an index.
Deep reinforcement learning-based controller for path following of an unmanned surface vehicle
Abstract
Abstract In this paper, a deep reinforcement learning (DRL)-based controller for path following of an unmanned surface vehicle (USV) is proposed. The proposed controller can self-develop a vehicle’s path following capability by interacting with the nearby environment. A deep deterministic policy gradient (DDPG) algorithm, which is an actor-critic-based reinforcement learning algorithm, was adapted to capture the USV’s experience during the path-following trials. A Markov decision process model, which includes the state, action, and reward formulation, specially designed for the USV path-following problem is suggested. The control policy was trained with repeated trials of path-following simulation. The proposed method’s path-following and self-learning capabilities were validated through USV simulation and a free-running test of the full-scale USV.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“It was able to learn different policies from traditional controllers during training because it learned how to map the system output to control actions [13].”
From Deep reinforcement learning-based adaptive fuzzy control for electro-hydraulic servo system (2025) · cites this paper as [13]
Citations
Cites
Human-level control through deep reinforcement learning
Mnih, V., Kavukcuoglu, K., Silver, D., Rusu, A. A., Veness, J., Bellemare, M. G., Graves, A., Riedmiller, M., Fidjeland, A. K., Ostrovski, G., et al. (2015) · Nature · doi:10.1038/nature14236 · Cited by 30,357
Continuous control with deep reinforcement learning
Lillicrap, T. P., Hunt, J. J., Pritzel, A., Heess, N., Erez, T., Tassa, Y., Silver, D., & Wierstra, D. (2015) · arXiv · doi:10.48550/arxiv.1509.02971 · Cited by 10,782
Introduction to Reinforcement Learning
Sutton, R. S., & Barto, A. G. (1998) · The MIT Press eBooks · Cited by 6,855
Marine Control Systems Guidance, Navigation, and Control of Ships, Rigs and Underwater Vehicles
Fossen, T. I. (2002) · Cited by 1,540
Vector Field Path Following for Miniature Air Vehicles
Nelson, D. R., Barber, D. B., McLain, T. W., et al. (2007) · IEEE Transactions on Robotics · doi:10.1109/tro.2007.898976 · Cited by 650
Cui, R., Yang, C., Li, Y., & Sharma, S. (2017) · IEEE Transactions on Systems, Man, and Cybernetics: Systems · doi:10.1109/tsmc.2016.2645699 · Cited by 538
Collision avoidance for an unmanned surface vehicle using deep reinforcement learning
Woo, J., & Kim, N. (2020) · Ocean Engineering · doi:10.1016/j.oceaneng.2020.107001 · Cited by 264
Self-citationConcise deep reinforcement learning obstacle avoidance for underactuated unmanned marine vessels
Cheng, Y., & Zhang, W. (2017) · Neurocomputing · doi:10.1016/j.neucom.2017.06.066 · Cited by 248
A Deep Reinforcement Learning Strategy for UAV Autonomous Landing on a Moving Platform
Rodriguez-Ramos, A., Sampedro, C., Bavle, H., de la Puente, P., & Campoy, P. (2019) · Journal of Intelligent & Robotic Systems · doi:10.1007/s10846-018-0891-8 · Cited by 229
Modeling, Identification, and Control of an Unmanned Surface Vehicle
Sonnenburg, C., & Woolsey, C. A. (2013) · Journal of Field Robotics · doi:10.1002/rob.21452 · Cited by 192
Research works printed but not linked: 4
- Bertram, 2008, Unmanned surface vehicles-a survey, Skibsteknisk Selskab, Copenhagen, Denmark, 1, 1
- Naeem, W., Sutton, R., Chudley, J., Modelling and control of an unmanned surface vehicle for environmental monitoring, in: UKACC International Control Conference, August, Glasgow, Scotland, 2006.
- Silver, D., Lever, G., Heess, N., Degris, T., Wierstra, D., Riedmiller, M., Deterministic policy gradient algorithms, in: ICML, 2014.
- Yiannis, P., Mitchel, W., Operations architecture and vector field guidance for the river scout subscale unmanned surface vehicle, in: Defence and Homeland Security Symposium, 2013, pp. 55–60.
Cited by
Survey of deep learning for autonomous surface vehicles in marine environments
Qiao, Y., Yin, J., Wang, W., Duarte, F., Yang, J., & Ratti, C. (2023) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2023.3235911 · Cited by 192
Dynamic robotic tracking of underwater targets using reinforcement learning
Masmitja, I., Martin, M., O’Reilly, T., Kieft, B., Palomeras, N., Navarro, J., & Katija, K. (2023) · Science Robotics · doi:10.1126/scirobotics.ade7811 · Cited by 96
Deep reinforcement learning-based adaptive fuzzy control for electro-hydraulic servo system
Khater, A. A., Fekry, M., El-Bardini, M., & El-Nagar, A. M. (2025) · Neural Computing and Applications · doi:10.1007/s00521-024-10741-x · Cited by 17
“It was able to learn different policies from traditional controllers during training because it learned how to map the system output to control actions [13].”
Metadata
- Authors
- Woo, J., Yu, C., & Kim, N.
- Affiliations
- Seoul National University (KR)
- Corresponding author
- Kim, Nakwan
- Venue
- Ocean Engineering
- Publisher
- Elsevier BV
- Published
- 2019-07
- Volume
- 183
- Pages
- 155-166
- ISSN
- 0029-8018 (ISSN-L)
- OpenAlex
- W2963917788
- Cited by
- 246
- References
- 31
- Citations by year
- 2026: 20 · 2025: 31 · 2024: 50 · 2023: 57 · 2022: 46 · 2021: 25 · 2020: 15 · 2019: 2
- FWCI
- 10.15
- Type
- journal article
- Language
- en
- Open access
- closed
- Licence
- www.elsevier.com/tdm/userlicense/1.0/ (tdm)
- Locations
- Ocean Engineering (publishedVersion)
- Funders
- Seoul National University, Republic of Korea
- Agency for Defense Development (UD160009DD)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Reinforcement Learning in Robotics
- Physical Sciences › Engineering › Ocean Engineering › Maritime Navigation and Safety
- Physical Sciences › Engineering › Control and Systems Engineering › Adaptive Dynamic Programming Control
- Keywords
- unmanned surface vehicles, path following, deep reinforcement learning, deep deterministic policy gradient, actor-critic, Markov decision process, autonomous navigation, self-learning control
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Deep reinforcement learning-based controller for path following of an unmanned surface vehicle. https://app.cochat.ai/papers/10.1016/j.oceaneng.2019.04.099
- The paper
- Woo, J., Yu, C., & Kim, N. (2019). Deep reinforcement learning-based controller for path following of an unmanned surface vehicle. Ocean Engineering. https://doi.org/10.1016/j.oceaneng.2019.04.099
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.
