Dynamic robotic tracking of underwater targets using reinforcement learning
Open accessThe DOI matches its Crossref record; 60 of the 60 references it cites were found in a registry or an index.
Abstract
To realize the potential of autonomous underwater robots that scale up our observational capacity in the ocean, new techniques are needed. Fleets of autonomous robots could be used to study complex marine systems and animals with either new imaging configurations or by tracking tagged animals to study their behavior. These activities can then inform and create new policies for community conservation. The role of animal connectivity via active movement of animals represents a major knowledge gap related to the distribution of deep ocean populations. Tracking underwater targets represents a major challenge for observing biological processes in situ, and methods to robustly respond to a changing environment during monitoring missions are needed. Analytical techniques for optimal sensor placement and path planning to locate underwater targets are not straightforward in such cases. The aim of this study was to investigate the use of reinforcement learning as a tool for range-only underwater target-tracking optimization, whose promising capabilities have been demonstrated in terrestrial scenarios. To evaluate its usefulness, a reinforcement learning method was implemented as a path planning system for an autonomous surface vehicle while tracking an underwater mobile target. A complete description of an open-source model, performance metrics in simulated environments, and evaluated algorithms based on more than 15 hours of at-sea field experiments are presented. These efforts demonstrate that deep reinforcement learning is a powerful approach that enhances the abilities of autonomous robots in the ocean and encourages the deployment of algorithms like these for monitoring marine biological systems in the future.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The check wasn’t confident either way, so it gives no verdict — never disputed.
In the citing paper
“Geocentric policies are not invariant to rotations and translations of the flow field and not suitable for autonomous deployment in the open ocean without continuous support from and communication with a terrestrial control center8.”
From Sensing flow gradients is necessary for learning autonomous underwater navigation (2025) · cites this paper as [8]
Experimental — accuracy not yet measuredThe check wasn’t confident either way, so it gives no verdict — never disputed.
In the citing paper
“Our analysis of the sensory requirements for autonomous underwater navigation (Table 1) indicates that egocentric sensing in the agent’s own world eliminates potential time delays and computations inherent to assessing inertial signals8,83 at the expense of requiring more sensors to observe spatial variations in the flow field at the scale of the navigator.”
From Sensing flow gradients is necessary for learning autonomous underwater navigation (2025) · cites this paper as [8]
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Human-level control through deep reinforcement learning
Mnih, V., Kavukcuoglu, K., Silver, D., Rusu, A. A., Veness, J., Bellemare, M. G., Graves, A., Riedmiller, M., Fidjeland, A. K., Ostrovski, G., et al. (2015) · Nature · doi:10.1038/nature14236 · Cited by 30,357 · cited 1×
“The use of deep Q-learning and off-policy algorithms in continuous state and action space presents a major challenge for stability and convergence ( 38 ).”
MethodMastering the game of Go with deep neural networks and tree search
Silver, D., Huang, A., Maddison, C. J., Guez, A., Sifre, L., van den Driessche, G., Schrittwieser, J., Antonoglou, I., Panneershelvam, V., Lanctot, M., et al. (2016) · Nature · doi:10.1038/nature16961 · Cited by 15,612 · cited 1×
“Whereas most of the attention in deep RL has focused on game theory (for example to solve Atari games ( 24 ) or to master the game of Go ( 25 )), the same principles can be used to solve path planning and trajectory optimization problems (Table 1).”
BackgroundObject Detection With Deep Learning: A Review
Zhao, Z.-Q., Zheng, P., Xu, S.-T., & Wu, X. (2019) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2018.2876865 · Cited by 5,545
The State of World Fisheries and Aquaculture 2020
(2020) · doi:10.4060/ca9229en · Cited by 3,167
Underwater acoustic communication channels: Propagation models and statistical characterization
Stojanovic, M., & Preisig, J. (2009) · IEEE Communications Magazine · doi:10.1109/mcom.2009.4752682 · Cited by 2,060
Learning quadrupedal locomotion over challenging terrain
Lee, J., Hwangbo, J., Wellhausen, L., Koltun, V., & Hutter, M. (2020) · Science Robotics · doi:10.1126/scirobotics.abc5986 · Cited by 1,156
Underwater sensor networks: applications, advances and challenges
Heidemann, J., Stojanovic, M., & Zorzi, M. (2012) · Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences · doi:10.1098/rsta.2011.0214 · Cited by 882
Learning quadrupedal locomotion over challenging terrain
Lee, J., Hwangbo, J., Wellhausen, L., Koltun, V., & Hutter, M. (2020) · Repository for Publications and Research Data (ETH Zurich) · doi:10.3929/ethz-b-000448343 · Cited by 752
The generation of plankton patchiness by turbulent stirring
Abraham, E. R. (1998) · Nature · doi:10.1038/35361 · Cited by 456 · cited 1×
“Multiple robotic platforms, in coordination with each other, have been used to characterize different oceanographic phenomena, such as fronts and eddies (10, 11).”
MethodAutonomous navigation of stratospheric balloons using reinforcement learning
Bellemare, M. G., Candido, S., Castro, P. S., Gong, J., Machado, M. C., Moitra, S., Ponda, S. S., & Wang, Z. (2020) · Nature · doi:10.1038/s41586-020-2939-8 · Cited by 286
Research works printed but not linked: 5
- R. S. Sutton A. G. Barto Reinforcement Learning: An Introduction (MIT Press ed. 2 2015).
- T. Haarnoja A. Zhou P. Abbeel S. Levine Soft actor-critic: Off-policy maximum entropy deep reinforcement learning with a stochastic actor in Proceedings of the 35th International Conference on (ICML 2018) (ICML 2018) vol. 5 pp. 2976–2989.
- J. K. Terry, B. Black, N. Grammel, M. Jayakumar, A. Hari, R. Sullivan, L. Santos, R. Perez, C. Horsch, C. Dieffendahl, N. L. Williams, Y. Lokesh, P. Ravi, PettingZoo: A standard API for multi-agent reinforcement learning. Adv. Neural Inf. Process. Syst. 18, 15032–15043 (2021).
- T. P. Lillicrap J. J. Hunt A. Pritzel N. Heess T. Erez Y. Tassa D. Silver D. Wierstra Continuous control with deep reinforcement learning in 4th International Conference on Learning Representations (ICLR 2016)–Conference Track Proceedings (ICLR 2016).
- K. Katija P. L. D. Roberts J. Daniels A. Lapides K. Barnard M. Risi B. Y. Ranaan B. G. Woodward J. Takahashi in 2021 IEEE Winter Conference on Applications of Computer Vision (WACV) (IEEE 2021) pp. 859–868.
Cited by
Chu, S., Lin, M., Li, D., Lin, R., & Xiao, S. (2024) · Ocean Engineering · doi:10.1016/j.oceaneng.2024.120139 · Cited by 79
Zheng, X., Jiang, Y., Mete, M., Li, J., Watanabe, I., Yamada, T., & Paik, J. K. (2025) · Science Robotics · doi:10.1126/scirobotics.adx1519 · Cited by 48
Dai, H., Zhang, C., Hu, H., Hu, Z., Sun, H., Liu, K., Li, T., Fu, J., Zhao, P., & Yang, H. (2024) · Advanced Science · doi:10.1002/advs.202405276 · Cited by 41
Chun, S., Kawamura, C., Ohkuma, K., & Maki, T. (2024) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2024.3364470 · Cited by 31
“Digital Object Identifier 10.1109/LRA.2024.3364470 To compensate for the limitations of biologging, Autonomous Underwater Vehicles (AUVs) have emerged as one of the remote sensing methods [4], [5], [6].”
BackgroundInfluentialA comprehensive review on the evolution of bio-inspired sensors from aquatic creatures
Zhao, Z., Yang, Q., Li, R., Yang, J., Liu, Q., Zhu, B., Weng, C., Liu, W., Hu, P., Ma, L. M., et al. (2024) · Cell Reports Physical Science · doi:10.1016/j.xcrp.2024.102064 · Cited by 33
Reinforcement learning approaches in the motion systems of autonomous underwater vehicles
Yu, T., Zhang, Q., & Liu, T. (2025) · Applied Ocean Research · doi:10.1016/j.apor.2025.104682 · Cited by 27
Bionic Recognition Technologies Inspired by Biological Mechanosensory Systems
Zhang, X., Wang, C., Pi, X., Li, B., Ding, Y., Yu, H., Sun, J., Wang, P., Chen, Y., Wang, Q., et al. (2025) · Advanced Materials · doi:10.1002/adma.202418108 · Cited by 35
Sensing flow gradients is necessary for learning autonomous underwater navigation
Jiao, Y., Hang, H., Merel, J., & Kanso, E. (2025) · Nature Communications · doi:10.1038/s41467-025-58125-6 · Cited by 23
NetTrack: Tracking Highly Dynamic Objects with a Net
Zheng, G., Lin, S., Zuo, H., Fu, C., & Pan, J. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.01811 · Cited by 49
“However, the demand for tracking arbitrary objects, especially highly dynamic objects, in open-world MOT tasks [31, 39, 47] severely challenges this assumption.”
BackgroundTerminal Line-of-Sight Angle-Constrained Target Tracking Guidance for Unmanned Surface Vehicles
Du, B., Yang, K., Zhang, W., & Chen, H. (2024) · IEEE Transactions on Vehicular Technology · doi:10.1109/tvt.2024.3390001 · Cited by 17
Multi-agent self-attention reinforcement learning for multi-USV hunting target
Xue, S., Zhao, N., Wang, L., Zhang, W., Zhang, J., & Zhu, F. (2025) · Neural Networks · doi:10.1016/j.neunet.2025.107574 · Cited by 18
Autonomous robotic systems for coral reef monitoring: Review and open research issues
Sultan, A., Saoud, L. S., Elmezain, M., Heshmat, M., Seneviratne, L. D., & Hussain, I. (2025) · Ecological Informatics · doi:10.1016/j.ecoinf.2025.103511 · Cited by 16
Cai, L., Chang, K., & Girdhar, Y. (2025) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra55743.2025.11128688 · Cited by 31
“Masmitja et al. [19] used RL to develop a tracking behavior for a surface vehicle, enabling it to follow underwater moving targets, but only focused on high-level steering commands.”
Dong, S., Liu, M., Dong, S., Zheng, R., & Wei, P. (2024) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2024.3417698 · Cited by 13
“Various methods, such as rule-based approaches [1], [10], artificial potential field meth-ods [8], particle swarm optimization (PSO) [3], [4], and RL [5], [6], [8], [9], [14], [15] are utilized for trajectory planning.”
BackgroundMethodInfluentialEstimating Lyapunov Region of Attraction for Robust Model-Based Reinforcement Learning USV
Xia, L., Cui, Y., Yi, Z., Li, H., & Wu, X. (2024) · IEEE Transactions on Automation Science and Engineering · doi:10.1109/tase.2024.3492174 · Cited by 14
“…decision process (MDP), the model-free RL approaches are abundantly implemented in different simulated scenarios [18], [19], [20], [21] while the real-world applications heavily rely on extensive training in simulations and have to endure performance degradation resulting from policy transfer [22].”
Effective Probabilistic Neural Networks Model for Model-Based Reinforcement Learning USV
Huang, W., Cui, Y., Li, H., & Wu, X. (2025) · IEEE Transactions on Automation Science and Engineering · doi:10.1109/tase.2025.3539317 · Cited by 17
“The most recent research trained a model-free RL agent in a simulation environment and successfully drove it in the real-world USV tracking task [18].”
Weng, Y., Chun, S., Sekimori, Y., Yokohata, H., Matsuda, T., Pajarinen, J., & Maki, T. (2025) · Ocean Engineering · doi:10.1016/j.oceaneng.2025.121047 · Cited by 13
Efficient navigation of a robotic fish swimming across the vortical flow field
Feng, H., Yuan, D., Miao, J.-L., You, J., Wang, Y., Zhu, Y. F., & Fan, D. (2024) · Journal of Hydrodynamics · doi:10.1007/s42241-025-0103-5 · Cited by 16
Fish-inspired tracking of underwater turbulent plumes
Gunnarson, P., & Dabiri, J. O. (2024) · Bioinspiration & Biomimetics · doi:10.1088/1748-3190/ad7181 · Cited by 11
Wu, X., Wei, C., Guan, D., & Ji, Z. (2025) · Ocean Engineering · doi:10.1016/j.oceaneng.2025.120446 · Cited by 11
Development and Application of the Coverage Path Planning Based on a Biomimetic Robotic Fish
Liu, J., Zhao, J., Liu, Z., Liu, Y., Ren, Y., An, D. U., & Wei, Y. (2025) · Journal of Field Robotics · doi:10.1002/rob.22531 · Cited by 11
“Following an extensive developmental period, RL has found successful applications in robotic control (Luo et al. 2024; Yu et al. 2023; Masmitja et al. 2023; Gan et al. 2024), emerging as a novel avenue for enhancing CPP.”
InfluentialUnmanned Surface Vehicle Navigation Under Disturbances: World Model Enhanced Reinforcement Learning
Jin, K., Liu, Z., Wang, J., & Wang, H. (2025) · IEEE/ASME Transactions on Mechatronics · doi:10.1109/tmech.2025.3552601 · Cited by 11
“Anincreasing numberofstudiesutilizetheRLmethodtorealizetheUSVcon-troltask[2],[19],[20].”
MarineGym: A High-Performance Reinforcement Learning Platform for Underwater Robotics
Chu, S., Huang, Z., Li, Y., Lin, M., Li, D., Carlucho, I., Pétillot, Y., & Yang, C. (2025) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros60139.2025.11246146 · Cited by 17
“Recent RL advancements have enhanced tasks for UUVs [1], [4], [5].”
An Image-Free Single-Pixel Detection System for Adaptive Multi-Target Tracking
Peng, Y., Yang, J., Feng, Y., Yu, S., Xing, F., & Sun, T. (2025) · Sensors · doi:10.3390/s25133879 · Cited by 9
“Multi-target tracking technology is a core requirement in many modern applications, such as drone navigation, autonomous driving and remote sensing [1–8].”
Control of Marine Robots in the Era of Data-Driven Intelligence
Hong, L., Liu, L., Peng, Z., & Zhang, F. (2025) · Annual Review of Control Robotics and Autonomous Systems · doi:10.1146/annurev-control-022723-033729 · Cited by 17
Sim-to-Real Transfer in Reinforcement Learning for Maneuver Control of a Variable-Pitch MAV
Wang, Z., & Zhao, S. (2025) · IEEE Transactions on Industrial Electronics · doi:10.1109/tie.2025.3558030 · Cited by 12
“First, it requires the design of a VPP MAV system that can execute rapid and precise pitch angle adjustments, essential for advanced maneuvers.”
Liu, S., & Guo, G. (2025) · IEEE Transactions on Systems Man and Cybernetics Systems · doi:10.1109/tsmc.2025.3572371 · Cited by 8
“Existing approaches mainly include learning-based methods, such as reinforcement learning [3], [4], optimization-based path planning techniques [5], [6], and kinematic guidance strategies [7].”
Li, S., Huang, W., Miao, C., Xu, K., Chen, Y., Sun, T., & Cui, Y. (2024) · Applied Sciences · doi:10.3390/app142210665 · Cited by 11
Huang, Z., Buchholz, M., Grimaldi, M., Yu, H., Carlucho, I., & Pétillot, Y. (2024) · MTS/IEEE OCEANS Conference (OCEANS) · doi:10.1109/oceans51537.2024.10682284 · Cited by 8
“The simulator can also be applied in training learning-based controllers to learn optimal policies in a low-risk environment[11, 12].”
BackgroundUnderwater Visual SLAM with Depth Uncertainty and Medium Modeling
Liu, R., Fan, S., Wang, W., & Yang, Y. (2025) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51701.2025.00098 · Cited by 9
“Although our method achieves successful convergence on most sequences, failures occasionally occur in particularly challenging long-duration trajectories, e.g. , the length of trajectory shown in (b) exceeds three times that of typical sequences.”
Liang, D., Chu, J., Cui, Y., Liang, D., & Feng, Y. (2024) · IEEE Transactions on Intelligent Vehicles · doi:10.1109/tiv.2024.3443393 · Cited by 6
“For underwater tracking control, experimental testing under simulation environment is an effective way for algorithm verification and comparisons on [35], [36].”
An Adaptive Controller with Disturbance Observer for Underwater Vehicle Manipulator Systems
Zheng, X., Wang, Y., Tian, Q., Zhang, Q., Wang, X., Xu, W., Wang, G., Yang, X., & Sun, Y. (2024) · Electronics · doi:10.3390/electronics13193938 · Cited by 6
“A neural network was used to obtain a desired action function [11] with a reinforcement learning framework.”
Super-resolution reconstruction of turbulent flows from a single Lagrangian trajectory
Wu, H.-L., Xu, A., & Xi, H.-D. (2026) · Journal of Fluid Mechanics · doi:10.1017/jfm.2025.11033 · Cited by 9
“…for a wide range of real-world applications, including the autonomous navigation of aerial and underwater vehicles (Lawrance & Sukkarieh 2009; Masmitja et al. 2023; Zhang et al. 2023), migration of microswimmers (Qiu et al. 2022 a , b ; Mousavi et al. 2024, 2025) and environmental…”
Multi-object detection and tracking for cross-domain operations of underwater robots
Liu, C., Ding, Z., & Li, D. (2026) · Ocean Engineering · doi:10.1016/j.oceaneng.2026.125348 · Cited by 5
Station-keeping strategy in emergency mode for wave gliders considering power shortages
Yu, P., Zhou, Y. S., Sun, X., Sang, H., & Zhang, S. (2024) · Applied Ocean Research · doi:10.1016/j.apor.2024.104236 · Cited by 5
Chen, T., Qi, Q., Xu, D., & Pan, Y. (2024) · IEEE Transactions on Instrumentation and Measurement · doi:10.1109/tim.2024.3522687 · Cited by 4
Adaptive backward-correction extended Kalman filter for small amounts of measurement data
Lan, C., Guo, R., Yang, J., Chen, H., & Wang, J. (2025) · Ocean Engineering · doi:10.1016/j.oceaneng.2025.122500 · Cited by 6
A novel underwater target tracking method in UASNs via collaborative deep reinforcement learning
Zheng, L., Liu, M., Zhang, S., & Dong, S. (2025) · Information Fusion · doi:10.1016/j.inffus.2025.103797 · Cited by 4
Learning an End-To-End Policy for AUV Control Within Just Forty Minutes Using Parallel Simulation
Chu, S., Li, D., & Lin, M. (2024) · MTS/IEEE OCEANS Conference (OCEANS) · doi:10.1109/oceans51537.2024.10682328 · Cited by 4
Liu, Z., An, K., Tang, J., Huo, J., & Chen, J. (2025) · Measurement · doi:10.1016/j.measurement.2025.119339 · Cited by 3
AUV decision-making in bistatic sonar target tracking via deep reinforcement learning
Zhan, W., Tian, Y., Zheng, F., Sang, Q., & Yu, J. (2025) · Robotics and Autonomous Systems · doi:10.1016/j.robot.2025.105262 · Cited by 3
Fu, T., Wang, B., Liu, Y., Ai, R., Liu, J., Wang, C., & Li, D. (2025) · Journal of Ocean Engineering and Science · doi:10.1016/j.joes.2025.12.019 · Cited by 3
He, Q., Li, W., Dai, G., Chen, H., Liu, Q., Tian, X., You, J., Cui, W., Triantafyllou, M. S., & Fan, D. (2026) · Soft Robotics · doi:10.1177/21695172261417752 · Cited by 3
MPPI-DWIT: Robust Bearing-Only Target Tracking of Single AUV With Detection Sector Constraints
Qiu, S., Xu, H., Wang, J., Ru, J., Zhang, W., & Wang, J. (2026) · IEEE Transactions on Automation Science and Engineering · doi:10.1109/tase.2026.3669656 · Cited by 4
“The network architecture and additional parameters follow the setup described in [5].”
InfluentialDesign and control of a bioinspired underwater robot with hydrogel-based flexible pectoral fins
Wang, C., Li, Z., Wang, Y., Wang, X., & Kang, H. (2026) · Bioinspiration & Biomimetics · doi:10.1088/1748-3190/ae4ce4 · Cited by 3
Lan, C., Guo, R., Huan, C., Yang, J., Wang, J., & Chen, Y. (2026) · Neurocomputing · doi:10.1016/j.neucom.2026.134126 · Cited by 3
Yao, P., Liu, Q., & Zhao, Z. (2025) · IEEE Internet of Things Journal · doi:10.1109/jiot.2025.3555243 · Cited by 3
“At the same time, the introduction of DRL into various autonomous vehicles has shown huge potential in autonomous navigation and path planning [24], [25], [26].”
Batet, G., Gandul, D. S., Real, M., Castro, S. G., Fernández, J. D. R., Palomeras, N., & Rusiñol, I. M. (2024) · IEEE International Conference on Automation Science and Engineering (CASE) · doi:10.1109/case59546.2024.10711493 · Cited by 3
“…possibilities that autonomous vehicles open to biotelemetry, combined with state-of-the-art navigation algorithms, various works have designed and proposed a robot fleet to automate and ease more extended oceanography and operational biology, such as the species migration tracking [10] [17] [11].”
BackgroundMethodSelf-citationSurfing vortex rings for energy-efficient propulsion
Gunnarson, P. J., & Dabiri, J. O. (2025) · PNAS Nexus · doi:10.1093/pnasnexus/pgaf031 · Cited by 6
Cao, Q., Wang, R., Huang, S. H., Zhang, T., Yin, B., Tan, M. H., & Wang, S. (2025) · IEEE/ASME Transactions on Mechatronics · doi:10.1109/tmech.2025.3579529 · Cited by 4
“With the development of and AI technology [26], deep reinforcement learning (DRL) provides an efficient solution to such a gait optimization based difficult task [27], [28], [29].”
Zhou, Y., Juan, R., Li, Y., Wang, T., Guo, W., Yang, M., & Gao, Z. (2025) · Ocean Engineering · doi:10.1016/j.oceaneng.2025.122445 · Cited by 4
Fault-tolerant heading control for a bionic gliding robotic dolphin with flipper faults
Zhang, Y., Wu, Z., Wang, J., Li, S., & Tan, M. H. (2024) · Ocean Engineering · doi:10.1016/j.oceaneng.2024.117485 · Cited by 2
Movement ecology of gelatinous zooplankton: approaches, challenges and future directions
Gemmell, B. J., Colin, S. P., & Costello, J. H. (2025) · Journal of Experimental Biology · doi:10.1242/jeb.247987 · Cited by 3
Inverse kinematics calculation in IEC 61131-3 Structured Text: KinIn
Cam, J. R. M., & López, F. M. S. (2025) · SoftwareX · doi:10.1016/j.softx.2025.102252 · Cited by 2
Goal-Oriented Control Strategies for Soft Growing Robots
Huang, W., Li, P., Zhang, Z., Wang, Z., Zhou, D., & Li, L. (2025) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2025.3598668 · Cited by 1
“Reinforcementlearning,asanonlinearself-modelingmethod, hasdemonstratedpowerfulcapabilitiesincontrolandplanning [26],[27],[28],[29].”
Joint Multi-Scale Channel Attention and Multi-Perception Head for Underwater Object Detection
Guo, C., Yang, Y. Z., Jiang, Y., Zhang, X., Zhao, X., & Wang, J. (2025) · Big Data Mining and Analytics · doi:10.26599/bdma.2025.9020030 · Cited by 2
Wang, W., Ma, Y., Zhu, G., Jiang, D., Hu, S., Wu, Y., Han, B., & Zhao, Y. (2025) · IEEE Sensors Journal · doi:10.1109/jsen.2025.3613254 · Cited by 1
“W ITH the development of autonomous technology, autonomous marine vehicles (AMVs) such as autonomous surface vehicles (ASVs) [1] and autonomous underwater vehicles (AUVs) [2] have shown great potential for applications in the fields of ocean resource exploration and maritime search and rescue [3].”
Cognitive embodied learning for anomaly active target tracking
Wu, Q., Li, J., Zhou, F., Ji, J., Wang, H., Liang, H., & Ma, K. (2025) · Communications Engineering · doi:10.1038/s44172-025-00556-6 · Cited by 6
A High-Sensitive Bio-Inspired Differential Pressure Sensor for Vortex Source Localization
Yang, X., Cao, Y., Gong, Z., Zhou, W., Ye, H., Ma, Z., & Jiang, Y. (2025) · IEEE Sensors Journal · doi:10.1109/jsen.2025.3641566 · Cited by 1
“H YDRODYNAMIC perception plays a pivotal role in providing essential environmental sensing abilities for underwater robotics, carrying profound implications for marine exploration [1], [2], [3], environmental surveillance [4], [5], subaquatic operations [6], [7], [8], and beyond.”
A simulation method for underwater acoustic positioning system of deep-sea neutrino telescope
Liu, J., Wei, H., Tian, X., Xu, D. L., Mei, H., & Xiang, X. (2025) · Ocean Engineering · doi:10.1016/j.oceaneng.2025.124038 · Cited by 1
Risk-Aware Robot Control in Dynamic Environments Using Belief Control Barrier Functions
Han, S., Vahs, M., & Tůmová, J. (2025) · IEEE Conference on Decision and Control (CDC) · doi:10.1109/cdc57313.2025.11312006 · Cited by 3
“This non-parametric form can capture distributions that could be multimodal and skewed [14]–[16].”
Meng, A., Hong, L., Feng, Y., Ling, Z., Li, X., & Hu, L. (2026) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2026.3664176 · Cited by 1
“[1] presented an RL-based controller of an autonomous surface vehicle that tracks an underwater mobile target based on range-only measurements.”
Deng, Y., Chen, L., He, H., Liu, F., Li, F., Xu, Y., Yan, J., & Zhao, D. (2026) · Measurement Science and Technology · doi:10.1088/1361-6501/ae74f0 · Cited by 1
Buoy and Winch Collaborative Control System Based on Deep Reinforcement Learning
Gu, Y., Ni, J., Geng, Z., Zhao, B., & Yang, H. (2025) · Journal of Marine Science and Engineering · doi:10.3390/jmse13020326 · Cited by 1
Implementation of two-roller scheduling path planning under road construction scenarios
Xu, T., Liu, J., & Zhou, Z. (2025) · Scientific Reports · doi:10.1038/s41598-025-91107-8 · Cited by 2
“…are summarized below: Firstly, the effective design of metrics and constraints has driven the development of robotics in various fields 20–22 , including obstacle avoidance and robot models, as well as evaluation metrics such as energy consumption, path smoothness, path length, and path…”
Underwater target 6D State Estimation via UUV Attitude Enhance Observability
Liu, F., Jia, C., Zhang, N., Yuan, S., & Su, R. (2025) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros60139.2025.11246376 · Cited by 1
“Meanwhile, reinforcement learning (RL) [3] has shown promise in adaptive decision-making for UUV path planning, enabling agents to learn optimal tracking strategies through interaction with the environment.”
SpineWave: Harnessing Fish Spinal Kinematics in Biomimetic Robotics for Enhanced Aquatic ExplorationPreprint
Li, W., He, Q., Dai, G., Chen, H., Liu, Q., Tian, X., You, J., Cui, W., Triantafyllou, M. S., & Fan, D. (2024) · Research Square · doi:10.21203/rs.3.rs-4544520/v1
Learning-based optimization method for attitude control of jet-driven spacecraft
Du, D., Qi, N., Cheng, H., Han, M., Zhou, R., & Liu, Y. (2024) · Aerospace Frontiers Conference (AFC) · doi:10.1117/12.3032458
“…technology, reinforcement learning (RL)-based control methods have been applied to various robotic systems such as racing drones [12,13] , bipedal robots [14] , and underwater robots [15] , and show superior performance in agility, robustness, and adaptability over traditional control methods.”
Liu, H., Yu, M., Li, Z., Shen, X., Xia, H., Xie, X., & Tang, Q. (2024) · International Conference on Advanced Robotics and Mechatronics (ICARM) · doi:10.1109/icarm62033.2024.10715873
“There have been numerous studies on maneuvering strategies to achieve better observability conditions [6-8].”
BackgroundBaba, A., & Alothman, B. (2024) · International Symposium on Innovative Approaches in Smart Technologies (ISAS) · doi:10.1109/isas64331.2024.10845482
“Range measurements are crucial in tracking missions, showing effectiveness as explored in (13).”
Efficient deployment of multiple jumping robots in uneven terrains using deep reinforcement learning
Zhou, Q., Li, G., Xu, Y., Zhang, W., Peng, L., & Shi, Q. (2025) · Expert Systems with Applications · doi:10.1016/j.eswa.2025.129159
Global Interception Control for Marine Surface Vehicle on a Moving Target Whose Velocity is Unknown
Liu, S., & Guo, G. (2025) · Chinese Control and Decision Conference (CCDC) · doi:10.1109/ccdc65474.2025.11090306
“To this end, target interception for MSVs has garnered increasing attention, with approaches primarily divided into three categories: reinforcement learning [1], path planning [2], and kinematic guidance [3].”
An Adaptive AUV Hunting Strategy for High-Mobility Target With Hierarchical MADRL
Gan, W., Guo, K., & Qiao, L. (2025) · IEEE Transactions on Aerospace and Electronic Systems · doi:10.1109/taes.2025.3630151 · Cited by 2
“The meta-strategy encompasses the combined pattern (in the absence of hunter detection, a cruise mode is adopted, with forces generated following the symmetric Lévy distribution with the Mantegna algorithm [25].”
Vertical Profiling in Estuaries Using AUV NetworksPreprint
Bandara, C. T., & Tanner, H. G. (2025) · Research Square · doi:10.21203/rs.3.rs-8140687/v1
Yao, C., Wang, F., Zhang, W., & Xia, Y. (2026) · International Conference on Neural Networks, Information and Communication Engineering (NNICE) · doi:10.1109/nnice68970.2026.11466203
Improving Odometry-Aided Position and Orientation Systems Through Bilateral Endpoints Constraint
Bao, J., Zhao, J. B., Yang, Z.-X., & Qin, H. (2026) · IEEE Transactions on Industrial Electronics · doi:10.1109/tie.2026.3675034
“Representative applications include underground infrastructure mapping, tunnel and pipeline inspection, high-density forest surveying, and underwater terrain surveying [3], [4].”
Calorimetric differential pressure sensor with high sensitivity for hydrodynamic perception
Cao, Y., Ma, Z., Kang, H., Zhou, W., Yang, X., Ye, H., Zhang, X., Cai, J. W., & Jiang, Y. (2026) · Microsystems & Nanoengineering · doi:10.1038/s41378-026-01270-9 · Cited by 1
“To evaluate the stability of the sensor, the CDP sensor was subjected to multiple load-unload cycles at differential pressures of 20 and 60 Pa on days 1, 3, 5, and 7 of a one-week period.”
Autonomous Surface Vehicle Motion Planning with Differentiable Value Guided Diffusers
Wang, J., Liu, R., & Zhang, W. (2025) · China Automation Congress (CAC) · doi:10.1109/cac67268.2025.11487621
“…optimization objectives and associated path planning challenges, typically addressed through systematic heuristic frameworks [30], analytical models combining Cram´er-Rao lower bound analysis with model predictive control [33], [34], or data-driven reinforcement learning paradigms [1], [31].”
InfluentialROV Robust Identification Method for Underwater Degradation Modeling and Structural Priors
Zhang, Z. (2026) · International Conference on Electronics Technology and Artificial Intelligence (ETAI) · doi:10.1109/etai68332.2026.11485189
“Visual perception is essential for underwater robots in object recognition [2] and state perception, but underwater imaging differs greatly from terrestrial scenes and remains challenging.”
Basic Research on Physical Intelligent System for Extreme Environment
Wang, S., Du, X., Duan, H., Yu, H., Da, L., Pan, F., Liu, H., Liu, H., Cao, H., & Wu, F. (2026) · 中国科学基金 · doi:10.3724/bnsfc-2025-0154
Zhang, G., Yao, G., Li, J., & Tong, S. (2026) · IEEE Transactions on Automation Science and Engineering · doi:10.1109/tase.2026.3694179
“W ITH recent developments in computing, sensing, guidance and control, aerial robots such as quadrotor unmanned aerial vehicles (UAVs) and maritime robots such as unmanned surface vessels (USVs) have entered into ocean engineering applications with extraordinary versatility [1], [2], [3].”
Multi-AUV Underwater Cooperative Target Encirclement Via Quantum-Assisted Software-Defined MARL
Tian, J., Lin, C., Han, G., Zhu, S., Wang, Z., & Ma, Y. (2026) · IEEE Transactions on Mobile Computing · doi:10.1109/tmc.2026.3697504
“…underwater targets may exhibit dynamic and intelligent behaviors, which considerably increase the difficulty of successful encirclement [8]; 3) Real-time and adaptability requirement : AUV swarms must make rapid decisions and adapt strategies in real-time under high-dimensional state…”
Ma, D., Wang, J., Lv, S., Wang, Z., & Zheng, H. (2026) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2026.3700649
“While studies have validated its feasibility in individual USV navigation and localized conflict resolution [18], [19], [20], [21], direct application to multi-agent USV tra ffi c scenarios su ff ers from tra ffi c state non-stationarity [22].”
Scalable Many-to-Many AUV Hunt-Evasion Game via MADRL With Phased Objectives
Gan, W., Guo, K., & Qiao, L. (2026) · IEEE Transactions on Network Science and Engineering · doi:10.1109/tnse.2026.3704819
“During training, evaders adopt a two-phase behavior model: they move under a Lévy-distributed force [43], obstacle avoidance, and random disturbances before detecting hunters, and switch to a bio-inspired APF evasion strategy [5] afterward.”
Learning to traverse convective flows at moderate to high Rayleigh numbers
Xu, A., Wu, H.-L., Xu, B.-R., & Xi, H.-D. (2026) · Journal of Fluid Mechanics · doi:10.1017/jfm.2026.11802 · Cited by 1
Nerites Framework: An Open-Source ROS2 Gym Environment for Intelligent Underwater Robotics
Stefanopoulou, A., Kapsacheilis, D., Karatzinis, G. D., Michailidis, I., Attia, Y., Wanderlingh, F., Simetti, E., Kosmatopoulos, E. B., & Boutalis, Y. (2026) · Mediterranean Conference on Control and Automation (MED) · doi:10.1109/med70602.2026.11598133
“In parallel, RL has been increasingly explored for under-water robotic control problems, including depth regulation [20] [19], station keeping [9] [3], trajectory tracking [22] [16] [7] [23], and collision avoidance [21] [6] [1] [12].”
Luo, W., Zhou, Z., Wang, A., & Zheng, H. (2026) · MTS/IEEE OCEANS Conference (OCEANS) · doi:10.1109/oceans66983.2026.11617112
“Deep reinforcement learning (RL) has expanded this toolbox to path following, collision avoidance, offline learning, and adaptation to unseen hull configurations [5]–[7], with related marine tracking results reinforcing the value of learned long-horizon control under disturbances [8].”
Wang, J., Li, Y., Cui, H., Xu, H., & Yan, J. (2026) · Engineering Applications of Artificial Intelligence · doi:10.1016/j.engappai.2026.115891
Ahn, H., 이호영, Kang, B., Choi, J., Yoon, Y., & Kim, J. (2026) · IEEE Access · doi:10.1109/access.2026.3725709
“Many of them constructed agents with MLP-style neural networks, including deep deterministic policy gradient (DDPG)-based defense tactics, to neutralize torpedo attacks and generated elementary evasive maneuvers [19], [20].”
Zhang, K., Wang, S., Liu, J., Wang, T., Hu, F., He, W., Meng, Z., Wan, X., Xu, G., Cui, Y., et al. (2026) · Nano Energy · doi:10.1016/j.nanoen.2026.112355
Cao, Y., Ye, H., Gong, Z., Zhou, W., Luo, Y., Jin, B., Cai, J., Ma, Z., & Jiang, Y. (2026) · Small · doi:10.1002/smll.75701
HexaWorld: The Role of Grid Geometries in Constrained RL Exploration
Lombardi, G., Bianchi, L. A., & Schneider, R. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-39028-8_10
Zhan, W., Tian, Y., Zheng, F., Yu, J., & Huang, Y. (2026) · Journal of Marine Science and Engineering · doi:10.3390/jmse14181733
Pattern-Aware Task Assignment and Depth-Based Deconfliction for Multi-AUV Oceanic Search Operations
Kılıç, D. K. (2026) · Modelling—International Open Access Journal of Modelling in Engineering Science · doi:10.3390/modelling7050203
Kramar, V. A., Liashko, A., & Norenko, E. (2026) · Journal of Marine Science and Engineering · doi:10.3390/jmse14201895
A vision-based docking planning network for robust recovery of autonomous underwater vehicles
Yu, T., Xu, G., Wang, Y., Zhang, Y., Zhang, Q., Zeng, J., & Li, S. (2026) · Ocean Engineering · doi:10.1016/j.oceaneng.2026.128864
Metadata
- Authors
- Masmitja, I., Martin, M., O’Reilly, T., Kieft, B., Palomeras, N., Navarro, J., & Katija, K.
- Affiliations
- Institut de Ciències del Mar (ICM), CSIC, Barcelona 95062, Spain.
- Knowledge Engineering and Machine Learning Group, Universitat Politècnica de Catalunya, Barcelona Tech., Barcelona 08034, Spain.
- Monterey Bay Aquarium Research Institute (US)
- Universitat de Girona (ES)
- Institut de Ciències del Mar (ES)
- Venue
- Science Robotics
- Published
- 2023-07-12
- Volume
- 8
- Issue
- 80
- Pages
- eade7811-eade7811
- ISSN
- 2470-9476 (ISSN-L)
- OpenAlex
- W4385286939
- Cited by
- 96
- References
- 59
- Citations by year
- 2026: 29 · 2025: 44 · 2024: 23
- FWCI
- 11.66
- Type
- journal article
- Language
- en
- Open access
- green
- hdl.handle.net
- Licence
- cc-by-nc-nd
- Locations
- Science Robotics (publishedVersion)
- PubMed (publishedVersion)
- UPCommons institutional repository (Universitat Politècnica de Catalunya) (submittedVersion, cc-by-nc-nd)
- DIGITAL.CSIC (Spanish National Research Council (CSIC)) (submittedVersion)
- Funders
- European Commission (CEX2019-000928-S, 893089)
- Barcelona Supercomputing Center
- Topics
- Physical Sciences › Engineering › Ocean Engineering › Underwater Vehicles and Communication Systems
- Physical Sciences › Computer Science › Artificial Intelligence › Reinforcement Learning in Robotics
- Physical Sciences › Engineering › Ocean Engineering › Maritime Navigation and Safety
- Keywords
- underwater target tracking, reinforcement learning, autonomous underwater vehicles, unmanned surface vehicles, path planning, deep reinforcement learning, oceanographic monitoring, marine conservation, field experiments
- MeSH terms
- Algorithms, Animals, Learning, Reinforcement, Psychology, Robotics
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.