CoChat

Dynamic robotic tracking of underwater targets using reinforcement learning

Open access
Masmitja, I., Martin, M., O’Reilly, T., Kieft, B., Palomeras, N., Navarro, J., Katija, K. · 2023 · Science Robotics

The DOI matches its Crossref record; 60 of the 60 references it cites were found in a registry or an index.

Abstract

To realize the potential of autonomous underwater robots that scale up our observational capacity in the ocean, new techniques are needed. Fleets of autonomous robots could be used to study complex marine systems and animals with either new imaging configurations or by tracking tagged animals to study their behavior. These activities can then inform and create new policies for community conservation. The role of animal connectivity via active movement of animals represents a major knowledge gap related to the distribution of deep ocean populations. Tracking underwater targets represents a major challenge for observing biological processes in situ, and methods to robustly respond to a changing environment during monitoring missions are needed. Analytical techniques for optimal sensor placement and path planning to locate underwater targets are not straightforward in such cases. The aim of this study was to investigate the use of reinforcement learning as a tool for range-only underwater target-tracking optimization, whose promising capabilities have been demonstrated in terrestrial scenarios. To evaluate its usefulness, a reinforcement learning method was implemented as a path planning system for an autonomous surface vehicle while tracking an underwater mobile target. A complete description of an open-source model, performance metrics in simulated environments, and evaluated algorithms based on more than 15 hours of at-sea field experiments are presented. These efforts demonstrate that deep reinforcement learning is a powerful approach that enhances the abilities of autonomous robots in the ocean and encourages the deployment of algorithms like these for monitoring marine biological systems in the future.

Overview

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1126/scirobotics.ade7811

Cites

60 of 60 references verified0 citation errors · 0 not checkable

Statistics

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Show all 60 references

Research works printed but not linked: 5

  1. R. S. Sutton A. G. Barto Reinforcement Learning: An Introduction (MIT Press ed. 2 2015).
  2. T. Haarnoja A. Zhou P. Abbeel S. Levine Soft actor-critic: Off-policy maximum entropy deep reinforcement learning with a stochastic actor in Proceedings of the 35th International Conference on (ICML 2018) (ICML 2018) vol. 5 pp. 2976–2989.
  3. J. K. Terry, B. Black, N. Grammel, M. Jayakumar, A. Hari, R. Sullivan, L. Santos, R. Perez, C. Horsch, C. Dieffendahl, N. L. Williams, Y. Lokesh, P. Ravi, PettingZoo: A standard API for multi-agent reinforcement learning. Adv. Neural Inf. Process. Syst. 18, 15032–15043 (2021).
  4. T. P. Lillicrap J. J. Hunt A. Pritzel N. Heess T. Erez Y. Tassa D. Silver D. Wierstra Continuous control with deep reinforcement learning in 4th International Conference on Learning Representations (ICLR 2016)–Conference Track Proceedings (ICLR 2016).
  5. K. Katija P. L. D. Roberts J. Daniels A. Lapides K. Barnard M. Risi B. Y. Ranaan B. G. Woodward J. Takahashi in 2021 IEEE Winter Conference on Applications of Computer Vision (WACV) (IEEE 2021) pp. 859–868.

Cited by

See all 96

Metadata

Authors
Masmitja, I., Martin, M., O’Reilly, T., Kieft, B., Palomeras, N., Navarro, J., & Katija, K.
Affiliations
  • Institut de Ciències del Mar (ICM), CSIC, Barcelona 95062, Spain.
  • Knowledge Engineering and Machine Learning Group, Universitat Politècnica de Catalunya, Barcelona Tech., Barcelona 08034, Spain.
  • Monterey Bay Aquarium Research Institute (US)
  • Universitat de Girona (ES)
  • Institut de Ciències del Mar (ES)
Venue
Science Robotics
Published
2023-07-12
Volume
8
Issue
80
Pages
eade7811-eade7811
ISSN
2470-9476 (ISSN-L)
OpenAlex
W4385286939
Cited by
96
References
59
Citations by year
2026: 29 · 2025: 44 · 2024: 23
FWCI
11.66
Type
journal article
Language
en
Open access
Licence
cc-by-nc-nd
Locations
  • Science Robotics (publishedVersion)
  • PubMed (publishedVersion)
  • UPCommons institutional repository (Universitat Politècnica de Catalunya) (submittedVersion, cc-by-nc-nd)
  • DIGITAL.CSIC (Spanish National Research Council (CSIC)) (submittedVersion)
Funders
  • European Commission (CEX2019-000928-S, 893089)
  • Barcelona Supercomputing Center
Topics
  • Physical Sciences › Engineering › Ocean Engineering › Underwater Vehicles and Communication Systems
  • Physical Sciences › Computer Science › Artificial Intelligence › Reinforcement Learning in Robotics
  • Physical Sciences › Engineering › Ocean Engineering › Maritime Navigation and Safety
Keywords
underwater target tracking, reinforcement learning, autonomous underwater vehicles, unmanned surface vehicles, path planning, deep reinforcement learning, oceanographic monitoring, marine conservation, field experiments
MeSH terms
Algorithms, Animals, Learning, Reinforcement, Psychology, Robotics

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: Dynamic robotic tracking of underwater targets using reinforcement learning. https://app.cochat.ai/papers/10.1126/scirobotics.ade7811
The paper
Masmitja, I., Martin, M., O’Reilly, T., Kieft, B., Palomeras, N., Navarro, J., & Katija, K. (2023). Dynamic robotic tracking of underwater targets using reinforcement learning. Science Robotics. https://doi.org/10.1126/scirobotics.ade7811

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.