Student or educator? Sign up with your school email and get 6 months of Pro free.
CoChat

The DOI matches its Crossref record; 119 of the 119 references it cites were found in a registry or an index; 24 of the 51 sources its claims cite were checked for support.

Sensing flow gradients is necessary for learning autonomous underwater navigation

Open access
Jiao, Yusheng, Hang, Haotian, Merel, Josh, Kanso, Eva · 2025 · Nature Communications

Abstract

Aquatic animals are much better at underwater navigation than robotic vehicles. Robots face major challenges in deep water because of their limited access to global positioning signals and flow maps. These limitations, and the changing nature of water currents, support the use of reinforcement learning approaches, where the navigator learns through trial-and-error interactions with the flow environment. But is it feasible to learn underwater navigation in the agent's Umwelt, without any land references? Here, we tasked an artificial swimmer with learning to reach a specific destination in unsteady flows by relying solely on egocentric observations, collected through on-board flow sensors in the agent's body frame, with no reference to a geocentric inertial frame. We found that while sensing local flow velocities is sufficient for geocentric navigation, successful egocentric navigation requires additional information of local flow gradients. Importantly, egocentric navigation strategies obey rotational symmetry and are more robust in unfamiliar conditions and flows not experienced during training. Our work expands underwater robot-centric learning, helps explain why aquatic organisms have arrays of flow sensors that detect gradients, and provides physics-based guidelines for transfer learning of learned policies to unfamiliar and diverse flow environments.

Overview

CoChat Summary

TL;DR

Egocentric reinforcement learning can navigate an artificial swimmer across an unsteady wake without any inertial or geocentric reference, but only if the agent senses local flow gradients in addition to local flow velocity. These egocentric policies are rotationally invariant and transfer more robustly than geocentric policies to unfamiliar flow conditions, though both degrade when flow physics changes substantially.

Key findings

  • Egocentric RL using only body-frame flow velocity and target position `(Δx_b, Δy_b, u_b, v_b)` failed in all 10 CFD training instances; adding transverse flow gradients `(n·∇u_b, n·∇v_b)` succeeded in all 16 CFD and 4 vortex-street training instances, reaching equally high reward as geocentric policies. Longitudinal gradients `(t·∇u_b, t·∇v_b)` also enabled success.
  • Geocentric RL required explicit heading orientation: with `(Δx, Δy, θ, u, v)` it converged in all 17 CFD sessions and 4 vortex-street sessions, while a flow-blind agent failed.
  • Egocentric policies were rotationally invariant: as the CFD wake was rotated relative to the inertial frame, geocentric success dropped to nearly zero, whereas egocentric performance remained high.
  • Transfer tests showed egocentric policies generalized better at lower Reynolds numbers and at Re = 600 when trained at Re = 1000; both policies declined at higher Re or when Re changes altered wake stability. In grid tests, egocentric success was 90% vs 55% geocentric upstream of the training domain, while geocentric was 63% vs 47% downstream.

Methods

  • The swimmer moved at constant speed `V = 0.8U` through a cylinder wake simulated with CFD at Re = 400 and Re = 200–1000, or through a reduced-order inviscid von Kármán vortex-street model with `λ = 4D`, `A = 0.2D`, `Γ = 3UD`. The action controlled heading rate, not heading directly.
  • Deep RL used V-RACER with a 128 × 128 feedforward network; reward combined a sparse 200D bonus for reaching within 0.15D of the target and a dense per-timestep distance-change reward. Training episodes randomized target and agent positions in circular regions of radius 2D on opposite sides of the wake, plus random orientation and flow phase.
  • Evaluation used 1000 random test cases per policy; 105 policies were tested for 105,000 total tests. Additional sweeps included 198,000 tests for Re = 400-trained policies, 60,000 for Re = 1000-trained policies, and 34,020 tests per policy over a grid of initial positions and 36 orientations. Two-sample t-tests compared egocentric and geocentric success rates.

Limitations

  • Stated: both policies struggle when changes in Re substantially alter the nature of the wake; transfer is asymmetric and depends on the training flow. At higher Re, wake instability introduces novel physical challenges, and at lower Re both policies eventually fail. Egocentric policies fail downstream because flow gradients weaken and preferred directions become ambiguous; geocentric policies fail upstream because they confidently choose a single wrong preferred direction. Training over the entire flow domain performed worse than training over the limited domain, apparently due to a local optimum.
  • Stated: egocentric policy with only angular target information performed worse inside the training domain but generalized better outside it, with 56.54% success for full target position versus 79.30% for angular-only outside the training domain.
  • Evident from the text: results are simulation-only, with no physical robot or animal validation; the swimmer has constant speed, no memory or prediction, and senses flow gradients only locally rather than through distributed arrays.

Model-written summary. Check it against the paper before relying on it.

Integrity signals

Registry

Crossref · 10.1038/s41467-025-58125-6

Cites

119 of 119 references verified0 citation errors · 0 not checkable

Statistics

Nothing to recompute.

Figures

Fig. 1 Autonomous underwater navigation in unsteady flows. A Unsteady flow generated by a uniform freestream flow U past a cylinder of diameter D at Re = 400. A swimmer moving at constant speed V = 0.8U must navigate the wake to reach the target (black star). Motion planning using time-optimal control (black trajectory) requires prior knowledge of the entire flow field and its time evolution. Flow and visual sensory cues in (B) a terrestrial geocentric frame (ex, ey) or (C) a body-fixed egocentric frame (t, n). D To navigate across unsteady flows, we train, using Deep RL, a swimmer that senses the ambient flow and target location locally.
Fig. 1 Autonomous underwater navigation in unsteady flows. A Unsteady flow generated by a uniform freestream flow U past a cylinder of diameter D at Re = 400. A swimmer moving at constant speed V = 0.8U must navigate the wake to reach the target (black star). Motion planning using time-optimal control (black trajectory) requires prior knowledge of the entire flow field and its time evolution. Flow and visual sensory cues in (B) a terrestrial geocentric frame (ex, ey) or (C) a body-fixed egocentric frame (t, n). D To navigate across unsteady flows, we train, using Deep RL, a swimmer that senses the ambient flow and target location locally.CC BY-NC-ND · PubMed Central
Fig. 2 Learning underwater navigation using egocentric observations requires sensing flow gradients. We trained RL policies with geocentric and egocentric observations in two flow environments: high-fidelity CFD simulations and a von Kármán vortex street (VS) model. All policies underwent training of equal length (2 × 107 timesteps for CFD and 0.5 × 107 timesteps for VS). Learning curves represent the moving mean of cumulative rewards per episode, calculated over a window of 500 episodes. A Geocentric observations: sensing (Δx, Δy, θ) only, the flow-blind agent failed to learn in six instances of learning. Adding flow sensing abilities (u, v), the agent learned to navigate in CFD wake in all 17 instances of learning. Training in VS wake with the same observations succeeded in all four instances, with faster convergence. B Egocentric observations: sensing (Δxb, Δyb, ub, vb) only, the agent failed in CFD wake in all 10 instances of learning. Adding local flow gradients (n ⋅ ∇ ub, n ⋅ ∇ vb) resulted in successful learning in all 16 instances of learning in the CFD wake and 4 instances of learning in the VS wake. Success rate of (C) geocentric and (D) egocentric agents for each of the trained policies are evaluated over a distribution of 1000 randomly generated test conditions (Supplementary Fig. 4). The larger variance of success rates in CFD compared to VS wakes reflects that CFD flows are more challenging to navigate. Additional training with entire domain for initialization and training in CFD wake at Re = 1000 is provided in Supplementary Fig. 3.
Fig. 2 Learning underwater navigation using egocentric observations requires sensing flow gradients. We trained RL policies with geocentric and egocentric observations in two flow environments: high-fidelity CFD simulations and a von Kármán vortex street (VS) model. All policies underwent training of equal length (2 × 107 timesteps for CFD and 0.5 × 107 timesteps for VS). Learning curves represent the moving mean of cumulative rewards per episode, calculated over a window of 500 episodes. A Geocentric observations: sensing (Δx, Δy, θ) only, the flow-blind agent failed to learn in six instances of learning. Adding flow sensing abilities (u, v), the agent learned to navigate in CFD wake in all 17 instances of learning. Training in VS wake with the same observations succeeded in all four instances, with faster convergence. B Egocentric observations: sensing (Δxb, Δyb, ub, vb) only, the agent failed in CFD wake in all 10 instances of learning. Adding local flow gradients (n ⋅ ∇ ub, n ⋅ ∇ vb) resulted in successful learning in all 16 instances of learning in the CFD wake and 4 instances of learning in the VS wake. Success rate of (C) geocentric and (D) egocentric agents for each of the trained policies are evaluated over a distribution of 1000 randomly generated test conditions (Supplementary Fig. 4). The larger variance of success rates in CFD compared to VS wakes reflects that CFD flows are more challenging to navigate. Additional training with entire domain for initialization and training in CFD wake at Re = 1000 is provided in Supplementary Fig. 3.CC BY-NC-ND · PubMed Central
Fig. 3 Trajectories of trained agents in physical and flow observation spaces. Trajectories are shown in black (geocentric) and blue (egocentric) for the same initial conditions and target location. A Geocentric and (B) egocentric agents, trained in CFD wake at Re = 400, learn to enter, slalom inside, and exit the wake. Agents trained in the reduced VS flow representation succeed when tested in (C, D) VS wake and (E, F) CFD wake (Supplementary Movies 1 and 2). Corresponding trajectories in the spaces of flow observations are shown in the right panels of (A, C, E) and (B, D, F) for the geocentric and egocentric agents, respectively. The three stages of navigation are marked using dark red for wake entry, dark blue for wake exit, black (geocentric), and light blue (egocentric) for slaloming inside the wake.
Fig. 3 Trajectories of trained agents in physical and flow observation spaces. Trajectories are shown in black (geocentric) and blue (egocentric) for the same initial conditions and target location. A Geocentric and (B) egocentric agents, trained in CFD wake at Re = 400, learn to enter, slalom inside, and exit the wake. Agents trained in the reduced VS flow representation succeed when tested in (C, D) VS wake and (E, F) CFD wake (Supplementary Movies 1 and 2). Corresponding trajectories in the spaces of flow observations are shown in the right panels of (A, C, E) and (B, D, F) for the geocentric and egocentric agents, respectively. The three stages of navigation are marked using dark red for wake entry, dark blue for wake exit, black (geocentric), and light blue (egocentric) for slaloming inside the wake.CC BY-NC-ND · PubMed Central
Fig. 4 Flow observations and transfer from low- to high-fidelity flow representations. Geocentric and egocentric agents trained and tested in (A, B) CFD wake at Re = 400, (C, D) VS wake, and (E, F) trained in VS and tested in CFD wake. Green colormap shows the success in reaching a target based on the 1000 random test cases (Supplementary Fig. 4); sample trajectories from Fig. 3 are superimposed; red (flow velocity) and gray (velocity gradient) colormaps show the probability density function (p.d.f.) of flow observations for all 1000 test cases. P.d.f. plots of observations of relative target locations are provided in Supplementary Fig. 6.
Fig. 4 Flow observations and transfer from low- to high-fidelity flow representations. Geocentric and egocentric agents trained and tested in (A, B) CFD wake at Re = 400, (C, D) VS wake, and (E, F) trained in VS and tested in CFD wake. Green colormap shows the success in reaching a target based on the 1000 random test cases (Supplementary Fig. 4); sample trajectories from Fig. 3 are superimposed; red (flow velocity) and gray (velocity gradient) colormaps show the probability density function (p.d.f.) of flow observations for all 1000 test cases. P.d.f. plots of observations of relative target locations are provided in Supplementary Fig. 6.CC BY-NC-ND · PubMed Central
Fig. 5 Transfer to unfamiliar flows and Reynolds numbers. A Agents trained at Re = 400 and tested in the same wake under -30∘ misalignment between the wake and inertial frame: geocentric agent fails where the egocentric agent succeeds (Supplementary Movies 1). B With increasing misalignment, the success rate of the geocentric agent quickly drops to nearly zero; whereas the performance of the egocentric agent is invariant to such rotations. Geocentric and egocentric agents trained at Re = 400 succeed when tested at (C) Re = 200 and (D) Re = 1000 (Supplementary Movies 2). E Success rates of geocentric and egocentric agents trained in CFD wake at Re = 400 and tested across a range of Reynolds numbers are summarized using box plots, where the median, lower, and upper quartiles are indicated with horizontal bars, and outliers are marked by “×”. All 17 geocentric and 16 egocentric policies are included, each tested over 1000 test cases. P.d.f. plots of visual and flow observations collected at Re = 200, 400, and 1000 are provided in Supplementary Fig. 7. Agents trained at Re = 1000 succeed when tested at (F) Re = 1000 and (G) Re = 600. H Success rates of geocentric and egocentric agents trained in CFD wake at Re = 1000 and tested across a range of Reynolds numbers are summarized using the same box plot convention as in (C). All 5 geocentric and 5 egocentric policies are included, each tested over 1000 test cases. In (E, H) to evaluate the difference in performance between the egocentric and geocentric policies, a two-sample t-test is used119,120. The null hypothesis states that there is no significant difference in success rates. A smaller p-value indicates stronger evidence against the null hypothesis, suggesting a more significant difference in performance. * p < 0.05, ** p < 0.01, *** p < 0.001.
Fig. 5 Transfer to unfamiliar flows and Reynolds numbers. A Agents trained at Re = 400 and tested in the same wake under -30∘ misalignment between the wake and inertial frame: geocentric agent fails where the egocentric agent succeeds (Supplementary Movies 1). B With increasing misalignment, the success rate of the geocentric agent quickly drops to nearly zero; whereas the performance of the egocentric agent is invariant to such rotations. Geocentric and egocentric agents trained at Re = 400 succeed when tested at (C) Re = 200 and (D) Re = 1000 (Supplementary Movies 2). E Success rates of geocentric and egocentric agents trained in CFD wake at Re = 400 and tested across a range of Reynolds numbers are summarized using box plots, where the median, lower, and upper quartiles are indicated with horizontal bars, and outliers are marked by “×”. All 17 geocentric and 16 egocentric policies are included, each tested over 1000 test cases. P.d.f. plots of visual and flow observations collected at Re = 200, 400, and 1000 are provided in Supplementary Fig. 7. Agents trained at Re = 1000 succeed when tested at (F) Re = 1000 and (G) Re = 600. H Success rates of geocentric and egocentric agents trained in CFD wake at Re = 1000 and tested across a range of Reynolds numbers are summarized using the same box plot convention as in (C). All 5 geocentric and 5 egocentric policies are included, each tested over 1000 test cases. In (E, H) to evaluate the difference in performance between the egocentric and geocentric policies, a two-sample t-test is used119,120. The null hypothesis states that there is no significant difference in success rates. A smaller p-value indicates stronger evidence against the null hypothesis, suggesting a more significant difference in performance. * p < 0.05, ** p < 0.01, *** p < 0.001.CC BY-NC-ND · PubMed Central
Fig. 6 Transfer to locations outside the training domain and interpretation of RL policies. A Geocentric and (B) egocentric agents starting from initial conditions unseen during training: geocentric agents fail upstream of the target location but outperform egocentric agents downstream of the target (Supplementary Movie 3). Success rates and consumed time of (C, E) geocentric and (D, F) egocentric agents reaching a fixed target (*) starting anywhere in the wake (Green colormap) with 100% success of both policies within the training domain (black circle), 58% and 66% in favor of egocentric policy outside the training domain, and overall 60% and 68% success across the entire domain. Both policies fail downstream: solid lines marking failure of the geocentric policy align with the direction of the “time-optimal” strategy (Supplementary Fig. 2A) and those of the egocentric policy align with the direction of the “drift-optimal” strategy (Supplementary Fig. 2B). The field of “preferred orientations” defined by the stable fixed points of the average policy for (G) geocentric and (H) egocentric agents explains the behavior of the trained agent inside and outside the training domain. Preferred orientations that align with time-optimal and drift-optimal strategies are highlighted in green and orange, respectively.
Fig. 6 Transfer to locations outside the training domain and interpretation of RL policies. A Geocentric and (B) egocentric agents starting from initial conditions unseen during training: geocentric agents fail upstream of the target location but outperform egocentric agents downstream of the target (Supplementary Movie 3). Success rates and consumed time of (C, E) geocentric and (D, F) egocentric agents reaching a fixed target (*) starting anywhere in the wake (Green colormap) with 100% success of both policies within the training domain (black circle), 58% and 66% in favor of egocentric policy outside the training domain, and overall 60% and 68% success across the entire domain. Both policies fail downstream: solid lines marking failure of the geocentric policy align with the direction of the “time-optimal” strategy (Supplementary Fig. 2A) and those of the egocentric policy align with the direction of the “drift-optimal” strategy (Supplementary Fig. 2B). The field of “preferred orientations” defined by the stable fixed points of the average policy for (G) geocentric and (H) egocentric agents explains the behavior of the trained agent inside and outside the training domain. Preferred orientations that align with time-optimal and drift-optimal strategies are highlighted in green and orange, respectively.CC BY-NC-ND · PubMed Central

Verification

Claims the paper makes

14 Valid · 3 Not valid · 9 Not checked · 96 Own results and interpretations

45 more claims · 1 with their cited sources checked

Create a free account to see all 122 claims and to verify the 3 references still open.

See all claims — free account
  • It may state two separate claims as one. 58
  • Probably not a substantial claim. 19
  • The quote only partly states this claim. 2

Create a free account to see these claims.

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

Each contradiction between two passages listed here was confirmed by an independent check.

3 of 3

Citations

Cites

Show all 116 references

Research works printed but not linked: 7

  1. Hájek, P. Metamathematics of Fuzzy Logic, vol. 4 (Springer Science & Business Media, 2013).
  2. Sutton, R. S. & Barto, A. G. Reinforcement Learning: An Introduction (MIT Press, 2018).
  3. Saffman, P. G. Vortex Dynamics (Cambridge University Press, 1995).
  4. Kolmogoroff, A. Interpolation und extrapolation von stationaren zufalligen folgen. Izvestiya Rossiiskoi Akademii Nauk. Seriya Matematicheskaya 5, 3–14 (1941).
  5. Arnold, V. I. Ordinary Differential Equations (Springer Science & Business Media, 1992).
  6. Lienhard, J. H. et al. Synopsis of Lift, Drag, and Vortex Frequency Data for Rigid Circular Cylinders, vol. 300 (Technical Extension Service, Washington State University, 1966).
  7. Novati, G. & Koumoutsakos, P. Remember and forget for experience replay. In Proc. International Conference on Machine Learning, 4851–4860 (PMLR, 2019).

Cited by

See all 23

Metadata

Authors
Jiao, Y., Hang, H., Merel, J., & Kanso, E.
Affiliations
University of Southern California (US)
Corresponding author
Kanso, Eva
Venue
Nature Communications
Published
2025-03-28
Volume
16
Issue
1
Pages
3044-3044
ISSN
2041-1723 (ISSN-L)
OpenAlex
W4408953897
Cited by
23
References
116
Citations by year
2026: 19 · 2025: 4
FWCI
8.80
Type
journal article
Language
en
Open access
Licence
Locations
  • Nature Communications (publishedVersion, cc-by-nc-nd)
  • Europe PMC (PubMed Central) (publishedVersion, cc-by-nc-nd)
  • PubMed Central (publishedVersion, cc-by-nc-nd)
  • PubMed (publishedVersion)
  • RePEc: Research Papers in Economics (submittedVersion)
  • DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
Funders
  • United States Department of Defense | United States Navy | Office of Naval Research (N00014-22-1-2655)
  • National Science Foundation (IOS 2034043, CBET 210020)
Topics
  • Physical Sciences › Engineering › Ocean Engineering › Underwater Vehicles and Communication Systems
  • Physical Sciences › Computer Science › Artificial Intelligence › Reinforcement Learning in Robotics
  • Physical Sciences › Engineering › Aerospace Engineering › Biomimetic flight and propulsion mechanisms
Keywords
underwater navigation, flow gradients, egocentric navigation, unsteady flow, artificial swimmers, reinforcement learning, flow sensors, rotational symmetry, policy transfer, deep-sea robotics, flow velocity sensing

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

The claim verdicts this report counts are experimental: their accuracy is not yet measured.

A language model drew the claims from the paper on 2026-10-10, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).

A second model labelled each claim on 2026-10-10 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).

Cited sources are checked with a language model’s help (gpt-6-sol).

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: Sensing flow gradients is necessary for learning autonomous underwater navigation. https://app.cochat.ai/papers/10.1038/s41467-025-58125-6
The paper
Jiao, Y., Hang, H., Merel, J., & Kanso, E. (2025). Sensing flow gradients is necessary for learning autonomous underwater navigation. Nature Communications. https://doi.org/10.1038/s41467-025-58125-6

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.