Efficient collective swimming by harnessing vortices through deep reinforcement learning
Open accessThe DOI matches its Crossref record; 41 of the 41 references it cites were found in a registry or an index.
Abstract
Significance: Can fish reduce their energy expenditure by schooling? We answer affirmatively this longstanding question by combining state-of-the-art direct numerical simulations of the 3D Navier–Stokes equations with reinforcement learning, using recurrent neural networks with long short-term memory cells to account for the unsteadiness of the flow field. Surprisingly, we find that swimming behind a leader is not always associated with energetic benefits for the follower. In turn, we demonstrate that fish can improve their sustained propulsive efficiency by placing themselves at appropriate locations in the wake of other swimmers and intercepting their wake vortices judiciously. The results show that autonomous, “smart” swimmers may exploit unsteady flow fields to reap substantial energetic benefits and have promising implications for robotic swarms.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“The V-RACER algorithm has proven suitable for control problems in complex flow fields34,118.”
From Sensing flow gradients is necessary for learning autonomous underwater navigation (2025) · cites this paper as [34]
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Human-level control through deep reinforcement learning
Mnih, V., Kavukcuoglu, K., Silver, D., Rusu, A. A., Veness, J., Bellemare, M. G., Graves, A., Riedmiller, M., Fidjeland, A. K., Ostrovski, G., et al. (2015) · Nature · doi:10.1038/nature14236 · Cited by 30,357 · cited 1×
“Here, we use deep reinforcement learning [deep RL (23)] to discover such strategies for two autonomous and self-propelled swimmers and elucidate the physical mechanisms that enable efficient and sustained coordinated swimming.”
MethodReinforcement Learning: An Introduction
Sutton, R. S., & Barto, A. G. (1998) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.1998.712192 · Cited by 27,225
Reinforcement Learning: An Introduction
(2005) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.2004.842673 · Cited by 24,496
Eddies, streams, and convergence zones in turbulent flows
Hunt, J. C. R., Wray, A. A., & Moin, P. (1988) · NASA Technical Reports Server (NASA) · Cited by 3,634
Functions of Shoaling Behaviour in Teleosts
Pitcher, T. J. (1986) · The Behaviour of Teleost Fishes · doi:10.1007/978-1-4684-8261-4_12 · Cited by 962
Fish Exploiting Vortices Decrease Muscle Activity
Liao, J. C., Beal, D. N., Lauder, G. V., & Triantafyllou, M. S. (2003) · Science · doi:10.1126/science.1088295 · Cited by 921
Hydromechanics of Fish Schooling
WEIHS, D. (1973) · Nature · doi:10.1038/241290a0 · Cited by 689
The lateral line can mediate rheotaxis in fish
Montgomery, J. C., Baker, C. F., & Carton, A. G. (1997) · Nature · doi:10.1038/40135 · Cited by 553 · cited 1×
“Fish sense and navigate in complex flow fields full of mechanical energy that is distributed across multiple scales by vortices generated by obstacles and other swimming organisms (11, 12).”
BackgroundBorazjani, I., & Sotiropoulos, F. (2008) · Journal of Experimental Biology · doi:10.1242/jeb.015644 · Cited by 512
The Kármán gait: novel body kinematics of rainbow trout swimming in a vortex street
Liao, J. C., Beal, D. N., Lauder, G. V., & Triantafyllou, M. S. (2003) · Journal of Experimental Biology · doi:10.1242/jeb.00209 · Cited by 509 · cited 3×
“Such vortex-induced benefits have been observed in trout, which curtail muscle use by capitalizing on energy injected in the flow by obstacles present in streams (13, 27).”
BackgroundResultSwimming and Flying in Nature: Volume 2
(1975) · Springer eBooks · doi:10.1007/978-1-4757-1326-8 · Cited by 481
Simulations of optimized anguilliform swimming
Kern, S., & Koumoutsakos, P. (2006) · Journal of Experimental Biology · doi:10.1242/jeb.02526 · Cited by 420
Self-citationAnimal Orientation Strategies for Movement in Flows
Chapman, J. W., Klaassen, R. H. G., Drake, V. A., Fossette, S., Hays, G. C., Metcalfe, J. D., Reynolds, A. M., Reynolds, D. R., & Alerstam, T. (2011) · Current Biology · doi:10.1016/j.cub.2011.08.014 · Cited by 298
From behavioural analyses to models of collective motion in fish schools
Lopez, U., Gautrais, J., Couzin, I. D., & Theraulaz, G. (2012) · Interface Focus · doi:10.1098/rsfs.2012.0033 · Cited by 279
Aerobic capacity influences the spatial position of individuals within fish schools
Killen, S. S., Marras, S., Steffensen, J. F., & McKenzie, D. J. (2012) · Proceedings of the Royal Society B: Biological Sciences · doi:10.1098/rspb.2011.1006 · Cited by 275
Herskin, J., & Steffensen, J. F. (1998) · Journal of Fish Biology · doi:10.1111/j.1095-8649.1998.tb00986.x · Cited by 266
Flow Navigation by Smart Microswimmers via Reinforcement Learning
Colabrese, S., Gustavsson, K., Celani, A., & Biferale, L. (2017) · Physical Review Letters · doi:10.1103/physrevlett.118.158004 · Cited by 237
Fish schooling as a basis for vertical axis wind turbine farm design
Whittlesey, R. W., Liska, S., & Dabiri, J. O. (2010) · Bioinspiration & Biomimetics · doi:10.1088/1748-3182/5/3/035005 · Cited by 209
Simple phalanx pattern leads to energy saving in cohesive fish schooling
Ashraf, I., Bradshaw, H., Ha, T.-T., Halloy, J., Godoy-Diana, R., & Thiria, B. (2017) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1706503114 · Cited by 204
HOW THE BODY CONTRIBUTES TO THE WAKE IN UNDULATORY FISH SWIMMING
MÜLLER, U. K., SMIT, J., STAMHUIS, E. J., & VIDELER, J. J. (2001) · Journal of Experimental Biology · doi:10.1242/jeb.204.16.2751 · Cited by 200
Optimal undulatory swimming for a single fish-like body and for a pair of interacting swimmers
Maertens, A. P., Gao, A., & Triantafyllou, M. S. (2017) · Journal of Fluid Mechanics · doi:10.1017/jfm.2016.845 · Cited by 200
A novel mechanism for mechanosensory-based rheotaxis in larval zebrafish
Oteiza, P., Odstrcil, I., Lauder, G., Portugues, R., & Engert, F. (2017) · Nature · doi:10.1038/nature23014 · Cited by 199
Aleyev, Y. G. (1977) · Springer eBooks · doi:10.1007/978-94-010-1324-6 · Cited by 194
Studying Turbulence Using Numerical Simulation Databases
nasa (2016) · Cited by 181
Herskin, J. (1998) · Journal of Fish Biology · doi:10.1006/jfbi.1998.0708 · Cited by 180
Optimal undulatory swimming for a single fish-like body and for a pair of interacting swimmersPreprint
Maertens, A., Gao, A., & Triantafyllou, M. S. (2016) · DSpace@MIT (Massachusetts Institute of Technology) · Cited by 171
Learning to soar in turbulent environments
Reddy, G., Celani, A., Sejnowski, T. J., & Vergassola, M. (2016) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1606075113 · Cited by 170
Synchronisation through learning for two self-propelled swimmers
Novati, G., Verma, S., Alexeev, D., Rossinelli, D., van Rees, W. M., & Koumoutsakos, P. (2017) · Bioinspiration & Biomimetics · doi:10.1088/1748-3190/aa6311 · Cited by 157 · cited 3×
“This consideration has motivated our present study, where we expand on our earlier work (33), combining RL with direct numerical simulations (DNSs) of the Navier– Stokes (NS) equations for self-propelled autonomous swimmers.”
BackgroundInfluentialSelf-citationThe hydrodynamic advantages of synchronized swimming in a rectangular pattern
Daghooghi, M., & Borazjani, I. (2015) · Bioinspiration & Biomimetics · doi:10.1088/1748-3190/10/5/056018 · Cited by 156
Fish schooling as a basis for vertical axis wind turbine farm design
Whittlesey, R. W., Liska, S., & Dabiri, J. O. (2010) · ScholarWorks@UMassAmherst (University of Massachusetts Amherst) · Cited by 147
Reinforcement Learning and Wavelet Adapted Vortex Methods for Simulations of Self-propelled Swimmers
Gazzola, M., Hejazialhosseini, B., & Koumoutsakos, P. (2014) · SIAM Journal on Scientific Computing · doi:10.1137/130943078 · Cited by 138
Self-citationLearning to school in the presence of hydrodynamic interactions
Gazzola, M., Tchieu, A. A., Alexeev, D., de Brauer, A., & Koumoutsakos, P. (2016) · Journal of Fluid Mechanics · doi:10.1017/jfm.2015.686 · Cited by 137
Self-citationBreeding Places and Migrations of the Eel
Schmidt, J. (1923) · Nature · doi:10.1038/111051a0 · Cited by 136
Biomimetic Survival Hydrodynamics and Flow Sensing
Triantafyllou, M. S., Weymouth, G. D., & Miao, J. (2016) · Annual Review of Fluid Mechanics · doi:10.1146/annurev-fluid-122414-034329 · Cited by 134
Some Hydrodynamical Aspects of Fish Schooling
Weihs, D. (1975) · Springer eBooks · doi:10.1007/978-1-4757-1326-8_16 · Cited by 123
Learning to school in the presence of hydrodynamic interactions
Gazzola, M., Tchieu, A., Alexeev, D., Brauer, A. D., & Koumoutsakos, P. (2016) · Repository for Publications and Research Data (ETH Zurich) · doi:10.3929/ethz-b-000113533 · Cited by 117
Self-citationBreder, C. M. (1965) · Zoologica scientific contributions of the New York Zoological Society · doi:10.5962/p.206663 · Cited by 80
Schooling reduces energy consumption in swimming male European eels, Anguilla anguilla L.
Burgerhout, E., Tudorache, C., Brittijn, S. A., Palstra, A. P., Dirks, R. P., & van den Thillart, G. E. E. J. M. (2013) · Journal of Experimental Marine Biology and Ecology · doi:10.1016/j.jembe.2013.05.015 · Cited by 67
Hydrodynamic Performance of Porpoises ( Stenella attenuata )
Lang, T. G., & Pryor, K. W. (1966) · Science · doi:10.1126/science.152.3721.531 · Cited by 61
Synchronised Swimming of Two Fish
Novati, G., Verma, S., Alexeev, D., Rossinelli, D., van Rees, W. M., & Koumoutsakos, P. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1610.04248 · Cited by 41
Self-citation(1966) · Deep Sea Research and Oceanographic Abstracts · doi:10.1016/0011-7471(76)90988-8 · Cited by 31
Research works printed but not linked: 11
- 7 Shaw E (1978) Schooling Fishes: The school, a truly egalitarian form of organization in which all members of the group are alike in influence, offers substantial benefits to its participants. Am. Sci. 66:166-175.
- 8 Pavlov DS, Kasumyan AO (2000) Patterns and mechanisms of schooling behavior in fish: A review. J. Ichthyol. 40:163-231.
- 35 Bertsekas DP, Bertsekas DP, Bertsekas DP, Bertsekas DP (1995) Dynamic programming and optimal control . (Athena scientific Belmont, MA) Vol. 1.
- 40 Verma S, Abbati G, Novati G, Koumoutsakos P (2017) Computing the force distribution on the surface of complex, deforming geometries using vortex methods and brinkman penalization. Int. J. Numer. Meth. Fluids 85(8):484-501.
- 42 Gholami A, Hill J, Malhotra D, Biros G (2015) AccFFT: A library for distributed-memory FFT on CPU and GPU architectures. arXiv preprint arXiv:1506.07933 . · arXiv
- 45 Bellman RE (2010) Dynamic Programming . (Princeton University Press, Princeton, NJ, USA).
- 46 van Hasselt H, Guez A, Silver D (2015) Deep reinforcement learning with double Q-learning. CoRR, abs/1509.06461 . · arXiv
- 48 Riedmiller M (2005) Neural fitted Q iteration - First experiences with a data efficient neural reinforcement learning method in Machine Learning: ECML 2005: Lecture Notes in Computer Science, vol 3720 , eds. Gama J, Camacho R, Brazdil PB, Jorge AM, Torgo L. (Springer Berlin Heidelberg, Berlin, Heidelberg), pp. 317-328.
- 49 Gers FA, Schmidhuber J, Cummins F (2000) Learning to forget: Continual prediction with LSTM. Neural Comput. 12(10):2451-2471.
- 50 Lin LJ (1992) Ph.D. thesis (Carnegie Mellon University, Pittsburgh, PA, USA).
- 51 Kingma DP, Ba J (2014) Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 [cs.LG] . · arXiv
Cited by
Machine Learning for Fluid Mechanics
Brunton, S. L., Noack, B. R., & Koumoutsakos, P. (2020) · Annual Review of Fluid Mechanics · doi:10.1146/annurev-fluid-010719-060214 · Cited by 3,063
“Generative adversarial networks (GANs) (Goodfellow et al. 2014) are also being used to infer dynamical systems from data (Wu et al. 2018).”
BackgroundMethodInfluentialSelf-citationRabault, J., Kuchta, M., Jensen, A., Réglade, U., & Cerardi, N. (2019) · Journal of Fluid Mechanics · doi:10.1017/jfm.2019.62 · Cited by 575
Deep learning methods for super-resolution reconstruction of turbulent flows
Liu, B., Tang, J., Huang, H., & Lu, X.-Y. (2020) · Physics of Fluids · doi:10.1063/1.5140772 · Cited by 359
Immersed Methods for Fluid–Structure Interaction
Griffith, B. E., & Patankar, N. A. (2020) · Annual Review of Fluid Mechanics · doi:10.1146/annurev-fluid-010719-060228 · Cited by 330
Machine learning for active matter
Cichos, F., Gustavsson, K., Mehlig, B., & Volpe, G. (2020) · Nature Machine Intelligence · doi:10.1038/s42256-020-0146-9 · Cited by 315
Metadata
- Authors
- Verma, S., Novati, G., & Koumoutsakos, P.
- Affiliations
- ETH Zurich (CH)
- Venue
- Proceedings of the National Academy of Sciences
- Publisher
- National Academy of Sciences
- Published
- 2018-05-21
- Volume
- 115
- Issue
- 23
- Pages
- 5849-5854
- ISSN
- 0027-8424 (ISSN-L)
- 1091-6490
- PMC
- PMC6003313
- OpenAlex
- W2785318639
- Cited by
- 10
- References
- 31
- Citations by year
- 2025: 2 · 2024: 2 · 2023: 3 · 2022: 1 · 2021: 1 · 2020: 1
- FWCI
- 0.41
- Type
- journal article
- Language
- en
- Open access
- green
- europepmc.org
- Locations
- Proceedings of the National Academy of Sciences (publishedVersion)
- Europe PMC (PubMed Central) (publishedVersion)
- PubMed Central (publishedVersion)
- PubMed (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Funders
- Schweizerischer Nationalfonds zur Förderung der Wissenschaftlichen Forschung (147675)
- Topics
- Physical Sciences › Engineering › Aerospace Engineering › Biomimetic flight and propulsion mechanisms
- Physical Sciences › Engineering › Computational Mechanics › Fluid Dynamics and Turbulent Flows
- Physical Sciences › Physics and Astronomy › Statistical and Nonlinear Physics › Model Reduction and Neural Networks
- Keywords
- collective swimming, fish schooling, wake vortices, energy conservation, propulsive efficiency, deep reinforcement learning, recurrent neural networks, long short-term memory, unsteady flow, body deformation, leader-follower interaction
- MeSH terms
- Animals, Behavior, Animal, Biomechanical Phenomena, Computer Simulation, Fishes, Learning, Models, Biological, Reinforcement, Psychology, Swimming, Spatial Navigation
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.