Deep reinforcement learning enabling a BCFbot to learn various undulatory patterns
Open accessAbstract
In bio-inspired marine robots, one particular motion pattern is generally adopted to achieve benefits of that pattern. However, multiple gait patterns can be utilized together in a single biomimetic design to employ their benefits, as required. However, there is a lack of a unified control scheme that can be used to optimize and mimic undulatory patterns observed among different organisms in the body and/or caudal fin (BCF) category. Thus, central pattern generators (CPGs) were incorporated into a deep reinforcement learning (DRL) architecture to train a robot to develop various swimming gaits. The proposed framework can not only develop and optimize distinct motion patterns but also seamlessly and instantly switch between them. Oscillators integrated into a learning paradigm provide a bioinspired framework to systematically develop a variety of swimming gaits. The prototyped BCFbot has multiple joints, which makes it easy to realize more than one tail undulation patterns. Three different swimming patterns (anguilliform, sub-carangiform, and carangiform) were learned through simulation and then verified on a physical robot. Testing and comparison results show that the claimed benefits of the three benchmark motion patterns can be well realized using the developed robot and can be freely switched and optimized using the developed DRL mechanism. This should be the first attempt for achieving a multimotion pattern optimization and switching within a single BCFbot and demonstrati…
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Hybrid approaches outperform in versatility, validated by a sub-0.017s recovery and 98% resilience.
“Table 1 summarizes PFTC and AFTC trade-offs, with hybrid approaches outperforming in versatility, validated by a sub-0.017s recovery [18] and 98% resilience [17].”
From Hybrid Fault-Tolerant Control in Cooperative Robotics: Advances in Resilience and Scalability (2025) · cites this paper as [17]
Scaling reinforcement learning with a 0.01 exploration rate supports adaptability in large fleets.
“Scaling reinforcement learning with a 0.01 exploration rate supports adaptability in large fleets [17]”
From Hybrid Fault-Tolerant Control in Cooperative Robotics: Advances in Resilience and Scalability (2025) · cites this paper as [17]
Cites
Human-level control through deep reinforcement learning
Mnih, V., Kavukcuoglu, K., Silver, D., Rusu, A. A., Veness, J., Bellemare, M. G., Graves, A., Riedmiller, M., Fidjeland, A. K., Ostrovski, G., et al. (2015) · Nature · doi:10.1038/nature14236 · Cited by 30,357 · see citing works
MuJoCo: A physics engine for model-based control
Todorov, E., Erez, T., & Tassa, Y. (2012) · 2012 IEEE/RSJ International Conference on Intelligent Robots and Systems · doi:10.1109/iros.2012.6386109 · Cited by 4,767 · see citing works
Trust region policy optimization
Schulman, J., Levine, S., Moritz, P., Jordan, M. I., & Abbeel, P. (2015) · arXiv · doi:10.48550/arxiv.1502.05477 · Cited by 4,578 · see citing works
Review of fish swimming modes for aquatic locomotion
Sfakiotakis, M., Lane, D. M., & Davies, J. B. C. (1999) · IEEE Journal of Oceanic Engineering · doi:10.1109/48.757275 · Cited by 1,942 · see citing works
Central pattern generators for locomotion control in animals and robots: A review
Ijspeert, A. J. (2008) · Neural Networks · doi:10.1016/j.neunet.2008.03.014 · Cited by 1,850 · see citing works
2018 IEEE International Conference on Robotics and Automation (ICRA)
(2018) · doi:10.1109/icra36916.2018 · Cited by 1,368 · see citing works
2011 IEEE International Conference on Robotics and Automation
(2011) · doi:10.1109/icra15140.2011 · Cited by 1,228 · see citing works
Self-powered soft robot in the Mariana Trench
Li, G., Chen, X., Zhou, F., Liang, Y., Xiao, Y., Cao, X., Zhang, Z., Zhang, M., Wu, B., Yin, S., et al. (2021) · Nature · doi:10.1038/s41586-020-03153-z · Cited by 1,218 · see citing works
From Swimming to Walking with a Salamander Robot Driven by a Spinal Cord Model
Ijspeert, A. J., Crespi, A., Ryczko, D., & Cabelguen, J.-M. (2007) · Science · doi:10.1126/science.1138353 · Cited by 1,214 · see citing works
Note on the swimming of slender fish
Lighthill, M. J. (1960) · Journal of Fluid Mechanics · doi:10.1017/s0022112060001110 · Cited by 1,187 · see citing works
2021 IEEE International Conference on Robotics and Automation (ICRA)
(2021) · doi:10.1109/icra48506.2021 · Cited by 1,049 · see citing works
Triantafyllou, M. S., & Triantafyllou, G. S. (1995) · Scientific American · doi:10.1038/scientificamerican0395-64 · Cited by 1,007 · see citing works
Exploration of underwater life with an acoustically controlled soft robotic fish
Katzschmann, R. K., DelPreto, J., MacCurdy, R. B., & Rus, D. L. (2018) · Science Robotics · doi:10.1126/scirobotics.aar3449 · Cited by 874 · see citing works
Form, Function, and Locomotory Habits in Fish
Lindsey, C. C. (1978) · Fish physiology · doi:10.1016/s1546-5098(08)60163-6 · Cited by 602 · see citing works
2011 IEEE International Conference on Robotics and Automation
(2009) · IEEE Transactions on Haptics · doi:10.1109/toh.2009.52 · Cited by 540 · see citing works
Breder, C. M. (1926) · Zoologica scientific contributions of the New York Zoological Society · doi:10.5962/p.203769 · Cited by 514 · see citing works
Development of a Biomimetic Robotic Fish and Its Control Algorithm
Yu, J., Tan, M. H., Wang, S., & Chen, E. (2004) · IEEE Transactions on Systems Man and Cybernetics Part B (Cybernetics) · doi:10.1109/tsmcb.2004.831151 · Cited by 453 · see citing works
Central Pattern Generation of Locomotion: A Review of the Evidence
MacKay-Lyons, M. J. (2002) · Physical Therapy · doi:10.1093/ptj/82.1.69 · Cited by 442 · see citing works
Zhu, J., White, C., Wainwright, D. K., Santo, V. D., LAUDER, G. V., & Bart‐Smith, H. (2019) · Science Robotics · doi:10.1126/scirobotics.aax4615 · Cited by 360 · see citing works
Online Optimization of Swimming and Crawling in an Amphibious Snake Robot
Crespi, A., & Ijspeert, A. J. (2008) · IEEE Transactions on Robotics · doi:10.1109/tro.2008.915426 · Cited by 331 · see citing works
The Geometric Mechanics of Undulatory Robotic Locomotion
Ostrowski, J., & Burdick, J. W. (1998) · The International Journal of Robotics Research · doi:10.1177/027836499801700701 · Cited by 331 · see citing works
Fish-inspired robots: design, sensing, actuation, and autonomy—a review of research
Raj, A., & Thakur, A. (2016) · Bioinspiration & Biomimetics · doi:10.1088/1748-3190/11/3/031001 · Cited by 289 · see citing works
A Novel Robot Fish With Wire-Driven Active Body and Compliant Tail
Zhong, Y., Li, Z., & Du, R. (2017) · IEEE/ASME Transactions on Mechatronics · doi:10.1109/tmech.2017.2712820 · Cited by 202 · see citing works
Development and Motion Control of Biomimetic Underwater Robots: A Survey
Wang, R., Wang, S., Wang, Y., Cheng, L., & Tan, M. H. (2020) · IEEE Transactions on Systems Man and Cybernetics Systems · doi:10.1109/tsmc.2020.3004862 · Cited by 180 · see citing works
Gravish, N., & LAUDER, G. V. (2018) · Journal of Experimental Biology · doi:10.1242/jeb.138438 · Cited by 157 · see citing works
Kopman, V., & Porfiri, M. (2012) · IEEE/ASME Transactions on Mechatronics · doi:10.1109/tmech.2012.2222431 · Cited by 150 · see citing works
Convergence of undulatory swimming kinematics across a diversity of fishes
Santo, V. D., Goerig, E., Wainwright, D. K., Akanyeti, O., Liao, J. C., Castro‐Santos, T., & LAUDER, G. V. (2021) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2113206118 · Cited by 137 · see citing works
Flow-relative control of an underwater robot
Salumäe, T., & Kruusmaa, M. (2013) · Proceedings of the Royal Society A Mathematical Physical and Engineering Sciences · doi:10.1098/rspa.2012.0671 · Cited by 134 · see citing works
CPG Network Optimization for a Biomimetic Robotic Fish via PSO
Yu, J., Wu, Z., Wang, M., & Tan, M. H. (2015) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2015.2459913 · Cited by 107 · see citing works
Flowfield measurements in the wake of a robotic lamprey
Hultmark, M., Leftwich, M. C., & Smits, A. J. (2007) · Experiments in Fluids · doi:10.1007/s00348-007-0412-1 · Cited by 75 · see citing works
Underwater Target Tracking Control of an Untethered Robotic Fish With a Camera Stabilizer
Yu, J., Wu, Z., Yang, X., Yang, Y., & Zhang, P. (2020) · IEEE Transactions on Systems Man and Cybernetics Systems · doi:10.1109/tsmc.2019.2963246 · Cited by 73 · see citing works
2011 15th International Conference on Advanced Robotics (ICAR)
(2011) · doi:10.1109/icar16317.2011 · Cited by 71 · see citing works
Autonomous Optimization of Swimming Gait in a Fish Robot With Multiple Onboard Sensors
Wang, W., Gu, D., & Xie, G. (2017) · IEEE Transactions on Systems Man and Cybernetics Systems · doi:10.1109/tsmc.2017.2683524 · Cited by 70 · see citing works
Trajectory stabilization for a planar carangiform robot fish
Morgansen, K. A., Vela, P. A., & Burdick, J. W. (2003) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/robot.2002.1013449 · Cited by 69 · see citing works
Bio-Inspired Octopus Robot Based on Novel Soft Fluidic Actuator
Fraś, J., Noh, Y., Maciaś, M., Wurdemann, H. A., & Althoefer, K. A. (2018) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra.2018.8460629 · Cited by 64 · see citing works
Motion Planning and Control of a Swimming Machine
Saimek, S., & Li, P. Y. (2004) · The International Journal of Robotics Research · doi:10.1177/0278364904038366 · Cited by 61 · see citing works
Locomotion Learning for an Anguilliform Robotic Fish Using Central Pattern Generator Approach
Niu, X., Xu, J., Ren, Q., & Wang, Q. (2013) · IEEE Transactions on Industrial Electronics · doi:10.1109/tie.2013.2288193 · Cited by 59 · see citing works
Parametric Research of Experiments on a Carangiform Robotic Fish
Qin, Y. F., Han, Z. Y., Zhang, S., & Yang, J. (2008) · Journal of Bionic Engineering · doi:10.1016/s1672-6529(08)60012-8 · Cited by 58 · see citing works
Learning for Attitude Holding of a Robotic Fish: An End-to-End Approach With Sim-to-Real Transfer
Zheng, J., Zhang, T., Wang, C., Xiong, M., & Xie, G. (2021) · IEEE Transactions on Robotics · doi:10.1109/tro.2021.3098239 · Cited by 57 · see citing works
From Simulation to Reality: A Learning Framework for Fish-Like Robots to Perform Control Tasks
Zhang, T., Tian, R., Yang, H., Wang, C., Sun, J., Zhang, S., & Xie, G. (2022) · IEEE Transactions on Robotics · doi:10.1109/tro.2022.3181014 · Cited by 57 · see citing works
Toward a Novel Robotic Manta With Unique Pectoral Fins
Meng, Y., Wu, Z., Dong, H., Wang, J., & Yu, J. (2020) · IEEE Transactions on Systems Man and Cybernetics Systems · doi:10.1109/tsmc.2020.3034503 · Cited by 54 · see citing works
iSplash-II: Realizing fast carangiform swimming to outperform a real fish
Clapham, R. J., & Hu, H. (2014) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros.2014.6942692 · Cited by 53 · see citing works
Wu, Z., Yu, J., Tan, M. H., & Zhang, J. (2014) · Journal of Bionic Engineering · doi:10.1016/s1672-6529(14)60037-8 · Cited by 50 · see citing works
Reinforcement Learning of CPG-Regulated Locomotion Controller for a Soft Snake Robot
Liu, X., Onal, C. D., & Fu, J. (2023) · IEEE Transactions on Robotics · doi:10.1109/tro.2023.3286046 · Cited by 46 · see citing works
Efficient Cooperative Structured Control for a Multijoint Biomimetic Robotic Fish
Yan, S., Wu, Z., Wang, J., Tan, M. H., & Yu, J. (2020) · IEEE/ASME Transactions on Mechatronics · doi:10.1109/tmech.2020.3041506 · Cited by 44 · see citing works
Inexpensive, small AUVs for studying ice-covered polar environments
Singh, H., Maksym, T., Wilkinson, J., & Williams, G. D. (2017) · Science Robotics · doi:10.1126/scirobotics.aan4809 · Cited by 42 · see citing works
Performance study of a fish robot propelled by a flexible caudal fin
Low, K. H., Chong, C. W., & Zhou, C. (2010) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/robot.2010.5509848 · Cited by 40 · see citing works
The Kinematics and Dynamics of Undulatory Motion of a Tuna-Mimetic Robot
Masoomi, S. F., Gutschmidt, S., Chen, X., & Sellier, M. (2015) · International Journal of Advanced Robotic Systems · doi:10.5772/60059 · Cited by 34 · see citing works
Design and dynamic analysis of fish robot: PoTuna
Kim, E.-J., & Youm, Y. (2004) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/robot.2004.1302492 · Cited by 34 · see citing works
A Framework for Steering Dynamic Robotic Locomotion Systems
McIsaac, K., & Ostrowski, J. P. (2003) · The International Journal of Robotics Research · doi:10.1177/0278364903022002001 · Cited by 34 · see citing works
A bio-mimetic design and control of a fish-like robot using compliant structures
Daou, H. E., Salumäe, T., Ristolainen, A., Toming, G., Listak, M., & Kruusmaa, M. (2011) · International Conference on Advanced Robotics (ICAR) · doi:10.1109/icar.2011.6088645 · Cited by 25 · see citing works
ManyQuadrupeds: Learning a Single Locomotion Policy for Diverse Quadruped Robots
Shafiee, M., Bellegarda, G., & Ijspeert, A. J. (2024) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra57147.2024.10610155 · Cited by 24 · see citing works
Development of Fishlike Robot that Imitates Carangiform and Subcarangiform Swimming Motions
Fujiwara, S., & Yamaguchi, S. (2017) · Journal of Aero Aqua Bio-mechanisms · doi:10.5226/jabmech.6.1 · Cited by 24 · see citing works
Gao, H., Xiao, X., Qiu, L., Meng, M. Q. ‐., King, N. K. K., & Ren, H. (2021) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra48506.2021.9560747 · Cited by 23 · see citing works
Li, W., Wang, T., Wu, G., & Li, J. (2011) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra.2011.5979907 · Cited by 23 · see citing works
Locomotion Generation and Motion Library Design for an Anguilliform Robotic Fish
Niu, X., Xu, J., Ren, Q., & Wang, Q. (2013) · Journal of Bionic Engineering · doi:10.1016/s1672-6529(13)60221-8 · Cited by 22 · see citing works
Benchmarking Safe Deep Reinforcement Learning in Aquatic Navigation
Marchesini, E., Corsi, D., & Farinelli, A. (2021) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros51168.2021.9635925 · Cited by 17 · see citing works
Yao, Q., Meng, L., Zhang, Q., Zhao, J., Pajarinen, J., Wang, X., Li, Z., & Wang, C. (2023) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2023.3323893 · Cited by 15 · see citing works
Adaptation to environmental change using reinforcement learning for robotic salamander
Cho, Y., Manzoor, S., & Choi, Y. (2019) · Intelligent Service Robotics · doi:10.1007/s11370-019-00279-6 · Cited by 11 · see citing works
CPG-Actor: Reinforcement Learning for Central Pattern Generators
Campanaro, L., Gangapurwala, S., Martini, D. D., Merkt, W., & Havoutis, I. (2021) · Lecture notes in computer science · doi:10.1007/978-3-030-89177-0_3 · Cited by 10 · see citing works
Towards Autonomous Robotic Systems
(2022) · Lecture notes in computer science · doi:10.1007/978-3-031-15908-4 · Cited by 10 · see citing works
Proceedings of Symposium on Autonomous Underwater Vehicle Technology
(1996) · doi:10.1109/auv.1996 · Cited by 10 · see citing works
Effects of cross-flow fan on hydrodynamic and acoustic performance of underwater fan-wing thruster
Gao, T., Yang, L., Qiu, L., Tse, Z. T. H., & Ren, H. (2021) · Ocean Engineering · doi:10.1016/j.oceaneng.2021.110078 · Cited by 8 · see citing works
Training Dynamic Motion Primitives using Deep Reinforcement Learning to Control a Robotic Tadpole
Hameed, I., Chao, X., Navarro-Alarcón, D., & Jing, X. (2022) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros47612.2022.9981112 · Cited by 3 · see citing works
Self-citation
Flagged first, then by citations
1 of the 66 references couldn’t be matched to a known paper.
Cited by
Sorted by citations
Hybrid Fault-Tolerant Control in Cooperative Robotics: Advances in Resilience and Scalability
Urrea, C. (2025) · Actuators · doi:10.3390/act14040177 · Cited by 19 · see citing works
“…this synthesis examines broader trends affecting fault tolerance, integrating evidence of bioinspired and learning-based approaches [15,17–19]. prior reviews that focus on specific implementations, this synthesis examines broader trends affecting fault tolerance, integrating evidence…”
InfluentialAdaptive multimodal swimming gaits in a reconfigurable modular soft robotic fish
Wang, B., Li, L., Xu, M., Hu, N., Gao, W., Zhang, J., Yin, B., Xin, Z., & Yu, J. (2026) · Science Advances · doi:10.1126/sciadv.aea1299 · Cited by 13 · see citing works
Zhang, C., Qin, Z., He, J., Li, T., Hou, J., Zhou, Q., & Zhang, L. (2025) · Physics of Fluids · doi:10.1063/5.0291398 · Cited by 6 · see citing works
Sayahkarajy, M., & Witte, H. (2025) · Applied Sciences · doi:10.3390/app151810045
Hydrodynamic study of a novel surface standing-and-turning behavior of robotic dolphins
Lei, M., Gai, Q., Zou, T., & Xia, D. (2026) · Journal of Fluids and Structures · doi:10.1016/j.jfluidstructs.2026.104512
Koca, G. Ö., Korkmaz, D., Bal, C., Ay, M., & Akpolat, Z. H. (2026) · Biomimetics · doi:10.3390/biomimetics11080534
- Authors
- Hameed, I., Chao, X., Navarro-Alarcon, D., & Jing, X.
- Affiliations
- Hong Kong Polytechnic University (HK)
- City University of Hong Kong (HK)
- Corresponding author
- Jing, Xingjian
- Venue
- Ocean Engineering
- Publisher
- Elsevier BV
- Published
- 2025-03
- Volume
- 320
- Pages
- 120322
- ISSN
- 0029-8018 (ISSN-L)
- OpenAlex
- W4406255022
- Cited by
- 5
- References
- 66
- Citations by year
- 2026: 3 · 2025: 2
- FWCI
- 1.89
- Type
- journal article
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- Locations
- Ocean Engineering (publishedVersion, cc-by-nc)
- CityU Scholars (submittedVersion, cc-by-nc)
- Funders
- City University of Hong Kong (9380140)
- Topics
- Physical Sciences › Engineering › Aerospace Engineering › Biomimetic flight and propulsion mechanisms
- Physical Sciences › Engineering › Biomedical Engineering › Robotic Locomotion and Control
- Physical Sciences › Computer Science › Artificial Intelligence › Reinforcement Learning in Robotics
- Keywords
- undulatory locomotion, deep reinforcement learning, central pattern generator, gait transition, swimming gait, anguilliform locomotion
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
