Model-Based Reinforcement Learning for Ultrasound-Driven Autonomous Microrobots
Open accessPreprint — published as 10.1038/s42256-025-01054-2 · Paper page
The DOI matches its Crossref record; its reference list is not available.
Abstract
AI has catalyzed transformative advancements across multiple sectors, from medical diagnostics to autonomous vehicles, enhancing precision and efficiency. As it ventures into microrobotics, AI offer innovative solutions to the formidable challenge of controlling and manipulating microrobots, which typically operate within imprecise, remotely actuated systems—a task often too complex for human operators. We implement state-of-the-art model-based reinforcement learning for autonomous control of an ultrasound-driven microrobot learning from recurrent imagined environments. Our non-invasive, AI-controlled microrobot offers precise propulsion, which efficiently learns from images in data-scarce environments. Transitioning from a pre-trained simulation environment, we achieve sample-efficient collision avoidance and channel navigation, reaching a 90% success rate in target navigation across various channels within an hour of fine-tuning. Moreover, our model initially successfully generalized in 50% of tasks in new environments, improving to over 90% with 30 minutes of further training. Furthermore, we have showcased real-time manipulation of microrobots within complex vasculatures and across stationary and physiological flows, underscoring AI’s potential to revolutionize microrobotics in biomedical applications, potentially transforming medical procedures.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Figures










Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
No citation lists for this paper
No citation lists for this paper yet — checked again daily.
References 72 · Cited by 55
Research works printed but not linked: 64
- 2020, Deep learning of circulating tumour cells, Nat Mach Intell, 2, 124, 10.1038/s42256-020-0153-x · DOI
- 2022, Super-resolution generative adversarial networks of randomly-seeded fields, Nat Mach Intell, 4, 1165, 10.1038/s42256-022-00572-7 · DOI
- 2024, β-Variational autoencoders and transformers for reduced-order modelling of fluid flows, Nat Commun, 15, 1361, 10.1038/s41467-024-45578-4 · DOI · Paper page
- 10.1038/s41592-020-0792-1 · DOI · Paper page
- 2020, Deep Learning in Ultrasound Imaging, Proceedings of the IEEE, 108, 11, 10.1109/JPROC.2019.2932116 · DOI · Paper page
- 2023, Deep-learning-enabled brain hemodynamic mapping using resting-state fMRI, npj Digit. Med, 6, 1
- 10.1038/s41467-021-26216-9 · DOI · Paper page
- 2023, Champion-level drone racing using deep reinforcement learning, Nature, 620, 982, 10.1038/s41586-023-06419-4 · DOI · Paper page
- 10.1109/TITS.2021.3054625 · DOI · Paper page
- 2020, Machine learning for active matter, Nat Mach Intell, 2, 94, 10.1038/s42256-020-0146-9 · DOI · Paper page
- 2024, Machine learning for micro- and nanorobots, Nat Mach Intell, 6, 605, 10.1038/s42256-024-00859-x · DOI
- 10.1038/s44222-024-00232-y · DOI
- 2024, Medical Microrobots, Annual Review of Biomedical Engineering, 26, 561, 10.1146/annurev-bioeng-081523-033131 · DOI
- 10.1146/annurev-bioeng-010510-103409 · DOI · Paper page
- 2024, Synchronous and Fully Steerable Active Particle Systems for Enhanced Mimicking of Collective Motion in Nature, Advanced Materials, 36, 2304759, 10.1002/adma.202304759 · DOI
- 2020, Micro-rocket robot with all-optic actuating and tracking in blood, Light Sci Appl, 9, 84, 10.1038/s41377-020-0323-y · DOI · Paper page
- 2023, Multi-Wavelength Light-Responsive Metal–Phenolic Network-Based Microrobots for Reactive Species Scavenging, Advanced Materials, 35, 2210994, 10.1002/adma.202210994 · DOI
- 10.1038/ncomms1550 · DOI · Paper page
- 2020, Chemically programmable microrobots weaving a web from hormones, Nat Mach Intell, 2, 711, 10.1038/s42256-020-00248-0 · DOI
- 2024, Urease-powered nanobots for radionuclide bladder cancer therapy, Nat. Nanotechnol, 19, 554, 10.1038/s41565-023-01577-y · DOI · Paper page
- 10.1126/scirobotics.abd2823 · DOI · Paper page
- 2022, Multifunctional 4D-Printed Sperm-Hybrid Microcarriers for Assisted Reproduction, Advanced Materials, 34, 2204257, 10.1002/adma.202204257 · DOI
- 10.1063/1.3079655 · DOI · Paper page
- 2022, Artificial microtubules for rapid and collective transport of magnetic microcargoes, Nat Mach Intell, 4, 678, 10.1038/s42256-022-00510-7 · DOI
- 2022, Magnetic torque–driven living microrobots for increased tumor infiltration, Science Robotics, 7, eabo0665, 10.1126/scirobotics.abo0665 · DOI · Paper page
- 10.1021/nl900186w · DOI · Paper page
- 2018, Sperm-Hybrid Micromotor for Targeted Drug Delivery, ACS Nano, 12, 327, 10.1021/acsnano.7b06398 · DOI · Paper page
- 2024, On-Command Disassembly of Microrobotic Superstructures for Transport and Delivery of Magnetic Micromachines, Advanced Materials, 36, 2310084, 10.1002/adma.202310084 · DOI
- 10.1126/sciadv.abb5696 · DOI
- 2023, Reconfigurable self-assembly of photocatalytic magnetic microrobots for water purification, Nat Commun, 14, 6969, 10.1038/s41467-023-42674-9 · DOI
- 2016, Rotational manipulation of single cells and organisms using acoustic waves, Nat Commun, 7, 11085, 10.1038/ncomms11085 · DOI · Paper page
- 2020, Addressable Acoustic Actuation of 3D Printed Soft Robotic Microsystems, Advanced Science, 7, 2001120, 10.1002/advs.202001120 · DOI
- 2015, Selectively manipulable acoustic-powered microswimmers, Sci Rep, 5, 9744, 10.1038/srep09744 · DOI · Paper page
- 2020, Model-Free Control for Dynamic-Field Acoustic Manipulation Using Reinforcement Learning, IEEE Access, 8, 20597, 10.1109/ACCESS.2020.2969277 · DOI
- 10.1126/sciadv.aax3084 · DOI · Paper page
- 10.1073/pnas.1920099117 · DOI · Paper page
- 10.1038/nature19755 · DOI · Paper page
- 1996, Reinforcement Learning: A Survey, Journal of Artificial Intelligence Research, 4, 237, 10.1613/jair.301 · DOI · Paper page
- 10.1038/nature14236 · DOI · Paper page
- 10.1126/science.aar6404 · DOI · Paper page
- 10.1126/scirobotics.abd9285 · DOI · Paper page
- 2022, Autonomous environment-adaptive microrobot swarm navigation enabled by deep learning-based real-time distribution planning, Nat Mach Intell, 4, 480, 10.1038/s42256-022-00482-8 · DOI · Paper page
- 2024, Autonomous 3D positional control of a magnetic microrobot using reinforcement learning, Nat Mach Intell, 6, 92, 10.1038/s42256-023-00779-2 · DOI
- 2022, Smart Magnetic Microrobots Learn to Swim with Deep Reinforcement Learning, Advanced Intelligent Systems, 4, 2200023, 10.1002/aisy.202200023 · DOI
- 2024, SonoTransformers: Transformable acoustically activated wireless microscale machines, Proceedings of the National Academy of Sciences, 121, e2314661121, 10.1073/pnas.2314661121 · DOI
- 2023, An acoustically controlled helical microrobot, Science Advances, 9, eadh5260, 10.1126/sciadv.adh5260 · DOI
- 2024, Robot-assisted chirality-tunable acoustic vortex tweezers for contactless, multifunctional, 4-DOF object manipulation, Science Advances, 10, eadm7698, 10.1126/sciadv.adm7698 · DOI
- 10.1021/nn301312z · DOI · Paper page
- 2021, Ultrasound-activated ciliary bands for microrobotic systems inspired by starfish, Nat Commun, 12, 6455, 10.1038/s41467-021-26607-y · DOI · Paper page
- 10.1109/TASE.2023.3292885 · DOI
- 2023, Acoustic Manipulation of Particles in Microfluidic Chips with an Adaptive Controller that Models Acoustic Fields, Advanced Intelligent Systems, 5, 2300058, 10.1002/aisy.202300058 · DOI
- Konda, V. & Tsitsiklis, J. Actor-Critic Algorithms. in Advances in Neural Information Processing Systems vol. 12 (MIT Press, 1999).
- 10.48550/arXiv.1707.06347 · DOI · Paper page
- 1998, Soft Lithography, Angewandte Chemie International Edition, 37, 550, 10.1002/(SICI)1521-3773(19980316)37:5<550::AID-ANIE550>3.0.CO;2-G · DOI · Paper page
- 10.48550/arXiv.2304.02643 · DOI · Paper page
- Brockman, G. et al. OpenAI Gym. Preprint at http://arxiv.org/abs/1606.01540 (2016). · arXiv
- Kalyanakrishnan, S. et al. An Analysis of Frame-skipping in Reinforcement Learning. Preprint at http://arxiv.org/abs/2102.03718 (2021). · arXiv
- 10.1126/scirobotics.aba5726 · DOI · Paper page
- 2021, Bioinspired acousto-magnetic microswarm robots with upstream motility, Nat Mach Intell, 3, 116, 10.1038/s42256-020-00275-x · DOI · Paper page
- 2017, Neutrophil-inspired propulsion in a combined acoustic and magnetic field, Nat Commun, 8, 770, 10.1038/s41467-017-00845-5 · DOI · Paper page
- 2022, In vivo acoustic manipulation of microparticles in zebrafish embryos, Science Advances, 8, eabm2785, 10.1126/sciadv.abm2785 · DOI
- 2022, Direct sound printing, Nat Commun, 13, 1800, 10.1038/s41467-022-29395-1 · DOI
- 2023, Ultrasound trapping and navigation of microrobots in the mouse brain vasculature, Nat Commun, 14, 5889, 10.1038/s41467-023-41557-3 · DOI
- 2022, Real-time 3D optoacoustic tracking of cell-sized magnetic microrobots circulating in the mouse brain vasculature, Science Advances, 8, eabm9132, 10.1126/sciadv.abm9132 · DOI · Paper page
Metadata
- Authors
- Medany, M., Piglia, L., Achenbach, L., Mukkavilli, S. K., & Ahmed, D.
- Affiliations
- ETH Zurich (CH)
- IBM Research - Zurich (CH)
- Corresponding author
- Ahmed, Daniel
- Venue
- Nature Machine Intelligence
- Publisher
- openRxiv
- Published
- 2024-09-30
- Volume
- 7
- Issue
- 7
- Pages
- 1076-1090
- ISSN
- 2522-5839 (ISSN-L)
- PMC
- PMC12283351
- OpenAlex
- W4411668692
- Cited by
- 55
- References
- 72
- Citations by year
- 2026: 46 · 2025: 8 · 2024: 1
- FWCI
- 16.73
- Type
- journal article
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- Locations
- Nature Machine Intelligence (publishedVersion, cc-by)
- bioRxiv (Cold Spring Harbor Laboratory) (acceptedVersion, cc-by-nc-nd)
- Europe PMC (PubMed Central) (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- Repository for Publications and Research Data (ETH Zurich) (publishedVersion, cc-by)
- ETH Zürich Research Collection (cc-by)
- Topics
- Physical Sciences › Physics and Astronomy › Condensed Matter Physics › Micro and Nano Robotics
- Physical Sciences › Computer Science › Artificial Intelligence › Reinforcement Learning in Robotics
- Physical Sciences › Engineering › Biomedical Engineering › Soft Robotics and Applications
- Keywords
- model-based reinforcement learning, sample efficiency, collision avoidance, channel navigation, fine-tuning, generalization across environments, image-based learning
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Model-Based Reinforcement Learning for Ultrasound-Driven Autonomous Microrobots. https://app.cochat.ai/papers/10.1101/2024.09.28.615576
- The paper
- Medany, M., Piglia, L., Achenbach, L., Mukkavilli, S. K., & Ahmed, D. (2024). Model-Based Reinforcement Learning for Ultrasound-Driven Autonomous Microrobots. Nature Machine Intelligence. https://doi.org/10.1101/2024.09.28.615576
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.