SwarmGPT: Combining Large Language Models With Safe Motion Planning for Drone Swarm Choreography
Open accessAbstract
Drone swarm performances—synchronized, expressive aerial displays set to music—have emerged as a captivating application of modern robotics. Yet designing smooth, safe choreographies remains a complex task requiring expert knowledge. We present SwarmGPT, a language-based choreographer that leverages the reasoning power of large language models (LLMs) to streamline drone performance design. The LLM is augmented by a safety filter that ensures deployability by making minimal corrections when safety or feasibility constraints are violated. By decoupling high-level choreographic design from low-level motion planning, our system enables non-experts to iteratively refine choreographies using natural language without worrying about collisions or actuator limits. We validate our approach through simulations with swarms up to 200 drones and real-world experiments with up to 20 drones performing choreographies to diverse types of songs, demonstrating scalable, synchronized, and safe performances. Beyond entertainment, this work offers a blueprint for integrating foundation models into safety-critical swarm robotics applications.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Cites
SciPy 1.0: fundamental algorithms for scientific computing in Python
Virtanen, P., Gommers, R., Oliphant, T. E., Haberland, M., Reddy, T., Cournapeau, D., Burovski, E., Peterson, P., Weckesser, W., Bright, J., et al. (2020) · Nature Methods · doi:10.1038/s41592-019-0686-2 · Cited by 38,430 · see citing works
On the Opportunities and Risks of Foundation Models
Bommasani, R., Hudson, D. A., Adeli, E., Altman, R., Arora, S., von Arx, S., Bernstein, M. S., Bohg, J., Bosselut, A., Brunskill, E., et al. (2021) · arXiv · doi:10.48550/arxiv.2108.07258 · Cited by 2,676 · see citing works
Safe Learning in Robotics: From Learning-Based Control to Safe Reinforcement Learning
Brunke, L., Greeff, M., Hall, A. W., Yuan, Z., Zhou, S., Panerati, J., & Schoellig, A. P. (2022) · Annual Review of Control Robotics and Autonomous Systems · doi:10.1146/annurev-control-042920-020211 · Cited by 825 · see citing works
Self-citationProgPrompt: Generating Situated Robot Task Plans using Large Language Models
Singh, I., Blukis, V., Mousavian, A., Goyal, A., Xu, D., Tremblay, J., Fox, D., Thomason, J., & Garg, A. (2023) · 2023 IEEE International Conference on Robotics and Automation (ICRA) · doi:10.1109/icra48891.2023.10161317 · Cited by 613 · see citing works
PaLM-E: An Embodied Multimodal Language Model
Driess, D., Xia, F., Sajjadi, M. S. M., Lynch, C., Chowdhery, A., Ichter, B., Wahid, A., Tompson, J., Vuong, Q., Yu, T., et al. (2023) · arXiv · doi:10.48550/arxiv.2303.03378 · Cited by 591 · see citing works
Mixed integer programming for multi-vehicle path planning
Schouwenaars, T., Moor, B. D., Féron, É., & How, J. P. (2001) · European Control Conference (ECC) · doi:10.23919/ecc.2001.7076321 · Cited by 581 · see citing works
ChatGPT for Robotics: Design Principles and Model Abilities
Vemprala, S. H., Bonatti, R., Bucker, A., & Kapoor, A. (2024) · IEEE Access · doi:10.1109/access.2024.3387941 · Cited by 570 · see citing works
Crazyswarm: A large nano-quadcopter swarm
Preiss, J. A., Hönig, W., Sukhatme, G. S., et al. (2017) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra.2017.7989376 · Cited by 432 · see citing works
Visual Language Maps for Robot Navigation
Huang, C., Mees, O., Zeng, A., & Burgard, W. (2023) · 2023 IEEE International Conference on Robotics and Automation (ICRA) · doi:10.1109/icra48891.2023.10160969 · Cited by 376 · see citing works
Augugliaro, F., Schoellig, A. P., & D’Andrea, R. (2012) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros.2012.6385823 · Cited by 328 · see citing works
Self-citationInner Monologue: Embodied Reasoning through Planning with Language Models
Huang, W., Xia, F., Xiao, T., Chan, H., Liang, J., Florence, P., Zeng, A., Tompson, J., Mordatch, I., Chebotar, Y., et al. (2022) · arXiv · doi:10.48550/arxiv.2207.05608 · Cited by 287 · see citing works
Foundation models in robotics: Applications, challenges, and the future
Firoozi, R., Tucker, J., Tian, S., Majumdar, A., Sun, J., Liu, W., Zhu, Y., Song, S., Kapoor, A., Hausman, K., et al. (2024) · The International Journal of Robotics Research · doi:10.1177/02783649241281508 · Cited by 267 · see citing works
Luis, C. E., Vukosavljev, M., & Schoellig, A. P. (2020) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2020.2964159 · Cited by 264 · see citing works
Self-citationConceptGraphs: Open-Vocabulary 3D Scene Graphs for Perception and Planning
Gu, Q., Kuwajerwala, A., Morin, S., Jatavallabhula, K. M., Sen, B., Agarwal, A., Rivera, C., Paul, W., Ellis, K., Chellappa, R., et al. (2024) · 2024 IEEE International Conference on Robotics and Automation (ICRA) · doi:10.1109/icra57147.2024.10610243 · Cited by 255 · cited 1× · see citing works
“Their semantic understanding and reasoning capabilities have been increasingly applied in robotics—for example, in contextual navigation [8], [9], affordance-driven grasping and manipulation [10], [11], and guiding reward design or data collection for skill learning [12].”
Panerati, J., Zheng, H., Zhou, S., Xu, J. Y., Prorok, A., & Schoellig, A. P. (2021) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros51168.2021.9635857 · Cited by 210 · see citing works
Self-citationLuis, C. E., & Schoellig, A. P. (2019) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2018.2890572 · Cited by 161 · see citing works
Self-citationBy leaps and bounds: An exclusive look at how Boston dynamics is redefining robot agility
Guizzo, E. (2019) · IEEE Spectrum · doi:10.1109/mspec.2019.8913831 · Cited by 125 · see citing works
Physically Grounded Vision-Language Models for Robotic Manipulation
Gao, J., Sarkar, B., Xia, F., Xiao, T. G., Wu, J., Ichter, B., Majumdar, A., & Sadigh, D. (2024) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra57147.2024.10610090 · Cited by 119 · see citing works
VIMA: General Robot Manipulation with Multimodal Prompts
Jiang, Y., Gupta, A., Zhang, Z., Wang, G., Dou, Y., Chen, Y., Fei-Fei, L., Anandkumar, A., Zhu, Y., & Fan, L. (2022) · arXiv · doi:10.48550/arxiv.2210.03094 · Cited by 95 · see citing works
Distributed Predictive Drone Swarms in Cluttered Environments
Soria, E., Schiano, F., & Floreano, D. (2021) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2021.3118091 · Cited by 75 · see citing works
Downwash-aware trajectory planning for large quadrotor teams
Preiss, J. A., Hönig, W., Ayanian, N., et al. (2017) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros.2017.8202165 · Cited by 69 · see citing works
LATTE: LAnguage Trajectory TransformEr
Bucker, A., Figueredo, L., Haddadin, S., Kapoor, A., Ma, S., Vemprala, S. H., & Bonatti, R. (2023) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra48891.2023.10161068 · Cited by 57 · see citing works
Synchronizing the motion of a quadrocopter to music
Schoellig, A. P., Augugliaro, F., Lupashin, S., et al. (2010) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/robot.2010.5509755 · Cited by 50 · see citing works
Self-citationAutonomous robot dancing driven by beats and emotions of music
Xia, G., Tay, J., Dannenberg, R. B., et al. (2012) · Adaptive Agents and Multi-Agents Systems · doi:10.5555/2343576.2343605 · Cited by 49
Dance of the Flying Machines: Methods for Designing and Executing an Aerial Dance Choreography
Augugliaro, F., Schoellig, A. P., & D’Andrea, R. (2013) · IEEE Robotics & Automation Magazine · doi:10.1109/mra.2013.2275693 · Cited by 39 · see citing works
Self-citationFeasiblity of motion primitives for choreographed quadrocopter flight
Schöllig, A., Hehn, M., Lupashin, S., et al. (2011) · Proceedings of the ... American Control Conference/Proceedings of the American Control Conference · doi:10.1109/acc.2011.5991482 · Cited by 39 · see citing works
RoboGroove: Creating Fluid Motion for Dancing Robotic Arms
Rogel, A., Savery, R. J., Yang, N., et al. (2022) · International Conference on Movement and Computing (MOCO) · doi:10.1145/3537972.3537985 · Cited by 25 · see citing works
Scaling Up and Distilling Down: Language-Guided Robot Skill Acquisition
Ha, H., Florence, P., & Song, S. (2023) · arXiv · doi:10.48550/arxiv.2307.14535 · Cited by 24 · see citing works
So You Think You Can Dance? Rhythmic Flight Performances with Quadrocopters
Schoellig, A. P., Siegel, H., Augugliaro, F., et al. (2014) · Springer eBooks · doi:10.1007/978-3-319-03904-6_4 · Cited by 22 · see citing works
Self-citationFast and In Sync: Periodic Swarm Patterns for Quadrotors
Du, X., Luis, C. E., Vukosavljev, M., et al. (2019) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra.2019.8794017 · Cited by 21 · see citing works
Adajania, V. K., Zhou, S., Singh, A. K., et al. (2023) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra48891.2023.10161063 · Cited by 21 · see citing works
Self-citationSayTap: Language to Quadrupedal Locomotion
Tang, Y., Yu, W., Tan, J., et al. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2306.07580 · Cited by 18
Language Embedded Radiance Fields for Zero-Shot Task-Oriented GraspingPreprint
Rashid, A., Sharma, S., Kim, C. M., et al. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2309.07970 · Cited by 12 · see citing works
Robot Dance Generation with Music Based Trajectory Optimization
Boukheddimi, M., Harnack, D., Kumar, S., et al. (2022) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros47612.2022.9981462 · Cited by 9 · see citing works
Dynamically feasible and safe shape transitions for teams of aerial robots
Desai, A., Cappo, E. A., & Michael, N. (2016) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros.2016.7759807 · Cited by 8 · see citing works
Flagged first, then by citations
Research works printed but not linked: 4
- Brohan, 2022, Do as i can, not as i say: Grounding language in robotic affordances, 287
- 10.1109/ICRA.2019.8794017 · DOI
- Zitkovich, 2023, RT-2: Vision-language-action models transfer web knowledge to robotic control, 229, 2165
- Mller, 2015, Fundamentals of Music Processing - Audio, Analysis, Algorithms, Applications
Cited by
Still being checked.
- Authors
- Schuck, M., Dahanaggamaarachchi, D. O., Sprenger, B., Vyas, V., Zhou, S., & Schoellig, A. P.
- Affiliations
- Technical University of Munich (DE)
- Venue
- IEEE Robotics and Automation Letters
- Published
- 2025-11
- Volume
- 10
- Issue
- 11
- Pages
- 12237-12244
- ISSN
- 2377-3766 (ISSN-L)
- 2377-3774
- OpenAlex
- W4415002936
- Semantic Scholar
- dd5702e89842963a2431235215a64c6bf36d5955
- Cited by
- 7
- References
- 35
- Citations by year
- 2026: 7
- FWCI
- 2.11
- Type
- journal article
- Language
- en
- Open access
- hybrid
- doi.org
- Locations
- IEEE Robotics and Automation Letters (publishedVersion, cc-by)
- Topics
- Physical Sciences › Engineering › Mechanical Engineering › Modular Robots and Swarm Intelligence
- Physical Sciences › Computer Science › Computer Networks and Communications › Distributed Control Multi-Agent Systems
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Robotic Path Planning Algorithms
- Keywords
- large language models, motion planning, safety filtering, collision avoidance, aerial display, swarm robotics, actuator constraints, safety-critical robotics
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
