Code as Policies: Language Model Programs for Embodied Control
Open accessAbstract
Large language models (LLMs) trained on code-completion have been shown to be capable of synthesizing simple Python programs from docstrings [1]. We find that these code-writing LLMs can be re-purposed to write robot policy code, given natural language commands. Specifically, policy code can express functions or feedback loops that process perception outputs (e.g., from object detectors [2], [3]) and parameterize control primitive APIs. When provided as input several example language commands (formatted as comments) followed by corresponding policy code (via few-shot prompting), LLMs can take in new commands and autonomously re-compose API calls to generate new policy code respectively. By chaining classic logic structures and referencing third-party libraries (e.g., NumPy, Shapely) to perform arithmetic, LLMs used in this way can write robot policies that (i) exhibit spatial-geometric reasoning, (ii) generalize to new instructions, and (iii) prescribe precise values (e.g., velocities) to ambiguous descriptions (‘faster’) depending on context (i.e., behavioral commonsense). This paper presents Code as Policies: a robot-centric formulation of language model generated programs (LMPs) that can represent reactive policies (e.g., impedance controllers), as well as waypoint-based policies (vision-based pick and place, trajectory-based control), demonstrated across multiple real robot platforms. Central to our approach is prompting hierarchical code-gen (recursively defining undefined functions), which can write more complex code and also improves state-of-the-art to solve 39.8% of problems on the HumanEval [1] benchmark. Code and videos are available at https://code-as-policies.github.io
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
LLM-generated robotic task plans are highly sensitive to prompt phrasing and vulnerable to adversarial manipulation.
“LLM-generated robotic task plans are highly sensitive to prompt phrasing (Liang et al., 2023) and vulnerable to adversarial manipulation (Zhang et al., 2024c).”
From Large Language Model Reasoning Failures (2026) · cites this paper as Liang et al., 2023
Appendix A, “Prompt Engineering” (page unavailable), says reliable code generation requires careful prompting and that ambiguous or inconsistent examples can produce unreliable responses. This supports sensitivity to prompt design, but it does not establish that robotic task plans are highly sensitive to phrasing specifically. The adversarial-manipulation clause is cited to Zhang et al., not Liang et al., and is not part of this verdict. (LLM, conf=0.82)
Experimental — accuracy not yet measuredNaive LLMs and VLMs generate plans step by step, lacking mechanisms to detect and correct earlier mistakes or execution errors.
“Naive LLMs and VLMs generate plans step by step, lacking mechanisms to detect and correct earlier mistakes or execution errors (Liang et al., 2023; Huang et al., 2022b; Duan et al., 2024).”
From Large Language Model Reasoning Failures (2026) · cites this paper as Liang et al., 2023
The Introduction describes earlier LLM robot approaches as planning sequences of steps and says that abstraction makes it difficult for LLMs to influence the perception–action feedback loop. The related-work discussion also describes a system that uses success-detector or VLM feedback to re-plan. The paper therefore supports the limitation for basic sequence-planning approaches, but does not establish that naive LLMs and VLMs generally cannot detect and correct earlier mistakes or execution errors. (LLM, conf=0.83)
Experimental — accuracy not yet measuredIncorporating feedback mechanisms or explicit error-handling strategies significantly reduces embodied errors.
“Incorporating feedback mechanisms or explicit error-handling strategies significantly reduces these errors (Liang et al., 2023; Wang et al., 2023a).”
From Large Language Model Reasoning Failures (2026) · cites this paper as Liang et al., 2023
The Abstract describes feedback loops for robot control, and Appendix A (Prompt Engineering) says hints or examples can fix some code-generation mistakes. Neither passage establishes that feedback or error-handling strategies significantly reduce embodied errors; the appendix also says attempted code editing produced inconsistent results. No page numbers are available in our copy. (LLM, conf=0.82)
Experimental — accuracy not yet measuredIn personalization and embodied control, agents author and revise executable policies, simulators, and skill libraries in response to environment feedback.
“In personalization and embodied control, agents author and revise executable policies, simulators, and skill libraries in response to environment feedback [9, 10, 32].”
From Code as Agent Harness (2026) · cites this paper as [10]
The paper describes LLMs writing executable robot policies and recursively defining functions that can accumulate into libraries (introductory Code as Policies passage; page unavailable). Its policies can react to perception feedback, but the paper does not describe agents authoring simulators or revising policies and libraries in response to environment feedback. Appendix A says code editing to fix errors was not used in the experiments. (LLM, conf=0.88)
Experimental — accuracy not yet measuredPrograms are increasingly used as the medium through which LLM agents reason, act, and model their environments.
“Programs are increasingly used as the medium through which LLM agents reason, act, and model their environments.”
From Code as Agent Harness (2026) · cites this paper as [10]
The Abstract and III. METHOD describe LLM-generated programs that perform spatial reasoning, process perceptual inputs, and control robots. The paper does not establish that programs are increasingly used across LLM agents or that the programs model their environments, rather than use perception outputs. (LLM, conf=0.84)
Experimental — accuracy not yet measured
Code and classical robotics software define the admissibility boundary by exposing typed robot APIs, parameterizing primitive skills, calling geometric libraries, invoking motion planners, and supporting inspection, replay, versioning, and verification.
“Code and classical robotics software define the admissibility boundary by exposing typed robot APIs, parameterizing primitive skills, calling geometric libraries, invoking motion planners, and supporting inspection, replay, versioning, and verification [124, 10, 434, 435].”
From Code as Agent Harness (2026) · cites this paper as [10]
Split into 2 atoms — 1 confirmed, 0 partially supported, 0 disputed, 0 misattributed, 1 unverifiable. Expand the claim to see per-atom verdicts.
Experimental — accuracy not yet measuredPrevious work needed to compose multiple models to control a robot without robot action data.
“Whereas previous work needed to compose multiple models to this end (Ahn et al., 2022; Kwon et al., 2024; Liang et al., 2023; Vemprala et al., 2023)”
From Gemini Robotics: Bringing AI into the Physical World (2025) · cites this paper as Liang et al., 2023
The Perception APIs/Control APIs section describes combining a code-writing LLM with perception outputs and control APIs to operate robots, using off-the-shelf models and few-shot prompting without additional training. That supports the multiple-model approach, but the passage does not establish that the system uses no robot action data at all; it establishes only that it needs no additional training. (LLM, conf=0.78)
Experimental — accuracy not yet measuredVLMs can provide powerful visual semantic understanding and useful guidance to robotic tasks.
“Trained on large datasets of images and language, VLMs can provide powerful visual semantic understanding and useful guidance to robotic tasks, such as which object a manipulator should pick up or which goal a mobile robot should reach [8, 9, 10].”
From RoboPoint: A Vision-Language Model for Spatial Affordance Prediction for Robotics (2024) · cites this paper as [9]
Several works have shown that language models are capable planners for robotic tasks.
“Several works [8, 9, 10] have shown that language model are capable planners for robotic tasks.”
From RoboPoint: A Vision-Language Model for Spatial Affordance Prediction for Robotics (2024) · cites this paper as [9]
Programs can naturally represent temporally extended and compositional actions, which are essential for many long-horizon tasks in Minecraft.
“We opt to use code as the action space instead of low-level motor commands because programs can naturally represent temporally extended and compositional actions [16, 22], which are essential for many long-horizon tasks in Minecraft.”
From Voyager: An Open-Ended Embodied Agent with Large Language Models (2023) · cites this paper as [16]
Robotic and embodied agents use generated programs as executable policies for interacting with physical or simulated worlds.
“robotic and embodied agents use generated programs as executable policies for interacting with physical or simulated worlds [9, 10]”
From Code as Agent Harness (2026) · cites this paper as [10]
Recent LLM-based agents harness world knowledge to generate consistent action plans or executable policies.
“Recent advances in large language model (LLM) based agents harness the world knowledge encapsulated in pre-trained LLMs to generate consistent action plans or executable policies [16, 22, 19].”
From Voyager: An Open-Ended Embodied Agent with Large Language Models (2023) · cites this paper as [16]
As a grounding interface, code translates high-level intent from LLMs into embodiment-respecting commands through primitive skill calls, synthesized Python control policies, and structured behavior-tree programs.
“As a grounding interface, it translates high-level intent from LLMs into embodiment-respecting commands through primitive skill calls [9, 110, 111, 112], synthesized Python control policies [10, 33, 124, 117, 118], and structured behavior-tree programs [34].”
From Code as Agent Harness (2026) · cites this paper as [10]
Many existing approaches leveraging foundation models include CodeAsPolicies, Voyager, VIMA, and VoxPoser.
“Many existing approaches leveraging foundation models include CodeAsPolicies [6], Voyager [35], VIMA [92], VoxPoser [77].”
From Embodied Agent Interface: Benchmarking LLMs for Embodied Decision Making (2024) · cites this paper as [6]
There has been a performance revolution in RL due to LLMs.
“there has been a performance revolution of sorts there (Yao et al., 2023b;Liang et al., 2023;Wang et al., 2023b).”
From LLMs Can't Plan, But Can Help Planning in LLM-Modulo Frameworks (2024) · cites this paper as Liang et al., 2023
The Introduction discusses LLMs planning robot actions using skills that may have been pretrained with reinforcement learning, and the Method describes LLM-generated robot policies. Neither passage reports a performance revolution in reinforcement learning due to LLMs; searches for RL performance improvements and a revolution found no such statement in our copy. (LLM, conf=0.72)
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Language Models are Few-Shot Learners
Brown, T. B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., et al. (2020) · arXiv · doi:10.48550/arxiv.2005.14165 · Cited by 23,372 · see citing works
Chain-Of-Thought Prompting Elicits Reasoning in Large Language Models
Wei, J., Wang, X., Schuurmans, D., Bosma, M., Ichter, B., Xia, F., Chi, E., Le, Q. V., & Zhou, D. (2022) · Advances in Neural Information Processing Systems 35 · doi:10.52202/068431-1800 · Cited by 4,505 · see citing works
Self-citationTraining language models to follow instructions with human feedback
Ouyang, L., Wu, J., Jiang, X., Almeida, D., Wainwright, C. L., Mishkin, P., Zhang, C., Agarwal, S., Slama, K., Ray, A., et al. (2022) · arXiv · doi:10.48550/arxiv.2203.02155 · Cited by 4,208 · see citing works
PaLM: Scaling Language Modeling with Pathways
Chowdhery, A., Narang, S., Devlin, J., Bosma, M., Mishra, G., Roberts, A., Barham, P., Chung, H. W., Sutton, C., Gehrmann, S., et al. (2022) · arXiv · doi:10.48550/arxiv.2204.02311 · Cited by 2,590 · see citing works
Large language models are zero-shot reasoners
Kojima, T., Gu, S. (., Reid, M., Matsuo, Y., & Iwasawa, Y. (2022) · Advances in Neural Information Processing Systems 35 · doi:10.52202/068431-1613 · Cited by 2,013 · see citing works
Evaluating Large Language Models Trained on Code
Chen, M., Tworek, J., Jun, H., Yuan, Q., Pinto, H. P. D. O., Kaplan, J., Edwards, H., Burda, Y., Joseph, N., Brockman, G., et al. (2021) · arXiv · doi:10.48550/arxiv.2107.03374 · Cited by 1,436 · see citing works
MDETR - Modulated Detection for End-to-End Multi-Modal Understanding
Kamath, A., Singh, M., LeCun, Y., Synnaeve, G., Misra, I., & Carion, N. (2021) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00180 · Cited by 761 · see citing works
Understanding Natural Language Commands for Robotic Navigation and Mobile Manipulation
Tellex, S., Kollar, T., Dickerson, S., Walter, M. R., Banerjee, A. G., Teller, S. J., & Roy, N. (2011) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v25i1.7979 · Cited by 654 · see citing works
Do As I Can, Not As I Say: Grounding Language in Robotic Affordances
Ahn, M., Brohan, A., Brown, N., Chebotar, Y., Cortes, O., David, B., Finn, C., Fu, C., Gopalakrishnan, K., Hausman, K., et al. (2022) · arXiv · doi:10.48550/arxiv.2204.01691 · Cited by 624 · see citing works
Self-citationOpen-vocabulary Object Detection via Vision and Language Knowledge DistillationPreprint
Gu, X., Lin, T.-Y., Kuo, W., & Cui, Y. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2104.13921 · Cited by 581 · see citing works
Training Verifiers to Solve Math Word Problems
Cobbe, K., Kosaraju, V., Bavarian, M., Chen, M., Jun, H., Kaiser, L., Plappert, M., Tworek, J., Hilton, J., Nakano, R., et al. (2021) · arXiv · doi:10.48550/arxiv.2110.14168 · Cited by 575 · see citing works
A systematic evaluation of large language models of code
Xu, F. F., Alon, U., Neubig, G., & Hellendoorn, V. J. (2022) · Proceedings of the 6th ACM SIGPLAN International Symposium on Machine Programming · doi:10.1145/3520312.3534862 · Cited by 540 · see citing works
Procedures as a representation for data in a computer program for understanding natural language
Winograd, T. (1971) · DSpace@MIT (Massachusetts Institute of Technology) · Cited by 521 · see citing works
Weakly Supervised Learning of Semantic Parsers for Mapping Instructions to Actions
Artzi, Y., & Zettlemoyer, L. (2013) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00209 · Cited by 441 · see citing works
Breazeal, C., Dautenhahn, K., & Kanda, T. (2016) · Springer handbooks · doi:10.1007/978-3-319-32552-1_72 · Cited by 395 · see citing works
Program Synthesis with Large Language Models
Austin, J., Odena, A., Nye, M., Bosma, M., Michalewski, H., Dohan, D., Jiang, E., Cai, C., Terry, M., Le, Q., et al. (2021) · arXiv · doi:10.48550/arxiv.2108.07732 · Cited by 388 · see citing works
Least-to-Most Prompting Enables Complex Reasoning in Large Language Models
Zhou, D., Schärli, N., Hou, L., Wei, J., Scales, N., Wang, X., Schuurmans, D., Cui, C., Bousquet, O., Le, Q., et al. (2022) · arXiv · doi:10.48550/arxiv.2205.10625 · Cited by 380 · see citing works
Solving Quantitative Reasoning Problems with Language Models
Lewkowycz, A., Andreassen, A., Dohan, D., Dyer, E., Michalewski, H., Ramasesh, V., Slone, A., Anil, C., Schlag, I., Gutman-Solo, T., et al. (2022) · arXiv · doi:10.48550/arxiv.2206.14858 · Cited by 358 · see citing works
(2022) · doi:10.52202/068431-0278 · Cited by 358 · see citing works
Learning to Parse Natural Language Commands to a Robot Control System
Matuszek, C., Herbst, E. V., Zettlemoyer, L., & Fox, D. (2013) · Springer tracts in advanced robotics · doi:10.1007/978-3-319-00065-7_28 · Cited by 315 · see citing works
Walk the talk: connecting language, knowledge, and action in route instructions
MacMahon, M., Stankiewicz, B. J., & Kuipers, B. J. (2006) · Cited by 310 · see citing works
Language Models as Zero-Shot Planners: Extracting Actionable Knowledge for Embodied Agents
Huang, W., Abbeel, P., Pathak, D., & Mordatch, I. (2022) · arXiv · doi:10.48550/arxiv.2201.07207 · Cited by 304 · see citing works
Self-citationToward understanding natural language directions
Kollar, T., Tellex, S., Roy, D. K., & Roy, N. (2010) · Cited by 287 · see citing works
Inner Monologue: Embodied Reasoning through Planning with Language Models
Huang, W., Xia, F., Xiao, T., Chan, H., Liang, J., Florence, P., Zeng, A., Tompson, J., Mordatch, I., Chebotar, Y., et al. (2022) · arXiv · doi:10.48550/arxiv.2207.05608 · Cited by 287 · see citing works
Self-citationMapping Instructions and Visual Observations to Actions with Reinforcement Learning
Misra, D. K., Langford, J., & Artzi, Y. (2017) · Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d17-1106 · Cited by 258 · see citing works
CLIPort: What and Where Pathways for Robotic Manipulation
Shridhar, M., Manuelli, L., & Fox, D. (2021) · arXiv · doi:10.48550/arxiv.2109.12098 · Cited by 223 · see citing works
Tellex, S., Gopalan, N., Kress‐Gazit, H., & Matuszek, C. (2020) · Annual Review of Control Robotics and Autonomous Systems · doi:10.1146/annurev-control-101119-071628 · Cited by 221 · see citing works
A Survey of Reinforcement Learning Informed by Natural Language
Luketina, J., Nardelli, N., Farquhar, G., Foerster, J., Andreas, J., Grefenstette, E., Whiteson, S., & Rocktäschel, T. (2019) · International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2019/880 · Cited by 208 · see citing works
Socratic Models: Composing Zero-Shot Multimodal Reasoning with Language
Zeng, A., Attarian, M., Ichter, B., Choromanski, K., Wong, A., Welker, S., Tombari, F., Purohit, A., Ryoo, M., Sindhwani, V., et al. (2022) · arXiv · doi:10.48550/arxiv.2204.00598 · Cited by 205 · see citing works
Self-citationCompositionality Decomposed: How do Neural Networks Generalise?
Hupkes, D., Dankers, V., Mul, M., & Bruni, E. (2020) · Journal of Artificial Intelligence Research · doi:10.1613/jair.1.11674 · Cited by 194 · see citing works
Mees, O., Hermann, L., Rosete-Beas, E., & Burgard, W. B. (2022) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2022.3180108 · Cited by 189 · see citing works
Drori, I., Zhang, S., Shuttleworth, R., Tang, L., Lu, A., Ke, E., Liu, K., Chen, L., Tran, S., Cheng, N., et al. (2022) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2123433119 · Cited by 187 · see citing works
LM-Nav: Robotic Navigation with Large Pre-Trained Models of Language, Vision, and Action
Shah, D., Osinski, B., Ichter, B., & Levine, S. (2022) · arXiv · doi:10.48550/arxiv.2207.04429 · Cited by 179 · see citing works
Self-citationDzifcak, J., Scheutz, M. J., Baral, C. R., & Schermerhorn, P. (2009) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/robot.2009.5152776 · Cited by 175 · see citing works
BC-Z: Zero-Shot Task Generalization with Robotic Imitation Learning
Jang, E., Irpan, A., Khansari, M., Kappler, D., Ebert, F., Lynch, C., Levine, S., & Finn, C. (2022) · arXiv · doi:10.48550/arxiv.2202.02005 · Cited by 167 · see citing works
Andreas, J., Klein, D., & Levine, S. (2017) · North American Chapter of the Association for Computational Linguistics · doi:10.18653/v1/n18-1197 · Cited by 144 · see citing works
Learning to interpret natural language commands through human-robot dialog
Thomason, J., Zhang, S., Mooney, R. J., & Stone, P. (2015) · Cited by 138 · see citing works
Florence, P., Lynch, C., Zeng, A., Ramirez, O., Wahid, A., Downs, L., Wong, A., Lee, J., Mordatch, I., & Tompson, J. (2021) · arXiv · doi:10.48550/arxiv.2109.00137 · Cited by 131 · see citing works
Self-citationDreamCoder: growing generalizable, interpretable knowledge with wake–sleep Bayesian program learning
Ellis, K., Wong, L., Nye, M., Sablé-Meyer, M., Cary, L., Anaya Pozo, L., Hewitt, L., Solar-Lezama, A., & Tenenbaum, J. B. (2023) · Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences · doi:10.1098/rsta.2022.0050 · Cited by 125 · see citing works
Language as an Abstraction for Hierarchical Deep Reinforcement Learning
Jiang, Y., Gu, S., Murphy, K., & Finn, C. (2019) · arXiv · doi:10.48550/arxiv.1906.07343 · Cited by 109 · see citing works
Language-Conditioned Imitation Learning for Robot Manipulation TasksPreprint
Stepputtis, S., Campbell, J., Phielipp, M., Lee, S., Baral, C. R., & Amor, H. B. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2010.12083 · Cited by 85 · see citing works
Correcting Robot Plans with Natural Language Feedback
Sharma, P., Sundaralingam, B., Blukis, V., Paxton, C., Hermans, T., Torralba, A. B., Andreas, J., & Fox, D. (2022) · Robotics: Science and Systems · doi:10.15607/rss.2022.xviii.065 · Cited by 83 · see citing works
Recursively Summarizing Books with Human Feedback
Wu, J., Ouyang, L., Ziegler, D. M., Stiennon, N., Lowe, R., Leike, J., & Christiano, P. (2021) · arXiv · doi:10.48550/arxiv.2109.10862 · Cited by 76 · see citing works
PixL2R: Guiding Reinforcement Learning Using Natural Language by Mapping Pixels to RewardsPreprint
Goyal, P., Niekum, S., & Mooney, R. (2020) · Conference on Robot Learning · doi:10.48550/arxiv.2007.15543 · Cited by 63 · see citing works
StructFormer: Learning Spatial Structure for Language-Guided Semantic Rearrangement of Novel Objects
Liu, W., Paxton, C., Hermans, T., & Fox, D. (2022) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra46639.2022.9811931 · Cited by 59 · see citing works
Learning Language-Conditioned Robot Behavior from Offline Data and Crowd-Sourced AnnotationPreprint
Nair, S., Mitchell, E., Chen, K., Ichter, B., Savarese, S., & Finn, C. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2109.01115 · Cited by 56 · see citing works
Self-citationBucker, A., Figueredo, L., Haddadinl, S., Kapoor, A., Ma, S., & Bonatti, R. (2022) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros47612.2022.9981810 · Cited by 49 · see citing works
Jointly Improving Parsing and Perception for Natural Language Commands through Human-Robot Dialog
Thomason, J., Padmakumar, A., Sinapov, J., Walker, N., Jiang, Y., Yedidsion, H., Hart, J., Stone, P., & Mooney, R. J. (2020) · Journal of Artificial Intelligence Research · doi:10.1613/jair.1.11485 · Cited by 45 · see citing works
XIRL: Cross-embodiment inverse reinforcement learning
Zakka, K., Zeng, A., Florence, P., Tompson, J., Bohg, J., & Dwibedi, D. (2021) · arXiv · doi:10.48550/arxiv.2106.03911 · Cited by 38 · see citing works
Self-citationSORNet: Spatial Object-Centric Representations for Sequential ManipulationPreprint
Yuan, W., Paxton, C., Desingh, K., & Fox, D. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2109.03891 · Cited by 28 · see citing works
Grounding Language to Autonomously-Acquired Skills via Goal GenerationPreprint
Akakzia, A., Colas, C., Oudeyer, P., Chétouani, M., & Sigaud, O. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2006.07185 · Cited by 27 · see citing works
Composing Pick-and-Place Tasks by Grounding Language
Mees, O., & Burgard, W. (2021) · Springer proceedings in advanced robotics · doi:10.1007/978-3-030-71151-1_43 · Cited by 22 · see citing works
HIGhER : Improving instruction following with Hindsight Generation for Experience ReplayPreprint
Cideron, G., Seurin, M., Strub, F., & Pietquin, O. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1910.09451 · Cited by 20 · see citing works
Language Conditioned Imitation Learning over Unstructured Data
Lynch, C., & Sermanet, P. (2020) · arXiv · doi:10.48550/arxiv.2005.07648 · Cited by 18 · see citing works
Learning Visual Affordances for Robotic Manipulation
Zeng, A. (2019) · Cited by 17 · see citing works
Self-citationLearning abstract structure for drawing by efficient motor program inductionPreprint
Tian, L. Y., Ellis, K., Kryven, M., & Tenenbaum, J. B. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2008.03519 · Cited by 16 · see citing works
Learning to Synthesize Programs as Interpretable and Generalizable PoliciesPreprint
Trivedi, D., Zhang, J. M., Sun, S.-H., & Lim, J. J. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2108.13643 · Cited by 16 · see citing works
Implicit Kinematic Policies: Unifying Joint and Cartesian Action Spaces in End-to-End Robot Learning
Ganapathi, A., Florence, P., Varley, J., Burns, K., Goldberg, K. Y., & Zeng, A. (2022) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra46639.2022.9812165 · Cited by 14 · see citing works
Self-citationLearning Perceptual Concepts by Bootstrapping From Human Queries
Bobu, A., Paxton, C., Yang, W., Sundaralingam, B., Chao, Y.-W., Çakmak, M., & Fox, D. (2022) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2022.3196164 · Cited by 8 · see citing works
OPT: Open Pre-trained Transformer Language Models
Zhang, S., Roller, S., Goyal, N., Artetxe, M., Chen, M., Chen, S., Dewan, C., Diab, M., Li, X., Lin, X. V., et al. (2022) · arXiv · doi:10.48550/arxiv.2205.01068
Flagged first, then by citations
Research works printed but not linked: 1
- D. Kalashnikov, A. Irpan, P. Pastor, J. Ibarz, A. Herzog, E. Jang, D. Quillen, E. Holly, M. Kalakrishnan, V. Vanhoucke et al., "Scalable deep reinforcement learning for vision-based robotic manipulation," in CoRL, 2018.
Cited by
Sorted by citations
The rise and potential of large language model based agents: a survey
Xi, Z., Chen, W., Guo, X., He, W., Ding, Y., Hong, B., Zhang, M., Wang, J., Jin, S., Zhou, E., et al. (2025) · Science China Information Sciences · doi:10.1007/s11432-024-4222-0 · Cited by 970 · see citing works
(2022) · doi:10.15607/rss.2023.xix.025 · Cited by 916 · see citing works
Self-citationChatGPT for Robotics: Design Principles and Model Abilities
Vemprala, S. H., Bonatti, R., Bucker, A., & Kapoor, A. (2024) · IEEE Access · doi:10.1109/access.2024.3387941 · Cited by 570 · see citing works
DriveGPT4: Interpretable End-to-End Autonomous Driving Via Large Language Model
Xu, Z., Zhang, Y., Xie, E., Zhen, Z., Guo, Y. F., Wong, K. K., Li, Z., & Zhao, H. (2024) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2024.3440097 · Cited by 494 · see citing works
Visual Language Maps for Robot Navigation
Huang, C., Mees, O., Zeng, A., & Burgard, W. (2023) · 2023 IEEE International Conference on Robotics and Automation (ICRA) · doi:10.1109/icra48891.2023.10160969 · Cited by 376 · see citing works
Self-citationA Survey on Multimodal Large Language Models for Autonomous Driving
Cui, C., Ma, Y., Cao, X., Ye, W., Zhou, Y., Liang, K., Chen, J., Lu, J., Yang, Z., Liao, K.-D., et al. (2024) · 2024 IEEE/CVF Winter Conference on Applications of Computer Vision Workshops (WACVW) · doi:10.1109/wacvw60836.2024.00106 · Cited by 369 · see citing works
Empowering biomedical discovery with AI agents
Gao, S., Fang, A., Huang, Y., Giunchiglia, V., Noori, A., Schwarz, J. R., Ektefaie, Y., Kondic, J., & Zitnik, M. (2024) · Cell · doi:10.1016/j.cell.2024.09.022 · Cited by 315 · see citing works
Large language models for human–robot interaction: A review
Zhang, C., Chen, J., Li, J., Peng, Y., & Mao, Z. (2023) · Biomimetic Intelligence and Robotics · doi:10.1016/j.birob.2023.100131 · Cited by 269 · see citing works
Foundation models in robotics: Applications, challenges, and the future
Firoozi, R., Tucker, J., Tian, S., Majumdar, A., Sun, J., Liu, W., Zhu, Y., Song, S., Kapoor, A., Hausman, K., et al. (2024) · The International Journal of Robotics Research · doi:10.1177/02783649241281508 · Cited by 267 · see citing works
Self-citationAn Empirical Study of the Non-Determinism of ChatGPT in Code Generation
Ouyang, S., Zhang, J. M., Harman, M., & Wang, M. (2025) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/3697010 · Cited by 258 · see citing works
“Section 8 concludes.”
Text2Motion: from natural language instructions to feasible plans
Lin, K., Agia, C., Migimatsu, T., Pavone, M., & Bohg, J. (2023) · Autonomous Robots · doi:10.1007/s10514-023-10131-7 · Cited by 233 · see citing works
TidyBot: personalized robot assistance with large language models
Wu, J., Antonova, R., Kan, A., Lepert, M., Zeng, A., Song, S., Bohg, J., Rusinkiewicz, S. M., & Funkhouser, T. A. (2023) · Autonomous Robots · doi:10.1007/s10514-023-10139-z · Cited by 229 · see citing works
Self-citationAligning Cyber Space With Physical World: A Comprehensive Survey on Embodied AI
Liu, Y., Chen, W., Bai, Y., Liang, X., Li, G., Gao, W., & Lin, L. (2025) · IEEE/ASME Transactions on Mechatronics · doi:10.1109/tmech.2025.3574943 · Cited by 220 · see citing works
RoCo: Dialectic Multi-Robot Collaboration with Large Language Models
Mandi, Z., Jain, S., & Song, S. (2024) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra57147.2024.10610855 · Cited by 206 · see citing works
Large language models for robotics: Opportunities, challenges, and perspectives
Wang, J., Shi, E., Hu, H., Ma, C., Liu, Y., Wang, X., Yao, Y., Liu, X., Ge, B., & Zhang, S. (2024) · Journal of Automation and Intelligence · doi:10.1016/j.jai.2024.12.003 · Cited by 204 · see citing works
Voyager: An Open-Ended Embodied Agent with Large Language Models
Wang, G., Xie, Y., Jiang, Y., Mandlekar, A., Xiao, C., Zhu, Y., Fan, L., & Anandkumar, A. (2023) · arXiv · doi:10.48550/arxiv.2305.16291 · Cited by 222 · see citing works
“[16] Jacky Liang, Wenlong Huang, Fei Xia, Peng Xu, Karol Hausman, Brian Ichter, Pete Florence, and Andy Zeng.”
BackgroundMethodInfluentialFan, H., Liu, X., Fuh, J. Y. H., Lu, W. F., & Li, B. (2024) · Journal of Intelligent Manufacturing · doi:10.1007/s10845-023-02294-y · Cited by 188 · see citing works
Tool Learning with Foundation Models
Qin, Y., Hu, S., Lin, Y., Chen, W., Ding, N., Cui, G., Zeng, Z., Huang, Y., Xiao, C., Han, C., et al. (2023) · arXiv · doi:10.48550/arxiv.2304.08354 · Cited by 184 · see citing works
(2024) · doi:10.1145/3704435 · Cited by 184 · see citing works
Survey on large language model-enhanced reinforcement learning: Concept, taxonomy, and methods
Cao, Y., Zhao, H., Cheng, Y., Shu, T., Chen, Y., Liu, G., Liang, G., Zhao, J., Yan, J., & Li, Y. (2025) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2024.3497992 · Cited by 174 · see citing works
A Multiagent-Driven Robotic AI Chemist Enabling Autonomous Chemical Research On Demand
Song, T., Luo, M., Zhang, X., Chen, L., Huang, Y., Cao, J., Zhu, Q., Liu, D., Zhang, B., Zou, G., et al. (2025) · Journal of the American Chemical Society · doi:10.1021/jacs.4c17738 · Cited by 171 · see citing works
ORGANA: A robotic assistant for automated chemistry experimentation and characterization
Darvish, K., Skreta, M., Zhao, Y., Yoshikawa, N., Som, S., Bogdanović, M., Cao, Y., Hao, H., Xu, H., Aspuru‐Guzik, A., et al. (2024) · Matter · doi:10.1016/j.matt.2024.10.015 · Cited by 149 · see citing works
A survey on integration of large language models with intelligent robots
Kim, Y., Kim, D., Choi, J., Park, J., Oh, N., & Park, D. (2024) · Intelligent Service Robotics · doi:10.1007/s11370-024-00550-5 · Cited by 148 · see citing works
Task and Motion Planning with Large Language Models for Object Rearrangement
Ding, Y. F., Zhang, X., Paxton, C., & Zhang, S. (2023) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros55552.2023.10342169 · Cited by 144 · see citing works
Embodied large language models enable robots to complete complex tasks in unpredictable environments
Mon-Williams, R., Li, G., Long, R., Du, W., & Lucas, C. G. (2025) · Nature Machine Intelligence · doi:10.1038/s42256-025-01005-x · Cited by 132 · see citing works
Large language models for chemistry robotics
Yoshikawa, N., Skreta, M., Darvish, K., Arellano-Rubach, S., Ji, Z., Bjørn Kristensen, L., Li, A. Z., Zhao, Y., Xu, H., Kuramshin, A., et al. (2023) · Autonomous Robots · doi:10.1007/s10514-023-10136-2 · Cited by 112 · see citing works
Scalable Multi-Robot Collaboration with Large Language Models: Centralized or Decentralized Systems?
Chen, Y., Arkin, J., Zhang, Y., Roy, N., & Fan, C. (2024) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra57147.2024.10610676 · Cited by 111 · see citing works
GraspGPT: Leveraging Semantic Knowledge From a Large Language Model for Task-Oriented Grasping
Tang, C., Huang, D., Ge, W., Liu, W., & Zhang, H. (2023) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2023.3320012 · Cited by 110 · see citing works
TaskMatrix.AI: Completing Tasks by Connecting Foundation Models with Millions of APIs
Liang, Y., Wu, C., Song, T., Wu, W., Xia, Y., Liu, Y., Ou, Y., Lu, S., Ji, L., Mao, S., et al. (2024) · Intelligent Computing · doi:10.34133/icomputing.0063 · Cited by 108 · see citing works
AutoTAMP: Autoregressive Task and Motion Planning with LLMs as Translators and Checkers
Chen, Y., Arkin, J., Dawson, C., Zhang, Y., Roy, N., & Fan, C. (2024) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra57147.2024.10611163 · Cited by 107 · see citing works
RobotGPT: Robot Manipulation Learning From ChatGPT
Jin, Y., Li, D., A, Y., Shi, J., Hao, P., Sun, F., Zhang, J., & Fang, B. (2024) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2024.3357432 · Cited by 103 · see citing works
Semantic anomaly detection with large language models
Elhafsi, A., Sinha, R., Agia, C., Schmerling, E., Nesnas, I. A. D., & Pavone, M. (2023) · Autonomous Robots · doi:10.1007/s10514-023-10132-6 · Cited by 101 · see citing works
Knowledge sharing in manufacturing using LLM-powered tools: user study and model benchmarking
Freire, S. K., Wang, C., Foosherian, M., Wellsandt, S., Ruiz-Arenas, S., & Niforatos, E. (2024) · Frontiers in Artificial Intelligence · doi:10.3389/frai.2024.1293084 · Cited by 101 · see citing works
Enhancing the LLM-Based Robot Manipulation Through Human-Robot Collaboration
Liu, H., Zhu, Y., Kato, K. H., Tsukahara, A., Kondo, I., Aoyama, T., & Hasegawa, Y. (2024) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2024.3415931 · Cited by 100 · see citing works
King, E., Yu, H., Lee, S., & Julien, C. (2024) · Proceedings of the ACM on Interactive Mobile Wearable and Ubiquitous Technologies · doi:10.1145/3643505 · Cited by 99 · see citing works
ChatGPT Empowered Long-Step Robot Control in Various Environments: A Case Application
Wake, N., Kanehira, A., Sasabuchi, K., Takamatsu, J., & Ikeuchi, K. (2023) · IEEE Access · doi:10.1109/access.2023.3310935 · Cited by 96 · see citing works
Science acceleration and accessibility with self-driving labs
Canty, R. B., Bennett, J. A., Brown, K. A., Buonassisi, T., Kalinin, S. V., Kitchin, J. R., Maruyama, B., Moore, R. G., Schrier, J., Seifrid, M. T., et al. (2025) · Nature Communications · doi:10.1038/s41467-025-59231-1 · Cited by 97 · see citing works
ProgPrompt: program generation for situated robot task planning using large language models
Singh, I., Blukis, V., Mousavian, A., Goyal, A., Xu, D., Tremblay, J., Fox, D., Thomason, J., & Garg, A. (2023) · Autonomous Robots · doi:10.1007/s10514-023-10135-3 · Cited by 95 · see citing works
Vision-Language-Action Models for Robotics: A Review Towards Real-World Applications
Kawaharazuka, K., Oh, J., Yamada, J., Posner, I., & Zhu, Y. (2025) · IEEE Access · doi:10.1109/access.2025.3609980 · Cited by 95 · see citing works
Generalized Planning in PDDL Domains with Pretrained Large Language Models
Silver, T., Dan, S., Srinivas, K., Tenenbaum, J. B., Kaelbling, L. P., & Katz, M. (2024) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v38i18.30006 · Cited by 90 · see citing works
Real-world robot applications of foundation models: a review
Kawaharazuka, K., Matsushima, T., Gambardella, A., Guo, J.-X., Paxton, C., & Zeng, A. (2024) · Adv. Robotics · doi:10.1080/01691864.2024.2408593 · Cited by 139 · see citing works
Self-citationPenetrative AI: Making LLMs Comprehend the Physical World
Xu, H., Han, L., Yang, Q., Li, M., & Srivastava, M. B. (2024) · IEEE Workshop on Mobile Computing Systems and Applications (WMCSA) · doi:10.1145/3638550.3641130 · Cited by 85 · see citing works
LLM-Grounder: Open-Vocabulary 3D Visual Grounding with Large Language Model as an Agent
Yang, J., Chen, X., Qian, S., Madaan, N., Iyengar, M., Fouhey, D. F., & Chai, J. Y. (2024) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra57147.2024.10610443 · Cited by 85 · see citing works
A Survey of Robot Intelligence with Large Language Models
Jeong, H., Lee, H., Kim, C., & Shin, S. (2024) · Applied Sciences · doi:10.3390/app14198868 · Cited by 80 · see citing works
TidyBot: Personalized Robot Assistance with Large Language Models
Wu, J., Antonova, R., Kan, A., Lepert, M., Zeng, A., Song, S., Bohg, J., Rusinkiewicz, S. M., & Funkhouser, T. A. (2023) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros55552.2023.10341577 · Cited by 79 · see citing works
Self-citationAI Reasoning in Deep Learning Era: From Symbolic AI to Neural–Symbolic AI
Liang, B., Wang, Y., & Tong, C. (2025) · Mathematics · doi:10.3390/math13111707 · Cited by 79 · see citing works
GPT-4V(ision) for Robotics: Multimodal Task Planning From Human Demonstration
Wake, N., Kanehira, A., Sasabuchi, K., Takamatsu, J., & Ikeuchi, K. (2024) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2024.3477090 · Cited by 77 · see citing works
Towards Learning a Generalist Model for Embodied Navigation
Zheng, D., Huang, S., Zhao, L., Zhong, Y., & Wang, L. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.01293 · Cited by 74 · see citing works
Cognitive Architectures for Language Agents
Sumers, T. R., Yao, S., Narasimhan, K., & Griffiths, T. L. (2023) · arXiv · doi:10.48550/arxiv.2309.02427 · Cited by 72 · see citing works
“…of external knowledge or computation, stateless digital APIs (e.g., search, calculator, translator) are often packaged as “ tools ” (Parisi et al., 2022; Schick et al., 2023; Xu et al., 2023a; Tang et al., 2023b; Qin et al., 2023), which can be viewed as special “single-use” digital environments.”
BackgroundProAgent: Building Proactive Cooperative Agents with Large Language Models
Zhang, C., Yang, K., Hu, S., Wang, Z., Li, G., Sun, Y., Zhang, C., Zhang, Z., Liu, A., Zhu, S.-C., et al. (2024) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v38i16.29710 · Cited by 88 · see citing works
Grounding Language with Visual Affordances over Unstructured Data
Mees, O., Diaz, J. L. B., & Burgard, W. (2023) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra48891.2023.10160396 · Cited by 71 · see citing works
Bridging Items and Language: A Transition Paradigm for Large Language Model-Based Recommendation
Lin, X., Wang, W., Li, Y., Feng, F., Ng, S.-K., & Chua, T. (2024) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/3637528.3671884 · Cited by 71 · see citing works
CoPa: General Robotic Manipulation through Spatial Constraints of Parts with Foundation Models
Huang, H., Lin, F., Hu, Y., Wang, S., & Gao, Y. (2024) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros58592.2024.10801352 · Cited by 66 · see citing works
Generative Expressive Robot Behaviors using Large Language Models
Mahadevan, K., Chien, J., Brown, N., Xu, Z., Parada, C., Xia, F., Zeng, A., Takayama, L. A., & Sadigh, D. (2024) · ACM/IEEE International Conference on Human-Robot Interaction (HRI) · doi:10.1145/3610977.3634999 · Cited by 64 · see citing works
Self-citationLanguage Models as Zero-Shot Trajectory Generators
Kwon, T., Palo, N. D., & Johns, E. (2024) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2024.3410155 · Cited by 59 · see citing works
From screens to scenes: A survey of embodied AI in healthcare
Liu, Y., Cao, X., Chen, T., Jiang, Y., You, J., Wu, M., Wang, X., Feng, M., Jin, Y., & Chen, J. (2025) · Information Fusion · doi:10.1016/j.inffus.2025.103033 · Cited by 59 · see citing works
Opportunities of applying Large Language Models in building energy sector
Zhang, L., & Chen, Z. (2025) · Renewable and Sustainable Energy Reviews · doi:10.1016/j.rser.2025.115558 · Cited by 58 · see citing works
Plug in the Safety Chip: Enforcing Constraints for LLM-driven Robot Agents
Yang, Z., Raman, S. S., Shah, A., & Tellex, S. (2024) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra57147.2024.10611447 · Cited by 57 · see citing works
LaMPilot: An Open Benchmark Dataset for Autonomous Driving with Language Model Programs
Ma, Y., Cui, C., Cao, X., Ye, W., Liu, P., Lu, J., Abdelraouf, A., Gupta, R., Han, K., Bera, A., et al. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.01434 · Cited by 56 · see citing works
Gu, Z., Li, J., Shen, W., Yu, W., Xie, Z., McCrory, S., Cheng, X., Shamsah, A., Griffin, R. J., Liu, C. K., et al. (2026) · IEEE/ASME Transactions on Mechatronics · doi:10.1109/tmech.2025.3579247 · Cited by 55 · see citing works
Scarecrows in Oz: The Use of Large Language Models in HRI
Williams, T., Matuszek, C., Mead, R., & DePalma, N. (2024) · ACM Transactions on Human-Robot Interaction · doi:10.1145/3606261 · Cited by 55 · see citing works
Yang, T., Feng, P., Guo, Q., Zhang, J., Zhang, X., Ning, J., Wang, X., & Mao, Z. (2025) · IEEE Transactions on Cognitive Communications and Networking · doi:10.1109/tccn.2025.3528892 · Cited by 53 · see citing works
A Survey of Reasoning with Foundation ModelsPreprint
Sun, J., Zheng, C., Xie, E., Liu, Z., Chu, R., Liu, J., Xu, J., Ding, M., Li, H., Geng, M., et al. (2023) · doi:10.31219/osf.io/ac4sp · Cited by 51 · see citing works
FlowMind: Automatic Workflow Generation with LLMs
Zeng, Z., Watson, W., Cho, N., Rahimi, S., Reynolds, S., Balch, T. R., & Veloso, M. (2023) · ACM International Conference on AI in Finance (ICAIF) · doi:10.1145/3604237.3626908 · Cited by 50 · see citing works
Prompt a Robot to Walk with Large Language Models
Wang, Y., Zhang, B., Chen, J., & Sreenath, K. (2024) · IEEE Conference on Decision and Control (CDC) · doi:10.1109/cdc56724.2024.10885862 · Cited by 50 · see citing works
Diffusion-ES: Gradient-Free Planning with Diffusion for Autonomous and Instruction-Guided Driving
Yang, B., Su, H., Gkanatsios, N., Ke, T.-W., Jain, A., Schneider, J., & Fragkiadaki, K. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.01453 · Cited by 49 · see citing works
Object detection with multimodal large vision-language models: An in-depth review
Sapkota, R., & Karkee, M. (2025) · Information Fusion · doi:10.1016/j.inffus.2025.103575 · Cited by 49 · see citing works
Shi, W., Xu, R., Zhuang, Y., Yu, Y., Zhang, J., Wu, H., Zhu, Y., Ho, J. C., Yang, C., & Wang, M. D. (2024) · Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/2024.emnlp-main.1245 · Cited by 54 · see citing works
“To mitigate ambiguities and misinterpretations in plan generation, an increasing number of LLM agents (Gao et al., 2023; Liang et al., 2023; Sun et al., 2023; Chen et al., 2023; Zhuang et al., 2024) employ code prompts as planner interface instead of natu-ral language prompts.”
BackgroundVision-Language Interpreter for Robot Task Planning
Shirai, K., Beltran-Hernandez, C. C., Hamaya, M., Hashimoto, A., Tanaka, S., Kawaharazuka, K., Tanaka, K., Ushiku, Y., & Mori, S. (2024) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra57147.2024.10611112 · Cited by 47 · see citing works
Robot learning in the era of foundation models: a survey
Xiao, X., Liu, J., Wang, Z., Zhou, Y., Qi, Y., Jiang, S., He, B., & Cheng, Q. (2025) · Neurocomputing · doi:10.1016/j.neucom.2025.129963 · Cited by 47 · see citing works
Advancing real-time infectious disease forecasting using large language models
Du, H., Zhao, Y., Zhao, J., Xu, S., Lin, X., Chen, Y., Gardner, L., & Yang, H. (2025) · Nature Computational Science · doi:10.1038/s43588-025-00798-6 · Cited by 47 · see citing works
Raptis, E. K., Kapoutsis, A. C., & Kosmatopoulos, E. B. (2025) · Frontiers in Robotics and AI · doi:10.3389/frobt.2025.1605405 · Cited by 47 · see citing works
VoicePilot: Harnessing LLMs as Speech Interfaces for Physically Assistive Robots
Padmanabha, A., Yuan, J., Gupta, J., Karachiwalla, Z., Majidi, C., Admoni, H., & Erickson, Z. (2024) · ACM Symposium on User Interface Software and Technology (UIST) · doi:10.1145/3654777.3676401 · Cited by 45 · see citing works
Electromagnetic metamaterial agent
Hu, S., Li, M., Xu, J., Zhang, H., Zhang, S., Cui, T. J., del Hougne, P., & Li, L. (2025) · Light Science & Applications · doi:10.1038/s41377-024-01678-w · Cited by 45 · see citing works
Industrial Foundation Models (IFMs) for intelligent manufacturing: A systematic review
Zhao, S., Liu, S., Jiang, Y., Zhao, B. M., Lv, Y., Zhang, J., Wang, L., & Zhong, R. Y. (2025) · Journal of Manufacturing Systems · doi:10.1016/j.jmsy.2025.06.011 · Cited by 46 · see citing works
RoboTAP: Tracking Arbitrary Points for Few-Shot Visual Imitation
Vecerík, M., Doersch, C., Yang, Y., Davchev, T., Aytar, Y., Zhou, G., Hadsell, R. T., Agapito, L., & Scholz, J. (2024) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra57147.2024.10611409 · Cited by 44 · see citing works
Personalized Autonomous Driving with Large Language Models: Field Experiments
Cui, C., Yang, Z., Zhou, Y., Ma, Y., Lu, J., Li, L., Chen, Y., Panchal, J. H., & Wang, Z. (2024) · IEEE International Conference on Intelligent Transportation Systems (ITSC) · doi:10.1109/itsc58415.2024.10919978 · Cited by 44 · see citing works
Embodied Multi-Modal Agent trained by an LLM from a Parallel TextWorld
Yang, Y., Zhou, T., Li, K., Tao, D., Li, L., Shen, L., He, X., Jiang, J., & Shi, Y. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.02482 · Cited by 43 · see citing works
Robotic Skill Acquisition via Instruction Augmentation with Vision-Language Models
Xiao, T. G., Chan, H., Sermanet, P., Wahid, A., Brohan, A., Hausman, K., Levine, S., & Tompson, J. (2023) · Robotics: Science and Systems · doi:10.15607/rss.2023.xix.029 · Cited by 42 · see citing works
Self-citationStatler: State-Maintaining Language Models for Embodied Reasoning
Yoneda, T., Fang, J., Li, P., Zhang, H., Jiang, T., Lin, S., Picker, B., Yunis, D., Mei, H., & Walter, M. R. (2024) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra57147.2024.10610634 · Cited by 41 · see citing works
Prompt, Plan, Perform: LLM-based Humanoid Control via Quantized Imitation Learning
Sun, J., Zhang, Q., Duan, Y., Jiang, X., Chen, C., & Xu, R. (2024) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra57147.2024.10610948 · Cited by 41 · see citing works
Driving Everywhere with Large Language Model Policy Adaptation
Li, B., Wang, Y., Mao, J., Ivanovic, B., Veer, S., Leung, K., & Pavone, M. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.01416 · Cited by 41 · see citing works
Deploying and Evaluating LLMs to Program Service Mobile Robots
Hu, Z., Lucchetti, F., Schlesinger, C., Saxena, Y., Freeman, A., Modak, S., Guha, A., & Biswas, J. (2024) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2024.3360020 · Cited by 40 · see citing works
CoPAL: Corrective Planning of Robot Actions with Large Language Models
Joublin, F., Ceravola, A., Smirnov, P., Ocker, F., Deigmoeller, J., Belardinelli, A., Wang, C., Hasler, S., Tanneberg, D., & Gienger, M. (2024) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra57147.2024.10610434 · Cited by 40 · see citing works
Incremental learning of humanoid robot behavior from natural interaction and large language models
Bärmann, L., Kartmann, R., Peller-Konrad, F., Niehues, J., Waibel, A., & Asfour, T. (2024) · Frontiers in Robotics and AI · doi:10.3389/frobt.2024.1455375 · Cited by 40 · see citing works
SayCanPay: Heuristic Planning with Large Language Models Using Learnable Domain Knowledge
Hazra, R., Martires, P. Z. D., & Raedt, L. D. (2024) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v38i18.29991 · Cited by 39 · see citing works
BTGenBot: Behavior Tree Generation for Robotic Tasks with Lightweight LLMs
Izzo, R. A., Bardaro, G., & Matteucci, M. (2024) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros58592.2024.10802304 · Cited by 39 · see citing works
Visual Programming for Zero-Shot Open-Vocabulary 3D Visual Grounding
Yuan, Z., Ren, J., Feng, C.-M., Zhao, H., Cui, S. R., & Li, Z. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.01949 · Cited by 38 · see citing works
Jailbreaking LLM-Controlled Robots
Robey, A., Ravichandran, Z., Kumar, V., Hassani, H., & Pappas, G. J. (2025) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra55743.2025.11128119 · Cited by 38 · see citing works
DoReMi: Grounding Language Model by Detecting and Recovering from Plan-Execution Misalignment
Guo, Y. F., Wang, Y., Zha, L., & Chen, J. (2024) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros58592.2024.10802284 · Cited by 37 · see citing works
RoboSpatial: Teaching Spatial Understanding to 2D and 3D Vision-Language Models for Robotics
Song, C. H., Blukis, V., Tremblay, J., Tyree, S., Su, Y., & Birchfield, S. T. (2025) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52734.2025.01470 · Cited by 37 · see citing works
Embodied AI: From LLMs to World Models [Feature]
Feng, T., Wang, X., Jiang, Y., & Zhu, W. (2025) · IEEE Circuits and Systems Magazine · doi:10.1109/mcas.2025.3603693 · Cited by 37 · see citing works
<b>JARVIS</b> -1: Open-World Multi-Task Agents With Memory-Augmented Multimodal Language Models
Wang, Z., Cai, S., Liu, A., Jin, Y., Hou, J., Zhang, B., Lin, H., He, Z., Zheng, Z., Yang, Y., et al. (2025) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2024.3511593 · Cited by 36 · see citing works
StructDiffusion: Language-Guided Creation of Physically-Valid Structures using Unseen Objects
Liu, W., Du, Y., Hermans, T., Chernova, S., & Paxton, C. (2023) · Robotics: Science and Systems · doi:10.15607/rss.2023.xix.031 · Cited by 36 · see citing works
How People Prompt Generative AI to Create Interactive VR Scenes
Manesh, S. A., Zhang, T., Onishi, Y., Hara, K., Bateman, S., Li, J., & Tang, A. (2024) · Designing Interactive Systems Conference · doi:10.1145/3643834.3661547 · Cited by 36 · see citing works
Distilling and Retrieving Generalizable Knowledge for Robot Manipulation via Language Corrections
Zha, L., Cui, Y., Lin, L.-H., Kwon, M., Arenas, M. G., Zeng, A., Xia, F., & Sadigh, D. (2024) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra57147.2024.10610455 · Cited by 36 · see citing works
Self-citationA Comprehensive Survey on Embodied Intelligence: Advancements, Challenges, and Future Perspectives
Sun, F., Chen, R., Ji, T., Luo, Y., Zhou, H., & Liu, H. (2024) · CAAI Artificial Intelligence Research · doi:10.26599/air.2024.9150042 · Cited by 36 · see citing works
Alchemist: LLM-Aided End-User Development of Robot Applications
Karli, U. B., Chen, J.-T., Antony, V. N., & Huang, C. (2024) · ACM/IEEE International Conference on Human-Robot Interaction (HRI) · doi:10.1145/3610977.3634969 · Cited by 35 · see citing works
Luo, H., Wu, J., Liu, J., & Antwi‐Afari, M. F. (2024) · Developments in the Built Environment · doi:10.1016/j.dibe.2024.100488 · Cited by 35 · see citing works
Interactive Planning Using Large Language Models for Partially Observable Robotic Tasks
Sun, L., Jha, D. K., Hori, C., Jain, S., Corcodel, R., Zhu, X., Tomizuka, M., & Romeres, D. (2024) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra57147.2024.10610981 · Cited by 35 · see citing works
- Authors
- Liang, J., Huang, W., Xia, F., Xu, P., Hausman, K., Ichter, B., Florence, P., & Zeng, A.
- Venue
- 2023 IEEE International Conference on Robotics and Automation (ICRA)
- Publisher
- IEEE
- Published
- 2023-05-29
- Pages
- 9493-9500
- ISSN
- 1050-4729 (ISSN-L)
- OpenAlex
- W4383097638
- Cited by
- 767
- References
- 59
- Citations by year
- 2026: 231 · 2025: 286 · 2024: 201 · 2023: 48 · 2022: 1
- FWCI
- 327.59
- Type
- conference paper
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- Proceedings - IEEE International Conference on Robotics and Automation/Proceedings (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Topics
- Physical Sciences › Engineering › Control and Systems Engineering › Robot Manipulation and Learning
- Physical Sciences › Computer Science › Artificial Intelligence › AI-based Problem Solving and Planning
- Physical Sciences › Computer Science › Information Systems › Software Engineering Research
- Keywords
- large language models, code generation, robot policy, embodied control, natural language commands, few-shot prompting, trajectory-based control, impedance control
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
