The DOI matches its DataCite record; 2 of the 2 references it cites were found in a registry or an index.
Language Models as Zero-Shot Planners: Extracting Actionable Knowledge for Embodied Agents
Open accessAbstract
Can world knowledge learned by large language models (LLMs) be used to act in interactive environments? In this paper, we investigate the possibility of grounding high-level tasks, expressed in natural language (e.g. "make breakfast"), to a chosen set of actionable steps (e.g. "open fridge"). While prior work focused on learning from explicit step-by-step examples of how to act, we surprisingly find that if pre-trained LMs are large enough and prompted appropriately, they can effectively decompose high-level tasks into mid-level plans without any further training. However, the plans produced naively by LLMs often cannot map precisely to admissible actions. We propose a procedure that conditions on existing demonstrations and semantically translates the plans to admissible actions. Our evaluation in the recent VirtualHome environment shows that the resulting method substantially improves executability over the LLM baseline. The conducted human evaluation reveals a trade-off between executability and correctness but shows a promising sign towards extracting actionable knowledge from language models. Website at https://huangwl18.github.io/language-planner
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Several works use LLMs as interactive policies, demonstrating their strong capabilities but also highlighting challenges in grounding them with observations and actions (Huang et al., 2022;Yao et al., 2023;Ahn et al., 2022;Huang et al., 2023;Zeng et al., 2022).”
From PARTNR: A Benchmark for Planning and Reasoning in Embodied Multi-agent Tasks (2024) · cites this paper as Huang et al., 2022
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Agents equipped with LLM-based planners have started to show the ability to learn a new task with a few training examples.”
From LLM-Planner: Few-Shot Grounded Planning for Embodied Agents with Large Language Models (2023) · cites this paper as [13]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“However, these pre-trained models, especially the VLMs, are not explicitly designed for physical interactions and, therefore, often lack the physical common sense understanding and result in suboptimal plans and execution.”
From Cosmos-Reason1: From Physical Common Sense To Embodied Reasoning (2025) · cites this paper as Huang et al., 2022
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“For instance, language models have been used via prompting to translate natural language instructions into actions executable by robots (Ahn et al., 2022; Huang et al., 2022), interact with users (Coenen et al., 2021; Wu et al., 2021; 2022a; Lee et al., 2022b), and facilitate multi-modal reasoning (Zeng et al., 2022; Alayrac et al., 2022).”
From Emergent Abilities of Large Language Models (2022) · cites this paper as Huang et al., 2022
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“Once learned, language can unlock planning, reasoning (e.g., Huang et al., 2022;Brohan et al., 2023b;Driess et al., 2023;Kim et al., 2023), and communication (Zeng et al., 2022) about grounded situations and tasks.”
From Scaling Instructable Agents Across Many Simulated Worlds (2024) · cites this paper as Huang et al., 2022
The cited source says what the claim attributes to it.
In the citing paper
“Large language models (LLMs) have the ability to understand tasks described in natural language and generate plans to solve them [20, 27, 49, 60].”
From AgentDojo: A Dynamic Environment to Evaluate Prompt Injection Attacks and Defenses for LLM Agents (2024) · cites this paper as [20]
The cited source says what the claim attributes to it.
In the citing paper
“In the specific arena of autonomous task completion, LLMs struggle with making task breakdowns that are both correct and executable [10].”
From Autonomous Task Completion Based on Goal-directed Answer Set Programming (2025) · cites this paper as [10]
The cited source says what the claim attributes to it.
In the citing paper
“In these works, LLMs are often used to help an agent plan its actions [4], [13], [14], despite evidence that they may not be capable of reliably planning [15].”
From Episodic Memory in AI Agents Poses Risks that Should be Studied and Mitigated (2025) · cites this paper as [13]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
The publisher hasn’t deposited this paper’s reference list.
Research works printed but not linked: 51
- [1][1] Peter Anderson, Qi Wu, Damien Teney, Jake Bruce, Mark Johnson, Niko Sünderhauf, Ian Reid, Stephen Gould, and Anton Van Den Hengel. Vision-and-language navigation: Interpreting visually-grounded navigation instructions in real environments. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition , pages 3674-3683, 2018. -
- [2][2] Yoav Artzi and Luke Zettlemoyer. Weakly supervised learning of semantic parsers for mapping instructions to actions. Transactions of the Association for Computational Linguistics , 1:49-62, 2013. -
- [6][6] Daniel Cer, Mona Diab, Eneko Agirre, Inigo Lopez-Gazpio, and Lucia Specia. Semeval-2017 task 1: Semantic textual similarity-multilingual and cross-lingual focused evaluation. arXiv preprint arXiv:1708.00055 , 2017. - · arXiv
- [7][7] Mark Chen, Jerry Tworek, Heewoo Jun, Qiming Yuan, Henrique Ponde, Jared Kaplan, Harri Edwards, Yura Burda, Nicholas Joseph, Greg Brockman, et al. Evaluating large language models trained on code. arXiv preprint arXiv:2107.03374 , 2021. - · arXiv
- [8][8] Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Jacob Hilton, Reiichiro Nakano, Christopher Hesse, and John Schulman. Training verifiers to solve math word problems. arXiv preprint arXiv:2110.14168 , 2021. - · arXiv
- [9][9] Joe Davison, Joshua Feldman, and Alexander M Rush. Commonsense knowledge mining from pretrained models. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP) , pages 1173-1178, 2019. -
- [11][11] Daniel Fried, Ronghang Hu, Volkan Cirik, Anna Rohrbach, Jacob Andreas, Louis-Philippe Morency, Taylor Berg-Kirkpatrick, Kate Saenko, Dan Klein, and Trevor Darrell. Speakerfollower models for vision-and-language navigation. arXiv preprint arXiv:1806.02724 , 2018. - · arXiv
- [12][12] Justin Fu, Anoop Korattikara, Sergey Levine, and Sergio Guadarrama. From language to goals: Inverse reinforcement learning for vision-based instruction following. arXiv preprint arXiv:1902.07742 , 2019. - · arXiv
- [13][13] Tianyu Gao, Adam Fisch, and Danqi Chen. Making pre-trained language models better few-shot learners. arXiv preprint arXiv:2012.15723 , 2020. - · arXiv
- [14][14] Brent Harrison and Mark O Riedl. Learning from stories: using crowdsourced narratives to train virtual agents. In Twelfth Artificial Intelligence and Interactive Digital Entertainment Conference , 2016. -
- [15][15] Dan Hendrycks, Steven Basart, Saurav Kadavath, Mantas Mazeika, Akul Arora, Ethan Guo, Collin Burns, Samir Puranik, Horace He, Dawn Song, et al. Measuring coding challenge competence with apps. arXiv preprint arXiv:2105.09938 , 2021. - · arXiv
- [16][16] Felix Hill, Sona Mokra, Nathaniel Wong, and Tim Harley. Human instruction-following with deep reinforcement learning via transfer-learning from text. arXiv preprint arXiv:2005.09382 , 2020. - · arXiv
- [17][17] Sepp Hochreiter and Jürgen Schmidhuber. Long short-term memory. Neural computation , 9(8): 1735-1780, 1997. -
- [18][18] Ari Holtzman, Jan Buys, Li Du, Maxwell Forbes, and Yejin Choi. The curious case of neural text degeneration. arXiv preprint arXiv:1904.09751 , 2019. - · arXiv
- [19][19] Gabriel Ilharco, Rowan Zellers, Ali Farhadi, and Hannaneh Hajishirzi. Probing text models for common ground with visual representations. arXiv e-prints , pages arXiv-2005, 2020. -
- [20][20] Peter A Jansen. Visually-grounded planning without vision: Language models infer detailed plans from high-level instructions. arXiv preprint arXiv:2009.14259 , 2020. - · arXiv
- [21][21] Zhengbao Jiang, Frank F Xu, Jun Araki, and Graham Neubig. How can we know what language models know? Transactions of the Association for Computational Linguistics , 8:423-438, 2020. -
- [22][22] Eric Kolve, Roozbeh Mottaghi, Winson Han, Eli VanderBilt, Luca Weihs, Alvaro Herrasti, Daniel Gordon, Yuke Zhu, Abhinav Gupta, and Ali Farhadi. Ai2-thor: An interactive 3d environment for visual ai. arXiv preprint arXiv:1712.05474 , 2017. - · arXiv
- [23][23] Belinda Z Li, Maxwell Nye, and Jacob Andreas. Implicit representations of meaning in neural language models. arXiv preprint arXiv:2106.00737 , 2021. - · arXiv
- [24][24] Shuang Li, Xavier Puig, Yilun Du, Clinton Wang, Ekin Akyurek, Antonio Torralba, Jacob Andreas, and Igor Mordatch. Pre-trained language models for interactive decision-making. arXiv preprint arXiv:2202.01771 , 2022. - · arXiv
- [25][25] Yuan-Hong Liao, Xavier Puig, Marko Boben, Antonio Torralba, and Sanja Fidler. Synthesizing environment-aware activities via activity sketches. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 6291-6299, 2019. -
- [26][26] Jiachang Liu, Dinghan Shen, Yizhe Zhang, Bill Dolan, Lawrence Carin, and Weizhu Chen. What makes good in-context examples for gpt3 ? arXiv preprint arXiv:2101.06804 , 2021. - · arXiv
- [27][27] Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. Roberta: A robustly optimized bert pretraining approach. arXiv preprint arXiv:1907.11692 , 2019. - · arXiv
- [28][28] Kevin Lu, Aditya Grover, Pieter Abbeel, and Igor Mordatch. Pretrained transformers as universal computation engines. arXiv preprint arXiv:2103.05247 , 2021. - · arXiv
- [29][29] Corey Lynch and Pierre Sermanet. Grounding language in play. arXiv preprint arXiv:2005.07648 , 2020. - · arXiv
- [30][30] Corey Lynch and Pierre Sermanet. Language conditioned imitation learning over unstructured data. Proceedings of Robotics: Science and Systems. doi , 10, 2021. -
- [31][31] Arjun Majumdar, Ayush Shrivastava, Stefan Lee, Peter Anderson, Devi Parikh, and Dhruv Batra. Improving vision-and-language navigation with image-text pairs from the web. In European Conference on Computer Vision , pages 259-274. Springer, 2020. -
- [32][32] Suvir Mirchandani, Siddharth Karamcheti, and Dorsa Sadigh. Ella: Exploration through learned language abstraction. arXiv preprint arXiv:2103.05825 , 2021. - · arXiv
- [33][33] Dipendra Misra, Kejia Tao, Percy Liang, and Ashutosh Saxena. Environment-driven lexicon induction for high-level instructions. In Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing (Volume 1: Long Papers) , pages 992-1002, 2015. -
- [34][34] Dipendra K Misra, Jaeyong Sung, Kevin Lee, and Ashutosh Saxena. Tell me dave: Contextsensitive grounding of natural language to manipulation instructions. The International Journal of Robotics Research , 35(1-3):281-300, 2016. -
- [35][35] Jeffrey Pennington, Richard Socher, and Christopher D Manning. Glove: Global vectors for word representation. In Proceedings of the 2014 conference on empirical methods in natural language processing (EMNLP) , pages 1532-1543, 2014. -
- [36][36] Fabio Petroni, Tim Rocktäschel, Patrick Lewis, Anton Bakhtin, Yuxiang Wu, Alexander H Miller, and Sebastian Riedel. Language models as knowledge bases? arXiv preprint arXiv:1909.01066 , 2019. - · arXiv
- [37][37] Gabriel Poesia, Oleksandr Polozov, Vu Le, Ashish Tiwari, Gustavo Soares, Christopher Meek, and Sumit Gulwani. Synchromesh: Reliable code generation from pre-trained language models. arXiv preprint arXiv:2201.11227 , 2022. - · arXiv
- [38][38] Xavier Puig, Kevin Ra, Marko Boben, Jiaman Li, Tingwu Wang, Sanja Fidler, and Antonio Torralba. Virtualhome: Simulating household activities via programs. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition , pages 8494-8502, 2018. -
- [39][39] Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, Ilya Sutskever, et al. Language models are unsupervised multitask learners. OpenAI blog , 1(8):9, 2019. -
- [40][40] Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J Liu. Exploring the limits of transfer learning with a unified text-to-text transformer. arXiv preprint arXiv:1910.10683 , 2019. - · arXiv
- [41][41] Nils Reimers and Iryna Gurevych. Sentence-bert: Sentence embeddings using siamese bertnetworks. arXiv preprint arXiv:1908.10084 , 2019. - · arXiv
- [42][42] Adam Roberts, Colin Raffel, and Noam Shazeer. How much knowledge can you pack into the parameters of a language model? arXiv preprint arXiv:2002.08910 , 2020. - · arXiv
- [43][43] Ohad Rubin, Jonathan Herzig, and Jonathan Berant. Learning to retrieve prompts for in-context learning. arXiv preprint arXiv:2112.08633 , 2021. - · arXiv
- [44][44] Manolis Savva, Abhishek Kadian, Oleksandr Maksymets, Yili Zhao, Erik Wijmans, Bhavana Jain, Julian Straub, Jia Liu, Vladlen Koltun, Jitendra Malik, et al. Habitat: A platform for embodied ai research. In Proceedings of the IEEE/CVF International Conference on Computer Vision , pages 9339-9347, 2019. -
- [45][45] Pratyusha Sharma, Antonio Torralba, and Jacob Andreas. Skill induction and planning with latent language. arXiv preprint arXiv:2110.01517 , 2021. - · arXiv
- [46][46] Jianhao Shen, Yichun Yin, Lin Li, Lifeng Shang, Xin Jiang, Ming Zhang, and Qun Liu. Generate &rank: A multi-task framework for math word problems. arXiv preprint arXiv:2109.03034 , 2021. - · arXiv
- [47][47] Mohit Shridhar, Jesse Thomason, Daniel Gordon, Yonatan Bisk, Winson Han, Roozbeh Mottaghi, Luke Zettlemoyer, and Dieter Fox. Alfred: A benchmark for interpreting grounded instructions for everyday tasks. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , pages 10740-10749, 2020. -
- [48][48] Mohit Shridhar, Xingdi Yuan, Marc-Alexandre Côté, Yonatan Bisk, Adam Trischler, and Matthew Hausknecht. Alfworld: Aligning text and embodied environments for interactive learning. arXiv preprint arXiv:2010.03768 , 2020. - · arXiv
- [49][49] Alessandro Suglia, Qiaozi Gao, Jesse Thomason, Govind Thattai, and Gaurav Sukhatme. Embodied bert: A transformer model for embodied, language-guided visual task completion. arXiv preprint arXiv:2108.04927 , 2021. - · arXiv
- [50][50] Alon Talmor, Yanai Elazar, Yoav Goldberg, and Jonathan Berant. olmpics-on what language model pre-training captures. Transactions of the Association for Computational Linguistics , 8: 743-758, 2020. -
- [51][51] Moritz Tenorth, Daniel Nyga, and Michael Beetz. Understanding and executing instructions for everyday manipulation tasks from the world wide web. In 2010 ieee international conference on robotics and automation , pages 1486-1491. IEEE, 2010. -
- [52][52] Maria Tsimpoukelli, Jacob Menick, Serkan Cabi, SM Eslami, Oriol Vinyals, and Felix Hill. Multimodal few-shot learning with frozen language models. arXiv preprint arXiv:2106.13884 , 2021. - · arXiv
- [53][53] Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. Attention is all you need. In Advances in neural information processing systems , pages 5998-6008, 2017. -
- [54][54] Xin Wang, Qiuyuan Huang, Asli Celikyilmaz, Jianfeng Gao, Dinghan Shen, Yuan-Fang Wang, William Yang Wang, and Lei Zhang. Reinforced cross-modal matching and self-supervised imitation learning for vision-language navigation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 6629-6638, 2019. -
- [55][55] Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumond, Clement Delangue, Anthony Moi, Pierric Cistac, Tim Rault, Rémi Louf, Morgan Funtowicz, et al. Huggingface's transformers: State-of-the-art natural language processing. arXiv preprint arXiv:1910.03771 , 2019. · arXiv
Cited by
Sorted by citations
Emergent Abilities of Large Language Models
Wei, J., Tay, Y., Bommasani, R., Raffel, C., Zoph, B., Borgeaud, S., Yogatama, D., Bosma, M., Zhou, D., Metzler, D., et al. (2022) · arXiv · doi:10.48550/arxiv.2206.07682 · Cited by 1,264
Tree of Thoughts: Deliberate Problem Solving with Large Language Models
Yao, S., Yu, D., Zhao, J., Shafran, I., Griffiths, T. L., Cao, Y., & Narasimhan, K. (2023) · arXiv · doi:10.48550/arxiv.2305.10601 · Cited by 1,158
The Rise and Potential of Large Language Model Based Agents: A Survey
Xi, Z., Chen, W., Guo, X., He, W., Ding, Y., Hong, B., Zhang, M., Wang, J., Jin, S., Zhou, E., et al. (2023) · arXiv · doi:10.48550/arxiv.2309.07864 · Cited by 972
The rise and potential of large language model based agents: a survey
Xi, Z., Chen, W., Guo, X., He, W., Ding, Y., Hong, B., Zhang, M., Wang, J., Jin, S., Zhou, E., et al. (2025) · Science China Information Sciences · doi:10.1007/s11432-024-4222-0 · Cited by 970
Code as Policies: Language Model Programs for Embodied Control
Liang, J., Huang, W., Xia, F., Xu, P., Hausman, K., Ichter, B., Florence, P., & Zeng, A. (2023) · 2023 IEEE International Conference on Robotics and Automation (ICRA) · doi:10.1109/icra48891.2023.10160591 · Cited by 767
Self-citationProgPrompt: Generating Situated Robot Task Plans using Large Language Models
Singh, I., Blukis, V., Mousavian, A., Goyal, A., Xu, D., Tremblay, J., Fox, D., Thomason, J., & Garg, A. (2023) · 2023 IEEE International Conference on Robotics and Automation (ICRA) · doi:10.1109/icra48891.2023.10161317 · Cited by 613
ReAct: Synergizing Reasoning and Acting in Language Models
Yao, S., Zhao, J., Yu, D., Du, N., Shafran, I., Narasimhan, K., & Cao, Y. (2022) · arXiv · doi:10.48550/arxiv.2210.03629 · Cited by 608
ChatGPT for Robotics: Design Principles and Model Abilities
Vemprala, S. H., Bonatti, R., Bucker, A., & Kapoor, A. (2024) · IEEE Access · doi:10.1109/access.2024.3387941 · Cited by 570
Graph of Thoughts: Solving Elaborate Problems with Large Language Models
Besta, M., Blach, N., Kubicek, A., Gerstenberger, R., Podstawski, M., Gianinazzi, L., Gajda, J., Lehmann, T., Niewiadomski, H., Nyczyk, P., et al. (2024) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v38i16.29720 · Cited by 556
Towards Reasoning in Large Language Models: A Survey
Huang, J., & Chang, K. C.-C. (2023) · Findings of the Association for Computational Linguistics: ACL 2023 · doi:10.18653/v1/2023.findings-acl.67 · Cited by 428
GPT-NeoX-20B: An Open-Source Autoregressive Language Model
Black, S., Biderman, S., Hallahan, E., Anthony, Q. G., Gao, L., Golding, L., He, H., Leahy, C., McDonell, K., Phang, J., et al. (2022) · doi:10.18653/v1/2022.bigscience-1.9 · Cited by 480
LLM-Planner: Few-Shot Grounded Planning for Embodied Agents with Large Language Models
Song, C. H., Sadler, B. M., Wu, J., Chao, W.-L., Washington, C., & Su, Y. (2023) · 2023 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.00280 · Cited by 435
Visual Language Maps for Robot Navigation
Huang, C., Mees, O., Zeng, A., & Burgard, W. (2023) · 2023 IEEE International Conference on Robotics and Automation (ICRA) · doi:10.1109/icra48891.2023.10160969 · Cited by 376
A Survey on Multimodal Large Language Models for Autonomous Driving
Cui, C., Ma, Y., Cao, X., Ye, W., Zhou, Y., Liang, K., Chen, J., Lu, J., Yang, Z., Liao, K.-D., et al. (2024) · 2024 IEEE/CVF Winter Conference on Applications of Computer Vision Workshops (WACVW) · doi:10.1109/wacvw60836.2024.00106 · Cited by 369
Plan-and-Solve Prompting: Improving Zero-Shot Chain-of-Thought Reasoning by Large Language Models
Wang, L., Xu, W., Lan, Y., Hu, Z., Lan, Y., Lee, R. K.-W., & Lim, E.-P. (2023) · Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/2023.acl-long.147 · Cited by 344
Empowering biomedical discovery with AI agents
Gao, S., Fang, A., Huang, Y., Giunchiglia, V., Noori, A., Schwarz, J. R., Ektefaie, Y., Kondic, J., & Zitnik, M. (2024) · Cell · doi:10.1016/j.cell.2024.09.022 · Cited by 315
Inner Monologue: Embodied Reasoning through Planning with Language Models
Huang, W., Xia, F., Xiao, T., Chan, H., Liang, J., Florence, P., Zeng, A., Tompson, J., Mordatch, I., Chebotar, Y., et al. (2022) · arXiv · doi:10.48550/arxiv.2207.05608 · Cited by 287
“Similar to ours are recent task planning approaches that leverage pre-trained autoregressive LLMs to decompose abstract, high-level instructions into a sequence of low-level steps executable by an agent [20, 21] in a zero-shot manner.”
InfluentialSelf-citationRT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Control
Brohan, A., Brown, N., Carbajal, J., Chebotar, Y., Chen, X., Choromanski, K., Ding, T., Driess, D., Dubey, A., Finn, C., et al. (2023) · arXiv · doi:10.48550/arxiv.2307.15818 · Cited by 276
“Other works have incorporated pre-trained language models, often either as an instruction encoder [63, 64, 43, 41, 44, 1, 65] or for high-level planning [66, 19, 17, 67, 68, 69].”
BackgroundSelf-citationSpatialVLM: Endowing Vision-Language Models with Spatial Reasoning Capabilities
Chen, B., Xu, Z., Kirmani, S., Ichter, B., Sadigh, D., Guibas, L., & Xia, F. (2024) · 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52733.2024.01370 · Cited by 281
SpatialVLM: Endowing Vision-Language Models with Spatial Reasoning Capabilities
Chen, B., Xu, Z., Kirmani, S., Ichter, B., Driess, D., Florence, P., Sadigh, D., Guibas, L., & Xia, F. (2024) · arXiv · doi:10.48550/arxiv.2401.12168 · Cited by 269
Voyager: An Open-Ended Embodied Agent with Large Language Models
Wang, G., Xie, Y., Jiang, Y., Mandlekar, A., Xiao, C., Zhu, Y., Fan, L., & Anandkumar, A. (2023) · arXiv · doi:10.48550/arxiv.2305.16291 · Cited by 222
“1) Large language models for robot learning: Many prior works apply LLMs to generate subgoals for robot planning [27, 27, 25].”
BackgroundMethodSMART-LLM: Smart Multi-Agent Robot Task Planning using Large Language Models
Kannan, S. S., Venkatesh, V. L. N., & Min, B. (2024) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros58592.2024.10802322 · Cited by 306
Social Simulacra: Creating Populated Prototypes for Social Computing Systems
Park, J. S., Popowski, L., Cai, C., Morris, M. R., Liang, P., & Bernstein, M. S. (2022) · Proceedings of the 35th Annual ACM Symposium on User Interface Software and Technology · doi:10.1145/3526113.3545616 · Cited by 212
Fan, H., Liu, X., Fuh, J. Y. H., Lu, W. F., & Li, B. (2024) · Journal of Intelligent Manufacturing · doi:10.1007/s10845-023-02294-y · Cited by 194
Expel: LLM agents are experiential learners
Zhao, A., Huang, D., Xu, Q., Lin, M., Liu, Y.-J., & Huang, G. (2024) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v38i17.29936 · Cited by 153
From text to insight: large language models for chemical data extraction
Schilling-Wilhelmi, M., Ríos-García, M., Shabih, S., Gil, M. V., Miret, S., Koch, C. T., Marquez, J. A., & Jablonka, K. M. (2024) · Chemical Society Reviews · doi:10.1039/d4cs00913d · Cited by 178
Open-vocabulary Queryable Scene Representations for Real World Planning
Chen, B., Xia, F., Ichter, B., Rao, K., Gopalakrishnan, K., Ryoo, M., Stone, A., & Kappler, D. (2022) · IEEE International Conference on Robotics and Automation · doi:10.1109/icra48891.2023.10161534 · Cited by 273
What does CLIP know about a red circle? Visual prompt engineering for VLMs
Shtedritski, A., Rupprecht, C., & Vedaldi, A. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.01101 · Cited by 191
Task and Motion Planning with Large Language Models for Object Rearrangement
Ding, Y. F., Zhang, X., Paxton, C., & Zhang, S. (2023) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros55552.2023.10342169 · Cited by 196
Large Language Models as Evolutionary Optimizers
Liu, S., Chen, C., Qu, X., Tang, K., & Ong, Y.-S. (2024) · IEEE Congress on Evolutionary Computation (CEC) · doi:10.1109/cec60901.2024.10611913 · Cited by 209
3D-VisTA: Pre-trained Transformer for 3D Vision and Text Alignment
Zhu, Z., Ma, X., Chen, Y., Deng, Z., Huang, S., & Li, Q. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.00272 · Cited by 155
Thinking in Space: How Multimodal Large Language Models See, Remember, and Recall Spaces
Yang, J., Yang, S., Gupta, A. W., Han, R., Fei-Fei, L., & Xie, S. (2025) · 2025 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52734.2025.00994 · Cited by 120
Interactive Language: Talking to Robots in Real Time
Lynch, C., Wahid, A., Tompson, J., Ding, T., Betker, J., Baruch, R., Armstrong, T., & Florence, P. (2022) · arXiv · doi:10.48550/arxiv.2210.06407 · Cited by 113
Large language models for chemistry robotics
Yoshikawa, N., Skreta, M., Darvish, K., Arellano-Rubach, S., Ji, Z., Bjørn Kristensen, L., Li, A. Z., Zhao, Y., Xu, H., Kuramshin, A., et al. (2023) · Autonomous Robots · doi:10.1007/s10514-023-10136-2 · Cited by 112
Physically Grounded Vision-Language Models for Robotic Manipulation
Gao, J., Sarkar, B., Xia, F., Xiao, T. G., Wu, J., Ichter, B., Majumdar, A., & Sadigh, D. (2024) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra57147.2024.10610090 · Cited by 185
GraspGPT: Leveraging Semantic Knowledge From a Large Language Model for Task-Oriented Grasping
Tang, C., Huang, D., Ge, W., Liu, W., & Zhang, H. (2023) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2023.3320012 · Cited by 117
RobotGPT: Robot Manipulation Learning From ChatGPT
Jin, Y., Li, D., A, Y., Shi, J., Hao, P., Sun, F., Zhang, J., & Fang, B. (2024) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2024.3357432 · Cited by 103
AutoTAMP: Autoregressive Task and Motion Planning with LLMs as Translators and Checkers
Chen, Y., Arkin, J., Dawson, C., Zhang, Y., Roy, N., & Fan, C. (2024) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra57147.2024.10611163 · Cited by 146
ChatGPT Empowered Long-Step Robot Control in Various Environments: A Case Application
Wake, N., Kanehira, A., Sasabuchi, K., Takamatsu, J., & Ikeuchi, K. (2023) · IEEE Access · doi:10.1109/access.2023.3310935 · Cited by 96
Scalable Multi-Robot Collaboration with Large Language Models: Centralized or Decentralized Systems?
Chen, Y., Arkin, J., Zhang, Y., Roy, N., & Fan, C. (2024) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra57147.2024.10610676 · Cited by 152
Large Language Models as Zero-Shot Human Models for Human-Robot Interaction
Zhang, B., & Soh, H. (2023) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros55552.2023.10341488 · Cited by 112
Can an Embodied Agent Find Your “Cat-shaped Mug”? LLM-Based Zero-Shot Object Navigation
Dorbala, V. S., Mullen, J. F., & Manocha, D. (2023) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2023.3346800 · Cited by 114
Theory of Mind for Multi-Agent Collaboration via Large Language Models
Li, H., Yu, C. H. A., Stepputtis, S., Campbell, J., Hughes, D., Lewis, C., & Sycara, K. P. (2023) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2023.emnlp-main.13 · Cited by 192
Generalized Planning in PDDL Domains with Pretrained Large Language Models
Silver, T., Dan, S., Srinivas, K., Tenenbaum, J. B., Kaelbling, L. P., & Katz, M. (2024) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v38i18.30006 · Cited by 150
Kumaran, V., Rowe, J., Mott, B., & Lester, J. (2023) · Proceedings of the AAAI Conference on Artificial Intelligence and Interactive Digital Entertainment · doi:10.1609/aiide.v19i1.27504 · Cited by 78
AgentAI: A comprehensive survey on autonomous agents in distributed AI for industry 4.0
Piccialli, F., Chiaro, D., Sarwar, S., Cerciello, D., Qi, P., & Mele, V. (2025) · Expert Systems with Applications · doi:10.1016/j.eswa.2025.128404 · Cited by 77
TidyBot: Personalized Robot Assistance with Large Language Models
Wu, J., Antonova, R., Kan, A., Lepert, M., Zeng, A., Song, S., Bohg, J., Rusinkiewicz, S. M., & Funkhouser, T. A. (2023) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros55552.2023.10341577 · Cited by 85
ISR-LLM: Iterative Self-Refined Large Language Model for Long-Horizon Sequential Task Planning
Zhou, Z., Song, J., Yao, K., Shu, Z., & Ma, L. (2024) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra57147.2024.10610065 · Cited by 102
Dehallucinating Large Language Models Using Formal Methods Guided Iterative Prompting
Jha, S., Jha, S. K., Lincoln, P. D., Bastian, N. D., Velasquez, A., & Neema, S. K. (2023) · IEEE International Conference on Assured Autonomy (ICAA) · doi:10.1109/icaa58325.2023.00029 · Cited by 81
Cognitive Architectures for Language Agents
Sumers, T. R., Yao, S., Narasimhan, K., & Griffiths, T. L. (2023) · arXiv · doi:10.48550/arxiv.2309.02427 · Cited by 72
“Empirically, many early language agents simply use LLMs to propose an action (Schick et al., 2023), a sequence of actions (Huang et al., 2022b), or evaluate a fixed set of actions (Ahn et al., 2022) without intermediate reasoning or retrieval.”
BackgroundInfluentialGPT-4V(ision) for Robotics: Multimodal Task Planning From Human Demonstration
Wake, N., Kanehira, A., Sasabuchi, K., Takamatsu, J., & Ikeuchi, K. (2023) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2024.3477090 · Cited by 147
Qin, Y., Liang, S., Ye, Y., Zhu, K., Yan, L., Lu, Y., Lin, Y., Cong, X., Tang, X., Qian, B., et al. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2307.16789 · Cited by 71
Semantic Scene Understanding with Large Language Models on Unmanned Aerial Vehicles
de Curtò, J., de Zarzà, I., & Calafate, C. T. (2023) · Drones · doi:10.3390/drones7020114 · Cited by 70
Bridging Zero-shot Object Navigation and Foundation Models through Pixel-Guided Navigation Skill
Cai, W., Huang, S., Cheng, G., Long, Y., Gao, P., Sun, C., & Dong, H. (2024) · 2024 IEEE International Conference on Robotics and Automation (ICRA) · doi:10.1109/icra57147.2024.10610499 · Cited by 63
Chat with the Environment: Interactive Multimodal Perception Using Large Language Models
Zhao, X., Li, M., Weber, C., Hafez, M. B., & Wermter, S. (2023) · 2023 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros55552.2023.10342363 · Cited by 62
CoPa: General Robotic Manipulation through Spatial Constraints of Parts with Foundation Models
Huang, H.-X., Lin, F., Hu, Y.-D., Wang, S.-J., & Gao, Y. (2024) · IEEE/RJS International Conference on Intelligent RObots and Systems · doi:10.1109/iros58592.2024.10801352 · Cited by 182
Language Models as Zero-Shot Trajectory Generators
Kwon, T., Palo, N. D., & Johns, E. (2024) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2024.3410155 · Cited by 59
A Survey of Optimization-Based Task and Motion Planning: From Classical to Learning Approaches
Zhao, Z., Cheng, S., Ding, Y. F., Zhou, Z., Zhang, S., Xu, D., & Zhao, Y. (2024) · IEEE/ASME Transactions on Mechatronics · doi:10.1109/tmech.2024.3452509 · Cited by 79
LATTE: LAnguage Trajectory TransformEr
Bucker, A., Figueredo, L., Haddadin, S., Kapoor, A., Ma, S., Vemprala, S. H., & Bonatti, R. (2023) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra48891.2023.10161068 · Cited by 70
Deng, Y., Liao, L., Chen, L., Wang, H., Lei, W., & Chua, T. (2023) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2023.findings-emnlp.711 · Cited by 113
Plug in the Safety Chip: Enforcing Constraints for LLM-driven Robot Agents
Yang, Z., Raman, S. S., Shah, A., & Tellex, S. (2024) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra57147.2024.10611447 · Cited by 84
Pallagani, V., Muppasani, B. C., Roy, K., Fabiano, F., Loreggia, A., Murugesan, K., Srivastava, B., Rossi, F., Lior Horesh, C., & Sheth, A. (2024) · Proceedings of the International Conference on Automated Planning and Scheduling · doi:10.1609/icaps.v34i1.31503 · Cited by 78
LLM-Based Multi-Agent Decision-Making: Challenges and Future Directions
Sun, C., Huang, S., & Pompili, D. (2025) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2025.3562371 · Cited by 65
Bucker, A., Figueredo, L., Haddadinl, S., Kapoor, A., Ma, S., & Bonatti, R. (2022) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros47612.2022.9981810 · Cited by 55
Shi, W., Xu, R., Zhuang, Y., Yu, Y., Zhang, J., Wu, H., Zhu, Y., Ho, J. C., Yang, C., & Wang, M. D. (2024) · Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/2024.emnlp-main.1245 · Cited by 54
“…and efficiently mitigating error propagation and learning from environmental feedback (Qiao et al., 2023; Shinn et al., 2023; Yang et al., 2023 In such cases, it is generally preferable to acknowledge failure rather than generate an incorrect answer, as this could lead to an inaccurate diagnosis.”
BackgroundRoboVQA: Multimodal Long-Horizon Reasoning for Robotics
Sermanet, P., Ding, T., Zhao, J. W., Xia, F., Dwibedi, D., Gopalakrishnan, K., Chan, C., Dulac-Arnold, G., Maddineni, S., Joshi, N. J., et al. (2024) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra57147.2024.10610216 · Cited by 108
Vision-Language Interpreter for Robot Task Planning
Shirai, K., Beltran-Hernandez, C. C., Hamaya, M., Hashimoto, A., Tanaka, S., Kawaharazuka, K., Tanaka, K., Ushiku, Y., & Mori, S. (2024) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra57147.2024.10611112 · Cited by 53
Transfer learning in robotics: An upcoming breakthrough? A review of promises and challenges
Jaquier, N., Welle, M. C., Gams, A., Yao, K., Fichera, B., Billard, A., Ude, A., Asfour, T., & Kragić, D. (2024) · The International Journal of Robotics Research · doi:10.1177/02783649241273565 · Cited by 51
Diffusion-ES: Gradient-Free Planning with Diffusion for Autonomous and Instruction-Guided Driving
Yang, B., Su, H., Gkanatsios, N., Ke, T.-W., Jain, A., Schneider, J., & Fragkiadaki, K. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.01453 · Cited by 88
Proactive Conversational AI: A Comprehensive Survey of Advancements and Opportunities
Deng, Y., Liao, L., Lei, W., Yang, G. H., Lam, W. P., & Chua, T. (2025) · ACM Transactions on Information Systems · doi:10.1145/3715097 · Cited by 67
Prompt a Robot to Walk with Large Language Models
Wang, Y., Zhang, B., Chen, J., & Sreenath, K. (2024) · IEEE Conference on Decision and Control (CDC) · doi:10.1109/cdc56724.2024.10885862 · Cited by 86
Robot learning in the era of foundation models: a survey
Xiao, X., Liu, J., Wang, Z., Zhou, Y., Qi, Y., Jiang, S., He, B., & Cheng, Q. (2025) · Neurocomputing · doi:10.1016/j.neucom.2025.129963 · Cited by 56
Prompt, Plan, Perform: LLM-based Humanoid Control via Quantized Imitation Learning
Sun, J., Zhang, Q., Duan, Y., Jiang, X., Chen, C., & Xu, R. (2024) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra57147.2024.10610948 · Cited by 52
Embodied Multi-Modal Agent trained by an LLM from a Parallel TextWorld
Yang, Y., Zhou, T., Li, K., Tao, D., Li, L., Shen, L., He, X., Jiang, J., & Shi, Y. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.02482 · Cited by 51
Incremental learning of humanoid robot behavior from natural interaction and large language models
Bärmann, L., Kartmann, R., Peller-Konrad, F., Niehues, J., Waibel, A., & Asfour, T. (2024) · Frontiers in Robotics and AI · doi:10.3389/frobt.2024.1455375 · Cited by 46
Cue-CoT: Chain-of-thought Prompting for Responding to In-depth Dialogue Questions with LLMs
Wang, H., Wang, R., Mi, F., Deng, Y., Wang, Z., Liang, B., Xu, R., & Wong, K.-F. (2023) · Findings of the Association for Computational Linguistics: EMNLP 2023 · doi:10.18653/v1/2023.findings-emnlp.806 · Cited by 38
Deploying and Evaluating LLMs to Program Service Mobile Robots
Hu, Z., Lucchetti, F., Schlesinger, C., Saxena, Y., Freeman, A., Modak, S., Guha, A., & Biswas, J. (2024) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2024.3360020 · Cited by 51
Natural language instructions induce compositional generalization in networks of neurons
Riveland, R., & Pouget, A. (2024) · Nature Neuroscience · doi:10.1038/s41593-024-01607-5 · Cited by 53
SayCanPay: Heuristic Planning with Large Language Models Using Learnable Domain Knowledge
Hazra, R., Martires, P. Z. D., & Raedt, L. D. (2024) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v38i18.29991 · Cited by 51
Distilling and Retrieving Generalizable Knowledge for Robot Manipulation via Language Corrections
Zha, L., Cui, Y., Lin, L.-H., Kwon, M., Arenas, M. G., Zeng, A., Xia, F., & Sadigh, D. (2024) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra57147.2024.10610455 · Cited by 56
Visual Programming for Zero-Shot Open-Vocabulary 3D Visual Grounding
Yuan, Z., Ren, J., Feng, C.-M., Zhao, H., Cui, S. R., & Li, Z. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.01949 · Cited by 51
LLM 3 : Large Language Model-based Task and Motion Planning with Motion Failure Reasoning
Wang, S., Han, M., Jiao, Z., Zhang, Z., Wu, Y. N., Zhu, S. C., & Liu, H. (2024) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros58592.2024.10801328 · Cited by 95
March in Chat: Interactive Prompting for Remote Embodied Referring Expression
Qiao, Y., Qi, Y., Zheng, Y., Liu, J., & Wu, Q. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.01444 · Cited by 47
CoPAL: Corrective Planning of Robot Actions with Large Language Models
Joublin, F., Ceravola, A., Smirnov, P., Ocker, F., Deigmoeller, J., Belardinelli, A., Wang, C., Hasler, S., Tanneberg, D., & Gienger, M. (2024) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra57147.2024.10610434 · Cited by 65
PANGeA: Procedural Artificial Narrative Using Generative AI for Turn-Based, Role-Playing Video Games
Buongiorno, S., Klinkert, L., Zhuang, Z., Chawla, T., & Clark, C. (2024) · Proceedings of the AAAI Conference on Artificial Intelligence and Interactive Digital Entertainment · doi:10.1609/aiide.v20i1.31876 · Cited by 37
Statler: State-Maintaining Language Models for Embodied Reasoning
Yoneda, T., Fang, J., Li, P., Zhang, H., Jiang, T., Lin, S., Picker, B., Yunis, D., Mei, H., & Walter, M. R. (2024) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra57147.2024.10610634 · Cited by 62
Language Models Meet World Models: Embodied Experiences Enhance Language Models
Xiang, J., Tao, T., Gu, Y., Shu, T., Wang, Z., Yang, Z., & Hu, Z. (2023) · Advances in Neural Information Processing Systems 36 · doi:10.52202/075280-3295 · Cited by 34
How to Prompt Your Robot: A PromptBook for Manipulation Skills with Code as Policies
Arenas, M. G., Xiao, T. G., Singh, S., Jain, V., Ren, A. Z., Vuong, Q., Varley, J., Herzog, A., Leal, I. M. P., Kirmani, S., et al. (2024) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra57147.2024.10610784 · Cited by 40
DoReMi: Grounding Language Model by Detecting and Recovering from Plan-Execution Misalignment
Guo, Y., Wang, Y.-J., Zha, L.-H., Jiang, Z., & Chen, J. (2023) · IEEE/RJS International Conference on Intelligent RObots and Systems · doi:10.1109/iros58592.2024.10802284 · Cited by 118
Robotic Skill Acquisition via Instruction Augmentation with Vision-Language Models
Xiao, T. G., Chan, H., Sermanet, P., Wahid, A., Brohan, A., Hausman, K., Levine, S., & Tompson, J. (2023) · Robotics: Science and Systems · doi:10.15607/rss.2023.xix.029 · Cited by 84
Zhu, X., Chen, Y., Tian, H., Tao, C., Su, W., Yang, C., Huang, G., Li, B., Lu, L., Wang, X., et al. (2023) · arXiv · doi:10.48550/arxiv.2305.17144 · Cited by 33
WebCPM: Interactive Web Search for Chinese Long-form Question Answering
Qin, Y., Cai, Z., Jin, D., Yan, L., Liang, S., Zhu, K., Lin, Y., Han, X., Ding, N., Wang, H., et al. (2023) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2023.acl-long.499 · Cited by 78
LLM-based Robot Task Planning with Exceptional Handling for General Purpose Service Robots
Wang, R., Yang, Z., Zhao, Z., Tong, X., Hong, Z., & Qian, K. (2024) · Chinese Control Conference (CCC) · doi:10.23919/ccc63176.2024.10661966 · Cited by 47
Embodied Multi-Agent Systems: A Review
Li, Z., Wu, W., Guo, Y., Sun, J. Q., & Han, Q. (2025) · IEEE/CAA Journal of Automatica Sinica · doi:10.1109/jas.2025.125552 · Cited by 42
A LLM-informed multi-agent AI system for drone-based visual inspection for infrastructure
Liu, J., Li, H., Chai, C., Chen, K., & Wang, D. (2025) · Advanced Engineering Informatics · doi:10.1016/j.aei.2025.103643 · Cited by 32
CLARA: Classifying and Disambiguating User Commands for Reliable Interactive Robotic Agents
Park, J., Lim, S., Lee, J., Park, S., Chang, M., Yu, Y., & Choi, S. (2023) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2023.3338514 · Cited by 44
TR2MTL: LLM based framework for Metric Temporal Logic Formalization of Traffic Rules
Manas, K., Zwicklbauer, S., & Paschke, A. (2024) · IEEE Intelligent Vehicles Symposium · doi:10.1109/iv55156.2024.10588650 · Cited by 34
Multimodal large language models for inclusive collaboration learning tasks
Lewis, A. (2022) · Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies: Student Research Workshop · doi:10.18653/v1/2022.naacl-srw.26 · Cited by 30
FLTRNN: Faithful Long-Horizon Task Planning for Robotics with Large Language Models
Zhang, J., Tang, L., Song, Y., Meng, Q., Qian, H., Shao, J., Song, W., Zhu, S., & Gu, J. J. (2024) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra57147.2024.10611663 · Cited by 32
Toward Grounded Commonsense Reasoning
Kwon, M., Hu, H., Myers, V., Karamcheti, S., Dragan, A. D., & Sadigh, D. (2024) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra57147.2024.10611218 · Cited by 41
Metadata
- Authors
- Huang, W., Abbeel, P., Pathak, D., & Mordatch, I.
- Affiliations
- University of California, Berkeley (US)
- Carnegie Mellon University (US)
- Venue
- arXiv
- Publisher
- arXiv
- Published
- 2022
- ISSN
- 2331-8422 (ISSN-L)
- arXiv
- 2201.07207
- OpenAlex
- W4221152848
- Cited by
- 304
- References
- 0
- Citations by year
- 2026: 6 · 2025: 62 · 2024: 146 · 2023: 83 · 2022: 7
- Type
- posted content
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Funders
- National Science Foundation (2024594)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › AI-based Problem Solving and Planning
- Physical Sciences › Engineering › Control and Systems Engineering › Robot Manipulation and Learning
- Physical Sciences › Computer Science › Artificial Intelligence › Reinforcement Learning in Robotics
- Keywords
- large language models, task decomposition, embodied agents, semantic translation, human evaluation
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
