The DOI matches its DataCite record; 2 of the 2 references it cites were found in a registry or an index.
Language Models as Zero-Shot Planners: Extracting Actionable Knowledge for Embodied Agents
Open accessAbstract
Can world knowledge learned by large language models (LLMs) be used to act in interactive environments? In this paper, we investigate the possibility of grounding high-level tasks, expressed in natural language (e.g. "make breakfast"), to a chosen set of actionable steps (e.g. "open fridge"). While prior work focused on learning from explicit step-by-step examples of how to act, we surprisingly find that if pre-trained LMs are large enough and prompted appropriately, they can effectively decompose high-level tasks into mid-level plans without any further training. However, the plans produced naively by LLMs often cannot map precisely to admissible actions. We propose a procedure that conditions on existing demonstrations and semantically translates the plans to admissible actions. Our evaluation in the recent VirtualHome environment shows that the resulting method substantially improves executability over the LLM baseline. The conducted human evaluation reveals a trade-off between executability and correctness but shows a promising sign towards extracting actionable knowledge from language models. Website at https://huangwl18.github.io/language-planner
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Several works use LLMs as interactive policies, demonstrating their strong capabilities but also highlighting challenges in grounding them with observations and actions (Huang et al., 2022;Yao et al., 2023;Ahn et al., 2022;Huang et al., 2023;Zeng et al., 2022).”
From PARTNR: A Benchmark for Planning and Reasoning in Embodied Multi-agent Tasks (2024) · cites this paper as Huang et al., 2022
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Agents equipped with LLM-based planners have started to show the ability to learn a new task with a few training examples.”
From LLM-Planner: Few-Shot Grounded Planning for Embodied Agents with Large Language Models (2023) · cites this paper as [13]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“However, these pre-trained models, especially the VLMs, are not explicitly designed for physical interactions and, therefore, often lack the physical common sense understanding and result in suboptimal plans and execution.”
From Cosmos-Reason1: From Physical Common Sense To Embodied Reasoning (2025) · cites this paper as Huang et al., 2022
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“For instance, language models have been used via prompting to translate natural language instructions into actions executable by robots (Ahn et al., 2022; Huang et al., 2022), interact with users (Coenen et al., 2021; Wu et al., 2021; 2022a; Lee et al., 2022b), and facilitate multi-modal reasoning (Zeng et al., 2022; Alayrac et al., 2022).”
From Emergent Abilities of Large Language Models (2022) · cites this paper as Huang et al., 2022
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“Once learned, language can unlock planning, reasoning (e.g., Huang et al., 2022;Brohan et al., 2023b;Driess et al., 2023;Kim et al., 2023), and communication (Zeng et al., 2022) about grounded situations and tasks.”
From Scaling Instructable Agents Across Many Simulated Worlds (2024) · cites this paper as Huang et al., 2022
The cited source says what the claim attributes to it.
In the citing paper
“Large language models (LLMs) have the ability to understand tasks described in natural language and generate plans to solve them [20, 27, 49, 60].”
From AgentDojo: A Dynamic Environment to Evaluate Prompt Injection Attacks and Defenses for LLM Agents (2024) · cites this paper as [20]
The cited source says what the claim attributes to it.
In the citing paper
“In the specific arena of autonomous task completion, LLMs struggle with making task breakdowns that are both correct and executable [10].”
From Autonomous Task Completion Based on Goal-directed Answer Set Programming (2025) · cites this paper as [10]
The cited source says what the claim attributes to it.
In the citing paper
“In these works, LLMs are often used to help an agent plan its actions [4], [13], [14], despite evidence that they may not be capable of reliably planning [15].”
From Episodic Memory in AI Agents Poses Risks that Should be Studied and Mitigated (2025) · cites this paper as [13]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
The publisher hasn’t deposited this paper’s reference list.
Research works printed but not linked: 51
- [1][1] Peter Anderson, Qi Wu, Damien Teney, Jake Bruce, Mark Johnson, Niko Sünderhauf, Ian Reid, Stephen Gould, and Anton Van Den Hengel. Vision-and-language navigation: Interpreting visually-grounded navigation instructions in real environments. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition , pages 3674-3683, 2018. -
- [2][2] Yoav Artzi and Luke Zettlemoyer. Weakly supervised learning of semantic parsers for mapping instructions to actions. Transactions of the Association for Computational Linguistics , 1:49-62, 2013. -
- [6][6] Daniel Cer, Mona Diab, Eneko Agirre, Inigo Lopez-Gazpio, and Lucia Specia. Semeval-2017 task 1: Semantic textual similarity-multilingual and cross-lingual focused evaluation. arXiv preprint arXiv:1708.00055 , 2017. - · arXiv
- [7][7] Mark Chen, Jerry Tworek, Heewoo Jun, Qiming Yuan, Henrique Ponde, Jared Kaplan, Harri Edwards, Yura Burda, Nicholas Joseph, Greg Brockman, et al. Evaluating large language models trained on code. arXiv preprint arXiv:2107.03374 , 2021. - · arXiv
- [8][8] Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Jacob Hilton, Reiichiro Nakano, Christopher Hesse, and John Schulman. Training verifiers to solve math word problems. arXiv preprint arXiv:2110.14168 , 2021. - · arXiv
- [9][9] Joe Davison, Joshua Feldman, and Alexander M Rush. Commonsense knowledge mining from pretrained models. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP) , pages 1173-1178, 2019. -
- [11][11] Daniel Fried, Ronghang Hu, Volkan Cirik, Anna Rohrbach, Jacob Andreas, Louis-Philippe Morency, Taylor Berg-Kirkpatrick, Kate Saenko, Dan Klein, and Trevor Darrell. Speakerfollower models for vision-and-language navigation. arXiv preprint arXiv:1806.02724 , 2018. - · arXiv
- [12][12] Justin Fu, Anoop Korattikara, Sergey Levine, and Sergio Guadarrama. From language to goals: Inverse reinforcement learning for vision-based instruction following. arXiv preprint arXiv:1902.07742 , 2019. - · arXiv
- [13][13] Tianyu Gao, Adam Fisch, and Danqi Chen. Making pre-trained language models better few-shot learners. arXiv preprint arXiv:2012.15723 , 2020. - · arXiv
- [14][14] Brent Harrison and Mark O Riedl. Learning from stories: using crowdsourced narratives to train virtual agents. In Twelfth Artificial Intelligence and Interactive Digital Entertainment Conference , 2016. -
- [15][15] Dan Hendrycks, Steven Basart, Saurav Kadavath, Mantas Mazeika, Akul Arora, Ethan Guo, Collin Burns, Samir Puranik, Horace He, Dawn Song, et al. Measuring coding challenge competence with apps. arXiv preprint arXiv:2105.09938 , 2021. - · arXiv
- [16][16] Felix Hill, Sona Mokra, Nathaniel Wong, and Tim Harley. Human instruction-following with deep reinforcement learning via transfer-learning from text. arXiv preprint arXiv:2005.09382 , 2020. - · arXiv
- [17][17] Sepp Hochreiter and Jürgen Schmidhuber. Long short-term memory. Neural computation , 9(8): 1735-1780, 1997. -
- [18][18] Ari Holtzman, Jan Buys, Li Du, Maxwell Forbes, and Yejin Choi. The curious case of neural text degeneration. arXiv preprint arXiv:1904.09751 , 2019. - · arXiv
- [19][19] Gabriel Ilharco, Rowan Zellers, Ali Farhadi, and Hannaneh Hajishirzi. Probing text models for common ground with visual representations. arXiv e-prints , pages arXiv-2005, 2020. -
- [20][20] Peter A Jansen. Visually-grounded planning without vision: Language models infer detailed plans from high-level instructions. arXiv preprint arXiv:2009.14259 , 2020. - · arXiv
- [21][21] Zhengbao Jiang, Frank F Xu, Jun Araki, and Graham Neubig. How can we know what language models know? Transactions of the Association for Computational Linguistics , 8:423-438, 2020. -
- [22][22] Eric Kolve, Roozbeh Mottaghi, Winson Han, Eli VanderBilt, Luca Weihs, Alvaro Herrasti, Daniel Gordon, Yuke Zhu, Abhinav Gupta, and Ali Farhadi. Ai2-thor: An interactive 3d environment for visual ai. arXiv preprint arXiv:1712.05474 , 2017. - · arXiv
- [23][23] Belinda Z Li, Maxwell Nye, and Jacob Andreas. Implicit representations of meaning in neural language models. arXiv preprint arXiv:2106.00737 , 2021. - · arXiv
- [24][24] Shuang Li, Xavier Puig, Yilun Du, Clinton Wang, Ekin Akyurek, Antonio Torralba, Jacob Andreas, and Igor Mordatch. Pre-trained language models for interactive decision-making. arXiv preprint arXiv:2202.01771 , 2022. - · arXiv
- [25][25] Yuan-Hong Liao, Xavier Puig, Marko Boben, Antonio Torralba, and Sanja Fidler. Synthesizing environment-aware activities via activity sketches. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 6291-6299, 2019. -
- [26][26] Jiachang Liu, Dinghan Shen, Yizhe Zhang, Bill Dolan, Lawrence Carin, and Weizhu Chen. What makes good in-context examples for gpt3 ? arXiv preprint arXiv:2101.06804 , 2021. - · arXiv
- [27][27] Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. Roberta: A robustly optimized bert pretraining approach. arXiv preprint arXiv:1907.11692 , 2019. - · arXiv
- [28][28] Kevin Lu, Aditya Grover, Pieter Abbeel, and Igor Mordatch. Pretrained transformers as universal computation engines. arXiv preprint arXiv:2103.05247 , 2021. - · arXiv
- [29][29] Corey Lynch and Pierre Sermanet. Grounding language in play. arXiv preprint arXiv:2005.07648 , 2020. - · arXiv
- [30][30] Corey Lynch and Pierre Sermanet. Language conditioned imitation learning over unstructured data. Proceedings of Robotics: Science and Systems. doi , 10, 2021. -
- [31][31] Arjun Majumdar, Ayush Shrivastava, Stefan Lee, Peter Anderson, Devi Parikh, and Dhruv Batra. Improving vision-and-language navigation with image-text pairs from the web. In European Conference on Computer Vision , pages 259-274. Springer, 2020. -
- [32][32] Suvir Mirchandani, Siddharth Karamcheti, and Dorsa Sadigh. Ella: Exploration through learned language abstraction. arXiv preprint arXiv:2103.05825 , 2021. - · arXiv
- [33][33] Dipendra Misra, Kejia Tao, Percy Liang, and Ashutosh Saxena. Environment-driven lexicon induction for high-level instructions. In Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing (Volume 1: Long Papers) , pages 992-1002, 2015. -
- [34][34] Dipendra K Misra, Jaeyong Sung, Kevin Lee, and Ashutosh Saxena. Tell me dave: Contextsensitive grounding of natural language to manipulation instructions. The International Journal of Robotics Research , 35(1-3):281-300, 2016. -
- [35][35] Jeffrey Pennington, Richard Socher, and Christopher D Manning. Glove: Global vectors for word representation. In Proceedings of the 2014 conference on empirical methods in natural language processing (EMNLP) , pages 1532-1543, 2014. -
- [36][36] Fabio Petroni, Tim Rocktäschel, Patrick Lewis, Anton Bakhtin, Yuxiang Wu, Alexander H Miller, and Sebastian Riedel. Language models as knowledge bases? arXiv preprint arXiv:1909.01066 , 2019. - · arXiv
- [37][37] Gabriel Poesia, Oleksandr Polozov, Vu Le, Ashish Tiwari, Gustavo Soares, Christopher Meek, and Sumit Gulwani. Synchromesh: Reliable code generation from pre-trained language models. arXiv preprint arXiv:2201.11227 , 2022. - · arXiv
- [38][38] Xavier Puig, Kevin Ra, Marko Boben, Jiaman Li, Tingwu Wang, Sanja Fidler, and Antonio Torralba. Virtualhome: Simulating household activities via programs. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition , pages 8494-8502, 2018. -
- [39][39] Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, Ilya Sutskever, et al. Language models are unsupervised multitask learners. OpenAI blog , 1(8):9, 2019. -
- [40][40] Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J Liu. Exploring the limits of transfer learning with a unified text-to-text transformer. arXiv preprint arXiv:1910.10683 , 2019. - · arXiv
- [41][41] Nils Reimers and Iryna Gurevych. Sentence-bert: Sentence embeddings using siamese bertnetworks. arXiv preprint arXiv:1908.10084 , 2019. - · arXiv
- [42][42] Adam Roberts, Colin Raffel, and Noam Shazeer. How much knowledge can you pack into the parameters of a language model? arXiv preprint arXiv:2002.08910 , 2020. - · arXiv
- [43][43] Ohad Rubin, Jonathan Herzig, and Jonathan Berant. Learning to retrieve prompts for in-context learning. arXiv preprint arXiv:2112.08633 , 2021. - · arXiv
- [44][44] Manolis Savva, Abhishek Kadian, Oleksandr Maksymets, Yili Zhao, Erik Wijmans, Bhavana Jain, Julian Straub, Jia Liu, Vladlen Koltun, Jitendra Malik, et al. Habitat: A platform for embodied ai research. In Proceedings of the IEEE/CVF International Conference on Computer Vision , pages 9339-9347, 2019. -
- [45][45] Pratyusha Sharma, Antonio Torralba, and Jacob Andreas. Skill induction and planning with latent language. arXiv preprint arXiv:2110.01517 , 2021. - · arXiv
- [46][46] Jianhao Shen, Yichun Yin, Lin Li, Lifeng Shang, Xin Jiang, Ming Zhang, and Qun Liu. Generate &rank: A multi-task framework for math word problems. arXiv preprint arXiv:2109.03034 , 2021. - · arXiv
- [47][47] Mohit Shridhar, Jesse Thomason, Daniel Gordon, Yonatan Bisk, Winson Han, Roozbeh Mottaghi, Luke Zettlemoyer, and Dieter Fox. Alfred: A benchmark for interpreting grounded instructions for everyday tasks. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , pages 10740-10749, 2020. -
- [48][48] Mohit Shridhar, Xingdi Yuan, Marc-Alexandre Côté, Yonatan Bisk, Adam Trischler, and Matthew Hausknecht. Alfworld: Aligning text and embodied environments for interactive learning. arXiv preprint arXiv:2010.03768 , 2020. - · arXiv
- [49][49] Alessandro Suglia, Qiaozi Gao, Jesse Thomason, Govind Thattai, and Gaurav Sukhatme. Embodied bert: A transformer model for embodied, language-guided visual task completion. arXiv preprint arXiv:2108.04927 , 2021. - · arXiv
- [50][50] Alon Talmor, Yanai Elazar, Yoav Goldberg, and Jonathan Berant. olmpics-on what language model pre-training captures. Transactions of the Association for Computational Linguistics , 8: 743-758, 2020. -
- [51][51] Moritz Tenorth, Daniel Nyga, and Michael Beetz. Understanding and executing instructions for everyday manipulation tasks from the world wide web. In 2010 ieee international conference on robotics and automation , pages 1486-1491. IEEE, 2010. -
- [52][52] Maria Tsimpoukelli, Jacob Menick, Serkan Cabi, SM Eslami, Oriol Vinyals, and Felix Hill. Multimodal few-shot learning with frozen language models. arXiv preprint arXiv:2106.13884 , 2021. - · arXiv
- [53][53] Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. Attention is all you need. In Advances in neural information processing systems , pages 5998-6008, 2017. -
- [54][54] Xin Wang, Qiuyuan Huang, Asli Celikyilmaz, Jianfeng Gao, Dinghan Shen, Yuan-Fang Wang, William Yang Wang, and Lei Zhang. Reinforced cross-modal matching and self-supervised imitation learning for vision-language navigation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 6629-6638, 2019. -
- [55][55] Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumond, Clement Delangue, Anthony Moi, Pierric Cistac, Tim Rault, Rémi Louf, Morgan Funtowicz, et al. Huggingface's transformers: State-of-the-art natural language processing. arXiv preprint arXiv:1910.03771 , 2019. · arXiv
Cited by
Sorted by citations
Items 201–300 of 340
Patient Centric Summarization of Radiology Findings using Large Language ModelsPreprint
Tariq, A., Fathizadeh, S., Ramaswamy, G., Trivedi, S., Urooj, A., Tan, N., Stib, M. T., Patel, B. N., & Banerjee, I. (2024) · medRxiv · doi:10.1101/2024.02.01.24302145 · Cited by 8
Goubard, C., & Demiris, Y. (2024) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra57147.2024.10611445 · Cited by 6
EDA-Aware RTL Generation with Large Language Models
Islam, M. A., Sami, H., Gaillardon, P., & Tenace, V. (2025) · Design, Automation, and Test in Europe · doi:10.23919/date64628.2025.10992789 · Cited by 6
LLM-Guided Reinforcement Learning for Interactive Environments
Yang, F., Liu, J., & Li, K. (2025) · Mathematics · doi:10.3390/math13121932 · Cited by 9
Large Language Models as SocioTechnical Systems
Dhole, K. (2023) · doi:10.18653/v1/2023.bigpicture-1.6 · Cited by 7
Developmental Scaffolding with Large Language Models
Celik, B., Ahmetoğlu, A., Uğur, E., & Öztop, E. (2023) · IEEE International Conference on Development and Learning (ICDL) · doi:10.1109/icdl55364.2023.10364374 · Cited by 7
Is GPT Powerful Enough to Analyze the Emotions of Memes?
Wang, J. Z., Luo, J., Yang, G., Hong, A., & Luo, F. (2023) · International Conference on Machine Learning and Applications · doi:10.1109/icmla58977.2023.00202 · Cited by 6
Bootstrapping Cognitive Agents with a Large Language Model
Zhu, F., & Simmons, R. G. (2024) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v38i1.27822 · Cited by 11
Strategies to Leverage Foundational Model Knowledge in Object Affordance Grounding
Rai, A., Buettner, K., & Kovashka, A. (2024) · IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw63382.2024.00178 · Cited by 9
Assessing Model Quality Using Large Language Models
Gutschmidt, A., & Nast, B. (2024) · Lecture notes in business information processing · doi:10.1007/978-3-031-77908-4_7 · Cited by 6
Language-Guided Pattern Formation for Swarm Robotics with Multi-Agent Reinforcement Learning
Liu, H.-S., Kuroki, S., Kozuno, T., Sun, W.-F., & Lee, C. (2024) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros58592.2024.10801665 · Cited by 7
Task Planning for Long-Horizon Cooking Tasks Based on Large Language Models
Shin, J., Han, J., Kim, W., Oh, Y., & Kim, E. (2024) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros58592.2024.10801687 · Cited by 6
Combining Ontological Knowledge and Large Language Model for User-Friendly Service Robots
Nakajima, H., & Miura, J. (2024) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros58592.2024.10802273 · Cited by 7
A Data Source for Reasoning Embodied Agents
Lanchantin, J., Sukhbaatar, S., Synnaeve, G., Sun, Y., Srinet, K., & Szlam, A. D. (2023) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v37i7.26017 · Cited by 6
A Zero-Shot Language Agent for Computer Control with Structured Reflection
Li, T., Li, G., Deng, Z., Wang, B. K., & Li, Y. (2023) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2023.findings-emnlp.753 · Cited by 33
Kang, J. H., Dhanaraj, N., Wadaskar, S., & Gupta, S. K. (2024) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra57147.2024.10610875 · Cited by 7
Planetarium: A Rigorous Benchmark for Translating Text to Structured Planning Languages
Zuo, M., Velez, F. P., Li, X., Littman, M., & Bach, S. (2025) · Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers) · doi:10.18653/v1/2025.naacl-long.560 · Cited by 5
Liu, J., Hao, W., Cheng, K., & Jin, D. (2025) · Engineering Applications of Artificial Intelligence · doi:10.1016/j.engappai.2025.110319 · Cited by 5
A Smart Interactive Camera Robot Based on Large Language Models
Bao, Z., Zhu, G., Ding, W., Guan, Y., Bai, W., & Gan, Z. (2023) · IEEE International Conference on Robotics and Biomimetics (ROBIO) · doi:10.1109/robio58561.2023.10354952 · Cited by 5
LGMCTS: Language-Guided Monte-Carlo Tree Search for Executable Semantic Object Rearrangement
Chang, H., Gao, K., Boyalakuntla, K., Lee, A. J., Huang, B., Yu, J., & Boularias, A. (2024) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros58592.2024.10802562 · Cited by 16
Visual Environment-Interactive Planning for Embodied Complex-Question Answering
Lan, N., Ou, B., Xie, X., & Shi, G. (2025) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2025.3538860 · Cited by 6
Efficient WSI classification with sequence reduction and transformers pretrained on text
Pisula, J. I., & Bożek, K. (2025) · Scientific Reports · doi:10.1038/s41598-025-88139-5 · Cited by 10
Episodic Memory in AI Agents Poses Risks that Should be Studied and Mitigated
DeChant, C. (2025) · 2025 IEEE Conference on Secure and Trustworthy Machine Learning (SaTML) · doi:10.1109/satml64287.2025.00024 · Cited by 5
Robot Planning based on Behavior Tree and Knowledge Graph
Zhou, Y., Zhu, S., Song, W. G., Gu, J., Ren, J., Xi, X., Jin, T., & Mu, Z. (2022) · IEEE International Conference on Robotics and Biomimetics (ROBIO) · doi:10.1109/robio55434.2022.10011944 · Cited by 5
Multimedia Generative Script Learning for Task Planning
Wang, Q., Li, M., Chan, H. P., Huang, L., Hockenmaier, J., Chowdhary, G. V., & Ji, H. (2023) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2023.findings-acl.63 · Cited by 9
Ground Manipulator Primitive Tasks to Executable Actions Using Large Language Models
Cao, Y., & Lee, C. S. G. (2024) · Proceedings of the AAAI Symposium Series · doi:10.1609/aaaiss.v2i1.27720 · Cited by 5
Buettner, K., Malakouti, S., Li, X. L., & Kovashka, A. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.01283 · Cited by 6
IEA-Plot: Conducting Wafer-Based Data Analytics Through Chat
Dupree, M., Yang, M. J., Zeng, Y. J., & Wang, L.-C. (2023) · IEEE International Test Conference (ITC) · doi:10.1109/itc51656.2023.00028 · Cited by 5
Challenges and Opportunities in Neuro-Symbolic Composition of Foundation Models
Jha, S., Roy, A., Cobb, A. D., Berenbeim, A., & Bastian, N. D. (2023) · IEEE Military Communications Conference (MILCOM) · doi:10.1109/milcom58377.2023.10356344 · Cited by 6
Language as a cognitive and social tool at the time of large language models
Borghi, A. M., Livio, C. D., Gervasi, A. M., Mannella, F., Nolfi, S., & Tummolini, L. (2024) · Journal of Cultural Cognitive Science · doi:10.1007/s41809-024-00152-8 · Cited by 9
A Survey on Collision Avoidance Algorithms for Multi-robot Systems
Oh, D., Lim, Y., Jung, H., Huh, J., Lee, J., Choi, C., Kim, H. J., & Park, J. (2025) · International Journal of Control Automation and Systems · doi:10.1007/s12555-024-1104-9 · Cited by 4
Talk Through It: End User Directed Manipulation Learning
Winge, C., Imdieke, A., Aldeeb, B., Kang, D., & Desingh, K. (2024) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2024.3433309 · Cited by 4
Embodied Intelligent Driving: Key Technologies and Applications
Wang, Y., Chen, S., Li, Z., Shen, T., & Wang, K. (2024) · IEEE International Conference on Digital Twins and Parallel Intelligence (DTPI) · doi:10.1109/dtpi61353.2024.10778821 · Cited by 4
Wicke, P. (2023) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2023.findings-acl.302 · Cited by 6
Enabling action crossmodality for a pretrained large language model
Caesar, A., Özdemir, O., Weber, C., & Wermter, S. (2024) · Natural Language Processing Journal · doi:10.1016/j.nlp.2024.100072 · Cited by 5
Adapting LLMs to Downstream Applications
Kucharavy, A. (2024) · Springer eBooks · doi:10.1007/978-3-031-54827-7_2 · Cited by 5
LLM-SAP: Large Language Models Situational Awareness-Based Planning
Wang, L., & Zhong, H. (2024) · IEEE International Conference on Multimedia and Expo workshops · doi:10.1109/icmew63481.2024.10645429 · Cited by 7
Task Planning for Robot Manipulator Using Natural Language Task Input with Large Language Models
Kawabe, T., Nishi, T., Liu, Z., & Fujiwara, T. (2024) · IEEE International Conference on Automation Science and Engineering (CASE) · doi:10.1109/case59546.2024.10711671 · Cited by 4
Suzuki, K., & Ogata, T. (2024) · IEEE/RJS International Conference on Intelligent RObots and Systems · doi:10.1109/iros58592.2024.10802349 · Cited by 5
Make a Donut: Hierarchical EMD-Space Planning for Zero-Shot Deformable Manipulation With Tools
You, Y., Shen, B., Deng, C., Geng, H., Wei, S., Wang, H., & Guibas, L. J. (2025) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2025.3537899 · Cited by 5
Cheng, Z., Lin, Z., Yang, Y., Wei, Z., & Chen, S. (2025) · Visual Informatics · doi:10.1016/j.visinf.2025.100260 · Cited by 6
AUSPEX: An integrated open-source decision-making framework for UAVs in rescue missions
Döschl, B., Sommer, K., & Kiam, J. J. (2025) · Frontiers in Robotics and AI · doi:10.3389/frobt.2025.1583479 · Cited by 7
Chen, W., Xiao, C., Gao, G., Sun, F., Zhang, C., & Zhang, J. (2025) · IEEE Transactions on Systems Man and Cybernetics Systems · doi:10.1109/tsmc.2025.3611698 · Cited by 4
Generating Commonsense Counterfactuals for Stable Relation Extraction
Miao, X., Li, Y., & Qian, T. (2023) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2023.emnlp-main.344 · Cited by 4
A Picture is Worth a Thousand Words: Language Models Plan from Pixels
Liu, A., Logeswaran, L., Sohn, S., & Lee, H. (2023) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2023.emnlp-main.1025 · Cited by 7
de Wynter, A. (2024) · IEEE Transactions on Games · doi:10.1109/tg.2024.3497601 · Cited by 9
AgentSquare: Automatic LLM Agent Search in Modular Design Space
Shang, Y., Li, Y., Zhao, K., Ma, L., Liu, J., Xu, F., & Li, Y. (2024) · arXiv · doi:10.48550/arxiv.2410.06153 · Cited by 3
“Engineering-oriented open resources like LangChain * and BabyAGI † have provided convenient ways to build an LLM-centric agentic system, however, they still need human participation to organize different modules and can’t support the optimization of the designed agent.”
Large Language Model Reasoning Failures
Song, P., Han, P., & Goodman, N. (2026) · arXiv · doi:10.48550/arxiv.2602.06176 · Cited by 3
ReTA: Recursively Thinking Ahead to Improve the Strategic Reasoning of Large Language Models
Duan, J., Wang, S., Diffenderfer, J., Sun, L., Chen, T., Kailkhura, B., & Xu, K. (2024) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/2024.naacl-long.123 · Cited by 7
Enhancing Robot Learning Through Cognitive Reasoning Trajectory Optimization Under Unknown Dynamics
Dong, Q., Wu, T., Zeng, P., Zang, C., Wan, G., & Cui, S. (2025) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2025.3558648 · Cited by 3
Liu, L., Zhang, S., Jiang, Y., Guo, J., & Zhao, W. (2025) · IEEE Access · doi:10.1109/access.2025.3559076 · Cited by 3
Autonomous embodied navigation task generation from natural language dialogues
Xu, H., Li, Y., Ma, L., Li, C., Dong, Y., Yuan, X., & Liu, H. (2025) · Science China Information Sciences · doi:10.1007/s11432-024-4404-4 · Cited by 4
Multitask Multimodal Prompted Training for Interactive Embodied Task Completion
Πανταζόπουλος, Γ., Nikandrou, M., Parekh, A., Hemanthage, B., Eshghi, A., Konstas, I., Rieser, V., Lemon, O., & Suglia, A. (2023) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2023.emnlp-main.50 · Cited by 7
Look Further Ahead: Testing the Limits of GPT-4 in Path Planning
Aghzal, M., Plaku, E., & Yao, Z. (2024) · IEEE International Conference on Automation Science and Engineering (CASE) · doi:10.1109/case59546.2024.10711396 · Cited by 6
Gunji, K., Ohno, K., Kurita, S., Sakurada, K., Neto, R. P. B., Kojima, S., Okada, Y., Konyo, M., & Tadokoro, S. (2024) · IEEE Access · doi:10.1109/access.2024.3514473 · Cited by 3
Kinematic Control for the Motion Generation of Robot Manipulators Using MoMask LLM
Obludzyner, A., Zaldivar, F., & Ramos, O. E. (2024) · IEEE International Conference on Electronics, Electrical Engineering and Computing (INTERCON) · doi:10.1109/intercon63140.2024.10833232 · Cited by 3
Embodied AI for Smart Robotic Cells in Manufacturing Applications
Gupta, S. K. (2025) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v39i27.35088 · Cited by 3
Santra, P., Ghosh, M., Ganguly, D., Basuchowdhuri, P., & Naskar, S. K. (2025) · Wiley Interdisciplinary Reviews Data Mining and Knowledge Discovery · doi:10.1002/widm.70021 · Cited by 5
Xiong, G., Liu, S., Yan, Y., Li, Q., & Li, H. (2025) · IEEE Access · doi:10.1109/access.2025.3580641 · Cited by 6
State-Based Dynamic Graph with Breadth First Progression For Autonomous Robots
Chugh, T., Tyagi, K., Srinivasan, P., & Challagundla, J. (2024) · IEEE Annual Computing and Communication Workshop and Conference (CCWC) · doi:10.1109/ccwc60891.2024.10427646 · Cited by 3
LLM-Assisted Analytics in Semiconductor Test (Invited)
Wang, L.-C. (2024) · ACM/IEEE Workshop on Machine Learning for CAD (MLCAD) · doi:10.1109/mlcad62225.2024.10740163 · Cited by 3
VIHE: Virtual In-Hand Eye Transformer for 3D Robotic Manipulation
Wang, W., Lei, Y., Jin, S., Hager, G. D., & Zhang, L. (2024) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros58592.2024.10802366 · Cited by 7
Minimizing learning experiences in embodied agents language and action learning
Milano, N., & Nolfi, S. (2025) · Neurocomputing · doi:10.1016/j.neucom.2025.129510 · Cited by 3
iCORPP: Interleaved commonsense reasoning and probabilistic planning on robots
Zhang, S., Khandelwal, P., & Stone, P. (2024) · Robotics and Autonomous Systems · doi:10.1016/j.robot.2023.104613 · Cited by 3
Martin, E., Hasegawa, S., Solis, J., Macq, B. M. M., Ronsse, R., Ricardez, G. A. G., Hafi, L. E., & Taniguchi, T. (2025) · IEEE/SICE International Symposium on System Integration (SII) · doi:10.1109/sii59315.2025.10871045 · Cited by 3
Instruction-Following Long-Horizon Manipulation by LLM-Empowered Symbolic Planner
Li, Z., Liu, J., Li, Z., Yu, M., Teng, T., Zhou, S., Li, M., Lam, T. L., & Chen, F. (2024) · IEEE International Conference on Robotics and Biomimetics (ROBIO) · doi:10.1109/robio64047.2024.10907585 · Cited by 2
Reward Generation via Large Vision-Language Model in Offline Reinforcement Learning
Lee, Y., Luu, T. M., Lee, D., & Yoo, C. D. (2025) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp49660.2025.10889042 · Cited by 2
A Language-First Approach for Procedure Planning
Liu, J., Li, S., Wang, Z., Li, M., & Ji, H. (2023) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2023.findings-acl.122 · Cited by 4
MetaFill: Text Infilling for Meta-Path Generation on Heterogeneous Information Networks
Liu, Z., Duan, K., Yang, J., Xu, H., Zhang, M., & Wang, S. (2022) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2022.emnlp-main.341 · Cited by 2
Household Navigation and Manipulation for Everyday Object Rearrangement Tasks
Iyer, S. R., Pal, A., Hu, J., Adeleye, A., Aggarwal, A., & Christensen, H. I. (2023) · IEEE International Conference on Robotic Computing (IRC) · doi:10.1109/irc59093.2023.00021 · Cited by 2
A Simple Baseline for Uncertainty-Aware Language-Oriented Task Planner for Embodied Agents
Ong, H., Yoon, Y., Choi, J.-W., & Jang, M.-S. (2024) · 2024 21st International Conference on Ubiquitous Robots (UR) · doi:10.1109/ur61395.2024.10597499 · Cited by 4
MANIP: A Modular Architecture for Integrating Interactive Perception for Robot Manipulation
Yu, J., Sadjadpour, T., O'Neill, A., Khfifi, M., Chen, L. Y., Cheng, R., Irshad, M. Z., Balakrishna, A., Kollar, T., & Goldberg, K. (2024) · IEEE/RJS International Conference on Intelligent RObots and Systems · doi:10.1109/iros58592.2024.10801746 · Cited by 8
Liang, J., Chang, Y., Wang, Q., Wang, Y., & Yi, X. (2024) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros58592.2024.10801472 · Cited by 4
A Survey on Large Language Models for Motion Generation from Text
Obludzyner, A., Zaldivar, F., & Ramos, O. E. (2024) · IEEE International Conference on Electronics, Electrical Engineering and Computing (INTERCON) · doi:10.1109/intercon63140.2024.10833491 · Cited by 2
Language Reasoning in Vision-Language-Action Model for Robotic Grasping
Fan, L., Chen, K., Xu, Z., Yuan, M., Huang, P., & Huang, W. (2024) · China Automation Congress (CAC) · doi:10.1109/cac63892.2024.10865585 · Cited by 2
Self-citationSmart Energy Management: From Conventional Optimization to Generative AI Techniques
Mongaillard, T., Lasaulce, S., & Varma, V. S. (2025) · Lecture notes in intelligent transportation and infrastructure · doi:10.1007/978-3-031-72959-1_15 · Cited by 2
3C Assembly Methods and Systems Based on Large Language Models
Xin, D., Jin, L., Mao, D., Li, F., Wang, C., Song, R., & Li, Y. (2024) · IEEE International Conference on Robotics and Biomimetics (ROBIO) · doi:10.1109/robio64047.2024.10907465 · Cited by 2
Reimagining the Data Landscape: A Multi-Agent Paradigm for Data Interfacing
Renney, H., Nethercott, M., Williams, O., Evetts, J., & Lang, J. (2025) · International Conference on Data Science and Machine Learning Applications (CDMA) · doi:10.1109/cdma61895.2025.00025 · Cited by 2
A Biologically Inspired Program-Level Imitation Approach for Robots
Aliasghari, P., Ghafurian, M., Nehaniv, C. L., & Dautenhahn, K. (2025) · IEEE Access · doi:10.1109/access.2025.3570922 · Cited by 2
Generative AI in Embodied Systems: System-Level Analysis of Performance, Efficiency and Scalability
Wan, Z., Qian, J., Du, Y., Jabbour, J., Du, Y., Zhao, Y. K., Raychowdhury, A., Krishna, T., & Reddi, V. J. (2025) · IEEE International Symposium on Performance Analysis of Systems and Software (ISPASS) · doi:10.1109/ispass64960.2025.00013 · Cited by 6
tagE: Enabling an Embodied Agent to Understand Human Instructions
Sarkar, C., Mitra, A., Pramanick, P., & Nayak, T. K. (2023) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2023.findings-emnlp.593 · Cited by 2
ExpNote: Black-box Large Language Models are better Task Solvers with Experience Notebook
Sun, W., Yu, X., He, S., Zhao, J., & Liu, K. (2023) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2023.findings-emnlp.1034 · Cited by 4
Generation of Robot Manipulation Plans Using Generative Large Language Models
Töberg, J.-P., & Cimiano, P. (2023) · IEEE International Conference on Robotic Computing (IRC) · doi:10.1109/irc59093.2023.00039 · Cited by 4
Iterative improvements from feedback for language modelsPreprint
Li, Y. (2023) · doi:10.14293/pr2199.000220.v1 · Cited by 3
Cosmos-Reason1: From Physical Common Sense To Embodied Reasoning
NVIDIA, :, Azzolini, A., Bai, J., Brandon, H., Cao, J., Chattopadhyay, P., Chen, H., Chu, J., Cui, Y., et al. (2025) · arXiv · doi:10.48550/arxiv.2503.15558 · Cited by 187
Preble: Efficient Distributed Prompt Scheduling for LLM Serving
Srivatsa, V., He, Z., Abhyankar, R., Li, D., & Zhang, Y. (2024) · arXiv · doi:10.48550/arxiv.2407.00023 · Cited by 1
“This section presents our study results of five popular new LLM use cases: tool (or API, agent) use [44], interacting with virtual environments as an embodied agent [13, 15], software program generation [32], answering questions about videos [56], and answer questions about long documents [22].”
BackgroundWang, Y., Huang, Z., Dong, S., Chu, H., Yin, X., & Gao, B. (2024) · International Conference on Computer and Automation Engineering · doi:10.1109/iccae59995.2024.10569281 · Cited by 3
Towards Coarse-grained Visual Language Navigation Task Planning Enhanced by Event Knowledge Graph
Zhao, K., Song, Y., Zhao, H., Liu, H., Li, T., & Li, Z. (2024) · ACM International Conference on Information and Knowledge Management (CIKM) · doi:10.1145/3627673.3679711 · Cited by 1
UniMotion-DM: Uniform Text-Motion Generation and Editing via Diffusion Model
Lin, S., & Hou, W. (2024) · IEEE Access · doi:10.1109/access.2024.3518300 · Cited by 1
Lu, J., Song, H., & Zhang, M. (2024) · International Conference on Computer Science and Management Technology (ICCSMT) · doi:10.1145/3708036.3708154 · Cited by 1
Small Language Model Agent for the Operations of Continuously Updating ICT Systems
Fukuda, N., Nozue, H., & Oishi, H. (2025) · IEEE Access · doi:10.1109/access.2025.3544637 · Cited by 3
Liu, Y., Zhao, Y., Guo, W., Sheng, X., & Ding, H. (2024) · IEEE International Conference on Robotics and Biomimetics (ROBIO) · doi:10.1109/robio64047.2024.10907623 · Cited by 1
Training-Free Task Planning by Parsing Language Signals With Common Sense
Zhang, X., Wang, W., Chai, S., Wang, X., & Xiaopeng Fan, B. (2025) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp49660.2025.10888803 · Cited by 1
Sample Efficient Reinforcement Learning via Large Vision Language Model Distillation
Lee, D., Luu, T. M., Lee, Y.-H., & Yoo, C. D. (2025) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp49660.2025.10888998 · Cited by 1
GNN-Transformer Task Planning Enhanced with Semantic-Driven Data Augmentation
Jeong, S., Byeon, S., Kim, S., Kwon, H., & Oh, Y. (2025) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v39i14.33598 · Cited by 1
Multi-Agent LLM System for Automatic Meeting Information Processing
Varjú, Á., & Varga, P. (2025) · IEEE/IFIP Network Operations and Management Symposium (NOMS) · doi:10.1109/noms57970.2025.11073611 · Cited by 1
TerraCraft: City-scale generative procedural modeling with natural languages
Xi, Z., Yao, Z., Huang, J., Lu, Z.-Q., Yan, H., Mu, T., Wang, Z., & Xu, Q. (2025) · Graphical Models · doi:10.1016/j.gmod.2025.101285 · Cited by 1
Plug'n Play Task-Level Autonomy for Robotics Using POMDPs and Probabilistic Programs
Wertheim, O., Suissa, D. R., & Brafman, R. I. (2023) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2023.3334682 · Cited by 1
Metadata
- Authors
- Huang, W., Abbeel, P., Pathak, D., & Mordatch, I.
- Affiliations
- University of California, Berkeley (US)
- Carnegie Mellon University (US)
- Venue
- arXiv
- Publisher
- arXiv
- Published
- 2022
- ISSN
- 2331-8422 (ISSN-L)
- arXiv
- 2201.07207
- OpenAlex
- W4221152848
- Cited by
- 304
- References
- 0
- Citations by year
- 2026: 6 · 2025: 62 · 2024: 146 · 2023: 83 · 2022: 7
- Type
- posted content
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Funders
- National Science Foundation (2024594)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › AI-based Problem Solving and Planning
- Physical Sciences › Engineering › Control and Systems Engineering › Robot Manipulation and Learning
- Physical Sciences › Computer Science › Artificial Intelligence › Reinforcement Learning in Robotics
- Keywords
- large language models, task decomposition, embodied agents, semantic translation, human evaluation
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
