SayNav: Grounding Large Language Models for Dynamic Planning to Navigation in New Environments
Open accessAbstract
Semantic reasoning and dynamic planning capabilities are crucial for an autonomous agent to perform complex navigation tasks in unknown environments. It requires a large amount of common-sense knowledge, that humans possess, to succeed in these tasks. We present SayNav, a new approach that leverages human knowledge from Large Language Models (LLMs) for efficient generalization to complex navigation tasks in unknown large-scale environments. SayNav uses a novel grounding mechanism, that incrementally builds a 3D scene graph of the explored environment as inputs to LLMs, for generating feasible and contextually appropriate high-level plans for navigation. The LLM-generated plan is then executed by a pre-trained low-level planner, that treats each planned step as a short-distance point-goal navigation sub-task. SayNav dynamically generates step-by-step instructions during navigation and continuously refines future steps based on newly perceived information. We evaluate SayNav on multi-object navigation (MultiON) task, that requires the agent to utilize a massive amount of human knowledge to efficiently search multiple different objects in an unknown environment. We also introduce a benchmark dataset for MultiON task employing ProcTHOR framework that provides large photo-realistic indoor environments with variety of objects. SayNav achieves state-of-the-art results and even outperforms an oracle based baseline with strong ground-truth assumptions by more than 8% in terms of success rate, highlighting its ability to generate dynamic plans for successfully locating objects in large-scale new environments. The code, benchmark dataset and demonstration videos are accessible at https://www.sri.com/ics/computer-vision/saynav.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Simulators take on the roles of plan evaluation and critiques performed by the respective critics in the LLM-Modulo framework.
“Simulators takes on the roles of plan evaluation and critiques performed by the respective critics in the LLM-Modulo framework”
From LLMs Can't Plan, But Can Help Planning in LLM-Modulo Frameworks (2024) · cites this paper as Rajvanshi et al., 2023
Section 4.3 uses AI2-Thor as the simulator for SayNav’s experiments, but does not assign it a plan-critique role. More decisively, Section 5 (Limitations and Discussion; page unavailable) says that mechanisms to verify and provide feedback on LLM-generated plans are future work, rather than roles already performed by the simulator. The paper does not discuss the LLM-Modulo framework. (LLM, conf=0.84)
Checked by a reasoning model — may be wrong.
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474 · see citing works
Learning Phrase Representations using RNN Encoder–Decoder for Statistical Machine Translation
Cho, K., van Merrienboer, B., Gulcehre, C., Bahdanau, D., Bougares, F., Schwenk, H., & Bengio, Y. (2014) · Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.3115/v1/d14-1179 · Cited by 25,545 · see citing works
Language Models are Few-Shot Learners
Brown, T. B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., et al. (2020) · arXiv · doi:10.48550/arxiv.2005.14165 · Cited by 23,372 · see citing works
Training language models to follow instructions with human feedback
Ouyang, L., Wu, J., Jiang, X., Almeida, D., Wainwright, C. L., Mishkin, P., Zhang, C., Agarwal, S., Slama, K., Ray, A., et al. (2022) · arXiv · doi:10.48550/arxiv.2203.02155 · Cited by 4,208 · see citing works
A Survey of Large Language Models
Zhao, W. X., Zhou, K., Li, J., Tang, T., Dong, Z., Hou, Y., Zhang, B., Min, Y., Zhang, J., Liu, P., et al. (2026) · Frontiers of Computer Science · doi:10.1007/s11704-026-60308-3 · Cited by 2,093 · see citing works
A Reduction of Imitation Learning and Structured Prediction to No-Regret Online Learning
Ross, S., Gordon, G. J., & Bagnell, J. A. (2018) · Carnegie Mellon University · doi:10.1184/r1/6550949 · Cited by 1,941 · see citing works
Target-driven visual navigation in indoor scenes using deep reinforcement learning
Zhu, Y., Mottaghi, R., Kolve, E., Lim, J. J., Gupta, A., Fei-Fei, L., & Farhadi, A. (2017) · 2017 IEEE International Conference on Robotics and Automation (ICRA) · doi:10.1109/icra.2017.7989381 · Cited by 1,533 · see citing works
Scaling Instruction-Finetuned Language Models
Chung, H. W., Hou, L., Longpre, S., Zoph, B., Tay, Y., Fedus, W., Li, Y., Wang, X., Dehghani, M., Brahma, S., et al. (2022) · arXiv · doi:10.48550/arxiv.2210.11416 · Cited by 1,441 · see citing works
Habitat: A Platform for Embodied AI Research
Savva, M., Kadian, A., Maksymets, O., Zhao, Y., Wijmans, E., Jain, B., Straub, J., Liu, J., Koltun, V., Malik, J., et al. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00943 · Cited by 1,237 · see citing works
Do As I Can, Not As I Say: Grounding Language in Robotic Affordances
Ahn, M., Brohan, A., Brown, N., Chebotar, Y., Cortes, O., David, B., Finn, C., Fu, C., Gopalakrishnan, K., Hausman, K., et al. (2022) · arXiv · doi:10.48550/arxiv.2204.01691 · Cited by 624 · see citing works
On evaluation of embodied navigation agents
(2018) · doi:10.48550/arxiv.1807.06757 · Cited by 622 · see citing works
ProgPrompt: Generating Situated Robot Task Plans using Large Language Models
Singh, I., Blukis, V., Mousavian, A., Goyal, A., Xu, D., Tremblay, J., Fox, D., Thomason, J., & Garg, A. (2023) · 2023 IEEE International Conference on Robotics and Automation (ICRA) · doi:10.1109/icra48891.2023.10161317 · Cited by 613 · see citing works
PaLM-E: An Embodied Multimodal Language Model
Driess, D., Xia, F., Sajjadi, M. S. M., Lynch, C., Chowdhery, A., Ichter, B., Wahid, A., Tompson, J., Vuong, Q., Yu, T., et al. (2023) · arXiv · doi:10.48550/arxiv.2303.03378 · Cited by 591 · see citing works
AI2-THOR: An Interactive 3D Environment for Visual AI
Kolve, E., Mottaghi, R., Han, W., VanderBilt, E., Weihs, L., Herrasti, A., Deitke, M., Ehsani, K., Gordon, D., Zhu, Y., et al. (2017) · arXiv · doi:10.48550/arxiv.1712.05474 · Cited by 578 · see citing works
LLM-Planner: Few-Shot Grounded Planning for Embodied Agents with Large Language Models
Song, C. H., Sadler, B. M., Wu, J., Chao, W.-L., Washington, C., & Su, Y. (2023) · 2023 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.00280 · Cited by 435 · see citing works
Kimera: From SLAM to spatial perception with 3D dynamic scene graphs
Rosinol, A., Violette, A., Abate, M., Hughes, N., Chang, Y., Shi, J., Gupta, A., & Carlone, L. (2021) · Int. J. Robotics Res. · doi:10.1177/02783649211056674 · Cited by 418 · see citing works
3D Scene Graph: A Structure for Unified Semantics, 3D Space, and Camera
Armeni, I., He, Z.-Y., Zamir, A., Gwak, J., Malik, J., Fischer, M., & Savarese, S. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00576 · Cited by 350 · see citing works
Object Goal Navigation using Goal-Oriented Semantic Exploration
Chaplot, D. S., Gandhi, D., Gupta, A., & Salakhutdinov, R. (2020) · arXiv · doi:10.48550/arxiv.2007.00643 · Cited by 321 · see citing works
Inner Monologue: Embodied Reasoning through Planning with Language Models
Huang, W., Xia, F., Xiao, T., Chan, H., Liang, J., Florence, P., Zeng, A., Tompson, J., Mordatch, I., Chebotar, Y., et al. (2022) · arXiv · doi:10.48550/arxiv.2207.05608 · Cited by 287 · see citing works
DD-PPO: Learning Near-Perfect PointGoal Navigators from 2.5 Billion FramesPreprint
Wijmans, E., Kadian, A., Morcos, A. S., Lee, S., Essa, I. A., Parikh, D., Savva, M., & Batra, D. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1911.00357 · Cited by 253 · see citing works
Learning 3D Semantic Scene Graphs From 3D Indoor Reconstructions
Wald, J., Dhamo, H., Navab, N., & Tombari, F. (2020) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr42600.2020.00402 · Cited by 246 · see citing works
Automatic Evaluation of Information Ordering: Kendall's Tau
Lapata, M. (2006) · Computational Linguistics · doi:10.1162/coli.2006.32.4.471 · Cited by 194 · see citing works
Peng, B., Li, C., He, P., Galley, M., & Gao, J. (2023) · arXiv · doi:10.48550/arxiv.2304.03277 · Cited by 183 · see citing works
Simple but Effective: CLIP Embeddings for Embodied AI
Khandelwal, A., Weihs, L., Mottaghi, R., & Kembhavi, A. (2022) · 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52688.2022.01441 · Cited by 172 · see citing works
SceneGraphFusion: Incremental 3D Scene Graph Prediction from RGB-D Sequences
Wu, S.-C., Wald, J., Tateno, K., Navab, N., & Tombari, F. (2021) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr46437.2021.00743 · Cited by 158 · see citing works
(2022) · doi:10.52202/068431-0433 · Cited by 155 · see citing works
Habitat 2.0: Training home assistants to rearrange their habitat
Szot, A., Clegg, A., Undersander, E., Wijmans, E., Zhao, Y., Turner, J., Maestre, N., Mukadam, M., Chaplot, D., Maksymets, O., et al. (2021) · arXiv · doi:10.48550/arxiv.2106.14405 · Cited by 143 · see citing works
L3MVN: Leveraging Large Language Models for Visual Target Navigation
Yu, B., Kasaei, H., & Cao, M. (2023) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros55552.2023.10342512 · Cited by 120 · see citing works
Kim, U.-H., Park, J.-M., Song, T.-J., & Kim, J.-H. (2019) · IEEE Transactions on Cybernetics · doi:10.1109/tcyb.2019.2931042 · Cited by 109 · see citing works
LLM+P: Empowering Large Language Models with Optimal Planning Proficiency
Liu, B., Jiang, Y., Zhang, X., Liu, Q., Zhang, S., Biswas, J., & Stone, P. (2023) · arXiv · doi:10.48550/arxiv.2304.11477 · Cited by 104 · see citing works
Habitat-Web: Learning Embodied Object-Search Strategies from Human Demonstrations at Scale
Ramrakhya, R., Undersander, E., Batra, D., & Das, A. (2022) · 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52688.2022.00511 · Cited by 90 · see citing works
PIRLNav: Pretraining with Imitation and RL Finetuning for OBJECTNAV
Ramrakhya, R., Batra, D., Wijmans, E., & Das, A. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.01716 · Cited by 71 · see citing works
Benchmarking Classic and Learned Navigation in Complex 3D EnvironmentsPreprint
Mishkin, D., Dosovitskiy, A., & Koltun, V. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1901.10915 · Cited by 57 · see citing works
Multion: Benchmarking semantic map memory using multi-object navigation
Wani, S., Patel, S., Jain, U., Chang, A. X., & Savva, M. (2020) · arXiv · doi:10.48550/arxiv.2012.03912 · Cited by 56 · see citing works
AllenAct: A Framework for Embodied AI ResearchPreprint
Weihs, L., Salvador, J., Kotar, K., Jain, U., Zeng, K.-H., Mottaghi, R., & Kembhavi, A. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2008.12760 · Cited by 52 · see citing works
ViNT: A Foundation Model for Visual Navigation
Shah, D., Sridhar, A., Dashora, N., Stachowicz, K., Black, K., Hirose, N., & Levine, S. (2023) · arXiv · doi:10.48550/arxiv.2306.14846 · Cited by 45 · see citing works
SayPlan: Grounding Large Language Models using 3D Scene Graphs for Scalable Robot Task Planning
Rana, K., Haviland, J., Garg, S., Abou-Chakra, J., Reid, I., & Suenderhauf, N. (2023) · arXiv · doi:10.48550/arxiv.2307.06135 · Cited by 29 · see citing works
Multi-Object Navigation Using Potential Target Position Policy Function
Zeng, H., Song, X., & Jiang, S. (2023) · IEEE Transactions on Image Processing · doi:10.1109/tip.2023.3263110 · Cited by 22 · see citing works
Teaching Agents how to Map: Spatial Reasoning for Multi-Object Navigation
Marza, P., Matignon, L., Simonin, O., & Wolf, C. (2022) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros47612.2022.9982216 · Cited by 18 · see citing works
Multi-Object Navigation with dynamically learned neural implicit representations
Marza, P., Matignon, L., Simonin, O., & Wolf, C. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.01010 · Cited by 17 · see citing works
Learning Active Camera for Multi-Object Navigation
Chen, P., Ji, D., Lin, K., Hu, W., Huang, W., Li, T., Tan, M., & Gan, C. (2022) · neural information processing systems · doi:10.52202/068431-2078 · Cited by 16 · see citing works
Sequence-Agnostic Multi-Object Navigation
Gireesh, N., Agrawal, A., Datta, A., Banerjee, S., Sridharan, M., Bhowmick, B., & Krishna, M. (2023) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra48891.2023.10160259 · Cited by 11 · see citing works
Flagged first, then by citations
1 of the 43 references couldn’t be matched to a known paper.
Research works printed but not linked: 2
- Deitke, M.; et al. 2022. ProcTHOR: Large-Scale Embodied AI Using Procedural Generation. NeurIPS, 35: 5982-5994.
- Wu, Y.; et al. 2018. Group normalization. In ECCV, 3-19.
Cited by
Sorted by citations
A Survey on Multimodal Large Language Models for Autonomous Driving
Cui, C., Ma, Y., Cao, X., Ye, W., Zhou, Y., Liang, K., Chen, J., Lu, J., Yang, Z., Liao, K.-D., et al. (2024) · 2024 IEEE/CVF Winter Conference on Applications of Computer Vision Workshops (WACVW) · doi:10.1109/wacvw60836.2024.00106 · Cited by 369 · see citing works
Hierarchical Open-Vocabulary 3D Scene Graphs for Language-Grounded Robot Navigation
Werby, A., Huang, C., Büchner, M., Valada, A., & Burgard, W. (2024) · Robotics: Science and Systems XX · doi:10.15607/rss.2024.xx.077 · Cited by 124 · see citing works
“SayNav [44] obtains LLM-generated plans from scene graphs and executes short-distance point-goal navigation sub-tasks.”
MethodLanguage-Grounded Dynamic Scene Graphs for Interactive Object Search With Mobile Manipulation
Honerkamp, D., Büchner, M., Despinoy, F., Welschehold, T., & Valada, A. (2024) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2024.3441495 · Cited by 69 · see citing works
Pallagani, V., Muppasani, B. C., Roy, K., Fabiano, F., Loreggia, A., Murugesan, K., Srivastava, B., Rossi, F., Lior Horesh, C., & Sheth, A. (2024) · Proceedings of the International Conference on Automated Planning and Scheduling · doi:10.1609/icaps.v34i1.31503 · Cited by 62 · see citing works
Robot learning in the era of foundation models: a survey
Xiao, X., Liu, J., Wang, Z., Zhou, Y., Qi, Y., Jiang, S., He, B., & Cheng, Q. (2025) · Neurocomputing · doi:10.1016/j.neucom.2025.129963 · Cited by 47 · see citing works
Jin, Y., Yang, R., Yi, Z., Shen, X., Peng, H., Liu, X., Qin, J., Li, J., Xie, J., Gao, P., et al. (2024) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros58592.2024.10802229 · Cited by 29 · see citing works
Large Language Models for Human-Like Autonomous Driving: A Survey
Li, Y., Katsumata, K., Javanmardi, E., & Tsukada, M. (2024) · IEEE International Conference on Intelligent Transportation Systems (ITSC) · doi:10.1109/itsc58415.2024.10919629 · Cited by 21 · see citing works
Liu, Y., Sun, Q., & Kapadia, D. R. (2025) · AI · doi:10.3390/ai6070158 · Cited by 21 · see citing works
UniGoal: Towards Universal Zero-shot Goal-oriented Navigation
Yin, H. H., Xu, X., Zhao, L., Wang, Z., Zhou, J., & Lu, J. (2025) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52734.2025.01775 · Cited by 21 · see citing works
Embodied AI: A Survey on the Evolution from Perceptive to Behavioral Intelligence
Yifan, C., Wei, M., Wang, X., Liu, Y., Wang, J., Song, H., Ma, L., Di, D., Sun, C., Liu, K., et al. (2025) · SmartBot · doi:10.1002/smb2.70003 · Cited by 27 · see citing works
Unlocking Robotic Autonomy: A Survey on the Applications of Foundation Models
Jang, D.-S., Cho, D.-H., Lee, W.-C., Ryu, S.-K., Jeong, B.-M., Hong, M., Jung, M., Kim, M., Lee, M., Lee, S., et al. (2024) · International Journal of Control Automation and Systems · doi:10.1007/s12555-024-0438-7 · Cited by 16 · see citing works
Open scene graphs for open-world object-goal navigation
Loo, J., Wu, Z., & Hsu, D. (2025) · The International Journal of Robotics Research · doi:10.1177/02783649251369549 · Cited by 13 · see citing works
LLMs Can't Plan, But Can Help Planning in LLM-Modulo Frameworks
Kambhampati, S., Valmeekam, K., Guan, L., Verma, M., Stechly, K., Bhambri, S., Saldyt, L., & Murthy, A. (2024) · arXiv · doi:10.48550/arxiv.2402.01817 · Cited by 11 · see citing works
“Analogous to the plan evaluation and critiques performed by the respective critics in the LLM-Modulo framework, environment simulators provide the required feedback.”
MethodOpenIN: Open-Vocabulary Instance-Oriented Navigation in Dynamic Domestic Environments
Tang, Y., Wang, M., Deng, Y., Zheng, Z., Deng, J., Zuo, S., & Yue, Y. (2025) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2025.3592071 · Cited by 11 · see citing works
Grounding Large Language Models for Robot Task Planning Using Closed‐Loop State Feedback
Bhat, V., Kaypak, A. U., Krishnamurthy, P., Karri, R., & Khorrami, F. (2025) · Advanced Robotics Research · doi:10.1002/adrr.202500072 · Cited by 14 · see citing works
“Where am I?” Scene Retrieval with Language
Chen, J., Baráth, D., Armeni, I., Pollefeys, M., & Blum, H. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-72913-3_12 · Cited by 9 · see citing works
Learning hierarchical scene graph and contrastive learning for object goal navigation
Luo, J. J., Zhang, J., Cai, B., Yu, Y., & Ke, A. (2025) · Knowledge-Based Systems · doi:10.1016/j.knosys.2025.113532 · Cited by 8 · see citing works
Autonomous Vehicle Maneuvering Using Vision–LLM Models for Marine Surface Vehicles
Kim, T., & Choi, W.-S. (2025) · Journal of Marine Science and Engineering · doi:10.3390/jmse13081553 · Cited by 9 · see citing works
Aim My Robot: Precision Local Navigation to Any Object
Meng, X., Yang, X., Jung, S., Ramos, F. T., Jujjavarapu, S. S., Paul, S. K., & Fox, D. (2025) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2025.3542323 · Cited by 6 · see citing works
Visual Environment-Interactive Planning for Embodied Complex-Question Answering
Lan, N., Ou, B., Xie, X., & Shi, G. (2025) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2025.3538860 · Cited by 5 · see citing works
Advancing Object-Goal Navigation through LLM-enhanced Object Affinities Transfer
Lin, M., Liu, S., Zhang, D. W., Chen, Y., Wang, Z., Li, H., & Zhao, D. (2025) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros60139.2025.11247308 · Cited by 5 · see citing works
Zhang, J., Dong, H., Yang, J., Liu, J., Huang, S., Li, K., Tang, X., Wei, X., & You, X. (2025) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra55743.2025.11128157 · Cited by 4 · see citing works
From abstraction to reality: DARPA's vision for robust sim‐to‐real autonomy
Noorani, E., Serlin, Z., Price, B., & Velasquez, A. (2025) · AI Magazine · doi:10.1002/aaai.70015 · Cited by 3 · see citing works
Self-citationMulti-modal scene graph inspired policy for visual navigation
He, Y. Q., Zhou, K., & Tian, T. L. (2024) · The Journal of Supercomputing · doi:10.1007/s11227-024-06541-8 · Cited by 3 · see citing works
Kinematic Control for the Motion Generation of Robot Manipulators Using MoMask LLM
Obludzyner, A., Zaldivar, F., & Ramos, O. E. (2024) · IEEE International Conference on Electronics, Electrical Engineering and Computing (INTERCON) · doi:10.1109/intercon63140.2024.10833232 · Cited by 3 · see citing works
A survey on the recent random walk-based methods for embedding graphs
Bozorgi, E., Alqaaidi, S. K., Shams, A., Arabnia, H. R., & Kochut, K. (2025) · The Journal of Supercomputing · doi:10.1007/s11227-025-07019-x · Cited by 3 · see citing works
Graph2Nav: 3D Object-Relation Graph Generation to Robot Navigation
Shan, T., Rajvanshi, A., Mithun, N. C., & Chiu, H.-P. (2025) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra55743.2025.11128782 · Cited by 3 · see citing works
Self-citationEnhancing Robustness in Language-Driven Robotics: A Modular Approach to Failure Reduction
Garrabé, É., Teixeira, P., Khoramshahi, M., & Doncieux, S. (2025) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros60139.2025.11246969 · Cited by 3 · see citing works
RP-SG: Relation Prediction in 3D Scene Graphs for Unobserved Objects Localization
Ying, Z., Yuan, X., Yang, B., Song, Y. H., Xu, Q., Zhou, F., & Sheng, W. (2023) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2023.3342666 · Cited by 3 · see citing works
Learning Bird’s Eye View scene graph and knowledge-inspired policy for embodied visual navigation
Luo, J. J., Zhang, J., Yang, J., Huang, S., & Cai, B. (2025) · Knowledge-Based Systems · doi:10.1016/j.knosys.2025.113959 · Cited by 2 · see citing works
Integrating large language models for intuitive robot navigation
Xue, Z., Elksnis, A., & Wang, N. (2025) · Frontiers Robotics AI · doi:10.3389/frobt.2025.1627937 · Cited by 5 · see citing works
MORE: Mobile Manipulation Rearrangement Through Grounded Language Reasoning
Mohammadi, M., Honerkamp, D., Büchner, M., Cassinelli, M., Welschehold, T., Despinoy, F., Gilitschenski, I., & Valada, A. (2025) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros60139.2025.11247596 · Cited by 2 · see citing works
Tang, Y., Wang, M., Deng, Y., Zheng, Z., Zhong, J., Liu, T., Zhao, C., & Yue, Y. (2025) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros60139.2025.11247413 · Cited by 2 · see citing works
MobileVLA-R1: Reinforcing Vision-Language-Action for Mobile Robots
Huang, T., Li, D., Yang, R., Zhang, Z., Yang, Z., & Tang, H. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-37196-6_19 · Cited by 4 · see citing works
A comprehensive review of recent advancements in vision-and-language navigation
Khan, J. T., Aafaq, N., Ali, Q., & Mohsin, M. (2026) · Discover Computing · doi:10.1007/s10791-026-09977-z · Cited by 3 · see citing works
Li, Z., Meng, X., He, X., Zhang, Y., & Yin, W. (2026) · Sensors · doi:10.3390/s26072022 · Cited by 4 · see citing works
A Survey on Large Language Models for Motion Generation from Text
Obludzyner, A., Zaldivar, F., & Ramos, O. E. (2024) · IEEE International Conference on Electronics, Electrical Engineering and Computing (INTERCON) · doi:10.1109/intercon63140.2024.10833491 · Cited by 2 · see citing works
The state of hybrid artificial intelligence for interstellar missions
Ellery, A. A. (2025) · Progress in Aerospace Sciences · doi:10.1016/j.paerosci.2025.101100 · Cited by 2 · see citing works
NavGPT: Leveraging LLM for Route Classification Integrating Pollution and Crime Analytics
Sehgal, M., Sehra, S. S., Mohanarajah, T., Josyula, S., & Sehra, S. K. (2025) · IEEE International Conference on Computing and Machine Intelligence (ICMI) · doi:10.1109/icmi65310.2025.11141304 · Cited by 1 · see citing works
DebateNav: Structured Multi-VLM Expert Debate for Robust Zero-Shot Object Navigation
Sun, H., Tan, W., Liu, L., Yan, Z., Lu, X., & Dai, H. (2025) · IEEE International Conference on High Performance Computing and Communications; IEEE International Conference on Smart City; IEEE International Conference on Data Science and Systems (HPCC/SmartCity/DSS) · doi:10.1109/hpcc67675.2025.00068 · Cited by 1 · see citing works
Benchmarking Long-Horizon Mobile Manipulation in Multi-Room Dynamic Environments
Zhang, J., & Ma, K. (2025) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros60139.2025.11245866 · Cited by 1 · see citing works
VLM-Nav: Mapless UAV navigation using monocular vision driven by vision-language models
Sarker, G. C., Azad, A., Rahman, S., & Hasan, M. M. (2026) · PLoS ONE · doi:10.1371/journal.pone.0345778 · Cited by 1 · see citing works
Li, Y., Qin, K., Chen, W., & Wu, H. (2026) · Electronics · doi:10.3390/electronics15102179 · Cited by 1 · see citing works
Autonomous navigation and visual navigation in robot mission execution
Wang, S., Wang, Y., & Sun, Z. (2025) · Image and Vision Computing · doi:10.1016/j.imavis.2025.105516 · Cited by 1 · see citing works
Time Is on My Side: Scene Graph Filtering for Dynamic Environment Perception in an LLM-Driven RobotPreprint
Colombani, S., Brini, L., Ognibene, D., & Boccignone, G. (2024) · Qeios · doi:10.32388/ixaddn · Cited by 1 · see citing works
Methods and Mechanics for Robot Navigation in Different Environments
Sherwin, C., Singh, C., & Kumar, P. (2025) · doi:10.1002/9781394385027.ch2
Scalable Task Planning via Large Language Models and Structured World Representations
Pérez‐Dattari, R., Li, Z., Babuška, R., Kober, J., & Santina, C. D. (2025) · Advanced Robotics Research · doi:10.1002/adrr.202500002
Multi-Object Grounding via Hierarchical Contrastive Siamese Transformers
Du, C., & Jin, K. (2025) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn64981.2025.11228633
CTSG: Integrating Context and Way Topology Into Scene Graph for Zero-shot Navigation
Ma, R., Xu, Y., Li, Y., Fang, Y., Yang, Z., Qi, J., Zhao, X., & Zhang, C. (2025) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros60139.2025.11246966
AVP Scene Graph: Hierarchical Visual Language Mapping and Navigation for Autonomous Valet Parking
Mu, X., Chen, F., Wang, R., Chen, S., Cai, J., Cai, J., Yang, M. Z., & Qin, T. (2025) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros60139.2025.11246107
Kordjamshidi, P., Moens, M.-F., & Pustejovsky, J. (2025) · Synthesis lectures on human language technologies · doi:10.1007/978-3-031-99594-1_6
Emergent Planning Capabilities in Embodied LLMs Through Memory-Constrained Symbolic Compression
Muglikar, A. S., Phunde, S., Sharma, R., Chaugule, I., & Waghmare, G. (2025) · IEEE International Conference on ICT in Business Industry & Government (ICTBIG) · doi:10.1109/ictbig68706.2025.11323801
Situationally-Aware Path Planning Exploiting 3D Scene Graphs
Ejaz, S., Giberna, M., Shaheer, M. S., Millan-Romera, J. A., Tourani, A., Kremer, P., Voos, H., & Sánchez-López, J. L. (2026) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2026.3656775
Pandora: A Large Language Model-Driven Self-Evolving Agent for On-Campus Task Execution
Zeng, W., Xie, X., Yan, X., Li, Z., Wang, Y.-F., Luo, G.-F., Wang, C., & Chen, L.-B. (2025) · 2025 IEEE Smart World Congress (SWC) · doi:10.1109/swc65939.2025.00200
Foundation Models Meet Spatial Representations: Advancing Zero-Shot Navigation for Embodied AI
Zhou, Z., Chen, W., Xia, H., & Tian, Z. (2025) · IEEE Smart World Congress (SWC) · doi:10.1109/swc65939.2025.00227
MENav: Multi-Expert Collaboration for Zero-Shot Visual Semantic Navigation
Chen, J., Duan, H., Liu, H. H., & Guo, D. (2025) · IEEE International Conference on Robotics and Biomimetics (ROBIO) · doi:10.1109/robio66223.2025.11376137
Task2Mission: language-driven task-to-mission mapping for adaptive water quality monitoring
Islam, S., Alam, T., & Quader, S. N. (2026) · International Journal of Intelligent Robotics and Applications · doi:10.1007/s41315-026-00539-6
LaViCoN: Language and Vision Cooperative Navigation for Assistive Mobile Robots
Yang, T. Y., Zhong, R., & Shi, W. (2026) · IEEE International Conference on Mobility, Operations, Services and Technologies (MOST) · doi:10.1109/most69733.2026.00016
Gao, P., Wang, P., Wang, F., Fujita, H., & Aljuaid, H. (2026) · Journal of King Saud University - Computer and Information Sciences · doi:10.1007/s44443-026-00549-1
GSR-Nav: Graph-based Semantic Reasoning for Zero-Shot Object Navigation
Zhang, P., Wu, X., & Wang, Y. (2025) · IEEE International Conference on Control Science and Systems Engineering (ICCSSE) · doi:10.1109/iccsse67426.2025.11552579
A review of open-vocabulary semantic mapping and navigation with foundation models for mobile robots
Hagiwara, Y., & Hasegawa, S. (2026) · Advanced Robotics · doi:10.1080/01691864.2026.2738311
Learning Category-Level Last-Meter Navigation From RGB Demonstrations of a Single-Instance
Lee, T.-H., Mahmudova, F., & Desingh, K. (2026) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2026.3707352
Smarter Robots, Fewer Queries: A Knowledge-Driven Approach to Reduce LLM Dependency
Mothiya, A., Rangu, G., & Nair, S. B. (2025) · Advances in Robotics - International Conference of The Robotics Society (AIR) · doi:10.1145/3787370.3787413
Interactive 3D scene graph alignment for zero-shot object navigation
He, Y., Zhou, K., & Tian, L. (2026) · Neural Networks · doi:10.1016/j.neunet.2026.109323
Interleaved LLM and Motion Planning for Scalable Multi-Object Collection in Large Scene Graphs
Yang, R., Zhou, Y., Zhang, F., & Hou, M. (2026) · International Conference on Ubiquitous Robots (UR) · doi:10.1109/ur69537.2026.11626721
Mobile manipulation of embodied robots: A survey on manipulators, methods, datasets and challengesPreprint
Chen, S., Xu, R., Zhang, K., Li, C., Yin, D., Pei, L., & Li, Q. (2026) · SSRN Electronic Journal · doi:10.2139/ssrn.7403472
SMaRTAban: a voice-controlled LLM agent for quadruped mobile robotics with integrated vision
Zelenay, E., Kocúr, M., Lukáč, M., Duchoň, F., & Marko, R. (2026) · International Journal of Intelligent Robotics and Applications · doi:10.1007/s41315-026-00593-0
Unordered Multi-Object Navigation via VLM-Expert and Reinforcement Learning Collaboration
Chen, J., Wang, P., Tang, L., & Guo, D. (2026) · IEEE International Conference on Cyber Technology in Automation, Control, and Intelligent Systems · doi:10.1109/cyber71431.2026.11695491
Wang, W., Lin, R., Hu, J., & Yu, C. (2026) · Sensors · doi:10.3390/s26196251
- Authors
- Rajvanshi, A., Sikka, K., Lin, X., Lee, B., Chiu, H.-P., & Velasquez, A.
- Affiliations
- SRI International (US)
- Defense Advanced Research Projects Agency (US)
- Venue
- arXiv
- Publisher
- arXiv
- Published
- 2023
- Volume
- 34
- Pages
- 464-474
- ISSN
- 2334-0835 (ISSN-L)
- arXiv
- 2309.04077
- OpenAlex
- W4399177289
- Cited by
- 67
- References
- 43
- Citations by year
- 2026: 17 · 2025: 38 · 2024: 11 · 2023: 1
- FWCI
- 19.48
- Type
- conference paper
- Language
- en
- Open access
- diamond
- doi.org
- Locations
- Proceedings of the International Conference on Automated Planning and Scheduling (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Funders
- Defense Advanced Research Projects Agency
- Advanced Research Projects Agency
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › AI-based Problem Solving and Planning
- Physical Sciences › Computer Science › Artificial Intelligence › Reinforcement Learning in Robotics
- Physical Sciences › Engineering › Control and Systems Engineering › Robot Manipulation and Learning
- Keywords
- large language models, dynamic planning, robot navigation, 3D scene graphs, grounding, unknown environments, human knowledge, autonomous agents, point-goal navigation, instruction generation, benchmark dataset, indoor environment
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
