The DOI matches its DataCite record; 61 of the 69 references it cites were found in a registry or an index.
SpatialVLM: Endowing Vision-Language Models with Spatial Reasoning Capabilities
Open accessAbstract
Understanding and reasoning about spatial relationships is a fundamental capability for Visual Question Answering (VQA) and robotics. While Vision Language Models (VLM) have demonstrated remarkable performance in certain VQA benchmarks, they still lack capabilities in 3D spatial reasoning, such as recognizing quantitative relationships of physical objects like distances or size differences. We hypothesize that VLMs' limited spatial reasoning capability is due to the lack of 3D spatial knowledge in training data and aim to solve this problem by training VLMs with Internet-scale spatial reasoning data. To this end, we present a system to facilitate this approach. We first develop an automatic 3D spatial VQA data generation framework that scales up to 2 billion VQA examples on 10 million real-world images. We then investigate various factors in the training recipe, including data quality, training pipeline, and VLM architecture. Our work features the first internet-scale 3D spatial reasoning dataset in metric space. By training a VLM on such data, we significantly enhance its ability on both qualitative and quantitative spatial VQA. Finally, we demonstrate that this VLM unlocks novel downstream applications in chain-of-thought spatial reasoning and robotics due to its quantitative estimation capability. Project website: https://spatial-vlm.github.io/
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Our results (Table 2, Fig. 5) show that our instruction-tuned model achieves much higher precision than baselines using in-context learning [17, 22].”
From RoboPoint: A Vision-Language Model for Spatial Affordance Prediction for Robotics (2024) · cites this paper as [22]
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“SpatialVLM [22] took a step forward to predict spatial relations in metric space, but we show that ROBOPOINT can achieve better performance on real-world spatial reasoning tasks by locating affordances as points.”
From RoboPoint: A Vision-Language Model for Spatial Affordance Prediction for Robotics (2024) · cites this paper as [22]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Microsoft COCO: Common Objects in Context
Lin, T.-Y., Maire, M., Belongie, S., Hays, J., Perona, P., Ramanan, D., Dollár, P., & Zitnick, C. L. (2014) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-10602-1_48 · Cited by 45,460
Chen, L.-C., Papandreou, G., Kokkinos, I., Murphy, K., & Yuille, A. L. (2018) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2017.2699184 · Cited by 22,392
Kirillov, A., Mintun, E., Ravi, N., Mao, H., Rolland, C., Gustafson, L., Xiao, T., Whitehead, S., Berg, A. C., Lo, W.-Y., et al. (2023) · 2023 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.00371 · Cited by 11,953
Visual Genome: Connecting Language and Vision Using Crowdsourced Dense Image Annotations
Krishna, R., Zhu, Y., Groth, O., Johnson, J., Hata, K., Kravitz, J., Chen, S., Kalantidis, Y., Li, L.-J., Shamma, D. A., et al. (2017) · International Journal of Computer Vision · doi:10.1007/s11263-016-0981-7 · Cited by 5,107
Chain-Of-Thought Prompting Elicits Reasoning in Large Language Models
Wei, J., Wang, X., Schuurmans, D., Bosma, M., Ichter, B., Xia, F., Chi, E., Le, Q. V., & Zhou, D. (2022) · Advances in Neural Information Processing Systems 35 · doi:10.52202/068431-1800 · Cited by 4,505
Self-citationSimultaneous localization and mapping: part I
Durrant-Whyte, H., & Bailey, T. (2006) · IEEE Robotics & Automation Magazine · doi:10.1109/mra.2006.1638022 · Cited by 4,199
Past, Present, and Future of Simultaneous Localization and Mapping: Toward the Robust-Perception Age
Cadena, C., Carlone, L., Carrillo, H., Latif, Y., Scaramuzza, D., Neira, J., Reid, I., & Leonard, J. J. (2016) · IEEE Transactions on Robotics · doi:10.1109/tro.2016.2624754 · Cited by 3,596
PaLM: Scaling Language Modeling with Pathways
Chowdhery, A., Narang, S., Devlin, J., Bosma, M., Mishra, G., Roberts, A., Barham, P., Chung, H. W., Sutton, C., Gehrmann, S., et al. (2022) · arXiv · doi:10.48550/arxiv.2204.02311 · Cited by 2,590
Liu, H., Li, C., Wu, Q., & Lee, Y. J. (2023) · Advances in Neural Information Processing Systems 36 · doi:10.52202/075280-1516 · Cited by 2,428
Making the V in VQA Matter: Elevating the Role of Image Understanding in Visual Question Answering
Goyal, Y., Khot, T., Summers-Stay, D., Batra, D., & Parikh, D. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.670 · Cited by 2,287
Research works printed but not linked: 8
- Rohan Anil, Andrew M Dai, Orhan Firat, Melvin Johnson, Dmitry Lepikhin, Alexandre Passos, Siamak Shakeri, Emanuel Taropa, Paige Bailey, Zhifeng Chen, et al. Palm 2 technical report. arXiv preprint arXiv:2305.10403, 2023. · arXiv
- Anonymous. Flexcap: Generating rich, localized, and flexible captions in images. In Submitted to The Twelfth International Conference on Learning Representations, 2023. URL https://openreview.net/forum?id=7Phicg0WAg. under review.
- Liangyu Chen, Bo Li, Sheng Shen, Jingkang Yang, Chunyuan Li, Kurt Keutzer, Trevor Darrell, and Ziwei Liu. Language models are visual reasoning coordinators. In ICLR 2023 Workshop on Mathematical and Empirical Understanding of Foundation Models, 2023.
- Martin Ester, Hans-Peter Kriegel, Jörg Sander, Xiaowei Xu, et al. A density-based algorithm for discovering clusters in large spatial databases with noise. In kdd, volume 96, pages 226-231, 1996.
- Kaiming He, Georgia Gkioxari, Piotr Dollár, and Ross Girshick. Mask r-cnn. In Proceedings of the IEEE international conference on computer vision, pages 2961-2969, 2017.
- Parsa Mahmoudieh, Deepak Pathak, and Trevor Darrell. Zero-shot reward specification via grounded natural language. In International Conference on Machine Learning, pages 14743-14752. PMLR, 2022.
- Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervision. In International conference on machine learning, pages 8748-8763. PMLR, 2021.
- Jesse Thomason, Mohit Shridhar, Yonatan Bisk, Chris Paxton, and Luke Zettlemoyer. Language grounding with 3d objects. In Conference on Robot Learning, pages 1691-1701. PMLR, 2022.
Cited by
Sorted by citations
Foundation models in robotics: Applications, challenges, and the future
Firoozi, R., Tucker, J., Tian, S., Majumdar, A., Sun, J., Liu, W., Zhu, Y., Song, S., Kapoor, A., Hausman, K., et al. (2025) · The International Journal of Robotics Research · doi:10.1177/02783649241281508 · Cited by 238
Self-citationMultimodal Large Language Models in Health Care: Applications, Challenges, and Future Outlook.
AlSaad, R., Abd-alrazaq, A., Boughorbel, S., Ahmed, A., Renault, M.-A., Damseh, R., & Sheikh, J. (2024) · Journal of medical Internet research · doi:10.2196/59505 · Cited by 349
“…has been made in the field of general-domain image-text M-LLMs through the development of models such as GLaMM [71], Qwen-VL [72], SpatialVLM [73], InternVL [74], Osprey [75], Vary [76], ShareGPT4V [77], OtterHD [78], LION [79], SPHINX [80], BLIVA [81], SVIT [82], LLaVA [46], and CoVLM [83].”
CoT-VLA: Visual Chain-of-Thought Reasoning for Vision-Language-Action Models
Zhao, Q., Lu, Y., Kim, M. J., Fu, Z., Zhang, Z., Wu, Y., Li, Z., Ma, Q., Han, S., Finn, C., et al. (2025) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52734.2025.00166 · Cited by 231
A survey on integration of large language models with intelligent robots
Kim, Y., Kim, D., Choi, J., Park, J., Oh, N., & Park, D. (2024) · Intelligent Service Robotics · doi:10.1007/s11370-024-00550-5 · Cited by 179
Causal Inference Meets Deep Learning: A Comprehensive Survey
Jiao, L., Wang, Y., Liu, X., Li, L., Liu, F., Ma, W., Guo, Y., Chen, P., Yang, S., & Hou, B. (2024) · Research · doi:10.34133/research.0467 · Cited by 141
UAVs meet LLMs: Overviews and perspectives towards agentic low-altitude mobility
Tian, Y., Lin, F., Li, Y., Zhang, T., Zhang, Q., Fu, X., Huang, J., Dai, X., Wang, Y., Tian, C., et al. (2025) · Information Fusion · doi:10.1016/j.inffus.2025.103158 · Cited by 139
ViewSpatial-Bench: Evaluating Multi-perspective Spatial Localization in Vision-Language Models
Li, D., Li, H.-X., Wang, Z., Yan, Y., Zhang, H., Chen, S., Hou, G., Jiang, S., Zhang, W., Shen, Y., et al. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-37098-3_6 · Cited by 125
Thinking in Space: How Multimodal Large Language Models See, Remember, and Recall Spaces
Yang, J., Yang, S., Gupta, A. W., Han, R., Fei-Fei, L., & Xie, S. (2025) · 2025 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52734.2025.00994 · Cited by 120
Real-world robot applications of foundation models: a review
Kawaharazuka, K., Matsushima, T., Gambardella, A., Guo, J., Paxton, C., & Zeng, A. (2024) · Advanced Robotics · doi:10.1080/01691864.2024.2408593 · Cited by 117
GPT-4V(ision) for Robotics: Multimodal Task Planning From Human Demonstration
Wake, N., Kanehira, A., Sasabuchi, K., Takamatsu, J., & Ikeuchi, K. (2023) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2024.3477090 · Cited by 147
SAT: Dynamic Spatial Aptitude Training for Multimodal Language ModelsPreprint
Ray, A., Duan, J., Brown, E., Tan, R., Bashkirova, D., Hendrix, R. C., Ehsani, K., Kembhavi, A., Plummer, B. A., Krishna, R., et al. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2412.07755 · Cited by 115
Xie, S., Kong, L., Dong, Y., Sima, C., Zhang, W., Chen, Q. A., Liu, Z., & Pan, L. (2025) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51701.2025.00621 · Cited by 105
ShapeLLM: Universal 3D Object Understanding for Embodied Interaction
Qi, Z., Dong, R., Zhang, S., Geng, H., Han, C., Ge, Z., Yi, L., & Ma, K. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-72775-7_13 · Cited by 102
Yang, R., Zhu, Z., Li, Y., Huang, J., Yan, S., Zhou, S., Liu, Z., Li, X., Li, S., Wang, W., et al. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-36969-7_11 · Cited by 98
MindCube: Spatial Mental Modeling from Limited ViewsPreprint
Wang, Q., Yin, B., Zhang, P., Zhang, J., Wang, K., Wang, Z., Zhang, J., Chandrasegaran, K., Liu, H., Krishna, R., et al. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2506.21458 · Cited by 93
Explore until Confident: Efficient Exploration for Embodied Question Answering
Ren, A. Z., Clark, J., Dixit, A., Itkina, M., Majumdar, A., & Sadigh, D. (2024) · Robotics: Science and Systems · doi:10.15607/rss.2024.xx.089 · Cited by 88
Self-citation3DSRBENCH: A Comprehensive 3D Spatial Reasoning Benchmark
Ma, W., Chen, H., Zhang, G., Chou, Y.-C., Chen, J., de Melo, C. M., & Yuille, A. (2025) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51701.2025.00651 · Cited by 85
LayoutVLM: Differentiable Optimization of 3D Layout via Vision-Language Models
Sun, F.-Y., Liu, W., Gu, S., Lim, D., Bhat, G., Tombari, F., Li, M., Haber, N., & Wu, J. (2025) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52734.2025.02744 · Cited by 80
A Survey of Optimization-Based Task and Motion Planning: From Classical to Learning Approaches
Zhao, Z., Cheng, S., Ding, Y. F., Zhou, Z., Zhang, S., Xu, D., & Zhao, Y. (2024) · IEEE/ASME Transactions on Mechatronics · doi:10.1109/tmech.2024.3452509 · Cited by 79
RoboSpatial: Teaching Spatial Understanding to 2D and 3D Vision-Language Models for Robotics
Song, C. H., Blukis, V., Tremblay, J., Tyree, S., Su, Y., & Birchfield, S. T. (2025) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52734.2025.01470 · Cited by 79
BadRobot: Jailbreaking Embodied LLM Agents in the Physical WorldPreprint
Zhang, H., Zhu, C., Wang, X., Zhou, Z., Yin, C., Li, M., Xue, L., Wang, Y., Hu, S., Liu, A., et al. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2407.20242 · Cited by 76
Scaling Proprioceptive-Visual Learning with Heterogeneous Pre-trained Transformers
Wang, L., Chen, X., Zhao, J., & He, K. (2024) · neural information processing systems · doi:10.52202/079017-3952 · Cited by 68
Zhang, W., Zhou, Z., Zeng, X., Liu, X., Fang, J., Gao, C., Cui, J., Li, Y., Chen, X., & Zhang, X.-P. S. (2025) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3746027.3758219 · Cited by 66
Urban attractiveness according to ChatGPT: Contrasting AI and human insights
Malekzadeh, M., Willberg, E. S., Torkko, J., & Toivonen, T. (2024) · Computers Environment and Urban Systems · doi:10.1016/j.compenvurbsys.2024.102243 · Cited by 63
Object detection with multimodal large vision-language models: An in-depth review
Sapkota, R., & Karkee, M. (2025) · Information Fusion · doi:10.1016/j.inffus.2025.103575 · Cited by 61
MFMDepth: MetaFormer-based monocular metric depth estimation for distance measurement in ports
Chen, X., Ma, F., Wu, Y., Han, B., Luo, L., & Biancardo, S. A. (2025) · Computers & Industrial Engineering · doi:10.1016/j.cie.2025.111325 · Cited by 60
Web2Code: A Large-scale Webpage-to-Code Dataset and Evaluation Framework for Multimodal LLMs
Yun, S., Lin, H., Thushara, R., Bhat, M., Wang, Y., Jiang, Z., Deng, M., Wang, J., Tao, T., Li, J., et al. (2024) · neural information processing systems · doi:10.52202/079017-3560 · Cited by 60
RePLan: Robotic Replanning with Perception and Language ModelsPreprint
Skreta, M., Zhou, Z., Yuan, J. L., Darvish, K., Aspuru‐Guzik, A., & Garg, A. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2401.04157 · Cited by 66
STI-Bench: Are MLLMs Ready for Precise Spatial-Temporal World Understanding?
Li, Y., Zhang, Y., Lin, T., Liu, X., Cai, W., Liu, Z., & Zhao, B. (2025) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51701.2025.00533 · Cited by 59
Guo, R., Wei, J., Sun, L., Yu, B., Chang, G., Liu, D., Zhang, S., Yao, Z., Xu, M., & Bu, L. (2024) · Computers in Biology and Medicine · doi:10.1016/j.compbiomed.2024.108709 · Cited by 58
An Empirical Analysis on Spatial Reasoning Capabilities of Large Multimodal Models
Shiri, F., Guo, X., Far, M. G., Yu, X., Haf, R., & Li, Y.-F. (2024) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2024.emnlp-main.1195 · Cited by 55
Mobility VLA: Multimodal Instruction Navigation with Long-Context VLMs and Topological GraphsPreprint
Chiang, H.-T. L., Xu, Z., Fu, Z., Jacob, M. G., Zhang, T., Lee, T.-W. E., Yu, W., Schenck, C., Rendleman, D., Shah, D., et al. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2407.07775 · Cited by 55
Self-citationMa, X., Smart, B., Bhalgat, Y. S., Chen, S., Li, X., Ding, J., Gu, J., Chen, D. Z., Peng, S., Bian, J.-W., et al. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2405.10255 · Cited by 54
Pan, M., Zhang, J., Wu, T., Zhao, Y., Gao, W., & Dong, H. (2025) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52734.2025.01618 · Cited by 54
Vision-as-Inverse-Graphics Agent via Interleaved Multimodal Reasoning
Yin, S., Ge, J., Wang, Z. Z., Wang, C., Li, X., Black, M. J., Darrell, T., Kanazawa, A., & Feng, H. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-37464-6_30 · Cited by 52
Liao, Y.-H., Mahmood, R., Fidler, S., & Acuna, D. (2024) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2024.emnlp-main.947 · Cited by 47
MM-Spatial: Exploring 3D Spatial Understanding in Multimodal LLMs
Daxberger, E., Wenzel, N., Griffiths, D., Gang, H., Lazarow, J., Kohavi, G., Kang, K., Eichner, M. L., Yang, Y., Dehghan, A., et al. (2025) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51701.2025.00694 · Cited by 47
Zhao, B., Wang, Z., Fang, J., Gao, C., Man, F., Cui, J., Wang, X. Y., Chen, X., Li, Y., & Zhu, W. (2025) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3746027.3755703 · Cited by 45
A Comprehensive Survey on Embodied Intelligence: Advancements, Challenges, and Future Perspectives
Sun, F., Chen, R., Ji, T., Luo, Y., Zhou, H., & Liu, H. (2024) · CAAI Artificial Intelligence Research · doi:10.26599/air.2024.9150042 · Cited by 45
Explainability and vision foundation models: A survey
Kazmierczak, R., Berthier, É., Frehse, G., & Franchi, G. (2025) · Information Fusion · doi:10.1016/j.inffus.2025.103184 · Cited by 45
BAT: Learning to Reason about Spatial Sounds with Large Language ModelsPreprint
Zheng, Z., Peng, P., Ma, Z., Chen, X., Choi, E., & Harwath, D. F. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2402.01591 · Cited by 44
Chen, Q. H., & Yin, X. (2025) · Advanced Engineering Informatics · doi:10.1016/j.aei.2025.103478 · Cited by 43
Training Strategies for Efficient Embodied ReasoningPreprint
Chen, W., Belkhale, S., Mirchandani, S., Mees, O., Driess, D., Pertsch, K., & Levine, S. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2505.08243 · Cited by 42
Self-citationCoA-VLA: Improving Vision-Language-Action Models via Visual-Textual Chain-of-Affordance
Li, J., Zhu, Y., Tang, Z., Wen, J., Zhu, M., Liu, X., Li, C., Cheng, R., Peng, Y., Peng, Y., et al. (2025) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51701.2025.00910 · Cited by 41
Scaffolding Coordinates to Promote Vision-Language Coordination in Large Multi-Modal ModelsPreprint
Lei, X., Yang, Z., Chen, X., Li, P., & Liu, Y. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2402.12058 · Cited by 41
GEOBench-VLM: Benchmarking Vision-Language Models for Geospatial Tasks
Danish, M. S., Munir, M. A., Shah, S. R. A., Kuckreja, K., Khan, F. S., Fraccaro, P., Lacoste, A., & Khan, S. A. (2025) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51701.2025.00670 · Cited by 41
A Real-to-Sim-to-Real Approach to Robotic Manipulation with VLM-Generated Iterative Keypoint Rewards
Patel, S., Yin, X., Huang, W., Garg, S., Nayyeri, H., Fei-Fei, L., Lazebnik, S., & Li, Y. (2025) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra55743.2025.11127585 · Cited by 39
Wu, D., Zheng, P., Zhao, Q., Zhang, S., Qi, J., Hu, J., Zhu, G., & Wang, L. (2025) · Robotics and Computer-Integrated Manufacturing · doi:10.1016/j.rcim.2025.103064 · Cited by 38
What foundation models can bring for robot learning in manipulation: A survey
Li, D., Jin, Y., Sun, Y.-H., Yong, A., Yu, H., Shi, J., Hao, X., Peng, H., Liu, H., Li, X., et al. (2025) · The International Journal of Robotics Research · doi:10.1177/02783649251390579 · Cited by 36
Merlin: Empowering Multimodal LLMs with Foresight Minds
Yu, E., Zhao, L., Wei, Y., Yang, J., Wu, D., Kong, L., Wei, H., Wang, T., Ge, Z., Zhang, X., et al. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-73235-5_24 · Cited by 36
Semantically Safe Robot Manipulation: From Semantic Scene Understanding to Motion Safeguards
Brunke, L., Zhang, Y., Römer, R., Naimer, J., Staykov, N., Zhou, S., & Schoellig, A. P. (2025) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2025.3553046 · Cited by 34
Tian, K., Mao, J., Zhang, Y., Jiang, J., Zhou, Y., & Tu, Z. (2025) · IEEE International Conference on Computer Vision Workshops · doi:10.1109/iccvw69036.2025.00480 · Cited by 34
TopV-Nav: Unlocking the Top-View Spatial Reasoning Potential of MLLM for Zero-Shot Object Navigation
Zhong, L., Gao, C., Ding, Z., Liao, Y., Ma, H., Zhang, S., Zhou, X., & Liu, S. (2026) · International Journal of Computer Vision · doi:10.1007/s11263-026-02996-9 · Cited by 32
Perception Tokens Enhance Visual Reasoning in Multimodal Language Models
Bigverdi, M., Luo, Z., Hsieh, C.-Y., Shen, E., Chen, D., Shapiro, L. G., & Krishna, R. (2025) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52734.2025.00363 · Cited by 32
Perspective-Aware Reasoning in Vision-Language Models via Mental Imagery Simulation
Lee, P. Y., Je, J., Park, C., Guibas, L. J., Uy, M. A., & Sung, M. (2025) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51701.2025.00863 · Cited by 31
Self-citationBenchmark Evaluations, Applications, and Challenges of Large Vision Language Models: A SurveyPreprint
Li, Z., Wu, X., Du, H., Nghiem, H. T., & Shi, G. (2025) · Qeios · doi:10.32388/gxr68q · Cited by 33
DetToolChain: A New Prompting Paradigm to Unleash Detection Ability of MLLM
Wu, Y., Wang, Y., Tang, S., Wu, W., He, T., Ouyang, W., Torr, P. H. S., & Wu, J. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-73411-3_10 · Cited by 30
Evaluating and enhancing spatial cognition abilities of large language models
Yang, A., Fu, C., Jia, Q., Dong, W., Ma, M., Chen, H., Yang, F., & Wu, H. (2025) · International Journal of Geographical Information Systems · doi:10.1080/13658816.2025.2490701 · Cited by 30
Agent3D-Zero: An Agent for Zero-Shot 3D Understanding
Zhang, S., Huang, D., Deng, J., Tang, S., Ouyang, W., He, T., & Zhang, Y. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-72655-2_11 · Cited by 27
Coarse Correspondences Boost Spatial-Temporal Reasoning in Multimodal Language Model
Liu, B., Dong, Y., Wang, Y., Ma, Z., Tang, Y., Tang, L., Rao, Y., Ma, W.-C., & Krishna, R. (2025) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52734.2025.00358 · Cited by 26
Is CLIP Ideal? No. Can We Fix It? Yes!
Kang, R., Song, Y., Gkioxari, G., & Perona, P. (2025) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51701.2025.02083 · Cited by 26
Social-LLaVA: Enhancing Social Robot Navigation through Human-Language Reasoning
Payandeh, A., Song, D., Nazeri, M., Liang, J., Mukherjee, P., Raj, A. H., Kong, Y., Manocha, D., & Xiao, X. (2025) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros60139.2025.11247618 · Cited by 26
Why Reasoning Matters? A Survey of Advancements in Multimodal Reasoning (v1)Preprint
Bi, J., Liang, S., Zhou, X., Liu, P., Guo, J., Tang, Y., Song, L., Huang, C., Vosoughi, A., Sun, G., et al. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2504.03151 · Cited by 26
Meteor: Mamba-based Traversal of Rationale for Large Language and Vision Models
Lee, B.-K., Kim, C. W., Park, B., & Ro, Y. M. (2024) · neural information processing systems · doi:10.52202/079017-1274 · Cited by 25
Eureka: Evaluating and Understanding Large Foundation ModelsPreprint
Balachandran, V., Chen, J., Joshi, N., Nushi, B., Palangi, H., Salinas, E., Vineet, V., Woffinden-Luey, J., & Yousefi, S. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2409.10566 · Cited by 25
UAD: Unsupervised Affordance Distillation for Generalization in Robotic Manipulation
Tang, Y., Huang, W., Wang, Y., Li, C., Yuan, R., Zhang, R., Wu, J., & Fei-Fei, L. (2025) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra55743.2025.11128868 · Cited by 24
Robot Failure Recovery Using Vision-Language Models With Optimized Prompts
Chen, H., Yao, Y., Liu, R., Liu, C., & Ichnowski, J. (2025) · Proceedings of the ... American Control Conference/Proceedings of the American Control Conference · doi:10.23919/acc63710.2025.11107751 · Cited by 23
Ma, C., Lü, K., Cheng, T.-Y., Trigoni, N., & Markham, A. (2024) · neural information processing systems · doi:10.52202/079017-2198 · Cited by 23
KiVA: Kid-inspired Visual Analogies for Testing Large Multimodal ModelsPreprint
Yiu, E., Qraitem, M., Majhi, A. N., Wong, C.-J., Bai, Y., Ginosar, S., Alison Gopnik, C., & Saenko, K. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2407.17773 · Cited by 24
Affordance-Guided Reinforcement Learning via Visual Prompting
Lee, O. Y., Xie, A., Fang, K., Pertsch, K., & Finn, C. (2024) · IEEE/RJS International Conference on Intelligent RObots and Systems · doi:10.1109/iros60139.2025.11246854 · Cited by 31
Scaling and Beyond: Advancing Spatial Reasoning in MLLMs Requires New RecipesPreprint
Zhang, H., Li, C., Wu, W., Mao, S., Zhang, Y., Tian, H., Vulić, I., Zhang, Z., Wang, L., Tan, T., et al. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2504.15037 · Cited by 22
Zhu, H., Chang, J., An, X., & Li, S. (2025) · Cities · doi:10.1016/j.cities.2025.106183 · Cited by 22
Towards Spatial Trace with Reasoning in Vision-Language Models for Robotics
Zhou, E., Li, Y., An, J., An, J., Rong, S., Liu, M., Han, Y., Ji, Y., Tan, H., He, J., et al. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-36969-7_21 · Cited by 20
InSpire: Vision-Language-Action Models with Intrinsic Spatial Reasoning
Zhang, J., Wu, S., Luo, X., Wu, H., Xie, J., Gao, L., Shen, H. T., & Song, J. (2026) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra57385.2026.11695884 · Cited by 20
How Far Are We from Intelligent Visual Deductive Reasoning?Preprint
Zhang, Y., Bai, H., Zhang, R., Gu, J., Zhai, S., Susskind, J., & Jaitly, N. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2403.04732 · Cited by 20
Mei, Y., Nie, T., Sun, J., & Tian, Y. (2025) · IEEE International Conference on Intelligent Transportation Systems (ITSC) · doi:10.1109/itsc60802.2025.11423790 · Cited by 19
Zhang, L., Wang, Y., Gu, H., Hamidizadeh, A., Zhang, Z., Liu, Y., Wang, Y., Bravo, D. G. A., Dong, J., Zhou, S., et al. (2025) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros60139.2025.11246658 · Cited by 19
Liu, J., Yang, X., Li, W., & Wang, P. (2024) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2024.emnlp-main.864 · Cited by 18
TroL: Traversal of Layers for Large Language and Vision Models
Lee, B., Chung, S., Kim, C. W., Park, B., & Ro, Y. M. (2024) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2024.emnlp-main.633 · Cited by 18
VLM Agents Generate Their Own Memories: Distilling Experience into Embodied Programs of Thought
Sarch, G. H., Jang, L., Tarr, M. J., Cohen, W. W., Marino, K., & Fragkiadaki, K. (2024) · neural information processing systems · doi:10.52202/079017-2418 · Cited by 18
ST-Think: How Multimodal Large Language Models Reason About 4D Worlds from Ego-Centric Videos
Wu, P., Liu, Y., Liu, M., & Shen, J. (2026) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv61042.2026.00502 · Cited by 18
UnrealZoo: Enriching Photo-Realistic Virtual Worlds for Embodied AI
Zhong, F., Wu, K., Wang, C., Chen, H., Ci, H., Li, Z., & Wang, Y. (2025) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51701.2025.00546 · Cited by 17
Naderi, H., Shojaei, A., & Huang, L. (2026) · Eng—Advances in Engineering · doi:10.3390/eng7070358 · Cited by 18
FOCUS: Internal MLLM Representations for Efficient Fine-Grained Visual Question Answering
Zhong, L., Rosenthal, F. P., Sicking, J., Hüger, F., Bagdonat, T., Gottschalk, H., & Schwinn, L. (2025) · neural information processing systems · doi:10.52202/085713-3227 · Cited by 16
Helpful DoggyBot: Open-World Object Fetching using Legged Robots and Vision-Language Models
Wu, Q. M., Fu, Z., Cheng, X., Wang, X., & Finn, C. (2025) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros60139.2025.11246368 · Cited by 16
EfficientEQA: An Efficient Approach to Open-Vocabulary Embodied Question Answering
Cheng, K., Li, Z., Sun, X., Min, B., Bedi, A. S., & Bera, A. (2025) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros60139.2025.11247438 · Cited by 16
SpatialLLM: From multi-modality data to urban spatial intelligence
Chen, J., Wang, H., Li, J., Liu, Y., Dong, Z., & Yang, B. (2026) · International Journal of Applied Earth Observation and Geoinformation · doi:10.1016/j.jag.2026.105177 · Cited by 15
Guan, L., Zhou, Y., Liu, D., Zha, Y., Amor, H. B., & Subbarao Kambhampati, B. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2402.04210 · Cited by 15
Lin, F., Tian, Y., Wang, Y., Zhang, T., Zhang, X., & Wang, F. (2024) · IEEE International Conference on Intelligent Transportation Systems (ITSC) · doi:10.1109/itsc58415.2024.10919532 · Cited by 14
SILMM: Self-Improving Large Multimodal Models for Compositional Text-to-Image Generation
Qu, L., Li, H., Wang, W., Liu, X., Li, J., Nie, L., & Chua, T. (2025) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52734.2025.01724 · Cited by 14
AerialVG: A Challenging Benchmark for Aerial Visual Grounding by Exploring Positional Relations
Liu, J., Chen, Q., Wang, Z., Tang, Y., Zhang, Y., Yan, C., Wang, D., Li, X., & Zhao, B. L. (2025) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51701.2025.00492 · Cited by 14
Scene Exploration by Vision-Language ModelsPreprint
Sripada, V., Carter, S., Guérin, F., & E., A. M. G. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2409.17641 · Cited by 14
PIVOT: Iterative Visual Prompting Elicits Actionable Knowledge for VLMs
Nasiriany, S., Xia, F., Yu, W., Xiao, T., Liang, J., Dasgupta, I., Xie, A., Driess, D., Wahid, A., Xu, Z., et al. (2024) · arXiv · doi:10.48550/arxiv.2402.07872 · Cited by 13
Self-citationYu, J., Zhang, Y., Zhang, Z., Zhao, Y., Zhao, G., Sun, F., Zhang, F., Liu, Q., Sun, J., Liang, J., et al. (2024) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3664647.3688990 · Cited by 13
PhysVLM: Enabling Visual Language Models to Understand Robotic Physical Reachability
Zhou, W., Tao, M., Zhao, C., Guo, H., Dong, H., Tang, M., & Wang, J. (2025) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52734.2025.00651 · Cited by 13
SpatialLLM: A Compound 3D-Informed Design towards Spatially-Intelligent Large Multimodal Models
Ma, W., Ye, L., de Melo, C. M., Yuille, A., & Chen, J. (2025) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52734.2025.01608 · Cited by 13
Wang, Y., Liu, Q., Jiang, Z., Wang, T., Jiao, J., Chu, H., Gao, B., & Chen, H. (2025) · IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw67362.2025.00369 · Cited by 13
From Pixels to Tokens: Revisiting Object Hallucinations in Large Vision-Language Models
Shang, Y., Zeng, X., Zhu, Y., Yang, X. H., Fang, Z., Zhang, J., Chen, J., Liu, Z., & Tian, Y. (2025) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3746027.3755728 · Cited by 13
SpatiaLab: Can Vision-Language Models Perform Spatial Reasoning in the Wild?Preprint
Wasi, A. T., Faisal, W., Rahman, A., Anik, M. A., Shahriar, M., Topu, M. M., Meem, S. T., Priti, R. N., Mitu, S. A., Hoque, M. I., et al. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2602.03916 · Cited by 13
TopViewRS: Vision-Language Models as Top-View Spatial Reasoners
Li, C., Zhang, C., Zhou, H., Collier, N., Korhonen, A., & Vulić, I. (2024) · Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/2024.emnlp-main.106 · Cited by 12
Metadata
- Authors
- Chen, B., Xu, Z., Kirmani, S., Ichter, B., Driess, D., Florence, P., Sadigh, D., Guibas, L., & Xia, F.
- Affiliations
- Google DeepMind (United Kingdom) (GB)
- Google (United States) (US)
- Venue
- arXiv
- Publisher
- arXiv
- Published
- 2024
- Pages
- 14455-14465
- ISSN
- 1063-6919 (ISSN-L)
- arXiv
- 2401.12168
- OpenAlex
- W4402716288
- Cited by
- 269
- References
- 69
- Citations by year
- 2026: 122 · 2025: 127 · 2024: 20
- FWCI
- 55.53
- Type
- conference paper
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Multimodal Machine Learning Applications
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
- Physical Sciences › Engineering › Control and Systems Engineering › Robot Manipulation and Learning
- Keywords
- vision-language models, spatial reasoning, 3D spatial reasoning, visual question answering, qualitative spatial reasoning, automatic data generation, metric spaces, chain-of-thought reasoning, robotics, distance estimation
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
