Vision Language Models are blind
Open accessIntegrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In 2D perception-based physical reasoning, vision-language models fail on anomaly detection, object counting and overlap identification, and spatial relation understanding from images.
“similar tasks reveal surprising failures in simple tasks such as anomaly detection (Bitton-Guetta et al., 2023; Zhou et al., 2023b), object counting and overlap identification (Rahmanzadehgervi et al., 2024), and spatial relation understanding from the image content (Liu et al., 2023a; Zhao et al., 2024a).”
From Large Language Model Reasoning Failures (2026) · cites this paper as Rahmanzadehgervi et al., 2024
Vision-language models reveal surprising failures in simple tasks such as anomaly detection, object counting and overlap identification, and spatial relation understanding from image content.
“Applied to vision-language models (VLMs), similar tasks reveal surprising failures in simple tasks such as anomaly detection (Bitton-Guetta et al., 2023; Zhou et al., 2023b), object counting and overlap identification (Rahmanzadehgervi et al., 2024), and spatial relation understanding from the image content (Liu et al., 2023a; Zhao et al., 2024a).”
From Large Language Model Reasoning Failures (2026) · cites this paper as Rahmanzadehgervi et al., 2024
Advanced models fail on very basic visual tests.
“Explanation: Advanced models fail on very basic visual tests.”
From Large Language Model Reasoning Failures (2026) · cites this paper as Rahmanzadehgervi et al., 2024
Cites
Intern VL: Scaling up Vision Foundation Models and Aligning for Generic Visual-Linguistic Tasks
Chen, Z., Wu, J., Wang, W., Su, W., Chen, G., Xing, S., Zhong, M., Zhang, Q., Zhu, X., Lu, L., et al. (2024) · 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52733.2024.02283 · Cited by 734 · see citing works
MMMU: A Massive Multi-Discipline Multimodal Understanding and Reasoning Benchmark for Expert AGI
Yue, X., Ni, Y., Zheng, T., Zhang, K., Liu, R., Zhang, G., Stevens, S., Jiang, D., Ren, W., Sun, Y., et al. (2024) · 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52733.2024.00913 · Cited by 429 · see citing works
ActivityNet-QA: A Dataset for Understanding Complex Web Videos via Question Answering
Yu, Z., Xu, D., Yu, J., Yu, T., Zhao, Z., Zhuang, Y., & Tao, D. (2019) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v33i01.33019127 · Cited by 281 · see citing works
Eyes Wide Shut? Exploring the Visual Shortcomings of Multimodal LLMs
Tong, S., Liu, Z., Zhai, Y., Ma, Y., LeCun, Y., & Xie, S. (2024) · 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52733.2024.00914 · Cited by 189 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
Large Language Model Reasoning Failures
Song, P., Han, P., & Goodman, N. (2026) · arXiv · doi:10.48550/arxiv.2602.06176 · Cited by 3 · see citing works
PerceptionBench: Evaluating Atomic Visual Perception in Multimodal Large Language Models
Lin, Z., Xie, Y., Qu, B., Wang, H., Li, J., Wu, H., Dong, Y., Yang, Z., Zhu, J., Lu, H., et al. (2026) · arXiv · doi:10.48550/arxiv.2607.24957
- Authors
- Rahmanzadehgervi, P., Bolton, L., Taesiri, M. R., & Nguyen, A. T.
- Affiliations
- Auburn University (US)
- University of Alberta (CA)
- Corresponding author
- Nguyen, Anh Totti
- Venue
- Lecture Notes in Computer Science
- Publisher
- Springer Nature Singapore
- Published
- 2024-12-08
- Pages
- 293-309
- ISSN
- 0302-9743 (ISSN-L)
- 1611-3349
- OpenAlex
- W4405140008
- Cited by
- 54
- References
- 4
- Citations by year
- 2026: 22 · 2025: 29 · 2024: 3
- FWCI
- 10.05
- Type
- conference paper
- Language
- en
- Open access
- green
- arxiv.org
- Licence
- Locations
- Lecture notes in computer science (publishedVersion)
- arXiv (Cornell University) (submittedVersion, cc-by)
- arXiv (Cornell University) (cc-by)
- Funders
- National Science Foundation
- Auburn University
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Multimodal Machine Learning Applications
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Handwritten Text Recognition Techniques
- Keywords
- vision-language models, geometric reasoning, spatial reasoning, geometric primitives, image-text understanding, visual grounding
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
