CoChat
Sign Up
PreprintCited by 1Multimodal Machine Learning ApplicationsVisual Attention and Saliency DetectionAdvanced Neural Network Applications

Position: Reasoning After Perception Means Reasoning Without Vision

Open access
Gao, Hongcheng, Huang, Zihao, Tang, Jingyi, Xu, Lin, Li, Xinhao, Li, Haoyang, Liu, Yue, Lin, Minhua +7 more·2025·arXiv
DOI 10.48550/arxiv.2507.16863arXiv 2507.16863
Open this paper in CoChat — free accountOpen PDF

Abstract

A common belief in multimodal research is that the perceptual weaknesses of vision--language models can be compensated by stronger language reasoning (e.g., chain-of-thought, in-context learning, or external tools). We challenge this assumption. We argue that for a broad class of visual tasks hard to specify in language, failures stem from a structural fatality where the temporal decision of \textit{when} to reason strictly dictates the spatial constraint of \textit{where} reasoning takes place. When visual reasoning is deferred to language generation, current architectures do not merely delay computation; they displace it from the continuous visual representation to a discrete textual space. Consequently, the sequential ``Perception-then-Reasoning'' paradigm degenerates perception into a passive, one-off feature encoding process, rendering it functionally equivalent to ``Reasoning-in-Text-Space'', where task-critical spatial signals are collapsed before reasoning begins. We substantiate this claim with the Turing Eye Test (TET): tasks that must be resolved in \emph{visual space} and are hard to verbalize; results show text-only reasoning cannot remedy these perceptual failures. Our findings suggest rethinking the architectural divide: shifting from reasoning \textit{about} perception to reasoning \textit{within} perception. This facilitates actively reasoning-driven perception that operates directly on pixel-level visual representations, rather than within a collapsed textual space.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

DataCite · 10.48550/arxiv.2507.16863

Cites

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Downstream reasoning cannot reliably recover visual content that was missed, misread, or hallucinated.

    “Since downstream reasoning cannot reliably recover visual content that was missed, misread, or hallucinated, uncorrected perceptual errors can invalidate subsequent reasoning (Buschoff et al. 2025; Gao, Zihao Huang, et al. 2025).”

    From PerceptionBench: Evaluating Atomic Visual Perception in Multimodal Large Language Models (2026) · cites this paper as Gao, Zihao Huang, et al. 2025

    Appendix C says the decoder misses essential visual elements and cannot recover spatial information lost before reasoning begins. The paper supports the claim about missed visual information, but our copy does not establish the same point for misread or hallucinated content. (LLM, conf=0.83)

    Experimental — accuracy not yet measured

  • Frontier MLLMs continue to struggle with basic visual perception.

    “these studies show that frontier MLLMs continue to struggle with basic visual perception”

    From PerceptionBench: Evaluating Atomic Visual Perception in Multimodal Large Language Models (2026) · cites this paper as Gao, Zihao Huang, et al. 2025

  • Uncorrected perceptual errors can invalidate subsequent reasoning.

    “Since downstream reasoning cannot reliably recover visual content that was missed, misread, or hallucinated, uncorrected perceptual errors can invalidate subsequent reasoning (Buschoff et al. 2025; Gao, Zihao Huang, et al. 2025).”

    From PerceptionBench: Evaluating Atomic Visual Perception in Multimodal Large Language Models (2026) · cites this paper as Gao, Zihao Huang, et al. 2025

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

No references could be resolved yet.

Research works printed but not linked: 85

  1. Aissi, M. S., Grislain, C., Chetouani, M., Sigaud, O., Soulier, L., and Thome, N. Viper: Visual perception and ex- plainable reasoning for sequential decision-making, 2025. URL https://arxiv.org/abs/2503.15108. · arXiv
  2. Alayrac, J.-B., Donahue, J., Luc, P., Miech, A., Barr, I., Hasson, Y., Lenc, K., Mensch, A., Millican, K., Reynolds, M., Ring, R., Rutherford, E., Cabi, S., Han, T., Gong, Z., Samangooei, S., Monteiro, M., Menick, J., Borgeaud, S., Brock, A., Nematzadeh, A., Sharifzadeh, S., Binkowski, M., Barreira, R., Vinyals, O., Zisser- man, A., and Simonyan, K. Flamingo: a visual lan- guage model for few-shot learning. In Advances in Neural Information Processing Systems, 2022. URL https://arxiv.org/abs/2204.14198. · arXiv
  3. An, W., Nie, J., Wu, Y., Tian, F., Lu, S., and Zheng, Q. Em- powering multimodal llms with external tools: A compre- hensive survey, 2025a. URL https://arxiv.org/ abs/2508.10955. · arXiv
  4. An, X., Xie, Y., Yang, K., Zhang, W., Zhao, X., Cheng, Z., Wang, Y., Xu, S., Chen, C., Zhu, D., et al. Llava- onevision-1.5: Fully open framework for democratized multimodal training. arXiv preprint arXiv:2509.23661, 2025b. Anthropic. Introducing Claude 4. Anthropic News, May 2025. URL https://www.anthropic.com/ news/claude-4. Accessed: [Insert Access Date]. · arXiv
  5. Bai, S., Cai, Y., Chen, R., Chen, K., Chen, X., Cheng, Z., Deng, L., Ding, W., Gao, C., Ge, C., Ge, W., Guo, Z., Huang, Q., Huang, J., Huang, F., Hui, B., Jiang, S., Li, Z., Li, M., Li, M., Li, K., Lin, Z., Lin, J., Liu, X., Liu, J., Liu, C., Liu, Y., Liu, D., Liu, S., Lu, D., Luo, R., Lv, C., Men, R., Meng, L., Ren, X., Ren, X., Song, S., Sun, Y., Tang, J., Tu, J., Wan, J., Wang, P., Wang, P., Wang, Q., Wang, Y., Xie, T., Xu, Y., Xu, H., Xu, J., Yang, Z., Yang, M., Yang, J., Yang, A., Yu, B., Zhang, F., Zhang, H., Zhang, X., Zheng, B., Zhong, H., Zhou, J., Zhou, F., Zhou, J., Zhu, Y., and Zhu, K. Qwen3-vl technical report, 2025a. URL https://arxiv.org/abs/2511.21631. · arXiv
  6. Bai, S., Chen, K., Liu, X., Wang, J., Ge, W., Song, S., Dang, K., Wang, P., Wang, S., Tang, J., Zhong, H., Zhu, Y., Yang, M., Li, Z., Wan, J., Wang, P., Ding, W., Fu, Z., Xu, Y., Ye, J., Zhang, X., Xie, T., Cheng, Z., Zhang, H., Yang, Z., Xu, H., and Lin, J. Qwen2.5-vl technical report. arXiv preprint arXiv:2502.13923, 2025b. · arXiv
  7. Bai, T., Fan, Y., Qiu, J., Sun, F., Song, J., Han, J., Liu, Z., He, C., Zhang, W., and Yuan, B. Hallucination at a glance: Controlled visual edits and fine-grained multi- modal learning. arXiv preprint arXiv:2506.07227, 2025c. · arXiv
  8. Cao, S., Li, J., Li, X., Pu, Y., Zhu, K., Gao, Y., Luo, S., Xin, Y., Qin, Q., Zhou, Y., et al. Unipercept: Towards unified perceptual-level image understanding across aes- thetics, quality, structure, and texture. arXiv preprint arXiv:2512.21675, 2025. · arXiv
  9. Cartuyvels, R., Spinks, G., and Moens, M.-F. Dis- crete and continuous representations and processing in deep learning: Looking forward. AI Open, 2:143-159, 2021. ISSN 2666-6510. doi: 10.1016/j.aiopen.2021. 07.002. URL http://dx.doi.org/10.1016/j. aiopen.2021.07.002. · DOI
  10. Chen, G., Xu, W., Zhang, H., Chan, H. P., Zhao, D., Luu, A. T., and Rong, Y. Geopqa: Bridging the visual percep- tion gap in mllms for geometric reasoning. In Proceed- ings of the Conference on Empirical Methods in Natural Language Processing, pp. 25680-25688, 2025a. URL https://arxiv.org/abs/2509.17437. · arXiv
  11. Chen, L., Li, J., Dong, X., Zhang, P., Zang, Y., Chen, Z., Duan, H., Wang, J., Qiao, Y., Lin, D., et al. Are we on the right way for evaluating large vision-language models? In Advances in Neural Information Processing Systems, 2024a. URL https://arxiv.org/abs/ 2403.20330.
  12. Chen, L., Xie, W., Liang, Y., He, H., Zhao, H., Yang, Z., Huang, Z., Wu, H., Lu, H., charles, Y., Bao, Y., Fan, Y., Li, G., Shen, H., Chen, X., Xu, W., Si, S., Cai, Z., Chai, W., Huang, Z., Liu, F., Liu, T., Chang, B., Hu, X., Chen, K., Ren, Y., Liu, Y., Gong, Y., and Li, K. Babyvision: Visual reasoning beyond language, 2026. URL https://arxiv.org/abs/2601.06521. · arXiv
  13. Chen, X., Wu, Z., Liu, X., Pan, Z., Liu, W., Xie, Z., Yu, X., and Ruan, C. Janus-pro: Unified multimodal understand- ing and generation with data and model scaling. arXiv preprint arXiv:2501.17811, 2025b. · arXiv
  14. Chen, Y., Li, L., Xi, T., Zeng, L., and Wang, J. Perception before reasoning: Two-stage reinforcement learning for visual reasoning in vision-language models, 2025c. URL https://arxiv.org/abs/2509.13031. · arXiv
  15. Chen, Z., Wu, J., Wang, W., Su, W., Chen, G., Xing, S., Zhong, M., Zhang, Q., Zhu, X., Lu, L., et al. Internvl: Scaling up vision foundation models and aligning for generic visual-linguistic tasks. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 24185-24198, 2024b.
  16. Chollet, F., Knoop, M., Kamradt, G., Landers, B., and Pinkard, H. Arc-agi-2: A new challenge for frontier ai reasoning systems, 2026. URL https://arxiv. org/abs/2505.11831.
  17. Chung, J., Kim, J., Kim, S., Lee, J., Kim, M. S., and Yu, Y. v1: Learning to point visual tokens for multimodal grounded reasoning, 2025. URL https://arxiv. org/abs/2505.18842.
  18. Dai, W., Li, J., Li, D., Tiong, A. M. H., Zhao, J., Wang, W., Li, B., Fung, P., and Hoi, S. Instructblip: Towards general-purpose vision-language models with instruction tuning. In Advances in Neural Information Processing Systems, 2023. URL https://arxiv.org/abs/ 2305.06500.
  19. Daxberger, E., Wenzel, N., Griffiths, D., Gang, H., Lazarow, J., Kohavi, G., Kang, K., Eichner, M., Yang, Y., Dehghan, A., et al. Mm-spatial: Exploring 3d spatial understanding in multimodal llms. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 7395- 7408, 2025.
  20. Deng, C., Zhu, D., Li, K., Gou, C., Li, F., Wang, Z., Zhong, S., Yu, W., Nie, X., Song, Z., et al. Emerging proper- ties in unified multimodal pretraining. arXiv preprint arXiv:2505.14683, 2025. · arXiv
  21. Fu, S., Bonnen, T., Guillory, D., and Darrell, T. Hid- den in plain sight: Vlms overlook their visual repre- sentations, 2025. URL https://arxiv.org/abs/ 2506.08008.
  22. Fu, X., Hu, Y., Li, B., Feng, Y., Wang, H., Lin, X., Roth, D., Smith, N. A., Ma, W.-C., and Krishna, R. Blink: Multi- modal large language models can see but not perceive. In European Conference on Computer Vision, 2024. URL https://arxiv.org/abs/2404.12390. · arXiv
  23. Guo, D., Wu, F., Zhu, F., Leng, F., Shi, G., Chen, H., Fan, H., Wang, J., Jiang, J., Wang, J., Chen, J., Huang, J., Lei, K., Yuan, L., Luo, L., Liu, P., Ye, Q., Qian, R., Yan, S., Zhao, S., Peng, S., Li, S., Yuan, S., Wu, S., Cheng, T., Liu, W., Wang, W., Zeng, X., Liu, X., Qin, X., Ding, X., Xiao, X., Zhang, X., Zhang, X., Xiong, X., Peng, Y., Chen, Y., Li, Y., Hu, Y., Lin, Y., Hu, Y., Zhang, Y., Wu, Y., Li, Y., Liu, Y., Ling, Y., Qin, Y., Wang, Z., He, Z., Zhang, A., Yi, B., Liao, B., Huang, C., Zhang, C., Deng, C., Deng, C., Lin, C., Yuan, C., Li, C., Gou, C., Lou, C., Wei, C., Liu, C., Li, C., Zhu, D., Zhong, D., Li, F., Zhang, F., Wu, G., Li, G., Xiao, G., Lin, H., Yang, H., Wang, H., Ji, H., Hao, H., Shen, H., Li, H., Li, J., Wu, J., Zhu, J., Jiao, J., Feng, J., Chen, J., Duan, J., Liu, J., Zeng, J., Tang, J., Sun, J., Chen, J., Long, J., Feng, J., Zhan, J., Fang, J., Lu, J., Hua, K., Liu, K., Shen, K., Zhang, K., Shen, K., Wang, K., Pan, K., Zhang, K., Li, K., Li, L., Li, L., Shi, L., Han, L., Xiang, L., Chen, L., Chen, L., Li, L., Yan, L., Chi, L., Liu, L., Du, M., Wang, M., Pan, N., Chen, P., Chen, P., Wu, P., Yuan, Q., Shuai, Q., Tao, Q., Zheng, R., Zhang, R., Zhang, R., Wang, R., Yang, R., Zhao, R., Xu, S., Liang, S., Yan, S., Zhong, S., Cao, S., Wu, S., Liu, S., Chang, S., Cai, S., Ao, T., Yang, T., Zhang, T., Zhong, W., Jia, W., Weng, W., Yu, W., Huang, W., Zhu, W., Yang, W., Wang, W., Long, X., Yin, X., Li, X., Zhu, X., Jia, X., Zhang, X., Liu, X., Zhang, X., Yang, X., Luo, X., Chen, X., Zhong, X., Xiao, X., Li, X., Wu, Y., Wen, Y., Du, Y., Zhang, Y., Ye, Y., Wu, Y., Liu, Y., Yue, Y., Zhou, Y., Yuan, Y., Xu, Y., Yang, Y., Zhang, Y., Fang, Y., Li, Y., Ren, Y., Xiong, Y., Hong, Z., Wang, Z., Sun, Z., Wang, Z., Cai, Z., Zha, Z., An, Z., Zhao, Z., Xu, Z., Chen, Z., Wu, Z., Zheng, Z., Wang, Z., Huang, Z., Zhu, Z., and Song, Z. Seed1.5-vl technical report, 2025. URL https://arxiv.org/abs/2505.07062. · arXiv
  24. He, H., Li, G., Geng, Z., Xu, J., and Peng, Y. Analyzing and boosting the power of fine-grained visual recognition for multi-modal large language models. In International Conference on Learning Representations, 2025. URL https://arxiv.org/abs/2501.15140. · arXiv
  25. Huang, J.-T., Dai, D., Huang, J.-Y., Yuan, Y., Liu, X., Wang, W., Jiao, W., He, P., Tu, Z., and Duan, H. Human cognitive benchmarks reveal foundational visual gaps in mllms, 2026. URL https://arxiv.org/abs/ 2502.16435.
  26. Huang, W., Jia, B., Zhai, Z., Cao, S., Ye, Z., Zhao, F., Xu, Z., Hu, Y., and Lin, S. Vision-r1: Incentivizing reasoning capability in multimodal large language models, 2025. URL https://arxiv.org/abs/2503.06749. · arXiv
  27. Jaech, A., Kalai, A., Lerer, A., Richardson, A., El-Kishky, A., Low, A., Helyar, A., Madry, A., Beutel, A., Car- ney, A., et al. Openai o1 system card. arXiv preprint arXiv:2412.16720, 2024. · arXiv
  28. Ke, F., Hsu, J., Cai, Z., Ma, Z., Zheng, X., Wu, X., Huang, S., Wang, W., Haghighi, P. D., Haffari, G., Krishna, R., Wu, J., and Rezatofighi, H. Explain before you answer: A survey on compositional visual reasoning, 2025. URL https://arxiv.org/abs/2508.17298. · arXiv
  29. Kim, S., Jin, J., Hong, S., Ka, D., Kim, H., and Noh, B. Dsc-llm: Driving scene context representation-based tra- jectory prediction framework with risk factor reasoning using llms. Sensors, 25(23):7112, 2025.
  30. Kuchibhotla, H. C., Kancheti, S. S., Reddy, A. G., and Balasubramanian, V. N. Efficient vocabulary-free fine- grained visual recognition in the age of multimodal llms. Transactions on Machine Learning Research, 2025. URL https://arxiv.org/abs/2505.01064. · arXiv
  31. Lai, X., Li, J., Li, W., Liu, T., Li, T., and Zhao, H. Mini- o3: Scaling up reasoning patterns and interaction turns for visual search, 2025. URL https://arxiv.org/ abs/2509.07969. · arXiv
  32. Li, C., Wu, W., Zhang, H., Xia, Y., Mao, S., Dong, L., Vulić, I., and Wei, F. Imagine while reasoning in space: Mul- timodal visualization-of-thought. In Proceedings of the International Conference on Machine Learning, 2025a. URL https://arxiv.org/abs/2501.07542. · arXiv
  33. Li, K., Shang, C., Karlinsky, L., Feris, R., Darrell, T., and Herzig, R. Latent implicit visual reasoning, 2025b. URL https://arxiv.org/abs/2512.21218. · arXiv
  34. Li, L., Wei, Y., Xie, Z., Yang, X., Song, Y., Wang, P., An, C., Liu, T., Li, S., Lin, B. Y., Kong, L., and Liu, Q. Vl-rewardbench: A challenging benchmark for vision-language generative reward models. In Pro- ceedings of the IEEE/CVF Conference on Computer Vi- sion and Pattern Recognition, pp. 24657-24668, 2025c. doi: 10.1109/CVPR52734.2025.02296. URL https: //arxiv.org/abs/2411.17451. · DOI
  35. Li, Z., Du, Y., Liu, Y., Zhang, Y., Liu, Y., Zhang, M., and Cai, X. Eagle: Elevating geometric reasoning through llm-empowered visual instruction tuning. arXiv preprint arXiv:2408.11397, 2024. · arXiv
  36. Liu, H., Li, C., Wu, Q., and Lee, Y. J. Visual instruction tuning. In Advances in Neural Information Processing Systems, 2023. URL https://arxiv.org/abs/ 2304.08485.
  37. Liu, H., Li, C., Li, Y., and Lee, Y. J. Improved baselines with visual instruction tuning. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 26296-26306, 2024.
  38. Liu, L., Chen, L., Zhang, Z., Tian, M., Cui, H., Li, R., Liu, Z., Ju, Q., Li, Q., and Zhou, H.-Y. Skinflow: Efficient information transmission for open dermatological diag- nosis via dynamic visual encoding and staged rl, 2026. URL https://arxiv.org/abs/2601.09136. · arXiv
  39. Liu, S., Yao, D., Lin, Y., Cong, G., and Bi, J. Traj-mllm: Can multimodal large language models reform trajectory data mining? arXiv preprint arXiv:2509.00053, 2025a. · arXiv
  40. Liu, Z., Gao, K., Liang, S., Xiao, B., Qiao, L., Ma, L., and Jiang, T. Beyond the visible: Benchmarking occlusion perception in multimodal large language models. arXiv preprint arXiv:2508.04059, 2025b. · arXiv
  41. Lu, P., Mishra, S., Xia, T., Qiu, L., Chang, K.-W., Zhu, S.-C., Tafjord, O., Clark, P., and Kalyan, A. Learn to explain: Multimodal reasoning via thought chains for science question answering. In Advances in Neural Infor- mation Processing Systems, volume 35, pp. 2507-2521, 2022.
  42. Lu, P., Bansal, H., Xia, T., Liu, J., Li, C., Hajishirzi, H., Cheng, H., Chang, K.-W., Galley, M., and Gao, J. Mathvista: Evaluating mathematical reasoning of foundation models in visual contexts. In International Conference on Learning Representations, 2024. URL https://arxiv.org/abs/2310.02255. · arXiv
  43. Ma, W., Wang, C., Yuan, R., Chen, H., Dai, N., Zhou, S. K., Yang, Y., Yuille, A., and Chen, J. Causalspatial: A bench- mark for object-centric causal spatial reasoning, 2026. URL https://arxiv.org/abs/2601.13304. · arXiv
  44. Mathew, M., Karatzas, D., and Jawahar, C. V. Docvqa: A dataset for vqa on document images. In Proceed- ings of the IEEE/CVF Winter Conference on Applica- tions of Computer Vision, pp. 2199-2208, 2021. doi: 10.1109/WACV48630.2021.00225. URL https:// arxiv.org/abs/2007.00398. · DOI
  45. Mishra, A., Shekhar, S., Singh, A. K., and Chakraborty, A. Ocr-vqa: Visual question answering by reading text in images. In 2019 international conference on document analysis and recognition (ICDAR), pp. 947-952. IEEE, 2019.
  46. Nagrani, A., Yang, S., Arnab, A., Jansen, A., Schmid, C., and Sun, C. Attention bottlenecks for multimodal fu- sion. In Advances in Neural Information Processing Systems, 2021. URL https://arxiv.org/abs/ 2107.00135.
  47. Ouyang, L., Qu, Y., Zhou, H., Zhu, J., Zhang, R., Lin, Q., Wang, B., Zhao, Z., Jiang, M., Zhao, X., et al. Om- nidocbench: Benchmarking diverse pdf document parsing with comprehensive annotations. In Proceedings of the Computer Vision and Pattern Recognition Conference, pp. 24838-24848, 2025.
  48. Qu, L., Zhang, H., Liu, Y., Wang, X., Jiang, Y., Gao, Y., Ye, H., Du, D. K., Yuan, Z., and Wu, X. Tokenflow: Unified image tokenizer for multimodal understanding and gener- ation. In Proceedings of the Computer Vision and Pattern Recognition Conference, pp. 2545-2555, 2025.
  49. Radford, A., Kim, J. W., Hallacy, C., Ramesh, A., Goh, G., Agarwal, S., Sastry, G., Askell, A., Mishkin, P., Clark, J., Krueger, G., and Sutskever, I. Learning transferable visual models from natural language supervision. In Proceedings of the International Conference on Machine Learning, 2021. URL https://arxiv.org/abs/ 2103.00020.
  50. Rudman, W., Golovanevsky, M., Bar, A., Palit, V., Le- Cun, Y., Eickhoff, C., and Singh, R. Forgotten poly- gons: Multimodal large language models are shape- blind. In Proceedings of the Annual Meeting of the As- sociation for Computational Linguistics, 2025. URL https://arxiv.org/abs/2502.15969. · arXiv
  51. Selvaraju, R. R., Cogswell, M., Das, A., Vedantam, R., Parikh, D., and Batra, D. Grad-cam: Visual explana- tions from deep networks via gradient-based localization. In Proceedings of the IEEE international conference on computer vision, pp. 618-626, 2017.
  52. Su, Z., Li, L., Song, M., Hao, Y., Yang, Z., Zhang, J., Chen, G., Gu, J., Li, J., Qu, X., and Cheng, Y. Openthinkimg: Learning to think with images via visual tool reinforce- ment learning, 2025. URL https://arxiv.org/ abs/2505.08617. · arXiv
  53. Surís, D., Menon, S., and Vondrick, C. Vipergpt: Visual inference via python execution for reasoning. In Pro- ceedings of the IEEE/CVF international conference on computer vision, pp. 11888-11898, 2023.
  54. Tang, H., Xie, C., Bao, X., Weng, T., Li, P., Zheng, Y., and Wang, L. Unilip: Adapting clip for unified multimodal understanding, generation and editing. arXiv preprint arXiv:2507.23278, 2025. · arXiv
  55. Team, G. . Gemini 2.5: Pushing the frontier with ad- vanced reasoning, multimodality, long context, and next generation agentic capabilities, 2025b. URL https: //arxiv.org/abs/2507.06261.
  56. Team, G. Gemini: A family of highly capable multimodal models, 2025c. URL https://arxiv.org/abs/ 2312.11805.
  57. Team, K. Kimi k1.5: Scaling reinforcement learning with llms, 2025d. URL https://arxiv.org/abs/ 2501.12599.
  58. Team, K. Kimi-vl technical report, 2025e. URL https: //arxiv.org/abs/2504.07491.
  59. Tong, S., Liu, Z., Zhai, Y., Ma, Y., LeCun, Y., and Xie, S. Eyes wide shut? exploring the visual shortcomings of multimodal llms. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 9568-9578, 2024.
  60. Wang, C., Luo, W., Dong, S., Xuan, X., Li, Z., Ma, L., and Gao, S. Mllm-tool: A multimodal large lan- guage model for tool agent learning. In Proceed- ings of the IEEE/CVF Winter Conference on Applica- tions of Computer Vision, pp. 6678-6687, 2025a. doi: 10.1109/WACV61041.2025.00650. URL https:// arxiv.org/abs/2401.10727. · DOI
  61. Wang, C., Meng, F., Pei, J., Xia, L., Liu, J., Yuan, X., and Di, X. Occ-mllm-cot: Self-correction enhanced occlusion recognition with large language models via 3d-aware supervision, chain-of-thoughts guidance. Image and Vision Computing, pp. 105881, 2025b.
  62. Wang, X., Huang, J., Zhang, X., Wang, T., and Ma, J. W. Your reasoning benchmark may not test reasoning: Re- vealing perception bottleneck in abstract reasoning bench- marks, 2026a. URL https://arxiv.org/abs/ 2512.21329.
  63. Wang, X., Zhang, K., Jia, Q., Chen, Z., Zhai, G., and Min, X. Kidvis: Do multimodal large language models possess the visual perceptual capabilities of a 6-year-old?, 2026b. URL https://arxiv.org/abs/2601.08292. · arXiv
  64. Wang, Y., Wu, S., Zhang, Y., Yan, S., Liu, Z., Luo, J., and Fei, H. Multimodal chain-of-thought reasoning: A comprehensive survey, 2025c. URL https://arxiv. org/abs/2503.12605.
  65. Wang, Z., Che, C., Wang, Q., Li, Y., Shi, Z., and Wang, M. Smolora: Exploring and defying dual catas- trophic forgetting in continual visual instruction tun- ing. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 177-186, 2025d. doi: 10.1109/ICCV51701.2025.00024. URL https: //arxiv.org/abs/2411.13949. · DOI
  66. Wang, Z., Zhang, T., Guo, M., Wang, Y., and Ye, Q. Videoanchor: Reinforcing subspace-structured visual cues for coherent visual-spatial reasoning, 2025e. URL https://arxiv.org/abs/2509.25151. · arXiv
  67. Wei, J., Wang, X., Schuurmans, D., Bosma, M., Ichter, B., Xia, F., Chi, E., Le, Q., and Zhou, D. Chain-of- thought prompting elicits reasoning in large language models. In Advances in Neural Information Processing Systems, 2022. URL https://arxiv.org/abs/ 2201.11903.
  68. Weng, Z., Gomez, L., Webb, T. W., and Bashivan, P. Caption this, reason that: Vlms caught in the middle, 2025. URL https://arxiv.org/abs/2505.21538. · arXiv
  69. Wu, D., Liu, F., Hung, Y.-H., and Duan, Y. Spatial-mllm: Boosting mllm capabilities in visual-based spatial intelli- gence. arXiv preprint arXiv:2505.23747, 2025a. · arXiv
  70. Wu, H., Zhang, Z., Zhang, W., Chen, C., Liao, L., Li, C., Gao, Y., Wang, A., Zhang, E., Sun, W., et al. Q-align: Teaching lmms for visual scoring via discrete text-defined levels. In Proceedings of the International Conference on Machine Learning, 2024. URL https://arxiv. org/abs/2312.17090.
  71. Wu, J., Xiong, Y., Li, X., Xia, Y., Wang, R., Wang, Y., Yu, T., Kim, S., Rossi, R. A., Yao, L., Shang, J., and McAuley, J. Mitigating visual knowledge forgetting in mllm instruction-tuning via modality-decoupled gradient descent. In Proceedings of the Conference on Empirical Methods in Natural Language Processing, 2025b. URL https://arxiv.org/abs/2502.11740. · arXiv
  72. Wu, M., Yang, J., Jiang, J., Li, M., Yan, K., Yu, H., Zhang, M., Zhai, C., and Nahrstedt, K. Vtool-r1: Vlms learn to think with images via reinforcement learning on multi- modal tool use, 2025c. URL https://arxiv.org/ abs/2505.19255. · arXiv
  73. Xiao, Y., Sun, E., Liu, T., and Wang, W. Logicvista: Multimodal llm logical reasoning benchmark in visual contexts, 2024. URL https://arxiv.org/abs/ 2407.04973.
  74. Xie, J., Yang, Z., and Shou, M. Z. Show-o2: Im- proved native unified multimodal models. arXiv preprint arXiv:2506.15564, 2025. · arXiv
  75. Xu, J., Guo, Z., He, J., Hu, H., He, T., Bai, S., Chen, K., Wang, J., Fan, Y., Dang, K., Zhang, B., Wang, X., Chu, Y., and Lin, J. Qwen2.5-omni technical report. arXiv preprint arXiv:2503.20215, 2025a. · arXiv
  76. Xu, L., Zhao, Y., Wang, J., Wang, Y., Pi, B., Wang, C., Zhang, M., Gu, J., Li, X., Zhu, X., et al. Geosense: Evaluating identification and application of geometric principles in multimodal reasoning. arXiv preprint arXiv:2504.12597, 2025b. · arXiv
  77. Yue, X., Ni, Y., Zhang, K., Zheng, T., Liu, R., Zhang, G., Stevens, S., Jiang, D., Ren, W., Sun, Y., Wei, C., Yu, B., Yuan, R., Sun, R., Yin, M., Zheng, B., Yang, Z., Liu, Y., Huang, W., Sun, H., Su, Y., and Chen, W. Mmmu: A massive multi-discipline multimodal understanding and reasoning benchmark for expert agi. In Proceedings of the IEEE/CVF Conference on Com- puter Vision and Pattern Recognition, pp. 9556-9567, 2024. doi: 10.1109/CVPR52733.2024.00913. URL https://arxiv.org/abs/2311.16502. · DOI · Paper page
  78. Yue, X., Zheng, T., Ni, Y., Wang, Y., Zhang, K., Tong, S., Sun, Y., Yu, B., Zhang, G., Sun, H., et al. Mmmu-pro: A more robust multi-discipline multimodal understanding benchmark. In Proceedings of the Annual Meeting of the Association for Computational Linguistics, 2025. URL https://arxiv.org/abs/2409.02813. · arXiv
  79. Zhai, Y., Tong, S., Li, X., Cai, M., Qu, Q., Lee, Y. J., and Ma, Y. Investigating the catastrophic forgetting in multimodal large language models, 2023. URL https: //arxiv.org/abs/2309.10313.
  80. Zhang, H., Wu, W., Li, C., Shang, N., Xia, Y., Huang, Y., Zhang, Y., Dong, L., Zhang, Z., Wang, L., Tan, T., and Wei, F. Latent sketchpad: Sketching visual thoughts to elicit multimodal reasoning in mllms, 2025a. URL https://arxiv.org/abs/2510.24514. · arXiv
  81. Zhang, J., Khayatkhoei, M., Chhikara, P., and Ilievski, F. Mllms know where to look: Training-free perception of small visual details with multimodal llms. In Interna- tional Conference on Learning Representations, 2025b. URL https://arxiv.org/abs/2502.17422. · arXiv
  82. Zhang, X., Quan, Y., Shen, C., Yuan, X., Yan, S., Xie, L., Wang, W., Gu, C., Tang, H., and Ye, J. From redundancy to relevance: Information flow in lvlms across reason- ing tasks, 2024a. URL https://arxiv.org/abs/ 2406.06579.
  83. Zhang, Z., Zhang, A., Li, M., Zhao, H., Karypis, G., and Smola, A. Multimodal chain-of-thought reasoning in language models. Transactions on Machine Learning Research, 2024b. URL https://arxiv.org/abs/ 2302.00923.
  84. Zheng, Z., Yang, M., Hong, J., Zhao, C., Xu, G., Yang, L., Shen, C., and Yu, X. Deepeyes: Incentivizing "thinking with images" via reinforcement learning, 2025. URL https://arxiv.org/abs/2505.14362. · arXiv
  85. Zhu, J., Wang, W., Chen, Z., Liu, Z., Ye, S., Gu, L., Tian, H., Duan, Y., Su, W., Shao, J., et al. Internvl3: Exploring advanced training and test-time recipes for open-source multimodal models. arXiv preprint arXiv:2504.10479, 2025. · arXiv

Cited by

Sorted by citations

Authors
Gao, H., Huang, Z., Tang, J., Xu, L., Li, X., Li, H., Liu, Y., Lin, M., Yang, X., Hu, T., Wu, G., Bi, B., Chen, H., Huang, O., & Zhang, W.
ORCID
Venue
arXiv
Publisher
arXiv
Published
2025
ISSN
2331-8422 (ISSN-L)
DOI
10.48550/arxiv.2507.16863
arXiv
2507.16863
OpenAlex
W4414877033
Cited by
0
References
0
Type
posted content
Language
en
Open access
Licence
arxiv.org/licenses/nonexclusive-distrib/1.0/
Locations
  • arXiv (Cornell University) (submittedVersion)
  • arXiv (Cornell University)
Funders
Ministry of Education of the People's Republic of China
Topics
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Multimodal Machine Learning Applications
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Visual Attention and Saliency Detection
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
Keywords
visual reasoning, vision-language models, chain-of-thought, in-context learning, external tools, spatial reasoning, multimodal architectures

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review