MMMU: A Massive Multi-Discipline Multimodal Understanding and Reasoning Benchmark for Expert AGI
Open accessAbstract
We introduce MMMU: a new benchmark designed to evaluate multimodal models on massive multi-discipline tasks demanding college-level subject knowledge and deliberate reasoning. MMMU includes 11.5K meticulously collected multimodal questions from college exams, quizzes, and text-books, covering six core disciplines: Art & Design, Busi-ness, Science, Health & Medicine, Humanities & Social Science, and Tech & Engineering. These questions span 30 subjects and 183 subfields, comprising 30 highly het-erogeneous image types, such as charts, diagrams, maps, tables, music sheets, and chemical structures. Unlike existing benchmarks, MMMU focuses on advanced perception and reasoning with domain-specific knowledge, challenging models to perform tasks akin to those faced by experts. The evaluation of 28 open-source LMMs as well as the propri-etary GPT-4V(ision) and Gemini highlights the substantial challenges posed by MMMU. Even the advanced GPT-4V and Gemini Ultra only achieve accuracies of 56% and 59% respectively, indicating significant room for improvement. We believe MMMU will stimulate the community to build next-generation multimodal foundation models towards expert artificial general intelligence.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Open-source models such as Qwen2.5-VL-72B-Instruct and Llama-4-Maverick perform far worse on SPOT than proprietary models, despite matching them on existing multi-modal benchmarks like MMMU or MathVista.
“Surprisingly, open-source models such as Qwen2.5-VL-72B-Instruct and Llama-4-Maverick, which match proprietary models on existing multi-modal benchmarks like MMMU [49] or MathVista [50], perform far worse on SPOT.”
From When AI Co-Scientists Fail: SPOT-a Benchmark for Automated Verification of Scientific Research (2025) · cites this paper as [49]
Self-explanations are a prevailing approach for understanding LLM-generated responses and are widely used in analyzing language model behavior.
“Self-explanations are a prevailing approach on par with traditional model explanations like LIME saliency maps [71] for understanding LLM-generated responses [25, 32, 53] and are widely used in analyzing language model behavior [66, 99].”
From Thinking in Space: How Multimodal Large Language Models See, Remember, and Recall Spaces (2025) · cites this paper as [99]
Section 5, “Error Analysis and Future Work” (page unavailable), describes expert annotators examining GPT-4V predictions using their knowledge and gold explanations where available. Section 4.1 discusses extracting answers from responses that may contain reasoning steps. Neither passage establishes that model-generated self-explanations are a prevailing or widely used approach to analyzing language-model behavior (searched: self-explanation and model-generated explanations). (LLM, conf=0.72)
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks
Ren, S., He, K., Girshick, R., & Sun, J. (2017) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2016.2577031 · Cited by 56,035 · cited 1× · see citing works
“This work relies on pre-trained visual representations like Faster RCNN features [61] to minimize the training sample complexity.”
MethodMicrosoft COCO: Common Objects in Context
Lin, T.-Y., Maire, M., Belongie, S., Hays, J., Perona, P., Ramanan, D., Dollár, P., & Zitnick, C. L. (2014) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-10602-1_48 · Cited by 45,460 · see citing works
Learning Transferable Visual Models From Natural Language Supervision
Radford, A., Kim, J. W., Hallacy, C., Ramesh, A., Goh, G., Agarwal, S., Sastry, G., Askell, A., Mishkin, P., Clark, J., et al. (2021) · arXiv · doi:10.48550/arxiv.2103.00020 · Cited by 15,293 · cited 3× · see citing works
“Later on, CLIP [60], ALIGN [27], SimVLM [71], CoCa [78], Flamingo [2], BLIP-2 [32], and Fuyu [6] (inter alia) have been proposed to train visual representation using ViT [18] from scratch with massive amount of web data.”
BackgroundMethodLLaMA: Open and Efficient Foundation Language Models
Touvron, H., Lavril, T., Izacard, G., Martinet, X., Lachaux, M.-A., Lacroix, T., Rozière, B., Goyal, N., Hambro, E., Azhar, F., et al. (2023) · arXiv · doi:10.48550/arxiv.2302.13971 · Cited by 4,602 · cited 3× · see citing works
“Rapid advances in large language models (LLMs) [13, 54, 67] have sparked broad discussions on the controversial concept of artificial general intelligence (AGI), often used to describe AI systems that perform on par or surpass humans at most tasks [1, 7, 21, 29, 49, 52].”
BackgroundMethodVQA: Visual Question Answering
Antol, S., Agrawal, A., Lu, J., Mitchell, M., Batra, D., Zitnick, C. L., & Parikh, D. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.279 · Cited by 4,487 · see citing works
InstructBLIP: Towards General-purpose Vision-Language Models with Instruction TuningPreprint
Dai, W., Li, J., Li, D., Tiong, A., Zhao, J.-Q., Wang, W., Li, B. A., Fung, P., & Hoi, S. (2023) · Neural Information Processing Systems · doi:10.48550/arxiv.2305.06500 · Cited by 4,191 · cited 3× · see citing works
“Models such as LLaMA-Adapter [20, 81], mPlug-OWL [74, 75], SVIT [82], LRV-Instruction [39], and InstructBLIP [16] exemplify these developments.”
BackgroundMethodGQA: A New Dataset for Real-World Visual Reasoning and Compositional Question Answering
Hudson, D. A., & Manning, C. D. (2019) · Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2019.00686 · Cited by 3,703 · cited 1× · see citing works
“With the surge of multi-modal pre-training and instruction tuning, the prior single-task evaluation benchmarks like VQA [3, 23], OK-VQA [48], MSCOCO [36], GQA [26], etc., have become insufficient to holistically evaluate LMMs’ general multimodal perception and reasoning abilities.”
BackgroundLlama 2: Open Foundation and Fine-Tuned Chat Models
Touvron, H., Martin, L., Stone, K., Albert, P., Almahairi, A., Babaei, Y., Bashlykov, N., Batra, S., Bhargava, P., Bhosale, S., et al. (2023) · arXiv · doi:10.48550/arxiv.2307.09288 · Cited by 2,604 · see citing works
PaLM: Scaling Language Modeling with Pathways
Chowdhery, A., Narang, S., Devlin, J., Bosma, M., Mishra, G., Roberts, A., Barham, P., Chung, H. W., Sutton, C., Gehrmann, S., et al. (2022) · arXiv · doi:10.48550/arxiv.2204.02311 · Cited by 2,590 · cited 1× · see citing works
“Rapid advances in large language models (LLMs) [13, 54, 67] have sparked broad discussions on the controversial concept of artificial general intelligence (AGI), often used to describe AI systems that perform on par or surpass humans at most tasks [1, 7, 21, 29, 49, 52].”
BackgroundLiu, H., Li, C., Wu, Q., & Lee, Y. J. (2023) · Advances in Neural Information Processing Systems 36 · doi:10.52202/075280-1516 · Cited by 2,428 · see citing works
LXMERT: Learning Cross-Modality Encoder Representations from Transformers
Tan, H., & Bansal, M. (2019) · Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP) · doi:10.18653/v1/d19-1514 · Cited by 2,319 · cited 1× · see citing works
“LXMERT [64], UNITER [10], VinVL [80], Oscar [34], VilBert [45], and VLP [86] are among the earliest work to train universal vision-language models to tackle many multimodal tasks.”
BackgroundMaking the V in VQA Matter: Elevating the Role of Image Understanding in Visual Question Answering
Goyal, Y., Khot, T., Summers-Stay, D., Batra, D., & Parikh, D. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.670 · Cited by 2,287 · see citing works
ViLBERT: Pretraining Task-Agnostic Visiolinguistic Representations for Vision-and-Language Tasks
Lu, J., Batra, D., Parikh, D., & Lee, S. (2019) · arXiv · doi:10.48550/arxiv.1908.02265 · Cited by 2,109 · cited 1× · see citing works
“LXMERT [64], UNITER [10], VinVL [80], Oscar [34], VilBert [45], and VLP [86] are among the earliest work to train universal vision-language models to tackle many multimodal tasks.”
BackgroundUNITER: UNiversal Image-TExt Representation Learning
Chen, Y.-C., Li, L., Yu, L., El Kholy, A., Ahmed, F., Gan, Z., Cheng, Y., & Liu, J. (2020) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-58577-8_7 · Cited by 2,055 · cited 1× · see citing works
“LXMERT [64], UNITER [10], VinVL [80], Oscar [34], VilBert [45], and VLP [86] are among the earliest work to train universal vision-language models to tackle many multimodal tasks.”
BackgroundScaling Up Visual and Vision-Language Representation Learning With Noisy Text Supervision
Jia, C., Yang, Y., Xia, Y., Chen, Y.-T., Parekh, Z., Pham, H., Le, Q. V., Sung, Y., Li, Z., & Duerig, T. (2021) · arXiv · doi:10.48550/arxiv.2102.05918 · Cited by 1,977 · cited 1× · see citing works
“Later on, CLIP [60], ALIGN [27], SimVLM [71], CoCa [78], Flamingo [2], BLIP-2 [32], and Fuyu [6] (inter alia) have been proposed to train visual representation using ViT [18] from scratch with massive amount of web data.”
MethodCoCa: Contrastive Captioners are Image-Text Foundation ModelsPreprint
Yu, J.-H., Wang, Z., Vasudevan, V., Yeung, L., Seyedhosseini, M., & Wu, Y.-H. (2022) · Trans. Mach. Learn. Res. · doi:10.48550/arxiv.2205.01917 · Cited by 1,899 · cited 1× · see citing works
“Later on, CLIP [60], ALIGN [27], SimVLM [71], CoCa [78], Flamingo [2], BLIP-2 [32], and Fuyu [6] (inter alia) have been proposed to train visual representation using ViT [18] from scratch with massive amount of web data.”
MethodImproved Baselines with Visual Instruction Tuning
Liu, H., Li, C., Li, Y., & Lee, Y. J. (2024) · 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52733.2024.02484 · Cited by 1,748 · cited 3× · see citing works
“5 [40] linearly projects the visual embedding into word embedding space of Vicuna [12], thus equipping the LLM with visual abilities.”
BackgroundMethodOscar: Object-Semantics Aligned Pre-training for Vision-Language Tasks
Li, X., Yin, X. P., Li, C., Zhang, P., Hu, X., Zhang, L., Wang, L., Hu, H., Li, D., Wei, F., et al. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-58577-8_8 · Cited by 1,736 · see citing works
Flamingo: a Visual Language Model for Few-Shot Learning
Alayrac, J.-B., Donahue, J., Luc, P., Miech, A., Barr, I., Hasson, Y., Lenc, K., Mensch, A., Millican, K., Reynolds, M., et al. (2022) · arXiv · doi:10.48550/arxiv.2204.14198 · Cited by 1,555 · cited 4× · see citing works
“Later on, CLIP [60], ALIGN [27], SimVLM [71], CoCa [78], Flamingo [2], BLIP-2 [32], and Fuyu [6] (inter alia) have been proposed to train visual representation using ViT [18] from scratch with massive amount of web data.”
BackgroundMethodInfluentialSparks of Artificial General Intelligence: Early experiments with GPT-4
Bubeck, S., Chandrasekaran, V., Eldan, R., Gehrke, J., Horvitz, E., Kamar, E., Lee, P., Lee, Y. T., Li, Y., Lundberg, S., et al. (2023) · arXiv · doi:10.48550/arxiv.2303.12712 · Cited by 1,526 · cited 1× · see citing works
“Rapid advances in large language models (LLMs) [13, 54, 67] have sparked broad discussions on the controversial concept of artificial general intelligence (AGI), often used to describe AI systems that perform on par or surpass humans at most tasks [1, 7, 21, 29, 49, 52].”
BackgroundScaling Instruction-Finetuned Language Models
Chung, H. W., Hou, L., Longpre, S., Zoph, B., Tay, Y., Fedus, W., Li, Y., Wang, X., Dehghani, M., Brahma, S., et al. (2022) · arXiv · doi:10.48550/arxiv.2210.11416 · Cited by 1,441 · cited 2× · see citing works
“(iii) BLIP-2 [32] introduces light-weight learnable visual queries to bridge the frozen CLIP ViT [60] and FLAN-T5 [14].”
BackgroundMethodmPLUG-Owl: Modularization Empowers Large Language Models with MultimodalityPreprint
Ye, Q.-H., Xu, H., Xu, G., Ye, J., Yan, M., Zhou, Y., Wang, J., Hu, A., Shi, P.-C., Shi, Y., et al. (2023) · arXiv.org · doi:10.48550/arxiv.2304.14178 · Cited by 1,318 · see citing works
ReferItGame: Referring to Objects in Photographs of Natural Scenes
Kazemzadeh, S., Ordóñez, V., Matten, M., & Berg, T. L. (2014) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.3115/v1/d14-1086 · Cited by 1,082 · cited 1× · see citing works
“For instance, CogVLM [70] achieves 85 % on VQA-v2 [23], 92 % on ScienceQA-IMG [46], and 93 % on RefCOCO [28].”
The Dawn of LMMs: Preliminary Explorations with GPT-4V(ision)Preprint
Yang, Z.-Y., Li, L., Lin, K., Wang, J., Lin, C.-C., Liu, Z., & Wang, L.-J. (2023) · arXiv.org · doi:10.48550/arxiv.2309.17421 · Cited by 1,036 · cited 1× · see citing works
“Meanwhile, large multi-modal models (LMMs) that can understand both text and images have been making a major stride towards more general AI [9, 16, 32, 40, 73].”
BackgroundSelf-citationGemini: A Family of Highly Capable Multimodal Models
Gemini Team, Anil, R., Borgeaud, S., Alayrac, J.-B., Yu, J., Soricut, R., Schalkwyk, J., Dai, A. M., Hauth, A., Millican, K., et al. (2023) · arXiv · doi:10.48550/arxiv.2312.11805 · Cited by 990 · see citing works
Self-citationVinVL: Revisiting Visual Representations in Vision-Language Models
Zhang, P., Li, X., Hu, X., Yang, J., Zhang, L., Wang, L., Choi, Y., & Gao, J. (2021) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr46437.2021.00553 · Cited by 965 · cited 1× · see citing works
“LXMERT [64], UNITER [10], VinVL [80], Oscar [34], VilBert [45], and VLP [86] are among the earliest work to train universal vision-language models to tackle many multimodal tasks.”
BackgroundUnified Vision-Language Pre-Training for Image Captioning and VQA
Zhou, L., Palangi, H., Zhang, L., Hu, H., Corso, J., & Gao, J. (2020) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v34i07.7005 · Cited by 864 · see citing works
Towards VQA Models That Can Read
Singh, A., Natarajan, V., Shah, M., Jiang, Y. L., Chen, X., Batra, D., Parikh, D., & Rohrbach, M. (2019) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2019.00851 · Cited by 777 · see citing works
LLaMA-Adapter V2: Parameter-Efficient Visual Instruction ModelPreprint
Gao, P., Han, J., Zhang, R., Lin, Z.-Y., Geng, S., Zhou, A., Zhang, W., Lu, P., He, C.-H., Yue, X., et al. (2023) · arXiv.org · doi:10.48550/arxiv.2304.15010 · Cited by 777 · cited 2× · see citing works
“Models such as LLaMA-Adapter [20, 81], mPlug-OWL [74, 75], SVIT [82], LRV-Instruction [39], and InstructBLIP [16] exemplify these developments.”
MethodSelf-citationOK-VQA: A Visual Question Answering Benchmark Requiring External Knowledge
Marino, K., Rastegari, M., Farhadi, A., & Mottaghi, R. (2019) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2019.00331 · Cited by 749 · see citing works
Intern VL: Scaling up Vision Foundation Models and Aligning for Generic Visual-Linguistic Tasks
Chen, Z., Wu, J., Wang, W., Su, W., Chen, G., Xing, S., Zhong, M., Zhang, Q., Zhu, X., Lu, L., et al. (2024) · 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52733.2024.02283 · Cited by 734 · see citing works
Liu, H., Li, C., Wu, Q., & Lee, Y. J. (2023) · arXiv · doi:10.48550/arxiv.2304.08485 · Cited by 682 · cited 1× · see citing works
“Inspired by open-source instruction-tuned LLMs like FLAN-T5 [14] and Vi-cuna [12], models like LLaVA [40, 41] and MiniGPT-4 [87] utilized open-source resources, to improve the instruction-following capabilities of LMMs.”
MethodOpenFlamingo: An Open-Source Framework for Training Large Autoregressive Vision-Language ModelsPreprint
Awadalla, A., Gao, I., Gardner, J., Hessel, J., Hanafy, Y., Zhu, W., Marathe, K., Bitton, Y., Gadre, S., Sagawa, S., et al. (2023) · arXiv.org · doi:10.48550/arxiv.2308.01390 · Cited by 674 · cited 2× · see citing works
“(vi) As an open-source alternative to Flamingo [2], OpenFlamingo [4] has close performance on most vision-language tasks.”
BackgroundMethodMMBench: Is Your Multi-modal Model an All-Around Player?
Liu, Y., Duan, H., Zhang, Y., Li, B., Zhang, S., Zhao, W., Yuan, Y., Wang, J., He, C., Liu, Z., et al. (2025) · Lecture Notes in Computer Science · doi:10.1007/978-3-031-72658-3_13 · Cited by 668 · see citing works
Self-citationEvaluating Object Hallucination in Large Vision-Language Models
Li, Y., Du, Y., Zhou, K., Wang, J., Xin, Z., & Wen, J.-R. (2023) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2023.emnlp-main.20 · Cited by 580 · see citing works
SimVLM: Simple Visual Language Model Pretraining with Weak SupervisionPreprint
Wang, Z., Yu, J., Yu, A. W., Dai, Z., Tsvetkov, Y., & Cao, Y. J. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2108.10904 · Cited by 517 · cited 1× · see citing works
“Later on, CLIP [60], ALIGN [27], SimVLM [71], CoCa [78], Flamingo [2], BLIP-2 [32], and Fuyu [6] (inter alia) have been proposed to train visual representation using ViT [18] from scratch with massive amount of web data.”
MethodMiniGPT-4: Enhancing Vision-Language Understanding with Advanced Large Language Models
Zhu, D., Chen, J., Shen, X., Li, X., & Elhoseiny, M. (2023) · arXiv · doi:10.48550/arxiv.2304.10592 · Cited by 469 · cited 2× · see citing works
“(xi) MiniGPT-4 [87] is built upon Vicuna [12] and designs a linear modality projection layer for visual understanding abilities.”
BackgroundMethod(2021) · International Conference on Learning Representations · Cited by 435 · see citing works
Learn to Explain: Multimodal Reasoning Via Thought Chains for Science Question Answering
Lu, P., Mishra, S. R., Xia, T., Qiu, L., Chang, K.-W., Zhu, S. C., Tafjord, O., Clark, P., & Kalyan, A. (2022) · neural information processing systems · doi:10.52202/068431-0182 · Cited by 420 · see citing works
Dong, X.-W., Zhang, P., Zang, Y., Cao, Y.-H., Wang, B., Ouyang, L.-K., Wei, X.-L., Zhang, S.-Y., Duan, H., Cao, M., et al. (2024) · arXiv.org · doi:10.48550/arxiv.2401.16420 · Cited by 401 · see citing works
Lin, Z.-Y., Liu, C., Zhang, R., Gao, P., Qiu, L.-T., Xiao, H., Qiu, H., Lin, C., Shao, W., Chen, K.-Q., et al. (2023) · arXiv.org · doi:10.48550/arxiv.2311.07575 · Cited by 316 · see citing works
Otter: A Multi-Modal Model With In-Context Instruction Tuning
Li, B., Zhang, Y., Chen, L., Wang, J., Pu, F., Cahyono, J. A., Yang, J., Li, C., & Liu, Z. (2025) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2025.3571946 · Cited by 253 · cited 2× · see citing works
“(x) Otter [31] is fine-tuned with diverse instruction-tuning data and able to perform in-context learning.”
BackgroundMethodCogVLM: Visual Expert for Pretrained Language Models
Wang, W., Lv, Q., Yu, W., Hong, W., Ji, J., Wang, Y., Ji, J., Yang, Z., Zhao, L., Song, X., et al. (2024) · neural information processing systems · doi:10.52202/079017-3860 · Cited by 217 · see citing works
Self-citationCogVLM: Visual Expert for Pretrained Language Models
Wang, W., Lv, Q., Yu, W., Hong, W., Qi, J., Wang, Y., Ji, J., Yang, Z., Zhao, L., Song, X., et al. (2023) · arXiv · doi:10.48550/arxiv.2311.03079 · Cited by 217 · see citing works
Self-citationAGIEval: A Human-Centric Benchmark for Evaluating Foundation Models
Zhong, W., Cui, R., Guo, Y., Liang, Y., Lu, S., Wang, Y., Saied, A., Chen, W., & Duan, N. (2024) · Findings of the Association for Computational Linguistics: NAACL 2024 · doi:10.18653/v1/2024.findings-naacl.149 · Cited by 211 · see citing works
Self-citationAGIEval: A Human-Centric Benchmark for Evaluating Foundation Models
Zhong, W., Cui, R., Guo, Y., Liang, Y., Lu, S., Wang, Y., Saied, A., Chen, W., & Duan, N. (2023) · arXiv · doi:10.48550/arxiv.2304.06364 · Cited by 211 · cited 1× · see citing works
“This strategy has been successfully adopted in benchmarks such as MMLU [24] and AGIEval [85], but only text-based questions are considered, while human experts are capable of solving multimodal problems.”
MethodSelf-citationLLaMA-Adapter: Efficient Fine-tuning of Language Models with Zero-init AttentionPreprint
Zhang, R., Han, J., Liu, C., Gao, P., Zhou, A., Hu, X., Yan, S., Lu, P., Li, H., & Qiao, Y. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2303.16199 · Cited by 202 · cited 1× · see citing works
“Models such as LLaMA-Adapter [20, 81], mPlug-OWL [74, 75], SVIT [82], LRV-Instruction [39], and InstructBLIP [16] exemplify these developments.”
MethodQwen-VL: A Versatile Vision-Language Model for Understanding, Localization, Text Reading, and Beyond
Bai, J., Bai, S., Yang, S., Wang, S., Tan, S., Wang, P., Lin, J., Zhou, C., & Zhou, J. (2023) · arXiv · doi:10.48550/arxiv.2308.12966 · Cited by 193 · cited 1× · see citing works
“(ix) Qwen-VL [5] introduces a set of trainable query embeddings and single-layer cross-attention module to bridge the modalities, supporting interleaved image-text input.”
BackgroundOCRBench: on the hidden mystery of OCR in large multimodal models
Liu, Y., Li, Z., Huang, M., Yang, B., Yu, W., Li, C., Yin, X.-C., Liu, C.-L., Jin, L., & Bai, X. (2024) · Science China Information Sciences · doi:10.1007/s11432-024-4235-6 · Cited by 167 · see citing works
Self-citationOpenAGI: When LLM Meets Domain Experts
Ge, Y., Hua, W., Mei, K., Li, Z., Tan, J., Xu, S., Li, Z., & Zhang, Y. (2023) · neural information processing systems · doi:10.52202/075280-0242 · Cited by 165 · see citing works
LVLM-EHub: A Comprehensive Evaluation Benchmark for Large Vision-Language Models
Xu, P., Shao, W., Zhang, K., Gao, P., Liu, S., Lei, M., Meng, F., Huang, S., Qiao, Y., & Luo, P. (2024) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2024.3507000 · Cited by 150 · cited 1× · see citing works
“More holistic evaluations have been conducted as well, such as LAMM [76], LVLM-eHub [72], SEED [30], MMBench [43], and MM-Vet [79].”
MethodSelf-citationGenerative Multimodal Models are In-Context Learners
Sun, Q., Cui, Y., Zhang, X., Zhang, F., Yu, Q., Wang, Y., Rao, Y., Liu, J., Huang, T., & Wang, X. (2024) · 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52733.2024.01365 · Cited by 147 · see citing works
Holistic Analysis of Hallucination in GPT-4V(ision): Bias and Interference ChallengesPreprint
Cui, C.-H., Zhou, Y.-Y., Yang, X.-Y., Wu, S., Zhang, L.-J., Zou, J., & Yao, H. (2023) · arXiv.org · doi:10.48550/arxiv.2311.03287 · Cited by 145 · cited 2× · see citing works
“…when the model accurately processes and understands the given information but fails in elementary visual interpretation, such as misjudging the sequence described as “from left to right, top to bottom. text, prioritizing textual information over visual inputs, a trend noted in recent studies [15].”
BackgroundPosition: Levels of AGI for Operationalizing Progress on the Path to AGIPreprint
Morris, M. R., Sohl-Dickstein, J. N., Fiedel, N., Warkentin, T., Dafoe, A., Faust, A., Farabet, C., & Legg, S. (2023) · International Conference on Machine Learning · doi:10.48550/arxiv.2311.02462 · Cited by 135 · see citing works
Kosmos-2: Grounding Multimodal Large Language Models to the WorldPreprint
Peng, Z., Wang, W., Dong, L., Hao, Y., Huang, S., Ma, S., & Wei, F. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2306.14824 · Cited by 129 · cited 1× · see citing works
“(i) Kos-mos2 [57] is pre-trained to ground fine-grained visual objects with texts and to follow instructions.”
BackgroundEvaluation and Analysis of Hallucination in Large Vision-Language ModelsPreprint
Wang, J., Zhou, Y., Xu, G., Shi, P.-C., Zhao, C., Xu, H., Ye, Q.-H., Yan, M.-S., Zhang, J., Zhu, J.-H., et al. (2023) · arXiv.org · doi:10.48550/arxiv.2308.15126 · Cited by 118 · see citing works
On Evaluating Adversarial Robustness of Large Vision-Language Models
Zhao, Y., Pang, T., Du, C., Yang, X., Li, C., Cheung, N. M., & Lin, M. (2023) · neural information processing systems · doi:10.52202/075280-2355 · Cited by 96 · see citing works
MM-Vet: Evaluating Large Multimodal Models for Integrated CapabilitiesPreprint
Yu, W., Yang, Z., Li, L., Wang, J., Lin, K., Liu, Z., Wang, X., & Wang, L. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2308.02490 · Cited by 74 · cited 2× · see citing works
“More holistic evaluations have been conducted as well, such as LAMM [76], LVLM-eHub [72], SEED [30], MMBench [43], and MM-Vet [79].”
BackgroundMethodSelf-citationLAMM: Language-Assisted Multi-Modal Instruction-Tuning Dataset, Framework, and Benchmark
Yin, Z., Wang, J., Cao, J., Shi, Z., Liu, D., Li, M., Huang, X., Wang, Z., Sheng, L., Bai, L., et al. (2023) · neural information processing systems · doi:10.52202/075280-1158 · Cited by 59 · see citing works
Yi: Open Foundation Models by 01.AI
AI, 0., :, Young, A., Chen, B., Li, C., Huang, C., Zhang, G., Zhang, G., Wang, G., Li, H., et al. (2024) · arXiv · doi:10.48550/arxiv.2403.04652 · Cited by 50 · see citing works
Self-citationPaLI-X: On Scaling up a Multilingual Vision and Language ModelPreprint
Chen, X., Djolonga, J., Padlewski, P., Mustafa, B., Changpinyo, S., Wu, J., Riquelme, C., Goodman, S., Wang, X., Tay, Y., et al. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2305.18565 · Cited by 42 · cited 1× · see citing works
“Meanwhile, large multi-modal models (LMMs) that can understand both text and images have been making a major stride towards more general AI [9, 16, 32, 40, 73].”
BackgroundmPLUG-Owl2: Revolutionizing Multi-modal Large Language Model with Modality CollaborationPreprint
Ye, Q., Xu, H., Ye, J., Yan, M., Hu, A., Liu, H., Qian, Q., Zhang, J., Huang, F., & Zhou, J. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2311.04257 · Cited by 28 · cited 2× · see citing works
“Models such as LLaMA-Adapter [20, 81], mPlug-OWL [74, 75], SVIT [82], LRV-Instruction [39], and InstructBLIP [16] exemplify these developments.”
BackgroundMethodSVIT: Scaling up Visual Instruction TuningPreprint
Zhao, B., Wu, B., He, M., & Huang, T. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2307.04087 · Cited by 18 · cited 1× · see citing works
“Models such as LLaMA-Adapter [20, 81], mPlug-OWL [74, 75], SVIT [82], LRV-Instruction [39], and InstructBLIP [16] exemplify these developments.”
MethodM$^3$IT: A Large-Scale Dataset towards Multi-Modal Multilingual Instruction TuningPreprint
Li, L., Yin, Y., Li, S., Chen, L., Wang, P., Ren, S., Li, M., Yang, Y., Xu, J., Sun, X., et al. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2306.04387 · Cited by 14 · cited 1× · see citing works
“This area has been explored in depth by models such as Flamingo [2] and OpenFlamingo [4], Otter [31], M3IT [33], MetaVL [51], Sparkles [25], and MMICL [83].”
MethodGAIA: a benchmark for General AI Assistants
Mialon, G., Fourrier, C., Swift, C., Wolf, T., LeCun, Y., & Scialom, T. (2023) · arXiv · doi:10.48550/arxiv.2311.12983 · Cited by 12 · cited 2× · see citing works
“In line with the motivation of our study, concurrently, GAIA [49] introduces 466 questions that test fundamental abilities of models such as reasoning, multimodality handling, or tool use.”
BackgroundMetaVL: Transferring In-Context Learning Ability From Language Models to Vision-Language Models
Monajatipoor, M., Li, L. H., Rouhsedaghat, M., Yang, L. F., & Chang, K.-W. (2023) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2023.acl-short.43 · Cited by 9 · see citing works
Liu, F., Guan, T., Li, Z., Chen, L., Yacoob, Y., Manocha, D., & Zhou, T. (2023) · doi:10.13140/rg.2.2.19065.65121 · Cited by 9 · see citing works
Sparkles: Unlocking Chats Across Multiple Images for Multimodal Instruction-Following ModelsPreprint
Huang, Y., Meng, Z., Liu, F., Su, Y., Collier, N., & Lu, Y. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2308.16463 · Cited by 4 · cited 1× · see citing works
“This area has been explored in depth by models such as Flamingo [2] and OpenFlamingo [4], Otter [31], M3IT [33], MetaVL [51], Sparkles [25], and MMICL [83].”
MethodSelf-citationSEED-Bench: Benchmarking Multimodal LLMs with Generative Comprehension
Li, B., Wang, R., Wang, G., Ge, Y., Ge, Y., & Shan, Y. (2023) · arXiv · doi:10.48550/arxiv.2307.16125 · cited 2×
“More holistic evaluations have been conducted as well, such as LAMM [76], LVLM-eHub [72], SEED [30], MMBench [43], and MM-Vet [79].”
BackgroundMethodMMICL: Empowering Vision-language Model with Multi-Modal In-Context Learning
Zhao, H., Cai, Z., Si, S., Ma, X., An, K., Chen, L., Liu, Z., Wang, S., Han, W., & Chang, B. (2023) · arXiv · doi:10.48550/arxiv.2309.07915 · cited 1×
“This area has been explored in depth by models such as Flamingo [2] and OpenFlamingo [4], Otter [31], M3IT [33], MetaVL [51], Sparkles [25], and MMICL [83].”
Method
Flagged first, then by citations
10 of the 80 references couldn’t be matched to a known paper.
Research works printed but not linked: 8
- Zhe Chen, Jiannan Wu, Wenhai Wang, Weijie Su, Guo Chen, Sen Xing, Zhong Muyan, Qinglong Zhang, Xizhou Zhu, Lewei Lu, et al. Internvl: Scaling up vision foundation mod- els and aligning for generic visual-linguistic tasks. arXiv preprint arXiv:2312.14238, 2023. 6, 15, 16, 17, 18, 19, 20, 21 · arXiv
- Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Syl- vain Gelly, Jakob Uszkoreit, and Neil Houlsby. An image is worth 16x16 words: Transformers for image recognition at scale. In International Conference on Learning Representa- tions, 2021. 3
- Ehsan Latif, Gengchen Mai, Matthew Nyaaba, Xuansheng Wu, Ninghao Liu, Guoyu Lu, Sheng Li, Tianming Liu, and Xiaoming Zhai. Artificial general intelligence (agi) for edu- cation. arXiv preprint arXiv:2304.12479, 2023. 1 · arXiv
- Junnan Li, Dongxu Li, Silvio Savarese, and Steven Hoi. Blip-2: Bootstrapping language-image pre-training with frozen image encoders and large language models. Inter- national Conference on Machine Learning, 2023. 2, 3, 5, 6, 7, 15, 16, 17, 18, 19, 20, 21
- Ji Lin, Hongxu Yin, Wei Ping, Yao Lu, Pavlo Molchanov, Andrew Tao, Huizi Mao, Jan Kautz, Mohammad Shoeybi, and Song Han. Vila: On pre-training for visual language models. arXiv preprint arXiv:2312.07533, 2023. 6, 15, 16, 17, 18, 19, 20, 21 · arXiv
- Fuxiao Liu, Kevin Lin, Linjie Li, Jianfeng Wang, Yaser Yacoob, and Lijuan Wang. Aligning large multi-modal model with robust instruction tuning. arXiv preprint arXiv:2306.14565, 2023. 3 · arXiv
- Pan Lu, Hritik Bansal, Tony Xia, Jiacheng Liu, Chunyuan Li, Hannaneh Hajishirzi, Hao Cheng, Kai-Wei Chang, Michel Galley, and Jianfeng Gao. Mathvista: Evaluating mathemat- ical reasoning of foundation models in visual contexts. arXiv preprint arXiv:2310.02255, 2023. 3 · arXiv
- OpenAI. Gpt-4 technical report. arXiv preprint arXiv:2303.08774, 2023. 1, 6, 15, 16, 17, 18, 19, 20, 21 · arXiv
Cited by
Sorted by citations
How far are we to GPT-4V? Closing the gap to commercial multimodal models with open-source suites
Chen, Z., Wang, W., Tian, H., Ye, S., Gao, Z., Cui, E., Tong, W., Hu, K., Luo, J., Ma, Z., et al. (2024) · Science China Information Sciences · doi:10.1007/s11432-024-4231-5 · Cited by 437 · see citing works
Video-MME: The First-Ever Comprehensive Evaluation Benchmark of Multi-modal LLMs in Video Analysis
Fu, C., Dai, Y., Luo, Y., Li, L., Ren, S., Zhang, R., Wang, Z., Zhou, C., Shen, Y., Zhang, M., et al. (2025) · 2025 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52734.2025.02245 · Cited by 291 · see citing works
Chen, L., Zhao, H., Liu, T., Bai, S., Lin, J., Zhou, C., & Chang, B. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-73004-7_2 · Cited by 156 · see citing works
Multimodal generative AI for medical image interpretation
Rao, V. M., Hla, M., Moor, M., Adithan, S., Kwak, S., Topol, E. J., & Rajpurkar, P. (2025) · Nature · doi:10.1038/s41586-025-08675-y · Cited by 128 · see citing works
Generative Multimodal Models are In-Context Learners
Sun, Q., Cui, Y., Zhang, X., Zhang, F., Yu, Q., Wang, Y., Rao, Y., Liu, J., Huang, T., & Wang, X. (2024) · 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52733.2024.01365 · Cited by 147 · see citing works
InjecAgent: Benchmarking Indirect Prompt Injections in Tool-Integrated Large Language Model Agents
Zhan, Q., Liang, Z., Ying, Z., & Kang, D. (2024) · Findings of the Association for Computational Linguistics: ACL 2024 · doi:10.18653/v1/2024.findings-acl.624 · Cited by 147 · see citing works
Foundation models for materials discovery – current state and future directions
Pyzer‐Knapp, E. O., Manica, M., Staar, P., Morin, L., Ruch, P., Laino, T., Smith, J. R., & Curioni, A. (2025) · npj Computational Materials · doi:10.1038/s41524-025-01538-0 · Cited by 120 · see citing works
LVLM-EHub: A Comprehensive Evaluation Benchmark for Large Vision-Language Models
Xu, P., Shao, W., Zhang, K., Gao, P., Liu, S., Lei, M., Meng, F., Huang, S., Qiao, Y., & Luo, P. (2024) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2024.3507000 · Cited by 150 · see citing works
Self-citationThinking in Space: How Multimodal Large Language Models See, Remember, and Recall Spaces
Yang, J., Yang, S., Gupta, A. W., Han, R., Fei-Fei, L., & Xie, S. (2025) · 2025 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52734.2025.00994 · Cited by 120 · see citing works
“For MCA tasks, we follow standard practice [24, 31, 99] by using Accuracy ( ACC ), based on exact matching (with possible fuzzy matching), as the primary metric.”
The Revolution of Multimodal Large Language Models: A Survey
Caffagni, D., Cocchi, F., Barsellotti, L., Moratelli, N., Sarto, S., Baraldi, L., Baraldi, L., Cornia, M., & Cucchiara, R. (2024) · Findings of the Association for Computational Linguistics ACL 2024 · doi:10.18653/v1/2024.findings-acl.807 · Cited by 98 · see citing works
“Similarly, MiniGPT-5 (Zheng et al., 2023) includes the reconstruction loss of diffusion models in addition to the alignment loss of GILL.”
MATHVERSE: Does Your Multi-modal LLM Truly See the Diagrams in Visual Math Problems?
Zhang, R., Jiang, D., Zhang, Y., Lin, H., Guo, Z., Qiu, P., Zhou, A., Lu, P., Chang, K., Qiao, Y., et al. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-73242-3_10 · Cited by 77 · see citing works
Self-citationBLINK: Multimodal Large Language Models Can See but Not Perceive
Fu, X., Hu, Y., Li, B., Feng, Y., Wang, H., Lin, X., Roth, D., Smith, N. A., Ma, W.-C., & Krishna, R. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-73337-6_9 · Cited by 101 · see citing works
Large language models for life cycle assessments: Opportunities, challenges, and risks
Preuss, N., Alshehri, A. S., & You, F. (2024) · Journal of Cleaner Production · doi:10.1016/j.jclepro.2024.142824 · Cited by 71 · see citing works
Honeybee: Locality-Enhanced Projector for Multimodal LLM
Cha, J., Kang, W., Mun, J., & Roh, B. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.01311 · Cited by 88 · see citing works
VisionZip: Longer is Better but Not Necessary in Vision Language Models
Yang, S., Chen, Y., Tian, Z., Wang, C., Li, J., Yu, B., & Jia, J. (2025) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52734.2025.01843 · Cited by 93 · see citing works
Self-citationLi, Z., Wu, X., Du, H., Liu, F., Nghiem, H. T., & Shi, G. (2025) · IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw67362.2025.00147 · Cited by 72 · see citing works
MM1: Methods, Analysis and Insights from Multimodal LLM Pre-training
McKinzie, B., Gan, Z., Fauconnier, J.-P., Dodge, S., Zhang, B., Dufter, P., Shah, D., Du, X., Peng, F., Belyi, A., et al. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-73397-0_18 · Cited by 120 · see citing works
Janus: Decoupling Visual Encoding for Unified Multimodal Understanding and Generation
Wu, C., Chen, X., Wu, Z., Ma, Y., Liu, X., Pan, Z., Wen, L., Xie, Z., Yu, X., Ruan, C., et al. (2025) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52734.2025.01210 · Cited by 74 · see citing works
Efficient GPT-4V level multimodal large language model for deployment on edge devices
Yao, Y., Yu, T., Zhang, A., Wang, C., Cui, J., Zhu, H., Cai, T., Chen, C., Li, H., Zhao, W., et al. (2025) · Nature Communications · doi:10.1038/s41467-025-61040-5 · Cited by 50 · see citing works
Doris, A. C., Grandi, D., Tomich, R., Alam, M. F., Ataei, M., Cheong, H., & Ahmed, F. (2024) · Journal of Computing and Information Science in Engineering · doi:10.1115/1.4067333 · Cited by 53 · see citing works
NVILA: Efficient Frontier Visual Language Models
Liu, Z., Zhu, L., Shi, B., Zhang, Z., Lou, Y., Yang, S. W., Xi, H., Cao, S., Gu, Y., Li, D., et al. (2025) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52734.2025.00390 · Cited by 73 · see citing works
Mini-InternVL: a flexible-transfer pocket multi-modal model with 5% parameters and 90% performance
Gao, Z., Chen, Z., Cui, E., Ren, Y., Wang, W., Zhu, J., Tian, H., He, J., He, J., Zhu, X., et al. (2024) · Visual Intelligence · doi:10.1007/s44267-024-00067-6 · Cited by 50 · see citing works
TokenPacker: Efficient Visual Projector for Multimodal LLM
Li, W., Yuan, Y., Liu, J., Tang, D., Wang, S., Qin, J., Zhu, J., & Zhang, L. (2025) · International Journal of Computer Vision · doi:10.1007/s11263-025-02491-7 · Cited by 63 · see citing works
Vision Language Models are blind
Rahmanzadehgervi, P., Bolton, L., Taesiri, M. R., & Nguyen, A. T. (2025) · Lecture Notes in Computer Science · doi:10.1007/978-981-96-0917-8_17 · Cited by 54 · see citing works
Good at captioning, bad at counting: Benchmarking GPT-4V on Earth observation data
Zhang, C., & Wang, S. (2024) · IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw63382.2024.00780 · Cited by 40 · see citing works
RoboSpatial: Teaching Spatial Understanding to 2D and 3D Vision-Language Models for Robotics
Song, C. H., Blukis, V., Tremblay, J., Tyree, S., Su, Y., & Birchfield, S. T. (2025) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52734.2025.01470 · Cited by 37 · see citing works
Self-citationTowards Open-Ended Visual Quality Comparison
Wu, H., Zhu, H., Zhang, Z., Zhang, E., Chen, Z., Liao, L., Li, C., Wang, A., Sun, W., Yan, Q., et al. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-72646-0_21 · Cited by 48 · see citing works
Putting ChatGPT vision (GPT-4V) to the test: risk perception in traffic images
Driessen, T., Dodou, D., Bazilinskyy, P., & de Winter, J. (2024) · Royal Society Open Science · doi:10.1098/rsos.231676 · Cited by 36 · see citing works
Large language models for batteries
Zuo, W., Zheng, H., He, T., Vishwanath, V., Chan, M. K. Y., Stevens, R. L., Amine, K., & Xu, G. (2025) · Joule · doi:10.1016/j.joule.2025.102037 · Cited by 32 · see citing works
From System 1 to System 2: A Survey of Reasoning Large Language Models
Zhang, D., Li, Z., Zhang, M.-L., Zhang, J., Liu, Z., Yao, Y., Xu, H., Zheng, J., Chen, X., Zhang, Y., et al. (2025) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2025.3637037 · Cited by 172 · see citing works
A Survey on Efficient Vision‐Language Models
Shinde, G., Ravi, A., Dey, E., Sakib, S., Rampure, M., & Roy, N. (2025) · Wiley Interdisciplinary Reviews Data Mining and Knowledge Discovery · doi:10.1002/widm.70036 · Cited by 33 · see citing works
TokenFlow: Unified Image Tokenizer for Multimodal Understanding and Generation
Qu, L., Zhang, H., Liu, Y., Wang, X., Jiang, Y., Gao, Y., Ye, H., Du, D. K., Yuan, Z., & Wu, X. (2025) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52734.2025.00243 · Cited by 29 · see citing works
Self-citationBenchmark Evaluations, Applications, and Challenges of Large Vision Language Models: A SurveyPreprint
Li, Z., Wu, X., Du, H., Nghiem, H. T., & Shi, G. (2025) · Qeios · doi:10.32388/gxr68q · Cited by 30 · see citing works
Ma, Y., Liu, X., Chen, X., Liu, W., Wu, C., Wu, Z., Pan, Z., Xie, Z., Zhang, H., Yu, X., et al. (2025) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52734.2025.00725 · Cited by 27 · see citing works
What Makes Multimodal In-Context Learning Work?
Baldassini, F. B., Shukor, M., Cord, M., Soulier, L., & Piwowarski, B. (2024) · IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw63382.2024.00161 · Cited by 25 · see citing works
How should we test AI for human-level intelligence? OpenAI’s o3 electrifies quest
Jones, N. (2025) · Nature · doi:10.1038/d41586-025-00110-6 · Cited by 25 · see citing works
When Do We Not Need Larger Vision Models?
Shi, B., Wu, Z., Mao, M., Wang, X., & Darrell, T. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-73242-3_25 · Cited by 33 · see citing works
PathMMU: A Massive Multimodal Expert-Level Benchmark for Understanding and Reasoning in Pathology
Sun, Y., Wu, H., Zhu, C., Zheng, S., Chen, Q., Zhang, K., Zhang, Y., Wan, D., Lan, X., Zheng, M., et al. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-73033-7_4 · Cited by 24 · see citing works
Self-citationHuang, K.-H., Chan, H. P., Fung, Y. R., Qiu, H., Zhou, M., Joty, S., Chang, S., & Ji, H. (2024) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2024.3513320 · Cited by 34 · see citing works
Yang, X., Wu, W., Feng, S., Wang, M., Wang, D., Li, Y., Sun, Q., Zhang, Y., Fu, X., & Poria, S. (2025) · Information Fusion · doi:10.1016/j.inffus.2025.103204 · Cited by 26 · see citing works
Attention Prompting on Image for Large Vision-Language Models
Yu, R., Yu, W., & Wang, X. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-73404-5_15 · Cited by 21 · see citing works
Seidel, L., Gehringer, S., Raczok, T., Ivens, S.-N., Eckardt, B., & Maerz, M. (2025) · Drones · doi:10.3390/drones9050347 · Cited by 20 · see citing works
Zhang, Q., Gao, S., Chen, W., Zhao, Y., Nie, Y., Chen, Z., Chen, S., Su, Y., & Sun, H. (2025) · Transactions in GIS · doi:10.1111/tgis.70135 · Cited by 17 · see citing works
Self-citationBai, S., Cai, Y., Chen, R., Chen, K., Chen, X., Cheng, Z., Deng, L., Ding, W., Gao, C., Ge, C., et al. (2025) · arXiv · doi:10.48550/arxiv.2511.21631 · Cited by 27 · see citing works
“We evaluate the Qwen3-VL series on a wide range of multimodal reasoning benchmarks, primarily focusing on STEM-related tasks and visual puzzles, including MMMU (Yue et al., 2024a), MMMU-Pro (Yue et al., 2024b), MathVision (Wang et al., 2024b), MathVision-Wild photo (hereafter MathVision WP ) , MathVista (Lu et al., 20…”
InfluentialSelf-citationWords or Vision: Do Vision-Language Models Have Blind Faith in Text?
Deng, A., Cao, T., Chen, Z., & Hooi, B. (2025) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52734.2025.00366 · Cited by 19 · see citing works
ImageNet-D: Benchmarking Neural Network Robustness on Diffusion Synthetic Object
Zhang, C., Pan, F., Kim, J., Kweon, I. S., & Mao, C. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.02055 · Cited by 15 · see citing works
Medical multimodal large language models: A systematic review
Hu, Y., Xu, C., Lin, B., Yang, W., & Tang, Y. Y. (2025) · Intelligent Oncology · doi:10.1016/j.intonc.2025.09.005 · Cited by 14 · see citing works
AgroBench: Vision-Language Model Benchmark in Agriculture
Shinoda, R., Inoue, N., Kataoka, H., Onishi, M., & Ushiku, Y. (2025) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51701.2025.00716 · Cited by 14 · see citing works
A Causality-Aware Paradigm for Evaluating Creativity of Multimodal Large Language Models
Huang, Z., Zhong, S., Zhou, P., Gao, S., Žitnik, M., & Lin, L. (2025) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2025.3539433 · Cited by 14 · see citing works
Multimodal large language model for wheat breeding: A new exploration of smart breeding
Yang, G., Li, Y., He, Y., Zhou, Z., Ye, L., Fang, H., Luo, Y., & Feng, X. (2025) · ISPRS Journal of Photogrammetry and Remote Sensing · doi:10.1016/j.isprsjprs.2025.03.027 · Cited by 16 · see citing works
Luo, G., Yang, X., Dou, W., Wang, Z., Liu, J., Dai, J., Qiao, Y., & Zhu, X. (2025) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52734.2025.02324 · Cited by 23 · see citing works
Truong, T.-D., Nguyen, H.-Q., Nguyen, X.-B., Dowling, A. P. G., Li, X., & Luu, K. (2025) · International Journal of Computer Vision · doi:10.1007/s11263-025-02521-4 · Cited by 14 · see citing works
EEE-Bench: A Comprehensive Multimodal Electrical And Electronics Engineering Benchmark
Li, M., Zhong, J., Chen, T., Lai, Y., & Psounis, K. (2025) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52734.2025.01245 · Cited by 12 · see citing works
Miao, Z., Zhang, Y., Fabian, Z., Celis, A. H., Beery, S., Li, C., Liu, Z., Gupta, A., Nasir, M., Li, W., et al. (2025) · Methods in Ecology and Evolution · doi:10.1111/2041-210x.70120 · Cited by 18 · see citing works
Project Sid: Many-agent simulations toward AI civilization
AL, A., Ahn, A., Becker, N., Carroll, S., Christie, N., Cortes, M., Demirci, A., Du, M., Li, F., Luo, S., et al. (2024) · arXiv · doi:10.48550/arxiv.2411.00114 · Cited by 12 · see citing works
Barman, K. G., Caron, S., Sullivan, E., de Regt, H. D., de Austri, R. D., Boon, M., Färber, M., Fröse, S., Golling, T., Lopez, L. G., et al. (2025) · The European Physical Journal C · doi:10.1140/epjc/s10052-025-14707-8 · Cited by 35 · see citing works
Li, Y., Yuan, S., Wang, H., Li, Q., Liu, M., Chen, X., Shi, G., & Zuo, W. (2025) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51701.2025.02035 · Cited by 11 · see citing works
Nasir, M. F., Rehman, M. U., & Hussain, I. (2026) · Frontiers in Plant Science · doi:10.3389/fpls.2025.1735096 · Cited by 11 · see citing works
“To connect model performance in agricultural tasks with overall scale, Figure 6 shows the relationship between parameter count, training compute, and general reasoning ability, measured by the MMMU (Massive Multi-discipline Multimodal Understanding) benchmark (Yue et al., 2024).”
Imp: Highly Capable Large Multimodal Models for Mobile Devices
Shao, Z., Zhou, Y., Yu, J., Ouyang, X., Zheng, L., Gai, Z., Wang, M., Kuang, Z., & Ding, J. (2025) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2025.3557680 · Cited by 15 · see citing works
Patnaik, S., Jain, R., Krishnamurthy, B., & Sarkar, M. (2025) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52734.2025.02207 · Cited by 11 · see citing works
All Languages Matter: Evaluating LMMs on Culturally Diverse 100 Languages
Vayani, A., Dissanayake, D., Watawana, H., Ahsan, N., Sasikumar, N., Thawakar, O., Ademtew, H. B., Hmaiti, Y., Kumar, A., Kuckreja, K., et al. (2025) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52734.2025.01822 · Cited by 25 · see citing works
Moëll, B., Farestam, F., & Beskow, J. (2025) · Frontiers in Artificial Intelligence · doi:10.3389/frai.2025.1557920 · Cited by 12 · see citing works
PhD: A ChatGPT-Prompted Visual hallucination Evaluation Dataset
Liu, J., Fu, Y., Xie, R., Xie, R., Sun, X., Lian, F., Kang, Z., & Li, X. (2025) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52734.2025.01849 · Cited by 10 · see citing works
MMVU: Measuring Expert-Level Multi-Discipline Video Understanding
Zhao, Y., Zhang, H., Xie, L., Hu, T., Gan, G., Long, Y., Hu, Z., Chen, W., Li, C., Xu, Z., et al. (2025) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52734.2025.00793 · Cited by 14 · see citing works
Self-citationProvatas, N., Konstantinou, I., & Koziris, N. (2025) · IEEE Access · doi:10.1109/access.2025.3535085 · Cited by 10 · see citing works
POEM: Interactive Prompt Optimization for Enhancing Multimodal Reasoning of Large Language Models
He, J., Wang, X., Liu, S., Wu, G., Silva, C. T., & Qu, H. (2025) · IEEE Pacific Visualization Conference (PacificVis) · doi:10.1109/pacificvis64226.2025.00010 · Cited by 14 · see citing works
Instruction-guided fusion of multi-layer visual features in Large Vision-Language Models
Xu, L., Zheng, Y., Chen, H., Chen, X., Liang, Y., Lai, C., Li, B., & Xue, X. (2025) · Pattern Recognition · doi:10.1016/j.patcog.2025.111932 · Cited by 9 · see citing works
FaceXBench: Evaluating Multimodal LLMs on Face Understanding
Narayan, K., Vibashan, V. S., & Patel, V. M. (2026) · IEEE Transactions on Biometrics Behavior and Identity Science · doi:10.1109/tbiom.2026.3655668 · Cited by 15 · see citing works
Teaching multimodal LLMs to comprehend 12-lead electrocardiographic images
Liu, R., Bai, Y., Yue, X., & Zhang, P. (2026) · npj Digital Medicine · doi:10.1038/s41746-026-02551-3 · Cited by 22 · see citing works
Self-citationPeng, H., Cui, S., Wei, J., Lu, T., Cai, Y., & Wang, S. (2025) · IEEE Transactions on Automation Science and Engineering · doi:10.1109/tase.2025.3567609 · Cited by 8 · see citing works
Enhancing Video-LLM Reasoning via Agent-of-Thoughts Distillation
Shi, Y., Di, S., Chen, Q., & Xie, W. (2025) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52734.2025.00797 · Cited by 8 · see citing works
How Many Are in This Image A Safety Evaluation Benchmark for Vision LLMs
Tu, H., Cui, C., Wang, Z., Zhou, Y., Zhao, B., Han, J., Zhou, W., Yao, H., & Xie, C. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-72983-6_3 · Cited by 8 · see citing works
EMOVA: Empowering Language Models to See, Hear and Speak with Vivid Emotions
Chen, K., Gou, Y.-H., Huang, R., Liu, Z.-L., Tan, D.-X., Xu, J., Wang, C., Zhu, Y., Zeng, Y., Yang, K., et al. (2024) · Computer Vision and Pattern Recognition · doi:10.1109/cvpr52734.2025.00513 · Cited by 65 · see citing works
Forensics-Bench: A Comprehensive Forgery Detection Benchmark Suite for Large Vision Language Models
Wang, J., Lv, C., Li, X., Dong, S., Li, H., Yao, K., Li, C., Shao, W., & Luo, P. (2025) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52734.2025.00400 · Cited by 7 · see citing works
Pan, X., Bai, Y., Zou, K., Zhou, Y., Zhou, J., Fu, H., Tham, Y. C., & Liu, Y. (2025) · Information Fusion · doi:10.1016/j.inffus.2025.103631 · Cited by 7 · see citing works
Self-citationUniToken: Harmonizing Multimodal Understanding and Generation Through Unified Visual Encoding
Jiao, Y., Qiu, H.-B., Jie, Z.-Q., Chen, S.-X., Chen, J.-J., Ma, L., & Jiang, Y.-G. (2025) · 2025 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw67362.2025.00345 · Cited by 56 · see citing works
Mitigating Hallucinations in Large Vision-Language Models via Accumulative Decoding
Zhang, H., & Zhang, Z. (2025) · International Conference on Electronic Information Engineering and Computer Science (EIECS) · doi:10.1109/eiecs67708.2025.11283452 · Cited by 7 · see citing works
SESR-Eval: Dataset for Evaluating LLMs in the Title-Abstract Screening of Systematic Reviews
Huotala, A., Kuutila, M., & Mäntylä, M. V. (2025) · International Symposium on Empirical Software Engineering and Measurement · doi:10.1109/esem64174.2025.00053 · Cited by 13 · see citing works
EgoPlan-Bench: Benchmarking Multimodal Large Language Models for Human-Level Planning
Chen, Y., Ge, Y., Ge, Y., Ding, M., Li, B., Wang, R., Xu, R., Shan, Y., & Liu, X. (2026) · International Journal of Computer Vision · doi:10.1007/s11263-025-02676-0 · Cited by 14 · see citing works
Tihanyi, N., Bisztray, T., Dubniczky, R. A., Tóth, R., Borsos, B., Cherif, B., Jain, R., Muzsai, L., Ferrag, M. A., Marinelli, R. A., et al. (2024) · IEEE International Conference on Big Data (BigData) · doi:10.1109/bigdata62323.2024.10825051 · Cited by 7 · see citing works
PathVLM-Eval: Evaluation of open vision language models in histopathology
Gilal, N. U., Zegour, R., Al-Thelaya, K., Özer, E., Agus, M., Schneider, J., & Boughorbel, S. (2025) · Journal of Pathology Informatics · doi:10.1016/j.jpi.2025.100455 · Cited by 7 · see citing works
MiniCPM-V LLaMA Model for Image Recognition: A Case Study on Satellite Datasets
Kömürcü, K., & Petkevičius, L. (2025) · IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing · doi:10.1109/jstars.2025.3547144 · Cited by 7 · see citing works
Space-LLaVA: A Vision-Language Model Adapted to Extraterrestrial Applications
Foutter, M., Gammelli, D., Kruger, J., Foss, E., Bhoj, P., Guffanti, T., D’Amico, S., & Pavone, M. (2025) · IEEE Aerospace Conference (AERO) · doi:10.1109/aero63441.2025.11068757 · Cited by 10 · see citing works
Chen, R., Lee, E., Wang, Y., Yadav, A., Zhong, M., Pragti, P., Sun, Y., & Diao, J. (2025) · Aggregate · doi:10.1002/agt2.70118 · Cited by 6 · see citing works
Self-citationBlueLM-V-3B: Algorithm and System Co-Design for Multimodal Large Language Models on Mobile Devices
Lü, X., Chen, Y., Chen, C., Tan, H. S., Chen, B., Xie, Y., Hu, R., Tan, G., Wu, R., Hu, Y., et al. (2025) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52734.2025.00392 · Cited by 12 · see citing works
Yang, C., Dong, X., Zhu, X., Su, W., Wang, J., Tian, H., Chen, Z., Wang, W., Lu, L., & Dai, J. (2025) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52734.2025.02322 · Cited by 10 · see citing works
Automated Generation of Challenging Multiple-Choice Questions for Vision Language Model Evaluation
Zhang, Y., Su, Y., Liu, Y., Wang, X., Burgess, J., Sui, E., Wang, C., Aklilu, J. G., Lozano, A., Wei, A., et al. (2025) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52734.2025.02754 · Cited by 6 · see citing works
Self-citationVLsI: Verbalized Layers-to-Interactions from Large to Small Vision Language Models
Lee, B., Hachiuma, R., Wang, Y.-C. F., Ro, Y. M., & Wu, Y.-H. (2025) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52734.2025.02751 · Cited by 6 · see citing works
MoE-Inference-Bench: Performance Evaluation of Mixture of Expert Large Language and Vision Models
Chitty-Venkata, K. T., Howland, S., Azar, G. A., Soboleva, D., Vassilieva, N., Raskar, S., Emani, M. K., & Vishwanath, V. (2025) · International Conference for High Performance Computing, Networking, Storage and Analysis (SC) · doi:10.1145/3731599.3767706 · Cited by 6 · see citing works
Visual language models show widespread visual deficits on neuropsychological tests
Tangtartharakul, G., & Storrs, K. R. (2026) · Nature Machine Intelligence · doi:10.1038/s42256-026-01179-y · Cited by 6 · see citing works
Park, S., Cui, C., Ma, Y., Moradipari, A., Gupta, R., Han, K., & Wang, Z. (2025) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51701.2025.00756 · Cited by 6 · see citing works
MMInstruct: a high-quality multi-modal instruction tuning dataset with extensive diversity
Liu, Y., Cao, Y. Q., Gao, Z., Wang, W., Chen, Z., Wang, W., Tian, H., Lu, L., Zhu, X., Lu, T., et al. (2024) · Science China Information Sciences · doi:10.1007/s11432-024-4187-3 · Cited by 7 · see citing works
Self-citationWhen AI Co-Scientists Fail: SPOT-a Benchmark for Automated Verification of Scientific Research
Son, G., Hong, J., Fan, H., Nam, H., Ko, H., Lim, S., Song, J., Choi, J., Paulo, G., Yu, Y., et al. (2025) · arXiv · doi:10.48550/arxiv.2505.11855 · Cited by 5 · see citing works
“Figure 3 illustrates the performance of o3 on six benchmarks: MathVista [50], MMLU-Pro [51], GPQA Di-amond [4], MMMU [49], HLE [52] and S POT (re-call). o3 exceeds 80% on the first four benchmarks, demonstrating robust general reasoning and code understanding.”
Videogamebunny: Towards Vision Assistants for Video Games
Taesiri, M. R., & Bezemer, C. (2025) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv61041.2025.00144 · Cited by 5 · see citing works
Accuracy-Aware MLLM Task Offloading and Resource Allocation in UAV-Assisted Satellite Edge Computing
Yan, H., Huang, H., Zhao, Z., Wang, Z., & Zhao, Z. (2025) · Drones · doi:10.3390/drones9070500 · Cited by 5 · see citing works
Jung, J., Kim, E., Kim, S., Lee, J. H., Kim, B., & Chang, B. (2025) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52734.2025.01330 · Cited by 6 · see citing works
Symmetry-Aware Advances in Multimodal Large Language Models: Architectures, Training, and Evaluation
Liu, X., & Liu, H. (2025) · Symmetry · doi:10.3390/sym17091400 · Cited by 5 · see citing works
Benchmarking Knowledge and Capability of Large Language Models in Building Science Domain
Jiang, G., Chen, K., Liu, J., Chen, J.-L., Zhe, S.-D., & Wang, Z. (2025) · Energy Use · doi:10.59717/ipj.energy-use.2025.100026 · Cited by 4 · see citing works
LLaVA-MORE: A Comparative Study of LLMs and Visual Backbones for Enhanced Visual Instruction Tuning
Cocchi, F., Moratelli, N., Caffagni, D., Sarto, S., Baraldi, L., Cornia, M., & Cucchiara, R. (2025) · IEEE International Conference on Computer Vision Workshops · doi:10.1109/iccvw69036.2025.00450 · Cited by 5 · see citing works
Evaluating the robustness and readiness of large frontier models in health AI applications
池谷, 裕., Fu, J., Liu, X., Valanarasu, J. M. J., Codella, N., Tan, R., Liu, Q., Jin, Y., Zhang, S., Wang, J., et al. (2026) · Nature Medicine · doi:10.1038/s41591-026-04501-8 · Cited by 21 · see citing works
“To address this, we conducted a structured, clinician-guided audit of nine representative benchmarks 26–30 .”
- Authors
- Yue, X., Ni, Y., Zheng, T., Zhang, K., Liu, R., Zhang, G., Stevens, S., Jiang, D., Ren, W., Sun, Y., Wei, C., Yu, B., Yuan, R., Sun, R., Yin, M., Zheng, B., Yang, Z., Liu, Y., Huang, W., Sun, H., Su, Y., & Chen, W.
- Affiliations
- IN. AI Research
- University of Waterloo (CA)
- Independent
- The Ohio State University (US)
- Carnegie Mellon University (US)
- Princeton University
- University of Victoria (CA)
- ORCID
- Venue
- 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)
- Publisher
- IEEE
- Published
- 2024-06-16
- Pages
- 9556-9567
- ISSN
- 1063-6919 (ISSN-L)
- OpenAlex
- W4402716477
- Cited by
- 429
- References
- 80
- Citations by year
- 2026: 213 · 2025: 175 · 2024: 41
- FWCI
- 88.55
- Type
- conference paper
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Multimodal Machine Learning Applications
- Physical Sciences › Computer Science › Artificial Intelligence › Domain Adaptation and Few-Shot Learning
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
- Keywords
- multimodal benchmark, large language models, domain knowledge, visual reasoning, charts and diagrams, GPT-4V, multimodal foundation models
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
