The DOI matches its Crossref record; 88 of the 88 references it cites were found in a registry or an index.
MMMU: A Massive Multi-Discipline Multimodal Understanding and Reasoning Benchmark for Expert AGI
Open accessAbstract
We introduce MMMU: a new benchmark designed to evaluate multimodal models on massive multi-discipline tasks demanding college-level subject knowledge and deliberate reasoning. MMMU includes 11.5K meticulously collected multimodal questions from college exams, quizzes, and text-books, covering six core disciplines: Art & Design, Busi-ness, Science, Health & Medicine, Humanities & Social Science, and Tech & Engineering. These questions span 30 subjects and 183 subfields, comprising 30 highly het-erogeneous image types, such as charts, diagrams, maps, tables, music sheets, and chemical structures. Unlike existing benchmarks, MMMU focuses on advanced perception and reasoning with domain-specific knowledge, challenging models to perform tasks akin to those faced by experts. The evaluation of 28 open-source LMMs as well as the propri-etary GPT-4V(ision) and Gemini highlights the substantial challenges posed by MMMU. Even the advanced GPT-4V and Gemini Ultra only achieve accuracies of 56% and 59% respectively, indicating significant room for improvement. We believe MMMU will stimulate the community to build next-generation multimodal foundation models towards expert artificial general intelligence.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source says what the claim attributes to it.
In the citing paper
“Surprisingly, open-source models such as Qwen2.5-VL-72B-Instruct and Llama-4-Maverick, which match proprietary models on existing multi-modal benchmarks like MMMU [49] or MathVista [50], perform far worse on SPOT.”
From When AI Co-Scientists Fail: SPOT-a Benchmark for Automated Verification of Scientific Research (2025) · cites this paper as [49]
The check wasn’t confident either way, so it gives no verdict — never disputed.
In the citing paper
“Self-explanations are a prevailing approach on par with traditional model explanations like LIME saliency maps [71] for understanding LLM-generated responses [25, 32, 53] and are widely used in analyzing language model behavior [66, 99].”
From Thinking in Space: How Multimodal Large Language Models See, Remember, and Recall Spaces (2025) · cites this paper as [99]
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks
Ren, S., He, K., Girshick, R., & Sun, J. (2017) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2016.2577031 · Cited by 73,472 · cited 1×
“This work relies on pre-trained visual representations like Faster RCNN features [61] to minimize the training sample complexity.”
MethodFaster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks
Ren, S., He, K., Girshick, R., & Sun, J. (2015) · arXiv · doi:10.48550/arxiv.1506.01497 · Cited by 70,812
Microsoft COCO: Common Objects in Context
Lin, T.-Y., Maire, M., Belongie, S., Hays, J., Perona, P., Ramanan, D., Dollár, P., & Zitnick, C. L. (2014) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-10602-1_48 · Cited by 45,460
Learning Transferable Visual Models From Natural Language Supervision
Radford, A., Kim, J. W., Hallacy, C., Ramesh, A., Goh, G., Agarwal, S., Sastry, G., Askell, A., Mishkin, P., Clark, J., et al. (2021) · arXiv · doi:10.48550/arxiv.2103.00020 · Cited by 15,293 · cited 3×
“Later on, CLIP [60], ALIGN [27], SimVLM [71], CoCa [78], Flamingo [2], BLIP-2 [32], and Fuyu [6] (inter alia) have been proposed to train visual representation using ViT [18] from scratch with massive amount of web data.”
BackgroundMethodLLaMA: Open and Efficient Foundation Language Models
Touvron, H., Lavril, T., Izacard, G., Martinet, X., Lachaux, M.-A., Lacroix, T., Rozière, B., Goyal, N., Hambro, E., Azhar, F., et al. (2023) · arXiv · doi:10.48550/arxiv.2302.13971 · Cited by 4,602 · cited 3×
“Rapid advances in large language models (LLMs) [13, 54, 67] have sparked broad discussions on the controversial concept of artificial general intelligence (AGI), often used to describe AI systems that perform on par or surpass humans at most tasks [1, 7, 21, 29, 49, 52].”
BackgroundMethodVQA: Visual Question Answering
Antol, S., Agrawal, A., Lu, J., Mitchell, M., Batra, D., Zitnick, C. L., & Parikh, D. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.279 · Cited by 4,487
Llama 2: Open Foundation and Fine-Tuned Chat Models
Touvron, H., Martin, L., Stone, K., Albert, P., Almahairi, A., Babaei, Y., Bashlykov, N., Batra, S., Bhargava, P., Bhosale, S., et al. (2023) · arXiv · doi:10.48550/arxiv.2307.09288 · Cited by 2,604
PaLM: Scaling Language Modeling with Pathways
Chowdhery, A., Narang, S., Devlin, J., Bosma, M., Mishra, G., Roberts, A., Barham, P., Chung, H. W., Sutton, C., Gehrmann, S., et al. (2022) · arXiv · doi:10.48550/arxiv.2204.02311 · Cited by 2,590 · cited 1×
“Rapid advances in large language models (LLMs) [13, 54, 67] have sparked broad discussions on the controversial concept of artificial general intelligence (AGI), often used to describe AI systems that perform on par or surpass humans at most tasks [1, 7, 21, 29, 49, 52].”
BackgroundLiu, H., Li, C., Wu, Q., & Lee, Y. J. (2023) · Advances in Neural Information Processing Systems 36 · doi:10.52202/075280-1516 · Cited by 2,428
LXMERT: Learning Cross-Modality Encoder Representations from Transformers
Tan, H., & Bansal, M. (2019) · Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP) · doi:10.18653/v1/d19-1514 · Cited by 2,319 · cited 1×
“LXMERT [64], UNITER [10], VinVL [80], Oscar [34], VilBert [45], and VLP [86] are among the earliest work to train universal vision-language models to tackle many multimodal tasks.”
Background
Flagged first, then by citations
Research works printed but not linked: 19
- Arcas, 2023, Artificial general intelligence is already here, Noema Magazine
- Bavishi, 2023, Introducing our multimodal models
- 2024, GitHub Repository
- Chiang, 2023, Vicuna: An open-source chatbot impressing gpt-4 with 90%* chatgpt quality
- Dosovitskiy, 2021, An image is worth 16×16 words: Transformers for image recognition at scale
- Team, 2024, Adept fuyu-heavy: A new multimodal model
- Li, 2023, Seed-bench: Benchmarking multimodalllms with generative comprehension, arXiv preprint
- Li, 2023, Blip-2: Bootstrapping language-image pretraining with frozen image encoders and large language models
- Li, 2023, M3it: A large-scale dataset towards multimodal multilingual instruction tuning, arXiv preprint
- Liu, 2023, Aligning large multimodal model with robust instruction tuning, arXiv preprint
- Liu, 2024, Llava-next: Improved reasoning, ocr, and world knowledge
- 2024, Minicpm-v
- 2024, Ominilmm-12b
- 2023, Gpt-4v(ision) system card, OpenAI
- Ormazabal, 2024, Reka flash: An efficient and capable multimodal language model
- 2023, Qwen-vl-plus
- 2024, Qwen-vl-max
- 2024, Introducing the next generation of claude
- 2024, Infimm: Advancing multimodal understanding from flamingos legacy through diverse llm integration, InfiMM Team.
Cited by
Sorted by citations
Items 401–445 of 445
NaVLM-PVC: Progressive Visual Compression for Efficient Native-Resolution Encoding in MLLMs
Sun, S., Zhang, Y., Song, H., Guo, Z., Chen, C., Zhang, Y., Yao, Y., Liu, Z., & Sun, M. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-37189-8_12
A critical assessment of the rise of agentic AI, its capabilities, task domains and open challenges
Khan, R. U., Aljuaid, H., & Ning, Z. (2026) · Discover Artificial Intelligence · doi:10.1007/s44163-026-02342-5 · Cited by 2
“MMMU : Yue et al. [135] designed MMU to test expert-level multimodal reasoning and understanding in 30 college-level subjects requiring integration of visual and text input.”
Learning When to Look: A Disentangled Curriculum for Strategic Perception in Multimodal Reasoning
Yang, S., Gao, Z., Qiu, H., Liu, F., Shi, P., Zeng, Z., Liao, Q., & Ma, L. (2026) · doi:10.1109/cvprf71909.2026.00932
Modality-Aware Bit Allocation for Mixed-Precision Quantization of Vision-Language Models
Zhang, X., Zhu, H., Wang, J., Wu, X., & Lin, W. (2026) · 2026 IEEE/CVF Conference on Computer Vision and Pattern Recognition- FINDINGS Track (CVPRF) · doi:10.1109/cvprf71909.2026.00892
“To assess progress, recent benchmarks like MMMU [50] and SEED-Bench [22] evaluate expert-level reasoning and generative comprehension across images and videos, revealing substantial room for improvement even among SOTA models.”
Large Multimodal Models as General In-Context Classifiers
Garosi, M., Farina, M., Conti, A., Mancini, M., & Ricci, E. (2026) · doi:10.1109/cvprf71909.2026.00646
“This is surprising, as LMMs are usually benchmarked in much more complex scenarios [15, 22, 40, 42], and should find classification a much easier task.”
Choi, J., Lee, J. W., You, S., & Lee, J. (2026) · doi:10.1109/cvprf71909.2026.00910 · Cited by 2
“For instance, the MMMU benchmark [40] evaluates models across six broad disciplines—including science, engineering, medicine, and the humanities—demanding expert-level domain knowledge and multi-step reasoning.”
Xia, G.-Y., Ding, Y.-F., Li, F.-F., Ren, L., Chen, W., Feng, F.-X., & Wang, X.-J. (2025) · 2026 IEEE/CVF Conference on Computer Vision and Pattern Recognition- FINDINGS Track (CVPRF) · doi:10.1109/cvprf71909.2026.00568 · Cited by 3
Self-citationDrawingVQA: A Real-World Benchmark for Multi-Depth Visual-Textual Reasoning on Construction Drawings
Jung, Y., Fu, J., & Golparvar‐Fard, M. (2026) · doi:10.1109/cvprf71909.2026.00208
Object-Centric Vision Token Pruning for Vision Language Models
Li, G., Zhao, R., Deng, J., Wang, Y., & Pajarinen, J. (2026) · doi:10.1109/cvprf71909.2026.00676
Do LLMs and VLMs Share Reasoning Neurons? Evidence and Mechanisms of Cross-Modal Transfer
Cui, C., Zhang, A., Chen, Y., Deng, G., Zheng, J., Liang, Z., Wang, X., & Chua, T. (2026) · doi:10.1109/cvprf71909.2026.00229
“We evaluate models on both the original MMMU [54] and the more challenging MMMU-Pro [55].”
Switch-KD: Visual-Switch Knowledge Distillation for Vision-Language Models
Sun, H., Wang, X., Mao, N., Wang, Q., Mu, L., Zheng, W., Wei, T., & Chen, W. (2026) · doi:10.1109/cvprf71909.2026.00893
“(2) Cognition & Reasoning: VQAv2 [8], GQA [13], ScienceQA [33], and MMMU [52] evaluate compositional reasoning and complex problem-solving abilities, reflecting higher-order cognition.”
Self-citationEgoTL: Egocentric Think-Aloud Chains for Long-Horizon Tasks
Liu, L., Li, D., Liang, Y., Jiang, S., Vijay, H., Hu, H., Xu, X., Liu, Z., Shakkottai, S., Li, M., et al. (2026) · doi:10.1109/cvprf71909.2026.00198
“For multiple-choice answer (MCA) tasks, we use standard accuracy ( ACC ) [15, 20, 56] based on exact matching (optionally with fuzzy matching for minor wording differences).”
Flash-Unified: A Training-Free and Task-Aware Acceleration Framework for Native Unified Models
Ke, J., Wen, Z., Yang, B., Yang, Y., Liu, X., Liao, C., Chen, Z., Wang, S., & Zhang, L. (2026) · doi:10.1109/cvprf71909.2026.00876
“For evaluation, we assess multimodal understanding on a suite of benchmarks including MME [19], GQA [24], MMBench [41], MMMU [77], MMStar [7], and AI2D [29].”
Feng, Y., Wu, W., Sun, J., Li, C., Zhang, F., Li, Z., Ai, J., Zhou, S., Chang, Y., Gao, C., et al. (2026) · doi:10.1109/cvprf71909.2026.00217
“More recently, benchmarks like MMBench [16], MMMU [35], and MM-VET [34] have offered a more comprehensive evaluation across various tasks [5, 6].”
Self-citationLin, Z., Zhang, Z., Sun, S., Zhao, J., & Yang, H. (2026) · 2026 IEEE/CVF Conference on Computer Vision and Pattern Recognition- FINDINGS Track (CVPRF) · doi:10.1109/cvprf71909.2026.00205
Mitigating Vision-Text Order Bias in Vision-Language Models
Gan, W., Song, Y., Yu, Z., & Li, S. (2026) · doi:10.1109/cvprf71909.2026.00925
“The rapid progress of Vision-Language Models (VLMs) is transforming human–AI interaction beyond unimodal boundaries [1, 19, 20], enabling complex tasks from mathematical reasoning [12, 24, 29] to nuanced spatial reasoning [17].”
Attention-Space Contrastive Guidance for Efficient Hallucination Mitigation in LVLMs
Jo, Y., Bae, S.-P., & Kim, T. (2026) · 2026 IEEE/CVF Conference on Computer Vision and Pattern Recognition- FINDINGS Track (CVPRF) · doi:10.1109/cvprf71909.2026.00929 · Cited by 1
Lian, S., Wu, C., Yang, L. T., Yuan, H., Yu, B., Zhang, L., & Chen, K. (2026) · doi:10.1109/cvprf71909.2026.00940
Self-citationQuantifying the Gap between Understanding and Generation within Unified Multimodal Models
Wang, C., Chen, Y., Hu, Z., Chen, D., Chen, W., Wiegreffe, S., & Zhou, T. (2026) · doi:10.1109/cvprf71909.2026.00570
“We provide more details on how the gap is computed in Appendix B.”
InfluentialSelf-citationOTPrune: Distribution-Aligned Visual Token Pruning via Optimal Transport
Chen, X., Zhu, W., Li, G., Dong, X., Xiong, Y., Wang, H., Qiu, P., Song, Q., Wang, Z., Tang, S., et al. (2026) · doi:10.1109/cvprf71909.2026.00562
Liu, Z., Wei, Y., Feng, L., Su, X., Xia, X., Guan, W., Xie, Z., & Yang, S. (2026) · doi:10.1109/cvprf71909.2026.00919
“For LLaVA [24], applying the TaLo method yields up to a 4.7% performance gain on the Tech&Engineering task of MMMU [54].”
InfluentialD 2 -STX: Decoupling Spatial-Temporal Cross-Attention for Dual-Branch Repetitive Action Counting
Wang, X., Wang, H., Liu, Y., Hu, H., & Yu, B. X. B. (2026) · doi:10.1109/cvprf71909.2026.00788 · Cited by 1
Self-citationAnyExperts: On-Demand Expert Allocation for Multimodal Language Models with Mixture of Experts
Gao, Y., Wang, L., Zhao, H., Huang, L., Liu, S., & Guo, Q. (2026) · 2026 IEEE/CVF Conference on Computer Vision and Pattern Recognition- FINDINGS Track (CVPRF) · doi:10.1109/cvprf71909.2026.00883
Vision Inference Former: Sustaining Visual Consistency in Multimodal Large Language Models
Dong, X., Zhang, M., Han, K., Tan, X., Wu, F., & Kuang, K. (2026) · 2026 IEEE/CVF Conference on Computer Vision and Pattern Recognition- FINDINGS Track (CVPRF) · doi:10.1109/cvprf71909.2026.00586
“We conducted extensive evaluations across a comprehensive set of 14 benchmark datasets, includin MMMU [45], RealWorldQA [43], MMBench [22], MMStar [4], OK-VQA [24], GQA [13], ScienceQA [32], MMVP [39], OCRBench [21], TextVQA [34], AI2D [12], InfographicVQA [27], DocVQA [26] and POPE [19].”
Myopia Rectification: KV Cache Pruning for MLLMs via Dynamic Attention Subsidy and Token Reclamation
Zhuang, J., Lu, L., Dai, M., Chen, J., Liu, Q., & Hu, H. (2026) · doi:10.1109/cvprf71909.2026.00866
“With these foundational developments, Multimodal Large Language Models (MLLMs) [34–36] have demonstrated remarkable performance in tasks such as visual question answering [21, 26, 41, 42, 49], visual reasoning [20, 38, 62] and visual grounding [14–16 , 68].”
TextBind: Your Vision-Language Models are Naturally Unified Multimodal Models
Ma, X., & Fu, Y. (2026) · doi:10.1109/cvprf71909.2026.00579
“For visual understanding, we report accuracy on MMBench [29], SEED-Bench [23], MM-Vet [56], and MMMU [57], which together cover multi-image perception, fine-grained recognition, and multimodal reasoning under open-ended, exam-style questions.”
Multimodal Reasoning with Explicit Reasoning Patterns and Rewards
Qiu, H., Jin, S., Zuo, Z., Wang, Z., She, Q., Shao, L., & Lu, S. (2026) · doi:10.1109/cvprf71909.2026.00914
“For general multimodal reasoning, we employ MMMU-Pro [43], MMMU [42], and EMMA [11] benchmarks.”
PaLMR: Towards Faithful Visual Reasoning via Multimodal Process Alignment
Li, Y., Hui, Q., Yan, C.-Y., Cheng, K., Zhao, F., Tan, C., Gao, H.-L., Zhang, J., Wang, K., Dai, X.-Y., et al. (2026) · 2026 IEEE/CVF Conference on Computer Vision and Pattern Recognition- FINDINGS Track (CVPRF) · doi:10.1109/cvprf71909.2026.00587 · Cited by 4
“However, most existing reward mechanisms focus solely on what the model answers, while overlooking how it reasons.”
Recursive Think-Answer Process for LLMs and VLMs
Lee, B.-K., Chee, Y., & Ro, Y. M. (2026) · 2026 IEEE/CVF Conference on Computer Vision and Pattern Recognition- FINDINGS Track (CVPRF) · doi:10.1109/cvprf71909.2026.00920
“%) of numerous vision language models (VLMs) on five evaluation benchmarks—MMMU [94], Math-Vista [58], OlympiadBench [36], MathVision [78], and MMMU-Pro [95].”
Elicit and Enhance: Advancing Multimodal Reasoning in Medical Scenarios
Huang, Z., Mu, L., Gu, Y., Hu, K., Hua, S., & Zhang, X. (2026) · 2026 IEEE/CVF Conference on Computer Vision and Pattern Recognition- FINDINGS Track (CVPRF) · doi:10.1109/cvprf71909.2026.00539
Reasoning Within the Mind: Dynamic Multimodal Interleaving in Latent Space
Liu, C., Yang, Y., Fan, Y., Wei, Q., Liu, S., & Wang, X. E. (2026) · doi:10.1109/cvprf71909.2026.00885
DraCo: Draft as CoT for Text-to-Image Preview and Rare Concept Generation
Jiang, D., Zhang, R., Li, H., Zong, Z., Guo, Z., He, J., Guo, C., Ye, J., Fang, R., Li, W., et al. (2026) · 2026 IEEE/CVF Conference on Computer Vision and Pattern Recognition- FINDINGS Track (CVPRF) · doi:10.1109/cvprf71909.2026.00567
Self-citationCREM: Compression-Driven Representation Enhancement for Multimodal Retrieval and Comprehension
Liu, L., Yang, B., Wang, Y., Li, D., Cao, J., Luo, Y., Chen, X., Wu, X., Yuan, W., Yang, F., et al. (2026) · doi:10.1109/cvprf71909.2026.00549
“These tokens can then populate the KV cache for decoding or be stored for future reuse, thereby eliminating redundant computation and reducing memory usage in long-context scenarios.”
Efficient Long-Context Modeling in Diffusion Language Models via Block Approximate Sparse Attention
Zhang, W., Wu, Y., Wang, H., Liu, Y., Dou, H., Yang, S., Wu, S., Zhao, H., & Jia, J. (2026) · doi:10.1109/cvprf71909.2026.00278
“Second, we adopt ChartQA [31], DocVQA [32], InfoVQA [33], Re-alworldQA [44], MMMU [56], and MMMU-Pro [57] for image–language tasks.”
Self-citationOTSUE, Y., & Yamada, H. (2026) · Transactions of Japan Society of Kansei Engineering · doi:10.5057/jjske.tjske-d-26-00026
Zhou, J. (2026) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn69854.2026.11729497
“MMMU (Massive Multi-discipline Multimodal Understanding) (Medical) [21] is the medical subset of the Massive Multi-discipline Multimodal Understanding benchmark, evaluating expert-level reasoning across clinical medicine and basic sciences.”
Jin, Z., & Mao, R. (2026) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn69854.2026.11728558
“Prominent examples include MMMU [4] and ScienceQA [5] for scientific reasoning, MMBench [6] for comprehensive vision-language evaluation, and GQA [7] for visual question answering.”
LexSign: Learning Sign Language from Lexical Descriptions
Min, Y., Zou, H., Jiao, P., & Chen, X. (2026) · International Journal of Computer Vision · doi:10.1007/s11263-026-03075-9
MAAP-VL: Modality-aware adaptive probing for reliability diagnosis of vision-language models
Wang, X., Zhan, Y., Liu, Z., Wang, K., Lian, S., & Du, J. (2026) · Image and Vision Computing · doi:10.1016/j.imavis.2026.106249
FlashAttention-Compatible Modality-Mutual Attention with Virtual Reordering
Seo, J. W., & Jung, H. (2026) · Mathematics · doi:10.3390/math14213852
A Study on Multimodal Encoding and Fusion Methods for Visual Question Answering
Yan, J. (2026) · International Conference on Image Processing, Machine Learning and Pattern Recognition (IPMLPR) · doi:10.1145/3847445.3847453
Marchetti, T., Capezzone, S., Esposito, R., Spirito, M. D., & Maulucci, G. (2026) · Nutrients · doi:10.3390/nu18213582
OmniGen2: Towards Instruction-Aligned Multimodal Generation
Wu, C., Zheng, P., Yan, R., Xiao, S., Luo, X., Wang, Y., Li, W., Jiang, X., Liu, Y., Zhou, J., et al. (2025) · arXiv · doi:10.48550/arxiv.2506.18871
Self-citationGrattafiori, A., Dubey, A., Jauhri, A., Pandey, A., Kadian, A., Al-Dahle, A., Letman, A., Mathur, A., Schelten, A., Vaughan, A., et al. (2024) · arXiv · doi:10.48550/arxiv.2407.21783 · Cited by 505
Self-citationGemini 1.5: Unlocking multimodal understanding across millions of tokens of context
Gemini Team, Georgiev, P., Lei, V. I., Burnell, R., Bai, L., Gulati, A., Tanzer, G., Vincent, D., Pan, Z., Wang, S., et al. (2024) · arXiv · doi:10.48550/arxiv.2403.05530 · Cited by 306
Self-citation
Metadata
- Authors
- Yue, X., Ni, Y., Zheng, T., Zhang, K., Liu, R., Zhang, G., Stevens, S., Jiang, D., Ren, W., Sun, Y., Wei, C., Yu, B., Yuan, R., Sun, R., Yin, M., Zheng, B., Yang, Z., Liu, Y., Huang, W., Sun, H., Su, Y., & Chen, W.
- Affiliations
- IN. AI Research
- University of Waterloo (CA)
- Independent
- The Ohio State University (US)
- Carnegie Mellon University (US)
- Princeton University
- University of Victoria (CA)
- ORCID
- Venue
- 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)
- Publisher
- IEEE
- Published
- 2024-06-16
- Pages
- 9556-9567
- ISSN
- 1063-6919 (ISSN-L)
- OpenAlex
- W4402716477
- Semantic Scholar
- b50d19c5c298f6562c3b3c6c3822a351bdc89260
- Cited by
- 429
- References
- 80
- Citations by year
- 2026: 213 · 2025: 175 · 2024: 41
- FWCI
- 88.55
- Type
- conference paper
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Multimodal Machine Learning Applications
- Physical Sciences › Computer Science › Artificial Intelligence › Domain Adaptation and Few-Shot Learning
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
- Keywords
- multimodal benchmark, large language models, domain knowledge, visual reasoning, charts and diagrams, GPT-4V, multimodal foundation models
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
