What matters when building vision-language models?
Open accessThe DOI matches its Crossref record; 9 of the 9 references it cites were found in a registry or an index.
Abstract
The growing interest in vision-language models (VLMs) has been driven by improvements in large language models and vision transformers. Despite the abundance of literature on this subject, we observe that critical decisions regarding the design of VLMs are often not justified. We argue that these unsupported decisions impede progress in the field by making it difficult to identify which choices improve model performance. To address this issue, we conduct extensive experiments around pre-trained models, architecture choice, data, and training methods. Our consolidation of findings includes the development of Idefics2, an efficient foundational VLM of 8 billion parameters. Idefics2 achieves state-of-the-art performance within its size category across various multimodal benchmarks, and is often on par with models four times its size. We release the model (base, instructed, and chat) along with the datasets created for its training.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Citations
Cites
The publisher hasn’t deposited this paper’s reference list.
Cited by
Items 101–200 of 660
RadVLM: a multitask conversational vision-language model for radiology
Deperrois, N., Matsuo, H., Ruipérez-Campillo, S., Vandenhirtz, M., Laguna, S., Ryser, A., Fujimoto, K., Nishio, M., Sutter, T. M., Vogt, J. E., et al. (2026) · Scientific Reports · doi:10.1038/s41598-026-66181-1 · Cited by 50
M3DocVQA: Multi-Modal Multi-Page Multi-Document Understanding
Cho, J., Mahata, D., İrsoy, O., He, Y., & Bansal, M. (2025) · IEEE International Conference on Computer Vision Workshops · doi:10.1109/iccvw69036.2025.00649 · Cited by 49
LLaVA-MORE: A Comparative Study of LLMs and Visual Backbones for Enhanced Visual Instruction Tuning
Cocchi, F., Moratelli, N., Caffagni, D., Sarto, S., Baraldi, L., Cornia, M., & Cucchiara, R. (2025) · IEEE International Conference on Computer Vision Workshops · doi:10.1109/iccvw69036.2025.00450 · Cited by 50
Unveiling the Ignorance of MLLMs: Seeing Clearly, Answering Incorrectly
Liu, Y., Liang, Z., Wang, Y., Wu, X., Tang, F., He, M., Li, J., Liu, Z., Yang, H., Lim, S.-N., et al. (2025) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52734.2025.00849 · Cited by 48
MOSSBench: Is Your Multimodal Language Model Oversensitive to Safe Queries?Preprint
Li, X., Zhou, H., Wang, R., Zhou, T., Cheng, M., & Hsieh, C. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2406.17806 · Cited by 48
A Survey on Agentic Multimodal Large Language ModelsPreprint
Yao, H., Zhang, R., Huang, J., Zhang, J., Wang, Y., Fang, B., Zhu, R., Jing, Y., Liu, S., Li, G., et al. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2510.10991 · Cited by 49
MIA-DPO: Multi-Image Augmented Direct Preference Optimization For Large Vision-Language Models
Liu, Z., Zang, Y., Dong, X., Zhang, P., Cao, Y., Duan, H., He, C., Xiong, Y., Lin, D., & Wang, J. (2024) · International Conference on Learning Representations · doi:10.48550/arxiv.2410.17637 · Cited by 47
MM-CoT:A Benchmark for Probing Visual Chain-of-Thought Reasoning in Multimodal ModelsPreprint
Zhang, J., Cai, K., Guo, X., Liu, S., Lv, Q., Chen, R., Yang, J., Fan, Y., Sun, X., Wang, J., et al. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2512.08228 · Cited by 48
SOLO: A Single Transformer for Scalable Vision-Language Modeling
Chen, Y., Wang, X., Peng, H., & Ji, H. (2024) · Transactions on machine learning research · doi:10.48550/arxiv.2407.06438 · Cited by 46
Distill Visual Chart Reasoning Ability from LLMs to MLLMs
He, W., Xi, Z., Zhao, W., Fan, X., Ding, Y., Shan, Z., Gui, T., Zhang, Q., & Huang, X. (2025) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2025.findings-emnlp.172 · Cited by 47
MMLongBench: Benchmarking Long-Context Vision-Language Models Effectively and Thoroughly
Wang, Z., Yu, W., Xiyu, R. E. N., Zhang, J., Zhao, Y., Saxena, R., Cheng, L., Wong, G. Y., See, S. C. W., Minervini, P., et al. (2025) · neural information processing systems · doi:10.52202/085713-3627 · Cited by 46
Modality-Aware Neuron Pruning for Unlearning in Multimodal Large Language Models
Liu, Z., Dou, G., Yuan, X., Zhang, C., Tan, Z., & Jiang, M. (2025) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2025.acl-long.295 · Cited by 45
Make LVLMs Focus: Context-Aware Attention Modulation for Better Multimodal In-Context Learning
Li, Y., Yang, J., Yang, Z., Li, B., Han, L., He, H., Yao, Z., Chen, Y., Fei, S., Liu, D., et al. (2026) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v40i8.37591 · Cited by 45
Automated Generation of Challenging Multiple-Choice Questions for Vision Language Model Evaluation
Zhang, Y., Su, Y., Liu, Y., Wang, X., Burgess, J., Sui, E., Wang, C., Aklilu, J. G., Lozano, A., Wei, A., et al. (2025) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52734.2025.02754 · Cited by 45
AlignVLM: Bridging Vision and Language Latent Spaces for Multimodal Document Understanding
Masry, A., Rodriguez, J., Zhang, T., Wang, S., Wang, C., Feizi, A., Suresh, A. K., Puri, A., Jian, X., Noël, P., et al. (2025) · neural information processing systems · doi:10.52202/085713-4875 · Cited by 44
DualToken: Towards Unifying Visual Understanding and Generation with Dual Visual Vocabularies
Song, W., Wang, Y., Song, Z., Li, Y., Zhou, Z., Chen, L., Xu, J. Q., Wang, J., & Yu, K. (2025) · International Conference on Learning Representations · doi:10.48550/arxiv.2503.14324 · Cited by 44
Temporal Preference Optimization for Long-Form Video UnderstandingPreprint
Li, R., Wang, X., Zhang, Y., Zohar, O., Wang, Z., & Yeung, S. Y. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2501.13919 · Cited by 44
FOLDER: Accelerating Multi-Modal Large Language Models with Enhanced Performance
Wang, H., Yu, Z., Spadaro, G., Ju, C., Quétu, V., Xiao, S., & Tartaglione, E. (2025) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51701.2025.02192 · Cited by 42
II-Bench: An Image Implication Understanding Benchmark for Multimodal Large Language Models
Liu, Z., Fang, F., Feng, X., Du, X., Zhang, C., Wang, Z., Bai, Y., Zhao, Q., Fan, L., Gan, C., et al. (2024) · neural information processing systems · doi:10.52202/079017-1474 · Cited by 43
On the Fundamental Limits of LLMs at Scale
Mohsin, M. A., Umer, M., Bilal, A., Memon, Z., Qadir, M. I., Bhattacharya, S., Rizwan, H., Gorle, A. R., Kazmi, M. Z., Amir, N., et al. (2026) · Transactions on machine learning research · doi:10.48550/arxiv.2511.12869 · Cited by 42
Jiao, Q., Chen, D., Huang, Y., Li, Y., & Shen, Y. (2026) · International Journal of Computer Vision · doi:10.1007/s11263-026-02877-1 · Cited by 42
OA-WAM: Object-Addressable World Action Model for Robust Robot ManipulationPreprint
Liu, Y., Sun, P., Li, S., Xie, Y., Zhang, L., Chao, X., Dong, S., Chen, F., Zhang, X.-P., & Ding, W. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2605.06481 · Cited by 43
MultiChartQA: Benchmarking Vision-Language Models on Multi-Chart Problems
Zhu, Z., Jia, M., Zhang, Z., Li, L., & Jiang, M. (2025) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/2025.naacl-long.566 · Cited by 41
What Factors Affect Multi-Modal In-Context Learning? An In-Depth Exploration
Qin, L., Chen, Q., Fei, H., Chen, Z., Li, M., & Che, W. (2024) · neural information processing systems · doi:10.52202/079017-3916 · Cited by 41
LATTE: Learning to Think with Vision Specialists
Ma, Z., Zhang, J., Liu, Z., Zhang, J., Tan, J., Shu, M., Niebles, J. C., Heinecke, S., Wang, H., Xiong, C., et al. (2025) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2025.emnlp-main.564 · Cited by 41
Bring Reason to Vision: Understanding Perception and Reasoning through Model MergingPreprint
Chen, S., Zhang, J., Zhu, T., Liu, W., Gao, S., Xiong, M., Li, M., & He, J. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2505.05464 · Cited by 42
MIBench: Evaluating Multimodal Large Language Models over Multiple Images
Liu, H., Zhang, X., Xu, H., Shi, Y., Jiang, C., Yan, M., Zhang, J., Huang, F., Yuan, C., Li, B., et al. (2024) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2024.emnlp-main.1250 · Cited by 40
Granular Privacy Control for Geolocation with Vision Language Models
Mendes, E., Chen, Y., Hays, J. H., Das, S., Xu, W. H., & Ritter, A. (2024) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2024.emnlp-main.957 · Cited by 40
He, H., Li, G., Geng, Z., Xu, J., & Peng, Y. (2025) · International Conference on Learning Representations · doi:10.48550/arxiv.2501.15140 · Cited by 40
Huang, K.-H., Qin, C., Qiu, H., Laban, P., Joty, S., Xiong, C., & Wu, C.-S. (2025) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2025.findings-acl.249 · Cited by 38
STORM: Token-Efficient Long Video Understanding for Multimodal LLMsPreprint
Jiang, J., Li, X., Liu, Z., Li, M., Guo, C., Li, Z., Huang, D.-A., Liu, G., Yu, Z., Keutzer, K., et al. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2503.04130 · Cited by 38
TUNA: Taming Unified Visual Representations for Native Unified Multimodal ModelsPreprint
Liu, Z., Ren, W., Liu, H., Zhou, Z., Chen, S., Qiu, H., Huang, X., An, Z., Yang, F., Patel, A., et al. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2512.02014 · Cited by 37
Tikzero: Zero-Shot Text-Guided Graphics Program Synthesis
Belouadi, J., Ilg, E., Keuper, M., Tanaka, H., Utiyama, M., Dabre, R., Eger, S., & Ponzetto, S. P. (2025) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51701.2025.01653 · Cited by 36
The Responsible Foundation Model Development Cheatsheet: A Review of Tools & Resources
Longpre, S., Biderman, S., Albalak, A., Schoelkopf, H., McDuff, D., Kapoor, S., Klyman, K., Lo, K., Ilharco, G., San, N., et al. (2024) · Transactions on machine learning research · doi:10.48550/arxiv.2406.16746 · Cited by 37
Self-citationEfficient Vision-Language Models by Summarizing Visual Tokens into Compact RegistersPreprint
Wen, Y., Cao, Q., Fu, Q., Mehta, S., & Najibi, M. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2410.14072 · Cited by 36
FaceXBench: Evaluating Multimodal LLMs on Face Understanding
Narayan, K., Vibashan, V. S., & Patel, V. M. (2026) · IEEE Transactions on Biometrics Behavior and Identity Science · doi:10.1109/tbiom.2026.3655668 · Cited by 35
Amini, H., Mia, M. J., Saadati, Y., Imteaj, A., Nabavirazavi, S., Thakker, U., Hossain, M. Z., Fime, A. A., & Iyengar, S. S. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2503.16585 · Cited by 34
HiMTok: Learning Hierarchical Mask Tokens for Image Segmentation with Large Multimodal Model
Wang, T., Cheng, C., Wang, L., Chen, S., & Zhao, W. (2025) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51701.2025.02160 · Cited by 33
Bee: A High-Quality Corpus and Full-Stack Suite to Unlock Advanced Fully Open MLLMs
Zhang, Y., Ni, B., Chen, X.-S., Zhang, H., Rao, Y., Peng, H., Lu, Q., Hu, H., Guo, M.-H., & Hu, S.-M. (2025) · International Conference on Learning Representations · doi:10.48550/arxiv.2510.13795 · Cited by 34
A Survey: Collaborative Hardware and Software Design in the Era of Large Language Models
Guo, C., Cheng, F., Du, Z., Kiessling, J., Ku, J., Li, S., Li, Z., Ma, M., Molom-Ochir, T., Morris, B., et al. (2025) · IEEE Circuits and Systems Magazine · doi:10.1109/mcas.2024.3476008 · Cited by 32
Can MLLMs Understand the Deep Implication Behind Chinese Images?
Zhang, C., Feng, X., Bai, Y., Du, X., Hou, J., Deng, K., Han, G., Li, Q., Wang, B., Liu, J., et al. (2025) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2025.acl-long.700 · Cited by 32
BLIP-3: A Family of Open Large Multimodal Models
Xue, L., Shu, M., Awadalla, A., Wang, J., Yan, A., Purushwalkam, S., Zhou, H., Prabhu, V., Dai, Y., Ryoo, M. S., et al. (2025) · IEEE International Conference on Computer Vision Workshops · doi:10.1109/iccvw69036.2025.00644 · Cited by 32
Xu, M.-Z., Gao, M.-F., Li, S.-Y., Lu, J.-S., Gan, Z., Lai, Z.-F., Cao, M., Kang, K., Yang, Y.-F., & Dehghan, A. (2025) · arXiv.org · doi:10.48550/arxiv.2503.18943 · Cited by 41
Vision-Language Models Can't See the Obvious
Dahou, Y., Huynh, N. D., Khac, P. H. L., Para, W. R., Singh, A., & Narayan, S. (2025) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51701.2025.02239 · Cited by 31
Img-Diff: Contrastive Data Synthesis for Multimodal Large Language Models
Jiao, Q., Chen, D., Huang, Y., Ding, B., Li, Y., & Shen, Y. (2025) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52734.2025.00868 · Cited by 31
BigDocs: An Open Dataset for Training Multimodal Models on Document and Code TasksPreprint
Rodriguez, J., Jian, X., Panigrahi, S. S., Zhang, T., Feizi, A., Puri, A., Kalkunte, A., Savard, F., Masry, A., Nayak, S., et al. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2412.04626 · Cited by 31
UniFlow: A Unified Pixel Flow Tokenizer for Visual Understanding and Generation
Yue, Z., Zhang, H., Zeng, X., Chen, B., Wang, C., Zhuang, S., Dong, L., Wang, Y., Wang, L., & Wang, L. (2025) · International Conference on Learning Representations · doi:10.48550/arxiv.2510.10575 · Cited by 31
AutoGUI: Scaling GUI Grounding with Automatic Functionality Annotations from LLMs
Li, H., Chen, J., Su, J., Chen, Y., Li, Q., & Zhang, Z. (2025) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2025.acl-long.510 · Cited by 30
Visual Perception by Large Language Model’s Weights
Ma, F., Xue, H., Shou, M. Z., Wang, G., Rao, F., Yan, S., Zhang, Y., Wu, S., Shou, M. Z., & Sun, X. (2024) · neural information processing systems · doi:10.52202/079017-0898 · Cited by 30
TACO: Enhancing Multimodal In-context Learning via Task Mapping-Guided Sequence Configuration
Li, Y., Yang, J., Yun, T., Feng, P., Huang, J., & Tang, R. (2025) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2025.emnlp-main.39 · Cited by 30
MVI-Bench: A Comprehensive Benchmark for Evaluating Robustness to Misleading Visual Inputs in LVLMsPreprint
Chen, H., Peng, J., Min, D., Sun, C., Chen, K., Yan, Y., Yang, X., & Cheng, L. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2511.14159 · Cited by 30
Seidel, L., Gehringer, S., Raczok, T., Ivens, S.-N., Eckardt, B., & Maerz, M. (2025) · Drones · doi:10.3390/drones9050347 · Cited by 29
Chen, T., Li, P., Zhou, K., Chen, T., & Wei, H. (2025) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2025.findings-acl.237 · Cited by 29
When does perceptual alignment benefit vision representations?
Sundaram, S., Fu, S., Muttenthaler, L., Tamir, N. Y., Chai, L., Kornblith, S., Darrell, T., & Isola, P. (2024) · neural information processing systems · doi:10.52202/079017-1756 · Cited by 29
ModernVBERT: Towards Smaller Visual Document RetrieversPreprint
Teiletche, P., Macé, Q., Conti, M., Loison, A., Viaud, G., Colombo, P., & Faysse, M. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2510.01149 · Cited by 30
TACO: Enhancing Multimodal In-context Learning via Task Mapping-Guided Sequence ConfigurationPreprint
Li, Y., Yang, J., Yun, T., Feng, P., Huang, J., & Tang, R. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2505.17098 · Cited by 29
AI-generated Image Quality Assessment in Visual Communication
Tian, Y., Li, Y., Chen, B., Zhu, H., Wang, S., & Kwong, S. (2025) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v39i7.32795 · Cited by 27
Evaluating Visual and Cultural Interpretation: The K-Viscuit Benchmark with Human-VLM Collaboration
Park, C., Baek, Y., Kim, J., Heo, Y., Chang, D.-S., & Choo, J. (2025) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2025.acl-long.1066 · Cited by 27
SEEA-R1: Tree-Structured Reinforcement Fine-Tuning for Self-Evolving Embodied Agents
Tian, W., Zhang, S., Zhang, K., Chi, X., Fan, C.-K., Lu, J., Luo, Y., Zhou, Q., Zhao, Y., Liu, N., et al. (2025) · neural information processing systems · doi:10.52202/085713-2628 · Cited by 27
Towards Visual Text Grounding of Multimodal Large Language ModelPreprint
Li, M., Zhang, R., Chen, J., Wang, C., Gu, J., Zhou, Y., Dernoncourt, F., Zhu, W., Zhou, T., & Sun, T. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2504.04974 · Cited by 27
Image Captioning Evaluation in the Age of Multimodal LLMs: Challenges and Future PerspectivesPreprint
Sarto, S., Cornia, M., & Cucchiara, R. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2503.14604 · Cited by 27
FaceLLM: A Multimodal Large Language Model for Face Understanding
Shahreza, H. O., & Marcel, S. (2025) · IEEE International Conference on Computer Vision Workshops · doi:10.1109/iccvw69036.2025.00390 · Cited by 26
Ren, W., Yang, H., Min, J., Wei, C., & Chen, W. (2025) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52734.2025.00360 · Cited by 25
Lost in Embeddings: Information Loss in Vision–Language Models
Li, W., Tang, R., Li, C., Zhang, C., Vulić, I., & Søgaard, A. (2025) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2025.findings-emnlp.1235 · Cited by 25
Multimodal language models in agriculture: A tutorial and survey
Haghighat, M., Saleh, A., & Azghadi, M. R. (2025) · Information Fusion · doi:10.1016/j.inffus.2025.104042 · Cited by 25
Onohara, S., Miyai, A., Imajuku, Y., Egashira, K., Baek, J., Yue, X., Neubig, G., & Aizawa, K. (2025) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/2025.naacl-long.43 · Cited by 25
Yu, X., Guan, D., & Gu, Y. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2506.01663 · Cited by 25
Mimic In-Context Learning for Multimodal Tasks
Jiang, Y., Fu, J., Hao, C., Hu, X., Peng, Y., Geng, X., & Xu, Y. (2025) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52734.2025.02776 · Cited by 24
A Token-Level Text Image Foundation Model for Document Understanding
Guan, T., Wang, Z., Fu, P., Guo, Z., Shen, W., Zhou, K., Yue, T., Duan, C., Sun, H., Jiang, Q., et al. (2025) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51701.2025.02155 · Cited by 24
Automated Black-box Prompt Engineering for Personalized Text-to-Image Generation
He, Y., Robey, A., Murata, N., Jiang, Y., Williams, J., Pappas, G. J., Hassani, H., Mitsufuji, Y., Salakhutdinov, R., & Kolter, J. Z. (2025) · Transactions on machine learning research · doi:10.48550/arxiv.2403.19103 · Cited by 24
VCR: A Task for Pixel-Level Complex Reasoning in Vision Language Models via Restoring Occluded Text
Zhang, T., Wang, S., Li, L., Zhang, G., Taslakian, P., Rajeswar, S., Fu, J., Liu, B., & Bengio, Y. (2024) · International Conference on Learning Representations · doi:10.48550/arxiv.2406.06462 · Cited by 24
ProVision: Programmatically Scaling Vision-centric Instruction Data for Multimodal Language ModelsPreprint
Zhang, J., Xue, L., Song, L., Wang, J., Huang, W., Shu, M., Yan, A., Ma, Z., Niebles, J. C., Savarese, S., et al. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2412.07012 · Cited by 24
Kong, D., Lin, S., Xu, Z., Wang, Z., Li, M., Li, Y., Zhang, Y., Peng, H., Chen, X., Sha, Z., et al. (2025) · arXiv · doi:10.48550/arxiv.2506.19676 · Cited by 23
MIBench: Evaluating Multimodal Large Language Models over Multiple ImagesPreprint
Liu, H., Zhang, X., Xu, H., Shi, Y., Jiang, C., Yan, M., Zhang, J., Huang, F., Yuan, C., Li, B., et al. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2407.15272 · Cited by 23
SAIL-VL2 Technical ReportPreprint
Yin, W., Ye, Y., Shu, F., Liao, Y., Kang, Z., Dong, H., Yu, H., Yang, D., Wang, J., Wang, H., et al. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2509.14033 · Cited by 24
CausalVLBench: Benchmarking Visual Causal Reasoning in Large Vision-Language Models
Komanduri, A., Bhaila, K., & Wu, X. (2025) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2025.emnlp-main.1561 · Cited by 22
Can Large Multimodal Models Understand Agricultural Scenes? Benchmarking with AgroMind
Li, Q., Zhang, Y., Mai, Z., Chen, Y., Lou, S., Huang, H., Zhang, J., Zhang, Z., Wen, Y., Li, W., et al. (2025) · neural information processing systems · doi:10.52202/085713-5271 · Cited by 22
Cao, Z., Zhang, J., & Zhang, R. (2025) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51701.2025.00023 · Cited by 22
Progress-Aware Video Frame Captioning
Xue, Z., An, J., Yang, X., & Grauman, K. L. (2025) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52734.2025.01273 · Cited by 22
Leopard: A Vision Language Model For Text-Rich Multi-Image Tasks
Jia, M., Yu, W., Ma, K., Fang, T., Zhang, Z., Ouyang, S., Zhang, H., Yu, D., & Jiang, M. (2025) · Transactions on machine learning research · doi:10.48550/arxiv.2410.01744 · Cited by 22
Vision-Language Models Create Cross-Modal Task RepresentationsPreprint
Luo, G., Darrell, T., & Bar, A. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2410.22330 · Cited by 22
Learning to See Before Seeing: Demystifying LLM Visual Priors from Language Pre-training
Han, J., Tong, S., Fan, D., Ren, Y., Sinha, K., Torr, P., & Kokkinos, F. (2025) · International Conference on Learning Representations · doi:10.48550/arxiv.2509.26625 · Cited by 22
Wang, J., Jiang, H., Liu, Y., Ma, C., Zhang, X., Pan, Y., Liu, M., Gu, P., Xia, S., Chen, L., et al. (2024) · arXiv · doi:10.48550/arxiv.2408.01319 · Cited by 21
EE-MLLM: A Data-Efficient and Compute-Efficient Multimodal Large Language ModelPreprint
Ma, F., Zhou, Y., Zhang, Z., Yan, S., Li, H., He, Z., Wu, S., Rao, F., Zhang, Y., & Sun, X. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2408.11795 · Cited by 21
POINTS: Improving Your Vision-language Model with Affordable StrategiesPreprint
Liu, Y., Zhao, Z., Zhuang, Z., Tian, L., Zhou, X. F., & Zhou, J. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2409.04828 · Cited by 21
POINTS1.5: Building a Vision-Language Model towards Real World ApplicationsPreprint
Liu, Y., Tian, L., Zhou, X., Gao, X., Yu, K., Yang, Y., & Zhou, J. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2412.08443 · Cited by 21
Lee, S., Yoon, S., Bui, T., Shi, J., & Yoon, S. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2412.15484 · Cited by 22
Why are Visually-Grounded Language Models Bad at Image Classification?
Zhang, Y., Unell, A., Wang, X., Ghosh, D., Su, Y., Schmidt, L., & Yeung-Levy, S. (2024) · Advances in Neural Information Processing Systems 37 · doi:10.52202/079017-1639 · Cited by 20
Image Captioning Evaluation in the Age of Multimodal LLMs: Challenges and Future Perspectives
Sarto, S., Cornia, M., & Cucchiara, R. (2025) · International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2025/1180 · Cited by 20
Weaving Context Across Images: Improving Vision-Language Models through Focus-Centric Visual Chains
Zhang, J., Cheng, C., Liu, Y., Liu, W., Luan, J., & Yan, R. (2025) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2025.acl-long.1347 · Cited by 20
Data or Language Supervision: What Makes CLIP Better than DINO?
Liu, Y., Zhang, Y., Ghosh, D., Schmidt, L., & Yeung, S. Y. (2025) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2025.findings-emnlp.98 · Cited by 19
PVChat: Personalized Video Chat with One-Shot Learning
Shi, Y., Yan, W., Xu, G., Li, Y., Chen, Y., Li, Z., Yu, F. R., Li, M., & Yeo, S. Y. (2025) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51701.2025.02165 · Cited by 19
Lin, J., Chen, H., Zhu, D., & Shen, X. (2024) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2024.emnlp-main.325 · Cited by 19
Videogamebunny: Towards Vision Assistants for Video Games
Taesiri, M. R., & Bezemer, C. (2025) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv61041.2025.00144 · Cited by 19
From Text to Pixel: Advancing Long-Context Understanding in MLLMsPreprint
Lu, Y., Li, X., Fu, T.-J., Eckstein, M. P., & Wang, W. Y. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2405.14213 · Cited by 19
Wu, S., Zhu, K., Bai, Y., Liang, Y., Li, Y., Wu, H., Liu, J. H., Liu, R., Qu, X., Cheng, X., et al. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2407.17379 · Cited by 19
Large-vocabulary forensic pathological analyses via prototypical cross-modal contrastive learning
Shen, C., Lian, C., Zhang, W., Wang, F., Zhang, J., Fan, S., Wei, X., Wang, G., Li, K., Mu, H., et al. (2025) · Nature Communications · doi:10.1038/s41467-025-62060-x · Cited by 18
Survey on Question Answering over Visually Rich Documents: Methods, Challenges, and TrendsPreprint
Barboule, C., Piwowarski, B., & Chabot, Y. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2501.02235 · Cited by 18
Open-Qwen2VL: Compute-Efficient Pre-Training of Fully-Open Multimodal LLMs on Academic ResourcesPreprint
Wang, W., Tian, Y., Yang, L., Wang, H., & Yan, X. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2504.00595 · Cited by 18
PaliGemma: A versatile 3B VLM for transfer
Beyer, L., Steiner, A., Pinto, A. S., Kolesnikov, A., Wang, X., Salz, D., Neumann, M., Alabdulmohsin, I., Tschannen, M., Bugliarello, E., et al. (2024) · arXiv · doi:10.48550/arxiv.2407.07726 · Cited by 17
Metadata
- Authors
- Laurençon, H., Tronchon, L., Cord, M., & Sanh, V.
- ORCID
- Cord, Matthieu
- Venue
- Advances in Neural Information Processing Systems 37
- Published
- 2024
- Pages
- 87874-87907
- ISSN
- 1049-5258 (ISSN-L)
- OpenAlex
- W4415796627
- Cited by
- 66
- References
- 0
- Citations by year
- 2026: 39 · 2025: 23 · 2024: 4
- FWCI
- 13.67
- Type
- conference paper
- Language
- en
- Open access
- green
- arxiv.org
- Licence
- cc-by
- Locations
- neural information processing systems (publishedVersion)
- arXiv (Cornell University) (submittedVersion, cc-by)
- arXiv (Cornell University) (cc-by)
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Multimodal Machine Learning Applications
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Keywords
- vision-language models, model design, multimodal representation learning
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.