What matters when building vision-language models?
Open accessThe DOI matches its Crossref record; 9 of the 9 references it cites were found in a registry or an index.
Abstract
The growing interest in vision-language models (VLMs) has been driven by improvements in large language models and vision transformers. Despite the abundance of literature on this subject, we observe that critical decisions regarding the design of VLMs are often not justified. We argue that these unsupported decisions impede progress in the field by making it difficult to identify which choices improve model performance. To address this issue, we conduct extensive experiments around pre-trained models, architecture choice, data, and training methods. Our consolidation of findings includes the development of Idefics2, an efficient foundational VLM of 8 billion parameters. Idefics2 achieves state-of-the-art performance within its size category across various multimodal benchmarks, and is often on par with models four times its size. We release the model (base, instructed, and chat) along with the datasets created for its training.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Citations
Cites
The publisher hasn’t deposited this paper’s reference list.
Cited by
Items 301–400 of 660
Chitranuvad: Adapting Multi-lingual LLMs for Multimodal Translation
Khan, S., Tarun, A. K., Faraz, A., Kamble, P., Dahiya, V., Pokala, P. K., Kulkarni, A., Khatri, C., Ravi, A., & Agarwal, S. (2024) · Conference on Machine Translation (WMT) · doi:10.18653/v1/2024.wmt-1.80 · Cited by 8
Preference-Optimized Retrieval and Ranking for Efficient Multimodal Recommendation
Yue, Z., Zeng, H., Wang, Y., McAuley, J., & Wang, D. (2025) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/3711896.3737088 · Cited by 8
Human-centered evaluation of data visualization understanding in vision-language models
Verma, A., Mukherjee, K., Potts, C., Kreiss, E., & Fan, J. E. (2026) · Scientific Reports · doi:10.1038/s41598-026-68119-z · Cited by 8
Analyzing Fine-Tuning Representation Shift for Multimodal LLMs Steering
Khayatan, P., Shukor, M., Parekh, J., Dapogny, A., & Cord, M. (2025) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51701.2025.00213 · Cited by 8
Self-citationKim, H., Ahn, S., & Seo, Y.-D. (2025) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51701.2025.01595 · Cited by 8
BREEN: Bridge Data-Efficient Encoder-Free Multimodal Learning with Learnable Queries
Li, T., Rao, Y., Hu, W., & Cheng, Y. (2026) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv61042.2026.00522 · Cited by 8
MMR: Evaluating Reading Ability of Large Multimodal ModelsPreprint
Chen, J., Zhang, R., Zhou, Y., Rossi, R. A., Gu, J., & Chen, C. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2408.14594 · Cited by 8
Vision language models are unreliable at trivial spatial cognitionPreprint
Khemlani, S., Tran, T., Gyory, N., Harrison, A. M., Lawson, W., Thielstrom, R., Thompson, H., Singh, T., & Trafton, J. G. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2504.16061 · Cited by 8
Multimodal Large Language Models for Text-rich Image Understanding: A Comprehensive ReviewPreprint
Fu, P., Guan, T.-K., Wang, Z.-N., Guo, Z.-T., Duan, C., Sun, H., Chen, B.-M., Jiang, Q., Ma, J.-Y., Zhou, K., et al. (2025) · Annual Meeting of the Association for Computational Linguistics · doi:10.48550/arxiv.2502.16586 · Cited by 19
Vision-Speech Models: Teaching Speech Models to Converse about ImagesPreprint
Royer, A., Böhle, M., de Marmiesse, G., Mazaré, L., Zeghidour, N., Défossez, A., & Pérez, P. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2503.15633 · Cited by 8
Baseer: A Vision-Language Model for Arabic Document-to-Markdown OCRPreprint
Hennara, K., Hreden, M., Hamed, M. M., Bastati, A., Aldallal, Z., Chrouf, S. M. B., & AlModhayan, S. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2509.18174 · Cited by 8
Why Representation Engineering Works: A Theoretical and Empirical Study in Vision-Language ModelsPreprint
Tian, B., Lyu, X., Liu, M., Wang, H., & Li, A. A. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2503.22720 · Cited by 8
CoMemo: LVLMs Need Image Context with Image MemoryPreprint
Liu, S., Su, W., Zhu, X., Wang, W., & Dai, J. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2506.06279 · Cited by 8
Chitranuvad: Adapting Multi-Lingual LLMs for Multimodal TranslationPreprint
Khan, S., Tarun, A. K., Faraz, A., Kamble, P., Dahiya, V., Pokala, P., Kulkarni, A., Khatri, C., Ravi, A., & Agarwal, S. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2502.20420 · Cited by 8
Chitrarth: Bridging Vision and Language for a Billion PeoplePreprint
Khan, S., Tarun, A. K., Ravi, A., Faraz, A., Patidar, A., Pokala, P. K., Bhangare, A., Kolla, R., Khatri, C., & Agarwal, S. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2502.15392 · Cited by 8
From Chatbot to Digital Colleague: The Paradigm Shift Toward Persistent Autonomous AIPreprint
Zhang, Y., Liu, Z., Zhu, J., Wang, S., Chen, X., Huang, H., Kuang, J., Chen, S., Shen, A., Wu, H., et al. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2606.14502 · Cited by 8
ColPali: Efficient Document Retrieval with Vision Language Models
Faysse, M., Sibille, H., Wu, T., Omrani, B., Viaud, G., Hudelot, C., & Colombo, P. (2024) · arXiv · doi:10.48550/arxiv.2407.01449 · Cited by 7
Where and What Matters: Sensitivity-Aware Task Vectors for Many-Shot Multimodal In-Context Learning
Ma, Z., Gou, C., Hu, Y., Wang, Y., Zhuang, B., & Cai, J. (2026) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v40i10.37733 · Cited by 7
ProactiveBench: Benchmarking Proactiveness in Multimodal Large Language Models
Min, T. D., Roy, S., Lathuilière, S., Ricci, E., & Mancini, M. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-37235-2_30 · Cited by 7
Benchmarking Multimodal Large Language Models for Face Recognition
Shahreza, H. O., & Marcel, S. (2026) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp55912.2026.11463782 · Cited by 7
ComiCap: A VLMs Pipeline for Dense Captioning of Comic Panels
Vivoli, E., Biondi, N., Bertini, M., & Karatzas, D. (2025) · Lecture notes in computer science · doi:10.1007/978-3-031-92808-6_4 · Cited by 7
Pay More Attention to Images: Numerous Images-Oriented Multimodal Summarization
Xiao, M., Zhu, J., Zhai, F., Zong, C., & Zhou, Y. (2025) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/2025.naacl-long.474 · Cited by 7
Beyond Perception: Evaluating Abstract Visual Reasoning through Multi-Stage Task
Jiang, Y., Ding, Y., Lei, C., Ao, J., Lau, J. H., & Ehinger, K. A. (2025) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2025.findings-acl.2 · Cited by 8
SMMILE: An expert-driven benchmark for multimodal medical in-context learning
Rieff, M., Varma, M., Rabow, O., Adithan, S., Kim, J. J., Chang, K., Lee, H., Rohatgi, N., Blüthgen, C., Muneer, M. S., et al. (2025) · neural information processing systems · doi:10.52202/085713-2180 · Cited by 7
Zhang, J., Fan, Q., Yu, Y., & Zhang, Y. (2025) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3746027.3755357 · Cited by 7
Wei, J., Su, Y., Xu, K., Zeng, L., Liu, B., & Wang, H. (2024) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3664647.3688995 · Cited by 7
Yang, Q., Yan, W., & Agrawal, A. (2024) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2024.emnlp-main.211 · Cited by 7
A High-Quality Text-Rich Image Instruction Tuning Dataset via Hybrid Instruction GenerationPreprint
Zhou, S., Zhang, R., Zhou, Y., & Chen, C. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2412.16364 · Cited by 7
PRISM-Bench: A Benchmark of Puzzle-Based Visual Tasks with CoT Error DetectionPreprint
Qian, Y., Wan, C., Jia, C., Yang, Y., Zhao, Q., & Gan, Z. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2510.23594 · Cited by 7
Diagnosing Bottlenecks in Data Visualization Understanding by Vision-Language ModelsPreprint
Tartaglini, A. R., Grant, S., Wurgaft, D., Potts, C., & Fan, J. E. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2510.21740 · Cited by 8
Multimodal Latent Reasoning via Hierarchical Visual Cues InjectionPreprint
Zhang, Y., Yan, Q., Jiang, B., & Han, K. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2602.05359 · Cited by 7
Liang, Y., Li, X., Chen, X., Chen, H., Zheng, Y., Liu, Z., Zhu, R., Li, B., & Xue, X. (2026) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2026.3663658 · Cited by 6
On the Perception Bottleneck of VLMs for Chart Understanding
Liu, J., Zeng, W., Zhang, X., Wang, Y., Shan, Z., & He, J. (2025) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2025.findings-emnlp.573 · Cited by 6
Test-Time Consistency in Vision Language Models
Chou, S.-H., Chandhok, S., Little, J. J., & Sigal, L. (2026) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv61042.2026.00752 · Cited by 6
Guo, T., Logan, P. A., Wackwitz, T., & MARTIN, D. J. (2025) · Lecture notes in computer science · doi:10.1007/978-981-95-4972-6_19 · Cited by 6
Buehler, M. J. (2026) · Nano Futures · doi:10.1088/2399-1984/ae4152 · Cited by 6
Mishra, A., Li, M., Fu, H., Noh, R., & Kim, M. K. M. (2025) · International Joint Conference on Natural Language Processing and Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics (IJCNLP-AACL) · doi:10.18653/v1/2025.ijcnlp-long.154 · Cited by 6
Bolaños-Burgos, F., Salatino, A. A., Osborne, F., & Motta, E. (2026) · PeerJ Computer Science · doi:10.7717/peerj-cs.3921 · Cited by 6
CANVAS: Commonsense-Aware Navigation System for Intuitive Human-Robot Interaction
Choi, S., Cho, Y., Kim, M., Jae-Yoon, J., Joe, M., Park, Y., Kim, M.-S., Kim, S., Lee, S., Park, H., et al. (2025) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra55743.2025.11127664 · Cited by 6
CIVET: Systematic Evaluation of Understanding in VLMs
Rizzoli, M., Alghisi, S., Khomyn, O., Roccabruna, G., Mousavi, S. M., & Riccardi, G. (2025) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2025.findings-emnlp.239 · Cited by 6
ViCToR: Improving Visual Comprehension via Token Reconstruction for Pretraining LMMs
Xie, Y., Yang, K., Liang, P., An, X., Zhao, Y., Wang, Y., Feng, Z., Miles, R., Elezi, I., & Deng, J. (2026) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v40i32.39924 · Cited by 6
Reverse Image Retrieval Cues Parametric Memory in Multimodal LLMsPreprint
Xu, J., Moor, M., & Leskovec, J. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2405.18740 · Cited by 6
Exploring the Design Space of Visual Context Representation in Video MLLMs
Du, Y., Huo, Y., Zhou, K., Zhao, Z., Lu, H., Huang, H. P., Zhao, W. X., Wang, B., Chen, W., & Wen, J.-R. (2024) · International Conference on Learning Representations · doi:10.48550/arxiv.2410.13694 · Cited by 6
Can Vision Language Models Learn from Visual Demonstrations of Ambiguous Spatial Reasoning?Preprint
Zhao, B., Dirac, L., & Varshavskaya, P. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2409.17080 · Cited by 6
Seeing is Deceiving: Exploitation of Visual Pathways in Multi-Modal Language ModelsPreprint
Janowczyk, P., Laurier, L., Giulietta, A., Octavia, A., & Cleti, M. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2411.05056 · Cited by 6
MMGenBench: Fully Automatically Evaluating LMMs from the Text-to-Image Generation PerspectivePreprint
Huang, H., Wang, Y., Huang, Z., Li, H., Huang, T., Chu, X., & Zhang, R. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2411.14062 · Cited by 6
Efficient Test-Time Scaling for Small Vision-Language Models
Kaya, M. O., Elliott, D., & Papadopoulos, D. P. (2025) · International Conference on Learning Representations · doi:10.48550/arxiv.2510.03574 · Cited by 6
Jarvis: Towards Personalized AI Assistant via Personal KV-Cache RetrievalPreprint
Xu, B., Feng, J., Lu, S., Luo, Y., Yan, S., Liang, H., Lu, M., & Zhang, W. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2510.22765 · Cited by 6
AIGVE-MACS: Unified Multi-Aspect Commenting and Scoring Model for AI-Generated Video EvaluationPreprint
Liu, X., & Zhang, J. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2507.01255 · Cited by 6
PairBench: Are Vision-Language Models Reliable at Comparing What They See?Preprint
Feizi, A., Rajeswar, S., Romero, A., Rabbany, R., Zantedeschi, V., Gella, S., & Monteiro, J. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2502.15210 · Cited by 8
ChartAB: A Benchmark for Chart Grounding & Dense AlignmentPreprint
Bansal, A., Soselia, D., Nguyen, D., & Zhou, T. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2510.26781 · Cited by 6
He, H., Geng, Z., & Peng, Y. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2602.07605 · Cited by 6
Wang, J., Xu, H., Zhang, X., Zhu, Z., Yan, M., Ye, J., & Sang, J. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2605.29324 · Cited by 6
Review of vision-based surface defect inspection methods with incomplete annotations
Ye, B., Kang, D., Xiaohua, X., & Jianhuang, L. (2025) · Journal of Image and Graphics · doi:10.11834/jig.240434 · Cited by 5
mOSCAR: A Large-scale Multilingual and Multimodal Document-level Corpus
Futeral, M., Zebaze, A. R., Suárez, P. O., Abadji, J., Lacroix, R., Schmid, C., Bawden, R., & Sagot, B. (2025) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2025.findings-acl.180 · Cited by 5
Öztürk, M. G., Şahin, D. Ö., & Kılıç, E. (2025) · IEEE Access · doi:10.1109/access.2025.3614147 · Cited by 5
Zooming from Context to Cue: Hierarchical Preference Optimization for Multi-Image MLLMs
Li, X., Zhang, M., Chen, P., Zheng, X., Zhang, Y., Zheng, J., Shen, Y., Li, K., Fu, C., Sun, X., et al. (2025) · neural information processing systems · doi:10.52202/085713-0008 · Cited by 5
Rezaei, Z., Samghabadi, S. S., & Banad, Y. M. (2025) · Computers in Biology and Medicine · doi:10.1016/j.compbiomed.2025.111397 · Cited by 5
Grainge, O., Waheed, S., Stilgoe, J., Milford, M. J., & Ehsan, S. (2025) · Proceedings of the AAAI Symposium Series · doi:10.1609/aaaiss.v7i1.36882 · Cited by 5
Multimodal Adaptive Retrieval Augmented Generation through Internal Representation Learning
Du, R., Sun, X., Liu, Q., Song, B., Chen, Z., Wang, W., & Wang, L. (2026) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn69854.2026.11729643 · Cited by 5
Rajabi, M. S., Ojelade, A., Kim, S., & Nussbaum, M. A. (2026) · Applied Ergonomics · doi:10.1016/j.apergo.2026.104831 · Cited by 5
UVU: Improving Multimodal Understanding via Vision-Language Unified Autoregressive Paradigm
Kan, Z., Jiang, X., Liu, Y., Liu, Y., Yang, X., Wei, Z., Liu, S., Liao, Q., Yang, W., Li, X., et al. (2026) · Enlighten: Publications (University of Glasgow) · doi:10.48550/arxiv.2609.27915 · Cited by 5
Chitrarth: Bridging Vision and Language for a Billion People
Khan, S., Tarun, A. K., Ravi, A., Faraz, A., Pokala, P. K., Bhangare, A., Kolla, R., Khatri, C., & Agarwal, S. (2025) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp49660.2025.10888601 · Cited by 5
PATFinger: Prompt-Adapted Transferable Fingerprinting against Unauthorized Multimodal Dataset Usage
Zhang, W., Jia, J., Jia, X., Huang, Y., Li, X., Wu, C., & Wang, L. (2025) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/3726302.3730054 · Cited by 5
Generative Visual Instruction TuningPreprint
Hernandez, J., Villegas, R., & Ordóñez, V. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2406.11262 · Cited by 5
VGA: Vision GUI Assistant -- Minimizing Hallucinations through Image-Centric Fine-TuningPreprint
Meng, Z., Dai, Y. M., Gong, Z., Guo, S., Tang, M., & Wei, T. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2406.14056 · Cited by 5
MUMU: Bootstrapping Multimodal Image Generation from Text-to-Image DataPreprint
Berman, W., & Peysakhovich, A. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2406.18790 · Cited by 5
Evaluating Linguistic Capabilities of Multimodal LLMs in the Lens of Few-Shot LearningPreprint
Dogan, M., Kesen, İ., Calixto, I., Erdem, A., & Erdem, E. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2407.12498 · Cited by 5
SMIR: Efficient Synthetic Data Pipeline To Improve Multi-Image ReasoningPreprint
Li, A. C., Thapa, R., Chalamala, R., Wu, Q., Chen, K., & Zou, J. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2501.03675 · Cited by 5
Research, M., :, Hsu, C.-J., Liu, C.-S., Chen, M.-H., Chen, M., Hsu, P.-C., Chen, Y.-C., & Shiu, D.-S. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2501.13921 · Cited by 5
Mishra, A., Li, M., Fu, H. X., Noh, R., & Kim, M. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2502.14780 · Cited by 5
Learning User Preferences for Image Generation ModelPreprint
Mo, W., Ba, Y., Zhang, T., Bai, Y., & Li, B. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2508.08220 · Cited by 5
Amebley, D., & Dibbo, S. V. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2511.20710 · Cited by 5
IGen: Scalable Data Generation for Robot Learning from Open-World ImagesPreprint
Gu, C., Kang, H., Lin, J., Wang, J., Wu, D., Xie, S., Huang, F., Ge, J., Gong, Z., Li, L., et al. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2512.01773 · Cited by 5
Visual Language Models as Zero-Shot Deepfake DetectorsPreprint
Pirogov, V. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2507.22469 · Cited by 5
MultiMat: Multimodal Program Synthesis for Procedural Materials using Large Multimodal ModelsPreprint
Belouadi, J., Boubekeur, T., & Kaiser, A. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2509.22151 · Cited by 5
What do vision-language models see in the context? Investigating multimodal in-context learningPreprint
Santos, G. O. D., Colombini, E. L., & Avila, S. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2510.24331 · Cited by 5
Multimodal Language Models Cannot Spot Spatial InconsistenciesPreprint
Khangaonkar, O., Jamali‐Rad, H., & Pirsiavash, H. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2604.00799 · Cited by 5
ChartMimic: Evaluating LMM's Cross-Modal Reasoning Capability via Chart-to-Code Generation
Yang, C., Shi, C., Liu, Y., Shui, B., Wang, J., Jing, M., Xu, L., Zhu, X., Li, S., Zhang, Y., et al. (2024) · arXiv · doi:10.48550/arxiv.2406.09961 · Cited by 4
Evaluating Multimodal Generative AI with Korean Educational Standards
Park, S., & Kim, G. (2025) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/2025.naacl-short.56 · Cited by 4
How Humans and LLMs Organize Conceptual Knowledge: Exploring Subordinate Categories in Italian
Pedrotti, A., Rambelli, G., Villani, C., & Bolognesi, M. (2025) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2025.acl-long.224 · Cited by 4
MATATA: Weakly Supervised End-to-End MAthematical Tool-Augmented Reasoning for Tabular Applications
Vinayagame, V., Senay, G., & Martí, L. (2025) · Lecture notes in computer science · doi:10.1007/978-3-032-04614-7_24 · Cited by 4
AutoJudger: An Agent-Driven Framework for Efficient Benchmarking of MLLMs
Ding, X., Pan, C., Li, Z., Zhang, J., Wang, S., & Wei, Z. (2026) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2026.acl-long.685 · Cited by 4
Instruction-Following Evaluation of Large Vision-Language Models
Shiono, D., Miyawaki, S., Tanaka, R., & Suzuki, J. (2026) · New Generation Computing · doi:10.1007/s00354-025-00308-2 · Cited by 4
Magic-MM-Embedding: Towards Visual-Token-Efficient Universal Multimodal Embedding with MLLMs
Li, Q., Zhao, Y., Zhou, Y., Wang, Y., Yang, Y., Zhou, Y., & Liu, J. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-37152-2_32 · Cited by 4
ThermEval: A Structured Benchmark for Evaluation of Vision-Language Models on Thermal Imagery
Shrivastava, A., Gangani, K., Jain, L., Goel, M., & Batra, N. (2026) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/3770855.3817469 · Cited by 4
Kim, M., Kim, T., Chung, J., Choi, H., Min, S., Lim, J., & Park, S. (2025) · IEEE/ACM International Conference on Software Engineering: Software Engineering in Society (ICSE-SEIS) · doi:10.1109/icse-seis66351.2025.00022 · Cited by 4
Beyond Sight: Towards Cognitive Alignment in LVLM via Enriched Visual Knowledge
Zhao, Y., Yin, Y., Li, L., Lin, M., Huang, V. S.-J., Chen, S., Chen, W., Yin, B., Zhou, Z., & Zhang, W. (2025) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52734.2025.02323 · Cited by 4
MMAT-1M: A Large Reasoning Dataset for Multimodal Agent Tuning
Gao, T., Fu, Y., Wu, W., Yue, H., Liu, S., & Zhang, G. (2025) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51701.2025.00146 · Cited by 4
Towards Multi-Modal Mastery: A 4.5B Parameter Truly Multi-Modal Small Language Model
Koska, B., & Horváth, M. (2024) · International Conference on Foundation and Large Language Models (FLLM) · doi:10.1109/fllm63129.2024.10852430 · Cited by 4
AITQE: An Adaptive Image–Text Quality Enhancer for Scalable MLLM Pretraining
Huang, H., Huo, Y., Zhao, Z., Lu, H., Wu, S., Wang, B., Liu, Q., Chen, W., & Wang, L. (2026) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2026.3657433 · Cited by 4
Using Game Play to Investigate Multimodal and Conversational Grounding in Large Multimodal ModelsPreprint
Hakimov, S., Abdullayeva, Y., Koshti, K., Schmidt, A., Weiser, Y., Beyer, A., & Schlangen, D. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2406.14035 · Cited by 4
E-THER: A Multimodal Dataset for Empathic AI -- Towards Emotional Mismatch AwarenessPreprint
Tahir, S. A., Johnson, J., Abu-Khalaf, J., & Shah, S. A. A. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2509.02100 · Cited by 4
NoTeS-Bank: Benchmarking Neural Transcription and Search for Scientific Notes UnderstandingPreprint
Pal, A., Biswas, S., Das, A., Lodh, A., Banerjee, P., Chattopadhyay, S., Karatzas, D., Lladós, J., & Jawahar, C. V. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2504.09249 · Cited by 4
Seeing Straight: Document Orientation Detection for Efficient OCRPreprint
Goswami, S., Ravi, A., Kolla, R., Faraz, A., Khan, S., Akash, Khatri, C., & Agarwal, S. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2511.04161 · Cited by 4
VISOR: Visual Input-based Steering for Output Redirection in Vision-Language ModelsPreprint
Phute, M., & Balakrishnan, R. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2508.08521 · Cited by 4
jina-vlm: Small Multilingual Vision Language ModelPreprint
Koukounas, A., Mastrapas, G., Hönicke, F., Eslami, S., Roncari, G., & Xiao, H. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2512.04032 · Cited by 4
Aligning Text, Images, and 3D Structure Token-by-TokenPreprint
Sahoo, A., Tibrewal, V., & Gkioxari, G. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2506.08002 · Cited by 4
SAILViT: Towards Robust and Generalizable Visual Backbones for MLLMs via Gradual Feature RefinementPreprint
Yin, W., Yang, D., Dong, H., Kang, Z., Wang, J., Liang, X. M., Feng, C., & Ran, J. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2507.01643 · Cited by 4
Efficient Motion-Aware Video MLLMPreprint
Zhao, Z., Huo, Y., Yue, T., Guo, L., Lu, H., Wang, B., Chen, W., & Liu, J. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2503.13016 · Cited by 4
Metadata
- Authors
- Laurençon, H., Tronchon, L., Cord, M., & Sanh, V.
- ORCID
- Cord, Matthieu
- Venue
- Advances in Neural Information Processing Systems 37
- Published
- 2024
- Pages
- 87874-87907
- ISSN
- 1049-5258 (ISSN-L)
- OpenAlex
- W4415796627
- Cited by
- 66
- References
- 0
- Citations by year
- 2026: 39 · 2025: 23 · 2024: 4
- FWCI
- 13.67
- Type
- conference paper
- Language
- en
- Open access
- green
- arxiv.org
- Licence
- cc-by
- Locations
- neural information processing systems (publishedVersion)
- arXiv (Cornell University) (submittedVersion, cc-by)
- arXiv (Cornell University) (cc-by)
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Multimodal Machine Learning Applications
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Keywords
- vision-language models, model design, multimodal representation learning
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.