Intern VL: Scaling up Vision Foundation Models and Aligning for Generic Visual-Linguistic Tasks
Open accessAbstract
The exponential growth of large language models (LLMs) has opened up numerous possibilities for multi-modal AGI systems. However, the progress in vision and vision-language foundation models, which are also critical elements of multi-modal AGI, has not kept pace with LLMs. In this work, we design a large-scale vision-language foun-dation model (Intern VL), which scales up the vision foun-dation model to 6 billion parameters and progressively aligns it with the LLM, using web-scale image-text data from various sources. This model can be broadly applied to and achieve state-of-the-art performance on 32 generic visual-linguistic benchmarks including visual perception tasks such as image-level or pixel-level recognition, vision-language tasks such as zero-shot image/video classification, zero-shot image/video-text retrieval, and link with LLMs to create multi-modal dialogue systems. It has powerful visual capabilities and can be a good alternative to the ViT-22B. We hope that our research could contribute to the development of multi-modal large models.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Context-aware modeling is widely used in natural language processing, computer vision, and recommendation systems, where it can substantially boost system performance and user experience.
“Context-aware modeling is widely used across various domains, including natural language processing [219], computer vision [220], and recommendation systems [221], where it can substantially boost both system performance and user experience.”
From Multimodal fusion and vision–language models: A survey for robot vision (2026) · cites this paper as [219]
Cites
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474 · see citing works
ImageNet classification with deep convolutional neural networks
Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017) · Communications of the ACM · doi:10.1145/3065386 · Cited by 112,155 · see citing works
ImageNet: A large-scale hierarchical image database
Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., & Fei-Fei, L. (2009) · 2009 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2009.5206848 · Cited by 65,333 · see citing works
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2019) · Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) · doi:10.18653/v1/n19-1423 · Cited by 48,293 · see citing works
Squeeze-and-Excitation Networks
Hu, J., Shen, L., & Sun, G. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00745 · Cited by 35,667 · see citing works
Swin Transformer: Hierarchical Vision Transformer using Shifted Windows
Liu, Z., Lin, Y., Cao, Y., Hu, H., Wei, Y., Zhang, Z., Lin, S., & Guo, B. (2021) · 2021 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00986 · Cited by 35,534 · see citing works
RoBERTa: A Robustly Optimized BERT Pretraining Approach
Liu, Y., Ott, M., Goyal, N., Du, J., Joshi, M., Chen, D., Levy, O., Lewis, M., Zettlemoyer, L., & Stoyanov, V. (2019) · arXiv · doi:10.48550/arxiv.1907.11692 · Cited by 16,515 · see citing works
Learning Transferable Visual Models From Natural Language Supervision
Radford, A., Kim, J. W., Hallacy, C., Ramesh, A., Goh, G., Agarwal, S., Sastry, G., Askell, A., Mishkin, P., Clark, J., et al. (2021) · arXiv · doi:10.48550/arxiv.2103.00020 · Cited by 15,293 · see citing works
Aggregated Residual Transformations for Deep Neural Networks
Xie, S., Girshick, R., Dollar, P., Tu, Z., & He, K. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.634 · Cited by 12,465 · see citing works
Quo Vadis, Action Recognition? A New Model and the Kinetics Dataset
Carreira, J., & Zisserman, A. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.502 · Cited by 9,570 · see citing works
Liu, Z., Mao, H., Wu, C.-Y., Feichtenhofer, C., Darrell, T., & Xie, S. (2022) · 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52688.2022.01167 · Cited by 8,494 · see citing works
Masked Autoencoders Are Scalable Vision Learners
He, K., Chen, X., Xie, S., Li, Y., Dollar, P., & Girshick, R. (2022) · 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52688.2022.01553 · Cited by 8,489 · cited 1× · see citing works
“Following common practices [30, 44, 84], we adopt the linear probing evaluation, i.e . training a linear classifier while keeping the backbone frozen.”
MethodDeformable Convolutional Networks
Dai, J., Qi, H., Xiong, Y., Li, Y., Zhang, G., Hu, H., & Wei, Y. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.89 · Cited by 7,513 · see citing works
Self-citationPyramid Vision Transformer: A Versatile Backbone for Dense Prediction without Convolutions
Wang, W., Xie, E., Li, X., Fan, D.-P., Song, K., Liang, D., Lu, T., Luo, P., & Shao, L. (2021) · 2021 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00061 · Cited by 5,259 · cited 1× · see citing works
“ViT and its variants [13, 23, 30, 71, 89, 111, 112, 125, 139, 140] have significantly increased their capacity and excelled in various important visual tasks.”
BackgroundSelf-citationLLaMA: Open and Efficient Foundation Language Models
Touvron, H., Lavril, T., Izacard, G., Martinet, X., Lachaux, M.-A., Lacroix, T., Rozière, B., Goyal, N., Hambro, E., Azhar, F., et al. (2023) · arXiv · doi:10.48550/arxiv.2302.13971 · Cited by 4,602 · see citing works
Chain-Of-Thought Prompting Elicits Reasoning in Large Language Models
Wei, J., Wang, X., Schuurmans, D., Bosma, M., Ichter, B., Xia, F., Chi, E., Le, Q. V., & Zhou, D. (2022) · Advances in Neural Information Processing Systems 35 · doi:10.52202/068431-1800 · Cited by 4,505 · see citing works
Chain-of-Thought Prompting Elicits Reasoning in Large Language Models
Wei, J., Wang, X., Schuurmans, D., Bosma, M., Ichter, B., Xia, F., Chi, E., Le, Q., & Zhou, D. (2022) · arXiv · doi:10.48550/arxiv.2201.11903 · Cited by 4,263 · see citing works
InstructBLIP: Towards General-purpose Vision-Language Models with Instruction TuningPreprint
Dai, W., Li, J., Li, D., Tiong, A., Zhao, J.-Q., Wang, W., Li, B. A., Fung, P., & Hoi, S. (2023) · Neural Information Processing Systems · doi:10.48550/arxiv.2305.06500 · Cited by 4,191 · see citing works
Self-citationGQA: A New Dataset for Real-World Visual Reasoning and Compositional Question Answering
Hudson, D. A., & Manning, C. D. (2019) · Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2019.00686 · Cited by 3,703 · see citing works
Scene Parsing through ADE20K Dataset
Zhou, B., Zhao, H., Puig, X., Fidler, S., Barriuso, A., & Torralba, A. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.544 · Cited by 3,326 · see citing works
RepVGG: Making VGG-style ConvNets Great Again
Ding, X., Zhang, X., Ma, N., Han, J., Ding, G., & Sun, J. (2021) · 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr46437.2021.01352 · Cited by 2,907 · see citing works
Llama 2: Open Foundation and Fine-Tuned Chat Models
Touvron, H., Martin, L., Stone, K., Albert, P., Almahairi, A., Babaei, Y., Bashlykov, N., Batra, S., Bhargava, P., Bhosale, S., et al. (2023) · arXiv · doi:10.48550/arxiv.2307.09288 · Cited by 2,604 · see citing works
Towards VQA Models That Can Read
Singh, A., Natarajan, V., Shah, M., Jiang, Y., Chen, X., Batra, D., Parikh, D., & Rohrbach, M. (2019) · Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2019.00851 · Cited by 2,473 · see citing works
Pvt v2: Improved baselines with pyramid vision transformer
Wang, W., Xie, E., Li, X., Fan, D.-P., Song, K., Liang, D., Lu, T., Luo, P., & Shao, L. (2022) · Computational Visual Media · doi:10.1007/s41095-022-0274-8 · Cited by 2,439 · see citing works
Self-citationLiu, H., Li, C., Wu, Q., & Lee, Y. J. (2023) · Advances in Neural Information Processing Systems 36 · doi:10.52202/075280-1516 · Cited by 2,428 · see citing works
Unified Perceptual Parsing for Scene Understanding
Xiao, T., Liu, Y., Zhou, B., Jiang, Y., & Sun, J. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-01228-1_26 · Cited by 2,201 · cited 1× · see citing works
“Additionally, Table 5b presents our further verification in three distinct settings, including linear probing, head tuning [122], and full-parameter tuning.”
BackgroundScaling Up Visual and Vision-Language Representation Learning With Noisy Text Supervision
Jia, C., Yang, Y., Xia, Y., Chen, Y.-T., Parekh, Z., Pham, H., Le, Q. V., Sung, Y., Li, Z., & Duerig, T. (2021) · arXiv · doi:10.48550/arxiv.2102.05918 · Cited by 1,977 · cited 2× · see citing works
“Vision models, trained on pure-vision data or aligned with the BERT series [52, 54, 70], often exhibit representation inconsistencies with LLMs. (3) Inefficient connection.”
BackgroundMethodCoCa: Contrastive Captioners are Image-Text Foundation ModelsPreprint
Yu, J.-H., Wang, Z., Vasudevan, V., Yeung, L., Seyedhosseini, M., & Wu, Y.-H. (2022) · Trans. Mach. Learn. Res. · doi:10.48550/arxiv.2205.01917 · Cited by 1,899 · see citing works
Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena
Zheng, L., Chiang, W.-L., Sheng, Y., Zhuang, S., Wu, Z., Zhuang, Y., Lin, Z., Li, Z., Li, D., Xing, E. P., et al. (2023) · arXiv · doi:10.48550/arxiv.2306.05685 · Cited by 1,888 · see citing works
Conceptual Captions: A Cleaned, Hypernymed, Image Alt-text Dataset For Automatic Image Captioning
Sharma, P., Ding, N., Goodman, S., & Soricut, R. (2018) · Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/p18-1238 · Cited by 1,879 · see citing works
Improved Baselines with Visual Instruction Tuning
Liu, H., Li, C., Li, Y., & Lee, Y. J. (2024) · 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52733.2024.02484 · Cited by 1,748 · see citing works
Li, J., Li, D., Xiong, C., & Hoi, S. (2022) · arXiv · doi:10.48550/arxiv.2201.12086 · Cited by 1,741 · see citing works
Microsoft COCO Captions: Data Collection and Evaluation Server
Chen, X., Fang, H., Lin, T.-Y., Vedantam, R., Gupta, S., Dollar, P., & Zitnick, C. L. (2015) · arXiv · doi:10.48550/arxiv.1504.00325 · Cited by 1,614 · see citing works
Flamingo: a Visual Language Model for Few-Shot Learning
Alayrac, J.-B., Donahue, J., Luc, P., Miech, A., Barr, I., Hasson, Y., Lenc, K., Mensch, A., Millican, K., Reynolds, M., et al. (2022) · arXiv · doi:10.48550/arxiv.2204.14198 · Cited by 1,555 · see citing works
Flickr30k Entities: Collecting Region-to-Phrase Correspondences for Richer Image-to-Sentence Models
Plummer, B. A., Wang, L., Cervantes, C. M., Caicedo, J. C., Hockenmaier, J., & Lazebnik, S. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.303 · Cited by 1,444 · see citing works
BEiT: BERT Pre-Training of Image Transformers
Bao, H., Dong, L., Piao, S., & Wei, F. (2021) · arXiv · doi:10.48550/arxiv.2106.08254 · Cited by 1,392 · see citing works
The Many Faces of Robustness: A Critical Analysis of Out-of-Distribution Generalization
Hendrycks, D., Basart, S., Mu, N., Kadavath, S., Wang, F., Dorundo, E., Desai, R., Zhu, T., Parajuli, S., Guo, M., et al. (2021) · 2021 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00823 · Cited by 1,250 · see citing works
Generation and Comprehension of Unambiguous Object Descriptions
Mao, J., Huang, J., Toshev, A., Camburu, O., Yuille, A., & Murphy, K. P. (2016) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2016.9 · Cited by 1,211 · see citing works
Hendrycks, D., Zhao, K., Basart, S., Steinhardt, J., & Song, D. (2021) · 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr46437.2021.01501 · Cited by 1,090 · see citing works
(2016) · doi:10.1007/978-3-319-46475-6_5 · Cited by 1,081 · see citing works
InternImage: Exploring Large-Scale Vision Foundation Models with Deformable Convolutions
Wang, W., Dai, J., Chen, Z., Huang, Z., Li, Z., Zhu, X., Hu, X., Lu, T., Lu, L., Li, H., et al. (2023) · 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52729.2023.01385 · Cited by 1,080 · see citing works
Self-citationThe Dawn of LMMs: Preliminary Explorations with GPT-4V(ision)Preprint
Yang, Z.-Y., Li, L., Lin, K., Wang, J., Lin, C.-C., Liu, Z., & Wang, L.-J. (2023) · arXiv.org · doi:10.48550/arxiv.2309.17421 · Cited by 1,036 · see citing works
GLM: General Language Model Pretraining with Autoregressive Blank Infilling
Du, Z., Qian, Y., Liu, X., Ding, M., Qiu, J., Yang, Z., & Tang, J. (2022) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2022.acl-long.26 · Cited by 1,028 · see citing works
LAION-5B: An open large-scale dataset for training next generation image-text models
Schuhmann, C., Beaumont, R., Vencu, R., Gordon, C., Wightman, R., Cherti, M., Coombes, T., Katta, A., Mullis, C., Wortsman, M., et al. (2022) · arXiv · doi:10.48550/arxiv.2210.08402 · Cited by 1,022 · see citing works
Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsity
Fedus, W., Zoph, B., & Shazeer, N. (2021) · arXiv · doi:10.48550/arxiv.2101.03961 · Cited by 1,005 · see citing works
Vision Transformer Adapter for Dense PredictionsPreprint
Chen, Z., Duan, Y., Wang, W., He, J.-J., Lu, T., Dai, J.-F., & Qiao, Y. (2022) · International Conference on Learning Representations · doi:10.48550/arxiv.2205.08534 · Cited by 959 · see citing works
Self-citationDenseNet: Implementing Efficient ConvNet Descriptor PyramidsPreprint
Iandola, F., Moskewicz, M. W., Karayev, S., Girshick, R., Darrell, T., & Keutzer, K. (2014) · arXiv (Cornell University) · doi:10.48550/arxiv.1404.1869 · Cited by 909 · see citing works
Zhai, X., Kolesnikov, A., Houlsby, N., & Beyer, L. (2022) · 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52688.2022.01179 · Cited by 891 · see citing works
Im2Text: Describing Images Using 1 Million Captioned Photographs
Ordóñez, V., Kulkarni, G. A., & Berg, T. L. (2011) · Cited by 879 · see citing works
MViTv2: Improved Multiscale Vision Transformers for Classification and Detection
Li, Y., Wu, C.-Y., Fan, H., Mangalam, K., Xiong, B., Malik, J., & Feichtenhofer, C. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.00476 · Cited by 837 · see citing works
Do ImageNet Classifiers Generalize to ImageNet?
Recht, B., Roelofs, R., Schmidt, L., & Shankar, V. (2019) · arXiv · doi:10.48550/arxiv.1902.10811 · Cited by 790 · see citing works
OK-VQA: A Visual Question Answering Benchmark Requiring External Knowledge
Marino, K., Rastegari, M., Farhadi, A., & Mottaghi, R. (2019) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2019.00331 · Cited by 748 · see citing works
OCR-VQA: Visual Question Answering by Reading Text in Images
Mishra, A., Shekhar, S., Singh, A., & Chakraborty, A. (2019) · IEEE International Conference on Document Analysis and Recognition · doi:10.1109/icdar.2019.00156 · Cited by 746 · see citing works
VisionLLM: Large Language Model is also an Open-Ended Decoder for Vision-Centric TasksPreprint
Wang, W., Chen, Z., Chen, X., et al. (2023) · Neural Information Processing Systems · doi:10.48550/arxiv.2305.11175 · Cited by 728 · see citing works
Self-citationLiu, H., Li, C., Wu, Q., & Lee, Y. J. (2023) · arXiv · doi:10.48550/arxiv.2304.08485 · Cited by 682 · see citing works
MM-REACT: Prompting ChatGPT for Multimodal Reasoning and ActionPreprint
Yang, Z.-Y., Li, L., Wang, J., Lin, K., Azarnasab, E., Ahmed, F., Liu, Z.-C., Liu, C., Zeng, M., & Wang, L.-J. (2023) · arXiv.org · doi:10.48550/arxiv.2303.11381 · Cited by 660 · see citing works
Conceptual 12M: Pushing Web-Scale Image-Text Pre-Training To Recognize Long-Tail Visual Concepts
Changpinyo, S., Sharma, P. K., Ding, N., & Soricut, R. (2021) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr46437.2021.00356 · Cited by 624 · see citing works
VizWiz Grand Challenge: Answering Visual Questions from Blind People
Gurari, D., Li, Q., Stangl, A. J., Guo, A., Lin, C., Grauman, K., Luo, J., & Bigham, J. P. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00380 · Cited by 621 · see citing works
Video-LLaMA: An Instruction-tuned Audio-Visual Language Model for Video Understanding
Zhang, H., Li, X. S., & Bing, L. (2023) · Conference on Empirical Methods in Natural Language Processing: System Demonstrations (EMNLP) · doi:10.18653/v1/2023.emnlp-demo.49 · Cited by 601 · see citing works
PaLM-E: An Embodied Multimodal Language Model
Driess, D., Xia, F., Sajjadi, M. S. M., Lynch, C., Chowdhery, A., Ichter, B., Wahid, A., Tompson, J., Vuong, Q., Yu, T., et al. (2023) · arXiv · doi:10.48550/arxiv.2303.03378 · Cited by 591 · see citing works
Evaluating Object Hallucination in Large Vision-Language Models
Li, Y., Du, Y., Zhou, K., Wang, J., Xin, Z., & Wen, J.-R. (2023) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2023.emnlp-main.20 · Cited by 580 · see citing works
EVA: Exploring the Limits of Masked Visual Representation Learning at Scale
Fang, Y., Wang, W., Xie, B., Sun, Q., Wu, L., Wang, X., Huang, T., Wang, X., & Cao, Y. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.01855 · Cited by 540 · see citing works
Self-citationHuggingGPT: Solving AI Tasks with ChatGPT and its Friends in Hugging Face
Shen, Y., Song, K., Tan, X., Li, D., Lu, W., & Zhuang, Y. (2023) · arXiv · doi:10.48550/arxiv.2303.17580 · Cited by 477 · see citing works
HuggingGPT: Solving AI Tasks with ChatGPT and its Friends in Hugging Face
Shen, Y., Song, K., Tan, X., Li, D., Lu, W., & Zhuang, Y. (2023) · Advances in Neural Information Processing Systems 36 · doi:10.52202/075280-1657 · Cited by 475 · see citing works
MiniGPT-4: Enhancing Vision-Language Understanding with Advanced Large Language Models
Zhu, D., Chen, J., Shen, X., Li, X., & Elhoseiny, M. (2023) · arXiv · doi:10.48550/arxiv.2304.10592 · Cited by 469 · see citing works
LISA: Reasoning Segmentation via Large Language Model
Lai, X., Tian, Z., Chen, Y., Li, Y., Yuan, Y., Liu, S., & Jia, J. (2024) · 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52733.2024.00915 · Cited by 466 · see citing works
Das, A., Kottur, S., Gupta, K., Singh, A., Yadav, D., Moura, J. M. F., Parikh, D., & Batra, D. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.121 · Cited by 445 · see citing works
VideoChat: chat-centric video understanding
Li, K., He, Y., Wang, Y., Li, Y., Wang, W., Luo, P., Wang, L., Wang, L., & Qiao, Y. (2025) · Science China Information Sciences · doi:10.1007/s11432-024-4321-9 · Cited by 426 · see citing works
Self-citationTouvron, H., Cord, M., & Jégou, H. (2022) · Lecture Notes in Computer Science · doi:10.1007/978-3-031-20053-3_30 · Cited by 409 · see citing works
Simple and Effective Multi-Paragraph Reading Comprehension
Clark, C., & Gardner, M. (2018) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/p18-1078 · Cited by 402 · see citing works
LiT: Zero-Shot Transfer with Locked-image text Tuning
Zhai, X., Wang, X., Mustafa, B., Steiner, A., Keysers, D., Kolesnikov, A., & Beyer, L. (2022) · 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52688.2022.01759 · Cited by 402 · see citing works
GPT4RoI: Instruction Tuning Large Language Model on Region-of-InterestPreprint
Zhang, S., Sun, P., Chen, S., Xiao, M., Shao, W., Zhang, W., Liu, Y., Chen, K., & Luo, P. (2023) · ECCV Workshops · doi:10.48550/arxiv.2307.03601 · Cited by 390 · see citing works
Self-citationFlorence: A New Foundation Model for Computer Vision
Yuan, L., Chen, D., Chen, Y.-L., Codella, N., Dai, X., Gao, J., Hu, H., Huang, X., Li, B., Li, C., et al. (2021) · arXiv · doi:10.48550/arxiv.2111.11432 · Cited by 389 · see citing works
Self-citationChartQA: A Benchmark for Question Answering about Charts with Visual and Logical Reasoning
Masry, A., Long, X., Tan, J. Q., Joty, S., & Hoque, E. (2022) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2022.findings-acl.177 · Cited by 387 · see citing works
nocaps: novel object captioning at scale
Agrawal, H., Desai, K., Wang, Y., Chen, X., Jain, R., Johnson, M., Batra, D., Parikh, D., Lee, S., & Anderson, P. (2019) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00904 · Cited by 386 · see citing works
GPT4Tools: Teaching Large Language Model to Use Tools via Self-instructionPreprint
Yang, R., Song, L., Li, Y., et al. (2023) · Neural Information Processing Systems · doi:10.48550/arxiv.2305.18752 · Cited by 340 · see citing works
Learning Robust Global Representations by Penalizing Local Predictive Power
Wang, H., Ge, S., Xing, E. P., & Lipton, Z. C. (2019) · arXiv · doi:10.48550/arxiv.1905.13549 · Cited by 337 · see citing works
LLaVAR: Enhanced Visual Instruction Tuning for Text-Rich Image UnderstandingPreprint
Zhang, Y., Zhang, R., Gu, J., Zhou, Y., Lipka, N., Yang, D.-Y., & Sun, T.-F. (2023) · arXiv.org · doi:10.48550/arxiv.2306.17107 · Cited by 335 · see citing works
A Short Note on the Kinetics-700-2020 Human Action Dataset
Smaira, L., Carreira, J., Noland, E., Clancy, E., Wu, A., & Zisserman, A. (2020) · arXiv · doi:10.48550/arxiv.2010.10864 · Cited by 334 · see citing works
Bai, J., Bai, S., Chu, Y., Cui, Z., Dang, K., Deng, X., Fan, Y., Ge, W., Han, Y., Huang, F., et al. (2023) · arXiv · doi:10.48550/arxiv.2309.16609 · Cited by 332 · see citing works
Self-citationScene Text Visual Question Answering
Biten, A. F., Tito, R., Mafla, A., Gómez, L., Rusiñol, M., Jawahar, C. V., Valveny, E., & Karatzas, D. (2019) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00439 · Cited by 311 · see citing works
EVA-CLIP: Improved Training Techniques for CLIP at ScalePreprint
Sun, Q., Fang, Y., Wu, L., Wang, X., & Cao, Y. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2303.15389 · Cited by 299 · see citing works
A-OKVQA: A Benchmark for Visual Question Answering Using World Knowledge
Schwenk, D., Khandelwal, A., Clark, C., Marino, K., & Mottaghi, R. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-20074-8_9 · Cited by 295 · see citing works
(2019) · DSpace@MIT (Massachusetts Institute of Technology) · Cited by 291 · see citing works
A Diagram is Worth a Dozen Images
Kembhavi, A., Salvato, M., Kolve, E., Seo, M., Hajishirzi, H., & Farhadi, A. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-46493-0_15 · Cited by 290 · see citing works
ViperGPT: Visual Inference via Python Execution for Reasoning
Surís, D., Menon, S., & Vondrick, C. (2023) · 2023 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.01092 · Cited by 273 · see citing works
Otter: A Multi-Modal Model With In-Context Instruction Tuning
Li, B., Zhang, Y., Chen, L., Wang, J., Pu, F., Cahyono, J. A., Yang, J., Li, C., & Liu, Z. (2025) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2025.3571946 · Cited by 253 · see citing works
Self-citationTextCaps: A Dataset for Image Captioning with Reading Comprehension
Sidorov, O., Hu, R., Rohrbach, M., & Singh, A. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-58536-5_44 · Cited by 242 · see citing works
PaLI: A Jointly-Scaled Multilingual Language-Image ModelPreprint
Chen, X., Wang, X., Changpinyo, S., Piergiovanni, A., Padlewski, P., Salz, D., Goodman, S., Grycner, A., Mustafa, B., Beyer, L., et al. (2022) · arXiv (Cornell University) · doi:10.48550/arxiv.2209.06794 · Cited by 235 · see citing works
ResT: An Efficient Transformer for Visual RecognitionPreprint
Zhang, Q., & Yang, Y.-B. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2105.13677 · Cited by 214 · see citing works
Self-citationLLaMA-Adapter: Efficient Fine-tuning of Language Models with Zero-init AttentionPreprint
Zhang, R., Han, J., Liu, C., Gao, P., Zhou, A., Hu, X., Yan, S., Lu, P., Li, H., & Qiao, Y. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2303.16199 · Cited by 202 · see citing works
Self-citationVisionLLM: Large Language Model is also an Open-Ended Decoder for Vision-Centric Tasks
Wang, W., Chen, Z., Chen, X., Wu, J., Zhu, X., Zeng, G., Luo, P., Lu, T., Zhou, J., Qiao, Y., et al. (2023) · neural information processing systems · doi:10.52202/075280-2688 · Cited by 198 · see citing works
Self-citationVisual ChatGPT: Talking, Drawing and Editing with Visual Foundation Models
Wu, C., Yin, S., Qi, W., Wang, X., Tang, Z., & Duan, N. (2023) · arXiv · doi:10.48550/arxiv.2303.04671 · Cited by 197 · see citing works
Mathew, M., Bagal, V., Tito, R., Karatzas, D., Valveny, E., & Jawahar, C. V. (2022) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv51458.2022.00264 · Cited by 188 · see citing works
Beyer, L., Hénaff, O. J., Kolesnikov, A., Zhai, X., & Oord, A. V. D. (2020) · arXiv · doi:10.48550/arxiv.2006.07159 · Cited by 176 · see citing works
Monkey: Image Resolution and Text Label are Important Things for Large Multi-Modal Models
Li, Z., Yang, B., Liu, Q., Ma, Z., Zhang, S., Yang, J., Sun, Y., Liu, Y., & Bai, X. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.02527 · Cited by 172 · see citing works
Penedo, G., Malartic, Q., Hesslow, D., Cojocaru, R., Cappelli, A., Alobeidli, H., Pannier, B., Almazrouei, E., & Launay, J. (2023) · arXiv · doi:10.48550/arxiv.2306.01116 · Cited by 164 · see citing works
Chinese CLIP: Contrastive Vision-Language Pretraining in ChinesePreprint
An, Y., Pan, J., Lin, J., Men, R., Zhang, Y., Zhou, J., & Zhou, C. (2022) · arXiv (Cornell University) · doi:10.48550/arxiv.2211.01335 · Cited by 162 · see citing works
MME: A Comprehensive Evaluation Benchmark for Multimodal Large Language Models
Fu, C., Chen, P., Shen, Y., Qin, Y., Zhang, M., Xu, L., Yang, J., Zheng, X., Li, K., Sun, X. W., et al. (2025) · neural information processing systems · doi:10.52202/085713-4899 · Cited by 148 · see citing works
GPT4RoI: Instruction Tuning Large Language Model on Region-of-Interest
Zhang, S., Sun, P., Chen, S., Xiao, M., Shao, W., Zhang, W., Liu, Y., Chen, K., & Luo, P. (2025) · Lecture notes in computer science · doi:10.1007/978-3-031-91813-1_4 · Cited by 141 · see citing works
Self-citationNExT-GPT: Any-to-Any Multimodal LLMPreprint
Wu, S., Fei, H., Qu, L., Ji, W., & Chua, T. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2309.05519 · Cited by 132 · see citing works
Kosmos-2: Grounding Multimodal Large Language Models to the WorldPreprint
Peng, Z., Wang, W., Dong, L., Hao, Y., Huang, S., Ma, S., & Wei, F. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2306.14824 · Cited by 129 · see citing works
Self-citationInternVid: A Large-scale Video-Text Dataset for Multimodal Understanding and GenerationPreprint
Wang, Y., He, Y., Li, Y., Li, K., Yu, J., Ma, X., Li, X., Guo, C., Chen, X., Wang, Y., et al. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2307.06942 · Cited by 126 · see citing works
Self-citationBaichuan 2: Open Large-scale Language Models
Yang, A., Xiao, B., Wang, B., Zhang, B., Bian, C., Yin, C., Lv, C., Pan, D., Wang, D., Yan, D., et al. (2023) · arXiv · doi:10.48550/arxiv.2309.10305 · Cited by 125 · see citing works
Self-citationZhang, P., Dong, X., Wang, B., Cao, Y., Xu, C., Ouyang, L., Zhao, Z., Duan, H., Zhang, S., Ding, S., et al. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2309.15112 · Cited by 124 · see citing works
Self-citationScaling vision transformers to 22 billion parameters
Dehghani, M., Djolonga, J., Mustafa, B., Padlewski, P., Heek, J., Gilmer, J., Steiner, A., Caron, M., Geirhos, R., Alabdulmohsin, I., et al. (2023) · arXiv · doi:10.48550/arxiv.2302.05442 · Cited by 112 · see citing works
Fluency-Guided Cross-Lingual Image Captioning
Lan, W., Li, X., & Dong, J. (2017) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3123266.3123366 · Cited by 94 · see citing works
Fengshenbang 1.0: Being the Foundation of Chinese Cognitive IntelligencePreprint
Zhang, J., Gan, R., Wang, J., Zhang, Y., Zhang, L., Yang, P., Gao, X., Wu, Z., Dong, X., He, J., et al. (2022) · arXiv (Cornell University) · doi:10.48550/arxiv.2209.02970 · Cited by 89 · see citing works
Self-citationVideoChat: Chat-Centric Video UnderstandingPreprint
Li, K., He, Y., Wang, Y., et al. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2305.06355 · Cited by 89 · see citing works
Otter: A Multi-Modal Model with In-Context Instruction TuningPreprint
Li, B., Zhang, Y., Chen, L., et al. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2305.03726 · Cited by 86 · see citing works
Self-citationOn the General Value of Evidence, and Bilingual Scene-Text Visual Question Answering
Wang, X., Liu, Y., Shen, C., Ng, C. C., Luo, C., Jin, L., Chan, C. S., van den Hengel, A., & Wang, L. (2020) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr42600.2020.01014 · Cited by 83 · see citing works
ONE-PEACE: Exploring One General Representation Model Toward Unlimited ModalitiesPreprint
Wang, P., Wang, S., Lin, J., Bai, S., Zhou, X., Zhou, J., Wang, X., & Zhou, C. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2305.11172 · Cited by 82 · see citing works
mPLUG-DocOwl: Modularized Multimodal Large Language Model for Document UnderstandingPreprint
Ye, J., Hu, A., Xu, H., Ye, Q., Yan, M., Dan, Y., Zhao, C., Xu, G., Li, C., Tian, J., et al. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2307.02499 · Cited by 74 · see citing works
Emu: Generative Pretraining in MultimodalityPreprint
Sun, Q., Yu, Q., Cui, Y., Zhang, F., Zhang, X., Wang, Y., Gao, H., Liu, J., Huang, T., & Wang, X. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2307.05222 · Cited by 74 · see citing works
Efficient and Effective Text Encoding for Chinese LLaMA and AlpacaPreprint
Cui, Y., Yang, Z., & Yao, X. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2304.08177 · Cited by 71 · see citing works
AltCLIP: Altering the Language Encoder in CLIP for Extended Language Capabilities
Chen, Z., Liu, G., Zhang, B., Yang, Q., & Wu, L. (2023) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2023.findings-acl.552 · Cited by 68 · see citing works
Self-citationWukong: A 100 Million Large-Scale Chinese Cross-Modal Pre-Training Benchmark
Gu, J., Meng, X., Lu, G., Hou, L., Minzhe, N., Liang, X., Yao, L., Huang, R., Zhang, W., Jiang, X., et al. (2022) · neural information processing systems · doi:10.52202/068431-1916 · Cited by 64 · see citing works
DreamLLM: Synergistic Multimodal Comprehension and CreationPreprint
Dong, R., Han, C., Peng, Y., Qi, Z., Ge, Z., Yang, J., Zhao, L., Sun, J., Zhou, H., Wei, H., et al. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2309.11499 · Cited by 59 · see citing works
EmbodiedGPT: Vision-Language Pre-Training via Embodied Chain of ThoughtPreprint
Mu, Y., Zhang, Q., Hu, M., Wang, W., Ding, M., Jin, J., Wang, B., Dai, J., Qiao, Y., & Luo, P. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2305.15021 · Cited by 58 · see citing works
Self-citationIconQA: A New Benchmark for Abstract Diagram Understanding and Visual Language ReasoningPreprint
Lu, P., Qiu, L., Chen, J., Xia, T., Zhao, Y., Zhang, W., Zhou, Y., Liang, X., & Zhu, S. C. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2110.13214 · Cited by 57 · see citing works
Cai, Y., Zhou, Y., Han, Q., Sun, J., Kong, X., Li, J., & Zhang, X. (2022) · arXiv · doi:10.48550/arxiv.2212.11696 · Cited by 56 · see citing works
The effectiveness of MAE pre-pretraining for billion-scale pretraining
Singh, M., Duval, Q., Alwala, K. V., Fan, H., Aggarwal, V., Adcock, A., Joulin, A., Dollár, P., Feichtenhofer, C., Girshick, R., et al. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.00505 · Cited by 56 · see citing works
Skywork: A More Open Bilingual Foundation ModelPreprint
Wei, T., Zhao, L., Zhang, L., Zhu, B., Wang, L., Yang, H., Li, B., Cheng, C., Lü, W., Hu, R., et al. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2310.19341 · Cited by 53 · see citing works
Self-citationInternGPT: Solving Vision-Centric Tasks by Interacting with ChatGPT Beyond LanguagePreprint
Liu, Z., He, Y., Wang, W., Wang, W., Wang, Y., Chen, S., Zhang, Q., Lai, Z., Yang, Y., Li, Q., et al. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2305.05662 · Cited by 46 · see citing works
Self-citationGPT4Tools: Teaching Large Language Model to Use Tools via Self-instruction
Yang, R., Song, L., Li, Y., Zhao, S., Ge, Y., Li, X., & Shan, Y. (2023) · neural information processing systems · doi:10.52202/075280-3149 · Cited by 46 · see citing works
PaLI-X: On Scaling up a Multilingual Vision and Language ModelPreprint
Chen, X., Djolonga, J., Padlewski, P., Mustafa, B., Changpinyo, S., Wu, J., Riquelme, C., Goodman, S., Wang, X., Tay, Y., et al. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2305.18565 · Cited by 42 · see citing works
Self-citationContrastive Language-Image Pre-training for the Italian LanguagePreprint
Bianchi, F., Attanasio, G., Pisoni, R., Terragni, S., Sarti, G., & Lakshmi, S. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2108.08688 · Cited by 35 · see citing works
ControlLLM: Augment Language Models with Tools by Searching on Graphs
Liu, Z., Lai, Z., Gao, Z., Cui, E., Li, Z., Zhu, X., Lu, L., Chen, Q., Qiao, Y., Dai, Y., et al. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-73254-6_6 · Cited by 34 · see citing works
Self-citationThe Physiological Response to the Feeling of Being WatchedPreprint
Carere, M. T. (2018) · EasyChair preprint · doi:10.29007/h68j · Cited by 31 · see citing works
Wukong: A 100 Million Large-scale Chinese Cross-modal Pre-training BenchmarkPreprint
Gu, J., Meng, X., Lu, G., et al. (2022) · arXiv (Cornell University) · doi:10.48550/arxiv.2202.06767 · Cited by 29 · see citing works
The All-Seeing Project: Towards Panoptic Visual Recognition and Understanding of the Open WorldPreprint
Wang, W., Shi, M., Li, Q., Wang, W., Huang, Z., Xing, L., Chen, Z., Li, H., Zhu, X., Cao, Z., et al. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2308.01907 · Cited by 29 · see citing works
Self-citationAn Empirical Study of Scaling Instruct-Tuned Large Multimodal ModelsPreprint
Lu, Y., Li, C., Liu, H., Yang, J., Gao, J., & Shen, Y. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2309.09958 · Cited by 21 · see citing works
SVIT: Scaling up Visual Instruction TuningPreprint
Zhao, B., Wu, B., He, M., & Huang, T. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2307.04087 · Cited by 18 · see citing works
ResT V2: Simpler, Faster and Stronger
Zhang, Q., & Yang, Y.-B. (2022) · neural information processing systems · doi:10.52202/068431-2640 · Cited by 17 · see citing works
Self-citationAlleviating data insufficiency for Chinese sign language recognition
Xue, W., Liu, J., Yan, S., Zhou, Y., Yuan, T., & Guo, Q. (2023) · Visual Intelligence · doi:10.1007/s44267-023-00028-5 · Cited by 15 · see citing works
ResT V2: Simpler, Faster and StrongerPreprint
Zhang, Q., & Yang, Y.-B. (2022) · arXiv (Cornell University) · doi:10.48550/arxiv.2204.07366 · Cited by 12 · see citing works
Self-citationToloka Visual Question Answering BenchmarkPreprint
Ustalov, D., Pavlichenko, N., Koshelev, S., Likhobaba, D., & Smirnova, A. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2309.16511 · Cited by 1 · see citing works
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2018) · arXiv · doi:10.48550/arxiv.1810.04805
Flagged first, then by citations
7 of the 145 references couldn’t be matched to a known paper.
Cited by
Sorted by citations
How far are we to GPT-4V? Closing the gap to commercial multimodal models with open-source suites
Chen, Z., Wang, W., Tian, H., Ye, S., Gao, Z., Cui, E., Tong, W., Hu, K., Luo, J., Ma, Z., et al. (2024) · Science China Information Sciences · doi:10.1007/s11432-024-4231-5 · Cited by 432 · see citing works
Self-citationMMMU: A Massive Multi-Discipline Multimodal Understanding and Reasoning Benchmark for Expert AGI
Yue, X., Ni, Y., Zheng, T., Zhang, K., Liu, R., Zhang, G., Stevens, S., Jiang, D., Ren, W., Sun, Y., et al. (2024) · 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52733.2024.00913 · Cited by 429 · see citing works
Multimodal Large Language Models in Health Care: Applications, Challenges, and Future Outlook.
AlSaad, R., Abd-alrazaq, A., Boughorbel, S., Ahmed, A., Renault, M.-A., Damseh, R., & Sheikh, J. (2024) · Journal of medical Internet research · doi:10.2196/59505 · Cited by 349 · see citing works
InternVideo2: Scaling Foundation Models for Multimodal Video Understanding
Wang, Y., Li, K., Li, X., Yu, J., He, Y., Chen, G., Pei, B., Zheng, R., Wang, Z., Shi, Y., et al. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-73013-9_23 · Cited by 239 · see citing works
Self-citationLlaVA-CoT: Let Vision Language Models Reason Step-By-Step
Xu, G., Jin, P., Wu, Z., Li, H., Song, Y., Sun, L., & Yuan, L. (2025) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51701.2025.00202 · Cited by 118 · see citing works
TextMonkey: An OCR-Free Large Multimodal Model for Understanding Document
Liu, Y., Yang, B., Liu, Q., Li, Z., Ma, Z., Zhang, S., & Bai, X. (2026) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2026.3653415 · Cited by 103 · see citing works
An overview of large AI models and their applications
Tu, X., He, Z., Huang, Y., Zhang, Z., Yang, M., & Zhao, J. C. (2024) · Visual Intelligence · doi:10.1007/s44267-024-00065-8 · Cited by 95 · see citing works
Thinking in Space: How Multimodal Large Language Models See, Remember, and Recall Spaces
Yang, J., Yang, S., Gupta, A. W., Han, R., Fei-Fei, L., & Xie, S. (2025) · 2025 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52734.2025.00994 · Cited by 120 · see citing works
VisionZip: Longer is Better but Not Necessary in Vision Language Models
Yang, S., Chen, Y., Tian, Z., Wang, C., Li, J., Yu, B., & Jia, J. (2025) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52734.2025.01843 · Cited by 93 · see citing works
Uni-MoE: Scaling Unified Multimodal LLMs With Mixture of Experts
Li, Y., Jiang, S., Hu, B., Wang, L., Zhong, W., Luo, W., Ma, L., & Zhang, M. (2025) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2025.3532688 · Cited by 91 · see citing works
Video Mamba Suite: State Space Model as a Versatile Alternative for Video Understanding
Chen, G., Huang, Y., Xu, J., Pei, B., Wang, J., Chen, Z., Li, Z., Lu, T., & Wang, L. (2025) · International Journal of Computer Vision · doi:10.1007/s11263-025-02597-y · Cited by 90 · see citing works
Self-citationOmniGen: Unified Image Generation
Xiao, S., Wang, Y., Zhou, J., Yuan, H., Xing, X., Yan, R., Li, C., Wang, S., Huang, T., & Liu, Z. (2025) · 2025 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52734.2025.01241 · Cited by 75 · see citing works
How to Bridge the Gap Between Modalities: Survey on Multimodal Large Language Model
Song, S., Li, X., Li, S., Zhao, S., Yu, J., Ma, J., Mao, X., Zhang, W., & Wang, M. (2025) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2025.3527978 · Cited by 74 · see citing works
Janus: Decoupling Visual Encoding for Unified Multimodal Understanding and Generation
Wu, C., Chen, X., Wu, Z., Ma, Y., Liu, X., Pan, Z., Wen, L., Xie, Z., Yu, X., Ruan, C., et al. (2025) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52734.2025.01210 · Cited by 74 · see citing works
Self-citationNVILA: Efficient Frontier Visual Language Models
Liu, Z., Zhu, L., Shi, B., Zhang, Z., Lou, Y., Yang, S. W., Xi, H., Cao, S., Gu, Y., Li, D., et al. (2025) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52734.2025.00390 · Cited by 73 · see citing works
Li, Z., Wu, X., Du, H., Liu, F., Nghiem, H. T., & Shi, G. (2025) · IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw67362.2025.00147 · Cited by 72 · see citing works
Natural Language Understanding and Inference with MLLM in Visual Question Answering: A Survey
Kuang, J., Shen, Y., Xie, J., Luo, H., Xu, Z., Li, R., Li, Y., Cheng, X., Lin, X., & Han, Y. (2025) · ACM Computing Surveys · doi:10.1145/3711680 · Cited by 69 · see citing works
TokenPacker: Efficient Visual Projector for Multimodal LLM
Li, W., Yuan, Y., Liu, J., Tang, D., Wang, S., Qin, J., Zhu, J., & Lei Zhang, A. (2025) · International Journal of Computer Vision · doi:10.1007/s11263-025-02491-7 · Cited by 63 · see citing works
AM-RADIO: Agglomerative Vision Foundation Model Reduce All Domains Into One
Ranzinger, M., Heinrich, G., Kautz, J., & Molchanov, P. A. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.01187 · Cited by 60 · see citing works
A Thorough Examination of Decoding Methods in the Era of LLMs
Shi, C., Yang, H., Cai, D., Zhang, Z., Wang, Y., Yang, Y., & Lam, W. P. (2024) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2024.emnlp-main.489 · Cited by 55 · see citing works
Groma: Localized Visual Tokenization for Grounding Multimodal Large Language Models
Ma, C., Jiang, Y., Wu, J., Yuan, Z., & Qi, X. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-72658-3_24 · Cited by 54 · see citing works
Self-citationGuo, R., Wei, J., Sun, L., Yu, B., Chang, G., Liu, D., Zhang, S., Yao, Z., Xu, M., & Bu, L. (2024) · Computers in Biology and Medicine · doi:10.1016/j.compbiomed.2024.108709 · Cited by 50 · see citing works
Paying More Attention to Image: A Training-Free Method for Alleviating Hallucination in LVLMs
Liu, S., Zheng, K., & Chen, W. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-73010-8_8 · Cited by 50 · see citing works
Mini-InternVL: a flexible-transfer pocket multi-modal model with 5% parameters and 90% performance
Gao, Z., Chen, Z., Cui, E., Ren, Y., Wang, W., Zhu, J., Tian, H., He, J., He, J., Zhu, X., et al. (2024) · Visual Intelligence · doi:10.1007/s44267-024-00067-6 · Cited by 49 · see citing works
Self-citationEarthDial: Turning Multi-sensory Earth Observations to Interactive Dialogues
Soni, S., Dudhane, A. A., Debary, H., Fiaz, M., Munir, M. A., Danish, M. S., Fraccaro, P., Watson, C., Klein, L. J., Khan, F. S., et al. (2025) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52734.2025.01334 · Cited by 48 · see citing works
Detecting and Mitigating Hallucination in Large Vision Language Models via Fine-Grained AI Feedback
Xiao, W., Huang, Z., Gan, L., He, W., Li, H., Yu, Z., Shu, F., Jiang, H., & Zhu, L. (2025) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v39i24.34744 · Cited by 49 · see citing works
Vision Language Models are blind
Rahmanzadehgervi, P., Bolton, L., Taesiri, M. R., & Nguyen, A. T. (2025) · Lecture Notes in Computer Science · doi:10.1007/978-981-96-0917-8_17 · Cited by 54 · see citing works
Trajectory prediction for autonomous driving: Progress, limitations, and future directions
Madjid, N. A., Ahmad, A., Mebrahtu, M., Baba'a, Y., Nasser, A. N., Malik, S., Hassan, B., Werghi, N., Dias, J. M. M., & Khonji, M. (2025) · Information Fusion · doi:10.1016/j.inffus.2025.103588 · Cited by 46 · see citing works
Beyond Text-Visual Attention: Exploiting Visual Cues for Effective Token Pruning in VLMs
Zhang, Q. F., Cheng, A., Lu, M., Zhang, R., Zhuo, Z., Cao, J., Guo, S., She, Q., & Zhang, S. (2025) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51701.2025.01939 · Cited by 45 · see citing works
Self-citationFineVQ: Fine-Grained User Generated Content Video Quality Assessment
Duan, H., Hu, Q., Wang, J., Yang, L., Xu, Z., Lü, L., Min, X., Cai, C., Ye, T., Zhang, X., et al. (2025) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52734.2025.00305 · Cited by 42 · see citing works
Multimodal fusion and vision–language models: A survey for robot vision
Han, X., Chen, S., Fu, Z., Feng, Z., Fan, L., An, D., Wang, C., Guo, L., Meng, W., Zhang, X., et al. (2026) · Information Fusion · doi:10.1016/j.inffus.2025.103652 · Cited by 41 · see citing works
“By integrating visual, depth, audio, and linguistic data, these datasets provide abundant resources for comprehensive model training, evaluation, and optimization, while laying a solid foundation for future research in multimodal fusion and cross-domain transfer learning.”
Hallucination of Multimodal Large Language Models: A Survey
Bai, Z., Wang, P., Xiao, T., He, T., Han, Z., Zhang, Z., & Shou, M. Z. (2024) · arXiv · doi:10.48550/arxiv.2404.18930 · Cited by 38 · see citing works
“Unlearning refers to a technique designed to induce a model to ’forget’ specific behaviors or data, primarily through the application of gradient ascent methods [9].”
Embodied Multi-Agent Systems: A Review
Li, Z., Wu, W., Guo, Y., Sun, J. Q., & Han, Q. (2025) · IEEE/CAA Journal of Automatica Sinica · doi:10.1109/jas.2025.125552 · Cited by 38 · see citing works
Li, J., Li, J., Guo, Y., Yang, L., Chi, H., & Yang, L. (2025) · Drones · doi:10.3390/drones9040238 · Cited by 37 · see citing works
Embodied AI: From LLMs to World Models [Feature]
Feng, T., Wang, X., Jiang, Y., & Zhu, W. (2025) · IEEE Circuits and Systems Magazine · doi:10.1109/mcas.2025.3603693 · Cited by 37 · see citing works
Fu, H., Zhang, D. H., Zhao, Z., Cui, J., Liang, D., Zhang, C., Zhang, D., Xie, H., Wang, B., & Bai, X. (2025) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51701.2025.02302 · Cited by 37 · see citing works
Zhu, Q., Lao, J., Ji, D., Luo, J., Wu, K. L., Zhang, Y., Ru, L., Wang, J., Chen, J., Yang, M., et al. (2025) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52734.2025.01373 · Cited by 33 · see citing works
Zhu, Y., Li, L., Chen, K., Liu, C., Zhou, F., & Shi, Z. (2024) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2024.3497338 · Cited by 32 · see citing works
RoboBrain: A Unified Brain Model for Robotic Manipulation from Abstract to Concrete
Ji, Y., Tan, H., Shi, J., Hao, X., Zhang, Y., Zhang, H., Wang, P., Zhao, M., Mu, Y., An, P., et al. (2025) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52734.2025.00168 · Cited by 32 · see citing works
Guo, K., Wong, P. K.-Y., Cheng, J. C. P., Chan, C.-F., Leung, P.-H., & Tao, X. (2025) · Automation in Construction · doi:10.1016/j.autcon.2025.106490 · Cited by 31 · see citing works
Cross-Modal Remote Sensing Image–Text Retrieval via Context and Uncertainty-Aware Prompt
Wang, Y., Tang, X., Ma, J., Zhang, X., Liu, F., & Jiao, L. (2024) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2024.3458898 · Cited by 30 · see citing works
Wang, X., Wang, F., Li, Y., Ma, Q., Wang, S., Jiang, B., & Tang, J. (2025) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52734.2025.00483 · Cited by 30 · see citing works
Benchmark Evaluations, Applications, and Challenges of Large Vision Language Models: A SurveyPreprint
Li, Z., Wu, X., Du, H., Nghiem, H. T., & Shi, G. (2025) · Qeios · doi:10.32388/gxr68q · Cited by 30 · see citing works
Uncertainty-Aware Medical Diagnostic Phrase Identification and Grounding
Zou, K., Bai, Y., Liu, B., Chen, Y., Chen, Z., Zhou, Y., Yuan, X., Wang, M., Shen, X., Cao, X., et al. (2025) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2025.3596878 · Cited by 29 · see citing works
Self-citationStreamingbench: Assessing the Gap for MLLMs to Achieve Streaming Video Understanding
Lin, J., Zheng, F., Chen, C., Cheng, H., Wan, Z., Luo, F., Wang, Z., Li, P., Liu, Y., & Sun, M. (2026) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp55912.2026.11463959 · Cited by 29 · see citing works
Kangaroo: A Powerful Video-Language Model Supporting Long-context Video Input
Liu, J., Wang, Y., Ma, H., Wu, X., Ma, X., Wei, X., Jiao, J., Wu, E., & Hu, J. (2026) · International Journal of Computer Vision · doi:10.1007/s11263-025-02620-2 · Cited by 28 · see citing works
ScreenSpot-Pro: GUI Grounding for Professional High-Resolution Computer Use
Li, K., Meng, Z., Lin, H., Luo, Z., Tian, Y., Ma, J., Huang, Z., & Chua, T.-S. (2025) · Proceedings of the 33rd ACM International Conference on Multimedia · doi:10.1145/3746027.3755688 · Cited by 27 · see citing works
MiniGPT-3D: Efficiently Aligning 3D Point Clouds with Large Language Models using 2D Priors
Tang, Y., Han, X., Li, X., Qiao, Y., Hao, Y., Hu, L., & Chen, M. (2024) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3664647.3681257 · Cited by 27 · see citing works
Self-citationHan, S., Huang, W., Shi, H., Zhuo, L., Su, X., Zhang, S., Zhou, X., Qi, X., Liao, Y., & Liu, S. (2025) · 2025 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52734.2025.02438 · Cited by 22 · see citing works
Qian, R., Ding, S., Dong, X., Zhang, P., Zang, Y., Cao, Y., Lin, D., & Wang, J. (2025) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52734.2025.02239 · Cited by 26 · see citing works
Liang, X., Xie, J., Zhao, T., Stouffs, R. M. F. A., & Biljecki, F. (2025) · ISPRS Journal of Photogrammetry and Remote Sensing · doi:10.1016/j.isprsjprs.2025.10.014 · Cited by 26 · see citing works
Gong, Z., Wei, Z., Wang, D., Hu, X., Ma, X., Chen, H., Jia, Y., Deng, Y., Ji, Z., Zhu, X., et al. (2025) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2025.3649001 · Cited by 26 · see citing works
Self-citationAll Languages Matter: Evaluating LMMs on Culturally Diverse 100 Languages
Vayani, A., Dissanayake, D., Watawana, H., Ahsan, N., Sasikumar, N., Thawakar, O., Ademtew, H. B., Hmaiti, Y., Kumar, A., Kuckreja, K., et al. (2025) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52734.2025.01822 · Cited by 25 · see citing works
Zhang, X., Zeng, F., & Gu, C. (2024) · Neural Networks · doi:10.1016/j.neunet.2024.107059 · Cited by 25 · see citing works
Yang, X., Wu, W., Feng, S., Wang, M., Wang, D., Li, Y., Sun, Q., Zhang, Y., Fu, X., & Poria, S. (2025) · Information Fusion · doi:10.1016/j.inffus.2025.103204 · Cited by 26 · see citing works
VisionUnite: A Vision-Language Foundation Model for Ophthalmology Enhanced With Clinical Knowledge
Li, Z., Song, D., Yang, Z., Wang, D., Li, F., Zhang, X., Kinahan, P. E., & Qiao, Y. (2025) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2025.3598734 · Cited by 25 · see citing works
Self-citationPointVLA: Injecting the 3D World Into Vision-Language-Action Models
Li, C., Wen, J., Peng, Y., Peng, Y., & Zhu, Y. (2026) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2026.3653303 · Cited by 25 · see citing works
Automated electrosynthesis reaction mining with multimodal large language models (MLLMs)
Leong, S. X., Pablo‐García, S., Zhang, Z., & Aspuru‐Guzik, A. (2024) · Chemical Science · doi:10.1039/d4sc04630g · Cited by 24 · see citing works
Luo, G., Yang, X., Dou, W., Wang, Z., Liu, J., Dai, J., Qiao, Y., & Zhu, X. (2025) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52734.2025.02324 · Cited by 23 · see citing works
Self-citationM 3 ixup: A multi-modal data augmentation approach for image captioning
Li, Y., Ji, J., Sun, X., Zhou, Y., Luo, Y., & Ji, R. (2024) · Pattern Recognition · doi:10.1016/j.patcog.2024.110941 · Cited by 22 · see citing works
Hong, W., Cheng, Y., Yang, Z., Wang, W., Wang, L., Gu, X., Huang, S., Dong, Y., & Tang, J. (2025) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52734.2025.00791 · Cited by 22 · see citing works
Self-citationTeaching multimodal LLMs to comprehend 12-lead electrocardiographic images
Liu, R., Bai, Y., Yue, X., & Zhang, P. (2026) · npj Digital Medicine · doi:10.1038/s41746-026-02551-3 · Cited by 22 · see citing works
Creating High-Quality 3D Content by Bridging the Gap between Text-to-2D and Text-to-3D Generation
Ma, Y., Fan, Y., Ji, J., Wang, H., Yin, H., Sun, X., & Ji, R. (2024) · ACM Transactions on Multimedia Computing Communications and Applications · doi:10.1145/3687475 · Cited by 22 · see citing works
Generative Semantic Communication: Architectures, Technologies, and Applications
Ren, J., Sun, Y., Du, H., Yuan, W., Wang, C., Wang, X., Zhou, Y., Zhu, Z., Wang, F., & Cui, S. R. (2025) · Engineering · doi:10.1016/j.eng.2025.07.022 · Cited by 22 · see citing works
MSCoTDet: Language-Driven Multi-Modal Fusion for Improved Multispectral Pedestrian Detection
Kim, T., Chung, S., Yeom, D., Yu, Y., Kim, H. G., & Ro, Y. M. (2024) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2024.3524645 · Cited by 21 · see citing works
Wu, C., Liang, Z., Ge, Y., Guo, Q., Lu, Z., Wang, J., Shan, Y., & Luo, P. (2025) · Findings of the Association for Computational Linguistics: NAACL · doi:10.18653/v1/2025.findings-naacl.164 · Cited by 21 · see citing works
Self-citationDon’t Miss the Forest for the Trees: Attentional Vision Calibration for Large Vision Language Models
Woo, S., Kim, D., Jang, J., Choi, Y., & Kim, C. (2025) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2025.findings-acl.99 · Cited by 21 · see citing works
Visual Instruction Tuning towards General-Purpose Multimodal Large Language Model: A Survey
Huang, J., Zhang, J., Jiang, K. Y., Qiu, H., Zhang, X., Shao, L., Lu, S., & Tao, D. (2025) · International Journal of Computer Vision · doi:10.1007/s11263-025-02572-7 · Cited by 21 · see citing works
X-Pose: Detecting Any Keypoints
Yang, J., Zeng, A., Zhang, R., & Zhang, L. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-72952-2_15 · Cited by 20 · see citing works
EmoVerse: Enhancing Multimodal Large Language Models for Affective Computing via Multitask Learning
Li, A., Xu, L., Ling, C., Zhang, J., & Wang, P. (2025) · Neurocomputing · doi:10.1016/j.neucom.2025.130810 · Cited by 20 · see citing works
Zhang, Z., Yu, J., Yang, B. Y., Du, K., Wang, S., & Xing, Q. (2025) · Advanced Engineering Informatics · doi:10.1016/j.aei.2025.103705 · Cited by 20 · see citing works
The All-Seeing Project V2: Towards General Relation Comprehension of the Open World
Wang, W., Ren, Y., Luo, H., Li, T., Yan, C., Chen, Z., Wang, W., Li, Q., Lu, L., Zhu, X., et al. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-73414-4_27 · Cited by 20 · see citing works
Self-citationZheng, X., Zeng, Z., Wang, H., Bai, Y., Liu, Y., & Luo, M. (2025) · International World Wide Web Conference (WWW) · doi:10.1145/3696410.3714532 · Cited by 19 · see citing works
Generative AI for Cel-Animation: A Survey
Tang, Y., Guo, J., Liu, P., Wang, Z., Hua, H., Zhong, J.-X., Xiao, Y., Huang, C., Song, L., Liang, S., et al. (2025) · IEEE International Conference on Computer Vision Workshops · doi:10.1109/iccvw69036.2025.00400 · Cited by 19 · see citing works
Ma, Y., Ye, W., Cui, C., Zhang, H., Xing, S., Ke, F., Wang, J., Miao, C., Chen, J., Rezatofighi, H., et al. (2025) · IEEE/CVF Winter Conference on Applications of Computer Vision Workshops (WACVW) · doi:10.1109/wacvw65960.2025.00114 · Cited by 19 · see citing works
Self-citationBui, M. D., von der Wense, K., & Lauscher, A. (2025) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/2025.naacl-long.490 · Cited by 18 · see citing works
Scalable Vision Language Model Training via High Quality Data Curation
Dong, H., Kang, Z., Yin, W., LiangXiao, L., ChaoFeng, C., & Jiao, R. (2025) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2025.acl-long.1595 · Cited by 18 · see citing works
DocKylin: A Large Multimodal Model for Visual Document Understanding with Efficient Visual Slimming
Zhang, J., Yang, W., Lai, S., Xie, Z., & Jin, L. (2025) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v39i9.33076 · Cited by 18 · see citing works
Reflective Instruction Tuning: Mitigating Hallucinations in Large Vision-Language Models
Zhang, J., Wang, T., Zhang, H., Lu, P., & Zheng, F. (2025) · Lecture Notes in Computer Science · doi:10.1007/978-3-031-73113-6_12 · Cited by 17 · see citing works
Foundation Models and Biometrics: A Survey and Outlook
Shahreza, H. O., & Marcel, S. (2025) · IEEE Transactions on Information Forensics and Security · doi:10.1109/tifs.2025.3602233 · Cited by 17 · see citing works
AvatarShield: Visual Reinforcement Learning for Human-Centric Synthetic Video Detection
Xu, Z., Zhang, X., Huang, Q., Zhou, X., & Zhang, J. (2026) · International Journal of Computer Vision · doi:10.1007/s11263-026-02954-5 · Cited by 17 · see citing works
RoboTron-Drive: All-in-One Large Multimodal Model for Autonomous Driving
Huang, Z., Feng, C., Yan, F., Xiao, B., Jie, Z., Zhong, Y., Liang, X., & Ma, L. (2025) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51701.2025.00751 · Cited by 17 · see citing works
Lu, J., Yu, H., Wang, Y., Ye, Y., Tang, J., Yang, Z., Wu, B., Liu, Q., Feng, H., Wang, H., et al. (2025) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2025.findings-acl.379 · Cited by 17 · see citing works
Urban Road Anomaly Monitoring Using Vision–Language Models for Enhanced Safety Management
Ding, H., Du, Y., & Xia, Z. (2025) · Applied Sciences · doi:10.3390/app15052517 · Cited by 16 · see citing works
Benchmarking vision-language models for diagnostics in emergency and critical care settings
Kurz, C., Merzhevich, T., Eskofier, B. M., Kather, J. N., & Gmeiner, B. (2025) · npj Digital Medicine · doi:10.1038/s41746-025-01837-2 · Cited by 16 · see citing works
Your Large Vision-Language Model Only Needs A Few Attention Heads For Visual Grounding
Kang, S., Kim, J.-Y., Kim, J., & Hwang, S. J. (2025) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52734.2025.00872 · Cited by 16 · see citing works
Multimodal learning with next-token prediction for large multimodal models
Wang, X., Cui, Y., Wang, J., Zhang, F., Wang, Y., Zhang, X., Luo, Z., Sun, Q., Li, Z., Wang, Y., et al. (2026) · Nature · doi:10.1038/s41586-025-10041-x · Cited by 16 · see citing works
Point Cloud-Based Deep Learning in Industrial Production: A Survey
Liu, Y., Zhang, C., Dong, X., & Ning, J. (2025) · ACM Computing Surveys · doi:10.1145/3715851 · Cited by 16 · see citing works
Critic-V: VLM Critics Help Catch VLM Errors in Multimodal Reasoning
Zhang, D., Lei, J., Li, J., Wang, X., Liu, Y., Yang, Z., Li, J., Wang, W., Yang, S., Wu, J., et al. (2025) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52734.2025.00846 · Cited by 16 · see citing works
Self-citationMMFuser: Multimodal Multi-layer Feature Fuser for Fine-Grained Vision-Language Understanding
Cao, Y., Huang, Y., Zhu, W., Liu, Y., Chen, Z., Shi, G., Fa, Y., Yang, Y., Mei, S., & Lü, T. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-31666-0_27 · Cited by 16 · see citing works
Self-citationNII-UIT at VBS2025: Multimodal Video Retrieval with LLM Integration and Dynamic Temporal Search
Gia, B. T., Khanh, T. B. C., Thanh, T., Doan, T. T., Le, K., Do, T., Mai, T.-D., Ngo, T. D., Le, D.-D., & Satoh, S. (2024) · Lecture notes in computer science · doi:10.1007/978-981-96-2074-6_38 · Cited by 15 · see citing works
Chat-based Person Retrieval via Dialogue-Refined Cross-Modal Alignment
Bai, Y., Ji, Y., Cao, M., Wang, J., & Ye, M. (2025) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52734.2025.00374 · Cited by 15 · see citing works
Imp: Highly Capable Large Multimodal Models for Mobile Devices
Shao, Z., Zhou, Y., Yu, J., Ouyang, X., Zheng, L., Gai, Z., Wang, M., Kuang, Z., & Ding, J. (2025) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2025.3557680 · Cited by 15 · see citing works
FaceXBench: Evaluating Multimodal LLMs on Face Understanding
Narayan, K., Vibashan, V. S., & Patel, V. M. (2026) · IEEE Transactions on Biometrics Behavior and Identity Science · doi:10.1109/tbiom.2026.3655668 · Cited by 15 · see citing works
Image Quality Assessment: From Human to Machine Preference
Li, C., Tian, Y., Ling, X., Zhang, Z., Duan, H., Wu, H., Jia, Z., Liu, X., Min, X., Lu, G. S., et al. (2025) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52734.2025.00709 · Cited by 14 · see citing works
PiercingEye: Dual-Space Video Violence Detection With Hyperbolic Vision-Language Guidance
Leng, J., Wu, Z., Tan, M., Mo, M., Zheng, J., Li, Q., Gan, J., & Gao, X. (2025) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2025.3617460 · Cited by 14 · see citing works
UrbanLLaVA: A Multi-Modal Large Language Model for Urban Intelligence
Feng, J., Wang, S. S., Liu, T., Xi, Y., & Li, Y. (2025) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51701.2025.00586 · Cited by 14 · see citing works
A Causality-Aware Paradigm for Evaluating Creativity of Multimodal Large Language Models
Huang, Z., Zhong, S., Zhou, P., Gao, S., Žitnik, M., & Lin, L. (2025) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2025.3539433 · Cited by 14 · see citing works
T2ISafety: Benchmark for Assessing Fairness, Toxicity, and Privacy in Image Generation
Li, L., Shi, Z., Hu, X., Dong, B., Qin, Y., Liu, X., Sheng, L., & Shao, J. (2025) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52734.2025.01249 · Cited by 14 · see citing works
MMVU: Measuring Expert-Level Multi-Discipline Video Understanding
Zhao, Y., Zhang, H., Xie, L., Hu, T., Gan, G., Long, Y., Hu, Z., Chen, W., Li, C., Xu, Z., et al. (2025) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52734.2025.00793 · Cited by 14 · see citing works
- Authors
- Chen, Z., Wu, J., Wang, W., Su, W., Chen, G., Xing, S., Zhong, M., Zhang, Q., Zhu, X., Lu, L., Li, B., Luo, P., Lu, T., Qiao, Y., & Dai, J.
- Affiliations
- Nanjing University (CN)
- University of Hong Kong (HK)
- Chinese University of Hong Kong (HK)
- University of Science and Technology of China (CN)
- Tsinghua University (CN)
- Shanghai Artificial Intelligence Laboratory (CN)
- Sensetime (China) (CN)
- Nanjing University
- Tsinghua University
- Venue
- 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)
- Publisher
- IEEE
- Published
- 2024-06-16
- Pages
- 24185-24198
- ISSN
- 1063-6919 (ISSN-L)
- OpenAlex
- W4402713111
- Cited by
- 734
- References
- 145
- Citations by year
- 2026: 360 · 2025: 331 · 2024: 43
- FWCI
- 151.51
- Type
- conference paper
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Funders
- National Natural Science Foundation of China (62376134,62372223)
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Multimodal Machine Learning Applications
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
- Physical Sciences › Computer Science › Artificial Intelligence › Domain Adaptation and Few-Shot Learning
- Keywords
- vision-language foundation models, vision foundation models, LLM alignment, zero-shot image classification, image-text retrieval, visual perception
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
