The DOI matches its Crossref record; 138 of the 145 references it cites were found in a registry or an index.
Intern VL: Scaling up Vision Foundation Models and Aligning for Generic Visual-Linguistic Tasks
Open accessAbstract
The exponential growth of large language models (LLMs) has opened up numerous possibilities for multi-modal AGI systems. However, the progress in vision and vision-language foundation models, which are also critical elements of multi-modal AGI, has not kept pace with LLMs. In this work, we design a large-scale vision-language foun-dation model (Intern VL), which scales up the vision foun-dation model to 6 billion parameters and progressively aligns it with the LLM, using web-scale image-text data from various sources. This model can be broadly applied to and achieve state-of-the-art performance on 32 generic visual-linguistic benchmarks including visual perception tasks such as image-level or pixel-level recognition, vision-language tasks such as zero-shot image/video classification, zero-shot image/video-text retrieval, and link with LLMs to create multi-modal dialogue systems. It has powerful visual capabilities and can be a good alternative to the ViT-22B. We hope that our research could contribute to the development of multi-modal large models.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Context-aware modeling is widely used across various domains, including natural language processing [219], computer vision [220], and recommendation systems [221], where it can substantially boost both system performance and user experience.”
From Multimodal fusion and vision–language models: A survey for robot vision (2026) · cites this paper as [219]
Citations
Cites
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474
ImageNet classification with deep convolutional neural networks
Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017) · Communications of the ACM · doi:10.1145/3065386 · Cited by 112,155
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2018) · arXiv · doi:10.48550/arxiv.1810.04805 · Cited by 73,173
ImageNet: A large-scale hierarchical image database
Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., & Fei-Fei, L. (2009) · 2009 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2009.5206848 · Cited by 65,333
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2019) · Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) · doi:10.18653/v1/n19-1423 · Cited by 54,325
Squeeze-and-Excitation Networks
Hu, J., Shen, L., & Sun, G. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00745 · Cited by 35,667
Swin Transformer: Hierarchical Vision Transformer using Shifted Windows
Liu, Z., Lin, Y., Cao, Y., Hu, H., Wei, Y., Zhang, Z., Lin, S., & Guo, B. (2021) · 2021 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00986 · Cited by 35,534
RoBERTa: A Robustly Optimized BERT Pretraining Approach
Liu, Y., Ott, M., Goyal, N., Du, J., Joshi, M., Chen, D., Levy, O., Lewis, M., Zettlemoyer, L., & Stoyanov, V. (2019) · arXiv · doi:10.48550/arxiv.1907.11692 · Cited by 16,515
Learning Transferable Visual Models From Natural Language Supervision
Radford, A., Kim, J. W., Hallacy, C., Ramesh, A., Goh, G., Agarwal, S., Sastry, G., Askell, A., Mishkin, P., Clark, J., et al. (2021) · arXiv · doi:10.48550/arxiv.2103.00020 · Cited by 15,293
Aggregated Residual Transformations for Deep Neural Networks
Xie, S., Girshick, R., Dollar, P., Tu, Z., & He, K. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.634 · Cited by 12,465
Research works printed but not linked: 23
- Achiam, 2023, Gpt-4 technical report, arXiv preprint
- Bai, 2023, Qwen tech-nical report, arXiv preprint
- Bai, 2023, Qwen-vl: A frontier large vision-language model with versatile abilities, arXiv preprint
- Byeon, 2022, Coyo-700m: Image-text pair dataset
- Carlsson, 2022, Cross-lingual and multilingual clip, LREC, 6848
- Chen, 2023, Shikra: Unleashing multi-modal 11ms referential dialogue magic, arXiv preprint
- Dosovitskiy, 2020, An image is worth 16×16 words: Transformers for image recognition at scale, ICLR
- Goyal, 2017, Making the v in vq a matter: El-evating the role of image understanding in visual question answering, CVPR, 6904
- 2023, Introducing idefics: An open reproduction of state-of-the-art visual language model
- Ilharco, 2021, Open-clip. Zenodo. Version 0.1
- 2023, Clip benchmark: Clip-like model evaluation
- Li, 2023, Blip-2: Bootstrapping language-image pre-training with frozen image encoders and large language models, 19730
- Li, 2019, Coco-en for cross-lingual image tagging, captioning, and retrieval, TMM, 21, 2347
- Liu, 2023, Aligning large multi-modal model with robust instruction tuning, arXiv preprint
- Sawada, 2022, Construction and public release of language image pretraining models in japanese, MIRU
- Oquab, 2023, Dinov2: Learning robust visual features with-out supervision, TMLR
- Schuhmann, 2022, Laion coco:600m synthetic captions from laion2b-en
- Sun, 2024, Generative pretraining in mul-timodality, ICLR
- Sun, 2023, Moss: Training conversationallanguage models from synthetic data, arXiv preprint, 7
- Taori, 2023, Alpaca: A strong, replicable instruction-following model, Stanford Center for Research on Foundation Models, 3, 7
- Team, 2023, Internlm: A multilingual language model with progressively enhanced capabilities
- Xie, 2022, Zero and r2d2: A large-scale chi-nese cross-modal benchmark and a vision-language frame-work, arXiv preprint
- Zhu, 2023, Ghost in the minecraft: Generally capable agents for open-world enviroments via large language models with text-based knowledge and memory, arXiv preprint
Cited by
Sorted by citations
How far are we to GPT-4V? Closing the gap to commercial multimodal models with open-source suites
Chen, Z., Wang, W., Tian, H., Ye, S., Gao, Z., Cui, E., Tong, W., Hu, K., Luo, J., Ma, Z., et al. (2024) · Science China Information Sciences · doi:10.1007/s11432-024-4231-5 · Cited by 1,058
Self-citationMMMU: A Massive Multi-Discipline Multimodal Understanding and Reasoning Benchmark for Expert AGI
Yue, X., Ni, Y., Zheng, T., Zhang, K., Liu, R., Zhang, G., Stevens, S., Jiang, D., Ren, W., Sun, Y., et al. (2024) · 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52733.2024.00913 · Cited by 429
Multimodal Large Language Models in Health Care: Applications, Challenges, and Future Outlook.
AlSaad, R., Abd-alrazaq, A., Boughorbel, S., Ahmed, A., Renault, M.-A., Damseh, R., & Sheikh, J. (2024) · Journal of medical Internet research · doi:10.2196/59505 · Cited by 349
InternVideo2: Scaling Foundation Models for Multimodal Video Understanding
Wang, Y., Li, K., Li, X., Yu, J., He, Y., Chen, G., Pei, B., Zheng, R., Wang, Z., Shi, Y., et al. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-73013-9_23 · Cited by 427
Self-citationLlaVA-CoT: Let Vision Language Models Reason Step-By-Step
Xu, G., Jin, P., Wu, Z., Li, H., Song, Y., Sun, L., & Yuan, L. (2025) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51701.2025.00202 · Cited by 118
TextMonkey: An OCR-Free Large Multimodal Model for Understanding Document
Liu, Y., Yang, B., Liu, Q., Li, Z., Ma, Z., Zhang, S., & Bai, X. (2026) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2026.3653415 · Cited by 103
An overview of large AI models and their applications
Tu, X., He, Z., Huang, Y., Zhang, Z., Yang, M., & Zhao, J. C. (2024) · Visual Intelligence · doi:10.1007/s44267-024-00065-8 · Cited by 106
Thinking in Space: How Multimodal Large Language Models See, Remember, and Recall Spaces
Yang, J., Yang, S., Gupta, A. W., Han, R., Fei-Fei, L., & Xie, S. (2025) · 2025 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52734.2025.00994 · Cited by 120
VisionZip: Longer is Better but Not Necessary in Vision Language Models
Yang, S., Chen, Y., Tian, Z., Wang, C., Li, J., Yu, B., & Jia, J. (2025) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52734.2025.01843 · Cited by 229
Uni-MoE: Scaling Unified Multimodal LLMs With Mixture of Experts
Li, Y., Jiang, S., Hu, B., Wang, L., Zhong, W., Luo, W., Ma, L., & Zhang, M. (2025) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2025.3532688 · Cited by 136
Video Mamba Suite: State Space Model as a Versatile Alternative for Video Understanding
Chen, G., Huang, Y., Xu, J., Pei, B., Wang, J., Chen, Z., Li, Z., Lu, T., & Wang, L. (2025) · International Journal of Computer Vision · doi:10.1007/s11263-025-02597-y · Cited by 154
Self-citationOmniGen: Unified Image Generation
Xiao, S., Wang, Y., Zhou, J., Yuan, H., Xing, X., Yan, R., Li, C., Wang, S., Huang, T., & Liu, Z. (2025) · 2025 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52734.2025.01241 · Cited by 75
How to Bridge the Gap Between Modalities: Survey on Multimodal Large Language Model
Song, S., Li, X., Li, S., Zhao, S., Yu, J., Ma, J., Mao, X., Zhang, W., & Wang, M. (2025) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2025.3527978 · Cited by 105
Janus: Decoupling Visual Encoding for Unified Multimodal Understanding and Generation
Wu, C., Chen, X., Wu, Z., Ma, Y., Liu, X., Pan, Z., Wen, L., Xie, Z., Yu, X., Ruan, C., et al. (2025) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52734.2025.01210 · Cited by 334
Self-citationNVILA: Efficient Frontier Visual Language Models
Liu, Z., Zhu, L., Shi, B., Zhang, Z., Lou, Y., Yang, S. W., Xi, H., Cao, S., Gu, Y., Li, D., et al. (2025) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52734.2025.00390 · Cited by 190
Li, Z., Wu, X., Du, H., Liu, F., Nghiem, H. T., & Shi, G. (2025) · IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw67362.2025.00147 · Cited by 91
Natural Language Understanding and Inference with MLLM in Visual Question Answering: A Survey
Kuang, J., Shen, Y., Xie, J., Luo, H., Xu, Z., Li, R., Li, Y., Cheng, X., Lin, X., & Han, Y. (2025) · ACM Computing Surveys · doi:10.1145/3711680 · Cited by 84
TokenPacker: Efficient Visual Projector for Multimodal LLM
Li, W., Yuan, Y., Liu, J., Tang, D., Wang, S., Qin, J., Zhu, J., & Zhang, L. (2025) · International Journal of Computer Vision · doi:10.1007/s11263-025-02491-7 · Cited by 159
AM-RADIO: Agglomerative Vision Foundation Model Reduce All Domains Into One
Ranzinger, M., Heinrich, G., Kautz, J., & Molchanov, P. A. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.01187 · Cited by 93
A Thorough Examination of Decoding Methods in the Era of LLMs
Shi, C., Yang, H., Cai, D., Zhang, Z., Wang, Y., Yang, Y., & Lam, W. P. (2024) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2024.emnlp-main.489 · Cited by 120
Groma: Localized Visual Tokenization for Grounding Multimodal Large Language Models
Ma, C., Jiang, Y., Wu, J., Yuan, Z., & Qi, X. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-72658-3_24 · Cited by 85
Self-citationVision Language Models are blind
Rahmanzadehgervi, P., Bolton, L., Taesiri, M. R., & Nguyen, A. T. (2025) · Lecture Notes in Computer Science · doi:10.1007/978-981-96-0917-8_17 · Cited by 54
Guo, R., Wei, J., Sun, L., Yu, B., Chang, G., Liu, D., Zhang, S., Yao, Z., Xu, M., & Bu, L. (2024) · Computers in Biology and Medicine · doi:10.1016/j.compbiomed.2024.108709 · Cited by 58
Paying More Attention to Image: A Training-Free Method for Alleviating Hallucination in LVLMs
Liu, S., Zheng, K., & Chen, W. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-73010-8_8 · Cited by 104
Mini-InternVL: a flexible-transfer pocket multi-modal model with 5% parameters and 90% performance
Gao, Z., Chen, Z., Cui, E., Ren, Y., Wang, W., Zhu, J., Tian, H., He, J., He, J., Zhu, X., et al. (2024) · Visual Intelligence · doi:10.1007/s44267-024-00067-6 · Cited by 84
Self-citationEarthDial: Turning Multi-sensory Earth Observations to Interactive Dialogues
Soni, S., Dudhane, A. A., Debary, H., Fiaz, M., Munir, M. A., Danish, M. S., Fraccaro, P., Watson, C., Klein, L. J., Khan, F. S., et al. (2025) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52734.2025.01334 · Cited by 79
Detecting and Mitigating Hallucination in Large Vision Language Models via Fine-Grained AI Feedback
Xiao, W., Huang, Z., Gan, L., He, W., Li, H., Yu, Z., Shu, F., Jiang, H., & Zhu, L. (2025) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v39i24.34744 · Cited by 79
Trajectory prediction for autonomous driving: Progress, limitations, and future directions
Madjid, N. A., Ahmad, A., Mebrahtu, M., Baba'a, Y., Nasser, A. N., Malik, S., Hassan, B., Werghi, N., Dias, J. M. M., & Khonji, M. (2025) · Information Fusion · doi:10.1016/j.inffus.2025.103588 · Cited by 60
Beyond Text-Visual Attention: Exploiting Visual Cues for Effective Token Pruning in VLMs
Zhang, Q. F., Cheng, A., Lu, M., Zhang, R., Zhuo, Z., Cao, J., Guo, S., She, Q., & Zhang, S. (2025) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51701.2025.01939 · Cited by 156
Self-citationFineVQ: Fine-Grained User Generated Content Video Quality Assessment
Duan, H., Hu, Q., Wang, J., Yang, L., Xu, Z., Lü, L., Min, X., Cai, C., Ye, T., Zhang, X., et al. (2025) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52734.2025.00305 · Cited by 42
Multimodal fusion and vision–language models: A survey for robot vision
Han, X., Chen, S., Fu, Z., Feng, Z., Fan, L., An, D., Wang, C., Guo, L., Meng, W., Zhang, X., et al. (2026) · Information Fusion · doi:10.1016/j.inffus.2025.103652 · Cited by 105
“By integrating visual, depth, audio, and linguistic data, these datasets provide abundant resources for comprehensive model training, evaluation, and optimization, while laying a solid foundation for future research in multimodal fusion and cross-domain transfer learning.”
Hallucination of Multimodal Large Language Models: A Survey
Bai, Z., Wang, P., Xiao, T., He, T., Han, Z., Zhang, Z., & Shou, M. Z. (2024) · arXiv · doi:10.48550/arxiv.2404.18930 · Cited by 38
“Unlearning refers to a technique designed to induce a model to ’forget’ specific behaviors or data, primarily through the application of gradient ascent methods [9].”
Embodied Multi-Agent Systems: A Review
Li, Z., Wu, W., Guo, Y., Sun, J. Q., & Han, Q. (2025) · IEEE/CAA Journal of Automatica Sinica · doi:10.1109/jas.2025.125552 · Cited by 42
Li, J., Li, J., Guo, Y., Yang, L., Chi, H., & Yang, L. (2025) · Drones · doi:10.3390/drones9040238 · Cited by 44
Embodied AI: From LLMs to World Models [Feature]
Feng, T., Wang, X., Jiang, Y., & Zhu, W. (2025) · IEEE Circuits and Systems Magazine · doi:10.1109/mcas.2025.3603693 · Cited by 59
Fu, H., Zhang, D. H., Zhao, Z., Cui, J., Liang, D., Zhang, C., Zhang, D., Xie, H., Wang, B., & Bai, X. (2025) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51701.2025.02302 · Cited by 123
Zhu, Q., Lao, J., Ji, D., Luo, J., Wu, K. L., Zhang, Y., Ru, L., Wang, J., Chen, J., Yang, M., et al. (2025) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52734.2025.01373 · Cited by 40
Zhu, Y., Li, L., Chen, K., Liu, C., Zhou, F., & Shi, Z. (2024) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2024.3497338 · Cited by 46
RoboBrain: A Unified Brain Model for Robotic Manipulation from Abstract to Concrete
Ji, Y., Tan, H., Shi, J., Hao, X., Zhang, Y., Zhang, H., Wang, P., Zhao, M., Mu, Y., An, P., et al. (2025) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52734.2025.00168 · Cited by 55
Guo, K., Wong, P. K.-Y., Cheng, J. C. P., Chan, C.-F., Leung, P.-H., & Tao, X. (2025) · Automation in Construction · doi:10.1016/j.autcon.2025.106490 · Cited by 38
Cross-Modal Remote Sensing Image–Text Retrieval via Context and Uncertainty-Aware Prompt
Wang, Y., Tang, X., Ma, J., Zhang, X., Liu, F., & Jiao, L. (2024) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2024.3458898 · Cited by 34
Wang, X., Wang, F., Li, Y., Ma, Q., Wang, S., Jiang, B., & Tang, J. (2025) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52734.2025.00483 · Cited by 37
Benchmark Evaluations, Applications, and Challenges of Large Vision Language Models: A SurveyPreprint
Li, Z., Wu, X., Du, H., Nghiem, H. T., & Shi, G. (2025) · Qeios · doi:10.32388/gxr68q · Cited by 33
Uncertainty-Aware Medical Diagnostic Phrase Identification and Grounding
Zou, K., Bai, Y., Liu, B., Chen, Y., Chen, Z., Zhou, Y., Yuan, X., Wang, M., Shen, X., Cao, X., et al. (2025) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2025.3596878 · Cited by 36
Self-citationStreamingbench: Assessing the Gap for MLLMs to Achieve Streaming Video Understanding
Lin, J., Zheng, F., Chen, C., Cheng, H., Wan, Z., Luo, F., Wang, Z., Li, P., Liu, Y., & Sun, M. (2026) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp55912.2026.11463959 · Cited by 31
Kangaroo: A Powerful Video-Language Model Supporting Long-context Video Input
Liu, J., Wang, Y., Ma, H., Wu, X., Ma, X., Wei, X., Jiao, J., Wu, E., & Hu, J. (2026) · International Journal of Computer Vision · doi:10.1007/s11263-025-02620-2 · Cited by 132
ScreenSpot-Pro: GUI Grounding for Professional High-Resolution Computer Use
Li, K., Meng, Z., Lin, H., Luo, Z., Tian, Y., Ma, J., Huang, Z., & Chua, T.-S. (2025) · Proceedings of the 33rd ACM International Conference on Multimedia · doi:10.1145/3746027.3755688 · Cited by 27
MiniGPT-3D: Efficiently Aligning 3D Point Clouds with Large Language Models using 2D Priors
Tang, Y., Han, X., Li, X., Qiao, Y., Hao, Y., Hu, L., & Chen, M. (2024) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3664647.3681257 · Cited by 27
Self-citationHan, S., Huang, W., Shi, H., Zhuo, L., Su, X., Zhang, S., Zhou, X., Qi, X., Liao, Y., & Liu, S. (2025) · 2025 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52734.2025.02438 · Cited by 22
Qian, R., Ding, S., Dong, X., Zhang, P., Zang, Y., Cao, Y., Lin, D., & Wang, J. (2025) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52734.2025.02239 · Cited by 26
Liang, X., Xie, J., Zhao, T., Stouffs, R. M. F. A., & Biljecki, F. (2025) · ISPRS Journal of Photogrammetry and Remote Sensing · doi:10.1016/j.isprsjprs.2025.10.014 · Cited by 31
Gong, Z., Wei, Z., Wang, D., Hu, X., Ma, X., Chen, H., Jia, Y., Deng, Y., Ji, Z., Zhu, X., et al. (2025) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2025.3649001 · Cited by 47
Self-citationAll Languages Matter: Evaluating LMMs on Culturally Diverse 100 Languages
Vayani, A., Dissanayake, D., Watawana, H., Ahsan, N., Sasikumar, N., Thawakar, O., Ademtew, H. B., Hmaiti, Y., Kumar, A., Kuckreja, K., et al. (2025) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52734.2025.01822 · Cited by 25
Zhang, X., Zeng, F., & Gu, C. (2024) · Neural Networks · doi:10.1016/j.neunet.2024.107059 · Cited by 27
Yang, X., Wu, W., Feng, S., Wang, M., Wang, D., Li, Y., Sun, Q., Zhang, Y., Fu, X., & Poria, S. (2025) · Information Fusion · doi:10.1016/j.inffus.2025.103204 · Cited by 29
VisionUnite: A Vision-Language Foundation Model for Ophthalmology Enhanced With Clinical Knowledge
Li, Z., Song, D., Yang, Z., Wang, D., Li, F., Zhang, X., Kinahan, P. E., & Qiao, Y. (2025) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2025.3598734 · Cited by 25
Self-citationPointVLA: Injecting the 3D World Into Vision-Language-Action Models
Li, C., Wen, J., Peng, Y., Peng, Y., & Zhu, Y. (2026) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2026.3653303 · Cited by 100
Automated electrosynthesis reaction mining with multimodal large language models (MLLMs)
Leong, S. X., Pablo‐García, S., Zhang, Z., & Aspuru‐Guzik, A. (2024) · Chemical Science · doi:10.1039/d4sc04630g · Cited by 26
Luo, G., Yang, X., Dou, W., Wang, Z., Liu, J., Dai, J., Qiao, Y., & Zhu, X. (2025) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52734.2025.02324 · Cited by 23
Self-citationM 3 ixup: A multi-modal data augmentation approach for image captioning
Li, Y., Ji, J., Sun, X., Zhou, Y., Luo, Y., & Ji, R. (2024) · Pattern Recognition · doi:10.1016/j.patcog.2024.110941 · Cited by 23
Hong, W., Cheng, Y., Yang, Z., Wang, W., Wang, L., Gu, X., Huang, S., Dong, Y., & Tang, J. (2025) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52734.2025.00791 · Cited by 22
Self-citationTeaching multimodal LLMs to comprehend 12-lead electrocardiographic images
Liu, R., Bai, Y., Yue, X., & Zhang, P. (2026) · npj Digital Medicine · doi:10.1038/s41746-026-02551-3 · Cited by 22
Creating High-Quality 3D Content by Bridging the Gap between Text-to-2D and Text-to-3D Generation
Ma, Y., Fan, Y., Ji, J., Wang, H., Yin, H., Sun, X., & Ji, R. (2024) · ACM Transactions on Multimedia Computing Communications and Applications · doi:10.1145/3687475 · Cited by 24
Generative Semantic Communication: Architectures, Technologies, and Applications
Ren, J., Sun, Y., Du, H., Yuan, W., Wang, C., Wang, X., Zhou, Y., Zhu, Z., Wang, F., & Cui, S. R. (2025) · Engineering · doi:10.1016/j.eng.2025.07.022 · Cited by 25
MSCoTDet: Language-Driven Multi-Modal Fusion for Improved Multispectral Pedestrian Detection
Kim, T., Chung, S., Yeom, D., Yu, Y., Kim, H. G., & Ro, Y. M. (2024) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2024.3524645 · Cited by 22
Wu, C., Liang, Z., Ge, Y., Guo, Q., Lu, Z., Wang, J., Shan, Y., & Luo, P. (2025) · Findings of the Association for Computational Linguistics: NAACL · doi:10.18653/v1/2025.findings-naacl.164 · Cited by 75
Self-citationDon’t Miss the Forest for the Trees: Attentional Vision Calibration for Large Vision Language Models
Woo, S., Kim, D., Jang, J., Choi, Y., & Kim, C. (2025) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2025.findings-acl.99 · Cited by 22
Visual Instruction Tuning towards General-Purpose Multimodal Large Language Model: A Survey
Huang, J., Zhang, J., Jiang, K. Y., Qiu, H., Zhang, X., Shao, L., Lu, S., & Tao, D. (2025) · International Journal of Computer Vision · doi:10.1007/s11263-025-02572-7 · Cited by 44
X-Pose: Detecting Any Keypoints
Yang, J., Zeng, A., Zhang, R., & Zhang, L. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-72952-2_15 · Cited by 39
EmoVerse: Enhancing Multimodal Large Language Models for Affective Computing via Multitask Learning
Li, A., Xu, L., Ling, C., Zhang, J., & Wang, P. (2025) · Neurocomputing · doi:10.1016/j.neucom.2025.130810 · Cited by 26
Zhang, Z., Yu, J., Yang, B. Y., Du, K., Wang, S., & Xing, Q. (2025) · Advanced Engineering Informatics · doi:10.1016/j.aei.2025.103705 · Cited by 20
The All-Seeing Project V2: Towards General Relation Comprehension of the Open World
Wang, W., Ren, Y., Luo, H., Li, T., Yan, C., Chen, Z., Wang, W., Li, Q., Lu, L., Zhu, X., et al. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-73414-4_27 · Cited by 64
Self-citationZheng, X., Zeng, Z., Wang, H., Bai, Y., Liu, Y., & Luo, M. (2025) · International World Wide Web Conference (WWW) · doi:10.1145/3696410.3714532 · Cited by 22
Generative AI for Cel-Animation: A Survey
Tang, Y., Guo, J., Liu, P., Wang, Z., Hua, H., Zhong, J.-X., Xiao, Y., Huang, C., Song, L., Liang, S., et al. (2025) · IEEE International Conference on Computer Vision Workshops · doi:10.1109/iccvw69036.2025.00400 · Cited by 19
Ma, Y., Ye, W., Cui, C., Zhang, H., Xing, S., Ke, F., Wang, J., Miao, C., Chen, J., Rezatofighi, H., et al. (2025) · IEEE/CVF Winter Conference on Applications of Computer Vision Workshops (WACVW) · doi:10.1109/wacvw65960.2025.00114 · Cited by 23
Self-citationBui, M. D., von der Wense, K., & Lauscher, A. (2025) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/2025.naacl-long.490 · Cited by 18
Scalable Vision Language Model Training via High Quality Data Curation
Dong, H., Kang, Z., Yin, W., LiangXiao, L., ChaoFeng, C., & Jiao, R. (2025) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2025.acl-long.1595 · Cited by 61
DocKylin: A Large Multimodal Model for Visual Document Understanding with Efficient Visual Slimming
Zhang, J., Yang, W., Lai, S., Xie, Z., & Jin, L. (2025) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v39i9.33076 · Cited by 33
Reflective Instruction Tuning: Mitigating Hallucinations in Large Vision-Language Models
Zhang, J., Wang, T., Zhang, H., Lu, P., & Zheng, F. (2025) · Lecture Notes in Computer Science · doi:10.1007/978-3-031-73113-6_12 · Cited by 17
Foundation Models and Biometrics: A Survey and Outlook
Shahreza, H. O., & Marcel, S. (2025) · IEEE Transactions on Information Forensics and Security · doi:10.1109/tifs.2025.3602233 · Cited by 20
AvatarShield: Visual Reinforcement Learning for Human-Centric Synthetic Video Detection
Xu, Z., Zhang, X., Huang, Q., Zhou, X., & Zhang, J. (2026) · International Journal of Computer Vision · doi:10.1007/s11263-026-02954-5 · Cited by 37
RoboTron-Drive: All-in-One Large Multimodal Model for Autonomous Driving
Huang, Z., Feng, C., Yan, F., Xiao, B., Jie, Z., Zhong, Y., Liang, X., & Ma, L. (2025) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51701.2025.00751 · Cited by 49
Lu, J., Yu, H., Wang, Y., Ye, Y., Tang, J., Yang, Z., Wu, B., Liu, Q., Feng, H., Wang, H., et al. (2025) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2025.findings-acl.379 · Cited by 17
Urban Road Anomaly Monitoring Using Vision–Language Models for Enhanced Safety Management
Ding, H., Du, Y., & Xia, Z. (2025) · Applied Sciences · doi:10.3390/app15052517 · Cited by 16
Benchmarking vision-language models for diagnostics in emergency and critical care settings
Kurz, C., Merzhevich, T., Eskofier, B. M., Kather, J. N., & Gmeiner, B. (2025) · npj Digital Medicine · doi:10.1038/s41746-025-01837-2 · Cited by 16
Your Large Vision-Language Model Only Needs A Few Attention Heads For Visual Grounding
Kang, S., Kim, J.-Y., Kim, J., & Hwang, S. J. (2025) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52734.2025.00872 · Cited by 27
Multimodal learning with next-token prediction for large multimodal models
Wang, X., Cui, Y., Wang, J., Zhang, F., Wang, Y., Zhang, X., Luo, Z., Sun, Q., Li, Z., Wang, Y., et al. (2026) · Nature · doi:10.1038/s41586-025-10041-x · Cited by 33
Point Cloud-Based Deep Learning in Industrial Production: A Survey
Liu, Y., Zhang, C., Dong, X., & Ning, J. (2025) · ACM Computing Surveys · doi:10.1145/3715851 · Cited by 18
Critic-V: VLM Critics Help Catch VLM Errors in Multimodal Reasoning
Zhang, D., Lei, J., Li, J., Wang, X., Liu, Y., Yang, Z., Li, J., Wang, W., Yang, S., Wu, J., et al. (2025) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52734.2025.00846 · Cited by 34
Self-citationMMFuser: Multimodal Multi-layer Feature Fuser for Fine-Grained Vision-Language Understanding
Cao, Y., Huang, Y., Zhu, W., Liu, Y., Chen, Z., Shi, G., Fa, Y., Yang, Y., Mei, S., & Lü, T. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-31666-0_27 · Cited by 16
Self-citationNII-UIT at VBS2025: Multimodal Video Retrieval with LLM Integration and Dynamic Temporal Search
Gia, B. T., Khanh, T. B. C., Thanh, T., Doan, T. T., Le, K., Do, T., Mai, T.-D., Ngo, T. D., Le, D.-D., & Satoh, S. (2024) · Lecture notes in computer science · doi:10.1007/978-981-96-2074-6_38 · Cited by 15
Chat-based Person Retrieval via Dialogue-Refined Cross-Modal Alignment
Bai, Y., Ji, Y., Cao, M., Wang, J., & Ye, M. (2025) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52734.2025.00374 · Cited by 20
Imp: Highly Capable Large Multimodal Models for Mobile Devices
Shao, Z., Zhou, Y., Yu, J., Ouyang, X., Zheng, L., Gai, Z., Wang, M., Kuang, Z., & Ding, J. (2025) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2025.3557680 · Cited by 27
FaceXBench: Evaluating Multimodal LLMs on Face Understanding
Narayan, K., Vibashan, V. S., & Patel, V. M. (2026) · IEEE Transactions on Biometrics Behavior and Identity Science · doi:10.1109/tbiom.2026.3655668 · Cited by 15
Image Quality Assessment: From Human to Machine Preference
Li, C., Tian, Y., Ling, X., Zhang, Z., Duan, H., Wu, H., Jia, Z., Liu, X., Min, X., Lu, G. S., et al. (2025) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52734.2025.00709 · Cited by 14
PiercingEye: Dual-Space Video Violence Detection With Hyperbolic Vision-Language Guidance
Leng, J., Wu, Z., Tan, M., Mo, M., Zheng, J., Li, Q., Gan, J., & Gao, X. (2025) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2025.3617460 · Cited by 14
UrbanLLaVA: A Multi-Modal Large Language Model for Urban Intelligence
Feng, J., Wang, S. S., Liu, T., Xi, Y., & Li, Y. (2025) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51701.2025.00586 · Cited by 28
A Causality-Aware Paradigm for Evaluating Creativity of Multimodal Large Language Models
Huang, Z., Zhong, S., Zhou, P., Gao, S., Žitnik, M., & Lin, L. (2025) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2025.3539433 · Cited by 16
T2ISafety: Benchmark for Assessing Fairness, Toxicity, and Privacy in Image Generation
Li, L., Shi, Z., Hu, X., Dong, B., Qin, Y., Liu, X., Sheng, L., & Shao, J. (2025) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52734.2025.01249 · Cited by 23
MMVU: Measuring Expert-Level Multi-Discipline Video Understanding
Zhao, Y., Zhang, H., Xie, L., Hu, T., Gan, G., Long, Y., Hu, Z., Chen, W., Li, C., Xu, Z., et al. (2025) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52734.2025.00793 · Cited by 80
Metadata
- Authors
- Chen, Z., Wu, J., Wang, W., Su, W., Chen, G., Xing, S., Zhong, M., Zhang, Q., Zhu, X., Lu, L., Li, B., Luo, P., Lu, T., Qiao, Y., & Dai, J.
- Affiliations
- Nanjing University (CN)
- University of Hong Kong (HK)
- Chinese University of Hong Kong (HK)
- University of Science and Technology of China (CN)
- Tsinghua University (CN)
- Shanghai Artificial Intelligence Laboratory (CN)
- Sensetime (China) (CN)
- Nanjing University
- Tsinghua University
- Venue
- 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)
- Publisher
- IEEE
- Published
- 2024-06-16
- Pages
- 24185-24198
- ISSN
- 1063-6919 (ISSN-L)
- OpenAlex
- W4402713111
- Cited by
- 734
- References
- 145
- Citations by year
- 2026: 360 · 2025: 331 · 2024: 43
- FWCI
- 151.51
- Type
- conference paper
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Funders
- National Natural Science Foundation of China (62376134,62372223)
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Multimodal Machine Learning Applications
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
- Physical Sciences › Computer Science › Artificial Intelligence › Domain Adaptation and Few-Shot Learning
- Keywords
- vision-language foundation models, vision foundation models, LLM alignment, zero-shot image classification, image-text retrieval, visual perception
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
