The DOI matches its Crossref record; 138 of the 145 references it cites were found in a registry or an index.
Intern VL: Scaling up Vision Foundation Models and Aligning for Generic Visual-Linguistic Tasks
Open accessAbstract
The exponential growth of large language models (LLMs) has opened up numerous possibilities for multi-modal AGI systems. However, the progress in vision and vision-language foundation models, which are also critical elements of multi-modal AGI, has not kept pace with LLMs. In this work, we design a large-scale vision-language foun-dation model (Intern VL), which scales up the vision foun-dation model to 6 billion parameters and progressively aligns it with the LLM, using web-scale image-text data from various sources. This model can be broadly applied to and achieve state-of-the-art performance on 32 generic visual-linguistic benchmarks including visual perception tasks such as image-level or pixel-level recognition, vision-language tasks such as zero-shot image/video classification, zero-shot image/video-text retrieval, and link with LLMs to create multi-modal dialogue systems. It has powerful visual capabilities and can be a good alternative to the ViT-22B. We hope that our research could contribute to the development of multi-modal large models.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Context-aware modeling is widely used across various domains, including natural language processing [219], computer vision [220], and recommendation systems [221], where it can substantially boost both system performance and user experience.”
From Multimodal fusion and vision–language models: A survey for robot vision (2026) · cites this paper as [219]
Citations
Cites
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474
ImageNet classification with deep convolutional neural networks
Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017) · Communications of the ACM · doi:10.1145/3065386 · Cited by 112,155
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2018) · arXiv · doi:10.48550/arxiv.1810.04805 · Cited by 73,173
ImageNet: A large-scale hierarchical image database
Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., & Fei-Fei, L. (2009) · 2009 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2009.5206848 · Cited by 65,333
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2019) · Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) · doi:10.18653/v1/n19-1423 · Cited by 54,325
Squeeze-and-Excitation Networks
Hu, J., Shen, L., & Sun, G. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00745 · Cited by 35,667
Swin Transformer: Hierarchical Vision Transformer using Shifted Windows
Liu, Z., Lin, Y., Cao, Y., Hu, H., Wei, Y., Zhang, Z., Lin, S., & Guo, B. (2021) · 2021 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00986 · Cited by 35,534
RoBERTa: A Robustly Optimized BERT Pretraining Approach
Liu, Y., Ott, M., Goyal, N., Du, J., Joshi, M., Chen, D., Levy, O., Lewis, M., Zettlemoyer, L., & Stoyanov, V. (2019) · arXiv · doi:10.48550/arxiv.1907.11692 · Cited by 16,515
Learning Transferable Visual Models From Natural Language Supervision
Radford, A., Kim, J. W., Hallacy, C., Ramesh, A., Goh, G., Agarwal, S., Sastry, G., Askell, A., Mishkin, P., Clark, J., et al. (2021) · arXiv · doi:10.48550/arxiv.2103.00020 · Cited by 15,293
Aggregated Residual Transformations for Deep Neural Networks
Xie, S., Girshick, R., Dollar, P., Tu, Z., & He, K. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.634 · Cited by 12,465
Flagged first, then by citations
7 of the 145 references couldn’t be matched to a known paper.
Research works printed but not linked: 23
- Achiam, 2023, Gpt-4 technical report, arXiv preprint
- Bai, 2023, Qwen tech-nical report, arXiv preprint
- Bai, 2023, Qwen-vl: A frontier large vision-language model with versatile abilities, arXiv preprint
- Byeon, 2022, Coyo-700m: Image-text pair dataset
- Carlsson, 2022, Cross-lingual and multilingual clip, LREC, 6848
- Chen, 2023, Shikra: Unleashing multi-modal 11ms referential dialogue magic, arXiv preprint
- Dosovitskiy, 2020, An image is worth 16×16 words: Transformers for image recognition at scale, ICLR
- Goyal, 2017, Making the v in vq a matter: El-evating the role of image understanding in visual question answering, CVPR, 6904
- 2023, Introducing idefics: An open reproduction of state-of-the-art visual language model
- Ilharco, 2021, Open-clip. Zenodo. Version 0.1
- 2023, Clip benchmark: Clip-like model evaluation
- Li, 2023, Blip-2: Bootstrapping language-image pre-training with frozen image encoders and large language models, 19730
- Li, 2019, Coco-en for cross-lingual image tagging, captioning, and retrieval, TMM, 21, 2347
- Liu, 2023, Aligning large multi-modal model with robust instruction tuning, arXiv preprint
- Sawada, 2022, Construction and public release of language image pretraining models in japanese, MIRU
- Oquab, 2023, Dinov2: Learning robust visual features with-out supervision, TMLR
- Schuhmann, 2022, Laion coco:600m synthetic captions from laion2b-en
- Sun, 2024, Generative pretraining in mul-timodality, ICLR
- Sun, 2023, Moss: Training conversationallanguage models from synthetic data, arXiv preprint, 7
- Taori, 2023, Alpaca: A strong, replicable instruction-following model, Stanford Center for Research on Foundation Models, 3, 7
- Team, 2023, Internlm: A multilingual language model with progressively enhanced capabilities
- Xie, 2022, Zero and r2d2: A large-scale chi-nese cross-modal benchmark and a vision-language frame-work, arXiv preprint
- Zhu, 2023, Ghost in the minecraft: Generally capable agents for open-world enviroments via large language models with text-based knowledge and memory, arXiv preprint
Cited by
Sorted by citations
Items 801–843 of 843
Do Audio-Visual Large Language Models Really See and Hear?
Selvakumar, R. K., Jayakumar, K., Sakshi, S., Ghosh, S., Gao, R., & Manocha, D. (2026) · doi:10.1109/cvprf71909.2026.00566
Lian, S., Wu, C., Yang, L. T., Yuan, H., Yu, B., Zhang, L., & Chen, K. (2026) · doi:10.1109/cvprf71909.2026.00940
Shin, J., Kim, J., Ko, D., & Choi, J. (2026) · doi:10.1109/cvprf71909.2026.00216
Liu, Z., Wei, Y., Feng, L., Su, X., Xia, X., Guan, W., Xie, Z., & Yang, S. (2026) · doi:10.1109/cvprf71909.2026.00919
Reasoning for Mobile User Experience with Multimodal LLMs: Task, Benchmark, and Approach
Mao, R., Fang, Z., Guo, T., Yang, H., Li, Y., Peng, S., Huang, M., Lin, X., Liu, S., Li, X., et al. (2026) · doi:10.1109/cvprf71909.2026.00862
Frequency-Modulated Visual Restoration for Matryoshka Large Multimodal Models
Pan, Q., Dou, Z., & Li, S. (2026) · doi:10.1109/cvprf71909.2026.00931
Liu, Z., Zhou, S., Liu, M., Deng, H., & Zhu, H. (2026) · doi:10.1109/cvprf71909.2026.00593
QuPAINT: Physics-Aware Instruction Tuning Approach to Quantum Material Discovery
Nguyen, X.-B., Nguyen, H.-Q., Pandey, S., Faltermeier, T., Borys, N., Churchill, H., & Luu, K. (2026) · doi:10.1109/cvprf71909.2026.00833
Vision Inference Former: Sustaining Visual Consistency in Multimodal Large Language Models
Dong, X., Zhang, M., Han, K., Tan, X., Wu, F., & Kuang, K. (2026) · 2026 IEEE/CVF Conference on Computer Vision and Pattern Recognition- FINDINGS Track (CVPRF) · doi:10.1109/cvprf71909.2026.00586
Beyond Static Artifacts: A Forensic Benchmark for Video Deepfake Reasoning in Vision Language Models
Gu, Z., Zhao, Q., Wang, Y., Huang, Z., Xinqi, L. I., Yuan, C., Shao, J., Zhang, C., & Li, X. (2026) · doi:10.1109/cvprf71909.2026.00824 · Cited by 1
Myopia Rectification: KV Cache Pruning for MLLMs via Dynamic Attention Subsidy and Token Reclamation
Zhuang, J., Lu, L., Dai, M., Chen, J., Liu, Q., & Hu, H. (2026) · doi:10.1109/cvprf71909.2026.00866
Self-citationVideoScaffold: Elastic-Scale Visual Hierarchies for Streaming Video Understanding in MLLMs
Zheng, N., Guo, Q., Huang, J., & Zhao, F. (2026) · doi:10.1109/cvprf71909.2026.00571
Parallel In-Context Learning for Large Vision Language Models
Yamaguchi, S., Chijiwa, D., Sakao, T., & Hasegawa, T. (2026) · doi:10.1109/cvprf71909.2026.00557
Clash: a Benchmark for Cross-Modal Contradiction Detection
Popordanoska, T., Li, J., & Blaschko, M. B. (2026) · doi:10.1109/cvprf71909.2026.00581
Aboukhadra, A. T., Rogge, M., Robertini, N., Arafa, A., Malik, J., Elhayek, A., & Stricker, D. (2026) · doi:10.1109/cvprf71909.2026.00330
Elicit and Enhance: Advancing Multimodal Reasoning in Medical Scenarios
Huang, Z., Mu, L., Gu, Y., Hu, K., Hua, S., & Zhang, X. (2026) · 2026 IEEE/CVF Conference on Computer Vision and Pattern Recognition- FINDINGS Track (CVPRF) · doi:10.1109/cvprf71909.2026.00539
ProSM: Progressive Soft Masking for Fine-Grained Remote Image Segmentation
Nian, B., Tang, F., Ning, Z., Jiang, D., Li, Y., Yang, J., Xiao, R., Zhou, S. K., & Liu, W. (2026) · doi:10.1109/cvprf71909.2026.00612
M-DocSum: Do LVLMs Genuinely Comprehend Interleaved Image-Text in Document Summarization?
Yan, H., Tan, K., Shen, Y., Huang, X., Wang, J., Ge, Z., Zhang, X., Li, S., & Jiang, D. (2026) · doi:10.1109/cvprf71909.2026.00266
Jena, P. K., Baburaj, B., Anand, T., Dutta, V., Ulavala, V., & Ali, S. A. (2026) · doi:10.1109/cvprf71909.2026.00188
EggHand: A Multimodal Foundation Model for Egocentric Hand Pose Forecasting
Choi, J., Kim, H., Kim, Y., & Park, D. (2026) · doi:10.1109/cvprf71909.2026.00342
Garcia, G. L., Passos, L. A., Manesco, J. R. R., Paiola, P. H., Correia, J. V. M., & Papa, J. P. (2026) · Proceedings - IEEE Symposium on Computer-Based Medical Systems · doi:10.1109/cbms69103.2026.00212
TARP: Text-Augmented Re-centered Projection for few-shot CLIP adaptation
Liu, C., Tang, G., Zhao, R., & Wang, L. (2026) · Computer Vision and Image Understanding · doi:10.1016/j.cviu.2026.104973
Task-Adaptive Reinforcement Learning for Unified OCR in Maritime and Complex Documents
Gao, Y., Zou, Y., Wang, X., & Wang, S. (2026) · Electronics · doi:10.3390/electronics15204684
Khan, G., Rani, M., Ahmed, W., Pimbblet, K., & Wertheim, K. Y. (2026) · Machine Learning with Applications · doi:10.1016/j.mlwa.2026.101042
CART-Doc: A Compact Design Space for Confidence-Routed Document Key Information Extraction
Oladele, D. A., & Sibiya, M. (2026) · Lecture notes in computer science · doi:10.1007/978-981-92-6510-7_6
Mohammad, F., Al-Ahmadi, S. A., Saeed, F., & Hussain, N. (2026) · Scientific Reports · doi:10.1038/s41598-026-75678-8
Zhou, J. (2026) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn69854.2026.11729497
Liu, X., Bai, Y., Lian, P., HaiFeng, C., & Li, H. (2026) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn69854.2026.11729283
Huang, X., Zhao, J., Liu, X., Xu, J., Zhang, Z., Luo, Z., & Chen, X. (2026) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn69854.2026.11729404
Fujitomi, T., Sogi, N., Shibata, T., & Terao, M. (2026) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn69854.2026.11728594
A Parcel-Level Land-Use Understanding Model Integrating Multi-Source Geographical Data
Yang, R., Su, Y., Chen, Z., Li, Y., Karanja, F., Boitt, M. K., Pellikka, P., Liu, Y., & Zhong, Y. (2026) · IEEE International Geoscience and Remote Sensing Symposium (IGARSS) · doi:10.1109/igarss60577.2026.11731194
Self-citationCausalDrive: Causal Reasoning Supervision for Interpretable Vision-Language-Action Driving
Yu, X., Lin, Z., Liu, C., Zhou, Z., & Zhang, F. (2026) · IEEE International Conference on Intelligent Systems · doi:10.1109/is71416.2026.11727104
MAAP-VL: Modality-aware adaptive probing for reliability diagnosis of vision-language models
Wang, X., Zhan, Y., Liu, Z., Wang, K., Lian, S., & Du, J. (2026) · Image and Vision Computing · doi:10.1016/j.imavis.2026.106249
Zheng, Y., Pan, S., Zhu, Z., Liu, Y., Ge, Y., Zhang, Y., Chen, Z., Xu, C., Yang, X., Jin, Y., et al. (2026) · International Journal of Computer Vision · doi:10.1007/s11263-026-03056-y
Self-citationMustapha, G. M., Ibrahim, N. M., Muhammed, A. S., & Ogar, A. O. (2026) · FUDMA Journal of Sciences · doi:10.33003/fjs-2026-1019-5746
A Comparative Assessment of Adversarial Robustness in Vision-Language Models
Yiğit, G., & Kale, A. (2026) · International Artificial Intelligence and Data Processing Symposium (IDAP) · doi:10.1109/idap71869.2026.11728083
CultureVLM: Characterizing and Improving Cultural Understanding of Vision-Language Models
Liu, S., Jin, Y., Li, C., Wen, Q., Sun, L., Chen, H., Wong, D. F., & Wang, J. (2026) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/2026.findings-emnlp.1603
Dimple: Discrete Diffusion Parallel Generation for Large Multimodal Modal
Yu, R., Ma, X., & Wang, X. (2026) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/2026.emnlp-main.824
Enhancing Multimodal Chain-of-Thought via Counterfactual-Symbolic Constrained Reasoning
Ding, X. (2026) · International Conference on Computer Engineering and Application (ICCEA) · doi:10.1109/iccea69962.2026.11743618
SigLIP-HGC: Hierarchical Image Captioning via Gated Cross-Scale Fusion
Hani, A., Tagougui, N., & Kherallah, M. (2026) · Procedia Computer Science · doi:10.1016/j.procs.2026.10.082
Li, D., Chen, R., Lyu, W., Lian, H., Liu, C., Wu, G., Liu, W., & Jing, F. (2026) · ACM International Conference on Information and Knowledge Management (CIKM) · doi:10.1145/3799682.3841110
de Vasconcelos, B. J. R., & Ferraz, C. A. G. (2026) · Brazilian Symposium on Multimedia and the Web · doi:10.5753/webmedia.2026.30811
Li, S., Wang, P., Liu, A., Yan, Y., Sun, F., & Tan, D. (2026) · International Conference on Energy Materials and Electrical Engineering (ICEMEE) · doi:10.1109/icemee71303.2026.11748488
Metadata
- Authors
- Chen, Z., Wu, J., Wang, W., Su, W., Chen, G., Xing, S., Zhong, M., Zhang, Q., Zhu, X., Lu, L., Li, B., Luo, P., Lu, T., Qiao, Y., & Dai, J.
- Affiliations
- Nanjing University (CN)
- University of Hong Kong (HK)
- Chinese University of Hong Kong (HK)
- University of Science and Technology of China (CN)
- Tsinghua University (CN)
- Shanghai Artificial Intelligence Laboratory (CN)
- Sensetime (China) (CN)
- Nanjing University
- Tsinghua University
- Venue
- 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)
- Publisher
- IEEE
- Published
- 2024-06-16
- Pages
- 24185-24198
- ISSN
- 1063-6919 (ISSN-L)
- OpenAlex
- W4402713111
- Cited by
- 734
- References
- 145
- Citations by year
- 2026: 360 · 2025: 331 · 2024: 43
- FWCI
- 151.51
- Type
- conference paper
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Funders
- National Natural Science Foundation of China (62376134,62372223)
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Multimodal Machine Learning Applications
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
- Physical Sciences › Computer Science › Artificial Intelligence › Domain Adaptation and Few-Shot Learning
- Keywords
- vision-language foundation models, vision foundation models, LLM alignment, zero-shot image classification, image-text retrieval, visual perception
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
