CoChat
Sign Up
Cited by 843Multimodal Machine Learning ApplicationsAdvanced Neural Network ApplicationsDomain Adaptation and Few-Shot Learning

The DOI matches its Crossref record; 138 of the 145 references it cites were found in a registry or an index.

Intern VL: Scaling up Vision Foundation Models and Aligning for Generic Visual-Linguistic Tasks

Open access
Chen, Zhe, Wu, Jiannan, Wang, Wenhai, Su, Weijie, Chen, Guo, Xing, Sen, Zhong, Muyan, Zhang, Qinglong, Zhu, Xizhou, Lu, Lewei, Li, Bin, Luo, Ping, Lu, Tong, Qiao, Yu, Dai, Jifeng·2024·2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)
DOI 10.1109/cvpr52733.2024.02283
Open this paper in CoChat — free accountOpen PDF

Abstract

The exponential growth of large language models (LLMs) has opened up numerous possibilities for multi-modal AGI systems. However, the progress in vision and vision-language foundation models, which are also critical elements of multi-modal AGI, has not kept pace with LLMs. In this work, we design a large-scale vision-language foun-dation model (Intern VL), which scales up the vision foun-dation model to 6 billion parameters and progressively aligns it with the LLM, using web-scale image-text data from various sources. This model can be broadly applied to and achieve state-of-the-art performance on 32 generic visual-linguistic benchmarks including visual perception tasks such as image-level or pixel-level recognition, vision-language tasks such as zero-shot image/video classification, zero-shot image/video-text retrieval, and link with LLMs to create multi-modal dialogue systems. It has powerful visual capabilities and can be a good alternative to the ViT-22B. We hope that our research could contribute to the development of multi-modal large models.

Overview

Integrity signals

Registry

Crossref · 10.1109/cvpr52733.2024.02283

Cites

138 of 138 references verified0 citation errors · 0 not checkable

Statistics

Recomputing p-values needs the paper’s full text.

Verification

Claims the paper makes

We couldn’t retrieve this paper’s full text, so its claims and self-check weren’t run. The registry and citation checks still apply.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Citations

Cites

Flagged first, then by citations

Show all 145 references

7 of the 145 references couldn’t be matched to a known paper.

Research works printed but not linked: 23

  1. Achiam, 2023, Gpt-4 technical report, arXiv preprint
  2. Bai, 2023, Qwen tech-nical report, arXiv preprint
  3. Bai, 2023, Qwen-vl: A frontier large vision-language model with versatile abilities, arXiv preprint
  4. Byeon, 2022, Coyo-700m: Image-text pair dataset
  5. Carlsson, 2022, Cross-lingual and multilingual clip, LREC, 6848
  6. Chen, 2023, Shikra: Unleashing multi-modal 11ms referential dialogue magic, arXiv preprint
  7. Dosovitskiy, 2020, An image is worth 16×16 words: Transformers for image recognition at scale, ICLR
  8. Goyal, 2017, Making the v in vq a matter: El-evating the role of image understanding in visual question answering, CVPR, 6904
  9. 2023, Introducing idefics: An open reproduction of state-of-the-art visual language model
  10. Ilharco, 2021, Open-clip. Zenodo. Version 0.1
  11. 2023, Clip benchmark: Clip-like model evaluation
  12. Li, 2023, Blip-2: Bootstrapping language-image pre-training with frozen image encoders and large language models, 19730
  13. Li, 2019, Coco-en for cross-lingual image tagging, captioning, and retrieval, TMM, 21, 2347
  14. Liu, 2023, Aligning large multi-modal model with robust instruction tuning, arXiv preprint
  15. Sawada, 2022, Construction and public release of language image pretraining models in japanese, MIRU
  16. Oquab, 2023, Dinov2: Learning robust visual features with-out supervision, TMLR
  17. Schuhmann, 2022, Laion coco:600m synthetic captions from laion2b-en
  18. Sun, 2024, Generative pretraining in mul-timodality, ICLR
  19. Sun, 2023, Moss: Training conversationallanguage models from synthetic data, arXiv preprint, 7
  20. Taori, 2023, Alpaca: A strong, replicable instruction-following model, Stanford Center for Research on Foundation Models, 3, 7
  21. Team, 2023, Internlm: A multilingual language model with progressively enhanced capabilities
  22. Xie, 2022, Zero and r2d2: A large-scale chi-nese cross-modal benchmark and a vision-language frame-work, arXiv preprint
  23. Zhu, 2023, Ghost in the minecraft: Generally capable agents for open-world enviroments via large language models with text-based knowledge and memory, arXiv preprint

Cited by

Sorted by citations

Items 701–800 of 843

Related

See all

Metadata

Authors
Chen, Z., Wu, J., Wang, W., Su, W., Chen, G., Xing, S., Zhong, M., Zhang, Q., Zhu, X., Lu, L., Li, B., Luo, P., Lu, T., Qiao, Y., & Dai, J.
Affiliations
  • Nanjing University (CN)
  • University of Hong Kong (HK)
  • Chinese University of Hong Kong (HK)
  • University of Science and Technology of China (CN)
  • Tsinghua University (CN)
  • Shanghai Artificial Intelligence Laboratory (CN)
  • Sensetime (China) (CN)
  • Nanjing University
  • Tsinghua University
ORCID
Venue
2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)
Publisher
IEEE
Published
2024-06-16
Pages
24185-24198
ISSN
1063-6919 (ISSN-L)
DOI
10.1109/cvpr52733.2024.02283
OpenAlex
W4402713111
Cited by
734
References
145
Citations by year
2026: 360 · 2025: 331 · 2024: 43
FWCI
151.51
Type
conference paper
Language
en
Open access
Licence
Locations
  • IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings (publishedVersion)
  • arXiv (Cornell University) (submittedVersion)
  • arXiv (Cornell University)
Funders
National Natural Science Foundation of China (62376134,62372223)
Topics
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Multimodal Machine Learning Applications
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
  • Physical Sciences › Computer Science › Artificial Intelligence › Domain Adaptation and Few-Shot Learning
Keywords
vision-language foundation models, vision foundation models, LLM alignment, zero-shot image classification, image-text retrieval, visual perception

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review