Cited by 1,051Multimodal Machine Learning ApplicationsDomain Adaptation and Few-Shot LearningAdvanced Neural Network Applications

The DOI matches its Crossref record; 194 of the 202 references it cites were found in a registry or an index.

Vision-Language Models for Vision Tasks: A Survey

Open access
Zhang, Jingyi, Huang, Jiaxing, Jin, Sheng, Lu, Shijian·2024·IEEE Transactions on Pattern Analysis and Machine Intelligence

Abstract

Most visual recognition studies rely heavily on crowd-labelled data in deep neural networks (DNNs) training, and they usually train a DNN for each single visual recognition task, leading to a laborious and time-consuming visual recognition paradigm. To address the two challenges, Vision-Language Models (VLMs) have been intensively investigated recently, which learns rich vision-language correlation from web-scale image-text pairs that are almost infinitely available on the Internet and enables zero-shot predictions on various visual recognition tasks with a single VLM. This paper provides a systematic review of visual language models for various visual recognition tasks, including: (1) the background that introduces the development of visual recognition paradigms; (2) the foundations of VLM that summarize the widely-adopted network architectures, pre-training objectives, and downstream tasks; (3) the widely-adopted datasets in VLM pre-training and evaluations; (4) the review and categorization of existing VLM pre-training methods, VLM transfer learning methods, and VLM knowledge distillation methods; (5) the benchmarking, analysis and discussion of the reviewed methods; (6) several research challenges and potential research directions that could be pursued in the future VLM studies for visual recognition.

Overview

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1109/tpami.2024.3369699

Cites

194 of 194 references verified0 citation errors · 0 not checkable

Statistics

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Flagged first, then by citations

Show all 202 references

8 of the 202 references couldn’t be matched to a known paper.

Research works printed but not linked: 5

  1. 10.5555/3524938.3525087 · DOI
  2. Radford, 2018, Improving language understanding by generative pre-training, OpenAI Blog
  3. Radford, 2019, Language models are unsupervised multitask learners, OpenAI Blog, 1
  4. 10.1109/cvpr52688.2022.01553 · DOI · Paper page
  5. Ordonez, Im2Text: Describing images using 1 million captioned photographs, 1143

Cited by

Sorted by citations

Items 101–200 of 1,002

Metadata

Authors
Zhang, J., Huang, J., Jin, S., & Lu, S.
Affiliations
Nanyang Technological University (SG)
Venue
IEEE Transactions on Pattern Analysis and Machine Intelligence
Published
2024-08
Volume
46
Issue
8
Pages
5625-5644
ISSN
  • 0162-8828 (ISSN-L)
  • 2160-9292
  • 1939-3539
OpenAlex
W4392172801
Cited by
1,051
References
202
Citations by year
2026: 432 · 2025: 486 · 2024: 124
FWCI
132.68
Type
journal article
Language
en
Open access
Locations
  • IEEE Transactions on Pattern Analysis and Machine Intelligence (publishedVersion)
  • PubMed (publishedVersion)
  • arXiv (Cornell University) (submittedVersion)
  • DR-NTU (Nanyang Technological University) (submittedVersion)
  • arXiv (Cornell University)
Topics
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Multimodal Machine Learning Applications
  • Physical Sciences › Computer Science › Artificial Intelligence › Domain Adaptation and Few-Shot Learning
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
Keywords
vision-language models, visual recognition, zero-shot learning, transfer learning, knowledge distillation, image-text dataset, pre-training objectives, network architecture, benchmarking

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review