Student or educator? Sign up with your school email and get 6 months of Pro free.
CoChat

The DOI matches its Crossref record; 220 of the 220 references it cites were found in a registry or an index.

Foundation Models Defining a New Era in Vision: A Survey and Outlook

Awais, Muhammad, Naseer, Muzammal, Khan, Salman, Anwer, Rao Muhammad, Cholakkal, Hisham, Shah, Mubarak, Yang, Ming-Hsuan, Khan, Fahad Shahbaz · 2025 · IEEE Transactions on Pattern Analysis and Machine Intelligence

Abstract

Vision systems that see and reason about the compositional nature of visual scenes are fundamental to understanding our world. The complex relations between objects and their locations, ambiguities, and variations in the real-world environment can be better described in human language, naturally governed by grammatical rules and other modalities such as audio and depth. The models learned to bridge the gap between such modalities and large-scale training data facilitate contextual reasoning, generalization, and prompt capabilities at test time. These models are referred to as foundation models. The output of such models can be modified through human-provided prompts without retraining, e.g., segmenting a particular object by providing a bounding box, having interactive dialogues by asking questions about an image or video scene or manipulating the robot's behavior through language instructions. In this survey, we provide a comprehensive review of such emerging foundation models, including typical architecture designs to combine different modalities (vision, text, audio, etc.), training objectives (contrastive, generative), pre-training datasets, fine-tuning mechanisms, and the common prompting patterns; textual, visual, and heterogeneous. We discuss the open challenges and research directions for foundation models in computer vision, including difficulties in their evaluations and benchmarking, gaps in their real-world understanding, limitations of contextual understanding, biases, vulnerability to adversarial attacks, and interpretability issues. We review recent developments in this field, covering a wide range of applications of foundation models systematically and comprehensively.

Overview

Integrity signals

Registry

Crossref · 10.1109/tpami.2024.3506283

Cites

220 of 220 references verified0 citation errors · 0 not checkable

Statistics

Recomputing p-values needs the paper’s full text.

Verification

Claims the paper makes

This paper’s full text isn’t openly available, so its claims and self-check weren’t run. The registry and citation checks still apply.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Citations

Cites

Show all 198 references

Research works printed but not linked: 10

  1. Radford, 2019, Language models are unsupervised multitask learners, OpenAI Blog, 1
  2. Schuhmann, LAION-5B: An open large-scale dataset for training next generation image-text models, 25278
  3. 10.5555/3524938.3525087 · doi:10.5555/3524938.3525087
  4. Liu, 2019, RoBERTa: A robustly optimized bert pretraining approach, ArXiv
  5. Zhang, GLIPv2: Unifying localization and vision-language understanding, 36067
  6. Kwon, Masked vision and language modeling for multi-modal representation learning, 4
  7. Ordonez, Im2text: Describing images using 1 million captioned photographs, 3
  8. 2023, GPT-4 technical report, PREPRINT
  9. 10.18653/v1/2023.emnlp-main.20 · DOI · Paper page
  10. Hu, LoRA: Low-rank adaptation of large language models, 3

Cited by

See all 417

Metadata

Authors
Awais, M., Naseer, M., Khan, S., Anwer, R. M., Cholakkal, H., Shah, M., Yang, M.-H., & Khan, F. S.
Affiliations
  • MBZ University of AI, Abu Dhabi, UAE
  • Australian National University (AU)
  • Mohamed bin Zayed University of Artificial Intelligence (AE)
  • University of Central Florida (US)
  • University of California, Merced (US)
  • Linköping University (SE)
Venue
IEEE Transactions on Pattern Analysis and Machine Intelligence
Published
2025-04
Volume
47
Issue
4
Pages
2245-2264
ISSN
  • 0162-8828 (ISSN-L)
  • 2160-9292
  • 1939-3539
OpenAlex
W4406208390
Cited by
304
References
198
Citations by year
2026: 184 · 2025: 118
FWCI
105.44
Type
journal article
Language
en
Open access
closed
Locations
  • IEEE Transactions on Pattern Analysis and Machine Intelligence (publishedVersion)
  • PubMed (publishedVersion)
Topics
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Multimodal Machine Learning Applications
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
  • Physical Sciences › Computer Science › Artificial Intelligence › Domain Adaptation and Few-Shot Learning
Keywords
vision foundation models, multimodal models, vision-language models, prompting, pre-training dataset, fine-tuning, contrastive learning, generative models, benchmarking, adversarial attacks, interpretability

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: Foundation Models Defining a New Era in Vision: A Survey and Outlook. https://app.cochat.ai/papers/10.1109/tpami.2024.3506283
The paper
Awais, M., Naseer, M., Khan, S., Anwer, R. M., Cholakkal, H., Shah, M., Yang, M.-H., & Khan, F. S. (2025). Foundation Models Defining a New Era in Vision: A Survey and Outlook. IEEE Transactions on Pattern Analysis and Machine Intelligence. https://doi.org/10.1109/tpami.2024.3506283

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.