The DOI matches its DataCite record; 185 of the 193 references it cites were found in a registry or an index, and 1 are not checked yet.
Learning Transferable Visual Models From Natural Language Supervision
Open accessAbstract
State-of-the-art computer vision systems are trained to predict a fixed set of predetermined object categories. This restricted form of supervision limits their generality and usability since additional labeled data is needed to specify any other visual concept. Learning directly from raw text about images is a promising alternative which leverages a much broader source of supervision. We demonstrate that the simple pre-training task of predicting which caption goes with which image is an efficient and scalable way to learn SOTA image representations from scratch on a dataset of 400 million (image, text) pairs collected from the internet. After pre-training, natural language is used to reference learned visual concepts (or describe new ones) enabling zero-shot transfer of the model to downstream tasks. We study the performance of this approach by benchmarking on over 30 different existing computer vision datasets, spanning tasks such as OCR, action recognition in videos, geo-localization, and many types of fine-grained object classification. The model transfers non-trivially to most tasks and is often competitive with a fully supervised baseline without the need for any dataset specific training. For instance, we match the accuracy of the original ResNet-50 on ImageNet zero-shot without needing to use any of the 1.28 million training examples it was trained on. We release our code and pre-trained model weights at https://github.com/OpenAI/CLIP.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source doesn’t support the claim as written.
In the citing paper
“During training,the model learns the joint distribution between the two modalities and optimizes the model through methods such as adversarial training or maximizing likelihood, enabling it to generate the target modality”
From Multimodal fusion and vision–language models: A survey for robot vision (2026) · cites this paper as [188]
Checked by a reasoning model — may be wrong.
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Large Image Models (LIMs) such as CLIP [86] and BLIP-2 [87] extend the agent's capabilities into the visual domain. Trained on image-text pairs, LIMs enable perception-based tasks including image classification, object detection, and vision-language grounding.”
From AI Agents vs. Agentic AI: A Conceptual taxonomy, applications and challenges (2026) · cites this paper as [86]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Foundation Models (FMs) [23, 35, 37] - generic models trained on web-scale datasets - have demonstrated remarkable zero-shot performance and can be quickly adapted to various downstream tasks with few additional tuning cost.”
From Towards More General Video-based Deepfake Detection through Facial Component Guided Adaptation for Foundation Model (2024) · cites this paper as [37]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“By encoding text descriptions of variations of the dataset bias (e.g.'a blue wall'), using CLIP's text encoder (CLIP T ), we vary visual representations without actually having seen images containing these variations (e.g. an image of a blue wall).”
From Language-Based Augmentation to Address Shortcut Learning in Object-Goal Navigation (2023) · cites this paper as [15]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“However, their capacity for semantic comprehension is comparatively limited when contrasted with vision-based perceptual models like CLIP (Radford et al., 2021).”
From WorldVLA: Towards Autoregressive Action World Model (2025) · cites this paper as Radford et al., 2021
Experimental — accuracy not yet measured
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“In natural language [1] and computer vision [39], general-purpose foundation models that are pre-trained on diverse multi-task data tend to outperform narrowly tailored and specialized solutions.”
From $π_0$: A Vision-Language-Action Flow Model for General Robot Control (2024) · cites this paper as [39]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Existing evaluation metrics such as CLIP-Image score [37] and DINO score [6], while commonly used, often fail to reflect subtle differences due to their reliance on high-dimensional semantic features for cosine similarity computation.”
From DiffSim: Taming Diffusion Models for Evaluating Visual Similarity (2024) · cites this paper as [37]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“CLIP cosine similarity is often used to measure text prompt and frame consistency, where a reference video dataset is not needed.”
From Video Diffusion Models: A Survey (2024) · cites this paper as Radford et al., 2021
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Trained on image-text pairs, LIMs enable perception-based tasks including image classification, object detection, and vision-language grounding.”
From AI Agents vs. Agentic AI: A Conceptual Taxonomy, Applications and Challenges (2025) · cites this paper as [88]
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“Vision-Language Models (VLMs) e.g., CLIP [15], allows us to augment at feature-level based on prior knowledge of the environment without any changes to training data.”
From Language-Based Augmentation to Address Shortcut Learning in Object-Goal Navigation (2023) · cites this paper as [15]
The cited source says what the claim attributes to it.
In the citing paper
“We find these methods perform poorly on language and image goals, particularly when relying on CLIP [25] features.”
From GOAT-Bench: A Benchmark for Multi-Modal Lifelong Navigation (2024) · cites this paper as [25]
The cited source says what the claim attributes to it.
In the citing paper
“A notable advancement in this domain is the recent work on CLIP [9, 32, 35, 43], which elevates open-vocabulary recognition performance by collecting millions of image-text pairs and employing contrastive learning techniques.”
From Active Open-Vocabulary Recognition: Let Intelligent Moving Mitigate CLIP Limitations (2024) · cites this paper as [32]
The cited source says what the claim attributes to it.
In the citing paper
“The recently proposed Contrastive Language-Image Pretraining (CLIP) [1] learns the correspondence between text and image by projecting them into a shared latent space.”
From Large-Scale Contrastive Language-Audio Pretraining with Feature Fusion and Keyword-to-Caption Augmentation (2023) · cites this paper as [1]
The cited source says what the claim attributes to it.
In the citing paper
“To evaluate the visual similarity of I R and I G , we use the similarity of their CLIP (Radford et al., 2021) embedding, denoted as CLIP ( I R , I G ) .”
From Design2Code: Benchmarking Multimodal Code Generation for Automated Front-End Engineering (2025) · cites this paper as Radford et al., 2021
The cited source says what the claim attributes to it.
In the citing paper
“This suggests the inability of CLIP [25] features in capturing crucial instance-specific and spatial features in language and image goals.”
From GOAT-Bench: A Benchmark for Multi-Modal Lifelong Navigation (2024) · cites this paper as [25]
The cited source says what the claim attributes to it.
In the citing paper
“Existing solutions in multimodal Transformer adopt a fixed fusion strategy [18,21,24,26,33].”
From Are Multimodal Transformers Robust to Missing Modality? (2022) · cites this paper as [33]
The cited source says what the claim attributes to it.
In the citing paper
“existing foundation models for vision and language such as CLIP [8], SigLIP [9], and Llama 2 [10] are capable of these types of generalization and more, stemming from the priors captured by their Internet-scale pretraining datasets.”
From OpenVLA: An Open-Source Vision-Language-Action Model (2024) · cites this paper as [8]
The cited source says what the claim attributes to it.
In the citing paper
“which shows that our model achieves the highest DINO [5] and CLIP-I [42] scores, with CLIP-T [42] scores that are comparable to other methods.”
From DreamingComics: A Story Visualization Pipeline via Subject and Layout Customized Generation using Video Models (2025) · cites this paper as [42]
The cited source says what the claim attributes to it.
In the citing paper
“In early 2021, OpenAI released an architecture termed Contrastive Language Image Pretraining (CLIP), which, when trained on millions of image-text pairs, matched the performance of competitive, fully supervised models without fine-tuning 106”
From Multimodal biomedical AI (2022) · cites this paper as [106]
The cited source says what the claim attributes to it.
In the citing paper
“Even when PwD data is available, data from marginalized groups is often automatically filtered out by techniques for removing 'low' quality data [57, 104], which rely on biased models such as CLIP [92].”
From Engaging Communities Meaningfully in Defining Disability Representation for AI Image Generation (2026) · cites this paper as [92]
The check wasn’t confident either way, so it gives no verdict — never disputed.
In the citing paper
“HOLODECK uses a Large Language Model (GPT-4 [37]) to design the floor plan, assign suitable materials, install the doorways and windows and arrange 3D assets coherently in the scene using constraintbased optimization.”
From Holodeck: Language Guided Generation of 3D Embodied AI Environments (2024) · cites this paper as [37]
Experimental — accuracy not yet measuredThe check wasn’t confident either way, so it gives no verdict — never disputed.
In the citing paper
“[41] famously found evidence of a ‘cat neuron’, a phenomenon that has been remarkably replicable across newer models and analyses [14,42,43]”
From Symbols and grounding in large language models (2023) · cites this paper as [43]
Experimental — accuracy not yet measuredThe check wasn’t confident either way, so it gives no verdict — never disputed.
In the citing paper
“Text-to-image models like Stable Diffusion, Midjourney, and DALL-E [26,27] enable educators to create visual learning materials from textual content.”
From Promises and challenges of generative artificial intelligence for human learning (2024) · cites this paper as [26]
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., & Polosukhin, I. (2017) · arXiv · doi:10.48550/arxiv.1706.03762 · Cited by 120,404
Adam: A Method for Stochastic Optimization
Kingma, D. P., & Ba, J. (2014) · arXiv · doi:10.48550/arxiv.1412.6980 · Cited by 98,946
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2018) · arXiv · doi:10.48550/arxiv.1810.04805 · Cited by 73,173
Scikit-learn: Machine Learning in Python
Pedregosa, F., Varoquaux, G., Gramfort, A., Michel, V., Thirion, B., Grisel, O., Blondel, M., Müller, A., Nothman, J., Louppe, G., et al. (2012) · arXiv · doi:10.48550/arxiv.1201.0490 · Cited by 71,016
ImageNet: A large-scale hierarchical image database
Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., & Fei-Fei, L. (2009) · 2009 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2009.5206848 · Cited by 65,333
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2019) · Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) · doi:10.18653/v1/n19-1423 · Cited by 54,325
An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale
Dosovitskiy, A., Beyer, L., Kolesnikov, A., Weissenborn, D., Zhai, X., Unterthiner, T., Dehghani, M., Minderer, M., Heigold, G., Gelly, S., et al. (2020) · arXiv · doi:10.48550/arxiv.2010.11929 · Cited by 47,472
SciPy 1.0: fundamental algorithms for scientific computing in Python
Virtanen, P., Gommers, R., Oliphant, T. E., Haberland, M., Reddy, T., Cournapeau, D., Burovski, E., Peterson, P., Weckesser, W., Bright, J., et al. (2020) · Nature Methods · doi:10.1038/s41592-019-0686-2 · Cited by 38,430
Glove: Global Vectors for Word Representation
Pennington, J., Socher, R., & Manning, C. (2014) · Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.3115/v1/d14-1162 · Cited by 34,071
Research works printed but not linked: 24
- Bengio, Y., Ducharme, R., Vincent, P., and Jauvin, C. A neural probabilistic language model. Journal of machine learning research , 3(Feb):1137-1155, 2003.
- Browne, S. Dark Matters: Surveillance of Blackness . Duke University Press, 2015.
- Chen, Y.-C., Li, L., Yu, L., Kholy, A. E., Ahmed, F., Gan, Z., Cheng, Y., and Liu, J. Uniter: Learning universal imagetext representations. arXiv preprint arXiv:1909.11740 , 2019. · arXiv
- Coates, A., Ng, A., and Lee, H. An analysis of singlelayer networks in unsupervised feature learning. In Proceedings of the fourteenth international conference on artificial intelligence and statistics , pp. 215-223, 2011.
- Crawford, K. The trouble with bias. NIPS 2017 Keynote , 2017. URL https://www.youtube.com/ watch?v=fMym\_BKWQzk .
- Frome, A., Corrado, G. S., Shlens, J., Bengio, S., Dean, J., Ranzato, M., and Mikolov, T. Devise: A deep visualsemantic embedding model. In Advances in neural information processing systems , pp. 2121-2129, 2013.
- Geirhos, R., Rubisch, P., Michaelis, C., Bethge, M., Wichmann, F. A., and Brendel, W. Imagenet-trained cnns are biased towards texture; increasing shape bias improves accuracy and robustness. arXiv preprint arXiv:1811.12231 , 2018. · arXiv
- Goodfellow, I. J., Shlens, J., and Szegedy, C. Explaining and harnessing adversarial examples. arXiv preprint arXiv:1412.6572 , 2014. · arXiv
- Ha, D., Dai, A., and Le, Q. V. Hypernetworks. arXiv preprint arXiv:1609.09106 , 2016. · arXiv
- He, K., Zhang, X., Ren, S., and Sun, J. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition , pp. 770-778, 2016b.
- Hendrycks, D. and Dietterich, T. Benchmarking neural network robustness to common corruptions and perturbations. arXiv preprint arXiv:1903.12261 , 2019. · arXiv
- Hendrycks, D., Liu, X., Wallace, E., Dziedzic, A., Krishnan, R., and Song, D. Pretrained transformers improve out-ofdistribution robustness. arXiv preprint arXiv:2004.06100 , 2020b. · arXiv
- Ilyas, A., Santurkar, S., Tsipras, D., Engstrom, L., Tran, B., and Madry, A. Adversarial examples are not bugs, they are features. In Advances in Neural Information Processing Systems , pp. 125-136, 2019.
- Jaderberg, M., Simonyan, K., Zisserman, A., et al. Spatial transformer networks. Advances in neural information processing systems , 28:2017-2025, 2015.
- Johnson, J., Hariharan, B., van der Maaten, L., Fei-Fei, L., Lawrence Zitnick, C., and Girshick, R. Clevr: A diagnostic dataset for compositional language and elementary visual reasoning. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition , pp. 2901-2910, 2017.
- Kiros, R., Zhu, Y., Salakhutdinov, R. R., Zemel, R., Urtasun, R., Torralba, A., and Fidler, S. Skip-thought vectors. Advances in neural information processing systems , 28: 3294-3302, 2015.
- Krizhevsky, A., Sutskever, I., and Hinton, G. E. Imagenet classification with deep convolutional neural networks. In Advances in neural information processing systems , pp. 1097-1105, 2012.
- Lin, T.-Y., Maire, M., Belongie, S., Hays, J., Perona, P., Ramanan, D., Doll´ ar, P., and Zitnick, C. L. Microsoft coco: Common objects in context. In European conference on computer vision , pp. 740-755. Springer, 2014.
- Proceedings of the European Conference on Computer Vision (ECCV) , pp. 181-196, 2018.
- McCann, B., Bradbury, J., Xiong, C., and Socher, R. Learned in translation: Contextualized word vectors. In Advances in neural information processing systems , pp. 6294-6305, 2017.
- Socher, R. and Fei-Fei, L. Connecting modalities: Semisupervised segmentation and annotation of images using unaligned text corpora. In 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition , pp. 966-973. IEEE, 2010.
- Srivastava, N. and Salakhutdinov, R. Multimodal learning with deep boltzmann machines. In NIPS , 2012.
- Thomee, B., Shamma, D. A., Friedland, G., Elizalde, B., Ni, K., Poland, D., Borth, D., and Li, L.-J. Yfcc100m: The new data in multimedia research. Communications of the ACM , 59(2):64-73, 2016.
- Touvron, H., Vedaldi, A., Douze, M., and J´ egou, H. Fixing the train-test resolution discrepancy. In Advances in neural information processing systems , pp. 8252-8262, 2019.
Cited by
Sorted by citations
Swin Transformer: Hierarchical Vision Transformer using Shifted Windows
Liu, Z., Lin, Y., Cao, Y., Hu, H., Wei, Y., Zhang, Z., Lin, S., & Guo, B. (2021) · 2021 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00986 · Cited by 35,534
“Its tremendous success in the language domain has led researchers to investigate its adaptation to computer vision, where it has recently demonstrated promising results on certain tasks, specifically image classification [19] and joint vision-language modeling [43].”
BackgroundKirillov, A., Mintun, E., Ravi, N., Mao, H., Rolland, C., Gustafson, L., Xiao, T., Whitehead, S., Berg, A. C., Lo, W.-Y., et al. (2023) · 2023 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.00371 · Cited by 11,953
SegFormer: Simple and Efficient Design for Semantic Segmentation with Transformers
Xie, E., Wang, W., Yu, Z., Anandkumar, A., Alvarez, J. M., & Luo, P. (2021) · arXiv · doi:10.48550/arxiv.2105.15203 · Cited by 5,090
Han (2023) · doi:10.1109/tpami.2022.3152247 · Cited by 4,682
Adding Conditional Control to Text-to-Image Diffusion Models
Zhang, L., Rao, A., & Agrawala, M. (2023) · 2023 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.00355 · Cited by 4,547
Transformers in Vision: A Survey
Khan, S., Naseer, M., Hayat, M., Zamir, S. W., Khan, F. S., & Shah, M. (2022) · ACM Computing Surveys · doi:10.1145/3505244 · Cited by 3,595
Diffusion Models Beat GANs on Image Synthesis
Dhariwal, P., & Nichol, A. (2021) · arXiv · doi:10.48550/arxiv.2105.05233 · Cited by 3,362
Learning to Prompt for Vision-Language Models
Zhou, K., Yang, J., Loy, C. C., & Liu, Z. (2022) · International Journal of Computer Vision · doi:10.1007/s11263-022-01653-1 · Cited by 3,304
“We briefly introduce vision-language pre-training with a particular focus on CLIP (Radford et al., 2021).”
BackgroundMethodInfluentialSwin Transformer V2: Scaling Up Capacity and Resolution
Liu, Z., Hu, H., Lin, Y., Yao, Z., Xie, Z., Wei, Y., Ning, J., Cao, Y., Zhang, Z., Dong, L., et al. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.01170 · Cited by 2,985
Swin Transformer V2: Scaling Up Capacity and Resolution
Liu, Z., Hu, H., Lin, Y., Yao, Z., Xie, Z., Wei, Y., Ning, J., Cao, Y., Zhang, Z., Dong, L., et al. (2021) · arXiv · doi:10.48550/arxiv.2111.09883 · Cited by 2,679
On the Opportunities and Risks of Foundation Models
Bommasani, R., Hudson, D. A., Adeli, E., Altman, R., Arora, S., von Arx, S., Bernstein, M. S., Bohg, J., Bosselut, A., Brunskill, E., et al. (2021) · arXiv · doi:10.48550/arxiv.2108.07258 · Cited by 2,676
Hierarchical Text-Conditional Image Generation with CLIP Latents
Ramesh, A., Dhariwal, P., Nichol, A., Chu, C., & Chen, M. (2022) · arXiv · doi:10.48550/arxiv.2204.06125 · Cited by 2,511
“Recent progress in computer vision has been driven by scaling models on large datasets of captioned images collected from the internet [10, 44, 60, 39, 31, 16].”
BackgroundMethodSelf-citationZero-shot text-to-image generation
Ramesh, A., Pavlov, M., Goh, G., Gray, S., Voss, C., Radford, A., Chen, M., & Sutskever, I. (2021) · arXiv · doi:10.48550/arxiv.2102.12092 · Cited by 2,417
Self-citationDreamBooth: Fine Tuning Text-to-Image Diffusion Models for Subject-Driven Generation
Ruiz, N., Li, Y., Jampani, V., Pritch, Y., Rubinstein, M., & Aberman, K. (2023) · 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52729.2023.02155 · Cited by 2,295
Scalable Diffusion Models with Transformers
Peebles, W., & Xie, S. (2023) · 2023 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.00387 · Cited by 2,284
Diffusion Models in Vision: A Survey
Croitoru, F.-A., Hondru, V., Ionescu, R. T., & Shah, M. (2023) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2023.3261988 · Cited by 2,189
Conditional Prompt Learning for Vision-Language Models
Zhou, K., Yang, J., Loy, C. C., & Liu, Z. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.01631 · Cited by 2,069
Foundation models for generalist medical artificial intelligence
Moor, M., Banerjee, O., Abad, Z. S. H., Krumholz, H. M., Leskovec, J., Topol, E. J., & Rajpurkar, P. (2023) · Nature · doi:10.1038/s41586-023-05881-4 · Cited by 2,028
Scaling Up Visual and Vision-Language Representation Learning With Noisy Text Supervision
Jia, C., Yang, Y., Xia, Y., Chen, Y.-T., Parekh, Z., Pham, H., Le, Q. V., Sung, Y., Li, Z., & Duerig, T. (2021) · arXiv · doi:10.48550/arxiv.2102.05918 · Cited by 1,977
“, 2018), CLIP (Radford et al., 2021), BiT (Kolesnikov et al.”
BackgroundMethodResultInfluentialJia, M., Tang, L., Chen, B.-C., Cardie, C., Belongie, S., Hariharan, B., & Lim, S.-N. (2022) · Lecture Notes in Computer Science · doi:10.1007/978-3-031-19827-4_41 · Cited by 1,965
Robust Speech Recognition via Large-Scale Weak Supervision
Radford, A., Kim, J. W., Xu, T., Brockman, G., McLeavey, C., & Sutskever, I. (2022) · arXiv · doi:10.48550/arxiv.2212.04356 · Cited by 1,953
Self-citationPrompt-to-Prompt Image Editing with Cross Attention ControlPreprint
Hertz, A., Mokady, R., Tenenbaum, J., Aberman, K., Pritch, Y., & Cohen‐Or, D. (2022) · arXiv (Cornell University) · doi:10.48550/arxiv.2208.01626 · Cited by 1,894
Improved Baselines with Visual Instruction Tuning
Liu, H., Li, C., Li, Y., & Lee, Y. J. (2024) · 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52733.2024.02484 · Cited by 1,748
Li, J., Li, D., Xiong, C., & Hoi, S. (2022) · arXiv · doi:10.48550/arxiv.2201.12086 · Cited by 1,741
Accurate medium-range global weather forecasting with 3D neural networks
Bi, K., Xie, L., Zhang, H., Chen, X., Gu, X., & Tian, Q. (2023) · Nature · doi:10.1038/s41586-023-06185-3 · Cited by 1,670
An Empirical Study of Training Self-Supervised Vision Transformers
Chen, X., Xie, S., & He, K. (2021) · 2021 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00950 · Cited by 1,648
CLIP-Adapter: Better Vision-Language Models with Feature Adapters
Gao, P., Geng, S., Zhang, R., Ma, T., Fang, R., Zhang, Y., Li, H., & Qiao, Y. (2023) · International Journal of Computer Vision · doi:10.1007/s11263-023-01891-x · Cited by 1,589
Flamingo: a Visual Language Model for Few-Shot Learning
Alayrac, J.-B., Donahue, J., Luc, P., Miech, A., Barr, I., Hasson, Y., Lenc, K., Mensch, A., Millican, K., Reynolds, M., et al. (2022) · arXiv · doi:10.48550/arxiv.2204.14198 · Cited by 1,555
Transformers in medical imaging: A survey
Shamshad, F., Khan, S., Zamir, S. W., Khan, M. H., Hayat, M., Khan, F. S., & Fu, H. (2023) · Medical Image Analysis · doi:10.1016/j.media.2023.102802 · Cited by 1,523
Instructpix2pix: Learning to follow image editing instructions
Brooks, T., Holynski, A., & Efros, A. A. (2023) · 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52729.2023.01764 · Cited by 1,466
Mou, C., Wang, X., Xie, L., Wu, Y., Zhang, J., Qi, Z., & Shan, Y. (2024) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v38i5.28226 · Cited by 1,477
A Brief Overview of ChatGPT: The History, Status Quo and Potential Future Development
Wu, T., He, S., Liu, J., Sun, S., Liu, K., Han, Q.-L., & Tang, Y. (2023) · IEEE/CAA Journal of Automatica Sinica · doi:10.1109/jas.2023.123618 · Cited by 1,452
Evaluating Large Language Models Trained on Code
Chen, M., Tworek, J., Jun, H., Yuan, Q., Pinto, H. P. D. O., Kaplan, J., Edwards, H., Burda, Y., Joseph, N., Brockman, G., et al. (2021) · arXiv · doi:10.48550/arxiv.2107.03374 · Cited by 1,436
“Generative models have been shown to encode bias in modalities such as natural language (Brown et al., 2020; Blodgett et al., 2020) and images (Radford et al., 2021), and we find that the same is true of models like Codex that generate code.”
BackgroundMethodSelf-citationMultiscale vision transformers
Fan, H., Xiong, B., Mangalam, K., Li, Y., Yan, Z., Malik, J., & Feichtenhofer, C. (2021) · 2021 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00675 · Cited by 1,407
“An emerging thread of work aims at applying transformers to vision tasks such as object detection [5], semantic segmentation [115, 99], 3D reconstruction [72], pose estimation [107], generative modeling [14], image retrieval [27], medical image segmentation [13, 97, 111], point clouds [40], video instance segmentation…”
BackgroundA Survey of AI-Generated Content (AIGC)
Cao, Y., Li, S., Liu, Y., Yan, Z., Dai, Y., Yu, P. S., & Sun, L. (2024) · ACM Computing Surveys · doi:10.1145/3704262 · Cited by 1,457
Acosta, J. N., Falcone, G. J., Rajpurkar, P., & Topol, E. J. (2022) · Nature Medicine · doi:10.1038/s41591-022-01981-2 · Cited by 1,365
GLIDE: Towards Photorealistic Image Generation and Editing with Text-Guided Diffusion Models
Nichol, A., Dhariwal, P., Ramesh, A., Shyam, P., Mishkin, P., McGrew, B., Sutskever, I., & Chen, M. (2021) · arXiv · doi:10.48550/arxiv.2112.10741 · Cited by 1,357
Self-citationGrounded Language-Image Pre-training
Li, L. H., Zhang, P., Zhang, H., Yang, J., Li, C., Zhong, Y., Wang, L., Yuan, L., Zhang, L., Hwang, J., et al. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.01069 · Cited by 1,336
Depth Anything: Unleashing the Power of Large-Scale Unlabeled Data
Yang, L., Kang, B., Huang, Z., Xu, X., Feng, J., & Zhao, H. (2024) · 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52733.2024.00987 · Cited by 1,296
CLIPScore: A Reference-free Evaluation Metric for Image Captioning
Hessel, J., Holtzman, A., Forbes, M., Le Bras, R., & Choi, Y. (2021) · Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/2021.emnlp-main.595 · Cited by 1,232
StyleCLIP: Text-Driven Manipulation of StyleGAN Imagery
Patashnik, O., Wu, Z., Shechtman, E., Cohen‐Or, D., & Lischinski, D. (2021) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00209 · Cited by 1,147
Sigmoid Loss for Language Image Pre-Training
Zhai, X., Mustafa, B., Kolesnikov, A., & Beyer, L. (2023) · 2023 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.01100 · Cited by 1,127
MaPLe: Multi-modal Prompt Learning
Khattak, M. U., Rasheed, H., Maaz, M., Khan, S., & Khan, F. S. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.01832 · Cited by 1,125
Zero-1-to-3: Zero-shot One Image to 3D Object
Liu, R., Wu, R., Hoorick, B. V., Tokmakov, P., Zakharov, S., & Vondrick, C. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.00853 · Cited by 1,130
A Comprehensive Survey of Continual Learning: Theory, Method and Application
Wang, L., Zhang, X., Su, H., & Zhu, J. (2024) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2024.3367329 · Cited by 1,113
“erally more efficient than images, IncCLIP [467] replays generated hard negative texts conditioned on images and performs multi-modal knowledge distillation for updating CLIP [347].”
BackgroundMethodInfluentialCLIP4Clip: An empirical study of CLIP for end to end video clip retrieval and captioning
Luo, H., Ji, L., Zhong, M., Chen, Y., Lei, W., Duan, N., & Li, T. (2022) · Neurocomputing · doi:10.1016/j.neucom.2022.07.028 · Cited by 1,105
Magic3D: High-Resolution Text-to-3D Content Creation
Lin, C.-H., Gao, J., Tang, L., Takikawa, T., Zeng, X., Huang, X., Kreis, K., Fidler, S., Liu, M.-Y., & Lin, T.-Y. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.00037 · Cited by 1,100
SpectralGPT: Spectral Remote Sensing Foundation Model
Hong, D., Zhang, B., Li, X., Li, Y., Li, C., Yao, J. T., Yokoya, N., Li, H., Ghamisi, P., Jia, X., et al. (2024) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2024.3362475 · Cited by 1,112
Align before Fuse: Vision and Language Representation Learning with Momentum Distillation
Li, J., Selvaraju, R. R., Gotmare, A. D., Joty, S., Xiong, C., & Hoi, S. (2021) · arXiv · doi:10.48550/arxiv.2107.07651 · Cited by 1,078
DINOv2: Learning Robust Visual Features without Supervision
Oquab, M., Darcet, T., Moutakanni, T., Vo, H., Szafraniec, M., Khalidov, V., Fernandez, P., Haziza, D., Massa, F., El-Nouby, A., et al. (2023) · arXiv · doi:10.48550/arxiv.2304.07193 · Cited by 1,078
Multimodal learning with transformers: A survey
Xu, P., Zhu, X., & Clifton, D. A. (2023) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2023.3275156 · Cited by 1,072
How far are we to GPT-4V? Closing the gap to commercial multimodal models with open-source suites
Chen, Z., Wang, W., Tian, H., Ye, S., Gao, Z., Cui, E., Tong, W., Hu, K., Luo, J., Ma, Z., et al. (2024) · Science China Information Sciences · doi:10.1007/s11432-024-4231-5 · Cited by 1,058
Vision-Language Models for Vision Tasks: A Survey
Zhang, J., Huang, J., Jin, S., & Lu, S. (2024) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2024.3369699 · Cited by 1,051
Unleashing the potential of prompt engineering for large language models
CHEN, B. ‐., Zhang, Z., Langrené, N., & Zhu, S. (2025) · Patterns · doi:10.1016/j.patter.2025.101260 · Cited by 1,097
Exploring Plain Vision Transformer Backbones for Object Detection
Li, Y., Mao, H., Girshick, R., & He, K. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-20077-9_17 · Cited by 1,027
LAION-5B: An open large-scale dataset for training next generation image-text models
Schuhmann, C., Beaumont, R., Vencu, R., Gordon, C., Wightman, R., Cherti, M., Coombes, T., Katta, A., Mullis, C., Wortsman, M., et al. (2022) · arXiv · doi:10.48550/arxiv.2210.08402 · Cited by 1,022
MedCLIP: Contrastive Learning from Unpaired Medical Images and Text
Wang, Z., Wu, Z., Agarwal, D., & Sun, J. (2022) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2022.emnlp-main.256 · Cited by 1,011
Pre-trained models: Past, present and future
Han, X., Zhang, Z., Ding, N., Gu, Y., Liu, X., Huo, Y., Qiu, J., Yao, Y., Zhang, A., Zhang, L., et al. (2021) · AI Open · doi:10.1016/j.aiopen.2021.08.002 · Cited by 994
LLaVA-Med: Training a Large Language-and-Vision Assistant for Biomedicine in One Day
Li, C., Wong, C., Zhang, S., Usuyama, N., Liu, H., Yang, J., Naumann, T., Poon, H., & Gao, J. (2023) · Advances in Neural Information Processing Systems 36 · doi:10.52202/075280-1240 · Cited by 986
MM-Vet: Evaluating Large Multimodal Models for Integrated CapabilitiesPreprint
Yu, W., Yang, Z., Li, L., Wang, J., Lin, K., Liu, Z., Wang, X., & Wang, L. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2308.02490 · Cited by 984
Imagic: Text-Based Real Image Editing with Diffusion Models
Kawar, B., Zada, S., Lang, O., Tov, O., Chang, H., Dekel, T., Mosseri, I., & Irani, M. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.00582 · Cited by 991
A visual-language foundation model for computational pathology
Lu, M. Y., Chen, B., Williamson, D. F. K., Chen, R. J., Liang, I., Ding, T., Jaume, G., Odintsov, I., Le, L. P., Gerber, G., et al. (2024) · Nature Medicine · doi:10.1038/s41591-024-02856-4 · Cited by 966
ViLT: Vision-and-Language Transformer Without Convolution or Region Supervision
Kim, W., Son, B., & Kim, I. (2021) · arXiv · doi:10.48550/arxiv.2102.03334 · Cited by 951
Exploring CLIP for Assessing the Look and Feel of Images
Wang, J., Chan, K. C. K., & Loy, C. C. (2023) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v37i2.25353 · Cited by 936
Point-BERT: Pre-training 3D Point Cloud Transformers with Masked Point Modeling
Yu, X., Tang, L., Rao, Y., Huang, T., Zhou, J., & Lu, J. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.01871 · Cited by 922
(2022) · doi:10.15607/rss.2023.xix.025 · Cited by 916
Li, J., Li, D., Savarese, S., & Hoi, S. (2023) · arXiv · doi:10.48550/arxiv.2301.12597 · Cited by 908
A survey on multimodal large language models
Yin, S., Fu, C., Zhao, S., Li, K., Sun, X., Xu, T., & Chen, E. (2024) · National Science Review · doi:10.1093/nsr/nwae403 · Cited by 893
Zhai, X., Kolesnikov, A., Houlsby, N., & Beyer, L. (2022) · 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52688.2022.01179 · Cited by 891
Generalized Out-of-Distribution Detection: A Survey
Yang, J., Zhou, K., Li, Y., & Liu, Z. (2024) · International Journal of Computer Vision · doi:10.1007/s11263-024-02117-4 · Cited by 884
Blended Diffusion for Text-driven Editing of Natural Images
Avrahami, O., Lischinski, D., & Fried, O. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.01767 · Cited by 889
ClipCap: CLIP Prefix for Image CaptioningPreprint
Mokady, R., Hertz, A., & Bermano, A. H. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2111.09734 · Cited by 929
ImageBind One Embedding Space to Bind Them All
Girdhar, R., El-Nouby, A., Liu, Z., Singh, M., Alwala, K. V., Joulin, A., & Misra, I. (2023) · 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52729.2023.01457 · Cited by 863
Open-vocabulary Object Detection via Vision and Language Knowledge DistillationPreprint
Gu, X.-Y., Lin, T.-Y., Kuo, W.-C., & Cui, Y. (2021) · International Conference on Learning Representations · doi:10.48550/arxiv.2104.13921 · Cited by 1,464
MotionDiffuse: Text-Driven Human Motion Generation With Diffusion Model
Zhang, M., Cai, Z., Pan, L., Hong, F., Guo, X., Yang, L., & Liu, Z. (2024) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2024.3355414 · Cited by 890
End-to-End Autonomous Driving: Challenges and Frontiers
Chen, L., Wu, P., Chitta, K., Jaeger, B., Geiger, A., & Li, H. (2024) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2024.3435937 · Cited by 843
YOLO-World: Real-Time Open-Vocabulary Object Detection
Cheng, T., Song, L., Ge, Y., Liu, W., Wang, X., & Shan, Y. (2024) · 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52733.2024.01599 · Cited by 815
Flamingo: a visual language model for few-shot learning
Alayrac, J.-B., Donahue, J., Luc, P., Miech, A., Barr, I., Hasson, Y., Lenc, K., Mensch, A., Millican, K., Reynolds, M., et al. (2022) · Advances in Neural Information Processing Systems 35 · doi:10.52202/068431-1723 · Cited by 966
“[85] we use “prompt-ensembling” in which we embed multiple texts using templates such as “A photo of a {class_name}” and average the resulting embedding.”
BackgroundMethodInfluentialMDETR - Modulated Detection for End-to-End Multi-Modal Understanding
Kamath, A., Singh, M., LeCun, Y., Synnaeve, G., Misra, I., & Carion, N. (2021) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00180 · Cited by 802
Tune-A-Video: One-Shot Tuning of Image Diffusion Models for Text-to-Video Generation
Wu, J. Z., Ge, Y., Wang, X., Lei, S. W., Gu, Y., Shi, Y., Hsu, W., Shan, Y., Qie, X., & Shou, M. Z. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.00701 · Cited by 799
ShareGPT4V: Improving Large Multi-modal Models with Better Captions
Chen, L., Li, J., Dong, X., Zhang, P., He, C., Wang, J., Zhao, F., & Lin, D. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-72643-9_22 · Cited by 816
Plug-and-Play Diffusion Features for Text-Driven Image-to-Image Translation
Tumanyan, N., Geyer, M., Bagon, S., & Dekel, T. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.00191 · Cited by 791
Point-E: A System for Generating 3D Point Clouds from Complex PromptsPreprint
Nichol, A., Jun, H., Dhariwal, P., Mishkin, P., & Chen, M. I.-C. (2022) · arXiv (Cornell University) · doi:10.48550/arxiv.2212.08751 · Cited by 853
Self-citationDINO: DETR with Improved DeNoising Anchor Boxes for End-to-End Object Detection
Zhang, H., Li, F., Liu, S., Zhang, L., Su, H., Zhu, J., Ni, L. M., & Shum, H.-Y. (2022) · arXiv · Cited by 1,154
“Large-scale pre-training have had a big impact on both natural language processing [10] and computer vision [27].”
BackgroundAlgorithmic fairness in artificial intelligence for medicine and healthcare
Chen, R. J., Wang, J. J., Williamson, D. F. K., Chen, T. Y., Lipkova, J., Lu, M. Y., Sahai, S., & Mahmood, F. (2023) · Nature Biomedical Engineering · doi:10.1038/s41551-023-01056-8 · Cited by 750
Null-text Inversion for Editing Real Images using Guided Diffusion Models
Mokady, R., Hertz, A., Aberman, K., Pritch, Y., & Cohen-Or, D. (2023) · 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52729.2023.00585 · Cited by 747
Intern VL: Scaling up Vision Foundation Models and Aligning for Generic Visual-Linguistic Tasks
Chen, Z., Wu, J., Wang, W., Su, W., Chen, G., Xing, S., Zhong, M., Zhang, Q., Zhu, X., Lu, L., et al. (2024) · 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52733.2024.02283 · Cited by 734
A Survey on Self-Supervised Learning: Algorithms, Applications, and Future Trends
Gui, J., Chen, T., Zhang, J., Cao, Q., Sun, Z., Luo, H., & Tao, D. (2024) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2024.3415112 · Cited by 730
Objaverse: A Universe of Annotated 3D Objects
Deitke, M., Schwenk, D., Salvador, J., Weihs, L., Michel, O., VanderBilt, E., Schmidt, L., Ehsanit, K., Kembhavi, A., & Farhadi, A. (2023) · 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52729.2023.01263 · Cited by 725
“Moreover, instances of objects often appear with many styles, which is critical for training and evaluating robust computer vision models [55].”
BackgroundInfluentialAlign Your Latents: High-Resolution Video Synthesis with Latent Diffusion Models
Blattmann, A., Rombach, R., Ling, H., Dockhorn, T., Kim, S. W., Fidler, S., & Kreis, K. (2023) · 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52729.2023.02161 · Cited by 724
Vector Quantized Diffusion Model for Text-to-Image Synthesis
Gu, S., Chen, D., Bao, J., Wen, F., Zhang, B., Chen, D., Yuan, L., & Guo, B. (2022) · 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52688.2022.01043 · Cited by 720
DenseCLIP: Language-Guided Dense Prediction with Context-Aware Prompting
Rao, Y., Zhao, W., Chen, G., Tang, Y., Zhu, Z., Huang, G., Zhou, J., & Lu, J. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.01755 · Cited by 718
GLIGEN: Open-Set Grounded Text-to-Image Generation
Li, Y., Liu, H., Wu, Q., Mu, F., Yang, J., Gao, J., Li, C., & Lee, Y. J. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.02156 · Cited by 715
RegNeRF: Regularizing Neural Radiance Fields for View Synthesis from Sparse Inputs
Niemeyer, M., Barron, J. T., Mildenhall, B., Sajjadi, M. S. M., Geiger, A., & Radwan, N. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.00540 · Cited by 705
Conceptual 12M: Pushing Web-Scale Image-Text Pre-Training To Recognize Long-Tail Visual Concepts
Changpinyo, S., Sharma, P. K., Ding, N., & Soricut, R. (2021) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr46437.2021.00356 · Cited by 701
SimVLM: Simple Visual Language Model Pretraining with Weak SupervisionPreprint
Wang, Z., Yu, J., Yu, A. W., Dai, Z., Tsvetkov, Y., & Cao, Y. J. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2108.10904 · Cited by 701
A visual–language foundation model for pathology image analysis using medical twitter
Huang, Z., Bianchi, F., Yuksekgonul, M., Montine, T. J., & Zou, J. (2023) · Nature Medicine · doi:10.1038/s41591-023-02504-3 · Cited by 693
Liu, H., Li, C., Wu, Q., & Lee, Y. J. (2023) · arXiv · doi:10.48550/arxiv.2304.08485 · Cited by 682
Bang, Y., Cahyawijaya, S., Lee, N., Dai, W., Su, D., Wilie, B., Lovenia, H., Ji, Z., Yu, T., Chung, W., et al. (2023) · Proceedings of the 13th International Joint Conference on Natural Language Processing and the 3rd Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/2023.ijcnlp-main.45 · Cited by 682
Gal, R., Patashnik, O., Maron, H., Bermano, A. H., Chechik, G., & Cohen‐Or, D. (2022) · ACM Transactions on Graphics · doi:10.1145/3528223.3530164 · Cited by 682
Metadata
- Authors
- Radford, A., Kim, J. W., Hallacy, C., Ramesh, A., Goh, G., Agarwal, S., Sastry, G., Askell, A., Mishkin, P., Clark, J., Krueger, G., & Sutskever, I.
- Affiliations
- OpenAI (United States) (US)
- ORCID
- Kim, Jong Wook
- Venue
- arXiv
- Publisher
- arXiv
- Published
- 2021
- ISSN
- 2331-8422 (ISSN-L)
- arXiv
- 2103.00020
- OpenAlex
- W3135367836
- Cited by
- 15,293
- References
- 193
- Citations by year
- 2026: 214 · 2025: 3988 · 2024: 6413 · 2023: 3389 · 2022: 932 · 2021: 349
- Type
- posted content
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Multimodal Machine Learning Applications
- Physical Sciences › Computer Science › Artificial Intelligence › Domain Adaptation and Few-Shot Learning
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
- Keywords
- natural language supervision, zero-shot transfer, image representation learning, image captioning, visual concept learning, cross-modal learning, human action recognition, geolocation, optical character recognition, CLIP
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
