The DOI matches its DataCite record; 185 of the 193 references it cites were found in a registry or an index, and 1 are not checked yet.
Learning Transferable Visual Models From Natural Language Supervision
Open accessAbstract
State-of-the-art computer vision systems are trained to predict a fixed set of predetermined object categories. This restricted form of supervision limits their generality and usability since additional labeled data is needed to specify any other visual concept. Learning directly from raw text about images is a promising alternative which leverages a much broader source of supervision. We demonstrate that the simple pre-training task of predicting which caption goes with which image is an efficient and scalable way to learn SOTA image representations from scratch on a dataset of 400 million (image, text) pairs collected from the internet. After pre-training, natural language is used to reference learned visual concepts (or describe new ones) enabling zero-shot transfer of the model to downstream tasks. We study the performance of this approach by benchmarking on over 30 different existing computer vision datasets, spanning tasks such as OCR, action recognition in videos, geo-localization, and many types of fine-grained object classification. The model transfers non-trivially to most tasks and is often competitive with a fully supervised baseline without the need for any dataset specific training. For instance, we match the accuracy of the original ResNet-50 on ImageNet zero-shot without needing to use any of the 1.28 million training examples it was trained on. We release our code and pre-trained model weights at https://github.com/OpenAI/CLIP.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source doesn’t support the claim as written.
In the citing paper
“During training,the model learns the joint distribution between the two modalities and optimizes the model through methods such as adversarial training or maximizing likelihood, enabling it to generate the target modality”
From Multimodal fusion and vision–language models: A survey for robot vision (2026) · cites this paper as [188]
Checked by a reasoning model — may be wrong.
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Large Image Models (LIMs) such as CLIP [86] and BLIP-2 [87] extend the agent's capabilities into the visual domain. Trained on image-text pairs, LIMs enable perception-based tasks including image classification, object detection, and vision-language grounding.”
From AI Agents vs. Agentic AI: A Conceptual taxonomy, applications and challenges (2026) · cites this paper as [86]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Foundation Models (FMs) [23, 35, 37] - generic models trained on web-scale datasets - have demonstrated remarkable zero-shot performance and can be quickly adapted to various downstream tasks with few additional tuning cost.”
From Towards More General Video-based Deepfake Detection through Facial Component Guided Adaptation for Foundation Model (2024) · cites this paper as [37]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“By encoding text descriptions of variations of the dataset bias (e.g.'a blue wall'), using CLIP's text encoder (CLIP T ), we vary visual representations without actually having seen images containing these variations (e.g. an image of a blue wall).”
From Language-Based Augmentation to Address Shortcut Learning in Object-Goal Navigation (2023) · cites this paper as [15]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“However, their capacity for semantic comprehension is comparatively limited when contrasted with vision-based perceptual models like CLIP (Radford et al., 2021).”
From WorldVLA: Towards Autoregressive Action World Model (2025) · cites this paper as Radford et al., 2021
Experimental — accuracy not yet measured
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“In natural language [1] and computer vision [39], general-purpose foundation models that are pre-trained on diverse multi-task data tend to outperform narrowly tailored and specialized solutions.”
From $π_0$: A Vision-Language-Action Flow Model for General Robot Control (2024) · cites this paper as [39]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Existing evaluation metrics such as CLIP-Image score [37] and DINO score [6], while commonly used, often fail to reflect subtle differences due to their reliance on high-dimensional semantic features for cosine similarity computation.”
From DiffSim: Taming Diffusion Models for Evaluating Visual Similarity (2024) · cites this paper as [37]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“CLIP cosine similarity is often used to measure text prompt and frame consistency, where a reference video dataset is not needed.”
From Video Diffusion Models: A Survey (2024) · cites this paper as Radford et al., 2021
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Trained on image-text pairs, LIMs enable perception-based tasks including image classification, object detection, and vision-language grounding.”
From AI Agents vs. Agentic AI: A Conceptual Taxonomy, Applications and Challenges (2025) · cites this paper as [88]
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“Vision-Language Models (VLMs) e.g., CLIP [15], allows us to augment at feature-level based on prior knowledge of the environment without any changes to training data.”
From Language-Based Augmentation to Address Shortcut Learning in Object-Goal Navigation (2023) · cites this paper as [15]
The cited source says what the claim attributes to it.
In the citing paper
“We find these methods perform poorly on language and image goals, particularly when relying on CLIP [25] features.”
From GOAT-Bench: A Benchmark for Multi-Modal Lifelong Navigation (2024) · cites this paper as [25]
The cited source says what the claim attributes to it.
In the citing paper
“A notable advancement in this domain is the recent work on CLIP [9, 32, 35, 43], which elevates open-vocabulary recognition performance by collecting millions of image-text pairs and employing contrastive learning techniques.”
From Active Open-Vocabulary Recognition: Let Intelligent Moving Mitigate CLIP Limitations (2024) · cites this paper as [32]
The cited source says what the claim attributes to it.
In the citing paper
“The recently proposed Contrastive Language-Image Pretraining (CLIP) [1] learns the correspondence between text and image by projecting them into a shared latent space.”
From Large-Scale Contrastive Language-Audio Pretraining with Feature Fusion and Keyword-to-Caption Augmentation (2023) · cites this paper as [1]
The cited source says what the claim attributes to it.
In the citing paper
“To evaluate the visual similarity of I R and I G , we use the similarity of their CLIP (Radford et al., 2021) embedding, denoted as CLIP ( I R , I G ) .”
From Design2Code: Benchmarking Multimodal Code Generation for Automated Front-End Engineering (2025) · cites this paper as Radford et al., 2021
The cited source says what the claim attributes to it.
In the citing paper
“This suggests the inability of CLIP [25] features in capturing crucial instance-specific and spatial features in language and image goals.”
From GOAT-Bench: A Benchmark for Multi-Modal Lifelong Navigation (2024) · cites this paper as [25]
The cited source says what the claim attributes to it.
In the citing paper
“Existing solutions in multimodal Transformer adopt a fixed fusion strategy [18,21,24,26,33].”
From Are Multimodal Transformers Robust to Missing Modality? (2022) · cites this paper as [33]
The cited source says what the claim attributes to it.
In the citing paper
“existing foundation models for vision and language such as CLIP [8], SigLIP [9], and Llama 2 [10] are capable of these types of generalization and more, stemming from the priors captured by their Internet-scale pretraining datasets.”
From OpenVLA: An Open-Source Vision-Language-Action Model (2024) · cites this paper as [8]
The cited source says what the claim attributes to it.
In the citing paper
“which shows that our model achieves the highest DINO [5] and CLIP-I [42] scores, with CLIP-T [42] scores that are comparable to other methods.”
From DreamingComics: A Story Visualization Pipeline via Subject and Layout Customized Generation using Video Models (2025) · cites this paper as [42]
The cited source says what the claim attributes to it.
In the citing paper
“In early 2021, OpenAI released an architecture termed Contrastive Language Image Pretraining (CLIP), which, when trained on millions of image-text pairs, matched the performance of competitive, fully supervised models without fine-tuning 106”
From Multimodal biomedical AI (2022) · cites this paper as [106]
The cited source says what the claim attributes to it.
In the citing paper
“Even when PwD data is available, data from marginalized groups is often automatically filtered out by techniques for removing 'low' quality data [57, 104], which rely on biased models such as CLIP [92].”
From Engaging Communities Meaningfully in Defining Disability Representation for AI Image Generation (2026) · cites this paper as [92]
The check wasn’t confident either way, so it gives no verdict — never disputed.
In the citing paper
“HOLODECK uses a Large Language Model (GPT-4 [37]) to design the floor plan, assign suitable materials, install the doorways and windows and arrange 3D assets coherently in the scene using constraintbased optimization.”
From Holodeck: Language Guided Generation of 3D Embodied AI Environments (2024) · cites this paper as [37]
Experimental — accuracy not yet measuredThe check wasn’t confident either way, so it gives no verdict — never disputed.
In the citing paper
“[41] famously found evidence of a ‘cat neuron’, a phenomenon that has been remarkably replicable across newer models and analyses [14,42,43]”
From Symbols and grounding in large language models (2023) · cites this paper as [43]
Experimental — accuracy not yet measuredThe check wasn’t confident either way, so it gives no verdict — never disputed.
In the citing paper
“Text-to-image models like Stable Diffusion, Midjourney, and DALL-E [26,27] enable educators to create visual learning materials from textual content.”
From Promises and challenges of generative artificial intelligence for human learning (2024) · cites this paper as [26]
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., & Polosukhin, I. (2017) · arXiv · doi:10.48550/arxiv.1706.03762 · Cited by 120,404
Adam: A Method for Stochastic Optimization
Kingma, D. P., & Ba, J. (2014) · arXiv · doi:10.48550/arxiv.1412.6980 · Cited by 98,946
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2018) · arXiv · doi:10.48550/arxiv.1810.04805 · Cited by 73,173
Scikit-learn: Machine Learning in Python
Pedregosa, F., Varoquaux, G., Gramfort, A., Michel, V., Thirion, B., Grisel, O., Blondel, M., Müller, A., Nothman, J., Louppe, G., et al. (2012) · arXiv · doi:10.48550/arxiv.1201.0490 · Cited by 71,016
ImageNet: A large-scale hierarchical image database
Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., & Fei-Fei, L. (2009) · 2009 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2009.5206848 · Cited by 65,333
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2019) · Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) · doi:10.18653/v1/n19-1423 · Cited by 54,325
An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale
Dosovitskiy, A., Beyer, L., Kolesnikov, A., Weissenborn, D., Zhai, X., Unterthiner, T., Dehghani, M., Minderer, M., Heigold, G., Gelly, S., et al. (2020) · arXiv · doi:10.48550/arxiv.2010.11929 · Cited by 47,472
SciPy 1.0: fundamental algorithms for scientific computing in Python
Virtanen, P., Gommers, R., Oliphant, T. E., Haberland, M., Reddy, T., Cournapeau, D., Burovski, E., Peterson, P., Weckesser, W., Bright, J., et al. (2020) · Nature Methods · doi:10.1038/s41592-019-0686-2 · Cited by 38,430
Glove: Global Vectors for Word Representation
Pennington, J., Socher, R., & Manning, C. (2014) · Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.3115/v1/d14-1162 · Cited by 34,071
Research works printed but not linked: 24
- Bengio, Y., Ducharme, R., Vincent, P., and Jauvin, C. A neural probabilistic language model. Journal of machine learning research , 3(Feb):1137-1155, 2003.
- Browne, S. Dark Matters: Surveillance of Blackness . Duke University Press, 2015.
- Chen, Y.-C., Li, L., Yu, L., Kholy, A. E., Ahmed, F., Gan, Z., Cheng, Y., and Liu, J. Uniter: Learning universal imagetext representations. arXiv preprint arXiv:1909.11740 , 2019. · arXiv
- Coates, A., Ng, A., and Lee, H. An analysis of singlelayer networks in unsupervised feature learning. In Proceedings of the fourteenth international conference on artificial intelligence and statistics , pp. 215-223, 2011.
- Crawford, K. The trouble with bias. NIPS 2017 Keynote , 2017. URL https://www.youtube.com/ watch?v=fMym\_BKWQzk .
- Frome, A., Corrado, G. S., Shlens, J., Bengio, S., Dean, J., Ranzato, M., and Mikolov, T. Devise: A deep visualsemantic embedding model. In Advances in neural information processing systems , pp. 2121-2129, 2013.
- Geirhos, R., Rubisch, P., Michaelis, C., Bethge, M., Wichmann, F. A., and Brendel, W. Imagenet-trained cnns are biased towards texture; increasing shape bias improves accuracy and robustness. arXiv preprint arXiv:1811.12231 , 2018. · arXiv
- Goodfellow, I. J., Shlens, J., and Szegedy, C. Explaining and harnessing adversarial examples. arXiv preprint arXiv:1412.6572 , 2014. · arXiv
- Ha, D., Dai, A., and Le, Q. V. Hypernetworks. arXiv preprint arXiv:1609.09106 , 2016. · arXiv
- He, K., Zhang, X., Ren, S., and Sun, J. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition , pp. 770-778, 2016b.
- Hendrycks, D. and Dietterich, T. Benchmarking neural network robustness to common corruptions and perturbations. arXiv preprint arXiv:1903.12261 , 2019. · arXiv
- Hendrycks, D., Liu, X., Wallace, E., Dziedzic, A., Krishnan, R., and Song, D. Pretrained transformers improve out-ofdistribution robustness. arXiv preprint arXiv:2004.06100 , 2020b. · arXiv
- Ilyas, A., Santurkar, S., Tsipras, D., Engstrom, L., Tran, B., and Madry, A. Adversarial examples are not bugs, they are features. In Advances in Neural Information Processing Systems , pp. 125-136, 2019.
- Jaderberg, M., Simonyan, K., Zisserman, A., et al. Spatial transformer networks. Advances in neural information processing systems , 28:2017-2025, 2015.
- Johnson, J., Hariharan, B., van der Maaten, L., Fei-Fei, L., Lawrence Zitnick, C., and Girshick, R. Clevr: A diagnostic dataset for compositional language and elementary visual reasoning. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition , pp. 2901-2910, 2017.
- Kiros, R., Zhu, Y., Salakhutdinov, R. R., Zemel, R., Urtasun, R., Torralba, A., and Fidler, S. Skip-thought vectors. Advances in neural information processing systems , 28: 3294-3302, 2015.
- Krizhevsky, A., Sutskever, I., and Hinton, G. E. Imagenet classification with deep convolutional neural networks. In Advances in neural information processing systems , pp. 1097-1105, 2012.
- Lin, T.-Y., Maire, M., Belongie, S., Hays, J., Perona, P., Ramanan, D., Doll´ ar, P., and Zitnick, C. L. Microsoft coco: Common objects in context. In European conference on computer vision , pp. 740-755. Springer, 2014.
- Proceedings of the European Conference on Computer Vision (ECCV) , pp. 181-196, 2018.
- McCann, B., Bradbury, J., Xiong, C., and Socher, R. Learned in translation: Contextualized word vectors. In Advances in neural information processing systems , pp. 6294-6305, 2017.
- Socher, R. and Fei-Fei, L. Connecting modalities: Semisupervised segmentation and annotation of images using unaligned text corpora. In 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition , pp. 966-973. IEEE, 2010.
- Srivastava, N. and Salakhutdinov, R. Multimodal learning with deep boltzmann machines. In NIPS , 2012.
- Thomee, B., Shamma, D. A., Friedland, G., Elizalde, B., Ni, K., Poland, D., Borth, D., and Li, L.-J. Yfcc100m: The new data in multimedia research. Communications of the ACM , 59(2):64-73, 2016.
- Touvron, H., Vedaldi, A., Douze, M., and J´ egou, H. Fixing the train-test resolution discrepancy. In Advances in neural information processing systems , pp. 8252-8262, 2019.
Cited by
Sorted by citations
Items 901–1,000 of 1,203
HairCLIP: Design Your Hair by Text and Reference Image
Wei, T., Chen, D., Zhou, W., Liao, J., Tan, Z., Yuan, L., Zhang, W., & Yu, N. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.01754 · Cited by 123
Feng, Z., Zhang, Z., Yu, X., Fang, Y., Li, L., Chen, X., Lu, Y., Liu, J., Yin, W., Feng, S., et al. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.00977 · Cited by 124
Forecast-MAE: Self-supervised Pre-training for Motion Forecasting with Masked Autoencoders
Cheng, J., Mei, X., & Liu, M. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.00797 · Cited by 123
GSVA: Generalized Segmentation via Multimodal Large Language Models
Xia, Z., Han, D., Han, Y., Pan, X., Song, S., & Huang, G. M. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.00370 · Cited by 127
Segment Anything Meets Point Tracking
Rajič, F., Ke, L., Tai, Y., Tang, C., Danelljan, M., & Yu, F. (2025) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv61041.2025.00901 · Cited by 123
Visual Language Pretrained Multiple Instance Zero-Shot Transfer for Histopathology Images
Lu, M. Y., Chen, B., Zhang, A., Williamson, D. F. K., Chen, R. J., Ding, T., Le, L. P., Chuang, Y.-S., & Mahmood, F. (2023) · 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52729.2023.01893 · Cited by 122
Creating, Using, Misusing, and Detecting Deep Fakes
Farid, H. (2022) · Journal of Online Trust and Safety · doi:10.54501/jots.v1i4.56 · Cited by 122
Krenn, M., Buffoni, L., Coutinho, B. C., Eppel, S., Foster, J. G., Gritsevskiy, A., Lee, H., Lu, Y., Moutinho, J. P., Sanjabi, N., et al. (2023) · Nature Machine Intelligence · doi:10.1038/s42256-023-00735-0 · Cited by 123
Synthbuster: Towards Detection of Diffusion Model Generated Images
Bammey, Q. (2023) · IEEE Open Journal of Signal Processing · doi:10.1109/ojsp.2023.3337714 · Cited by 125
Preventing Zero-Shot Transfer Degradation in Continual Learning of Vision-Language Models
Zheng, Z., Ma, M., Wang, K., Qin, Z., Yue, X., & You, Y. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.01752 · Cited by 122
LLM-Grounder: Open-Vocabulary 3D Visual Grounding with Large Language Model as an Agent
Yang, J., Chen, X., Qian, S., Madaan, N., Iyengar, M., Fouhey, D. F., & Chai, J. Y. (2024) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra57147.2024.10610443 · Cited by 128
Chen, Z., Chen, Y., Sun, Y., Tang, L., Zhang, L., Hu, Y., He, M., Li, Z., Cheng, S., Yuan, J., et al. (2024) · Signal Transduction and Targeted Therapy · doi:10.1038/s41392-024-01932-y · Cited by 122
KNN-Diffusion: Image Generation via Large-Scale RetrievalPreprint
Sheynin, S., Ashual, O., Polyak, A., Singer, U., Gafni, O., Nachmani, E., & Taigman, Y. (2022) · arXiv (Cornell University) · doi:10.48550/arxiv.2204.02849 · Cited by 126
Typology of Risks of Generative Text-to-Image Models
Bird, C., Ungless, E., & Kasirzadeh, A. (2023) · Proceedings of the 2023 AAAI/ACM Conference on AI, Ethics, and Society · doi:10.1145/3600211.3604722 · Cited by 121
Exploring fusion strategies for accurate RGBT visual object tracking
Tang, Z., Xu, T., Li, H., Wu, X., Zhu, X., & Kittler, J. (2023) · Information Fusion · doi:10.1016/j.inffus.2023.101881 · Cited by 121
UniTune: Text-Driven Image Editing by Fine Tuning a Diffusion Model on a Single Image
Valevski, D., Kalman, M., Molad, E., Segalis, E., Matias, Y., & Leviathan, Y. (2023) · ACM Transactions on Graphics · doi:10.1145/3592451 · Cited by 135
Finetune like you pretrain: Improved finetuning of zero-shot vision models
Goyal, S., Kumar, A., Garg, S., Kolter, J. Z., & Raghunathan, A. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.01853 · Cited by 121
Localizing Object-level Shape Variations with Text-to-Image Diffusion Models
Patashnik, O., Garibi, D., Azuri, I., Averbuch‐Elor, H., & Cohen‐Or, D. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.02107 · Cited by 122
Align Your Gaussians: Text-to-4D with Dynamic 3D Gaussians and Composed Diffusion Models
Ling, H., Kim, S. W., Torralba, A. B., Fidler, S., & Kreis, K. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.00819 · Cited by 127
PLOT: Prompt Learning with Optimal Transport for Vision-Language ModelsPreprint
Chen, G., Yao, W., Song, X., Li, X., Rao, Y., & Zhang, K. (2022) · arXiv (Cornell University) · doi:10.48550/arxiv.2210.01253 · Cited by 121
RePrompt: Automatic Prompt Editing to Refine AI-Generative Art Towards Precise Expressions
Wang, Y., Shen, S., & Lim, B. Y. (2023) · Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3544548.3581402 · Cited by 120
Thinking in Space: How Multimodal Large Language Models See, Remember, and Recall Spaces
Yang, J., Yang, S., Gupta, A. W., Han, R., Fei-Fei, L., & Xie, S. (2025) · 2025 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52734.2025.00994 · Cited by 120
Denoising Pretraining for Semantic Segmentation
Brempong, E. A., Kornblith, S., Chen, T., Parmar, N., Minderer, M., & Norouzi, M. (2022) · 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw56347.2022.00462 · Cited by 120
Debiased Learning from Naturally Imbalanced Pseudo-Labels
Wang, X., Wu, Z., Lian, L., & Yu, S. X. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.01424 · Cited by 120
Fake it Till You Make it: Learning Transferable Representations from Synthetic ImageNet Clones
Sarıyıldız, M. B., Alahari, K., Larlus, D., & Kalantidis, Y. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.00774 · Cited by 120
Revisiting Classifier: Transferring Vision-Language Models for Video Recognition
Wu, W., Sun, Z., & Ouyang, W. (2023) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v37i3.25386 · Cited by 120
Taming the Power of Diffusion Models for High-Quality Virtual Try-On with Appearance Flow
Gou, J., Sun, S., Zhang, J., Si, J., Qian, C., & Zhang, L. (2023) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3581783.3612255 · Cited by 122
Vision Transformers are Parameter-Efficient Audio-Visual Learners
Lin, Y.-B., Sung, Y.-L., Lei, J., Bansal, M., & Bertasius, G. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.00228 · Cited by 120
Unsafe Diffusion: On the Generation of Unsafe Images and Hateful Memes From Text-To-Image Models
Qu, Y., Shen, X., He, X., Michael Backes, A., Zannettou, S., & Zhang, Y. (2023) · ACM SIGSAC Conference on Computer and Communications Security (CCS) · doi:10.1145/3576915.3616679 · Cited by 120
SuS-X: Training-Free Name-Only Transfer of Vision-Language Models
Udandarao, V., Gupta, A., & Albanie, S. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.00257 · Cited by 120
Akrout, M., Gyepesi, B., Holló, P., Poór, A., Blága, K., Solis, S., Cirone, K. D., Kawahara, J., Slade, D., Abid, L., et al. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-53767-7_10 · Cited by 120
MotionGPT: Finetuned LLMs Are General-Purpose Motion Generators
Zhang, Y., Huang, D., Liu, B., Tang, S., Lu, Y., Chen, L., Bai, L., Chu, Q., Yu, N., & Ouyang, W. (2024) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v38i7.28567 · Cited by 128
AutoRepo: A general framework for multimodal LLM-based automated construction reporting
Pu, H., Yang, X., Li, J., & Guo, R. (2024) · Expert Systems with Applications · doi:10.1016/j.eswa.2024.124601 · Cited by 120
XCube: Large-Scale 3D Generative Modeling using Sparse Voxel Hierarchies
Ren, X., Huang, J., Zeng, X., Museth, K., Fidler, S., & Williams, F. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.00403 · Cited by 122
Sniffer: Multimodal Large Language Model for Explainable Out-of-Context Misinformation Detection
Qi, P., Yan, Z., Hsu, W., & Lee, M. L. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.01240 · Cited by 120
Slide-Transformer: Hierarchical Vision Transformer with Local Self-Attention
Pan, X., Ye, T., Xia, Z., Song, S., & Huang, G. M. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.00207 · Cited by 119
Paiss, R., Ephrat, A., Tov, O., Zada, S., Mosseri, I., Irani, M., & Dekel, T. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.00294 · Cited by 121
Efficient Diffusion Model for Image Restoration by Residual Shifting
Yue, Z., Wang, J., & Loy, C. C. (2024) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2024.3461721 · Cited by 126
Investigating Style Similarity in Diffusion Models
Somepalli, G., Gupta, A., Gupta, K., Palta, S., Goldblum, M., Geiping, J., Shrivastava, A., & Goldstein, T. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-72848-8_9 · Cited by 119
Foundation model of neural activity predicts response to new stimulus types
Wang, E. Y., Fahey, P. G., Ding, Z., Papadopoulos, S., Ponder, K., Weis, M. A., Chang, A., Muhammad, T., Patel, S. S., Ding, Z., et al. (2025) · Nature · doi:10.1038/s41586-025-08829-y · Cited by 119
The generative era of medical AI
Fahrner, J., Chen, E., Topol, E. J., & Rajpurkar, P. (2025) · Cell · doi:10.1016/j.cell.2025.05.018 · Cited by 120
RedCaps: web-curated image-text data created by the people, for the peoplePreprint
Desai, K., Kaul, G., Aysola, Z., & Johnson, J. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2111.11431 · Cited by 119
Kraljevic, Z., Bean, D., Shek, A., Bendayan, R., Hemingway, H., Yeung, J. A., Deng, A., Balston, A., Ross, J., Idowu, E., et al. (2024) · The Lancet Digital Health · doi:10.1016/s2589-7500(24)00025-6 · Cited by 169
“…on the generated timelines with, for example, a learningtorank model similar to how the contrastive language–image pretraining (CLIP) model 21 works with DALLE. 22 Additionally, an intriguing future use case would be whether the knowledge representation of a historical EHR can be used as…”
A visual–omics foundation model to bridge histopathology with spatial transcriptomics
Chen, W., Zhang, P., Tran, T. N., Xiao, Y., Li, S., Shah, V. V., Cheng, H., Brannan, K. W., Youker, K., Lai, L., et al. (2025) · Nature Methods · doi:10.1038/s41592-025-02707-1 · Cited by 118
Nath, S., Marie, A., Ellershaw, S., Korot, E., & Keane, P. A. (2022) · British Journal of Ophthalmology · doi:10.1136/bjophthalmol-2022-321141 · Cited by 118
Cross Modal Retrieval with Querybank Normalisation
Bogolin, S.-V., Croitoru, I., Jin, H., Liu, Y., & Albanie, S. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.00513 · Cited by 118
MolCA: Molecular Graph-Language Modeling with Cross-Modal Projector and Uni-Modal Adapter
Liu, Z., Li, S., Luo, Y., Fei, H., Cao, Y., Kawaguchi, K., Wang, X., & Chua, T. (2023) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2023.emnlp-main.966 · Cited by 122
VideoLLM-online: Online Video Large Language Model for Streaming Video
Chen, J., Lv, Z., Wu, S., Lin, K. Q., Song, C., Gao, D., Liu, J.-W., Gao, Z., Mao, D., & Shou, M. Z. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.01742 · Cited by 124
A Multimodal Vision Foundation Model for Clinical Dermatology
Yan, S., Yu, Z., Primiero, C., Vico-Alonso, C., Wang, Z., Yang, L., Tschandl, P., Hu, M., Ju, L., Tan, G., et al. (2025) · Nature Medicine · doi:10.1038/s41591-025-03747-y · Cited by 117
“Furthermore, PanDerm also surpassed vision-language models such as CLIP 36 , MONET 39 and biomedical-specific CLIP (BiomedCLIP) 40 in benchmark evaluations (Supplementary Table 1), while showing emergent capabilities in dermatology similar to those of DINOv2 in natural images, with linear probing performance comparabl…”
InfluentialMuse: Text-To-Image Generation via Masked Generative Transformers
Chang, H., Zhang, H., Barber, J., Maschinot, A., Lezama, J., Jiang, L., Yang, M.-H., Murphy, K., Freeman, W. T., Rubinstein, M., et al. (2023) · arXiv · doi:10.48550/arxiv.2301.00704 · Cited by 157
“Our 3B parameter model achieves a CLIP score of 0.32 and an FID score of 7.88 on the COCO (Lin et al., 2014) zero-shot validation benchmark, which compare favorably with that of other large-scale text-to-image models (see Table 2).”
BackgroundMethodInfluentialCLIP-Event: Connecting Text and Images with Event Structures
Li, M., Xu, R., Wang, S., Zhou, L., Lin, X., Zhu, C., Zeng, M., Ji, H., & Chang, S.-F. (2022) · 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52688.2022.01593 · Cited by 117
Diffusion Models, Image Super-Resolution, and Everything: A Survey
Moser, B. B., Shanbhag, A. S., Raue, F., Frolov, S., Palacio, S., & Dengel, A. (2025) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2024.3476671 · Cited by 117
TF-ICON: Diffusion-Based Training-Free Cross-Domain Image Composition
Lu, S., Liu, Y., & Kong, A. W. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.00218 · Cited by 119
Machine learning for the advancement of membrane science and technology: A critical review
Ignácz, G., Bader, L., Beke, Á. K., Ghunaim, Y., Shastry, T., Vovusha, H., Carbone, M. R., Ghanem, B., & Székely, G. (2024) · Journal of Membrane Science · doi:10.1016/j.memsci.2024.123256 · Cited by 117
Rich Human Feedback for Text-to-Image Generation
Liang, Y., He, J., Li, G., Li, P., Klimovskiy, A., Carolan, N., Sun, J., Pont-Tuset, J., Young, S., Yang, F., et al. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.01835 · Cited by 122
Self-Supervised Remote Sensing Feature Learning: Learning Paradigms, Challenges, and Future Works
Tao, C., Qi, J., Guo, M., Zhu, Q., & Li, H. (2023) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2023.3276853 · Cited by 116
Prompting for Multimodal Hateful Meme Classification
Cao, R., Lee, R. K.-W., Chong, W.-H., & Jiang, J. (2022) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2022.emnlp-main.22 · Cited by 116
X2-VLM: All-in-One Pre-Trained Model for Vision-Language Tasks
Zeng, Y., Zhang, X., Li, H., Wang, J., Zhang, J., & Zhou, W. (2023) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2023.3339661 · Cited by 116
Improving Sample Quality of Diffusion Models Using Self-Attention Guidance
Hong, S., Lee, G., Jang, W., & Kim, S. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.00686 · Cited by 120
Transformer in reinforcement learning for decision-making: a survey
Yuan, W., Chen, J., Chen, S., Lu, L., Hu, Z., Li, P., Feng, D., Liu, F., & Chen, J. (2024) · Frontiers of Information Technology & Electronic Engineering · doi:10.1631/fitee.2300548 · Cited by 118
MedLSAM: Localize and segment anything model for 3D CT images
Lei, W., Xu, W., Li, K., Zhang, X., & Zhang, S. (2024) · Medical Image Analysis · doi:10.1016/j.media.2024.103370 · Cited by 135
CLIP-AD: A Language-Guided Staged Dual-Path Model for Zero-Shot Anomaly Detection
Chen, X., Zhang, J., Tian, G., He, H., Zhang, W., Wang, Y., Wang, C., & Liu, Y. (2024) · Communications in computer and information science · doi:10.1007/978-981-97-9003-6_2 · Cited by 121
Large language models for building energy applications: Opportunities and challenges
Liu, M., Zhang, L., Chen, J., Chen, W.-A., Yang, Z., Lo, L. J., Wen, J., & O’Neill, Z. D. (2025) · Building Simulation · doi:10.1007/s12273-025-1235-9 · Cited by 116
PromptCharm: Text-to-Image Generation through Multi-modal Prompting and Refinement
Wang, Z., Huang, Y., Song, D., Ma, L., & Zhang, T. (2024) · Proceedings of the CHI Conference on Human Factors in Computing Systems · doi:10.1145/3613904.3642803 · Cited by 115
SAM-Med3D: Towards General-purpose Segmentation Models for Volumetric Medical Images
Wang, H., Guo, S., Ye, J., Deng, Z., Cheng, J., Li, T., Chen, J., Su, Y., Huang, Z., Shen, Y., et al. (2023) · arXiv · doi:10.48550/arxiv.2310.15161 · Cited by 115
Exploring scalable medical image encoders beyond text supervision
Pérez-García, F., Sharma, H., Bond-Taylor, S., Bouzid, K., Salvatelli, V., Ilse, M., Bannur, S., Castro, D. C., Schwaighofer, A., Lungren, M. P., et al. (2025) · Nature Machine Intelligence · doi:10.1038/s42256-024-00965-w · Cited by 115
Generating Representative Samples for Few-Shot Classification
Xu, J., & Le, H. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.00880 · Cited by 115
Optimization of Image Transmission in Cooperative Semantic Communication Networks
Zhang, W., Wang, Y., Chen, M., Luo, T., & Niyato, D. T. (2023) · IEEE Transactions on Wireless Communications · doi:10.1109/twc.2023.3282906 · Cited by 115
When brain-inspired AI meets AGI
Zhao, L., Zhang, L., Wu, Z., Chen, Y., Dai, H., Yu, X., Liu, Z., Zhang, T., Hu, X., Jiang, X., et al. (2023) · Meta-Radiology · doi:10.1016/j.metrad.2023.100005 · Cited by 116
Diffusion-SDF: Text-to-Shape via Voxelized Diffusion
Li, M., Duan, Y., Zhou, J., & Lu, J. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.01216 · Cited by 114
VectorFusion: Text-to-SVG by Abstracting Pixel-Based Diffusion Models
Jain, A., Xie, A., & Abbeel, P. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.00190 · Cited by 121
VGSG: Vision-Guided Semantic-Group Network for Text-Based Person Search
He, S., Luo, H., Jiang, W., Jiang, X., & Ding, H. (2023) · IEEE Transactions on Image Processing · doi:10.1109/tip.2023.3337653 · Cited by 115
Latency-Aware Generative Semantic Communications With Pre-Trained Diffusion Models
Qiao, L., Mashhadi, M. B., Gao, Z., Foh, C. H., Xiao, P., & Bennis, M. (2024) · IEEE Wireless Communications Letters · doi:10.1109/lwc.2024.3429295 · Cited by 115
A Survey of Multimodel Large Language Models
Liang, Z., Xu, Y., Hong, Y., Shang, P., Wang, Q., Fu, Q., & Liu, K. (2024) · International Conference on Computer, Artificial Intelligence and Control Engineering (CAICE) · doi:10.1145/3672758.3672824 · Cited by 115
Multimodal Web Navigation with Instruction-Finetuned Foundation Models
Furuta, H., Lee, K.-H., Nachum, O., Matsuo, Y., Faust, A., Gu, S. S., & Gur, I. (2023) · arXiv · doi:10.48550/arxiv.2305.11854 · Cited by 115
Hierarchical Open-Vocabulary 3D Scene Graphs for Language-Grounded Robot Navigation
Werby, A., Huang, C., Büchner, M., Valada, A., & Burgard, W. (2024) · Robotics: Science and Systems XX · doi:10.15607/rss.2024.xx.077 · Cited by 124
“On account of recent advancements in large vision-language models such as CLIP [21] and their fine-tuned counterparts, a number of works proposed map representations that integrate visual-language features into geometric maps, enabling open-vocabulary indexing of objects [11, 16, 10, 13, 35, 17,…”
BackgroundMethodTubeDETR: Spatio-Temporal Video Grounding with Transformers
Yang, A., Miech, A., Šivic, J., Laptev, I., & Schmid, C. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.01595 · Cited by 114
DALL-E-Bot: Introducing Web-Scale Diffusion Models to Robotics
Kapelyukh, I., Vosylius, V., & Johns, E. (2023) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2023.3272516 · Cited by 114
Can an Embodied Agent Find Your “Cat-shaped Mug”? LLM-Based Zero-Shot Object Navigation
Dorbala, V. S., Mullen, J. F., & Manocha, D. (2023) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2023.3346800 · Cited by 114
GIT-Mol: A multi-modal large language model for molecular science with graph, image, and text
Liu, P., Ren, Y., Tao, J., & Ren, Z. (2024) · Computers in Biology and Medicine · doi:10.1016/j.compbiomed.2024.108073 · Cited by 114
TeCH: Text-Guided Reconstruction of Lifelike Clothed Humans
Huang, Y., Yi, H., Xiu, Y., Liao, T., Tang, J., Cai, D., & Thies, J. (2024) · International Conference on 3D Vision (3DV) · doi:10.1109/3dv62453.2024.00152 · Cited by 116
4D-fy: Text-to-4D Generation Using Hybrid Score Distillation Sampling
Bahmani, S., Skorokhodov, I., Rong, V., Wetzstein, G., Guibas, L. J., Wonka, P., Tulyakov, S., Park, J. J., Tagliasacchi, A., & Lindell, D. B. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.00764 · Cited by 114
VideoAgent: A Memory-Augmented Multimodal Agent for Video Understanding
Fan, Y., Ma, X., Wu, R., Du, Y., Li, J., Gao, Z. F., & Li, Q. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-72670-5_5 · Cited by 114
Grounding Large Language Models in Interactive Environments with Online Reinforcement LearningPreprint
Carta, T., Romac, C., Wolf, T., Lamprier, S., Sigaud, O., & Oudeyer, P. (2023) · International Conference on Machine Learning · doi:10.48550/arxiv.2302.02662 · Cited by 307
Frozen Pretrained Transformers as Universal Computation Engines
Lu, K., Grover, A., Abbeel, P., & Mordatch, I. (2022) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v36i7.20729 · Cited by 113
“2021), and multimodal tasks comprising of both images and text (Lu et al. 2019; Radford et al. 2021).”
BackgroundMethodInfluentialNaphade, M., Wang, S., Anastasiu, D. C., Tang, Z., Chang, M., Yang, X., Yao, Y., Zheng, L., Chakraborty, P., Sharma, A., et al. (2021) · IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw53098.2021.00482 · Cited by 113
Cross-Modal Retrieval With Partially Mismatched Pairs
Hu, P., Huang, Z., Peng, D., Wang, X., & Peng, X. (2023) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2023.3247939 · Cited by 113
What Artificial Neural Networks Can Tell Us about Human Language Acquisition
Warstadt, A., & Bowman, S. R. (2022) · Algebraic Structures in Natural Language · doi:10.1201/9781003205388-2 · Cited by 111
A vision–language foundation model for the generation of realistic chest X-ray images
Bluethgen, C., Chambon, P., Delbrouck, J.-B., van der Sluijs, R., Połacin, M., Zambrano Chaves, J. M., Abraham, T. M., Purohit, S., Langlotz, C. P., & Chaudhari, A. S. (2024) · Nature Biomedical Engineering · doi:10.1038/s41551-024-01246-y · Cited by 108
Lee, U., Han, A., Lee, J., Lee, E., Kim, J., Kim, H., & Lim, C. (2024) · Education and Information Technologies · doi:10.1007/s10639-023-12150-4 · Cited by 107
Self-citationPredictive Coding: a Theoretical and Experimental Review
Millidge, B., Seth, A., & Buckley, C. L. (2021) · arXiv · doi:10.48550/arxiv.2107.12979 · Cited by 106
Stable Video Diffusion: Scaling Latent Video Diffusion Models to Large Datasets
Blattmann, A., Dockhorn, T., Kulal, S., Mendelevitch, D., Kilian, M., Lorenz, D., Levi, Y., English, Z., Voleti, V., Letts, A., et al. (2023) · arXiv · doi:10.48550/arxiv.2311.15127 · Cited by 105
Face Generation and Editing With StyleGAN: A Survey
Melnik, A., Miasayedzenkau, M., Makaravets, D., Pirshtuk, D., Akbulut, E., Holzmann, D., Renusch, T., Reichert, G., & Ritter, H. (2024) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2024.3350004 · Cited by 105
Multimodal fusion and vision–language models: A survey for robot vision
Han, X., Chen, S., Fu, Z., Feng, Z., Fan, L., An, D., Wang, C., Guo, L., Meng, W., Zhang, X., et al. (2026) · Information Fusion · doi:10.1016/j.inffus.2025.103652 · Cited by 105
Antaki, F., Touma, S., Milad, D., El-Khoury, J., & Duval, R. (2023) · doi:10.1101/2023.01.22.23284882 · Cited by 563
Opal: Multimodal Image Generation for News Illustration
Liu, V., Qiao, H., & Chilton, L. (2022) · Proceedings of the 35th Annual ACM Symposium on User Interface Software and Technology · doi:10.1145/3526113.3545621 · Cited by 104
ETPNav: Evolving Topological Planning for Vision-Language Navigation in Continuous Environments
An, D., Wang, H., Wang, W., Wang, Z., Huang, Y., He, K., & Wang, L. (2025) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2024.3386695 · Cited by 103
Holodeck: Language Guided Generation of 3D Embodied AI Environments
Yang, Y., Sun, F.-Y., Weihs, L., Vanderbilt, E., Herrasti, A., Han, W., Wu, J., Haber, N., Krishna, R., Liu, L., et al. (2024) · 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52733.2024.01536 · Cited by 101
GenAICHI: Generative AI and HCI
Muller, M., Chilton, L. B., Kantosalo, A., Martin, C. P., & Walsh, G. (2022) · CHI Conference on Human Factors in Computing Systems Extended Abstracts · doi:10.1145/3491101.3503719 · Cited by 101
Conwell, C., Prince, J. S., Kay, K. N., Alvarez, G. A., & Konkle, T. (2024) · Nature Communications · doi:10.1038/s41467-024-53147-y · Cited by 100
Metadata
- Authors
- Radford, A., Kim, J. W., Hallacy, C., Ramesh, A., Goh, G., Agarwal, S., Sastry, G., Askell, A., Mishkin, P., Clark, J., Krueger, G., & Sutskever, I.
- Affiliations
- OpenAI (United States) (US)
- ORCID
- Kim, Jong Wook
- Venue
- arXiv
- Publisher
- arXiv
- Published
- 2021
- ISSN
- 2331-8422 (ISSN-L)
- arXiv
- 2103.00020
- OpenAlex
- W3135367836
- Cited by
- 15,293
- References
- 193
- Citations by year
- 2026: 214 · 2025: 3988 · 2024: 6413 · 2023: 3389 · 2022: 932 · 2021: 349
- Type
- posted content
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Multimodal Machine Learning Applications
- Physical Sciences › Computer Science › Artificial Intelligence › Domain Adaptation and Few-Shot Learning
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
- Keywords
- natural language supervision, zero-shot transfer, image representation learning, image captioning, visual concept learning, cross-modal learning, human action recognition, geolocation, optical character recognition, CLIP
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
