The DOI matches its DataCite record; 185 of the 193 references it cites were found in a registry or an index, and 1 are not checked yet.
Learning Transferable Visual Models From Natural Language Supervision
Open accessAbstract
State-of-the-art computer vision systems are trained to predict a fixed set of predetermined object categories. This restricted form of supervision limits their generality and usability since additional labeled data is needed to specify any other visual concept. Learning directly from raw text about images is a promising alternative which leverages a much broader source of supervision. We demonstrate that the simple pre-training task of predicting which caption goes with which image is an efficient and scalable way to learn SOTA image representations from scratch on a dataset of 400 million (image, text) pairs collected from the internet. After pre-training, natural language is used to reference learned visual concepts (or describe new ones) enabling zero-shot transfer of the model to downstream tasks. We study the performance of this approach by benchmarking on over 30 different existing computer vision datasets, spanning tasks such as OCR, action recognition in videos, geo-localization, and many types of fine-grained object classification. The model transfers non-trivially to most tasks and is often competitive with a fully supervised baseline without the need for any dataset specific training. For instance, we match the accuracy of the original ResNet-50 on ImageNet zero-shot without needing to use any of the 1.28 million training examples it was trained on. We release our code and pre-trained model weights at https://github.com/OpenAI/CLIP.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source doesn’t support the claim as written.
In the citing paper
“During training,the model learns the joint distribution between the two modalities and optimizes the model through methods such as adversarial training or maximizing likelihood, enabling it to generate the target modality”
From Multimodal fusion and vision–language models: A survey for robot vision (2026) · cites this paper as [188]
Checked by a reasoning model — may be wrong.
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Large Image Models (LIMs) such as CLIP [86] and BLIP-2 [87] extend the agent's capabilities into the visual domain. Trained on image-text pairs, LIMs enable perception-based tasks including image classification, object detection, and vision-language grounding.”
From AI Agents vs. Agentic AI: A Conceptual taxonomy, applications and challenges (2026) · cites this paper as [86]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Foundation Models (FMs) [23, 35, 37] - generic models trained on web-scale datasets - have demonstrated remarkable zero-shot performance and can be quickly adapted to various downstream tasks with few additional tuning cost.”
From Towards More General Video-based Deepfake Detection through Facial Component Guided Adaptation for Foundation Model (2024) · cites this paper as [37]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“By encoding text descriptions of variations of the dataset bias (e.g.'a blue wall'), using CLIP's text encoder (CLIP T ), we vary visual representations without actually having seen images containing these variations (e.g. an image of a blue wall).”
From Language-Based Augmentation to Address Shortcut Learning in Object-Goal Navigation (2023) · cites this paper as [15]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“However, their capacity for semantic comprehension is comparatively limited when contrasted with vision-based perceptual models like CLIP (Radford et al., 2021).”
From WorldVLA: Towards Autoregressive Action World Model (2025) · cites this paper as Radford et al., 2021
Experimental — accuracy not yet measured
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“In natural language [1] and computer vision [39], general-purpose foundation models that are pre-trained on diverse multi-task data tend to outperform narrowly tailored and specialized solutions.”
From $π_0$: A Vision-Language-Action Flow Model for General Robot Control (2024) · cites this paper as [39]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Existing evaluation metrics such as CLIP-Image score [37] and DINO score [6], while commonly used, often fail to reflect subtle differences due to their reliance on high-dimensional semantic features for cosine similarity computation.”
From DiffSim: Taming Diffusion Models for Evaluating Visual Similarity (2024) · cites this paper as [37]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“CLIP cosine similarity is often used to measure text prompt and frame consistency, where a reference video dataset is not needed.”
From Video Diffusion Models: A Survey (2024) · cites this paper as Radford et al., 2021
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Trained on image-text pairs, LIMs enable perception-based tasks including image classification, object detection, and vision-language grounding.”
From AI Agents vs. Agentic AI: A Conceptual Taxonomy, Applications and Challenges (2025) · cites this paper as [88]
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“Vision-Language Models (VLMs) e.g., CLIP [15], allows us to augment at feature-level based on prior knowledge of the environment without any changes to training data.”
From Language-Based Augmentation to Address Shortcut Learning in Object-Goal Navigation (2023) · cites this paper as [15]
The cited source says what the claim attributes to it.
In the citing paper
“We find these methods perform poorly on language and image goals, particularly when relying on CLIP [25] features.”
From GOAT-Bench: A Benchmark for Multi-Modal Lifelong Navigation (2024) · cites this paper as [25]
The cited source says what the claim attributes to it.
In the citing paper
“A notable advancement in this domain is the recent work on CLIP [9, 32, 35, 43], which elevates open-vocabulary recognition performance by collecting millions of image-text pairs and employing contrastive learning techniques.”
From Active Open-Vocabulary Recognition: Let Intelligent Moving Mitigate CLIP Limitations (2024) · cites this paper as [32]
The cited source says what the claim attributes to it.
In the citing paper
“The recently proposed Contrastive Language-Image Pretraining (CLIP) [1] learns the correspondence between text and image by projecting them into a shared latent space.”
From Large-Scale Contrastive Language-Audio Pretraining with Feature Fusion and Keyword-to-Caption Augmentation (2023) · cites this paper as [1]
The cited source says what the claim attributes to it.
In the citing paper
“To evaluate the visual similarity of I R and I G , we use the similarity of their CLIP (Radford et al., 2021) embedding, denoted as CLIP ( I R , I G ) .”
From Design2Code: Benchmarking Multimodal Code Generation for Automated Front-End Engineering (2025) · cites this paper as Radford et al., 2021
The cited source says what the claim attributes to it.
In the citing paper
“This suggests the inability of CLIP [25] features in capturing crucial instance-specific and spatial features in language and image goals.”
From GOAT-Bench: A Benchmark for Multi-Modal Lifelong Navigation (2024) · cites this paper as [25]
The cited source says what the claim attributes to it.
In the citing paper
“Existing solutions in multimodal Transformer adopt a fixed fusion strategy [18,21,24,26,33].”
From Are Multimodal Transformers Robust to Missing Modality? (2022) · cites this paper as [33]
The cited source says what the claim attributes to it.
In the citing paper
“existing foundation models for vision and language such as CLIP [8], SigLIP [9], and Llama 2 [10] are capable of these types of generalization and more, stemming from the priors captured by their Internet-scale pretraining datasets.”
From OpenVLA: An Open-Source Vision-Language-Action Model (2024) · cites this paper as [8]
The cited source says what the claim attributes to it.
In the citing paper
“which shows that our model achieves the highest DINO [5] and CLIP-I [42] scores, with CLIP-T [42] scores that are comparable to other methods.”
From DreamingComics: A Story Visualization Pipeline via Subject and Layout Customized Generation using Video Models (2025) · cites this paper as [42]
The cited source says what the claim attributes to it.
In the citing paper
“In early 2021, OpenAI released an architecture termed Contrastive Language Image Pretraining (CLIP), which, when trained on millions of image-text pairs, matched the performance of competitive, fully supervised models without fine-tuning 106”
From Multimodal biomedical AI (2022) · cites this paper as [106]
The cited source says what the claim attributes to it.
In the citing paper
“Even when PwD data is available, data from marginalized groups is often automatically filtered out by techniques for removing 'low' quality data [57, 104], which rely on biased models such as CLIP [92].”
From Engaging Communities Meaningfully in Defining Disability Representation for AI Image Generation (2026) · cites this paper as [92]
The check wasn’t confident either way, so it gives no verdict — never disputed.
In the citing paper
“HOLODECK uses a Large Language Model (GPT-4 [37]) to design the floor plan, assign suitable materials, install the doorways and windows and arrange 3D assets coherently in the scene using constraintbased optimization.”
From Holodeck: Language Guided Generation of 3D Embodied AI Environments (2024) · cites this paper as [37]
Experimental — accuracy not yet measuredThe check wasn’t confident either way, so it gives no verdict — never disputed.
In the citing paper
“[41] famously found evidence of a ‘cat neuron’, a phenomenon that has been remarkably replicable across newer models and analyses [14,42,43]”
From Symbols and grounding in large language models (2023) · cites this paper as [43]
Experimental — accuracy not yet measuredThe check wasn’t confident either way, so it gives no verdict — never disputed.
In the citing paper
“Text-to-image models like Stable Diffusion, Midjourney, and DALL-E [26,27] enable educators to create visual learning materials from textual content.”
From Promises and challenges of generative artificial intelligence for human learning (2024) · cites this paper as [26]
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., & Polosukhin, I. (2017) · arXiv · doi:10.48550/arxiv.1706.03762 · Cited by 120,404
Adam: A Method for Stochastic Optimization
Kingma, D. P., & Ba, J. (2014) · arXiv · doi:10.48550/arxiv.1412.6980 · Cited by 98,946
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2018) · arXiv · doi:10.48550/arxiv.1810.04805 · Cited by 73,173
Scikit-learn: Machine Learning in Python
Pedregosa, F., Varoquaux, G., Gramfort, A., Michel, V., Thirion, B., Grisel, O., Blondel, M., Müller, A., Nothman, J., Louppe, G., et al. (2012) · arXiv · doi:10.48550/arxiv.1201.0490 · Cited by 71,016
ImageNet: A large-scale hierarchical image database
Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., & Fei-Fei, L. (2009) · 2009 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2009.5206848 · Cited by 65,333
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2019) · Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) · doi:10.18653/v1/n19-1423 · Cited by 54,325
An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale
Dosovitskiy, A., Beyer, L., Kolesnikov, A., Weissenborn, D., Zhai, X., Unterthiner, T., Dehghani, M., Minderer, M., Heigold, G., Gelly, S., et al. (2020) · arXiv · doi:10.48550/arxiv.2010.11929 · Cited by 47,472
SciPy 1.0: fundamental algorithms for scientific computing in Python
Virtanen, P., Gommers, R., Oliphant, T. E., Haberland, M., Reddy, T., Cournapeau, D., Burovski, E., Peterson, P., Weckesser, W., Bright, J., et al. (2020) · Nature Methods · doi:10.1038/s41592-019-0686-2 · Cited by 38,430
Glove: Global Vectors for Word Representation
Pennington, J., Socher, R., & Manning, C. (2014) · Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.3115/v1/d14-1162 · Cited by 34,071
Research works printed but not linked: 24
- Bengio, Y., Ducharme, R., Vincent, P., and Jauvin, C. A neural probabilistic language model. Journal of machine learning research , 3(Feb):1137-1155, 2003.
- Browne, S. Dark Matters: Surveillance of Blackness . Duke University Press, 2015.
- Chen, Y.-C., Li, L., Yu, L., Kholy, A. E., Ahmed, F., Gan, Z., Cheng, Y., and Liu, J. Uniter: Learning universal imagetext representations. arXiv preprint arXiv:1909.11740 , 2019. · arXiv
- Coates, A., Ng, A., and Lee, H. An analysis of singlelayer networks in unsupervised feature learning. In Proceedings of the fourteenth international conference on artificial intelligence and statistics , pp. 215-223, 2011.
- Crawford, K. The trouble with bias. NIPS 2017 Keynote , 2017. URL https://www.youtube.com/ watch?v=fMym\_BKWQzk .
- Frome, A., Corrado, G. S., Shlens, J., Bengio, S., Dean, J., Ranzato, M., and Mikolov, T. Devise: A deep visualsemantic embedding model. In Advances in neural information processing systems , pp. 2121-2129, 2013.
- Geirhos, R., Rubisch, P., Michaelis, C., Bethge, M., Wichmann, F. A., and Brendel, W. Imagenet-trained cnns are biased towards texture; increasing shape bias improves accuracy and robustness. arXiv preprint arXiv:1811.12231 , 2018. · arXiv
- Goodfellow, I. J., Shlens, J., and Szegedy, C. Explaining and harnessing adversarial examples. arXiv preprint arXiv:1412.6572 , 2014. · arXiv
- Ha, D., Dai, A., and Le, Q. V. Hypernetworks. arXiv preprint arXiv:1609.09106 , 2016. · arXiv
- He, K., Zhang, X., Ren, S., and Sun, J. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition , pp. 770-778, 2016b.
- Hendrycks, D. and Dietterich, T. Benchmarking neural network robustness to common corruptions and perturbations. arXiv preprint arXiv:1903.12261 , 2019. · arXiv
- Hendrycks, D., Liu, X., Wallace, E., Dziedzic, A., Krishnan, R., and Song, D. Pretrained transformers improve out-ofdistribution robustness. arXiv preprint arXiv:2004.06100 , 2020b. · arXiv
- Ilyas, A., Santurkar, S., Tsipras, D., Engstrom, L., Tran, B., and Madry, A. Adversarial examples are not bugs, they are features. In Advances in Neural Information Processing Systems , pp. 125-136, 2019.
- Jaderberg, M., Simonyan, K., Zisserman, A., et al. Spatial transformer networks. Advances in neural information processing systems , 28:2017-2025, 2015.
- Johnson, J., Hariharan, B., van der Maaten, L., Fei-Fei, L., Lawrence Zitnick, C., and Girshick, R. Clevr: A diagnostic dataset for compositional language and elementary visual reasoning. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition , pp. 2901-2910, 2017.
- Kiros, R., Zhu, Y., Salakhutdinov, R. R., Zemel, R., Urtasun, R., Torralba, A., and Fidler, S. Skip-thought vectors. Advances in neural information processing systems , 28: 3294-3302, 2015.
- Krizhevsky, A., Sutskever, I., and Hinton, G. E. Imagenet classification with deep convolutional neural networks. In Advances in neural information processing systems , pp. 1097-1105, 2012.
- Lin, T.-Y., Maire, M., Belongie, S., Hays, J., Perona, P., Ramanan, D., Doll´ ar, P., and Zitnick, C. L. Microsoft coco: Common objects in context. In European conference on computer vision , pp. 740-755. Springer, 2014.
- Proceedings of the European Conference on Computer Vision (ECCV) , pp. 181-196, 2018.
- McCann, B., Bradbury, J., Xiong, C., and Socher, R. Learned in translation: Contextualized word vectors. In Advances in neural information processing systems , pp. 6294-6305, 2017.
- Socher, R. and Fei-Fei, L. Connecting modalities: Semisupervised segmentation and annotation of images using unaligned text corpora. In 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition , pp. 966-973. IEEE, 2010.
- Srivastava, N. and Salakhutdinov, R. Multimodal learning with deep boltzmann machines. In NIPS , 2012.
- Thomee, B., Shamma, D. A., Friedland, G., Elizalde, B., Ni, K., Poland, D., Borth, D., and Li, L.-J. Yfcc100m: The new data in multimedia research. Communications of the ACM , 59(2):64-73, 2016.
- Touvron, H., Vedaldi, A., Douze, M., and J´ egou, H. Fixing the train-test resolution discrepancy. In Advances in neural information processing systems , pp. 8252-8262, 2019.
Cited by
Sorted by citations
Items 1,101–1,200 of 1,203
The Platonic Representation Hypothesis
Huh, M., Cheung, B., Wang, T., & Isola, P. (2024) · arXiv · doi:10.48550/arxiv.2405.07987 · Cited by 35
“…vision models in total: • 17 ViT models ranging from ViT-tiny to ViT-giant, trained on tasks including ImageNet-21k (Dosovitskiy et al., 2020) classification, Masked Autoencoders (He et al., 2021), DINO (Caron et al., 2021), and CLIP (Radford et al., 2021), including some finetuned on ImageNet-12k.”
InfluentialLearning cortical representations through perturbed and adversarial dreaming
Deperrois, N., Petrovici, M. A., Senn, W., & Jordan, J. (2022) · eLife · doi:10.7554/elife.76384 · Cited by 28
Learning Spatial Adaptation and Temporal Coherence in Diffusion Models for Video Super-Resolution
Chen, Z., Long, F., Qiu, Z., Yao, T., Zhou, W., Luo, J., & Mei, T. (2024) · 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52733.2024.00882 · Cited by 27
AIGCOIQA2024: Perceptual Quality Assessment of AI Generated Omnidirectional Images
Yang, L., Duan, H., Teng, L., Zhu, Y., Liu, X., Hu, M., Min, X., Zhai, G., & Callet, P. L. (2024) · arXiv · doi:10.48550/arxiv.2404.01024 · Cited by 27
Development of an AI model for DILI-level prediction using liver organoid brightfield images
Tan, S., Ding, Y., Wang, W., Rao, J., Cheng, F., Zhang, Q., Xu, T., Hu, T., Hu, Q., Ye, Z., et al. (2025) · Communications Biology · doi:10.1038/s42003-025-08205-6 · Cited by 26
Deeper evaluation of a single-cell foundation model
Boiarsky, R., Singh, N. M., Buendia, A., Amini, A. P., Getz, G., & Sontag, D. (2024) · Nature Machine Intelligence · doi:10.1038/s42256-024-00949-w · Cited by 25
MMDialog: A Large-scale Multi-turn Dialogue Dataset Towards Multi-modal Open-domain Conversation
Feng, J., Sun, Q., Xu, C., Zhao, P., Yang, Y., Tao, C., Zhao, D., & Lin, Q. (2023) · Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/2023.acl-long.405 · Cited by 26
Formal Mathematics Statement Curriculum Learning
Polu, S., Han, J. M., Zheng, K., Baksys, M., Babuschkin, I., & Sutskever, I. (2022) · arXiv · doi:10.48550/arxiv.2202.01344 · Cited by 26
Self-citationGenerating Executable Action Plans with Environmentally-Aware Language Models
Gramopadhye, M., & Szafir, D. (2023) · 2023 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros55552.2023.10341989 · Cited by 26
Scaling Up and Distilling Down: Language-Guided Robot Skill Acquisition
Ha, H., Florence, P., & Song, S. (2023) · arXiv · doi:10.48550/arxiv.2307.14535 · Cited by 24
“This policy takes as input a task description CLIP [56] feature, proprioception history, and visual observations, and outputs a sequence of end effector control commands.”
MethodMindstorms in Natural Language-Based Societies of Mind
Zhuge, M., Liu, H., Faccio, F., Ashley, D. R., Csordás, R., Gopalakrishnan, A., Hamdi, A., Hammoud, H. A. A. K., Herrmann, V., Irie, K., et al. (2023) · arXiv · doi:10.48550/arxiv.2305.17066 · Cited by 24
Self-citationMindstorms in Natural Language-Based Societies of Mind
Zhuge, M., Liu, H., Faccio, F., Ashley, D. R., Csordás, R., Gopalakrishnan, A., Hamdi, A., Hammoud, H. A. A. K., Herrmann, V., Irie, K., et al. (2025) · Computational Visual Media · doi:10.26599/cvm.2025.9450460 · Cited by 24
Self-citationCUDA: Convolution-Based Unlearnable Datasets
Sadasivan, V. S., Soltanolkotabi, M., & Feizi, S. (2023) · 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52729.2023.00376 · Cited by 24
VideoMAC: Video Masked Autoencoders Meet ConvNets
Pei, G., Chen, T., Jiang, X., Liu, H., Sun, Z., & Yao, Y. (2024) · 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52733.2024.02145 · Cited by 23
“A key advantage of SSL [2, 5–8, 18, 22, 48, 57, 69, 72] is its independence from costly human annotations, thereby providing robust prior knowledge across various visual tasks.”
BackgroundTurgut, Ö., Müller, P., Hager, P., Shit, S., Starck, S., Menten, M. J., Martens, E., & Rueckert, D. (2025) · Medical Image Analysis · doi:10.1016/j.media.2024.103451 · Cited by 22
Wu, X., Hao, Y., Sun, K., Chen, Y., Zhu, F., Zhao, R., & Li, H. (2023) · arXiv · doi:10.48550/arxiv.2306.09341 · Cited by 22
“We also plotted HPS v2 against existing evaluation metrics, CLIP Score [20] and Aesthetic Score [28].”
BackgroundMethodInfluentialProof Artifact Co-training for Theorem Proving with Language Models
Han, J. M., Rute, J., Wu, Y., Ayers, E. W., & Polu, S. (2021) · arXiv · doi:10.48550/arxiv.2102.06203 · Cited by 21
Large language model for medical images: a survey of taxonomy, systematic review, and future trends
Wang, P., Lu, W., Lu, C., Zhou, R., Li, M., & Qin, L. (2025) · Big Data Mining and Analytics · doi:10.26599/bdma.2024.9020090 · Cited by 21
Inspecting the Geographical Representativeness of Images from Text-to-Image Models
Basu, A., Babu, R. V., & Pruthi, D. (2023) · 2023 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.00474 · Cited by 21
Bridging the Gap between 2D and 3D Visual Question Answering: A Fusion Approach for 3D VQA
Mo, W., & Liu, Y. (2024) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v38i5.28222 · Cited by 20
Improving Multi-Agent Debate with Sparse Communication Topology
Li, Y., Du, Y., Zhang, J., Hou, L., Grabowski, P., Li, Y., & Ie, E. (2024) · Findings of the Association for Computational Linguistics: EMNLP 2024 · doi:10.18653/v1/2024.findings-emnlp.427 · Cited by 20
Abdulgalil, H. D., & Basir, O. A. (2025) · Natural Language Processing Journal · doi:10.1016/j.nlp.2025.100159 · Cited by 20
GOAT-Bench: A Benchmark for Multi-Modal Lifelong Navigation
Khanna, M., Ramrakhya, R., Chhablani, G., Yenamandra, S., Gervet, T., Chang, M., Kira, Z., Chaplot, D. S., Batra, D., & Mottaghi, R. (2024) · 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52733.2024.01549 · Cited by 23
Weimann, K., & Conrad, T. O. F. (2024) · IEEE Journal of Biomedical and Health Informatics · doi:10.1109/jbhi.2024.3427787 · Cited by 19
Detecting clinical medication errors with AI enabled wearable cameras
Chan, J., Nsumba, S., Wortsman, M., Dave, A., Schmidt, L., Gollakota, S., & Michaelsen, K. (2024) · npj Digital Medicine · doi:10.1038/s41746-024-01295-2 · Cited by 18
IMU2CLIP: Language-grounded Motion Sensor Translation with Multimodal Contrastive Learning
Moon, S., Madotto, A., Lin, Z., Saraf, A., Bearman, A., & Damavandi, B. (2023) · Findings of the Association for Computational Linguistics: EMNLP 2023 · doi:10.18653/v1/2023.findings-emnlp.883 · Cited by 18
(2025) · doi:10.1145/3833415 · Cited by 18
Design2Code: Benchmarking Multimodal Code Generation for Automated Front-End Engineering
Si, C., Zhang, Y., Li, R., Yang, Z., Liu, R., & Yang, D. (2025) · Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers) · doi:10.18653/v1/2025.naacl-long.199 · Cited by 17
“On the other hand, the color similarity and the CLIP similarity steadily increase until 4000 − 5000 steps.”
InfluentialMultimodal Large Language Models are Generalist Medical Image Interpreters
Han, T., Adams, L. C., Nebelung, S., Kather, J. N., Bressem, K. K., & Truhn, D. (2023) · doi:10.1101/2023.12.21.23300146 · Cited by 18
WebCode2M: A Real-World Dataset for Code Generation from Webpage Designs
Gui, Y., Li, Z., Wan, Y., Shi, Y., Zhang, H., Chen, B., Su, Y., Chen, D., Wu, S., Zhou, X., et al. (2025) · Proceedings of the ACM on Web Conference 2025 · doi:10.1145/3696410.3714889 · Cited by 17
Exploring the Transferability of Visual Prompting for Multimodal Large Language Models
Zhang, Y., Dong, Y., Zhang, S., Min, T., Su, H., & Zhu, J. (2024) · 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52733.2024.02508 · Cited by 17
Prefix Conditioning Unifies Language and Label Supervision
Saito, K., Sohn, K., Zhang, X., Li, C.-L., Lee, C.-Y., Saenko, K., & Pfister, T. (2022) · arXiv · doi:10.48550/arxiv.2206.01125 · Cited by 17
Goal Misgeneralization: Why Correct Specifications Aren't Enough For Correct Goals
Shah, R., Varma, V., Kumar, R., Phuong, M., Krakovna, V., Uesato, J., & Kenton, Z. (2022) · arXiv · doi:10.48550/arxiv.2210.01790 · Cited by 16
Multimodal Federated Learning: A Survey
Che, L., Wang, J., Zhou, Y., & Ma, F. (2023) · doi:10.20944/preprints202307.1420.v1 · Cited by 127
In conversation with Artificial Intelligence: aligning language models with human values
Kasirzadeh, A., & Gabriel, I. (2022) · arXiv · doi:10.48550/arxiv.2209.00731 · Cited by 128
“2For GPT-3, see [19]; for BERT, see [28]; for Turing-NLG-A-17, see [85]; for CLIP, see [81]; for Gopher, see [82].”
BackgroundEnabling Intelligent Interactions between an Agent and an LLM: A Reinforcement Learning Approach
Hu, B., Zhao, C., Zhang, P., Zhou, Z., Yang, Y., Xu, Z., & Liu, B. (2023) · arXiv · doi:10.48550/arxiv.2306.03604 · Cited by 15
“Alter-natively, pre-trained image translation models such as CLIP [22] can also be used for this purpose.”
MethodEcosystem Graphs: The Social Footprint of Foundation Models
Bommasani, R., Soylu, D., Liao, T., Creel, K., & Liang, P. (2023) · doi:10.21203/rs.3.rs-2961271/v1 · Cited by 14
Beyond Medical Imaging - A Review of Multimodal Deep Learning in Radiology
Heiliger, L., Sekuboyina, A., Menze, B., Egger, J., & Kleesiek, J. (2022) · doi:10.36227/techrxiv.19103432 · Cited by 13
Prioritized Semantic Learning for Zero-Shot Instance Navigation
Sun, X., Liu, L., Zhi, H., Qiu, R., & Liang, J. (2025) · Lecture Notes in Computer Science · doi:10.1007/978-3-031-73254-6_10 · Cited by 12
Diffusion Policy: Visuomotor Policy Learning via Action Diffusion
Chi, C., Xu, Z., Feng, S., Cousineau, E., Du, Y., Burchfiel, B., Tedrake, R., & Song, S. (2023) · arXiv · doi:10.48550/arxiv.2303.04137 · Cited by 12
“This is especially true for the CLIP-trained ViT-B/16, which reaches 98% success rate with only 50 epochs of training.”
Yager, K. G. (2024) · Digital Discovery · doi:10.1039/d4dd00178h · Cited by 12
Exploiting Modality-Specific Features for Multi-Modal Manipulation Detection and Grounding
Wang, J., Liu, B., Miao, C., Zhao, Z., Zhuang, W., Chu, Q., & Yu, N. (2024) · ICASSP 2024 - 2024 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) · doi:10.1109/icassp48485.2024.10448385 · Cited by 12
“Categories Binary Cls Multi-Label Cls Image Grounding Text Grounding Methods Params AUC EER ACC mAP CF1 OF1 IoUmean IoU50 IoU75 Precision Recall F1 CLIP [26] - 83.”
MethodWang, S., Zhou, D., Xie, L., Xu, C., Yan, Y., & Yin, E. (2025) · Neural Networks · doi:10.1016/j.neunet.2025.107320 · Cited by 11
Sparks of Large Audio Models: A Survey and Outlook
Latif, S., Shoukat, M., Shamshad, F., Usama, M., Ren, Y., Cuayáhuitl, H., Wang, W., Zhang, X., Togneri, R., Cambria, E., et al. (2023) · arXiv · doi:10.48550/arxiv.2308.12792 · Cited by 17
“This model harnesses the capabilities of an audio transformer HTSAT [243] as an audio encoder, along with CLIP’s [244] text encoder and GPT2-base as a language model.”
Method$π_0$: A Vision-Language-Action Flow Model for General Robot Control
Black, K., Brown, N., Driess, D., Esmail, A., Equi, M., Finn, C., Fusai, N., Groom, L., Hausman, K., Ichter, B., et al. (2024) · arXiv · doi:10.48550/arxiv.2410.24164 · Cited by 13
“In natural language [1] and computer vision [39], general-purpose foundation models that are pre-trained on diverse multi-task data tend to outperform narrowly tailored and specialized solutions.”
On the Effectiveness of Retrieval, Alignment, and Replay in Manipulation
Palo, N. D., & Johns, E. (2024) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2024.3349832 · Cited by 14
Generative AI in the Era of 'Alternative Facts'
Gabriel, S., Lyu, L., Siderius, J., Ghassemi, M., Andreas, J., & Ozdaglar, A. (2024) · An MIT Exploration of Generative AI · doi:10.21428/e4baedd9.82175d26 · Cited by 11
Towards Geographic Inclusion in the Evaluation of Text-to-Image Models
Hall, M., Bell, S. J., Ross, C., Williams, A., Drozdzal, M., & Soriano, A. R. (2024) · The 2024 ACM Conference on Fairness Accountability and Transparency · doi:10.1145/3630106.3658927 · Cited by 11
Toward Long Video Understanding via Fine-Detailed Video Story Generation
You, Z., Wen, Z., Chen, Y., Li, X., Zeng, R., Wang, Y., & Tan, M. (2025) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2024.3514820 · Cited by 11
PENSIEVE-AI a brief cognitive test to detect cognitive impairment across diverse literacy
Liew, T. M., Foo, J. Y. H., Yang, H., Tay, S. Y., Koay, W. I., Yip, K. F., Ting, S. K. S., Narasimhalu, K., Li, W., Tan, C., et al. (2025) · Nature Communications · doi:10.1038/s41467-025-58201-x · Cited by 11
Unleashing the potential of cell painting assays for compound activities and hazards prediction
Odje, F., Meijer, D., von Coburg, E., van der Hooft, J. J. J., Dunst, S., Medema, M. H., & Volkamer, A. (2024) · Frontiers in Toxicology · doi:10.3389/ftox.2024.1401036 · Cited by 11
LEDNet: a multimodal foundation model for robust deepfake detection
Tao, R., Tang, S., Qin, H., Wang, W., Wei, Y., & Zhao, Y. (2025) · Science China Information Sciences · doi:10.1007/s11432-024-4400-8 · Cited by 9
Jan, R. (2024) · Advances in Educational Technologies and Instructional Design · doi:10.4018/979-8-3693-0884-4.ch014 · Cited by 9
VRSBench: A Versatile Vision-Language Benchmark Dataset for Remote Sensing Image Understanding
Li, X., Ding, J., & Elhoseiny, M. (2024) · arXiv · doi:10.48550/arxiv.2406.12384 · Cited by 47
“We employ CLIP-ViT(L-14) [39] as the vision encoder and use the Vicuna-7B model [40] as the Large Language Model (LLM).”
Li, Z., Li, H., Ralescu, A. L., Dillman, J. R., Altaye, M., Cecil, K. M., Parikh, N. A., & He, L. (2024) · Artificial Intelligence in Medicine · doi:10.1016/j.artmed.2024.102993 · Cited by 10
A multi-modal deep language model for contaminant removal from metagenome-assembled genomes
Zou, B., Wang, J., Ding, Y., Zhang, Z., Huang, Y., Fang, X., Cheung, K. C., See, S., & Zhang, L. (2024) · Nature Machine Intelligence · doi:10.1038/s42256-024-00908-5 · Cited by 9
Gemini Robotics: Bringing AI into the Physical World
Gemini Robotics Team, Abeyruwan, S., Ainslie, J., Alayrac, J.-B., Arenas, M. G., Armstrong, T., Balakrishna, A., Baruch, R., Bauza, M., Blokzijl, M., et al. (2025) · arXiv · doi:10.48550/arxiv.2503.20020 · Cited by 8
UniMTS: Unified Pre-training for Motion Time Series
Zhang, X., Teng, D., Chowdhury, R., Li, S., Hong, D., Gupta, R., & Shang, J. (2024) · Advances in Neural Information Processing Systems 37 · doi:10.52202/079017-3414 · Cited by 8
Learning Music Sequence Representation From Text Supervision
Chen, T., Xie, Y., Zhang, S., Huang, S., Zhou, H., & Li, J. (2022) · ICASSP 2022 - 2022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) · doi:10.1109/icassp43922.2022.9746131 · Cited by 8
Massive Activations in Large Language Models
Sun, M., Chen, X., Kolter, J. Z., & Liu, Z. (2024) · arXiv · doi:10.48550/arxiv.2402.17762 · Cited by 8
Li, Y., Kong, C., Zhao, G., & Zhao, Z. (2025) · Artificial Intelligence Review · doi:10.1007/s10462-025-11337-0 · Cited by 7
Prismatic VLMs: Investigating the Design Space of Visually-Conditioned Language Models
Karamcheti, S., Nair, S., Balakrishna, A., Liang, P., Kollar, T., & Sadigh, D. (2024) · arXiv · doi:10.48550/arxiv.2402.07865 · Cited by 7
“For TextVQA, we also run a variant of the evaluation where VLMs are additionally prompted with input tokens parsed by an OCR-system.”
BackgroundMethodInfluentialXiao, J., Wan, Y., Huo, Y., Wang, Z., Xu, X., Wang, W., Xu, Z., Wang, Y., & Lyu, M. R. (2024) · arXiv · doi:10.48550/arxiv.2411.03292 · Cited by 7
RoboPoint: A Vision-Language Model for Spatial Affordance Prediction for Robotics
Yuan, W., Duan, J., Blukis, V., Pumacay, W., Krishna, R., Murali, A., Mousavian, A., & Fox, D. (2024) · arXiv · doi:10.48550/arxiv.2406.10721 · Cited by 7
ColPali: Efficient Document Retrieval with Vision Language Models
Faysse, M., Sibille, H., Wu, T., Omrani, B., Viaud, G., Hudelot, C., & Colombo, P. (2024) · arXiv · doi:10.48550/arxiv.2407.01449 · Cited by 7
Active Open-Vocabulary Recognition: Let Intelligent Moving Mitigate CLIP Limitations
Fan, L., Zhou, J., Xing, X., & Wu, Y. (2024) · 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52733.2024.01551 · Cited by 6
V-JEPA 2: Self-Supervised Video Models Enable Understanding, Prediction and Planning
Assran, M., Bardes, A., Fan, D., Garrido, Q., Howes, R., Mojtaba, Komeili, Muckley, M., Rizvi, A., Roberts, C., et al. (2025) · arXiv · doi:10.48550/arxiv.2506.09985 · Cited by 8
“Despite using a common evaluation protocol, the baseline encoders are trained on different data (e.g., DINOv2 on LVD-142M, PE core G on MetaCLIP) and are thus not directly comparable.”
Semi-supervised multimodal representation learning through a global workspace
Devillers, B., Maytié, L., & VanRullen, R. (2025) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2024.3416701 · Cited by 5
Do Vision and Language Models Share Concepts? A Vector Space Alignment Study
Li, J., Kementchedjhieva, Y., Fierro, C., & Søgaard, A. (2024) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00698 · Cited by 6
“CLIP Results.”
MethodInfluentialProcedural Generalization by Planning with Self-Supervised World Models
Anand, A., Walker, J., Li, Y., Vértes, E., Schrittwieser, J., Ozair, S., Weber, T., & Hamrick, J. B. (2021) · arXiv · doi:10.48550/arxiv.2111.01587 · Cited by 5
CAVEN: An Embodied Conversational Agent for Efficient Audio-Visual Navigation in Noisy Environments
Liu, X., Paul, S., Chatterjee, M., & Cherian, A. (2024) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v38i4.28167 · Cited by 6
“…ResNet-152 features from g view , coupled with the position encoding of g sub , and the embeddings of hitherto produced words in the question (e.g., GloVe (Pennington, Socher, and Manning 2014) or CLIP (Radford et al. 2021)), and proceeds to generate the next word in the question autoregressively.”
BackgroundMethodInfluentialSafetywashing: Do AI Safety Benchmarks Actually Measure Safety Progress?
Ren, R., Basart, S., Khoja, A., Gatti, A., Phan, L., Yin, X., Mazeika, M., Pan, A., Mukobi, G., Kim, R. H., et al. (2024) · arXiv · doi:10.48550/arxiv.2407.21792 · Cited by 10
Enhancing Multimodal Model Robustness Under Missing Modalities via Memory-Driven Prompt Learning
Zhao, Y., Xi, W., Fu, X., & Zhao, J. (2025) · Proceedings of the Thirty-Fourth International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2025/274 · Cited by 6
Visual Conceptual Blending with Large-scale Language and Vision Models
Ge, S., & Parikh, D. (2021) · arXiv · doi:10.48550/arxiv.2106.14127 · Cited by 4
Unveiling the Lack of LVLM Robustness to Fundamental Visual Variations: Why and Path Forward
Fan, Z., Wang, Y., Polisetty, S., & Fung, Y. R. (2025) · Findings of the Association for Computational Linguistics: ACL 2025 · doi:10.18653/v1/2025.findings-acl.1037 · Cited by 4
Efficient neural decoding based on multimodal training
Wang, Y. (2024) · Brain Sciences · doi:10.3390/brainsci14100988 · Cited by 3
‘We can see a savage’: a case study of the colonial gaze in generative AI algorithms
Alenichev, A., Shaffer, J. D., Kingori, P., Grietens, K. P., Muldoon, J., & Rocher, L. (2026) · AI & SOCIETY · doi:10.1007/s00146-025-02685-0 · Cited by 3
“Because contemporary multimodal models are trained on web-scale corpora and repeated imagery (Radford et al. 2021; (Birhane and Prabhu 2021)), we intentionally sampled images that are already ubiquitous online.”
Human-inspired Perspectives: A Survey on AI Long-term Memory
He, Z., Lin, W., Zheng, H., Zhang, F., Jones, M. W., Aitchison, L., Xu, X., Liu, M., Kristensson, P. O., & Shen, J. (2024) · arXiv · doi:10.48550/arxiv.2411.00489 · Cited by 3
“In contrast, dense retrieval matches queries with memory fragments in deep semantic representations format [224, 225] that cater to various modalities, including text, images, audio and code [227, 228, 229].”
SmolVLA: A Vision-Language-Action Model for Affordable and Efficient Robotics
Shukor, M., Aubakirova, D., Capuano, F., Kooijmans, P., Palma, S., Zouitine, A., Aractingi, M., Pascal, C., Russi, M., Marafioti, A., et al. (2025) · arXiv · doi:10.48550/arxiv.2506.01844 · Cited by 695
Scaling Laws for Galaxy Images
Walmsley, M., Bowles, M., Scaife, A. M. M., Makechemu, J. S., Gordon, A. J., Ferguson, A. M. N., Mann, R. G., Pearson, J., Popp, J. J., Bovy, J., et al. (2024) · arXiv · doi:10.48550/arxiv.2404.02973 · Cited by 2
“Notably, self-supervised training on web-scale datasets shows remarkable few-and zero-shot generalization performance (e.g. [7, 38]).”
BackgroundThe Future of Continual Learning in the Era of Foundation Models: Three Key Directions
Bell, J., Quarantiello, L., Coleman, E. N., Li, L., Li, M., Madeddu, M., Piccoli, E., & Lomonaco, V. (2025) · arXiv · doi:10.48550/arxiv.2506.03320 · Cited by 2
Language-Based Augmentation to Address Shortcut Learning in Object-Goal Navigation
Hoftijzer, D., Burghouts, G., & Spreeuwers, L. (2023) · 2023 Seventh IEEE International Conference on Robotic Computing (IRC) · doi:10.1109/irc59093.2023.00007 · Cited by 2
WorldVLA: Towards Autoregressive Action World Model
Cen, J., Yu, C., Yuan, H., Jiang, Y., Huang, S., Guo, J., Li, X., Song, Y., Luo, H., Wang, F., et al. (2025) · arXiv · doi:10.48550/arxiv.2506.21539 · Cited by 1
“However, their capacity for semantic comprehension is comparatively limited when contrasted with vision-based perceptual models like CLIP (Radford et al., 2021).”
GAIA-2: A Controllable Multi-View Generative World Model for Autonomous Driving
Russell, L., Hu, A., Bertoni, L., Fedoseev, G., Shotton, J., Arani, E., & Corrado, G. (2025) · arXiv · doi:10.48550/arxiv.2503.20523 · Cited by 1
“To enable semantic scene conditioning, GAIA-2 supports conditioning on CLIP embeddings [24].”
UniMTS: Unified Pre-training for Motion Time Series
Zhang, X., Teng, D., Chowdhury, R. R., Li, S., Hong, D., Gupta, R. K., & Shang, J. (2024) · arXiv · Cited by 9
“In other fields such as natural language processing [39, 56] and computer vision [44, 30], pre-trained foundation models have shown remarkable performance in such settings with insufficient data.”
InfluentialVideo Diffusion Models: A Survey
Melnik, A., Ljubljanac, M., Lu, C., Yan, Q., Ren, W., & Ritter, H. (2024) · arXiv · doi:10.48550/arxiv.2405.03150 · Cited by 4
Han, Y.-H., Huang, T.-M., Hua, K.-L., & Chen, J.-C. (2024) · arXiv · doi:10.48550/arxiv.2404.05583 · Cited by 45
Engaging Communities Meaningfully in Defining Disability Representation for AI Image Generation
Thieme, A., Faia Marques, R., Grayson, M., Balachandar, S., Tyler Cassidy, C., Choksi, M. Z., Longden, C., Huda, R. S., Ileve Kalovwe, N., Mallon, C., et al. (2026) · Proceedings of the 2026 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3772318.3790768 · Cited by 1
Qu, Y., Huang, Z., Sheng, Z., Chen, J., Leng, Y., Labi, S., & Chen, S. (2025) · arXiv · doi:10.48550/arxiv.2505.16377 · Cited by 1
Back into Plato's Cave: Examining Cross-modal Representational Convergence at Scale
Koepke, A. S., Zverev, D., Ginosar, S., & Efros, A. A. (2026) · arXiv · doi:10.48550/arxiv.2604.18572
“The CLIP model [78] by Radford et al. formed a turning point by demonstrating that contrastive learning on web-scale image-text pairs could produce shared embedding spaces.”
Akdemir, K., & Yanardag, P. (2024) · arXiv · doi:10.48550/arxiv.2406.02820 · Cited by 8
“We measure the normalized cosine similarity between the image and prompt text embedding us-ing CLIP (Radford et al. 2021) in order to evaluate the image prompt similarity.”
MethodKwon, P., & Chen, C. (2025) · arXiv · doi:10.48550/arxiv.2512.01686 · Cited by 1
“We also present the results for DreamBench++ in Table 3, which shows that our model achieves the highest DINO [5] and CLIP-I [42] scores, with CLIP-T [42] scores that are comparable to other methods.”
Ai, Q., Zhan, J., & Liu, Y. (2025) · arXiv · doi:10.48550/arxiv.2501.02842
“In terms of training methods, there are mainly four types of training objectives: • Contrastive Learning or Image-Text matching: These tasks require the model to correctly categorize images and their corresponding textual descriptions, aligning the representations of text and images [61, 63, 64].”
Haresamudram, H., Tang, C. I., Suh, S., Lukowicz, P., & Ploetz, T. (2024) · arXiv · Cited by 31
“Looking beyond works in re-purposing language and vision foundation models for human activity recognition, significant efforts have been dedicated by researchers to developing foundational models for time-series data.”
InfluentialMemVerse: Multimodal Memory for Lifelong Learning Agents
Liu, J., Sun, Y., Cheng, W., Lei, H., Chen, Y., Wen, L., Yang, X., Fu, D., Cai, P., Deng, N., et al. (2025) · arXiv · doi:10.48550/arxiv.2512.03627
SVG: 3D Stereoscopic Video Generation via Denoising Frame Matrix
Dai, P., Tan, F., Xu, Q., Futschik, D., Du, R., Fanello, S., Qi, X., & Zhang, Y. (2024) · arXiv · doi:10.48550/arxiv.2407.00367
Yang, C., & Kim, M. (2026) · Applied Sciences · doi:10.3390/app16073391
Xu, Z., Choi, Y., Yi, C., Park, C., Park, J., Park, H., & Song, S. (2026) · Aerospace · doi:10.3390/aerospace13030222
Multimodal Promptable Token Merging for Diffusion Models
Hong, C.-Y., & Liu, T.-L. (2025) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v39i16.33894
Dai, R., Jian, A., Zhang, R., Jia, Y., Mo, L., Zhan, S., & He, T. (2026) · Knowledge-Based Systems · doi:10.1016/j.knosys.2026.116272
Metadata
- Authors
- Radford, A., Kim, J. W., Hallacy, C., Ramesh, A., Goh, G., Agarwal, S., Sastry, G., Askell, A., Mishkin, P., Clark, J., Krueger, G., & Sutskever, I.
- Affiliations
- OpenAI (United States) (US)
- ORCID
- Kim, Jong Wook
- Venue
- arXiv
- Publisher
- arXiv
- Published
- 2021
- ISSN
- 2331-8422 (ISSN-L)
- arXiv
- 2103.00020
- OpenAlex
- W3135367836
- Cited by
- 15,293
- References
- 193
- Citations by year
- 2026: 214 · 2025: 3988 · 2024: 6413 · 2023: 3389 · 2022: 932 · 2021: 349
- Type
- posted content
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Multimodal Machine Learning Applications
- Physical Sciences › Computer Science › Artificial Intelligence › Domain Adaptation and Few-Shot Learning
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
- Keywords
- natural language supervision, zero-shot transfer, image representation learning, image captioning, visual concept learning, cross-modal learning, human action recognition, geolocation, optical character recognition, CLIP
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
