The DOI matches its DataCite record; 12 of the 12 references it cites were found in a registry or an index.
A Simple Framework for Contrastive Learning of Visual Representations
Open accessAbstract
This paper presents SimCLR: a simple framework for contrastive learning of visual representations. We simplify recently proposed contrastive self-supervised learning algorithms without requiring specialized architectures or a memory bank. In order to understand what enables the contrastive prediction tasks to learn useful representations, we systematically study the major components of our framework. We show that (1) composition of data augmentations plays a critical role in defining effective predictive tasks, (2) introducing a learnable nonlinear transformation between the representation and the contrastive loss substantially improves the quality of the learned representations, and (3) contrastive learning benefits from larger batch sizes and more training steps compared to supervised learning. By combining these findings, we are able to considerably outperform previous methods for self-supervised and semi-supervised learning on ImageNet. A linear classifier trained on self-supervised representations learned by SimCLR achieves 76.5% top-1 accuracy, which is a 7% relative improvement over previous state-of-the-art, matching the performance of a supervised ResNet-50. When fine-tuned on only 1% of the labels, we achieve 85.8% top-5 accuracy, outperforming AlexNet with 100X fewer labels.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Such multi-task objectives may be more effective than single-task ones (e.g., ImageNet classification) due to the fact that they impose more task constraints on the representation, leading to a smaller and higher-quality solution space (Chen et al., 2020;He et al., 2020;Radford et al., 2017;2019).”
From The Platonic Representation Hypothesis (2024) · cites this paper as Chen et al., 2020
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“Similarly to other self-supervised frameworks, using a projection head [12] improves greatly the accuracy of our method.”
From Emerging Properties in Self-Supervised Vision Transformers (2021) · cites this paper as [12]
The cited source says what the claim attributes to it.
In the citing paper
“We note that this is a common setting for self-supervised contrastive learners with NCE objectives (Gutmann & Hyvärinen, 2010;Oord et al., 2018), including SimCLR (Chen et al., 2020) and SimCSE (Gao et al., 2021).”
From The Platonic Representation Hypothesis (2024) · cites this paper as Chen et al., 2020
The cited source says what the claim attributes to it.
In the citing paper
“Since the choice of augmentation function encodes the learning objective and invariance that the model should capture, the original SimCLR framework demonstrates significant performance differences among the choices of augmentation functions [28].”
From Past, Present, and Future of Sensor-based Human Activity Recognition Using Wearables: A Surveying Tutorial on a Still Challenging Task (2025) · cites this paper as [28]
The cited source says what the claim attributes to it.
In the citing paper
“This framework has resulted in excellent representation learning performance across domains such as computer vision [12, 33, 35], natural language [49], speech recognition [6, 58] and time-series data [39].”
From Contrastive Predictive Coding for Human Activity Recognition (2021) · cites this paper as [12]
The cited source says what the claim attributes to it.
In the citing paper
“Powerful self-supervised learning techniques such as DINO (Caron et al., 2021) and SimCLR (Chen et al., 2020) have now also enabled the training of these feature encoders without any labels, further increasing the attractiveness of this approach.”
From Navigating the landscape of multimodal AI in medicine: A scoping review on technical challenges and clinical applications (2025) · cites this paper as Chen et al., 2020
The cited source says what the claim attributes to it.
In the citing paper
“As mentioned in the original paper [6], the choice of augmentations has a large impact on the pre-training quality.”
From Assessing the State of Self-Supervised Human Activity Recognition Using Wearables (2022) · cites this paper as [6]
The cited source says what the claim attributes to it.
In the citing paper
“existing self-supervised learning approaches [1, 2, 3, 4, 5, 6] are mainly designed for natural images.”
From SatMAE: Pre-Training Transformers for Temporal and Multi-Spectral Satellite Imagery (2022) · cites this paper as [4]
The cited source says what the claim attributes to it.
In the citing paper
“The OOD detection method CSI is based on contrastive learning [12, 29], data and class augmentations, and results ensembling [64].”
From Open-world continual learning: Unifying novelty detection and continual learning (2025) · cites this paper as [12]
In the citing paper
“For example, the recently proposed SimCLR method defines a context-based contrasting task for self-supervised learning [34], obtaining comparable performance as fully supervised models.”
From Empowering things with intelligence: a survey of the progress, challenges, and opportunities in artificial intelligence of things (2021) · cites this paper as [34]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474
ImageNet classification with deep convolutional neural networks
Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017) · Communications of the ACM · doi:10.1145/3065386 · Cited by 112,155
Self-citationImageNet Large Scale Visual Recognition Challenge
Russakovsky, O., Deng, J., Su, H., Krause, J., Satheesh, S., Ma, S., Huang, Z., Karpathy, A., Khosla, A., Bernstein, M., et al. (2015) · International Journal of Computer Vision · doi:10.1007/s11263-015-0816-y · Cited by 39,380
Efficient Estimation of Word Representations in Vector Space
Mikolov, T., Chen, K., Corrado, G., & Dean, J. (2013) · arXiv · doi:10.48550/arxiv.1301.3781 · Cited by 26,568
Auto-Encoding Variational Bayes
Kingma, D. P., & Welling, M. (2013) · arXiv · doi:10.48550/arxiv.1312.6114 · Cited by 20,067
FaceNet: A unified embedding for face recognition and clustering
Schroff, F., Kalenichenko, D., & Philbin, J. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298682 · Cited by 15,267
Representation Learning with Contrastive Predictive Coding
Oord, A. V. D., Li, Y., & Vinyals, O. (2018) · arXiv · doi:10.48550/arxiv.1807.03748 · Cited by 6,242
DeCAF: A Deep Convolutional Activation Feature for Generic Visual Recognition
Donahue, J., Jia, Y., Vinyals, O., Hoffman, J., Zhang, N., Tzeng, E., & Darrell, T. (2013) · arXiv · doi:10.48550/arxiv.1310.1531 · Cited by 4,019
Unsupervised Feature Learning via Non-parametric Instance Discrimination
Wu, Z., Xiong, Y., Yu, S. X., & Lin, D. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00393 · Cited by 3,664
Automated Flower Classification over a Large Number of Classes
Nilsback, M.-E., & Zisserman, A. (2008) · 2008 Sixth Indian Conference on Computer Vision, Graphics & Image Processing · doi:10.1109/icvgip.2008.47 · Cited by 3,442
Research works printed but not linked: 48
- Asano, Y. M., Rupprecht, C., and Vedaldi, A. A critical analysis of self-supervision, or what we can learn from a single image. arXiv preprint arXiv:1904.13132 , 2019. · arXiv
- Bachman, P., Hjelm, R. D., and Buchwalter, W. Learning representations by maximizing mutual information across views. In Advances in Neural Information Processing Systems , pp. 15509-15519, 2019.
- Becker, S. and Hinton, G. E. Self-organizing neural network that discovers surfaces in random-dot stereograms. Nature , 355 (6356):161-163, 1992.
- Berg, T., Liu, J., Lee, S. W., Alexander, M. L., Jacobs, D. W., and Belhumeur, P. N. Birdsnap: Large-scale fine-grained visual categorization of birds. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR) , pp. 2019-2026. IEEE, 2014.
- Berthelot, D., Carlini, N., Goodfellow, I., Papernot, N., Oliver, A., and Raffel, C. A. Mixmatch: A holistic approach to semisupervised learning. In Advances in Neural Information Processing Systems , pp. 5050-5060, 2019.
- Bossard, L., Guillaumin, M., and Van Gool, L. Food-101-mining discriminative components with random forests. In European conference on computer vision , pp. 446-461. Springer, 2014.
- Chen, T., Sun, Y., Shi, Y., and Hong, L. On sampling strategies for neural network-based collaborative filtering. In Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining , pp. 767-776, 2017.
- Chen, T., Zhai, X., Ritter, M., Lucic, M., and Houlsby, N. Selfsupervised gans via auxiliary rotation loss. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition , pp. 12154-12163, 2019.
- Cubuk, E. D., Zoph, B., Mane, D., Vasudevan, V., and Le, Q. V. Autoaugment: Learning augmentation strategies from data. In Proceedings of the IEEE conference on computer vision and pattern recognition , pp. 113-123, 2019.
- DeVries, T. and Taylor, G. W. Improved regularization of convolutional neural networks with cutout. arXiv preprint arXiv:1708.04552 , 2017. · arXiv
- Doersch, C. and Zisserman, A. Multi-task self-supervised visual learning. In Proceedings of the IEEE International Conference on Computer Vision , pp. 2051-2060, 2017.
- Doersch, C., Gupta, A., and Efros, A. A. Unsupervised visual representation learning by context prediction. In Proceedings of the IEEE International Conference on Computer Vision , pp. 1422-1430, 2015.
- Donahue, J. and Simonyan, K. Large scale adversarial representation learning. In Advances in Neural Information Processing Systems , pp. 10541-10551, 2019.
- Dosovitskiy, A., Springenberg, J. T., Riedmiller, M., and Brox, T. Discriminative unsupervised feature learning with convolutional neural networks. In Advances in neural information processing systems , pp. 766-774, 2014.
- Everingham, M., Van Gool, L., Williams, C. K., Winn, J., and Zisserman, A. The pascal visual object classes (voc) challenge. International Journal of Computer Vision , 88(2):303-338, 2010.
- Fei-Fei, L., Fergus, R., and Perona, P. Learning generative visual models from few training examples: An incremental bayesian approach tested on 101 object categories. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR) Workshop on Generative-Model Based Vision , 2004.
- Gidaris, S., Singh, P., and Komodakis, N. Unsupervised representation learning by predicting image rotations. arXiv preprint arXiv:1803.07728 , 2018. · arXiv
- Goodfellow, I., Pouget-Abadie, J., Mirza, M., Xu, B., WardeFarley, D., Ozair, S., Courville, A., and Bengio, Y. Generative adversarial nets. In Advances in neural information processing systems , pp. 2672-2680, 2014.
- Goyal, P., Dollár, P., Girshick, R., Noordhuis, P., Wesolowski, L., Kyrola, A., Tulloch, A., Jia, Y., and He, K. Accurate, large minibatch sgd: Training imagenet in 1 hour. arXiv preprint arXiv:1706.02677 , 2017. · arXiv
- Hadsell, R., Chopra, S., and LeCun, Y. Dimensionality reduction by learning an invariant mapping. In 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06) , volume 2, pp. 1735-1742. IEEE, 2006.
- He, K., Fan, H., Wu, Y., Xie, S., and Girshick, R. Momentum contrast for unsupervised visual representation learning. arXiv preprint arXiv:1911.05722 , 2019. · arXiv
- Hénaff, O. J., Razavi, A., Doersch, C., Eslami, S., and Oord, A. v. d. Data-efficient image recognition with contrastive predictive coding. arXiv preprint arXiv:1905.09272 , 2019. · arXiv
- Hinton, G. E., Osindero, S., and Teh, Y.-W. A fast learning algorithm for deep belief nets. Neural computation , 18(7):15271554, 2006.
- Hjelm, R. D., Fedorov, A., Lavoie-Marchildon, S., Grewal, K., Bachman, P., Trischler, A., and Bengio, Y. Learning deep representations by mutual information estimation and maximization. arXiv preprint arXiv:1808.06670 , 2018. · arXiv
- Howard, A. G. Some improvements on deep convolutional neural network based image classification. arXiv preprint arXiv:1312.5402 , 2013. · arXiv
- Ioffe, S. and Szegedy, C. Batch normalization: Accelerating deep network training by reducing internal covariate shift. arXiv preprint arXiv:1502.03167 , 2015. · arXiv
- Ji, X., Henriques, J. F., and Vedaldi, A. Invariant information clustering for unsupervised image classification and segmentation. In Proceedings of the IEEE International Conference on Computer Vision , pp. 9865-9874, 2019.
- Kolesnikov, A., Zhai, X., and Beyer, L. Revisiting self-supervised visual representation learning. In Proceedings of the IEEE conference on Computer Vision and Pattern Recognition , pp. 1920-1929, 2019.
- Kornblith, S., Shlens, J., and Le, Q. V . Do better ImageNet models transfer better? In Proceedings of the IEEE conference on computer vision and pattern recognition , pp. 2661-2671, 2019.
- Krause, J., Deng, J., Stark, M., and Fei-Fei, L. Collecting a large-scale dataset of fine-grained cars. In Second Workshop on Fine-Grained Visual Categorization , 2013.
- Loshchilov, I. and Hutter, F. Sgdr: Stochastic gradient descent with warm restarts. arXiv preprint arXiv:1608.03983 , 2016. · arXiv
- Maaten, L. v. d. and Hinton, G. Visualizing data using t-sne. Journal of machine learning research , 9(Nov):2579-2605, 2008.
- Misra, I. and van der Maaten, L. Self-supervised learning of pretext-invariant representations. arXiv preprint arXiv:1912.01991 , 2019. · arXiv
- Noroozi, M. and Favaro, P. Unsupervised learning of visual representations by solving jigsaw puzzles. In European Conference on Computer Vision , pp. 69-84. Springer, 2016.
- Parkhi, O. M., Vedaldi, A., Zisserman, A., and Jawahar, C. Cats and dogs. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR) , pp. 3498-3505. IEEE, 2012.
- Simonyan, K. and Zisserman, A. Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556 , 2014. · arXiv
- Sohn, K. Improved deep metric learning with multi-class n-pair loss objective. In Advances in neural information processing systems , pp. 1857-1865, 2016.
- Sohn, K., Berthelot, D., Li, C.-L., Zhang, Z., Carlini, N., Cubuk, E. D., Kurakin, A., Zhang, H., and Raffel, C. Fixmatch: Simplifying semi-supervised learning with consistency and confidence. arXiv preprint arXiv:2001.07685 , 2020. · arXiv
- Szegedy, C., Liu, W., Jia, Y ., Sermanet, P., Reed, S., Anguelov, D., Erhan, D., Vanhoucke, V., and Rabinovich, A. Going deeper with convolutions. In Proceedings of the IEEE conference on computer vision and pattern recognition , pp. 1-9, 2015.
- Tian, Y., Krishnan, D., and Isola, P. Contrastive multiview coding. arXiv preprint arXiv:1906.05849 , 2019. · arXiv
- Tschannen, M., Djolonga, J., Rubenstein, P. K., Gelly, S., and Lucic, M. On mutual information maximization for representation learning. arXiv preprint arXiv:1907.13625 , 2019. · arXiv
- Xiao, J., Hays, J., Ehinger, K. A., Oliva, A., and Torralba, A. Sun database: Large-scale scene recognition from abbey to zoo. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR) , pp. 3485-3492. IEEE, 2010.
- Xie, Q., Dai, Z., Hovy, E., Luong, M.-T., and Le, Q. V. Unsupervised data augmentation. arXiv preprint arXiv:1904.12848 , 2019. · arXiv
- Ye, M., Zhang, X., Yuen, P. C., and Chang, S.-F. Unsupervised embedding learning via invariant and spreading instance feature. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition , pp. 6210-6219, 2019.
- You, Y., Gitman, I., and Ginsburg, B. Large batch training of convolutional networks. arXiv preprint arXiv:1708.03888 , 2017. · arXiv
- Zhai, X., Oliver, A., Kolesnikov, A., and Beyer, L. S4l: Selfsupervised semi-supervised learning. In The IEEE International Conference on Computer Vision (ICCV) , October 2019.
- Zhang, R., Isola, P., and Efros, A. A. Colorful image colorization. In European conference on computer vision , pp. 649-666. Springer, 2016.
- Zhuang, C., Zhai, A. L., and Yamins, D. Local aggregation for unsupervised learning of visual embeddings. In Proceedings of the IEEE International Conference on Computer Vision , pp. 6002-6012, 2019.
Cited by
Sorted by citations
Items 101–189 of 189
Tian, Y., Pang, G., Liu, F., Chen, Y., Shin, S. H., Verjans, J. W., Singh, R., & Carneiro, G. (2021) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-87240-3_13 · Cited by 77
ContIG: Self-supervised Multimodal Contrastive Learning for Medical Imaging with Genetics
Taleb, A., Kirchler, M., Monti, R., & Lippert, C. (2022) · 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52688.2022.02024 · Cited by 83
Guha Roy, A., Ren, J., Azizi, S., Loh, A., Natarajan, V., Mustafa, B., Pawlowski, N., Freyberg, J., Liu, Y., Beaver, Z., et al. (2022) · Medical Image Analysis · doi:10.1016/j.media.2021.102274 · Cited by 77
Longitudinal self-supervised learning
Zhao, Q., Liu, Z., Adeli, E., & Pohl, K. M. (2021) · Medical Image Analysis · doi:10.1016/j.media.2021.102051 · Cited by 71
Subject-Aware Contrastive Learning for Biosignals
Cheng, J. Y., Goh, H., Dogrusoz, K., Tuzel, O., & Azemi, E. (2020) · arXiv · doi:10.48550/arxiv.2007.04871 · Cited by 74
Multiple Instance Captioning: Learning Representations from Histopathology Textbooks and Articles
Gamper, J., & Rajpoot, N. (2021) · 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr46437.2021.01628 · Cited by 75
Contrastive Learning Reduces Hallucination in Conversations
Sun, W., Shi, Z., Gao, S., Ren, P., De Rijke, M., & Ren, Z. (2023) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v37i11.26596 · Cited by 69
Zheng, Y., Wang, X., Deng, R., Bao, T., Zhao, R., & Wu, L. (2022) · 2022 IEEE International Conference on Multimedia and Expo (ICME) · doi:10.1109/icme52920.2022.9859925 · Cited by 80
Solving Inefficiency of Self-supervised Representation Learning
Wang, G., Wang, K., Wang, G., Torr, P. H. S., & Lin, L. (2021) · 2021 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00937 · Cited by 65
Supervised Contrastive Learning for Generalizable and Explainable DeepFakes Detection
Xu, Y., Raja, K., & Pedersen, M. (2022) · 2022 IEEE/CVF Winter Conference on Applications of Computer Vision Workshops (WACVW) · doi:10.1109/wacvw54805.2022.00044 · Cited by 62
Radar Camera Fusion via Representation Learning in Autonomous Driving
Dong, X., Zhuang, B., Mao, Y., & Liu, L. (2021) · 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw53098.2021.00183 · Cited by 62
SkeletonMAE: Spatial-Temporal Masked Autoencoders for Self-Supervised Skeleton Action Recognition
Wu, W., Hua, Y., Zheng, C., Wu, S., Chen, C., & Lu, A. (2023) · 2023 IEEE International Conference on Multimedia and Expo Workshops (ICMEW) · doi:10.1109/icmew59549.2023.00045 · Cited by 60
“More self-supervised representation learning approaches adopt the so-called contrastive learning manner [13, 14] to boost their performance.”
BackgroundZero Experience Required: Plug & Play Modular Transfer Learning for Semantic Visual Navigation
Al-Halah, Z., Ramakrishnan, S. K., & Grauman, K. (2022) · 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52688.2022.01652 · Cited by 58
Cheng, D., Zhang, L., Bu, C., Wu, H., & Song, A. (2023) · Knowledge-Based Systems · doi:10.1016/j.knosys.2023.110789 · Cited by 55
Product1M: Towards Weakly Supervised Instance-Level Product Retrieval via Cross-Modal Pretraining
Zhan, X., Wu, Y., Dong, X., Wei, Y., Lu, M., Zhang, Y., Xu, H., & Liang, X. (2021) · 2021 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.01157 · Cited by 54
Deep Self-Supervised Representation Learning for Free-Hand Sketch
Xu, P., Song, Z., Yin, Q., Song, Y.-Z., & Wang, L. (2021) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2020.3003048 · Cited by 53
Deep learning methods for protein function prediction
Boadu, F., Lee, A., & Cheng, J. (2025) · PROTEOMICS · doi:10.1002/pmic.202300471 · Cited by 53
A Survey on Contrastive Self-Supervised Learning
Jaiswal, A., Babu, A. R., Zadeh, M. Z., Banerjee, D., & Makedon, F. (2020) · Technologies · doi:10.3390/technologies9010002 · Cited by 154
An Information-Theoretic Perspective on Intrinsic Motivation in Reinforcement Learning: A Survey
Aubret, A., Matignon, L., & Hassas, S. (2023) · Entropy · doi:10.3390/e25020327 · Cited by 52
Negative Selection by Clustering for Contrastive Learning in Human Activity Recognition
Wang, J., Zhu, T., Chen, L. L., Ning, H., & Wan, Y. (2023) · IEEE Internet of Things Journal · doi:10.1109/jiot.2023.3239945 · Cited by 52
Mining for Strong Gravitational Lenses with Self-supervised Learning
Stein, G., Blaum, J., Harrington, P., Medan, T., & Lukić, Z. (2022) · The Astrophysical Journal · doi:10.3847/1538-4357/ac6d63 · Cited by 52
An Improved Inter-Intra Contrastive Learning Framework on Self-Supervised Video Representation
Tao, L., Wang, X., & Yamasaki, T. (2022) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2022.3141051 · Cited by 50
Yang, P., Yin, X., Lu, H., Hu, Z., Zhang, X., Jiang, R., & Lv, H. (2022) · Medical Image Analysis · doi:10.1016/j.media.2022.102539 · Cited by 50
Where are we in the search for an Artificial Visual Cortex for Embodied Intelligence?
Majumdar, A., Yadav, K., Arnaud, S., Ma, Y. J., Chen, C., Silwal, S., Jain, A., Berges, V.-P., Abbeel, P., Malik, J., et al. (2023) · arXiv · doi:10.48550/arxiv.2303.18240 · Cited by 49
“These algorithms use contrastive [21, 22], distillation-based [19, 20], or reconstructive [18, 23] objectives for training.”
Self-supervised Visual Representation Learning for Histopathological Images
Yang, P., Hong, Z., Yin, X., Zhu, C., & Jiang, R. (2021) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-87196-3_5 · Cited by 56
ConDA: Contrastive Domain Adaptation for AI-generated Text Detection
Bhattacharjee, A., Kumarage, T., Moraffah, R., & Liu, H. (2023) · Proceedings of the 13th International Joint Conference on Natural Language Processing and the 3rd Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/2023.ijcnlp-main.40 · Cited by 43
Zheng, Y., Conrad, R. D., Green, E. J., Burks, E. J., Betke, M., Beane, J. E., & Kolachalama, V. B. (2024) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2024.3386108 · Cited by 44
“After training, we used the embedding vectors for graph construction. patch through a convolutional neural network (CNN) that has been trained using contrastive learning [13] (Fig.”
MethodSelf-Supervised Wasserstein Pseudo-Labeling for Semi-Supervised Image Classification
Taherkhani, F., Dabouei, A., Soleymani, S., Dawson, J., & Nasrabadi, N. M. (2021) · 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr46437.2021.01209 · Cited by 45
Breaking away from labels: The promise of self-supervised machine learning in intelligent health
Spathis, D., Perez-Pozuelo, I., Marques-Fernandez, L., & Mascolo, C. (2022) · Patterns · doi:10.1016/j.patter.2021.100410 · Cited by 39
Lin, Z., Zhang, D., Shi, D., Xu, R., Tao, Q., Wu, L., He, M., & Ge, Z. (2023) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2023.104281 · Cited by 39
Continual Learning for Generative Retrieval over Dynamic Corpora
Chen, J., Zhang, R., Guo, J., de Rijke, M., Chen, W., Fan, Y., & Cheng, X. (2023) · Proceedings of the 32nd ACM International Conference on Information and Knowledge Management · doi:10.1145/3583780.3614821 · Cited by 40
Khosravi, P., Fuchs, T. J., & Ho, D. J. (2025) · Cancer Research · doi:10.1158/0008-5472.can-24-3630 · Cited by 41
Self-Supervised Learning for Anomaly Detection With Dynamic Local Augmentation
Yoa, S., Lee, S., Kim, C., & Kim, H. J. (2021) · IEEE Access · doi:10.1109/access.2021.3124525 · Cited by 38
COOKIE: Contrastive Cross-Modal Knowledge Sharing Pre-training for Vision-Language Representation
Wen, K., Xia, J., Huang, Y., Li, L., Xu, J., & Shao, J. (2021) · 2021 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00221 · Cited by 37
Improved Contrastive Divergence Training of Energy Based Models
Du, Y., Li, S., Tenenbaum, J., & Mordatch, I. (2020) · arXiv · doi:10.48550/arxiv.2012.01316 · Cited by 36
DFIL: Deepfake Incremental Learning by Exploiting Domain-invariant Forgery Clues
Pan, K., Yin, Y., Wei, Y., Lin, F., Ba, Z., Liu, Z., Wang, Z., Cavallaro, L., & Ren, K. (2023) · Proceedings of the 31st ACM International Conference on Multimedia · doi:10.1145/3581783.3612377 · Cited by 37
Cai, L., Huang, S., Zhang, Y., Lu, J., & Zhang, Y. (2025) · Medical Image Analysis · doi:10.1016/j.media.2025.103631 · Cited by 33
SimTriplet: Simple Triplet Representation Learning with a Single GPU
Liu, Q., Louis, P. C., Lu, Y., Jha, A., Zhao, M., Deng, R., Yao, T., Roland, J. T., Yang, H., Zhao, S., et al. (2021) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-87196-3_10 · Cited by 38
Human Hands as Probes for Interactive Object Understanding
Goyal, M., Modi, S., Goyal, R., & Gupta, S. (2022) · 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52688.2022.00329 · Cited by 31
Giga-SSL: Self-Supervised Learning for Gigapixel Images
Lazard, T., Lerousseau, M., Decencière, E., & Walter, T. (2023) · 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw59228.2023.00453 · Cited by 30
DeAR: Debiasing Vision-Language Models with Additive Residuals
Seth, A., Hemani, M., & Agarwal, C. (2023) · 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52729.2023.00659 · Cited by 34
Pre-training in Medical Data: A Survey
Qiu, Y., Lin, F., Chen, W., & Xu, M. (2023) · Machine Intelligence Research · doi:10.1007/s11633-022-1382-8 · Cited by 30
The Platonic Representation Hypothesis
Huh, M., Cheung, B., Wang, T., & Isola, P. (2024) · arXiv · doi:10.48550/arxiv.2405.07987 · Cited by 35
“We note that this is a common setting for self-supervised contrastive learners with NCE objectives (Gutmann & Hyv¨arinen, 2010; Oord et al., 2018), including SimCLR (Chen et al., 2020) and SimCSE (Gao et al., 2021).”
Learning cortical representations through perturbed and adversarial dreaming
Deperrois, N., Petrovici, M. A., Senn, W., & Jordan, J. (2022) · eLife · doi:10.7554/elife.76384 · Cited by 28
Contrasting Contrastive Self-Supervised Representation Learning Pipelines
Kotar, K., Ilharco, G., Schmidt, L., Ehsani, K., & Mottaghi, R. (2021) · 2021 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00980 · Cited by 28
Deep Active Learning for Computer Vision: Past and Future
Takezoe, R., Liu, X., Mao, S., Chen, M. T., Feng, Z., Zhang, S., & Wang, X. (2023) · APSIPA Transactions on Signal and Information Processing · doi:10.1561/116.00000057 · Cited by 26
Investigating Enhancements to Contrastive Predictive Coding for Human Activity Recognition
Haresamudram, H., Essa, I., & Plötz, T. (2023) · 2023 IEEE International Conference on Pervasive Computing and Communications (PerCom) · doi:10.1109/percom56429.2023.10099197 · Cited by 27
Enriched music representations with multiple cross-modal contrastive learning
Ferraro, A., Favory, X., Drossos, K., Kim, Y., & Bogdanov, D. (2021) · IEEE Signal Processing Letters · doi:10.1109/lsp.2021.3071082 · Cited by 23
VideoMAC: Video Masked Autoencoders Meet ConvNets
Pei, G., Chen, T., Jiang, X., Liu, H., Sun, Z., & Yao, Y. (2024) · 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52733.2024.02145 · Cited by 23
Self-citationTurgut, Ö., Müller, P., Hager, P., Shit, S., Starck, S., Menten, M. J., Martens, E., & Rueckert, D. (2025) · Medical Image Analysis · doi:10.1016/j.media.2024.103451 · Cited by 22
Xing, S., Wang, Y., & Liu, W. (2025) · Sensors · doi:10.3390/s25113396 · Cited by 23
COALA: Co-Aligned Autoencoders for Learning Semantically Enriched Audio Representations
Favory, X., Drossos, K., Virtanen, T., & Serra, X. (2020) · arXiv · doi:10.48550/arxiv.2006.08386 · Cited by 20
Xie, R., Pang, K., Bader, G. D., & Wang, B. (2023) · 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52729.2023.00321 · Cited by 17
Kadi, H. A., & Terzić, K. (2023) · Sensors · doi:10.3390/s23052389 · Cited by 27
Open-world continual learning: Unifying novelty detection and continual learning
Kim, G., Xiao, C., Konishi, T., Ke, Z., & Liu, B. (2025) · Artificial Intelligence · doi:10.1016/j.artint.2024.104237 · Cited by 19
SSLP: Spatial Guided Self-supervised Learning on Pathological Images
Li, J., Lin, T., & Xu, Y. (2021) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-87196-3_1 · Cited by 26
Text Transformations in Contrastive Self-Supervised Learning: A Review
Bhattacharjee, A., Karami, M., & Liu, H. (2022) · Proceedings of the Thirty-First International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2022/757 · Cited by 20
IMU2CLIP: Language-grounded Motion Sensor Translation with Multimodal Contrastive Learning
Moon, S., Madotto, A., Lin, Z., Saraf, A., Bearman, A., & Damavandi, B. (2023) · Findings of the Association for Computational Linguistics: EMNLP 2023 · doi:10.18653/v1/2023.findings-emnlp.883 · Cited by 18
Abstraction for Deep Reinforcement Learning
Shanahan, M., & Mitchell, M. (2022) · Proceedings of the Thirty-First International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2022/780 · Cited by 17
If only we had more data!: Sensor-Based Human Activity Recognition in Challenging Scenarios
Plötz, T. (2023) · 2023 IEEE International Conference on Pervasive Computing and Communications Workshops and other Affiliated Events (PerCom Workshops) · doi:10.1109/percomworkshops56833.2023.10150267 · Cited by 16
What Do Self-Supervised Vision Transformers Learn?
Park, N., Kim, W., Heo, B., Kim, T., & Yun, S. (2023) · arXiv · doi:10.48550/arxiv.2305.00729 · Cited by 29
“Negative samples is an effective technique to avoid the collapsing problems, but they cause dimensional collapse (Jing et al., 2022) and require extra large batches (Chen et al., 2020a) or memory queues (He et al., 2020; Chen et al., 2020b) to retrieve them.”
BackgroundMethodImproving Representation Learning for Histopathologic Images with Cluster Constraints
Wu, W., Gao, C., DiPalma, J., Vosoughi, S., & Hassanpour, S. (2023) · 2023 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.01957 · Cited by 16
Self-Supervised Video GANs: Learning for Appearance Consistency and Motion Coherency
Hyun, S., Kim, J., & Heo, J.-P. (2021) · 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr46437.2021.01068 · Cited by 16
Gender-tuning: Empowering Fine-tuning for Debiasing Pre-trained Language Models
Ghanbarzadeh, S., Huang, Y., Palangi, H., Cruz Moreno, R., & Khanpour, H. (2023) · Findings of the Association for Computational Linguistics: ACL 2023 · doi:10.18653/v1/2023.findings-acl.336 · Cited by 17
Detecting galaxy tidal features using self-supervised representation learning
Desmons, A., Brough, S., & Lanusse, F. (2024) · Monthly Notices of the Royal Astronomical Society · doi:10.1093/mnras/stae1402 · Cited by 15
Self-supervised Regularization for Text Classification
Zhou, M., Li, Z., & Xie, P. (2021) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00389 · Cited by 14
Haresamudram, H., Essa, I., & Plötz, T. (2024) · Sensors · doi:10.3390/s24041238 · Cited by 14
Beyond Medical Imaging - A Review of Multimodal Deep Learning in Radiology
Heiliger, L., Sekuboyina, A., Menze, B., Egger, J., & Kleesiek, J. (2022) · doi:10.36227/techrxiv.19103432 · Cited by 13
Self-supervised similarity search for large scientific datasets
Stein, G., Harrington, P., Blaum, J., Medan, T., & Lukic, Z. (2021) · arXiv · doi:10.48550/arxiv.2110.13151 · Cited by 13
Case Discrimination: Self-supervised Feature Learning for the Classification of Focal Liver Lesions
Dong, H., Iwamoto, Y., Han, X., Lin, L., Hu, H., Cai, X., & Chen, Y.-W. (2021) · Smart Innovation, Systems and Technologies · doi:10.1007/978-981-16-3013-2_20 · Cited by 12
Learning beyond sensations: How dreams organize neuronal representations
Deperrois, N., Petrovici, M. A., Senn, W., & Jordan, J. (2024) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2023.105508 · Cited by 9
Detecting Online Grooming By Simple Contrastive Chat Embeddings
Rezaee Borj, P., Raja, K., & Bours, P. (2023) · Proceedings of the 9th ACM International Workshop on Security and Privacy Analytics · doi:10.1145/3579987.3586564 · Cited by 11
Enhancing the Robustness of AI-Generated Text Detectors: A Survey
Liu, X., Li, Y., & Li, K. (2025) · Mathematics · doi:10.3390/math13132145 · Cited by 10
Visual information processing through the interplay between fine and coarse signal pathways
Zou, X., Ji, Z., Zhang, T., Huang, T., & Wu, S. (2023) · Neural Networks · doi:10.1016/j.neunet.2023.07.048 · Cited by 9
Kazi, S., Khoja, S., & Daud, A. (2025) · Computer Science Review · doi:10.1016/j.cosrev.2025.100756 · Cited by 9
Li, Z., Li, H., Ralescu, A. L., Dillman, J. R., Altaye, M., Cecil, K. M., Parikh, N. A., & He, L. (2024) · Artificial Intelligence in Medicine · doi:10.1016/j.artmed.2024.102993 · Cited by 10
Towards Attention-based Contrastive Learning for Audio Spoof Detection
Goel, C., Koppisetti, S., Colman, B., Shahriyari, A., & Bharaj, G. (2023) · INTERSPEECH 2023 · doi:10.21437/interspeech.2023-245 · Cited by 7
Value-Consistent Representation Learning for Data-Efficient Reinforcement Learning
Yue, Y., Kang, B., Xu, Z., Huang, G., & Yan, S. (2022) · arXiv · doi:10.48550/arxiv.2206.12542 · Cited by 4
WISE: Rethinking the Knowledge Memory for Lifelong Model Editing of Large Language Models
Wang, P., Li, Z., Zhang, N., Xu, Z., Yao, Y., Jiang, Y., Xie, P., Huang, F., & Chen, H. (2024) · arXiv · doi:10.48550/arxiv.2405.14768 · Cited by 19
Bi, S., Hu, Z., Zhang, H., Di, J., & Sun, Z. (2024) · Neural Networks · doi:10.1016/j.neunet.2024.106578 · Cited by 3
Video Diffusion Models: A Survey
Melnik, A., Ljubljanac, M., Lu, C., Yan, Q., Ren, W., & Ritter, H. (2024) · arXiv · doi:10.48550/arxiv.2405.03150 · Cited by 4
Oishi, N., Birch, P., Lago, P., & Roggen, D. (2026) · Knowledge-Based Systems · doi:10.1016/j.knosys.2026.116367 · Cited by 1
Prasha, A. A., Rachmadi, C. O., Syahlan, M. F. I., Anugerah, N. R., Raditya, N. G., Amelia, P., Mutiara, S. L., & Ramadhan, H. S. (2025) · arXiv · doi:10.48550/arxiv.2512.06642
“More recently, Alexander et al. (2023) and collaborators have explored self-supervised contrastive learning (in particular, a SimCLR / SimSiam framework (Chen et al., 2020)) on lensing images to learn representations that transfer to multiple tasks.”
ReasoningBank: Scaling Agent Self-Evolving with Reasoning Memory
Ouyang, S., Yan, J., Hsu, I.-H., Chen, Y., Jiang, K., Wang, Z., Han, R., Le, L. T., Daruki, S., Tang, X., et al. (2025) · arXiv · doi:10.48550/arxiv.2509.25140 · Cited by 295
“By comparing and contrasting ( self-contrast (Chen et al., 2020)) across different trajectories , the agent can identify consistent reasoning patterns while filtering out spurious solutions.”
Haresamudram, H., Tang, C. I., Suh, S., Lukowicz, P., & Ploetz, T. (2024) · arXiv · Cited by 31
Investigating Enhancements to Contrastive Predictive Coding for Human Activity Recognition
Haresamudram, H., Essa, I., & Ploetz, T. (2022) · arXiv · Cited by 27
“In scenarios where fine-grained activities need to be recognized, both CPC and enhanced CPC perform poorly compared to techniques such as SimCLR and Autoencoder.”
BackgroundMethodInfluentialUnbiased Embodied Visual Representation Learning with Causal Inference and Cross-Modality Alignment
Kang, J., Chen, B., Zhong, P., Wang, Y., Yang, H., & Sheng, Y. (2025) · ACM Transactions on Multimedia Computing, Communications, and Applications · doi:10.1145/3760261
From Illusion to Insight: A Taxonomic Survey of Hallucination Mitigation Techniques in LLMs
Kazlaris, I., Antoniou, E., Diamantaras, K., & Bratsas, C. (2025) · doi:10.20944/preprints202508.1942.v1 · Cited by 13
Enhancing Vision Foundation Models via Multimodal Continual Pre-Training
Chen, Y., Meng, L., Peng, W., Tao, J., Xu, C., Wu, Z., & Jiang, Y.-G. (2025) · arXiv · doi:10.48550/arxiv.2503.18931
Metadata
- Authors
- Chen, T., Kornblith, S., Norouzi, M., & Hinton, G.
- Affiliations
- Google (United States) (US)
- Venue
- arXiv
- Publisher
- arXiv
- Published
- 2020
- ISSN
- 2331-8422 (ISSN-L)
- arXiv
- 2002.05709
- OpenAlex
- W3005680577
- Cited by
- 10,857
- References
- 12
- Citations by year
- 2026: 80 · 2025: 1296 · 2024: 2446 · 2023: 2914 · 2022: 2085 · 2021: 1588 · 2020: 439
- Type
- posted content
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Domain Adaptation and Few-Shot Learning
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
- Physical Sciences › Computer Science › Artificial Intelligence › Machine Learning and Algorithms
- Keywords
- contrastive learning, self-supervised learning, visual representation learning, SimCLR, batch size, semi-supervised learning, ImageNet
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
