Self-Supervised Visual Feature Learning With Deep Neural Networks: A Survey
Open accessThe DOI matches its Crossref record; 204 of the 206 references it cites were found in a registry or an index.
Abstract
Large-scale labeled data are generally required to train deep neural networks in order to obtain better performance in visual feature learning from images or videos for computer vision applications. To avoid extensive cost of collecting and annotating large-scale datasets, as a subset of unsupervised learning methods, self-supervised learning methods are proposed to learn general image and video features from large-scale unlabeled data without using any human-annotated labels. This paper provides an extensive review of deep learning-based self-supervised general visual feature learning methods from images or videos. First, the motivation, general pipeline, and terminologies of this field are described. Then the common deep neural network architectures that used for self-supervised learning are summarized. Next, the schema and evaluation metrics of self-supervised learning methods are reviewed followed by the commonly used datasets for images, videos, audios, and 3D data, as well as the existing self-supervised visual feature learning methods. Finally, quantitative performance comparisons of the reviewed methods on benchmark datasets are summarized and discussed for both image and video feature learning. At last, this paper is concluded and lists a set of promising future directions for self-supervised visual feature learning.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Jing and Tian [18] summarize the self-supervised feature learning methods on image and video data, and four categories are discussed: generation-based, context-based, free semantic label-based, and cross modal-based, where cross-modal-based methods construct learning task using RGB frame sequence an optical flow sequence, which are unique features in the video.”
From Self-Supervised Learning for Time Series Analysis: Taxonomy, Progress, and Prospects (2024) · cites this paper as [18]
In the citing paper
“Another hot research sub-area is self-supervised learning, which learns discriminative visual representation by solving predefined pretext tasks [194].”
From Empowering things with intelligence: a survey of the progress, challenges, and opportunities in artificial intelligence of things (2021) · cites this paper as [194]
Citations
Cites
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474
Very Deep Convolutional Networks for Large-Scale Image Recognition
Simonyan, K., & Zisserman, A. (2014) · arXiv · doi:10.48550/arxiv.1409.1556 · Cited by 114,148 · cited 10×
“Figure is reproduced based on VGG [9].”
BackgroundMethodInfluentialImageNet classification with deep convolutional neural networks
Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017) · Communications of the ACM · doi:10.1145/3065386 · Cited by 112,155 · cited 8×
“Trained with accurate human-annotated labels, the supervised learning methods obtained break-through results on different computer vision applications [1], [4], [8], [16].”
MethodInfluentialHochreiter, S., & Schmidhuber, J. (1997) · Neural Computation · doi:10.1162/neco.1997.9.8.1735 · Cited by 102,318 · cited 1×
“Compared to standard RNN [72], long short term memory (LSTM) uses memory cells to store, modify, and access internal states, to better model the long-term temporal relationships within video frames [73].”
BackgroundFaster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks
Ren, S., He, K., Girshick, R., & Sun, J. (2017) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2016.2577031 · Cited by 73,472 · cited 2×
“DUE to the powerful ability to learn different levels of general visual features, deep neural networks have been used as the basic structure to many computer vision applications such as object detection [1], [2], [3], semantic segmentation [4], [5], [6], image captioning [7], etc.”
BackgroundMethodFaster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks
Ren, S., He, K., Girshick, R., & Sun, J. (2015) · arXiv · doi:10.48550/arxiv.1506.01497 · Cited by 70,812 · cited 2×
“DUE to the powerful ability to learn different levels of general visual features, deep neural networks have been used as the basic structure to many computer vision applications such as object detection [1], [2], [3], semantic segmentation [4], [5], [6], image captioning [7], etc.”
BackgroundMethodImageNet: A large-scale hierarchical image database
Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., & Fei-Fei, L. (2009) · 2009 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2009.5206848 · Cited by 65,333 · cited 5×
“• ImageNet: The ImageNet dataset [13] contains 1.”
BackgroundInfluentialGradient-based learning applied to document recognition
Lecun, Y., Bottou, L., Bengio, Y., & Haffner, P. (1998) · Proceedings of the IEEE · doi:10.1109/5.726791 · Cited by 58,805 · cited 1×
“• MNIST: The MNIST is a dataset of handwritten digits consisting of 70, 000 images while 60, 000 images belong to training set and the rest 10, 000 images are for testing [110].”
BackgroundDistinctive Image Features from Scale-Invariant Keypoints
Lowe, D. G. (2004) · International Journal of Computer Vision · doi:10.1023/b:visi.0000029664.99615.94 · Cited by 55,589
You Only Look Once: Unified, Real-Time Object Detection
Redmon, J., Divvala, S., Girshick, R., & Farhadi, A. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.91 · Cited by 53,803 · cited 1×
“Recently, many datasets such as MSCOCO [99] and OpenImage [14] have been proposed for object detection and many ConvNet-based models [1], [2], [3], [100], [101], [102], [103], [104] have been proposed and obtained great performance.”
Background
Cited by
Items 201–300 of 1,005
High-Dimensional Bayesian Optimisation with Variational Autoencoders and Deep Metric LearningPreprint
Grosnit, A., Tutunov, R., Maraval, A., Griffiths, R., Cowen-Rivers, A. I., Yang, L., Zhu, L., Lyu, W., Chen, Z., Wang, J., et al. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2106.03609 · Cited by 106
“Generally, this dichotomisation requires access to class labels, which are either readily available in supervised settings or constructed implicitly during data augmentation in self-supervised learning [41, 42].”
BackgroundOn the role of features in human activity recognition
Haresamudram, H., Anderson, D. V., & Plötz, T. (2019) · Proceedings of the 23rd International Symposium on Wearable Computers · doi:10.1145/3341163.3347727 · Cited by 90
“They are utilized to learn features by predicting geometric transformations, and solving jigsaw puzzles [24].”
MethodEnhancing Sequential Recommendation with Graph Contrastive Learning
Zhang, Y., Liu, Y., Xu, Y., Xiong, H., Lei, C., He, W., Cui, L., & Miao, C. (2022) · International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2022/333 · Cited by 108
Cao, Y., Chen, L., Yuan, Y., & Sun, G. (2023) · Computers and Electronics in Agriculture · doi:10.1016/j.compag.2023.107993 · Cited by 99
Label-efficient learning in agriculture: A comprehensive review
Li, J., Chen, D., Qi, X., Li, Z., Huang, Y., Morris, D., & Tan, X. (2023) · Computers and Electronics in Agriculture · doi:10.1016/j.compag.2023.108412 · Cited by 105
Self-Consistent Contrastive Attributed Graph Clustering With Pseudo-Label Prompt
Xia, W., Wang, Q., Gao, Q., Yang, M., & Gao, X. (2022) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2022.3213208 · Cited by 100
“studied and applied in various machine learning tasks, especially in representation learning [45], [46].”
BackgroundHolmberg, O. G., Köhler, N. D., Martins, T., Siedlecki, J., Herold, T., Keidel, L., Asani, B., Schiefelbein, J., Priglinger, S., Kortuem, K. U., et al. (2020) · Nature Machine Intelligence · doi:10.1038/s42256-020-00247-1 · Cited by 88
Deep Learning in Biomedical Optics
Tian, L., Hunt, B., Bell, M. A. L., Yi, J., Smith, J. T., Ochoa, M., Intes, X., & Durr, N. J. (2021) · Lasers in Surgery and Medicine · doi:10.1002/lsm.23414 · Cited by 93
“sets, although strategies to overcome this are an active topic in the literature, such as unsupervised [8], self‐ supervised [9], and semi‐supervised learning [10].”
BackgroundAkçakaya, M., Yaman, B., Chung, H., & Ye, J. C. (2022) · IEEE Signal Processing Magazine · doi:10.1109/msp.2021.3119273 · Cited by 108
“Self-supervised learning aims to generate supervisory labels automatically from the data itself to solve some tasks, and has found applications in many machine learning applications [4].”
BackgroundHu, M., Wu, C., & Zhang, L. (2022) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2022.3218795 · Cited by 97
“Self-supervised feature learning is a powerful feature learning paradigm on unlabeled data [18], [19], [21], [42], and has achieved great success on visual image processing, e.”
BackgroundS3VAE: Self-Supervised Sequential VAE for Representation Disentanglement and Data Generation
Zhu, Y., Min, M. R., Kadav, A., & Graf, H. P. (2020) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr42600.2020.00657 · Cited by 139
Pre-training Graph Transformer with Multimodal Side Information for Recommendation
Liu, Y., Yang, S., Lei, C., Wang, G., Tang, H., Zhang, J., Sun, A., & Miao, C. (2021) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3474085.3475709 · Cited by 115
Liu, S., Chen, J., He, S., Shi, Z., & Zhou, Z. (2022) · Mechanical Systems and Signal Processing · doi:10.1016/j.ymssp.2022.110071 · Cited by 91
Beyond Robustness: A Taxonomy of Approaches towards Resilient Multi-Robot SystemsPreprint
Prorok, A., Malencia, M., Carlone, L., Sukhatme, G. S., Sadler, B. M., & Kumar, V. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2109.12343 · Cited by 110
SpectralEarth: Training Hyperspectral Foundation Models at Scale
Braham, N. A. A., Albrecht, C. M., Mairal, J., Chanussot, J., Wang, Y., & Zhu, X. X. (2025) · IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing · doi:10.1109/jstars.2025.3581451 · Cited by 101
“SSL emerged as a powerful paradigm for learning representations from unlabeled data [36].”
BackgroundFew-Shot Learning With a Strong Teacher
Ye, H.-J., Lu, M., Zhan, D., & Chao, W. (2022) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2022.3160362 · Cited by 100
“[32] applied KD mainly to construct a portable few-shot learner; [31] applied self-supervised learning [93], [94] to train robust features and use them to guide the few-shot learner.”
MethodA Survey of Label-Efficient Deep Learning for 3D Point Clouds
Xiao, A., Zhang, X., Shao, L., & Lu, S. (2024) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2024.3416302 · Cited by 102
“In addition, several surveys [4], [5], [6], [7], [8] focus on label-efficient learning such as self-supervised learning [5], [9], [10], small sample learning [6], weakly-supervised learning [11], [12], [13], and generalizing across domains [7], [8].”
BackgroundMixCo: Mix-up Contrastive Learning for Visual RepresentationPreprint
Kim, S., Lee, G., Bae, S., & Yun, S.-Y. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2010.06300 · Cited by 108
“Self-supervised learning is an approach that exploits supervised signals through pretext (or proxy) tasks [18].”
BackgroundMulti-Task Self-Training for Learning General Representations
Ghiasi, G., Zoph, B., Cubuk, E. D., Le, Q. V., & Lin, T.-Y. (2021) · 2021 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00873 · Cited by 84
“Self-supervised training is a popular method for representation learning without supervised data [24, 5, 16, 17, 21, 50].”
MethodAbbas, A., Abdelsamea, M. M., & Gaber, M. M. (2021) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2021.3082015 · Cited by 83
Applying self-supervised learning for semantic cloud segmentation of all-sky images
Fabel, Y., Nouri, B., Wilbert, S., Blum, N. B., Triebel, R., Hasenbalg, M., Kuhn, P., Zarzalejo, L. F., & Pitz‐Paal, R. (2022) · Atmospheric measurement techniques · doi:10.5194/amt-15-797-2022 · Cited by 93
Tao, C., Qi, J., Zhang, G., Zhu, Q., Lu, W., & Li, H. (2023) · IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing · doi:10.1109/jstars.2023.3271312 · Cited by 98
“Most recently, a new machine learning paradigm, selfsupervised learning (SSL), has emerged in the field of natural language process (NLP) and computer vision (CV) [13], [14], [15], [16].”
BackgroundSimple Unsupervised Multi-Object TrackingPreprint
Karthik, S., Prabhu, A., & Gandhi, V. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2006.02609 · Cited by 100
“We propose to learn our model in an unsupervised manner in the free-labels paradigm (Section 6.3.2 in [21]) in a two-step manner.”
MethodInfluentialContIG: Self-supervised Multimodal Contrastive Learning for Medical Imaging with Genetics
Taleb, A., Kirchler, M., Monti, R., & Lippert, C. (2022) · 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52688.2022.02024 · Cited by 83
“These methods witnessed a surge of interest after proving successful in several application domains [54].”
BackgroundMethodBerg, P., Pham, M., & Courty, N. (2022) · Remote Sensing · doi:10.3390/rs14163995 · Cited by 89
“For more insightful surveys of self-supervised approaches, readers are invited to read dedicated review papers [23–25].”
BackgroundContinual Object Detection: A review of definitions, strategies, and challenges
Menezes, A. G., de Moura, G., Alves, C., & de Carvalho, A. C. P. L. F. (2023) · Neural Networks · doi:10.1016/j.neunet.2023.01.041 · Cited by 103
Towards Utilizing Unlabeled Data in Federated Learning: A Survey and ProspectivePreprint
Jin, Y., Wei, X., Liu, Y., & Yang, Q. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2002.11545 · Cited by 95
“…feature learning is popular, commonly achieved by learning coloriza-Figure tion, positioning and rotation information [ Trinh et al. , 2019; Pathak et al. , 2016 ] , and has been used for boosting performance in semantic segmentation, clustering, and object detection [ Jing and Tian, 2019 ] .”
ProtoCLIP: Prototypical Contrastive Language Image Pretraining
Chen, D., Zhao, W., Liu, F., Yang, Z., Zheng, S., Tan, Y., & Zhou, E. (2023) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2023.3335859 · Cited by 99
“Early works on SSL focus on exploring pretext tasks [36].”
Real-Time Semantic Image Segmentation with Deep Learning for Autonomous Driving: A Survey
Papadeas, I., Tsochatzidis, L., Amanatiadis, A. A., & Pratikakis, I. (2021) · Applied Sciences · doi:10.3390/app11198802 · Cited by 95
“As reported in [71], the cost of obtaining weak supervision labels is generally much cheaper than fine-grained labels for supervised methods.”
BackgroundSun, S., Wang, T., Yang, H., & Chu, F. (2022) · Applied Energy · doi:10.1016/j.apenergy.2022.118882 · Cited by 87
Self-supervised vision transformer-based few-shot learning for facial expression recognition
Chen, X., Zheng, X., Sun, K., Liu, W., & Zhang, Y. (2023) · Information Sciences · doi:10.1016/j.ins.2023.03.105 · Cited by 88
Dissecting self-supervised learning methods for surgical computer vision
Ramesh, S., Srivastav, V., Alapatt, D., Yu, T., Murali, A., Sestini, L., Nwoye, C. I., Hamoud, I., Sharma, S., Fleurentin, A., et al. (2023) · Medical Image Analysis · doi:10.1016/j.media.2023.102844 · Cited by 112
Self-Supervised Learning Across Domains
Bucci, S., D’Innocente, A., Liao, Y., Carlucci, F. M., Caputo, B., & Tommasi, T. (2021) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2021.3070791 · Cited by 115
Zhou, Y., Xu, H., Zhang, W., Gao, B., & Heng, P. (2021) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00695 · Cited by 108
“Here we focus on the most related literature for dense prediction tasks and refer readers to [25, 21] for other details.”
BackgroundSelf-Supervised Representation Learning from Electroencephalography Signals
Banville, H. J., Albuquerque, I., Hyvärinen, A., Moffat, G., Engemann, D. A., & Gramfort, A. (2019) · Machine learning for signal processing ... · doi:10.1109/mlsp.2019.8918693 · Cited by 90
Graph Regularized Residual Subspace Clustering Network for hyperspectral image clustering
Cai, Y., Zeng, M., Cai, Z., Liu, X., & Zhang, Z. (2021) · Information Sciences · doi:10.1016/j.ins.2021.07.003 · Cited by 78
Memorizing Structure-Texture Correspondence for Image Anomaly Detection
Zhou, K. Q., Li, J., Xiao, Y., Yang, J., Cheng, J. T., Liu, W., Luo, W., Liu, J., & Gao, S. (2021) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2021.3101403 · Cited by 84
“Recently, many self-supervised learning methods are proposed to learn the general image features from unlabeled data without any human annotations, which can be used for down-stream tasks such as image classification and segmentation [49].”
BackgroundCross-Modal Center Loss for 3D Cross-Modal Retrieval
Jing, L., Vahdani, E., Tan, J., & Tian, Y. (2021) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr46437.2021.00316 · Cited by 93
Self-citationLuengo, J., Moreno, R., Sevillano-García, I., Charte, D., Peláez-Vegas, A., Fernández-Moreno, M., Mesejo, P., & Herrera, F. (2021) · Information Fusion · doi:10.1016/j.inffus.2021.09.018 · Cited by 80
Progressive Video Summarization via Multimodal Self-supervised Learning
Li, H., Ke, Q., Gong, M., & Drummond, T. W. (2023) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv56688.2023.00554 · Cited by 102
“Self-supervised learning has been widely used for the pretraining of deep models, which boosts their performance to a large extent in various fields [10, 37, 15, 39, 20, 23, 8].”
BackgroundMedIAnomaly: A comparative study of anomaly detection in medical images
Cai, Y., Zhang, W., Chen, H., & Cheng, K.-T. T. (2025) · Medical Image Analysis · doi:10.1016/j.media.2025.103500 · Cited by 110
Shadow-Consistent Semi-Supervised Learning for Prostate Ultrasound Segmentation
Xu, X., Sanford, T., Türkbey, B., Xu, S., Wood, B. J., & Yan, P. (2021) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2021.3139999 · Cited by 84
Choi, E., Lee, S., Jeong, E., Shin, S., Park, H., Youm, S., Son, Y., & Pang, K. (2022) · Scientific Reports · doi:10.1038/s41598-022-06483-2 · Cited by 92
“Finally, the various techniques developed in the machine learning society, such as ensemble learning 33 , self-supervised learning 34 , and contrastive learning 35 , can be utilized for further improvement of the performance of our models even in situations where the total number of cases is…”
MethodSelf-supervised Learning for Electroencephalogram: A Systematic Survey
Weng, W., Gu, Y., Guo, S., Ma, Y., Yang, Z., Liu, Y., & Chen, Y. (2025) · ACM Computing Surveys · doi:10.1145/3736574 · Cited by 98
“…been investigated to extract channel correlation and brain structure, which are listed as follows: EEG jigsaw task [88] is analogous to the image jigsaw pretext task in CV. EEG jigsaw task involves the random shuffling of EEG channels, followed by an expectation that the model can reconstruct…”
Multiple Instance Captioning: Learning Representations from Histopathology Textbooks and Articles
Gamper, J., & Rajpoot, N. (2021) · 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr46437.2021.01628 · Cited by 75
Multi-Modal Pain Intensity Assessment Based on Physiological Signals: A Deep Learning Perspective
Thiam, P., Hihn, H., Braun, D. A., Kestler, H. A., & Schwenker, F. (2021) · Frontiers in Physiology · doi:10.3389/fphys.2021.720464 · Cited by 86
Latif, S., Rana, R. K., Khalifa, S., Jurdak, R., & Schuller, B. W. (2022) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2022.3167013 · Cited by 90
“Reconstruction is widely used as a pretext task in the computer vision literature [51], [86], wherein an autoen-coder network is used to reconstruct the input back from the compressed representation.”
Few-Shot Hyperspectral Image Classification With Self-Supervised Learning
Li, Z., Guo, H., Chen, Y., Liu, C., Du, Q., Fang, Z., & Wang, Y. (2023) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2023.3298851 · Cited by 88
“As a promising paradigm for learning useful image representations, SSL utilizes the input data itself as supervision and facilitates almost all types of downstream tasks [36].”
MethodAutomating crystal-structure phase mapping by combining deep learning with constraint reasoning
Chen, D., Bai, Y., Ament, S., Zhao, W., Guevarra, D., Zhou, L., Selman, B., van Dover, R. B., Gregoire, J. M., & Gomes, C. P. (2021) · Nature Machine Intelligence · doi:10.1038/s42256-021-00384-1 · Cited by 68
“Review of Scientific Instruments, 80(10):103902, 2009.”
Radio SLAM: A Review on Radio-Based Simultaneous Localization and Mapping
Amjad, B., Ahmed, Q. Z., Lazaridis, P. I., Hafeez, M., Khan, F. A., & Zaharis, Z. D. (2023) · IEEE Access · doi:10.1109/access.2023.3237330 · Cited by 84
“NN based DL algorithms [78] have become significantly popular in the field of computer vision [79], natural language processing [80], and more recently, in the applications of wireless communications [81].”
BackgroundDong, H., Ma, W., Wu, Y., Zhang, J., & Jiao, L. (2020) · Remote Sensing · doi:10.3390/rs12111868 · Cited by 77
“In fact, self-supervised learning algorithms, as a branch of unsupervised learning algorithms, have been very early put forward to learn general feature representations from unlabeled image and video data without using any manual labels [26].”
BackgroundLongitudinal self-supervised learning
Zhao, Q., Liu, Z., Adeli, E., & Pohl, K. M. (2021) · Medical Image Analysis · doi:10.1016/j.media.2021.102051 · Cited by 71
DL4SciVis: A State-of-the-Art Survey on Deep Learning for Scientific Visualization
Wang, C., & Han, J. (2022) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2022.3167896 · Cited by 77
“1, depending on howmuch labeled data are used for neural network training, DL tasks can be supervised (full labels), semi-supervised (partial labels), or unsupervised (no labels) [77].”
BackgroundA Self-Supervised Residual Feature Learning Model for Multifocus Image Fusion
Wang, Z., Li, X., Duan, H., & Zhang, X. (2022) · IEEE Transactions on Image Processing · doi:10.1109/tip.2022.3184250 · Cited by 82
“Self-supervised learning usually consists of two steps [24]: the pretext task and the downstream task.”
Wang, P., Wen, J., Si, C., Qian, Y., & Wang, L. (2022) · IEEE Transactions on Image Processing · doi:10.1109/tip.2022.3207577 · Cited by 75
Xu, M., Kim, H., Yang, J., Fuentes, A., Meng, Y., Yoon, S., Kim, T.-H., & Park, D. S. (2023) · Frontiers in Plant Science · doi:10.3389/fpls.2023.1225409 · Cited by 81
“In SSL, a pretext task should fi rst be de fi ned without using annotation to train a deep learning – based model (Jing and Tian, 2020).”
MethodLee, I., Lee, E., & Yoo, S. B. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.00148 · Cited by 90
“We drew inspiration from anomaly detection tasks [10, 45, 47, 64] with one-class classification algorithms [20, 32, 39, 40, 52] and proposed a model that learns only from unoccluded datasets and can generalize well to unseen datasets.”
MethodAn expert knowledge-empowered CNN approach for welding radiographic image recognition
Liu, T., Zheng, H., Zheng, P., Bao, J., Wang, J., Liu, X., & Yang, C. (2023) · Advanced Engineering Informatics · doi:10.1016/j.aei.2023.101963 · Cited by 74
Transforming Image Super-Resolution: A ConvFormer-Based Efficient Approach
Wu, G., Jiang, J., Jiang, J., & Liu, X. (2024) · IEEE Transactions on Image Processing · doi:10.1109/tip.2024.3477350 · Cited by 76
“Recently, deep learning methods have achieved dramatic improvements in SISR tasks [2], [37], [38].”
MethodLi, B., Keikhosravi, A., Loeffler, A. G., & Eliceiri, K. W. (2020) · Medical Image Analysis · doi:10.1016/j.media.2020.101938 · Cited by 71
PathoDuet: Foundation models for pathological slide analysis of H&E and IHC stains
Hua, S., Yan, F., Shen, T., Ma, L., & Zhang, X. (2024) · Medical Image Analysis · doi:10.1016/j.media.2024.103289 · Cited by 98
Semi-Supervised Contrastive Learning With Similarity Co-Calibration
Zhang, Y., Zhang, X., Li, J., Qiu, R. C., Xu, H., & Tian, Q. (2023) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2022.3158069 · Cited by 67
“Semi-supervised Learning makes uses of few shot labeled data in conjunction with a large number of unlabeled data[2, 11, 17, 18, 22, 26, 24, 29].”
BackgroundDang, B., & Li, Y. (2021) · Remote Sensing · doi:10.3390/rs13163122 · Cited by 73
“As a result, many SSL methods have recently demonstrated promising results [20,24,25].”
Consequential Advancements of Self-Supervised Learning (SSL) in Deep Learning Contexts
Abdulrazzaq, M. M., Ramaha, N. T. A., Hameed, A. A., Salman, M., Yon, D. K., Fitriyani, N. L., Syafrudin, M., & Lee, S. W. (2024) · Mathematics · doi:10.3390/math12050758 · Cited by 74
“The representations should be distinguished with respect to the downstream tasks while being sufficiently generic to be utilized on untrained actions [11].”
BackgroundA comprehensive overview of core modules in visual SLAM framework
Cai, D., Li, R., Hu, Z., Lu, J., Li, S., & Zhao, Y. (2024) · Neurocomputing · doi:10.1016/j.neucom.2024.127760 · Cited by 75
VideoSSL: Semi-Supervised Learning for Video Classification
Jing, L., Parag, T., Wu, Z., Tian, Y., & Wang, H. (2021) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv48630.2021.00115 · Cited by 83
“Self-Supervised Learning: Self-supervised learning is another trend of approach to learn visual features from unlabeled data [23, 19, 31, 22].”
Self-citationImage2Point: 3D Point-Cloud Understanding with 2D Image Pretrained Models
Xu, C., Yang, S., Galanti, T., Wu, B., Yue, X., Zhai, B., Zhan, W., Vajda, P., Keutzer, K., & Tomizuka, M. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-19836-6_36 · Cited by 89
“Pretraining in 2D computer vision is an effective approach using supervised [17,21], self-supervised [23,33], and contrastive learning [2,8,10,12,26,29].”
MethodChristopoulou, M. I., Barmpounakis, S., Koumaras, H. G., & Kaloxylos, A. (2022) · Vehicular Communications · doi:10.1016/j.vehcom.2022.100569 · Cited by 77
FedCL: Federated contrastive learning for multi-center medical image classification
Liu, Z., Wu, F., Wang, Y., Yang, M., & Pan, X. (2023) · Pattern Recognition · doi:10.1016/j.patcog.2023.109739 · Cited by 79
MERBench: A Unified Evaluation Benchmark for Multimodal Emotion Recognition
Lian, Z., Sun, L., Ren, Y., Gu, H., Sun, H., Chen, L., Liu, B., & Tao, J. (2026) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2026.3653457 · Cited by 83
“Recently, visual features extracted from weakly-supervised or self-supervised models have achieved remarkable results on many tasks [53], [54].”
Learning from Few Samples: A SurveyPreprint
Bendre, N., Terashima‐Marín, H., & Najafirad, P. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2007.15484 · Cited by 72
“Studies have shown that self supervised learning can be used along with few-shot learning to boost the performance of the model towards novel categories [100], [101].”
BackgroundMeta-heuristic as manager in federated learning approaches for image processing purposes
Połap, D., & Woźniak, M. (2021) · Applied Soft Computing · doi:10.1016/j.asoc.2021.107872 · Cited by 65
Intra- and Inter-Slice Contrastive Learning for Point Supervised OCT Fluid Segmentation
He, X., Fang, L., Tan, M., & Chen, X. (2022) · IEEE Transactions on Image Processing · doi:10.1109/tip.2022.3148814 · Cited by 68
“The general idea of contrastive learning is to exploit representative features by learning the intrinsic similarity and dissimilarity of the unlabeled data [61].”
BackgroundSketch3T: Test-Time Training for Zero-Shot SBIR
Sain, A., Bhunia, A. K., Potlapalli, V., Chowdhury, P. N., Xiang, T., & Song, Y.-Z. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.00731 · Cited by 90
“Selfsupervision involves designing pretext tasks that can learn semantic information without human annotations [23], such as image colorization [69], super-resolution [24], frame order recognition [32], solving jigsaw puzzles [36,40], image in-painting [41], relative patch location prediction [15], etc.”
BackgroundSiFall: Practical Online Fall Detection with RF Sensing
Ji, S., Xie, Y., & Li, M. (2022) · ACM Conference on Embedded Networked Sensor Systems (SenSys) · doi:10.1145/3560905.3568517 · Cited by 80
“Self-supervised learning techniques support learning representations from a large amount of unlabeled data and based on that representation to serve downstream classification tasks with a few labeled instances [26].”
BackgroundCai, Y., Chen, H., Yang, X. Y., Zhou, Y. C., & Cheng, K.-T. T. (2023) · Medical Image Analysis · doi:10.1016/j.media.2023.102794 · Cited by 79
“Self-supervised learning (Jing and Tian, 2020), referring to learning methods in which networks are explicitly trained using pretext tasks with generated pseudo labels, has also been extensively studied for anomaly detection.”
Lei, B., yu, L., Xie, J., Wu, Y., Liang, E., Liu, Y., Yang, P., Wang, T., Liu, C., Du, J., et al. (2024) · Pattern Recognition · doi:10.1016/j.patcog.2024.110423 · Cited by 76
Vectorization and Rasterization: Self-Supervised Learning for Sketch and Handwriting
Bhunia, A. K., Chowdhury, P. N., Yang, Y., Hospedales, T. M., Xiang, T., & Song, Y.-Z. (2021) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr46437.2021.00562 · Cited by 87
“Self-supervised Learning: Self-supervised learning is now a large field, too big to review in detail here, with recent surveys [29] providing a broader overview.”
BackgroundBess, A., Berglind, F., Mukhopadhyay, S., Bryliński, M., Griggs, N. W., Cho, T., Galliano, C., & Wasan, K. M. (2021) · Drug Discovery Today · doi:10.1016/j.drudis.2021.10.022 · Cited by 70
Materials Data toward Machine Learning: Advances and Challenges
Zhu, L., Zhou, J., & Sun, Z. (2022) · The Journal of Physical Chemistry Letters · doi:10.1021/acs.jpclett.2c00576 · Cited by 68
A Review of Predictive and Contrastive Self-supervised Learning for Medical Images
Wang, W.-C., Ahn, E., Feng, D. D., & Kim, J. (2023) · Machine Intelligence Research · doi:10.1007/s11633-022-1406-4 · Cited by 72
“There were several comprehensive reviews on natural images, such as contrastive learning of visual representations [28], generative learning and contrastive learning[3], pre-trained language models[29], and self-supervised contrastive learning[30].”
BackgroundThe color out of space: learning self-supervised representations for Earth Observation imagery
Vincenzi, S., Porrello, A., Buzzega, P., Cipriano, M., Fronte, P., Cuccu, R., Ippoliti, C., Conte, A., & Calderara, S. (2021) · 2020 25th International Conference on Pattern Recognition (ICPR) · doi:10.1109/icpr48806.2021.9413112 · Cited by 52
“Unsupervised and self-supervised methods were introduced to learn general visual features from unlabeled data [22].”
MethodA Survey on Learning to Reject
Zhang, X.-Y., Xie, G.-S., Li, X., Mei, T., & Liu, C. (2023) · Proceedings of the IEEE · doi:10.1109/jproc.2023.3238024 · Cited by 86
“A lot of objectives and tasks have been proposed in the literature to efficiently and effectively realize self-supervised learning [128].”
BackgroundGeometric Visual Similarity Learning in 3D Medical Image Self-Supervised Pre-training
He, Y., Yang, G., Ge, R., Chen, Y., Coatrieux, J.-L., Wang, B., & Li, S. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.00920 · Cited by 92
“, CT, MR) self-supervised pretraining (SSP) [20].”
MethodA self-learning deep neural network for classification of breast histopathological images
Abdulaal, A. H., Valizadeh, M., Amirani, M. C., & Shah, A. F. M. S. (2023) · Biomedical Signal Processing and Control · doi:10.1016/j.bspc.2023.105418 · Cited by 71
Deep Learning Methods for Semantic Segmentation in Remote Sensing with Small Data: A Survey
Yu, A., Quan, Y., Yu, R., Guo, W., Wang, X., Hong, D., Zhang, H., Chen, J., Hu, Q., & He, P. (2023) · Remote Sensing · doi:10.3390/rs15204987 · Cited by 70
Vision transformers in domain adaptation and domain generalization: a study of robustness
Alijani, S., Fayyad, J., & Najjaran, H. (2024) · Neural Computing and Applications · doi:10.1007/s00521-024-10353-5 · Cited by 76
“Through SSL, models undertake pretext tasks that generate pseudo-labels from the data itself, fostering a foundational understanding of data patterns and features without the necessity for explicit labeling [68, 69].”
A Survey on Self-supervised Pre-training for Sequential Transfer Learning in Neural NetworksPreprint
Mao, H. H. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2007.00800 · Cited by 73
“Generative approaches for self-supervised learning involve the process of producing all or parts of the training data as part of the model’s output (Jing and Tian, 2019).”
BackgroundMethodInfluentialA Geometric Analysis of Neural Collapse with Unconstrained FeaturesPreprint
Zhu, Z., Ding, T., Zhou, J., Li, X., You, C., Sulam, J., & Qu, Q. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2105.02375 · Cited by 229
“This is also the case for contrastive learning [121,122] in self-supervised feature learning [123], where augmented views of every sample are treated as a “class” so that the total number of classes grows with the size of the dataset.”
BackgroundShot Contrastive Self-Supervised Learning for Scene Boundary Detection
Chen, S., Nie, X., Fan, D., Zhang, D., Bhat, V., & Hamid, R. (2021) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr46437.2021.00967 · Cited by 91
“Recently, several self-supervised learning approaches have been applied to learn generalized visual representations for images [22] [1] [16] [18] [28] [48] [51] [43] and short video”
BackgroundLearning With Nested Scene Modeling and Cooperative Architecture Search for Low-Light Vision
Liu, R., Ma, L., Ma, T., Fan, X., & Luo, Z. (2022) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2022.3212995 · Cited by 80
“Actually, the training process of our hierarchical strategy is a little bit similar to the training idea in self-supervised learning [43].”
ResultImage Coding for Machines with Omnipotent Feature Learning
Feng, R., Jin, X., Guo, Z., Feng, R., Gao, Y., He, T., Zhang, Z., Sun, S., & Chen, Z. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-19836-6_29 · Cited by 88
Unleashing the potential of AI for pathology: challenges and recommendations
Asif, A., Rajpoot, K., Graham, S., Snead, D. R. J., Minhas, F., & Rajpoot, N. (2023) · The Journal of Pathology · doi:10.1002/path.6168 · Cited by 75
“The invariances can be learned through various augmentation strategies, self-supervised learning [96,97], and contrastive learning [100].”
BackgroundPunn, N. S., & Agarwal, S. (2022) · Machine Learning · doi:10.1007/s10994-022-06219-3 · Cited by 72
Incorporating Context Graph with Logical Reasoning for Inductive Relation Prediction
Lin, Q., Liu, J., Xu, F., Pan, Y., Zhu, Y., Zhang, L., & Zhao, T. (2022) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/3477495.3531996 · Cited by 82
“Besides, we intend to explore how to further strengthen the semantic representations of relations by some self-supervised strategies [17, 20].”
BackgroundZhang, H., Liu, W., Shi, J., Chang, S., Wang, H., He, J. L., & Huang, Q. (2022) · IEEE Transactions on Instrumentation and Measurement · doi:10.1109/tim.2022.3228267 · Cited by 101
“In the area of self-supervised learning, the contrastive learning method and generative learning method are the main [11].”
BackgroundMethodInfluentialMuRCL: Multi-Instance Reinforcement Contrastive Learning for Whole Slide Image Classification
Zhu, Z., Yu, L., Wu, W., Yu, R., Zhang, D., & Wang, L. (2022) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2022.3227066 · Cited by 75
“They can be grouped into three categories [18], [21]: 1) context-based, 2) generative-based, and 3) contrastive-based methods.”
BackgroundFan, C., Lei, Y., Sun, Y., & Mo, L. (2023) · Energy · doi:10.1016/j.energy.2023.127972 · Cited by 68
Masked Autoencoders in 3D Point Cloud Representation Learning
Jiang, J., Lu, X., Zhao, L., Dazeley, R., & Wang, M. (2023) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2023.3314973 · Cited by 79
“, training a neural network with supervisory signals generated by the data itself [16].”
Graph Self-Supervised Learning With Application to Brain Networks Analysis
Wen, G., Cao, P., Liu, L., Yang, J., Zhang, X., Wang, F., & Zai͏̈ane, O. R. (2023) · IEEE Journal of Biomedical and Health Informatics · doi:10.1109/jbhi.2023.3274531 · Cited by 70
Metadata
- Authors
- Jing, L., & Tian, Y.
- Affiliations
- The Graduate Center, CUNY (US)
- Venue
- IEEE Transactions on Pattern Analysis and Machine Intelligence
- Published
- 2021-11-01
- Volume
- 43
- Issue
- 11
- Pages
- 4037-4058
- ISSN
- 0162-8828 (ISSN-L)
- 2160-9292
- 1939-3539
- OpenAlex
- W3023371261
- Cited by
- 2,036
- References
- 206
- Citations by year
- 2026: 160 · 2025: 283 · 2024: 354 · 2023: 405 · 2022: 377 · 2021: 305
- FWCI
- 119.51
- Type
- journal article
- Language
- en
- Open access
- green
- arxiv.org
- Licence
- Locations
- IEEE Transactions on Pattern Analysis and Machine Intelligence (publishedVersion)
- PubMed (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Funders
- National Science Foundation (IIS-1400802)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Domain Adaptation and Few-Shot Learning
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Image and Video Retrieval Techniques
- Keywords
- self-supervised learning, visual feature learning, imaging features, video features, deep learning architectures, unlabeled data, audio features, 3D data, benchmark dataset, evaluation metrics
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.