Self-Supervised Visual Feature Learning With Deep Neural Networks: A Survey
Open accessThe DOI matches its Crossref record; 204 of the 206 references it cites were found in a registry or an index.
Abstract
Large-scale labeled data are generally required to train deep neural networks in order to obtain better performance in visual feature learning from images or videos for computer vision applications. To avoid extensive cost of collecting and annotating large-scale datasets, as a subset of unsupervised learning methods, self-supervised learning methods are proposed to learn general image and video features from large-scale unlabeled data without using any human-annotated labels. This paper provides an extensive review of deep learning-based self-supervised general visual feature learning methods from images or videos. First, the motivation, general pipeline, and terminologies of this field are described. Then the common deep neural network architectures that used for self-supervised learning are summarized. Next, the schema and evaluation metrics of self-supervised learning methods are reviewed followed by the commonly used datasets for images, videos, audios, and 3D data, as well as the existing self-supervised visual feature learning methods. Finally, quantitative performance comparisons of the reviewed methods on benchmark datasets are summarized and discussed for both image and video feature learning. At last, this paper is concluded and lists a set of promising future directions for self-supervised visual feature learning.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Jing and Tian [18] summarize the self-supervised feature learning methods on image and video data, and four categories are discussed: generation-based, context-based, free semantic label-based, and cross modal-based, where cross-modal-based methods construct learning task using RGB frame sequence an optical flow sequence, which are unique features in the video.”
From Self-Supervised Learning for Time Series Analysis: Taxonomy, Progress, and Prospects (2024) · cites this paper as [18]
In the citing paper
“Another hot research sub-area is self-supervised learning, which learns discriminative visual representation by solving predefined pretext tasks [194].”
From Empowering things with intelligence: a survey of the progress, challenges, and opportunities in artificial intelligence of things (2021) · cites this paper as [194]
Citations
Cites
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474
Very Deep Convolutional Networks for Large-Scale Image Recognition
Simonyan, K., & Zisserman, A. (2014) · arXiv · doi:10.48550/arxiv.1409.1556 · Cited by 114,148 · cited 10×
“Figure is reproduced based on VGG [9].”
BackgroundMethodInfluentialImageNet classification with deep convolutional neural networks
Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017) · Communications of the ACM · doi:10.1145/3065386 · Cited by 112,155 · cited 8×
“Trained with accurate human-annotated labels, the supervised learning methods obtained break-through results on different computer vision applications [1], [4], [8], [16].”
MethodInfluentialHochreiter, S., & Schmidhuber, J. (1997) · Neural Computation · doi:10.1162/neco.1997.9.8.1735 · Cited by 102,318 · cited 1×
“Compared to standard RNN [72], long short term memory (LSTM) uses memory cells to store, modify, and access internal states, to better model the long-term temporal relationships within video frames [73].”
BackgroundFaster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks
Ren, S., He, K., Girshick, R., & Sun, J. (2017) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2016.2577031 · Cited by 73,472 · cited 2×
“DUE to the powerful ability to learn different levels of general visual features, deep neural networks have been used as the basic structure to many computer vision applications such as object detection [1], [2], [3], semantic segmentation [4], [5], [6], image captioning [7], etc.”
BackgroundMethodFaster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks
Ren, S., He, K., Girshick, R., & Sun, J. (2015) · arXiv · doi:10.48550/arxiv.1506.01497 · Cited by 70,812 · cited 2×
“DUE to the powerful ability to learn different levels of general visual features, deep neural networks have been used as the basic structure to many computer vision applications such as object detection [1], [2], [3], semantic segmentation [4], [5], [6], image captioning [7], etc.”
BackgroundMethodImageNet: A large-scale hierarchical image database
Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., & Fei-Fei, L. (2009) · 2009 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2009.5206848 · Cited by 65,333 · cited 5×
“• ImageNet: The ImageNet dataset [13] contains 1.”
BackgroundInfluentialGradient-based learning applied to document recognition
Lecun, Y., Bottou, L., Bengio, Y., & Haffner, P. (1998) · Proceedings of the IEEE · doi:10.1109/5.726791 · Cited by 58,805 · cited 1×
“• MNIST: The MNIST is a dataset of handwritten digits consisting of 70, 000 images while 60, 000 images belong to training set and the rest 10, 000 images are for testing [110].”
BackgroundDistinctive Image Features from Scale-Invariant Keypoints
Lowe, D. G. (2004) · International Journal of Computer Vision · doi:10.1023/b:visi.0000029664.99615.94 · Cited by 55,589
You Only Look Once: Unified, Real-Time Object Detection
Redmon, J., Divvala, S., Girshick, R., & Farhadi, A. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.91 · Cited by 53,803 · cited 1×
“Recently, many datasets such as MSCOCO [99] and OpenImage [14] have been proposed for object detection and many ConvNet-based models [1], [2], [3], [100], [101], [102], [103], [104] have been proposed and obtained great performance.”
Background
Cited by
Items 301–400 of 1,005
Self-supervised learning for point cloud data: A survey
Zeng, C., Wang, W., Nguyen, A., Xiao, J., & Yue, Y. (2023) · Expert Systems with Applications · doi:10.1016/j.eswa.2023.121354 · Cited by 68
Yang, G., Luo, S., & Greer, P. B. (2024) · Multimedia Tools and Applications · doi:10.1007/s11042-024-19298-2 · Cited by 60
“By using non-annotated samples to train on pretext tasks [51] and then transferring the knowledge to a downstream task – usually supervised training using labelled data – the performance of a downstream model can be improved.”
Joint Visual-Temporal Embedding for Unsupervised Learning of Actions in Untrimmed Sequences
VidalMata, R. G., Scheirer, W., Kukleva, A., Cox, D., & Kuehne, H. (2020) · IEEE Workshop/Winter Conference on Applications of Computer Vision · doi:10.1109/wacv48630.2021.00128 · Cited by 69
“Various approaches have been proposed for learning visual representations without labels [12, 16, 1, 33].”
BackgroundSelf-Supervised Graph Learning With Hyperbolic Embedding for Temporal Health Event Prediction
Lu, C., Reddy, C. K., & Ning, Y. (2023) · IEEE Transactions on Cybernetics · doi:10.1109/tcyb.2021.3109881 · Cited by 51
“Self-supervised learning refers to training models with automatically generated labels [23].”
BackgroundSelf-supervision advances morphological profiling by unlocking powerful image representations
Kim, V., Adaloglou, N., Osterland, M., Morelli, F. M., Halawa, M., König, T., Gnutt, D., & Zapata, P. A. M. (2025) · Scientific Reports · doi:10.1038/s41598-025-88825-4 · Cited by 76
“Our results indicate that SSL methods provide a robust, efficient, and segmentation-free alternative to CellProfiler, and that SSL features demonstrate strong performance across a variety of clustering and predictive tasks.”
Federated Learning Approach for Breast Cancer Detection Based on DCNN
AlSalman, H., Al-Rakhami, M. S., Alfakih, T. M., & Hassan, M. M. (2024) · IEEE Access · doi:10.1109/access.2024.3374650 · Cited by 59
“With a larger dataset, the network has access to a more diverse set of features, including subtle variations in breast cancer presentations [15].”
BackgroundAeroRIT: A New Scene for Hyperspectral Image Analysis
Rangnekar, A., Mokashi, N., Ientilucci, E. J., Kanan, C., & Hoffman, M. J. (2020) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2020.2987199 · Cited by 60
“Self-supervised learning (survey: [32]) has shown much potential in helping randomly initialized neural networks learn better initialization points before being applied for their original task in other domains.”
BackgroundFast Adaptive Meta-Learning for Few-Shot Image Generation
Phaphuangwittayakul, A., Guo, Y., & Ying, F. (2021) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2021.3077729 · Cited by 68
Self-supervised audiovisual representation learning for remote sensing data
Heidler, K., Mou, L., Hu, D., Jin, P., Li, G., Gan, C., Wen, J.-R., & Zhu, X. X. (2022) · International Journal of Applied Earth Observation and Geoinformation · doi:10.1016/j.jag.2022.103130 · Cited by 70
“Self-supervised learning is a subset of unsupervised learning methods [23].”
Lu, A. X., Lu, A. X., Pritišanac, I., Zarin, T., Forman-Kay, J. D., & Moses, A. M. (2022) · PLOS Computational Biology · doi:10.1371/journal.pcbi.1010238 · Cited by 48
“Self-supervised learning trains models on “proxy” tasks resembling play and exploration [49], for which the labels can be automatically generated from data.”
BackgroundCRS-CONT: A Well-Trained General Encoder for Facial Expression Analysis
Li, H., Wang, N., Yang, X., & Gao, X. (2022) · IEEE Transactions on Image Processing · doi:10.1109/tip.2022.3186536 · Cited by 62
Wu, R., Liang, C., Li, Y., Shi, X., Zhang, J., & Huang, H. B. (2022) · Expert Systems with Applications · doi:10.1016/j.eswa.2022.119339 · Cited by 57
Robust Cross-Modal Representation Learning with Progressive Self-Distillation
Andonian, A., Chen, S., & Hamid, R. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.01594 · Cited by 90
“Self-Supervised Representation Learning: Selfsupervised learning (SSL) approaches use a pretext task to automatically generate a supervision signal from the data itself, thereby eliminating the dependence on expensive manual data-labeling [31].”
MethodAcoustic scene classification: A comprehensive survey
Ding, B., Zhang, T., Wang, C., Liu, G., Liang, J., Hu, R., Wu, Y., & Guo, D. (2023) · Expert Systems with Applications · doi:10.1016/j.eswa.2023.121902 · Cited by 73
Mitosis detection, fast and slow: Robust and efficient detection of mitotic figures
Jahanifar, M., Shephard, A., Tajeddin, N. Z., Graham, S., Raza, S. E. A., Minhas, F., & Rajpoot, N. (2024) · Medical Image Analysis · doi:10.1016/j.media.2024.103132 · Cited by 63
“In this pursuit, self-supervised learning (SSL) algorithms have shown success in extracting relevant visual features from unlabelled data (Jing and Tian, 2020).”
BackgroundFully 1 × 1 Convolutional Network for Lightweight Image Super-resolution
Wu, G., Jiang, J., Jiang, K., & Liu, X. (2024) · Machine Intelligence Research · doi:10.1007/s11633-024-1501-9 · Cited by 72
“Recently, deep learning methods have achieved dramatic improvements in SISR tasks [20], [21].”
MethodZemouri, R. M. (2025) · Machines · doi:10.3390/machines13020125 · Cited by 68
Self-supervised Visual Representation Learning for Histopathological Images
Yang, P., Hong, Z., Yin, X., Zhu, C., & Jiang, R. (2021) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-87196-3_5 · Cited by 56
Semi-supervised Time Series Classification Model with Self-supervised Learning
Xi, L., Yun, Z., Liu, H., Wang, R., Huang, X., & Fan, H. (2022) · Engineering Applications of Artificial Intelligence · doi:10.1016/j.engappai.2022.105331 · Cited by 56
Wahab, N., Miligy, I. M., Dodd, K., Sahota, H., Toss, M. S., Lu, W., Jahanifar, M., Bilal, M., Graham, S., Park, Y., et al. (2022) · The Journal of Pathology Clinical Research · doi:10.1002/cjp2.256 · Cited by 68
Contrastive Self-Supervised Learning: A Survey on Different Architectures
Khan, A., Albarri, S., & Manzoor, M. A. (2022) · doi:10.1109/icai55435.2022.9773725 · Cited by 74
Classification of Plant Leaf Disease Recognition Based on Self-Supervised Learning
Wang, Y., Yin, Y., Li, Y., Qu, T., Guo, Z., Peng, M., Jia, S., Wang, Q., Zhang, W., & Li, F. (2024) · Agronomy · doi:10.3390/agronomy14030500 · Cited by 65
“Currently, self-supervised learning in agriculture is increasingly favored [13–15], and its learning advantages and good performance in classification tasks are exactly what is needed for plant leaf disease recognition [16].”
BackgroundFu, D., Liu, J., Zhong, H., Zhang, X., & Zhang, F. (2024) · Knowledge-Based Systems · doi:10.1016/j.knosys.2024.111846 · Cited by 59
Dilated densely connected U-Net with uncertainty focus loss for 3D ABUS mass segmentation
Cao, X., Chen, H., Li, Y., Peng, Y., Wang, S., & Cheng, L. (2021) · Computer Methods and Programs in Biomedicine · doi:10.1016/j.cmpb.2021.106313 · Cited by 60
Mou, L., Zhou, C., Xie, P., Zhao, P., Jain, R., Gao, W., & Yin, B. (2021) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2021.3128738 · Cited by 62
“Self-supervised learning [45] is an unsupervised learning approach.”
BackgroundContrastive Learning-Based Dual Dynamic GCN for SAR Image Scene Classification
Liu, F., Qian, X., Jiao, L., Zhang, X., Li, L., & Cui, Y. (2022) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2022.3174873 · Cited by 64
“Self-supervised learning [50] model aims to learn effective representations from the data itself without supervision.”
BackgroundSelf-PU: Self Boosted and Calibrated Positive-Unlabeled TrainingPreprint
Chen, X., Chen, W., Chen, T., Yuan, Y., Gong, C., Chen, K., & Wang, Z. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2006.11280 · Cited by 87
“It is known to benefit data-efficient learning (Trinh et al., 2019; Jing & Tian, 2020), adversarial robustness (Chen et al., 2020), and outlier detection (Mohseni et al., 2020).”
BackgroundSelf-supervised 2D face presentation attack detection via temporal sequence sampling
Muhammad, U., Yu, Z., & Komulainen, J. (2022) · Pattern Recognition Letters · doi:10.1016/j.patrec.2022.03.001 · Cited by 54
Chattopadhyay, S., Manna, S., Bhattacharya, S., & Pal, U. (2022) · Proceedings - International Conference on Pattern Recognition/Proceedings/International Conference on Pattern Recognition · doi:10.1109/icpr56361.2022.9956442 · Cited by 62
“Self-supervised learning [1], [2] has emerged as a powerful paradigm that seeks to learn meaningful representations from unlabelled data to recognise underlying patterns without the”
BackgroundWhen Geoscience Meets Foundation Models: Toward a general geoscience artificial intelligence system
Zhang, H., Xu, J., Cui, H., Li, L., Yang, Y., Tang, C., & Boers, N. (2024) · IEEE Geoscience and Remote Sensing Magazine · doi:10.1109/mgrs.2024.3496478 · Cited by 82
“The SSL workflow typically involves defining a pretext task based on unlabeled inputs to generate meaningful representations [114], [116], [120], [121].”
Jin, Q., Yuan, M., Qiao, Q., & Song, Z. (2022) · Knowledge-Based Systems · doi:10.1016/j.knosys.2022.108278 · Cited by 53
Combining pretrained CNN feature extractors to enhance clustering of complex natural images
Guerin, J. M., Thiery, S., Nyiri, É., Gibaru, O., & Boots, B. (2020) · Neurocomputing · doi:10.1016/j.neucom.2020.10.068 · Cited by 64
Adversarial Spatiotemporal Contrastive Learning for Electrocardiogram Signals
Wang, N., Feng, P., Ge, Z., Zhou, Y., Zhou, B., & Wang, Z. (2023) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2023.3272153 · Cited by 82
“Self-supervised learning can learn useful representations in unlabeled data with pretext tasks [5], such as jigsaw puzzles [6], rotation prediction [7], multiview association [8], instance discrimination [9], and so on.”
BackgroundSelf-supervised Calorie-aware Heterogeneous Graph Networks for Food Recommendation
Song, Y., Yang, X., & Xu, C. (2022) · ACM Transactions on Multimedia Computing Communications and Applications · doi:10.1145/3524618 · Cited by 62
“Then, to effectively model the relations between ingredients to help capture the user’s personalized ingredient combination preference, we adopt the idea of self-supervised learning [8, 25, 27, 43, 50], which is a newly emerging paradigm aiming to let the model learn from the intrinsic structure of the raw data and ex…”
MethodSource-Free Unsupervised Domain Adaptation: Current research and future directions
Zhang, N., Lü, J., Li, K., Fang, Z., & Zhang, G. (2023) · Neurocomputing · doi:10.1016/j.neucom.2023.126921 · Cited by 57
Small data, big challenges: Machine- and deep-learning strategies for data-limited drug discovery
Pallikkavaliyaveetil, N., & Chandrasekaran, S. (2025) · Advanced Drug Delivery Reviews · doi:10.1016/j.addr.2025.115762 · Cited by 71
Unsupervised Deep Representation Learning and Few-Shot Classification of PolSAR Images
Zhang, L., Zhang, S., Zou, B., & Dong, H. (2020) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2020.3043191 · Cited by 60
“It has been proved that the training of supervised learning is inefficient and it converges to a fragile and task-specific solution [33], [61].”
BackgroundSelf-supervised Video Representation Learning by Uncovering Spatio-temporal Statistics
Wang, J., Jiao, J., Bao, L., He, S., Liu, W., & Liu, Y. (2020) · PubMed · doi:10.1109/tpami.2021.3057833 · Cited by 65
Feature Reuse and Scaling: Understanding Transfer Learning with Protein Language Models
Li, F.-Z., Amini, A. P., Yue, Y., Yang, K. K., & Lu, A. X. (2024) · doi:10.1101/2024.02.05.578959 · Cited by 51
“Pioneering work in generative self-supervised models also demonstrates that these models often saturate in downstream task performance in an intermediate layer of the model and degrade after (Jing & Tian, 2020).”
BackgroundChatterjee, S., Hazra, D., Byun, Y.-C., & Kim, Y. (2022) · Mathematics · doi:10.3390/math10091541 · Cited by 63
Reason and Discovery: A New Paradigm for Open Set Recognition
Fu, Y., Liu, Z., & Lyu, J. (2025) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2025.3552760 · Cited by 62
“The training process of single-stage methods typically begins with class-independent self-supervised learning [44].”
Jiang, F., Pan, C., Dong, L., Wang, K., Debbah, M. A., Niyato, D. T., & Han, Z. (2026) · IEEE Communications Surveys & Tutorials · doi:10.1109/comst.2026.3660844 · Cited by 80
“The model is then trained using these internal representations, similar to supervised learning but without external labels [100].”
Contrastive Self-Supervised Learning for Text-Independent Speaker Verification
Zhang, H., Zou, Y., & Wang, H. (2021) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp39728.2021.9413351 · Cited by 71
“More and more researchers divert attention to self-supervised learning (SSL) which benefits from large amount of unlabeled data [5, 6, 7].”
BackgroundPGL: Prior-Guided Local Self-supervised Learning for 3D Medical Image SegmentationPreprint
Xie, Y., Zhang, J., Liao, Z., Xia, Y., & Shen, C. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2011.12640 · Cited by 61
“Self-supervised learning (SSL) [39, 32, 16, 20] is able”
Yang, P., Yin, X., Lu, H., Hu, Z., Zhang, X., Jiang, R., & Lv, H. (2022) · Medical Image Analysis · doi:10.1016/j.media.2022.102539 · Cited by 50
CTRL-C: Camera calibration TRansformer with Line-Classification
Lee, J., Go, H., Lee, H., Cho, S., Sung, M., & Kim, J. (2021) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.01592 · Cited by 64
“For future work, we investigate to improve the generalization capability of the proposed network using the self-supervised based pre-training stage training [17, 24].”
MethodDeep active learning models for imbalanced image classification
Jin, Q., Yuan, M., Wang, H., Wang, M., & Song, Z. (2022) · Knowledge-Based Systems · doi:10.1016/j.knosys.2022.109817 · Cited by 62
Spatial Structure Constraints for Weakly Supervised Semantic Segmentation
Chen, T., Yao, Y., Huang, X., Li, Z., Nie, L., & Tang, J. (2024) · IEEE Transactions on Image Processing · doi:10.1109/tip.2024.3359041 · Cited by 64
“In contrast, self-supervised learning (SSL), a branch of unsupervised learning methods, aims to learn general image and video features from large numbers of unlabeled examples [39]–[41].”
BackgroundPerFedRec++: Enhancing Personalized Federated Recommendation with Self-Supervised Pre-Training
Luo, S., Xiao, Y., Zhang, X., Liu, Y., Ding, W., & Song, L. (2024) · ACM Transactions on Intelligent Systems and Technology · doi:10.1145/3664927 · Cited by 65
“Unlike traditional supervised learning, where labeled data is required for training, self-supervised learning leverages unlabeled data to learn representations that can be used for downstream tasks [21, 35].”
BackgroundGAN-Based Image Colorization for Self-Supervised Visual Feature Learning
Treneska, S., Zdravevski, E., Pires, I. M., Lameski, P., & Gievska, S. (2022) · Sensors · doi:10.3390/s22041599 · Cited by 57
“Self-supervised visual feature learning methods can be grouped into four categories based on the used pretext task: context-based, free-semantic label-based, crossmodal-based, and generation-based [13].”
MethodKang, D., Koniusz, P., Cho, M., & Murray, N. (2023) · Computer Vision and Pattern Recognition · doi:10.1109/cvpr52729.2023.01880 · Cited by 47
Brain-Inspired Remote Sensing Foundation Models and Open Problems: A Comprehensive Survey
Jiao, L., Huang, Z., Lu, X., Liu, X., Yang, Y., Zhao, J., Zhang, J., Hou, B., Yang, S., Liu, F., et al. (2023) · IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing · doi:10.1109/jstars.2023.3316302 · Cited by 58
“1) SSL for Natural Images: Based on the different pretext task approaches, three different SSL methods can be identified: generative, contrastive, and predictive [70], [77], as shown in Fig.”
BackgroundMethodSelf-supervised learning for Environmental Sound Classification
Tripathi, A. M., & Mishra, A. (2021) · Applied Acoustics · doi:10.1016/j.apacoust.2021.108183 · Cited by 51
Application of Machine Learning in Spatial Proteomics
Mou, M., Pan, Z., Lu, M., Sun, H., Wang, Y., Luo, Y., & Zhu, F. (2022) · Journal of Chemical Information and Modeling · doi:10.1021/acs.jcim.2c01161 · Cited by 50
Self-Supervised Learning Based on Spatial Awareness for Medical Image Analysis
Nguyen, X.-B., Lee, G.-S., Kim, S.-H., & Yang, H.-J. (2020) · IEEE Access · doi:10.1109/access.2020.3021469 · Cited by 60
“According to [17], based on the data attributes, pre-text tasks can be divided into four groups, which are generation-based methods, context-based methods, semantic label-based methods and cross modal-based methods.”
BackgroundSelf-supervised reinforcement learning-based energy management for a hybrid electric vehicle
Qi, C., Zhu, Y., Song, C., Cao, J., Xiao, F., Zhang, X., Xu, Z., & Song, S. (2021) · Journal of Power Sources · doi:10.1016/j.jpowsour.2021.230584 · Cited by 48
Self-Supervised Tumor Segmentation With Sim2Real Adaptation
Zhang, X., Xie, W., Huang, C., Zhang, Y., Chen, X., Tian, Q. C., & Wang, Y. (2023) · IEEE Journal of Biomedical and Health Informatics · doi:10.1109/jbhi.2023.3240844 · Cited by 52
Nonintrusive Load Monitoring Based on Self-Supervised Learning
Chen, S., Zhao, B., Zhong, M., Luan, W., & Yu, Y. (2023) · IEEE Transactions on Instrumentation and Measurement · doi:10.1109/tim.2023.3246504 · Cited by 56
“Note that self-supervised pretraining is treated as initialization for the general neural network f for each appliance category, contributing to performance improvement and overfitting mitigation [37].”
BackgroundInfluentialKnowledge-infused Contrastive Learning for Urban Imagery-based Socioeconomic Prediction
Liu, Y., Zhang, X., Ding, J., Xi, Y., & Li, Y. (2023) · International World Wide Web Conference (WWW) · doi:10.1145/3543507.3583876 · Cited by 68
“Motivated by the milestones of self-supervised learning achieved in computer vision [7, 19, 44], researchers also leverage self-supervised learning especially contrastive learning for urban imagery-based socioeconomic prediction, and focus on similarity metric design to distill expressive representations of urban imag…”
BackgroundUrban Flood Mapping With Bitemporal Multispectral Imagery Via a Self-Supervised Learning Framework
Peng, B., Huang, Q., Vongkusolkit, J., Gao, S., Wright, D. B., Fang, Z. N., & Qiang, Y. (2020) · IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing · doi:10.1109/jstars.2020.3047677 · Cited by 51
“Self-supervised learning is a special type of unsupervised learning [26], [27].”
BackgroundLearning and Sharing: A Multitask Genetic Programming Approach to Image Feature Learning
Bi, Y., Xue, B., & Zhang, M. (2021) · IEEE Transactions on Evolutionary Computation · doi:10.1109/tevc.2021.3097043 · Cited by 48
“Most of them are based on neural networks (NNs), particularly convolutional neural networks (CNNs) [13, 15].”
BackgroundMethodUnderstanding and Improving the Role of Projection Head in Self-Supervised LearningPreprint
Gupta, K., Ajanthan, T., van den Hengel, A., & Gould, S. J. (2022) · arXiv (Cornell University) · doi:10.48550/arxiv.2212.11491 · Cited by 62
“Due to space constraints, we briefly discuss previous works that are closely related to our work and/or inspired our thinking, and we refer the interested reader to the surveys [19, 17, 23] for a comprehensive study.”
BackgroundJigsawGAN: Auxiliary Learning for Solving Jigsaw Puzzles With Generative Adversarial Networks
Li, R., Liu, S., Wang, G., Liu, G., & Zeng, B. (2022) · IEEE Transactions on Image Processing · doi:10.1109/tip.2021.3120052 · Cited by 47
No-Reference Image Quality Assessment by Hallucinating Pristine Features
Chen, B., Zhu, L., Kong, C., Zhu, H., Wang, S., & Li, Z. (2022) · IEEE Transactions on Image Processing · doi:10.1109/tip.2022.3205770 · Cited by 55
“As indicated 223 in [58], the self-supervised learning methods greatly depend 224 on the difficulty level of the task and the amount of training 225 data.”
BackgroundLearning From Partially Labeled Data for Multi-Organ and Tumor Segmentation
Xie, Y., Zhang, J., Xia, Y., & Shen, C. (2023) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2023.3312587 · Cited by 56
“Although self-supervised learning [15], [59], [60], [61], [62], [63], [64], [65] has shown great potential to address this issue, it still suffers from the lack of a big dataset with millions or trillions of free data for self-supervised learning, especially for 3D volumes.”
BackgroundIntelligent Photonics: A Disruptive Technology to Shape the Present and Redefine the Future
Xu, D., Ma, Y., Jin, G., & Cao, L. (2024) · Engineering · doi:10.1016/j.eng.2024.08.016 · Cited by 57
Senanayaka, J. S. L., Khang, H. V., & Robbersmyr, K. G. (2020) · IEEE Transactions on Industrial Informatics · doi:10.1109/tii.2020.3014422 · Cited by 53
“The self-supervised learning is a relatively new concept, and their existing applications mainly focus on image classification, pattern recognition, and robotics [21]–[23].”
BackgroundSemantic Implicit Neural Scene Representations With Semi-Supervised Training
Kohli, A., Sitzmann, V., & Wetzstein, G. (2020) · International Conference on 3D Vision (3DV) · doi:10.1109/3dv50981.2020.00052 · Cited by 58
“A large body of work explores self-supervised representation learning on images [3, 6, 12, 14–16, 27, 31, 36, 38, 51, 53, 59, 60, 81, 85].”
Overview of deep learning application on visual SLAM
Li, S., Zhang, D., Xian, Y., Li, B., Zhang, T., & Zhong, C. (2022) · Displays · doi:10.1016/j.displa.2022.102298 · Cited by 54
Self-Supervised Learning for data scarcity in a fatigue damage prognostic problem
Akrim, A., Gogu, C., Vingerhoeds, R. A., & Salaün, M. (2023) · Engineering Applications of Artificial Intelligence · doi:10.1016/j.engappai.2023.105837 · Cited by 52
“SSL aims to improve predictive performance on the downstream task through the use of unlabelled data, thus avoiding the extensive cost of collecting and annotating large-scale datasets [30].”
BackgroundExploring simple triplet representation learning
Ren, Z., Lan, Q., Zhang, Y., & Wang, S. (2024) · Computational and Structural Biotechnology Journal · doi:10.1016/j.csbj.2024.04.004 · Cited by 47
“Moreover, self-supervised learning-based methods can get the same or superior performance than the fully supervised learning-based methods [1].”
Duong, S. T. M., Phung, S. L., Bouzerdoum, A., & Schira, M. M. (2020) · Magnetic Resonance Imaging · doi:10.1016/j.mri.2020.04.004 · Cited by 50
Memory-Efficient Semi-Supervised Continual Learning: The World is its Own Replay Buffer
Smith, J., Balloch, J., Hsu, Y.-C., & Kira, Z. (2021) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn52387.2021.9534361 · Cited by 55
Digital twin-assisted intelligent anomaly detection system for Internet of Things
Bolat-Akça, B., & Bozkaya, E. (2024) · Ad Hoc Networks · doi:10.1016/j.adhoc.2024.103484 · Cited by 45
Interpretable Failure Localization for Microservice Systems Based on Graph Autoencoder
Sun, Y., Lin, Z., Shi, B., Zhang, S., Ma, S., Jin, P., Zhong, Z., Pan, L., Guo, Y., & Pei, D. (2024) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/3695999 · Cited by 59
“Among them, prediction tasks require the manual design of complex and difficult tasks that may require domain expertise and experience [32].”
A descriptive framework for the field of deep learning applications in medical images
Tian, Y., & Fu, S. (2020) · Knowledge-Based Systems · doi:10.1016/j.knosys.2020.106445 · Cited by 45
Self-citationHow Well Do Self-Supervised Models Transfer?
Ericsson, L., Gouk, H., & Hospedales, T. M. (2021) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr46437.2021.00537 · Cited by 341
“Self-supervised learning Self-supervised representation learning is now a large topic that it is impossible to cover completely here, and we point the reader to excellent recent surveys [26, 37] for thorough reviews.”
BackgroundUnlabeled skin lesion classification by self-supervised topology clustering network
Wang, D., Pang, N., Wang, Y., & Zhao, H. (2021) · Biomedical Signal Processing and Control · doi:10.1016/j.bspc.2021.102428 · Cited by 50
Beyond Supervised Learning for Pervasive Healthcare
Gu, X., Deligianni, F., Han, J., Liu, X., Chen, W., Yang, G., & Lo, B. P. L. (2023) · IEEE Reviews in Biomedical Engineering · doi:10.1109/rbme.2023.3296938 · Cited by 52
“It should be noted that existing surveys have provided insightful ideas in terms of different aspects of model generalization and robustness [9], [10], [11], [12], [13].”
BackgroundTransformer-based contrastive prototypical clustering for multimodal remote sensing data
Cai, Y., Zhang, Z., Ghamisi, P., Rasti, B., Liu, X., & Cai, Z. (2023) · Information Sciences · doi:10.1016/j.ins.2023.119655 · Cited by 48
A Review of Pseudo-Labeling for Computer Vision
Kage, P., Rothenberger, J. C., Andreadis, P., & Diochnos, D. I. (2026) · Journal of Artificial Intelligence Research · doi:10.1613/jair.1.19656 · Cited by 55
“…example, EsViT C. Li et al. (2022), DINO Caron et al. (2021), BYOL Grill et al. (2020), and SwAV Caron et al. (2020) all use some form of label assignment as a representation learning pretext task , i.e., a task which prepares a model for transfer learning to a downstream task Jing and Tian (2021).”
Multi-modal hypergraph contrastive learning for medical image segmentation
Jing, W., Wang, J., Di, D., Li, D., Song, Y. S., & Fan, L. (2025) · Pattern Recognition · doi:10.1016/j.patcog.2025.111544 · Cited by 51
Biological data annotation via a human-augmenting AI-based labeling system
van der Wal, D., Jhun, I., Laklouk, I. A., Nirschl, J., Richer, L., Rojansky, R. B., Theparee, T., Wheeler, J. R., Sander, J., Feng, F. Y.-C., et al. (2021) · npj Digital Medicine · doi:10.1038/s41746-021-00520-6 · Cited by 55
“Recently, self-supervised learning techniques — in which synthetic labels are extracted from unlabeled data — have started to mature, demonstrating promise in decreasing the need for abundant labeled data 15 .”
BackgroundInteractive Contrastive Learning for Self-Supervised Entity Alignment
Zeng, K., Dong, Z., Hou, L., Cao, Y., Hu, M., Yu, J., Lv, X., Cao, L., Wang, X., Liu, H., et al. (2022) · ACM International Conference on Information and Knowledge Management (CIKM) · doi:10.1145/3511808.3557364 · Cited by 52
Self-Supervised Similarity Learning for Digital Pathology
Gildenblat, J., & Klaiman, E. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1905.08139 · Cited by 29
“Self-supervised learning is a subset of unsupervised learning methods in which CNNs are explicitly trained with automatically generated labels [6].”
Applications of Self-Supervised Learning to Biomedical Signals: A Survey
Pup, F. D., & Atzori, M. (2023) · IEEE Access · doi:10.1109/access.2023.3344531 · Cited by 43
“Self-supervised learning has been successfully applied in many fields, such as natural language processing [11], computer vision [12], speech recognition [13], and robotics [14].”
MethodApplying Self-Supervised Representation Learning for Emotion Recognition Using Physiological Signals
Quispe, K. G. M., Utyiama, D. M. D. S., Santos, E. M. D., Oliveira, H. A. B. F., & Souto, E. (2022) · Sensors · doi:10.3390/s22239102 · Cited by 51
“This has led to advances in the field of machine learning such as the introduction of transfer learning methods, semi-supervised learning, and self-supervised learning [10].”
BackgroundMethodInfluentialYin, C., Yang, B., Cheng, J. C. P., Gan, V. J. L., Wang, B., & Yang, J. (2023) · Automation in Construction · doi:10.1016/j.autcon.2023.104757 · Cited by 49
Liu, Z., Kainth, K., Zhou, A., Deyer, T. W., Fayad, Z. A., Greenspan, H. K., & Mei, X. (2024) · NMR in Biomedicine · doi:10.1002/nbm.5143 · Cited by 54
“35,36 Pretext tasks are artificial tasks designed for SSL setups with the primary goal of enabling the deep learning model to understand and capture the underlying structure or patterns in data without using any external labels.”
AutoProSAM: Automated Prompting SAM for 3D Multi-Organ Segmentation
Li, C., Sultan, R. I., Khanduri, P., Qiang, Y., Chetty, I. J., & Zhu, D. (2025) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv61041.2025.00352 · Cited by 68
“Recently, there has been significant interest among computer vision researchers in large, adaptable foundation models leveraging pre-training techniques such as self-supervised learning [16], contrastive learning [40], and language-vision pre-training [32].”
Learning Spatial-Semantic Relationship for Facial Attribute Recognition with Limited Labeled Data
Shu, Y., Yan, Y., Chen, S., Xue, J., Shen, C., & Wang, H. (2021) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr46437.2021.01174 · Cited by 51
“The training of self-supervised learning involves two tasks: a pretext task and a downstream task [16].”
MethodZheng, J., Wu, W., Yuan, S., Zhao, Y., Li, W., Zhang, L., Dong, R., & Fu, H. (2021) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2021.3105302 · Cited by 57
“1) Self-Supervised Approach: Self-supervised learning algorithms are designed to learn visual representations from a large amount of data in the absence of any human interpretation [31].”
Wang, Z., Xu, Q., Yang, Z., Cao, X., & Huang, Q. (2021) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3474085.3475446 · Cited by 53
“Self-Supervised Learning (SSL) is a popular form of unsupervised learning where data itself provides supervision signals [12].”
BackgroundHierarchy-aware contrastive learning with late fusion for skin lesion classification
Hsu, B. W., & Tseng, V. S. M. (2022) · Computer Methods and Programs in Biomedicine · doi:10.1016/j.cmpb.2022.106666 · Cited by 48
Contrastive Learning Assisted-Alignment for Partial Domain Adaptation
Yang, C., Cheung, Y., Ding, J., Tan, K. C., Xue, B., & Zhang, M. (2022) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2022.3145034 · Cited by 50
“With the augmentation, contrastive learning encourages encoding a sample (an anchor) and its augmentation (also known as similar sample, positive sample) be closer while the rest samples (different samples, negative sample) are far from each other [20], [48].”
Luo, C., Jin, L., & Chen, J. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.00111 · Cited by 55
“Recently selfsupervised representation learning has become a promising alternative and is thus attracting growing interest [24,34].”
BackgroundTransfuse: A Unified Transformer-Based Image Fusion Framework Using Self-Supervised LearningPreprint
Qu, L., Liu, S., Wang, M., Li, S., Yin, S., Qiao, Q., & Song, Z. (2022) · SSRN Electronic Journal · doi:10.2139/ssrn.4130858 · Cited by 46
Deep learning-based motion artifact removal in functional near-infrared spectroscopy
Gao, Y., Chao, H., Cavuoto, L. A., Yan, P., Krüger, U., Norfleet, J. E., Makled, B. A., Schwaitzberg, S. D., De, S., & Intes, X. (2022) · Neurophotonics · doi:10.1117/1.nph.9.4.041406 · Cited by 47
“Specifically, various pretext tasks have been used to train the deep network, 54 such as colorizing grayscale images, 55 image inpainting, 56 and image jigsaw puzzle.”
MethodFLPurifier: Backdoor Defense in Federated Learning via Decoupled Contrastive Training
Zhang, J., Zhu, C., Sun, X., Ge, C., Chen, B., Susilo, W., & Yu, S. (2024) · IEEE Transactions on Information Forensics and Security · doi:10.1109/tifs.2024.3384846 · Cited by 61
“Contrastive learning, a form of self-supervised learning, has garnered significant attention and implementation due to its remarkable performance [42].”
BackgroundSelf-Supervised Deep Learning to Reconstruct Seismic Data With Consecutively Missing Traces
Huang, H., Wang, T., Cheng, J., Xiong, Y., Wang, C., & Geng, J. (2022) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2022.3148994 · Cited by 52
“Thus, based on the pseudolabels, the features of the unannotated input data can be captured in a supervised manner and help to solve the target problems [55].”
Background
Metadata
- Authors
- Jing, L., & Tian, Y.
- Affiliations
- The Graduate Center, CUNY (US)
- Venue
- IEEE Transactions on Pattern Analysis and Machine Intelligence
- Published
- 2021-11-01
- Volume
- 43
- Issue
- 11
- Pages
- 4037-4058
- ISSN
- 0162-8828 (ISSN-L)
- 2160-9292
- 1939-3539
- OpenAlex
- W3023371261
- Cited by
- 2,036
- References
- 206
- Citations by year
- 2026: 160 · 2025: 283 · 2024: 354 · 2023: 405 · 2022: 377 · 2021: 305
- FWCI
- 119.51
- Type
- journal article
- Language
- en
- Open access
- green
- arxiv.org
- Licence
- Locations
- IEEE Transactions on Pattern Analysis and Machine Intelligence (publishedVersion)
- PubMed (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Funders
- National Science Foundation (IIS-1400802)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Domain Adaptation and Few-Shot Learning
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Image and Video Retrieval Techniques
- Keywords
- self-supervised learning, visual feature learning, imaging features, video features, deep learning architectures, unlabeled data, audio features, 3D data, benchmark dataset, evaluation metrics
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.