Self-Supervised Visual Feature Learning With Deep Neural Networks: A Survey
Open accessThe DOI matches its Crossref record; 204 of the 206 references it cites were found in a registry or an index.
Abstract
Large-scale labeled data are generally required to train deep neural networks in order to obtain better performance in visual feature learning from images or videos for computer vision applications. To avoid extensive cost of collecting and annotating large-scale datasets, as a subset of unsupervised learning methods, self-supervised learning methods are proposed to learn general image and video features from large-scale unlabeled data without using any human-annotated labels. This paper provides an extensive review of deep learning-based self-supervised general visual feature learning methods from images or videos. First, the motivation, general pipeline, and terminologies of this field are described. Then the common deep neural network architectures that used for self-supervised learning are summarized. Next, the schema and evaluation metrics of self-supervised learning methods are reviewed followed by the commonly used datasets for images, videos, audios, and 3D data, as well as the existing self-supervised visual feature learning methods. Finally, quantitative performance comparisons of the reviewed methods on benchmark datasets are summarized and discussed for both image and video feature learning. At last, this paper is concluded and lists a set of promising future directions for self-supervised visual feature learning.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Jing and Tian [18] summarize the self-supervised feature learning methods on image and video data, and four categories are discussed: generation-based, context-based, free semantic label-based, and cross modal-based, where cross-modal-based methods construct learning task using RGB frame sequence an optical flow sequence, which are unique features in the video.”
From Self-Supervised Learning for Time Series Analysis: Taxonomy, Progress, and Prospects (2024) · cites this paper as [18]
In the citing paper
“Another hot research sub-area is self-supervised learning, which learns discriminative visual representation by solving predefined pretext tasks [194].”
From Empowering things with intelligence: a survey of the progress, challenges, and opportunities in artificial intelligence of things (2021) · cites this paper as [194]
Citations
Cites
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474
Very Deep Convolutional Networks for Large-Scale Image Recognition
Simonyan, K., & Zisserman, A. (2014) · arXiv · doi:10.48550/arxiv.1409.1556 · Cited by 114,148 · cited 10×
“Figure is reproduced based on VGG [9].”
BackgroundMethodInfluentialImageNet classification with deep convolutional neural networks
Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017) · Communications of the ACM · doi:10.1145/3065386 · Cited by 112,155 · cited 8×
“Trained with accurate human-annotated labels, the supervised learning methods obtained break-through results on different computer vision applications [1], [4], [8], [16].”
MethodInfluentialHochreiter, S., & Schmidhuber, J. (1997) · Neural Computation · doi:10.1162/neco.1997.9.8.1735 · Cited by 102,318 · cited 1×
“Compared to standard RNN [72], long short term memory (LSTM) uses memory cells to store, modify, and access internal states, to better model the long-term temporal relationships within video frames [73].”
BackgroundFaster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks
Ren, S., He, K., Girshick, R., & Sun, J. (2017) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2016.2577031 · Cited by 73,472 · cited 2×
“DUE to the powerful ability to learn different levels of general visual features, deep neural networks have been used as the basic structure to many computer vision applications such as object detection [1], [2], [3], semantic segmentation [4], [5], [6], image captioning [7], etc.”
BackgroundMethodFaster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks
Ren, S., He, K., Girshick, R., & Sun, J. (2015) · arXiv · doi:10.48550/arxiv.1506.01497 · Cited by 70,812 · cited 2×
“DUE to the powerful ability to learn different levels of general visual features, deep neural networks have been used as the basic structure to many computer vision applications such as object detection [1], [2], [3], semantic segmentation [4], [5], [6], image captioning [7], etc.”
BackgroundMethodImageNet: A large-scale hierarchical image database
Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., & Fei-Fei, L. (2009) · 2009 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2009.5206848 · Cited by 65,333 · cited 5×
“• ImageNet: The ImageNet dataset [13] contains 1.”
BackgroundInfluentialGradient-based learning applied to document recognition
Lecun, Y., Bottou, L., Bengio, Y., & Haffner, P. (1998) · Proceedings of the IEEE · doi:10.1109/5.726791 · Cited by 58,805 · cited 1×
“• MNIST: The MNIST is a dataset of handwritten digits consisting of 70, 000 images while 60, 000 images belong to training set and the rest 10, 000 images are for testing [110].”
BackgroundDistinctive Image Features from Scale-Invariant Keypoints
Lowe, D. G. (2004) · International Journal of Computer Vision · doi:10.1023/b:visi.0000029664.99615.94 · Cited by 55,589
You Only Look Once: Unified, Real-Time Object Detection
Redmon, J., Divvala, S., Girshick, R., & Farhadi, A. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.91 · Cited by 53,803 · cited 1×
“Recently, many datasets such as MSCOCO [99] and OpenImage [14] have been proposed for object detection and many ConvNet-based models [1], [2], [3], [100], [101], [102], [103], [104] have been proposed and obtained great performance.”
Background
Cited by
Items 901–1,000 of 1,005
Zhang, J., Xing, T., Jiang, Y., Wu, X., Yan, D., Xue, W., Zhuang, S., Chen, L., Luo, L., & Ni, D. (2022) · Frontiers in Oncology · doi:10.3389/fonc.2022.938413 · Cited by 13
“Direct training of large-scale segmentation networks with tens of millions of parameters may introduce potential over-fi tting (23).”
BackgroundGlobal-and-Local sampling for efficient hybrid task self-supervised learning
Zhao, W., Xu, Y., Li, L., & Yang, H. (2023) · Knowledge-Based Systems · doi:10.1016/j.knosys.2023.110479 · Cited by 13
Self-Paced Hard Task-Example Mining for Few-Shot Classification
Xu, R., Yang, X., Yao, X., Tao, D., Cao, W., Lu, X., & Liu, W. (2023) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2023.3263593 · Cited by 12
“After that, many researchers applied SPL in different fields, e.g., few-shot learning [5], self-supervised learning [27], and multimodal learning [28] to get better use of training samples.”
BackgroundYu, Y., Sun, S., Huang, M., & Zhu, Q. (2023) · IEEE Transactions on Instrumentation and Measurement · doi:10.1109/tim.2023.3273689 · Cited by 12
“…large enough NIR spectra as the self-supervision [24] and is easy to synergize with label-free data augmentation techniques to increase source model generalization (since separating the pretraining of the source model from the labels can reduce the labeling bias of expanding spectral data) [27].”
BackgroundSelf-supervised learning for robust object retrieval without human annotations
den Herrewegen, J. V., Tourwé, T., & wyffels, F. (2023) · Computers & Graphics · doi:10.1016/j.cag.2023.06.029 · Cited by 16
CoRe: Contrastive and Restorative Self-Supervised Learning for Surface Defect Inspection
Wu, H., Li, B., Tian, L., Sun, Z., Dong, C., & Liao, W. (2023) · IEEE Transactions on Instrumentation and Measurement · doi:10.1109/tim.2023.3291776 · Cited by 14
“After accomplishing the above two stages, we follow the common protocol in SSL [21], [37] to evaluate the learned representation quality of the proposed method.”
MethodBouchard, C., Bernatchez, R., & Lavoie‐Cardinal, F. (2023) · Neurophotonics · doi:10.1117/1.nph.10.4.044405 · Cited by 14
“Self-supervised learning 22,23 (SSL) is a promising avenue for addressing annotated data scarcity when large datasets are available but the accessibility of ground truth annotations is limited.”
BackgroundModel-Contrastive Learning for Backdoor Elimination
Yue, Z., Xia, J., Ling, Z., Hu, M., Wang, T., Wei, X. H., & Chen, M. (2023) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3581783.3612415 · Cited by 14
“Contrastive learning (CL) has been widely investigated in self-supervised learning to achieve desired feature representations from unlabeled data [4, 5, 8, 19].”
BackgroundPolSAR Image Classification Based on Multi-Modal Contrastive Fully Convolutional Network
Hua, W., Wang, Y., Yang, S., & Jin, X. (2024) · Remote Sensing · doi:10.3390/rs16020296 · Cited by 12
Zhang, Y., Sun, L., Gupta, D., Ning, X., & Tiwari, P. (2024) · IEEE Journal of Biomedical and Health Informatics · doi:10.1109/jbhi.2024.3357168 · Cited by 12
“Thisenhancesthemodel’sgeneralizationabilitytohandleun-seen data, resulting in more robust outcomes [8].”
BackgroundMotion-artifact-augmented pseudo-label network for semi-supervised brain tumor segmentation
Qu, G., Lu, B., Shi, J.-L., Wang, Z., Yuan, Y.-P., Xia, Y.-F., Pan, Z., & Lin, Y. (2024) · Physics in Medicine and Biology · doi:10.1088/1361-6560/ad2634 · Cited by 12
Luo, S.-H., Zhao, X., Cao, M., Xu, J. J., Wang, W., Lu, X., Huang, Q.-T., Yue, X., Liu, G., Liu, Y., et al. (2024) · Analytical Chemistry · doi:10.1021/acs.analchem.3c04609 · Cited by 12
A framework for general-purpose microscopic image analysis via self-supervised learning
Zheng, Z., Yue, X., Wang, J., & Hou, J. (2024) · Materials Characterization · doi:10.1016/j.matchar.2024.114003 · Cited by 12
Model Inversion Attack Against Transfer Learning: Inverting a Model Without Querying It
Ye, D., Chen, H., Zhou, S., Zhu, T., Zhou, W., & Ji, S. (2025) · IEEE Transactions on Dependable and Secure Computing · doi:10.1109/tdsc.2025.3548119 · Cited by 16
“This claim is supported by the self-supervised learning research [27], [28], [29], where researchers often use well-trained image classifiers, such as VGG16 [30], to extract the features of unlabeled images for unseen classes.”
A Self-supervised GAN for Unsupervised Few-shot Object Recognition
Nguyen, K., & Todorović, S. (2021) · Proceedings - International Conference on Pattern Recognition/Proceedings/International Conference on Pattern Recognition · doi:10.1109/icpr48806.2021.9412539 · Cited by 12
“use the randomly sampled codes not only for the adversarial learning but also as a “free” ground-truth for self-supervised learning [10].”
MethodDeep Active Learning for Effective Pulmonary Nodule Detection
Liu, J., Cao, L., & Tian, Y. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-59725-2_59 · Cited by 15
“Recently, semi-supervised and unsupervised learning-based methods utilize partial or unlabeled data to avoid the annotation cost [5, 6].”
Self-citationLearning with self-supervision on EEG data
Gramfort, A., Banville, H. J., Chehab, O., Hyvärinen, A., & Engemann, D. A. (2021) · International Winter Conference on Brain-Computer Interface (BCI) · doi:10.1109/bci51272.2021.9385345 · Cited by 12
“To date, most applications of SSL have focused on domains where plentiful annotated data is already available, such as computer vision [5] and natural language processing [6].”
BackgroundMVCNet: Multiview Contrastive Network for Unsupervised Representation Learning for 3-D CT Lesions
Zhai, P., Cong, H., Zhu, E., Zhao, G., Yu, Y., & Li, J. (2022) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2022.3203412 · Cited by 13
Banitalebi-Dehkordi, A. (2021) · IEEE International Conference on Computer Vision Workshops · doi:10.1109/iccvw54120.2021.00091 · Cited by 17
“The idea is to learn general representations from one or more auxiliary tasks [14].”
BackgroundMachine learning-enabled feature classification of evaporation-driven multi-scale 3D printing
Ghosh, S., Johnson, M. V., Neupane, R., Hardin, J. O., Berrigan, J. D., Kalidindi, S. R., & Kong, Y. L. (2022) · Flexible and Printed Electronics · doi:10.1088/2058-8585/ac518a · Cited by 12
SCMS-Net: Self-Supervised Clustering-Based 3D Meshes Segmentation Network
Xue, J., Chen, Y., & Yang, X. (2023) · Computer-Aided Design · doi:10.1016/j.cad.2023.103512 · Cited by 12
Pretext-Contrastive Learning: Toward Good Practices in Self-supervised Video Representation LeaningPreprint
Li, T., Wang, X., & Yamasaki, T. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2010.15464 · Cited by 24
Multimodal supervised contrastive learning in remote sensing downstream tasks
Berg, P., Uzun, B., Pham, M.-T., & Courty, N. (2024) · IEEE Geoscience and Remote Sensing Letters · doi:10.1109/lgrs.2024.3385995 · Cited by 11
“downstream tasks (e.g., classification, detection, segmentation) with an improved performance compared with only performing the downstream task-specific training with a random model weights’ initialization or with the well-known ImageNet pretrained weights [4].”
BackgroundLoo, J., Teo, K. Y. C., Vyas, C. H., Jordan-Yu, J. M., Juhari, A., Jaffe, G. J., Cheung, C. M. G., & Farsiu, S. (2023) · Ophthalmology Science · doi:10.1016/j.xops.2023.100292 · Cited by 12
“36,37 Self-supervised pretraining enabled the network to learn about the structural content of the images and the pretrained weights provided a better initialization of the network during training of the segmentation task.”
Zhan, T., Dang, Q., & Zhu, Y. (2024) · IEEE Geoscience and Remote Sensing Letters · doi:10.1109/lgrs.2024.3415819 · Cited by 11
“In contrast to supervised learning, SSL [15] is capable of learning valuable feature representations from raw data without manual sample labeling.”
BackgroundYin, K., Ji, X., Wang, Y., & Wang, Z. (2024) · Defence Technology · doi:10.1016/j.dt.2024.08.015 · Cited by 11
Progress and Thinking on Self-Supervised Learning Methods in Computer Vision: A Review
Chen, Z., Hu, B., Chen, Z., & Zhang, J. (2024) · IEEE Sensors Journal · doi:10.1109/jsen.2024.3443885 · Cited by 17
“So far, several review papers have been published, but some of them only focused on SSL algorithms themselves [14], [15], [16], and others paid more attention on the applications of SSL models in specific CV fields [17], [18], [19].”
BackgroundDual-Feature Attention-Based Contrastive Prototypical Clustering for Multimodal Remote Sensing Data
Xu, S., Ding, X., Zhang, Y., Zhang, Z., Gao, H., & Zhang, B. (2024) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2024.3492500 · Cited by 12
“Recently, self-supervised learning has emerged as a promising approach to address this issue by extracting intrinsic attributes and semantics from large-scale unlabeled data, thereby reducing the need for costly labeled data [23], [24].”
Li, G., Togo, R., Mabe, K., Nishida, S., Tomoda, Y., Shiratani, F., Hirota, M., Ogawa, T., & Haseyama, M. (2025) · Journal of Gastroenterology · doi:10.1007/s00535-024-02209-5 · Cited by 12
Decoupled Doubly Contrastive Learning for Cross-Domain Facial Action Unit Detection
Li, Y., Liu, M., Cui, Z., Ding, Y., Zong, Y., Zheng, W., Shan, S., & Guan, C. (2025) · IEEE Transactions on Image Processing · doi:10.1109/tip.2025.3546479 · Cited by 14
“The fundamental objective of CL is to establish an embedding space wherein similar sample pairs remain closely clustered, while dissimilar pairs are distinctly separated.”
CLIP-Vision Guided Few-Shot Metal Surface Defect Recognition
Wang, T., Li, Z., Xu, Y., Zhai, Y., Xing, X., Guo, K., Coscia, P., Genovese, A., Piuri, V., & Scotti, F. (2025) · IEEE Transactions on Industrial Informatics · doi:10.1109/tii.2025.3547353 · Cited by 13
“2) Evaluation Metric: SSL evaluates model performance by FT and LE [8].”
InfluentialCRCFound: A Colorectal Cancer CT Image Foundation Model Based on Self‐Supervised Learning
Yang, J., Cai, D., Liu, J., Zhuang, Z., Zhao, Y., Wang, F., Li, C., Hu, C., Gai, B., Chen, Y., et al. (2025) · Advanced Science · doi:10.1002/advs.202407339 · Cited by 13
REAL: Representation enhanced analytic learning for exemplar-free class-incremental learning
He, R., Fang, D., Chen, Y., Tong, K., Chen, C., Wang, Y., Chau, L., & Zhuang, H. (2025) · Knowledge-Based Systems · doi:10.1016/j.knosys.2025.114901 · Cited by 15
“(2) The knowledge captured by the backbone is not e ff ectively utilized by the classifier.”
Deep feature annotation by iterative meta-pseudo-labeling on 2D projections
Benato, B. C., Telea, A. C., & Falcão, A. X. (2023) · Pattern Recognition · doi:10.1016/j.patcog.2023.109649 · Cited by 11
Smart Healthcare System Based on AIoT Emerging Technologies: A Brief Review
Prabha, C., Mittal, P., Gahlot, K., & Phul, V. (2023) · Lecture notes in electrical engineering · doi:10.1007/978-981-99-0601-7_23 · Cited by 11
Tian, Z., Li, W., Hu, J., & Deng, C. (2023) · Information Sciences · doi:10.1016/j.ins.2023.119542 · Cited by 11
Feng, Y., Zhou, K., Li, J., & Zhou, M. (2023) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2023.3313576 · Cited by 11
“For a comprehensive overview of self-supervised learning, the readers can refer to [42] and [43].”
BackgroundBeyond Instance Discrimination: Relation-Aware Contrastive Self-Supervised Learning
Zhang, Y., Liu, C., Zhou, Y., Wang, W., Ye, Q., & Ji, X. (2023) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2023.3324588 · Cited by 15
“growing attention [7], [8], [9], [10], [11], [12], [13], [14], [15], [16], [17], [18].”
Multicue Contrastive Self-Supervised Learning for Change Detection in Remote Sensing
Yang, M., Jiao, L., Liu, F., Hou, B., Yang, S., Zhang, Y., & Wang, J. (2023) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2023.3330494 · Cited by 12
“Various pretext tasks are designed in SSL to provide cues for pretraining networks, such as colorizing images or solving jigsaw puzzles [31].”
BackgroundSelf-Supervised Learning Guided by SAR Image Factors for Terrain Classification
Ren, Z., Du, Z., Liu, S., Hou, B., Li, W., Zhu, H., Ren, B., & Jiao, L. (2024) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2024.3386963 · Cited by 11
“Consequently, it has garnered extensive attention and research focus in natural and remote sensing domains [25], [26], [27].”
BackgroundHao, Z., Cai, X., Ge, Y., Foody, G. M., Li, X., Yin, Z., Du, Y., & Ling, F. (2024) · Journal of Hydrology · doi:10.1016/j.jhydrol.2024.131673 · Cited by 11
Learning Student Network Under Universal Label Noise
Tang, J., Jiang, N., Zhu, H., Zhou, J. T., & Gong, C. (2024) · IEEE Transactions on Image Processing · doi:10.1109/tip.2024.3430539 · Cited by 11
“Recent studies [23], [38] in self-supervised learning propose to explore the unlabeled examples to learn a DNN with powerful representational capability, which is useful for downstream tasks.”
Singh, A., Gorade, V., & Mishra, D. (2024) · IEEE Journal of Biomedical and Health Informatics · doi:10.1109/jbhi.2024.3455337 · Cited by 13
“Discrimination SSL falls into two main categories: contrastive and non-contrastive methods, which widely adopt the joint-embedding framework, often realized through Siamese network architecture [19].”
Deep Collaborative Multi-Modal Learning for Unsupervised Kinship Estimation
Dong, G.-N., Pun, C.-M., & Zhang, Z. (2021) · IEEE Transactions on Information Forensics and Security · doi:10.1109/tifs.2021.3098165 · Cited by 11
Gu, X., Guo, Y., Yang, G., & Lo, B. P. L. (2021) · IEEE Journal of Biomedical and Health Informatics · doi:10.1109/jbhi.2021.3107532 · Cited by 11
Deep Neural Networks Based Approach for Pothole Detection
Pandey, A. K., Iqbal, R., Amin, S. A., Maniak, T., Palade, V., & Karyotis, C. (2021) · International Conference on Signal Processing and Information Security (ICSPIS) · doi:10.1109/icspis53734.2021.9652434 · Cited by 11
Self-Supervised Enhancement for Named Entity Disambiguation via Multimodal Graph Convolution
Zhou, P., Ying, K., Wang, Z., Guo, D., & Bai, C. (2022) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2022.3173179 · Cited by 13
“SSL is a much-anticipated direction of unsupervised learning [39].”
BackgroundSemi-supervised learning for topographic map analysis over time: a study of bridge segmentation
Wong, C.-S., Liao, H.-M., Tsai, R. T., & Chang, M. (2022) · Scientific Reports · doi:10.1038/s41598-022-23364-w · Cited by 11
Image Segmentation Method on Quartz Particle-Size Detection by Deep Learning Networks
Nie, X., Zhang, C., & Cao, Q. (2022) · Minerals · doi:10.3390/min12121479 · Cited by 11
Self-supervised image co-saliency detection
Liu, Y., Li, T., Wu, Y., Song, H., & Zhang, K. (2022) · Computers & Electrical Engineering · doi:10.1016/j.compeleceng.2022.108533 · Cited by 13
SSSD: Self-Supervised Self Distillation
Chen, W.-C., & Chu, W.-T. (2023) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv56688.2023.00279 · Cited by 11
“More literature surveys on self-supervised learning or contrastive learning can be found in [15] and [16].”
BackgroundAdaptive Self-Supervised SAR Image Registration With Modifications of Alignment Transformation
Mao, S., Yang, J., Gou, S., Lü, K., Jiao, L., Xiong, T., & Xiong, L. (2023) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2023.3246964 · Cited by 14
“Recently, self-supervised learning [22], [23], [24] is given more attention due to reducing the requirement of manual annotations.”
BackgroundDiverse Imagenet Models Transfer Better
Nayman, N., Golbert, A., Noy, A., & Zelnik‐Manor, L. (2024) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv57701.2024.00192 · Cited by 13
“[3] estimated the number of redundant filters in each layer, by hierarchically clustering those according to their relative cosine distances in filter space.”
BackgroundAdding Seemingly Uninformative Labels Helps in Low Data RegimesPreprint
Matsoukas, C., Hernandez, A. B., Liu, Y., Dembrower, K., Miranda, G., Konuk, E., Haslum, J. F., Zouzos, A., Lindholm, P., Strand, F., et al. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2008.00807 · Cited by 21
“Beyond transfer learning, modern methods such as unsupervised or weakly/semi-supervised approaches, can learn representations yielding comparable performance to supervised learning (Jing & Tian, 2019) provided the domain gap is relatively small.”
MethodConterfactual Generative Zero-Shot Semantic SegmentationPreprint
Shen, F., Liu, J., & Hu, P. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2106.06360 · Cited by 13
“We generally use three groups of methods to train unsupervised representation: transformation-equivariant representations [9], self-supervised methods [10] and generative models [11].”
MethodImproving ClusterGAN Using Self-Augmented Information Maximization of Disentangling Latent SpacesPreprint
Dam, T., Anavatti, S. G., & Abbass, H. A. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2107.12706 · Cited by 13
Multi-source Few-shot Domain AdaptationPreprint
Yue, X., Zheng, Z., Reed, C., Das, H., Keutzer, K., & Sangiovanni-Vincentelli, A. L. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2109.12391 · Cited by 16
“SSL is a subset of unsupervised learning where supervision is automatically generated from the data [73, 74, 75, 76, 77, 78].”
BackgroundSelf-supervised feature learning for acoustic data analysis
Pala, A., Oleynik, A., Malde, K., & Handegard, N. O. (2024) · Ecological Informatics · doi:10.1016/j.ecoinf.2024.102878 · Cited by 11
UNet-Att: a self-supervised denoising and recovery model for two-photon microscopic image
Lu, Y., Ying, Y., Chen, L., Wang, Y., Jin, J. H., Jiang, X., Shuai, J., Li, X., & Zhong, J. (2024) · Complex & Intelligent Systems · doi:10.1007/s40747-024-01633-7 · Cited by 12
“To obviate the need for clean ground-truth image, self-supervised image denoising methods have emerged as a powerful alternative [45–47 ].”
Chen, S., Feng, Z., Yang, S., Ma, Y., Liu, J., & Qi, Z. (2024) · IEEE Transactions on Wireless Communications · doi:10.1109/twc.2024.3513980 · Cited by 10
“The primary goal of SSL is to acquire feature representations by pretraining a model that can be transferred to downstream tasks via pretext tasks [7], [8].”
From Global to Hybrid: A Review of Supervised Deep Learning for 2-D Image Feature Representation
Dong, X., Wang, Q., Deng, H., Yang, Z., Ruan, W., Liu, W., Liang, L., Wu, X. M., & Tian, Y. (2025) · IEEE Transactions on Artificial Intelligence · doi:10.1109/tai.2025.3526138 · Cited by 12
“D image feature representation is observed in application cases such as image retrieval [86], image recognition [49], [102] and image dehazing [117], [209], [210].”
Self-citationApplications of Computer Vision for Infectious Keratitis: A Systematic Review
Assaf, J. F., Ahuja, A. S., Kannan, V., Yazbeck, H., Krivit, J., & Redd, T. K. (2025) · Ophthalmology Science · doi:10.1016/j.xops.2025.100861 · Cited by 12
“56 Additionally, calibrating AI models so that probability estimates reflect true likelihoods of disease and systematically incorporating feedback from clinicians to iteratively refine models could enhance clinical relevance.”
Deep Feature Learning From Electromyographic Signals for Gesture Recognition Systems
Zhong, W., Jiang, X., Szymaniak, K., Jabbari, M., Ma, C., & Nazarpour, K. (2025) · IEEE Transactions on Neural Systems and Rehabilitation Engineering · doi:10.1109/tnsre.2025.3635419 · Cited by 14
“In the pretraining phase, the model learn from pretext tasks, which are auxiliary tasks that automatically generate data-label pairs from all input data points, whether labeled or not [51], [168].”
Liu, Z., Ye, S., Li, X., Luo, X., Pan, J., He, M., Qin, W., Xiao, J., Cai, Y., Shen, T., et al. (2026) · Computers and Electronics in Agriculture · doi:10.1016/j.compag.2026.111952 · Cited by 10
Gehlot, S., & Gupta, A. (2021) · Lecture notes in computer science · doi:10.1007/978-3-030-87589-3_49 · Cited by 12
Human Embryo Classification Using Self-Supervised Learning
Wicaksono, R. S. H., Septiandri, A. A., & Jamal, A. (2021) · International Conference on Artificial Intelligence and Data Sciences (AiDAS) · doi:10.1109/aidas53897.2021.9574328 · Cited by 10
Multimodal Violent Video Recognition Based on Mutual Distillation
Shang, Y., Wu, X., & Liu, R. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-18913-5_48 · Cited by 12
Self-Supervised Learning for the Distinction between Computer-Graphics Images and Natural Images
Wang, K. (2023) · Applied Sciences · doi:10.3390/app13031887 · Cited by 10
Disturbed Augmentation Invariance for Unsupervised Visual Representation Learning
Cheng, H., Li, H., Wu, Q., Qiu, H., Zhang, X., Meng, F., & Zhao, T. (2023) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2023.3272741 · Cited by 11
“The above methods are usually classified as generative methods in some papers [29], [30].”
MethodSelf-Supervised Learning for Seismic Data: Enhancing Model Interpretability With Seismic Attributes
Salazar, J. J., Maldonado-Cruz, E., Ochoa, J., & Pyrcz, M. J. (2023) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2023.3285820 · Cited by 11
“Once the secondary task is over, the model has learned intermediate representations that may contain valuable information to benefit a variety of downstream tasks like regression or classification [15].”
BackgroundDetecting Anomalies in System Logs With a Compact Convolutional Transformer
Larisch, R., Vitay, J., & Hamker, F. H. (2023) · IEEE Access · doi:10.1109/access.2023.3323252 · Cited by 10
“This procedure is considered as a self-supervised training scheme as it uses automatically generated labels and does not rely on human annotations [36].”
MethodDeep learning in optics—a tutorial
Hadad, B., Froim, S., Yosef, E., Giryes, R., & Bahabad, A. (2023) · Journal of Optics · doi:10.1088/2040-8986/ad08dc · Cited by 13
D4D: An RGBD Diffusion Model to Boost Monocular Depth Estimation
Papa, L., Russo, P., & Amerini, I. (2024) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2024.3404256 · Cited by 10
“This issue led researchers to focus on several techniques to reduce the data requirements, such as unsupervised [2] or self-supervised [3] learning strategies, with the objective of categorizing unlabeled or partially labeled data.”
BackgroundMethodNon-parametric Representation Learning with Kernels
Esser, P., Fleissner, M., & Ghoshdastidar, D. (2024) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v38i11.29077 · Cited by 19
Parsa, S., & Khatibi, T. (2024) · Multimedia Tools and Applications · doi:10.1007/s11042-024-18968-5 · Cited by 14
“1 Samples of IDRiD dataset images illustrate different severity grades of DR Self-supervised learning presents a potentially effective alternative to conventional supervised pre-training methods, as it leverages unlabeled data to pre-train neural networks [13].”
An Analysis of Initial Training Strategies for Exemplar-Free Class-Incremental Learning
Petit, G., Soumm, M., Feillet, E., Popescu, A., Delezoide, B., Picard, D., & Hudelot, C. (2024) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv57701.2024.00185 · Cited by 15
“A second family of methods considers a network of constant size throughout the incremental process (except the classifier) and proposes various strategies for obtaining models which ensure a balance between stability, i.e. preserving the performance for past classes, and plasticity, i.e. learning…”
BackgroundEfficient Masked Autoencoders With Self-Consistency
Li, Z., Zhu, Y., Chen, Z., Li, W., Zhao, R., Zhao, C., Tang, M., & Wang, J. (2024) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2024.3409826 · Cited by 12
“Self-supervised learning [1], [2], [3], [4], [5], [6], [7], [8], [9], [10], [11], [12], [13], [14], [15], [16], [17], [18], [19], [20], [21], [22], [23], [24], [25], [26], [27], [28], [29], [30], [42], [43], [44], [45] builds general representations by exploiting the internal priors or structures…”
Kang, S., Kim, S.-W., & Seong, J. (2024) · NeuroImage · doi:10.1016/j.neuroimage.2024.120737 · Cited by 14
“Self-supervised learning has emerged as a promising technique to capture complex features and generate biologically interpretable latent representations (Wang et al., 2021; Ericsson et al., 2022; Jing and Tian, 2020; Leeb et al., 2022).”
MethodHeterogeneous Network-Based Contrastive Learning Method for PolSAR Land Cover Classification
Cai, J., Ma, Y., Feng, Z., Yang, S., & Jiao, L. (2024) · IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing · doi:10.1109/jstars.2024.3429538 · Cited by 10
“In contrast to supervised learning, self-supervised learning (SSL) [33], where the data provide supervision, has the advantage of learning general representation from unlabeled data, which is more desirable and meaningful, and contrastive learning (CL), which constructs simple and easy-to-use…”
BackgroundSelf-supervised probabilistic models for exploring shape memory alloys
Wang, Y., Li, T., Zong, H., Ding, X., Xu, S., Sun, J., & Lookman, T. (2024) · npj Computational Materials · doi:10.1038/s41524-024-01379-3 · Cited by 10
Multimodal masked siamese network improves chest X-ray representation learning
Shurrab, S., Guerra-Manzanares, A., & Shamout, F. E. (2024) · Scientific Reports · doi:10.1038/s41598-024-74043-x · Cited by 12
Introduction to Domain Adaptation
Venkateswara, H., & Panchanathan, S. (2020) · Springer eBooks · doi:10.1007/978-3-030-45529-3_1 · Cited by 11
Learning by Distillation: A Self-Supervised Learning Framework for Optical Flow Estimation
Liu, P., Lyu, M. R.-T., King, I., & Xu, J. (2021) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2021.3085525 · Cited by 11
Forecasting the future clinical events of a patient through contrastive learning
Zhang, Z., Yan, C., Zhang, X., Nyemba, S., & Malin, B. (2022) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocac086 · Cited by 14
Learning neural decoders without labels using multiple data streams
Peterson, S. M., Rao, R. P. N., & Brunton, B. W. (2022) · Journal of Neural Engineering · doi:10.1088/1741-2552/ac857c · Cited by 14
“One promising approach is to use self-supervised learning techniques, which generate pseudo-labels from the data itself and then use those pseudo-labels to train a model iteratively without prior labels (see [21, 22] for comprehensive reviews of self-supervised approaches).”
MethodDenoised Labels for Financial Time Series Data via Self-Supervised Learning
Ma, Y., Ventre, C., & Polukarov, M. (2022) · ACM International Conference on AI in Finance (ICAIF) · doi:10.1145/3533271.3561687 · Cited by 13
Spikebench: An open benchmark for spike train time-series classification
Lazarevich, I. A., Prokin, I., Gutkin, B. S., & Kazantsev, V. (2023) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1010792 · Cited by 12
“We defined a set of 172 spike train classification tasks constructed in a self-supervised manner [35].”
MethodSelf-supervised Orientation-Guided Deep Network for Segmentation of Carbon Nanotubes in SEM Imagery
Nguyen, N. P., Surya, R., Maschmann, M. R., Calyam, P. P., Palaniappan, K., & Bunyak, F. (2023) · Lecture notes in computer science · doi:10.1007/978-3-031-25085-9_24 · Cited by 10
Learning Image Aesthetics by Learning Inpainting
Ching, J. H., See, J., & Wong, L.-K. (2020) · Proceedings - International Conference on Image Processing · doi:10.1109/icip40778.2020.9191130 · Cited by 10
“Across recent literature [19], there are four known categories of pretext tasks for SSL: generation-based, context-based, free semantic label-based and cross modal-based.”
BackgroundLeveraging Acoustic Images for Effective Self-supervised Audio Representation Learning
Sanguineti, V., Morerio, P., Pozzetti, N., Greco, D., Cristani, M., & Murino, V. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-58542-6_8 · Cited by 12
“More in detail, in deep self-supervised learning schemes, a network is trained to solve a so-called pretext task , and the quality of learned features is then assessed on a variety of downstream tasks , which are usually supervised (e.g., classification), showing beneficial effects [18].”
BackgroundSelf-Supervised Joint Encoding of Motion and Appearance for First Person Action Recognition
Planamente, M., Bottino, A., & Barbara Caputo, B. (2021) · Proceedings - International Conference on Pattern Recognition/Proceedings/International Conference on Pattern Recognition · doi:10.1109/icpr48806.2021.9411972 · Cited by 10
“Index Terms—Egocentric Vision, Action Recognition, Multitask Learning, Motion Prediction, Self-supervised Learning I. INTRODUCTION Recognizing human actions from videos is one of the most critical challenges in computer vision since its infancy.”
MethodPapadopoulos, A., & Delopoulos, A. N. (2023) · IEEE Journal of Biomedical and Health Informatics · doi:10.1109/jbhi.2023.3267095 · Cited by 9
“Approaches like self-supervised learning [25] and consistencybased regularization [26] before it, have been successfully applied to many image classification problems, achieving performance comparable to their fully-supervised counterparts, while using only a handful of labels [27], [28], [29].”
Unsupervised learning-based approach for detecting 3D edges in depth maps
Aggarwal, A., Stolkin, R., & Marturi, N. (2024) · Scientific Reports · doi:10.1038/s41598-023-50899-3 · Cited by 9
“The concept of unsupervised learning has been proposed in the literature to understand intrinsic patterns within the data 24 .”
BackgroundLin, L., Li, C., Zhang, T., Xia, C., Bai, Q., Jin, L., & Shen, Y. (2024) · Analytica Chimica Acta · doi:10.1016/j.aca.2024.342419 · Cited by 9
MSFFT: Multi-Scale Feature Fusion Transformer for cross platform vehicle re-identification
Holla, B. A., Pai, M. M. M., Verma, U., & Pai, R. M. (2024) · Neurocomputing · doi:10.1016/j.neucom.2024.127514 · Cited by 9
DCPoint: Global-Local Dual Contrast for Self-Supervised Representation Learning of 3-D Point Clouds
Shi, L., Zhang, G., Cao, Q. P., Zhang, L., Cen, Y., & Cen, Y. (2024) · IEEE Sensors Journal · doi:10.1109/jsen.2024.3405079 · Cited by 10
“As such, the models will not easily overfit with few labeled training data compared with the random initialization [29].”
BackgroundMethodA deep learning based interval type-2 fuzzy approach for image retrieval systems
Ghozzi, Y., Hamdani, T. M., Hagras, H., Ouahada, K., Chabchoub, H., & Alimi, A. M. (2024) · Neurocomputing · doi:10.1016/j.neucom.2024.128251 · Cited by 11
Ge, Y., Zhang, H., Shi, J., Luo, D., Chang, S., He, J. W., Huang, Q., & Wang, H. (2025) · Knowledge-Based Systems · doi:10.1016/j.knosys.2025.113090 · Cited by 11
Zou, X., Li, J., Wu, W., Liang, F., Yang, B., & Dong, Z. (2025) · ISPRS Journal of Photogrammetry and Remote Sensing · doi:10.1016/j.isprsjprs.2025.03.029 · Cited by 9
Bai, L., Zhang, X., Qin, W., Long, J., Wang, H., Dong, X., & Du, S. (2025) · IEEE Geoscience and Remote Sensing Magazine · doi:10.1109/mgrs.2025.3588505 · Cited by 10
“For the former, although many studies have different definitions of SSL, their overall understanding and core ideas are essentially consistent [30], [51].”
Metadata
- Authors
- Jing, L., & Tian, Y.
- Affiliations
- The Graduate Center, CUNY (US)
- Venue
- IEEE Transactions on Pattern Analysis and Machine Intelligence
- Published
- 2021-11-01
- Volume
- 43
- Issue
- 11
- Pages
- 4037-4058
- ISSN
- 0162-8828 (ISSN-L)
- 2160-9292
- 1939-3539
- OpenAlex
- W3023371261
- Cited by
- 2,036
- References
- 206
- Citations by year
- 2026: 160 · 2025: 283 · 2024: 354 · 2023: 405 · 2022: 377 · 2021: 305
- FWCI
- 119.51
- Type
- journal article
- Language
- en
- Open access
- green
- arxiv.org
- Licence
- Locations
- IEEE Transactions on Pattern Analysis and Machine Intelligence (publishedVersion)
- PubMed (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Funders
- National Science Foundation (IIS-1400802)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Domain Adaptation and Few-Shot Learning
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Image and Video Retrieval Techniques
- Keywords
- self-supervised learning, visual feature learning, imaging features, video features, deep learning architectures, unlabeled data, audio features, 3D data, benchmark dataset, evaluation metrics
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.