Self-Supervised Visual Feature Learning With Deep Neural Networks: A Survey
Open accessThe DOI matches its Crossref record; 204 of the 206 references it cites were found in a registry or an index.
Abstract
Large-scale labeled data are generally required to train deep neural networks in order to obtain better performance in visual feature learning from images or videos for computer vision applications. To avoid extensive cost of collecting and annotating large-scale datasets, as a subset of unsupervised learning methods, self-supervised learning methods are proposed to learn general image and video features from large-scale unlabeled data without using any human-annotated labels. This paper provides an extensive review of deep learning-based self-supervised general visual feature learning methods from images or videos. First, the motivation, general pipeline, and terminologies of this field are described. Then the common deep neural network architectures that used for self-supervised learning are summarized. Next, the schema and evaluation metrics of self-supervised learning methods are reviewed followed by the commonly used datasets for images, videos, audios, and 3D data, as well as the existing self-supervised visual feature learning methods. Finally, quantitative performance comparisons of the reviewed methods on benchmark datasets are summarized and discussed for both image and video feature learning. At last, this paper is concluded and lists a set of promising future directions for self-supervised visual feature learning.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Jing and Tian [18] summarize the self-supervised feature learning methods on image and video data, and four categories are discussed: generation-based, context-based, free semantic label-based, and cross modal-based, where cross-modal-based methods construct learning task using RGB frame sequence an optical flow sequence, which are unique features in the video.”
From Self-Supervised Learning for Time Series Analysis: Taxonomy, Progress, and Prospects (2024) · cites this paper as [18]
In the citing paper
“Another hot research sub-area is self-supervised learning, which learns discriminative visual representation by solving predefined pretext tasks [194].”
From Empowering things with intelligence: a survey of the progress, challenges, and opportunities in artificial intelligence of things (2021) · cites this paper as [194]
Citations
Cites
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474
Very Deep Convolutional Networks for Large-Scale Image Recognition
Simonyan, K., & Zisserman, A. (2014) · arXiv · doi:10.48550/arxiv.1409.1556 · Cited by 114,148 · cited 10×
“Figure is reproduced based on VGG [9].”
BackgroundMethodInfluentialImageNet classification with deep convolutional neural networks
Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017) · Communications of the ACM · doi:10.1145/3065386 · Cited by 112,155 · cited 8×
“Trained with accurate human-annotated labels, the supervised learning methods obtained break-through results on different computer vision applications [1], [4], [8], [16].”
MethodInfluentialHochreiter, S., & Schmidhuber, J. (1997) · Neural Computation · doi:10.1162/neco.1997.9.8.1735 · Cited by 102,318 · cited 1×
“Compared to standard RNN [72], long short term memory (LSTM) uses memory cells to store, modify, and access internal states, to better model the long-term temporal relationships within video frames [73].”
BackgroundFaster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks
Ren, S., He, K., Girshick, R., & Sun, J. (2017) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2016.2577031 · Cited by 73,472 · cited 2×
“DUE to the powerful ability to learn different levels of general visual features, deep neural networks have been used as the basic structure to many computer vision applications such as object detection [1], [2], [3], semantic segmentation [4], [5], [6], image captioning [7], etc.”
BackgroundMethodFaster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks
Ren, S., He, K., Girshick, R., & Sun, J. (2015) · arXiv · doi:10.48550/arxiv.1506.01497 · Cited by 70,812 · cited 2×
“DUE to the powerful ability to learn different levels of general visual features, deep neural networks have been used as the basic structure to many computer vision applications such as object detection [1], [2], [3], semantic segmentation [4], [5], [6], image captioning [7], etc.”
BackgroundMethodImageNet: A large-scale hierarchical image database
Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., & Fei-Fei, L. (2009) · 2009 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2009.5206848 · Cited by 65,333 · cited 5×
“• ImageNet: The ImageNet dataset [13] contains 1.”
BackgroundInfluentialGradient-based learning applied to document recognition
Lecun, Y., Bottou, L., Bengio, Y., & Haffner, P. (1998) · Proceedings of the IEEE · doi:10.1109/5.726791 · Cited by 58,805 · cited 1×
“• MNIST: The MNIST is a dataset of handwritten digits consisting of 70, 000 images while 60, 000 images belong to training set and the rest 10, 000 images are for testing [110].”
BackgroundDistinctive Image Features from Scale-Invariant Keypoints
Lowe, D. G. (2004) · International Journal of Computer Vision · doi:10.1023/b:visi.0000029664.99615.94 · Cited by 55,589
You Only Look Once: Unified, Real-Time Object Detection
Redmon, J., Divvala, S., Girshick, R., & Farhadi, A. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.91 · Cited by 53,803 · cited 1×
“Recently, many datasets such as MSCOCO [99] and OpenImage [14] have been proposed for object detection and many ConvNet-based models [1], [2], [3], [100], [101], [102], [103], [104] have been proposed and obtained great performance.”
Background
Cited by
Items 401–500 of 1,005
Hop-Count Based Self-supervised Anomaly Detection on Attributed Networks
Huang, T., Pei, Y., Menkovski, V., & Pechenizkiy, M. (2023) · Lecture notes in computer science · doi:10.1007/978-3-031-26387-3_14 · Cited by 73
Liu, M., Meng, F., & Liang, Y. (2022) · Cyborg and Bionic Systems · doi:10.34133/cbsystems.0002 · Cited by 44
“Following the formulation of self-supervised learning [13], a new and useful type of unsupervised learning, the key to the problem is to design a proper supervision signal that can drive the network to learn distinctive representation.”
MethodInfluentialOne-Shot Weakly-Supervised Segmentation in 3D Medical Images
Lei, W., Su, Q., Jiang, T., Gu, R., Wang, N., Liu, X., Wang, G., Zhang, X., & Zhang, S. (2023) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2023.3294975 · Cited by 48
Wang, H., & Li, Y. (2023) · International Conference on System Reliability and Safety Engineering (SRSE) · doi:10.1109/srse59585.2023.10336112 · Cited by 58
“[19].”
Lee, J., & Sohn, S. Y. (2020) · Scientometrics · doi:10.1007/s11192-020-03731-y · Cited by 42
“Self-supervised learning is a subset of unsupervised learning methods which converts the problem into supervised learning by creating surrogate labels from the unlabeled dataset of target task (Kolesnikov et al. 2019; Jing et al. 2020).”
BackgroundLi, K., Qin, Y., Ling, Q., Wang, Y., Lin, Z., & An, W. (2021) · IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing · doi:10.1109/jstars.2021.3063335 · Cited by 47
“Self-supervised learning aims at learning general features without using any human-annotated labels [36].”
BackgroundA Dense Neural Network Approach for Detecting Clone ID Attacks on the RPL Protocol of the IoT
Morales-Molina, C. D., Hernandez-Suarez, A., Sánchez-Pérez, G., Toscano, K., Perez-Meana, H. M., Olivares-Mercado, J., Portillo-Portillo, J., Sánchez, V., & Villalba, L. J. G. (2021) · Sensors · doi:10.3390/s21093173 · Cited by 46
“For the most part, the hidden units in Layer L2 learn from a lower-dimensional representation of the input, as this layer has fewer units [56].”
BackgroundLabels in a haystack: Approaches beyond supervised learning in biomedical applications
Yakimovich, A., Beaugnon, A., Huang, Y., & Özkırımlı, E. (2021) · Patterns · doi:10.1016/j.patter.2021.100383 · Cited by 50
“17,75,76 Overall, these approaches attempt to improve the efficiency of labeled data utilization by high expressive capacity models.”
BackgroundImportance of Self-Consistency in Active Learning for Semantic Segmentation
Golestaneh, S. A., & Kitani, K. (2020) · British Machine Vision Conference (BMVC) · doi:10.5244/c.34.3 · Cited by 50
RecON: Online learning for sensorless freehand 3D ultrasound reconstruction
Luo, M., Yang, X., Wang, H., Dou, H., Hu, X., Huang, Y., Ravikumar, N., Xu, S., Zhang, Y., Xiong, Y., et al. (2023) · Medical Image Analysis · doi:10.1016/j.media.2023.102810 · Cited by 51
Self-Supervised Relational Reasoning for Representation LearningPreprint
Patacchiola, M., & Storkey, A. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2006.05849 · Cited by 65
“classification and image retrieval (Jing and Tian, 2020).”
BackgroundMethodInfluentialRevisiting Contrastive Methods for Unsupervised Learning of Visual RepresentationsPreprint
Gansbeke, W. V., Vandenhende, S., Georgoulis, S., & Gool, L. V. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2106.05967 · Cited by 76
“SSL methods obtain positives and negatives by treating each image as a separate class [58].”
BackgroundInfluentialToward Understanding the Feature Learning Process of Self-supervised Contrastive LearningPreprint
Wen, Z., & Li, Y. (2021) · International Conference on Machine Learning · doi:10.48550/arxiv.2105.15134 · Cited by 174
“Self-supervised learning (Devlin et al., 2019; Mikolov et al., 2013; Sutskever et al., 2014; Jing & Tian, 2020) has demonstrated its immense power in different areas of machine learning (e.g. BERT (Devlin et al., 2019) in natural language processing).”
BackgroundBreaking away from labels: The promise of self-supervised machine learning in intelligent health
Spathis, D., Perez-Pozuelo, I., Marques-Fernandez, L., & Mascolo, C. (2022) · Patterns · doi:10.1016/j.patter.2021.100410 · Cited by 39
“8 However, the two terms are sometimes used interchangeably.)”
BackgroundA Review of Transformer-Based Approaches for Image Captioning
Ondeng, O., Absaloms, H. O., & Akuon, P. O. (2023) · Applied Sciences · doi:10.3390/app131911103 · Cited by 44
Mohammad‐Rahimi, H., Dianat, O., Abbasi, R., Zahedrozegar, S., Ashkan, A., Motamedian, S. R., Rohban, M. H., & Nosrat, A. (2023) · Journal of Endodontics · doi:10.1016/j.joen.2023.11.004 · Cited by 46
Geo-Foundation Models: Reality, Gaps and Opportunities
Xie, Y., Wang, Z., Mai, G., Li, Y., Jia, X., Gao, S., & Wang, S. (2023) · ACM SIGSPATIAL International Conference on Advances in Geographic Information Systems (SIGSPATIAL GIS) · doi:10.1145/3589132.3625616 · Cited by 44
“The success of ChatGPT-type of foundational LLMs (and broad generative models) has two important ingredients: (1) A very large model with a number of parameters at billions-to-trillions level; and (2) Self-supervised learning (SSL) based pretraining [11].”
BackgroundMultimodal image encoding pre-training for diabetic retinopathy grading
Hervella, Á. S., Rouco, J., Novo, J., & Ortega, M. (2022) · Computers in Biology and Medicine · doi:10.1016/j.compbiomed.2022.105302 · Cited by 49
Few-Shot Scene Classification of Optical Remote Sensing Images Leveraging Calibrated Pretext Tasks
Ji, H., Gao, Z. Y., Zhang, Y., Wan, Y., Li, C., & Mei, T. (2022) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2022.3184080 · Cited by 44
Machine learning in cartography
Harrie, L., Touya, G., Oucheikh, R., Ai, T., Courtial, A., & Richter, K. (2024) · Cartography and Geographic Information Science · doi:10.1080/15230406.2023.2295948 · Cited by 42
“However, a change of perspective has emerged, and researchers have worked on less data-demanding meth-ods (Jing & Tian, 2019).”
BackgroundSIEGE: Self-Supervised Incremental Deep Graph Learning for Ethereum Phishing Scam Detection
Li, S., Wang, R., Wu, H., Zhong, S., & Xu, F. (2023) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3581783.3612461 · Cited by 51
“For applying SSL in graphs, rich structural information could be both inspiration and barrier for the pretext tasks design [22].”
MethodSenane, Z., Cao, L., Buchner, V., Tashiro, Y., You, L., Herman, P. A., Nordahl, M. G., Tu, R., & von Ehrenheim, V. (2024) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/3637528.3671673 · Cited by 58
SVFAP: Self-Supervised Video Facial Affect Perceiver
Sun, L., Lian, Z., Wang, K., He, Y., Xu, M., Sun, H., Liu, B., & Tao, J. (2024) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2024.3436913 · Cited by 52
“Specifically, in contrast to supervised learning, self-supervised learning leverages input data itself as the supervisory signal for self-supervised pre-training and thus can learn powerful representations from large-scale data without using any human-annotated labels [19], [20].”
Self-supervised learning for label-free segmentation in cardiac ultrasound
Ferreira, D. L., Lau, C., Salaymang, Z., & Arnaout, R. (2025) · Nature Communications · doi:10.1038/s41467-025-59451-5 · Cited by 44
Self-supervised Feature Learning by Cross-modality and Cross-view Correspondences
Jing, L., Zhang, L., & Tian, Y. (2021) · IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw53098.2021.00174 · Cited by 45
Self-citationUnsupervised Outlier Detection Using Memory and Contrastive Learning
Huyan, N., Quan, D., Zhang, X., Liang, X., Chanussot, J., & Jiao, L. (2022) · IEEE Transactions on Image Processing · doi:10.1109/tip.2022.3211476 · Cited by 42
“trastive learning [12], [13], [14] is applied to learn more robust semantic features.”
BackgroundSemi-Supervised Contrastive Learning for Generalizable Motor Imagery EEG Classification
Han, J., Gu, X., & Lo, B. P. L. (2021) · Wearable and Implantable Body Sensor Networks · doi:10.1109/bsn51625.2021.9507038 · Cited by 48
“Self-supervised learning (SSL) has opened the possibility of making use of self-generated pseudo labels for training on unlabelled data, with limited access to ground truth labels [9].”
BackgroundMahmoud, H., Wu, W., & Gaber, M. M. (2022) · Energies · doi:10.3390/en15030914 · Cited by 40
Li, W., Hacid, H., Almazrouei, E., & Debbah, M. A. (2023) · AI · doi:10.3390/ai4030039 · Cited by 47
Masked Event Modeling: Self-Supervised Pretraining for Event Cameras
Klenk, S., Bonello, D., Koestler, L., Araslanov, N., & Cremers, D. (2024) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv57701.2024.00237 · Cited by 61
“The idea of SSL is to first train a network on an unlabeled dataset by solving a pretext task [27] and then to finetune the network on a downstream task.”
BackgroundLeSAM: Adapt Segment Anything Model for Medical Lesion Segmentation
Gu, Y., Wu, Q., Tang, H. N., Mai, X., Shu, H., Li, B., & Chen, Y. (2024) · IEEE Journal of Biomedical and Health Informatics · doi:10.1109/jbhi.2024.3406871 · Cited by 41
Self-Supervised Learning to Increase the Performance of Skin Lesion Classification
Kwasigroch, A., Grochowski, M., & Mikołajczyk, A. (2020) · Electronics · doi:10.3390/electronics9111930 · Cited by 40
“A different approach taken in this paper, which bears the name of self-supervised learning [10], consists of introducing a pretraining stage in the form of learning the features from unlabeled datasets.”
Hyperspectral Image Classification with Feature-Oriented Adversarial Active Learning
Wang, G., & Ren, P. (2020) · Remote Sensing · doi:10.3390/rs12233879 · Cited by 38
“Overall, the GAN undertakes a pretext task [36] for the mainstream classification task of the classifier.”
MethodBooth, T. C., Grzeda, M. T., Chelliah, A., Roman, A., Busaidi, A. A., Dragos, C., Shuaib, H., Luis, A., Mirchandani, A., Alparslan, B., et al. (2022) · Frontiers in Oncology · doi:10.3389/fonc.2022.799662 · Cited by 43
3-D CNN-Based Multichannel Contrastive Learning for Alzheimer’s Disease Automatic Diagnosis
Li, J., Wei, Y. J., Wang, C., Hu, Q., Liu, Y., & Xu, L. (2022) · IEEE Transactions on Instrumentation and Measurement · doi:10.1109/tim.2022.3162265 · Cited by 64
Parcollet, T., Nguyen, H., Evain, S. V., Boito, M. Z., Pupier, A., Mdhaffar, S., Le, H., Alisamir, S., Tomashenko, N., Dinarelli, M., et al. (2024) · Computer Speech & Language · doi:10.1016/j.csl.2024.101622 · Cited by 59
Foundation Models in Computational Pathology: A Review of Challenges, Opportunities, and ImpactPreprint
Bilal, M., Aadam, Raza, M., Altherwy, Y. N., Alsuhaibani, A., Abduljabbar, A., Almarshad, F. A., Golding, P., & Rajpoot, N. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2502.08333 · Cited by 66
Pseudo-Data Based Self-Supervised Federated Learning for Classification of Histopathological Images
Zhang, Y., Li, Z., Han, X., Ding, S., Li, J., Wang, J., Ying, S., & Shi, J. (2023) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2023.3323540 · Cited by 38
“Self-supervised learning (SSL) then provides a feasible way to explore and learn inherent information from these pseudo histopathological images, because it generates supervision directly from the training samples themselves to design pretext tasks [25], [26].”
BackgroundBlockchain-based biometric identity management
Salem, S. H. G., Hassan, A. Y., Moustafa, M. S., & Hassan, M. N. (2023) · Cluster Computing · doi:10.1007/s10586-023-04180-x · Cited by 38
“In real situations, the collected dataset is small size and the goal is to train powerful discriminative mode using very few samples for each individual or identity (zero/one shot in extreme case) [14].”
Generic Knowledge Boosted Pretraining for Remote Sensing Images
Huang, Z., Zhang, M., Gong, Y., Liu, Q., & Wang, Y. (2024) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2024.3354031 · Cited by 50
“Jing and Tian [27] summarized the pretext tasks into four categories: generation-based, context-based, free semantic label-based, and cross-modal-based.”
BackgroundMethodIdentification of Geochemical Anomalies Using an End-to-End Transformer
Yu, S., Deng, H., Liu, Z., Chen, J., Xiao, K., & Mao, X. (2024) · Natural Resources Research · doi:10.1007/s11053-024-10334-4 · Cited by 42
“This leads to a self-super-vised learning scheme (Jing and Tian, 2021; Liu et al., 2023), a branch of unsupervised learning, for geochemical anomaly recognition.”
Is it enough to optimize CNN architectures on ImageNet?
Tuggener, L., Schmidhuber, J., & Stadelmann, T. (2022) · Frontiers in Computer Science · doi:10.3389/fcomp.2022.1041703 · Cited by 38
“Self-supervised learning of visual representations Jing and Tian (2019) could serve as remedy for this issue.”
BackgroundA Self-Supervised Learning Based Channel Attention MLP-Mixer Network for Motor Imagery Decoding
He, Y., Lu, Z., Wang, J., Ying, S., & Shi, J. (2022) · IEEE Transactions on Neural Systems and Rehabilitation Engineering · doi:10.1109/tnsre.2022.3199363 · Cited by 36
“Self-supervised learning (SSL) has attracted considerable attention as an effective way for few-shot learning [15], [16].”
BackgroundWang, R., Chen, H., & Guan, C. (2023) · Ocean Engineering · doi:10.1016/j.oceaneng.2022.113437 · Cited by 38
Self-Supervised Global–Local Contrastive Learning for Fine-Grained Change Detection in VHR Images
Jiang, F., Gong, M., Zheng, H., Liu, T., Zhang, M., & Liu, J. (2023) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2023.3238327 · Cited by 44
“In general, the self-supervised meth-ods leverage supervised information available from the data itself and raise various pretext tasks for models to solve, while the features can be learned through this process [28], [29].”
BackgroundAuxiliary Learning by Implicit DifferentiationPreprint
Navon, A., Achituve, I., Maron, H., Chechik, G., & Fetaya, E. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2007.02693 · Cited by 48
“For self-supervised learning (SSL), many approaches have been proposed (see [21] for a recent survey), but the joint representation learned through SSL may suffer from negative transfer and hurt the main task [43].”
BackgroundA Comprehensive Survey on Self-Supervised Learning for Specific Emitter Identification
Liu, C., Gui, G., Wang, Y., Ohtsuki, T. O., Niyato, D. T., & Shen, X. (2025) · IEEE Communications Surveys & Tutorials · doi:10.1109/comst.2025.3588171 · Cited by 41
“Self-supervised learning (SSL), a subset of unsupervised learning (USL) that constructs pre-text task to learn useful representations without labels, has achieved remarkable success in fields such as Computer Vision (CV) [12] and natural language processing [13].”
Noise Estimation Using Density Estimation for Self-Supervised Multimodal Learning
Amrani, E., Ben‐Ari, R., Rotman, D., & Bronstein, A. (2021) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v35i8.16822 · Cited by 83
What machine learning can do for developmental biology
Villoutreix, P. (2021) · Development · doi:10.1242/dev.188474 · Cited by 38
“Finally, machine learning is a quickly changing field; new paradigms such as federated learning (Yang et al., 2019) or self-supervised learning (Jing and Tian, 2020) are emerging.”
BackgroundMethodYang, M., Jiao, L., Liu, F., Hou, B., Yang, S., Zhang, Y., & Wang, J. (2022) · IEEE Transactions on Image Processing · doi:10.1109/tip.2022.3211472 · Cited by 37
“With the development of self-supervised learning (SSL), it is possible to learn visual features from unlabeled data [32].”
BackgroundAlbert-Weiss, D., & Osman, A. (2022) · Sensors · doi:10.3390/s22020414 · Cited by 37
“It is largely motivated by the idea of learning meaningful representations by optimising the unlabelled dataset based on on the labelled one [53].”
BackgroundExploring Edge Disentanglement for Node Classification
Zhao, T., Zhang, X. D., & Wang, S. (2022) · International World Wide Web Conference (WWW) · doi:10.1145/3485447.3511929 · Cited by 45
“Besides its success in computer vision [14] and natural language processing [18], there is a trend of developing this idea in the graph learning domain [12, 19].”
BackgroundSelf-Supervised Learning Methods for Label-Efficient Dental Caries Classification
Taleb, A., Rohrer, C., Bergner, B., Leon, G. D., Rodrigues, J. A., Schwendicke, F., Lippert, C., & Krois, J. (2022) · Diagnostics · doi:10.3390/diagnostics12051237 · Cited by 42
“Self-supervised learning has found many application fields [7], and has shown to improve the data and annotation efficiency of deep learning models.”
BackgroundSemi-Supervised Learning for Joint SAR and Multispectral Land Cover Classification
Montanaro, A., Valsesia, D., Fracastoro, G., & Magli, E. (2022) · IEEE Geoscience and Remote Sensing Letters · doi:10.1109/lgrs.2022.3195259 · Cited by 35
“A wide variety of pretext tasks have been proposed [4].”
BackgroundLongo, L. (2022) · Brain Sciences · doi:10.3390/brainsci12101416 · Cited by 37
“Contrary to all the existing methods of cognitive load modeling, the method proposed here is self-supervised [57,58].”
MethodScenario-based test reduction and prioritization for multi-module autonomous driving systems
Deng, Y., Zheng, X., Zhang, M., Lou, G., & Zhang, T. (2022) · ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE) · doi:10.1145/3540250.3549152 · Cited by 52
“proposed to use self-supervised learning [32] to learn better feature representations for driving recordings.”
BackgroundLi, Y., Li, X., Zhang, Y., Peng, D., & Bruzzone, L. (2023) · International Journal of Applied Earth Observation and Geoinformation · doi:10.1016/j.jag.2023.103345 · Cited by 46
Self-supervised learning for remote sensing scene classification under the few shot scenario
Alosaimi, N., Alhichri, H. S., Bazi, Y., Youssef, B. B., & Alajlan, N. (2023) · Scientific Reports · doi:10.1038/s41598-022-27313-5 · Cited by 39
“potentially useful knowledge from unlabeled data, thus eliminating the need for costly labeling efforts 12 .”
BackgroundChen, Z., Pan, Y., & Xia, Y. (2023) · IEEE Journal of Biomedical and Health Informatics · doi:10.1109/jbhi.2023.3266576 · Cited by 37
“It can be attributed to the fact that the reconstruction alignment can provide more self-supervised guidance from a large number of images [52].”
Li, G., Shi, G., & Jiao, J. (2023) · Sensors · doi:10.3390/s23115242 · Cited by 36
“This is not conducive to intelligent [4–6], healthy, and precise pig farming [7,8].”
BackgroundAl‐Dahidi, S., Alrbai, M., Rinchi, B., Alahmer, H., Al‐Ghussain, L., Hayajneh, H. S., & Alahmer, A. I. (2025) · Energy · doi:10.1016/j.energy.2025.135877 · Cited by 33
Harnessing Text Insights With Visual Alignment for Medical Image Segmentation
Zeng, Q., Luo, H., Lu, Z., Xie, Y., Wang, Z., Zhang, Y., & Xia, Y. (2025) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2025.3601359 · Cited by 62
CLAD: Robust audio deepfake detection against manipulation attacks with contrastive learning
Wu, H., Chen, J., Du, R., Wu, C., He, K., Shang, X., Ren, H., & Xu, G. (2025) · Knowledge-Based Systems · doi:10.1016/j.knosys.2025.115179 · Cited by 46
Designing accurate emulators for scientific processes using calibration-driven deep models
Thiagarajan, J. J., Venkatesh, B., Anirudh, R., Bremer, P., Gaffney, J., Anderson, G. J., & Spears, B. K. (2020) · Nature Communications · doi:10.1038/s41467-020-19448-8 · Cited by 38
Moro, G., & Valgimigli, L. (2021) · Sensors · doi:10.3390/s21196430 · Cited by 35
“Different loss functions have been proposed based on this kind of relationships, such as soft triplet loss [27], angular loss [28], multi-similarity loss [29]; a comprehensive study in [30] collects and evaluates several variants under the new research thread of deep metric learning and metric loss functions [31,32].”
BackgroundEfficient multi-view clustering networks
Ke, G., Hong, Z., Yu, W., Zhang, X., & Liu, Z. (2022) · Applied Intelligence · doi:10.1007/s10489-021-03129-0 · Cited by 35
“Pseudo label generation is a vital sub-task in self-supervised learning [21].”
BackgroundWang, X., Zhu, J., Yan, Z., Zhang, Z., Zhang, Y., Chen, Y., & Li, H. (2022) · IEEE Geoscience and Remote Sensing Letters · doi:10.1109/lgrs.2022.3185088 · Cited by 37
“3) Label-free contextual learning: Two network architectures are often used as the backbone in SSL: convolutional neural networks (CNNs) [6] and vision transformers (ViTs) [14].”
MethodSelf-supervised Learning for Sonar Image Classification
Preciado-Grijalva, A., Wehbe, B., Firvida, M. B., & Valdenegro-Toro, M. (2022) · IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw56347.2022.00156 · Cited by 47
“SSL has been mainly used in color images demonstrating not only the capability to reach same performance than pure supervised learning methods with much less labeled data for many tasks, but even surpassing their performance in some cases [19].”
Domínguez, C., Heras, J., Mata, E. J., Pascual, V., Fibla, D. R., & Zapata, M. Á. (2022) · Computer Methods and Programs in Biomedicine · doi:10.1016/j.cmpb.2022.107302 · Cited by 34
Exploiting Unlabelled Photos for Stronger Fine-Grained SBIR
Sain, A., Bhunia, A. K., Koley, S., Chowdhury, P. N., Chattopadhyay, S., Xiang, T., & Song, Y.-Z. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.00664 · Cited by 40
“The former spans over a large literature [40], including generative models [29] like VAEs [41], contrastive learning ap-2 proaches [16,33], clustering [13], etc.”
BackgroundBuilding towards Automated Cyberbullying Detection: A Comparative Analysis
Al-Harigy, L. M., Al-Nuaim, H. A., Moradpoor, N., & Tan, Z. (2022) · Computational Intelligence and Neuroscience · doi:10.1155/2022/4794227 · Cited by 32
“(i) It is used to avoid time-consuming and expensive data annotations by learning features from largescale unlabelled images or videos without using any human annotations [95].”
BackgroundArtopoulos, G., Maslioukova, M. I., Zavou, C., Loizou, M., Deligiorgi, M., & Averkiou, M. (2023) · Journal of Cultural Heritage · doi:10.1016/j.culher.2023.07.016 · Cited by 33
TDAD: Self-supervised industrial anomaly detection with a two-stage diffusion model
Wei, C., Han, H., Xia, Y., & Ji, Z. (2024) · Computers in Industry · doi:10.1016/j.compind.2024.104192 · Cited by 38
Image Search with Text Feedback by Deep Hierarchical Attention Mutual Information Maximization
Gu, C., Bu, J., Zhang, Z., Yu, Z. H., Ma, D., & Wang, W. (2021) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3474085.3475619 · Cited by 39
“Since mutual information between different modalities usually has higher semantic meaning compared with information that is modality-specific, people verify whether two input data correspond to each other by capturing mutual information between the two modalities [33, 52].”
BackgroundTrusting our machines: validating machine learning models for single-molecule transport experiments
Bro‐Jørgensen, W., Hamill, J. M., Bro, R., & Solomon, G. C. (2022) · Chemical Society Reviews · doi:10.1039/d1cs00884f · Cited by 34
Zhao, H., Zhou, Z., Wu, F., Xiang, D., Zhao, H. Z., We, Z., Li, L., Li, Z., Huang, J. M., Hu, H., et al. (2022) · Cell Reports Medicine · doi:10.1016/j.xcrm.2022.100775 · Cited by 32
Pruning Convolutional Neural Networks with Self-SupervisionPreprint
Caron, M., Morcos, A. S., Bojanowski, P., Mairal, J., & Joulin, A. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2001.03554 · Cited by 35
“Indeed, it has been observed that most self-supervised approaches produce good shallow and mid-level features but poorer high level features [41].”
Deep Learning based Cell Classification in Imaging Flow Cytometer
Gu, Y., Chen, A., Zhang, X., Fan, C., Li, K., & Shen, J. (2021) · ASP Transactions on Pattern Recognition and Intelligent Systems · doi:10.52810/tpris.2021.100050 · Cited by 31
Li, D., Wang, X., Zhang, J., & Ji, Z. (2023) · Global Energy Interconnection · doi:10.1016/j.gloei.2023.10.008 · Cited by 33
Abdullahi, M., Oyelade, O. N., Kana, A. F. D., Bagiwa, M. A., Abdullahi, F. B., Junaidu, S. B., Iliyasu, I., Ore-Ofe, A., & Chiroma, H. (2024) · Multimedia Tools and Applications · doi:10.1007/s11042-024-19823-3 · Cited by 37
Cross- and Intra-Image Prototypical Learning for Multi-Label Disease Diagnosis and Interpretation
Wang, C., Liu, F., Chen, Y., Frazer, H. M. L., & Carneiro, G. (2025) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2025.3541830 · Cited by 39
“In line with popular self-supervised learning techniques [60], we adopt two different views, denoted by x b , x b ′ , augmented from the same training image, as demonstrated in Fig.”
Advancing Volumetric Medical Image Segmentation via Global-Local Masked Autoencoders
Zhuang, J., Luo, L., Wang, Q., Wu, M., Luo, L., & Chen, H. (2025) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2025.3569782 · Cited by 51
“Self-Supervised Learning (SSL) approaches [6] have emerged as effective solutions by involving pre-training on large sets of unlabeled data, followed by fine-tuning on smaller labeled datasets.”
A Survey of Defenses Against AI-Generated Visual Media: Detection, Disruption, and Authentication
Deng, J., Lin, C., Zhao, Z., Liu, S., Peng, Z., Wang, Q., & Shen, C. Q. (2025) · ACM Computing Surveys · doi:10.1145/3770916 · Cited by 45
“Beyond disruption itself, its trustworthiness concern of robustness is threatened by adversarial purification, which aims to eliminate the perturbations and thereby leads to ineffective disruption.”
Zheng, H., Wu, K., Park, J.-H., Zhu, W., & Luo, J. (2021) · IEEE International Conference on Big Data (BigData) · doi:10.1109/bigdata52589.2021.9671563 · Cited by 31
“In [8], [15], `2 distance d(i, j) = |i −j |(2)2 is used as the item-wise dis-similarity measurement.”
BackgroundTavakolian, M., López, M. B., & Liu, L. (2020) · Pattern Recognition Letters · doi:10.1016/j.patrec.2020.09.012 · Cited by 37
“To avoid time-consuming and costly data annotations, self-supervised learning has been put forward to learn visual representations from large-scale unlabeled data (Jing and Tian (2020)).”
MethodCensus-independent population estimation using representation learning
Neal, I., Seth, S., Watmough, G. R., & Diallo, M. S. (2022) · Scientific Reports · doi:10.1038/s41598-022-08935-1 · Cited by 34
“Compared to supervised learning which maximizes the objective function of a pre-defined task such as classification, self-supervised learning generates pseudolabels from a pretext task in the absence of ground truth data, and different algorithms differ in the way they define the pretext tasks [14].”
BackgroundArtificial Intelligence Methodologies for Data Management
Serey, J., Quezada, L., Alfaro, M. D., Fuertes, G., Vargas, M., Ternero, R., Sabattin, J., Durán, C., & Gutiérrez, S. (2021) · Symmetry · doi:10.3390/sym13112040 · Cited by 36
“Characterization of images and videos Classification/Survey [49] Self-supervised approaches ANN”
Socially Enhanced Situation Awareness from Microblogs Using Artificial Intelligence: A Survey
Lamsal, R., Harwood, A., & Rodriguez, M. A. (2022) · ACM Computing Surveys · doi:10.1145/3524498 · Cited by 35
“Some methods to visual feature learning include learning through image colorization, image inpainting, image super-resolution, image generation with Generative Adversarial Networks (GANs) [50] and game engines, context similarity and spatial relations between image patches, shuffling of image patches, image clustering…”
MethodYan, Z., & Liu, H. (2022) · Mathematics · doi:10.3390/math10152796 · Cited by 34
Getting the bugs out of AI : Advancing ecological research on arthropods through computer vision
Schneider, S., Taylor, G. W., Kremer, S. C., & Fryxell, J. M. (2023) · Ecology Letters · doi:10.1111/ele.14239 · Cited by 34
DL4ALL: Multi-Task Cross-Dataset Transfer Learning for Acute Lymphoblastic Leukemia Detection
Genovese, A., Piuri, V., Plataniotis, K. N., & Scotti, F. (2023) · IEEE Access · doi:10.1109/access.2023.3289219 · Cited by 34
“ing usually performs better than unsupervised learning [47].”
BackgroundBetter Safe Than Sorry: Preventing Delusive Adversaries with Adversarial TrainingPreprint
Tao, L., Feng, L., Yi, J., Huang, S.-J., & Chen, S. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2102.04716 · Cited by 82
“These results show the possibility of delusive attacks and defenses for SSL, and suggest that studying the robustness of other SSL methods [58, 17] against data poisoning is a promising direction for future research.”
FOOL: Addressing the Downlink Bottleneck in Satellite Computing With Neural Feature Compression
Furutanpey, A., Zhang, Q., Raith, P., Pfandzelter, T., Wang, S., & Dustdar, S. (2025) · IEEE Transactions on Mobile Computing · doi:10.1109/tmc.2025.3544516 · Cited by 42
“Second, even if the trend toward transfer learning with foundational models [44], [45] can accommodate various predictors, client preferences for architectures may vary.”
Huang, Y., Liu, Y., Steel, P. A. D., Axsom, K., Lee, J. R., Tummalapalli, S. L., Wang, F., Pathak, J., Subramanian, L., & Zhang, Y. (2021) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocab203 · Cited by 33
Self-Supervised Clustering for Leaf Disease Identification
Monowar, M. M., Hamid, M. A., Kateb, F., Ohi, A. Q., & Mridha, M. F. (2022) · Agriculture · doi:10.3390/agriculture12060814 · Cited by 34
Zhao, Z., Hu, J., Zeng, Z., Yang, X., Qian, P., Veeravalli, B., & Guan, C. (2022) · Proceedings - International Conference on Image Processing · doi:10.1109/icip46576.2022.9897591 · Cited by 30
“Recently, self-supervised learning (SSL) has received much attention because of its powerful ability to learn imagelevel representations from unlabeled data, which is beneficial to various downstream tasks, such as classification [11].”
Sun, Y., Wang, Y., Liu, H., Hu, L., Zhang, C., & Wang, S. (2023) · Remote Sensing · doi:10.3390/rs15030708 · Cited by 30
Chen, J., He, P., Zhu, J., Guo, Y., Sun, G., Deng, M., & Li, H. (2023) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2023.3262576 · Cited by 36
“Nowadays, the self-supervised learning model can be pretrained using a large number of unlabeled images and then be fine-tuned on a downstream task with a small number of labeled samples [24].”
BackgroundAn overview on Meta-learning approaches for Few-shot Weakly-supervised Segmentation
Gama, P. H. T., Oliveira, H., dos Santos, J. A., & Junior, R. M. C. (2023) · Computers & Graphics · doi:10.1016/j.cag.2023.05.009 · Cited by 31
Deep learning for time series classificationPreprint
Fawaz, H. I. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2010.00567 · Cited by 38
Metadata
- Authors
- Jing, L., & Tian, Y.
- Affiliations
- The Graduate Center, CUNY (US)
- Venue
- IEEE Transactions on Pattern Analysis and Machine Intelligence
- Published
- 2021-11-01
- Volume
- 43
- Issue
- 11
- Pages
- 4037-4058
- ISSN
- 0162-8828 (ISSN-L)
- 2160-9292
- 1939-3539
- OpenAlex
- W3023371261
- Cited by
- 2,036
- References
- 206
- Citations by year
- 2026: 160 · 2025: 283 · 2024: 354 · 2023: 405 · 2022: 377 · 2021: 305
- FWCI
- 119.51
- Type
- journal article
- Language
- en
- Open access
- green
- arxiv.org
- Licence
- Locations
- IEEE Transactions on Pattern Analysis and Machine Intelligence (publishedVersion)
- PubMed (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Funders
- National Science Foundation (IIS-1400802)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Domain Adaptation and Few-Shot Learning
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Image and Video Retrieval Techniques
- Keywords
- self-supervised learning, visual feature learning, imaging features, video features, deep learning architectures, unlabeled data, audio features, 3D data, benchmark dataset, evaluation metrics
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.