Self-Supervised Visual Feature Learning With Deep Neural Networks: A Survey
Open accessThe DOI matches its Crossref record; 204 of the 206 references it cites were found in a registry or an index.
Abstract
Large-scale labeled data are generally required to train deep neural networks in order to obtain better performance in visual feature learning from images or videos for computer vision applications. To avoid extensive cost of collecting and annotating large-scale datasets, as a subset of unsupervised learning methods, self-supervised learning methods are proposed to learn general image and video features from large-scale unlabeled data without using any human-annotated labels. This paper provides an extensive review of deep learning-based self-supervised general visual feature learning methods from images or videos. First, the motivation, general pipeline, and terminologies of this field are described. Then the common deep neural network architectures that used for self-supervised learning are summarized. Next, the schema and evaluation metrics of self-supervised learning methods are reviewed followed by the commonly used datasets for images, videos, audios, and 3D data, as well as the existing self-supervised visual feature learning methods. Finally, quantitative performance comparisons of the reviewed methods on benchmark datasets are summarized and discussed for both image and video feature learning. At last, this paper is concluded and lists a set of promising future directions for self-supervised visual feature learning.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Jing and Tian [18] summarize the self-supervised feature learning methods on image and video data, and four categories are discussed: generation-based, context-based, free semantic label-based, and cross modal-based, where cross-modal-based methods construct learning task using RGB frame sequence an optical flow sequence, which are unique features in the video.”
From Self-Supervised Learning for Time Series Analysis: Taxonomy, Progress, and Prospects (2024) · cites this paper as [18]
In the citing paper
“Another hot research sub-area is self-supervised learning, which learns discriminative visual representation by solving predefined pretext tasks [194].”
From Empowering things with intelligence: a survey of the progress, challenges, and opportunities in artificial intelligence of things (2021) · cites this paper as [194]
Citations
Cites
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474
Very Deep Convolutional Networks for Large-Scale Image Recognition
Simonyan, K., & Zisserman, A. (2014) · arXiv · doi:10.48550/arxiv.1409.1556 · Cited by 114,148 · cited 10×
“Figure is reproduced based on VGG [9].”
BackgroundMethodInfluentialImageNet classification with deep convolutional neural networks
Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017) · Communications of the ACM · doi:10.1145/3065386 · Cited by 112,155 · cited 8×
“Trained with accurate human-annotated labels, the supervised learning methods obtained break-through results on different computer vision applications [1], [4], [8], [16].”
MethodInfluentialHochreiter, S., & Schmidhuber, J. (1997) · Neural Computation · doi:10.1162/neco.1997.9.8.1735 · Cited by 102,318 · cited 1×
“Compared to standard RNN [72], long short term memory (LSTM) uses memory cells to store, modify, and access internal states, to better model the long-term temporal relationships within video frames [73].”
BackgroundFaster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks
Ren, S., He, K., Girshick, R., & Sun, J. (2017) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2016.2577031 · Cited by 73,472 · cited 2×
“DUE to the powerful ability to learn different levels of general visual features, deep neural networks have been used as the basic structure to many computer vision applications such as object detection [1], [2], [3], semantic segmentation [4], [5], [6], image captioning [7], etc.”
BackgroundMethodFaster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks
Ren, S., He, K., Girshick, R., & Sun, J. (2015) · arXiv · doi:10.48550/arxiv.1506.01497 · Cited by 70,812 · cited 2×
“DUE to the powerful ability to learn different levels of general visual features, deep neural networks have been used as the basic structure to many computer vision applications such as object detection [1], [2], [3], semantic segmentation [4], [5], [6], image captioning [7], etc.”
BackgroundMethodImageNet: A large-scale hierarchical image database
Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., & Fei-Fei, L. (2009) · 2009 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2009.5206848 · Cited by 65,333 · cited 5×
“• ImageNet: The ImageNet dataset [13] contains 1.”
BackgroundInfluentialGradient-based learning applied to document recognition
Lecun, Y., Bottou, L., Bengio, Y., & Haffner, P. (1998) · Proceedings of the IEEE · doi:10.1109/5.726791 · Cited by 58,805 · cited 1×
“• MNIST: The MNIST is a dataset of handwritten digits consisting of 70, 000 images while 60, 000 images belong to training set and the rest 10, 000 images are for testing [110].”
BackgroundDistinctive Image Features from Scale-Invariant Keypoints
Lowe, D. G. (2004) · International Journal of Computer Vision · doi:10.1023/b:visi.0000029664.99615.94 · Cited by 55,589
You Only Look Once: Unified, Real-Time Object Detection
Redmon, J., Divvala, S., Girshick, R., & Farhadi, A. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.91 · Cited by 53,803 · cited 1×
“Recently, many datasets such as MSCOCO [99] and OpenImage [14] have been proposed for object detection and many ConvNet-based models [1], [2], [3], [100], [101], [102], [103], [104] have been proposed and obtained great performance.”
Background
Cited by
Items 701–800 of 1,005
Object Priors for Classifying and Localizing Unseen Actions
Mettes, P., Thong, W. E., & Snoek, C. G. M. (2021) · International Journal of Computer Vision · doi:10.1007/s11263-021-01454-y · Cited by 24
“The general pipeline is to train a pre-text task on unlabeled data and transfer the knowledge to a supervised downstream task (Jing and Tian 2020) or by clustering video datasets without manual supervision (Asano et al. 2020).”
Knowledge memorization and generation for action recognition in still images
Dong, J., Yang, W., Yao, Y., & Porikli, F. (2021) · Pattern Recognition · doi:10.1016/j.patcog.2021.108188 · Cited by 21
GeoCLR: Georeference Contrastive Learning for Efficient Seafloor Image Interpretation
Yamada, T., Prügel‐Bennett, A., Williams, S. B., Pizarro, O. R., & Thornton, B. (2022) · Field Robotics · doi:10.55417/fr.2022037 · Cited by 19
Xiao, J., Xu, H., Fang, D., Chen, C., & Gao, H. (2021) · Wireless Networks · doi:10.1007/s11276-021-02713-z · Cited by 20
Model-Based Self-Advising for Multi-Agent Learning
Ye, D., Zhu, T., Zhu, C., Zhou, W., & Yu, P. S. (2022) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2022.3147221 · Cited by 21
Self-supervised Test-time Adaptation on Video Data
Azimi, F., Palacio, S., Raue, F., Hees, J., Bertinetto, L., & Dengel, A. R. (2022) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv51458.2022.00266 · Cited by 37
“In recent years there has been a surge of interest in self-supervised methods for different applications [16], including correspondence matching and dense tracking.”
BackgroundTriBYOL: Triplet BYOL for Self-Supervised Representation Learning
Li, G., Togo, R., Ogawa, T., & Haseyama, M. (2022) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp43922.2022.9746967 · Cited by 23
“As an alternative, self-supervised learning is a learning framework that conforms to human cognition, which can learn information from the data itself without the need for manually designed labels [3].”
Self-Supervised Motion Perception for Spatiotemporal Representation Learning
Liu, C., Yao, Y., Luo, D., Zhou, Y., & Ye, Q. (2022) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2022.3160860 · Cited by 22
“SSL has been a stepping-stone toward robust and generic representation learning, which capitalizes on large amounts of unlabeled data to learn semantics from their intrinsic structures [35], [36].”
BackgroundSelf-supervised Assisted Active Learning for Skin Lesion Segmentation
Zhao, Z., Lü, W., Zeng, Z., Xu, K., Veeravalli, B., & Guan, C. (2022) · Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC) · doi:10.1109/embc48229.2022.9871734 · Cited by 22
Nie, G., Zhang, Z., Shao, M., Jiao, Z., Li, Y., & Li, L. (2023) · Sensors · doi:10.3390/s23041858 · Cited by 21
“In actual working conditions, when the bearing fails, the mechanical equipment cannot run for a long period of time, so it is very difficult for us to obtain extensive marked failure data [31,32].”
BackgroundPredNet and Predictive Coding: A Critical Review
Rane, R. P., Szügyi, E., Saxena, V., Ofner, A., & Stober, S. (2020) · International Conference on Multimedia Retrieval · doi:10.1145/3372278.3390694 · Cited by 29
“As illustrated by [15], self-supervised learning allows to extract this complex structure of the real-world without a need for expensive labeled data.”
BackgroundGeneric Neural Architecture Search via RegressionPreprint
Li, Y., Hao, C., Li, P., Xiong, J., & Chen, D. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2108.01899 · Cited by 24
“Self-supervised learning is a form of unsupervised learning, where the neural architectures are trained with automatically generated labels to gain a good degree of comprehension or understanding [28, 29, 30, 31, 27].”
BackgroundA semantically driven self-supervised algorithm for detecting anomalies in image sets
Wheeler, B. J., & Karimi, H. A. (2021) · Computer Vision and Image Understanding · doi:10.1016/j.cviu.2021.103279 · Cited by 19
Self-Supervised Bi-Channel Transformer Networks for Computer-Aided Diagnosis
Gong, R., Han, X., Wang, J., Ying, S., & Shi, J. (2022) · IEEE Journal of Biomedical and Health Informatics · doi:10.1109/jbhi.2022.3153902 · Cited by 19
“Since most SSL models are fine-tuned in the downstream tasks [8], we use the term “fine-tuning based” instead of “network-based” for easy understanding in the following content.”
BackgroundMethodInfluentialUnderstanding common human driving semantics for autonomous vehicles
Xia, Y., Geng, M., Chen, Y., Sun, S., Liao, C., Zhu, Z., Li, Z., Ochieng, W. Y., Angeloudis, P., Elhajj, M., et al. (2023) · Patterns · doi:10.1016/j.patter.2023.100730 · Cited by 19
“54 Then, we employed an SNN as the cerebral behavior recognizer and trained the SNN model to predict topic evolution recurrently in a self-super-vised and natural language generation manner 55,56 (see Note S2, Figure S3, and Table S4 for details).”
MethodBiegel, T., Helm, P., Jourdan, N., & Metternich, J. (2023) · Journal of Intelligent Manufacturing · doi:10.1007/s10845-023-02156-7 · Cited by 24
“…real-world downstream tasks from unlabeled data by solving a supervised pretext task with automatically generated pseudo-labels, e.g., solving jigsaw puzzles or predicting image rotations (Doersch et al., 2015; Gidaris et al., 2018; Jing & Tian, 2020; Noroozi & Favaro, 2016; Noroozi et al., 2018).”
Parameterization-based neural network: predicting non-linear stress–strain response of composites
Feng, H., & Prabhakar, P. (2023) · Engineering With Computers · doi:10.1007/s00366-023-01849-0 · Cited by 18
“Then, we utilize the concepts of self-supervised learning[24] and transfer learning[25] to effectively extract the latent features from the composite geometric model using an Encoder-Decoder Neural Network.”
MethodWang, T., Li, Z., Xu, Y. W., Chen, J., Genovese, A., Piuri, V., & Scotti, F. (2023) · IEEE Transactions on Instrumentation and Measurement · doi:10.1109/tim.2023.3315404 · Cited by 22
Self-Supervised Classification of SAR Images With Optical Image Assistance
Li, C., Guo, W., Zhang, Z., & Zhang, T. (2023) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2023.3330826 · Cited by 20
“Self-supervised learning (SSL) methods carefully design elaborate pretext tasks to train the models where the supervised signals are generated by utilizing data transformations and characteristics of the data itself, such as spatial context structure and content similarity [5].”
BackgroundSun, K., Xia, C., Zhang, X., Chen, H., & Zhang, J. (2024) · Proceedings of the ACM on Interactive Mobile Wearable and Ubiquitous Technologies · doi:10.1145/3643553 · Cited by 21
“SSL has proven to be a promising solution for vision [40], motion sensor [24, 36–38, 41] , and other modalities [”
BackgroundSelf-supervised learning for medical image data with anatomy-oriented imaging planes
Zhang, T., Wei, D., Zhu, M., Gu, S., & Zheng, Y. (2024) · Medical Image Analysis · doi:10.1016/j.media.2024.103151 · Cited by 23
“In SSL, the supervision signal is defined by intrinsic properties of the data, eliminating the need for extrinsic annotation (Jing and Tian, 2020).”
Variational tensor neural networks for deep learning
Jahromi, S. S., & Orús, R. (2024) · Scientific Reports · doi:10.1038/s41598-024-69366-8 · Cited by 28
“However, due to its high potential for efficient data representation and compression, novel applications of TNs are emerging in different branches of data science such as machine learning and optimization.”
BackgroundPartly Supervised Multi-Task Learning
Imran, A.-A.-Z., Huang, C., Tang, H. N., Fan, W., Yuan, X. H., Hao, D., Qian, Z., & Terzopoulos, D. (2020) · International Conference on Machine Learning and Applications · doi:10.1109/icmla51294.2020.00126 · Cited by 25
“learning [3], adversarial learning [4], and multi-task learning [5].”
BackgroundInfluentialUncertainty-Aware Self-Supervised Learning of Spatial Perception Tasks
Nava, M., Paolillo, A., Guzzi, J., Gambardella, L. M., & Giusti, A. (2021) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2021.3095269 · Cited by 22
Self-Supervised Contrastive Learning for Singing Voices
Yakura, H., Watanabe, K., & Goto, M. (2022) · IEEE/ACM Transactions on Audio Speech and Language Processing · doi:10.1109/taslp.2022.3169627 · Cited by 26
Drukker, L. (2022) · Ultrasound in Obstetrics and Gynecology · doi:10.1002/uog.24869 · Cited by 21
Tang, L., Han, M., Yang, S., Sun, Y., Qiu, L., & Zhao, W. (2025) · International Journal of Extreme Manufacturing · doi:10.1088/2631-7990/add197 · Cited by 19
“…have proposed a series of innovative learning paradigms and training strategies to address this issue, including weakly supervised learning [40, 41] , self-supervised learning [42] , unsupervised learning [43–46] , comparison learning strategies [47–49] and transfer learning strategies [50, 51] .”
Zhang, J., Jin, Z., Xia, Y., Yuan, X., Wang, Y., Li, N., Yu, Y., & Li, D. (2025) · Neural Networks · doi:10.1016/j.neunet.2025.107881 · Cited by 18
Artificial Intelligence for Remote Sensing: Progress, Challenges, and Perspectives
Shang, Y., Cheng, B., Zhang, Z., Wang, Q., Jin, P., Huang, L., Liu, C., Huang, L., Ding, X., Shen, T., et al. (2026) · IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing · doi:10.1109/jstars.2026.3656185 · Cited by 19
“Task adaptation, conversely, integrates specialized model architectures and learn-ingstrategieswithinaunifiedframeworktoconcurrentlyaddressmultipledownstreamtasks[192].”
Understanding pre-training data effects in retinal foundation models using two large fundus cohorts
Zhou, Y., Wang, Z., Wu, Y., Ong, A. Y., Wagner, S. K., Ruffell, E., Chia, M. A., Guan, Z., Ju, L., Engelmann, J., et al. (2026) · Nature Communications · doi:10.1038/s41467-026-70077-z · Cited by 22
“…application-speci fi c AI models that rely solely on labelled data for training, FMs learn generalisable features through extensive pre-training on substantial unlabelled data (e.g. via self-supervised learning 33 – 35 ), followed by fi ne-tuning on labelled data for speci fi c target applications.”
Posit Process Element for Using in Energy-Efficient DNN Accelerators
Zolfagharinejad, M., Kamal, M., Afzali‐Kusha, A., & Pedram, M. (2022) · IEEE Transactions on Very Large Scale Integration (VLSI) Systems · doi:10.1109/tvlsi.2022.3165510 · Cited by 17
Human-guided auto-labeling for network traffic data: The GELM approach
Kim, M., & Lee, I. (2022) · Neural Networks · doi:10.1016/j.neunet.2022.05.007 · Cited by 19
Self-Supervised Scene-Debiasing for Video Representation Learning via Background Patching
Assefa, M., Jiang, W., Gedamu, K., Yilma, G., Kumeda, B., & Ayalew, M. (2022) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2022.3193559 · Cited by 17
A Dual-Branch Self-Boosting Framework for Self-Supervised 3D Hand Pose Estimation
Ren, P., Sun, H., Hao, J., Qi, Q., Wang, J., & Liao, J. (2022) · IEEE Transactions on Image Processing · doi:10.1109/tip.2022.3192708 · Cited by 19
Robust and Rotation-Equivariant Contrastive Learning
Bai, G., Xi, W., Hong, X., Liu, X., Yue, Y., & Zhao, S. (2023) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2023.3243258 · Cited by 21
“In order to learn object representation more effectively, self-supervised learning [7] uses pretext tasks to assist the learning process.”
Contrastive Learning Based on Multiscale Hard Features for Remote-Sensing Image Scene Classification
Li, Z., Hou, B., Guo, X., Ma, S., Cui, Y., Wang, S., & Jiao, L. (2023) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2023.3291878 · Cited by 19
“Generally, the success of a self-supervised learning algorithm [14], [15], [16], [47] depends on two aspects: the pretext task and loss function.”
BackgroundAlkhamaiseh, K. N., Grantner, J. L., Shebrain, S., & Abdel-Qader, I. M. (2023) · Surgical Endoscopy · doi:10.1007/s00464-023-10306-4 · Cited by 17
“In the self-supervised learning [21] approach, a pre-training step with an unsupervised strategy is deployed for learning useful and better representations of the data samples.”
MethodHNSSL: Hard Negative-Based Self-Supervised Learning
Zhu, W., Liu, J., & Huang, Y. (2023) · IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw59228.2023.00506 · Cited by 19
“The main effort of general self-supervised learning approaches mainly focuses on pretext task construction [18].”
BackgroundInfluentialExemplar-Based Continual Learning via Contrastive Learning
Chen, S., Zhang, M., Zhang, J., & Huang, K. (2024) · IEEE Transactions on Artificial Intelligence · doi:10.1109/tai.2024.3355879 · Cited by 19
Yin, B., Chen, Z., & Tao, M. (2024) · IEEE Transactions on Mobile Computing · doi:10.1109/tmc.2024.3364673 · Cited by 22
“Besides, self-supervised learning methods [37] enable models to generate their own labels directly from the original data, eliminating the dependence on external annotations.”
MethodKaragöz, M. A., & Nalbantoğlu, Ö. U. (2024) · Applied Intelligence · doi:10.1007/s10489-024-05358-5 · Cited by 17
“Commonly used pretext tasks are categorized as generation based, context based, free-semantic-label based, or cross-modal based [24].”
Use of Learning Media to Increase Student Motivation in Junior High School
Suyahman, S., Pattiruhu, C. M., Zuhriyah, A., Rintaningrum, R., & Manyullei, S. (2024) · World Psychology · doi:10.55849/wp.v3i1.605 · Cited by 18
Intxausti, E., Skočaj, D., Cernuda, C., & Zugasti, E. (2024) · Applied Sciences · doi:10.3390/app14072785 · Cited by 20
Literature review on deep learning for the segmentation of seismic images
Monteiro, B. A. A., Canguçu, G. L., Jorge, L. M. S., Vareto, R. H., Oliveira, B. S., Silva, T. H., Lima, L. A., Machado, A. M. C., Schwartz, W. R., & de Melo, P. O. S. V. (2024) · Earth-Science Reviews · doi:10.1016/j.earscirev.2024.104955 · Cited by 17
Review of deep representation learning techniques for brain–computer interfaces
Guetschel, P., Ahmadi, S., & Tangermann, M. (2024) · Journal of Neural Engineering · doi:10.1088/1741-2552/ad8962 · Cited by 25
“SSL is a form of unsupervised-learning that uses pseudo-labels, or automatically generated labels, to train a network [47].”
Dynamically Scaled Temperature in Self-Supervised Contrastive Learning
Manna, S., Chattopadhyay, S., Dey, R., Pal, U., & Bhattacharya, S. (2025) · IEEE Transactions on Artificial Intelligence · doi:10.1109/tai.2024.3524979 · Cited by 25
“SSL approaches [5], [6], [10], [11], [12], [13], [14], [15], [16] have become the de facto standard in unsupervised representation learning with the aim to learn powerful features from unlabeled data that can be effectively transferred to downstream tasks.”
Self-Supervised Learning of LiDAR 3D Point Clouds via 2D-3D Neural Calibration
Zhang, Y., Hou, J., Ren, S., Wu, J., Yuan, Y., & Shi, G. (2025) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2025.3584625 · Cited by 21
“The self-supervised approaches have been very successful in 2D (images), even reaching the level of supervised pre-training [1], [2].”
Self-supervised Representation Learning on Document Images
Cosma, A., Ghidoveanu, M., Panaitescu-Liess, M., & Popescu, M. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-57058-3_8 · Cited by 17
Reflective-net: learning from explanations
Schneider, J., & Vlachos, M. (2023) · Data Mining and Knowledge Discovery · doi:10.1007/s10618-023-00920-0 · Cited by 23
“It can be based on aligning multiple modalities [19] or based on structural insights of the data [11].”
IoT-Powered Monitoring Systems for Geriatric Healthcare: Overview
Petrushin, A., Freddolini, M., Barresi, G., Bustreo, M., Laffranchi, M., Bue, A. D., & Michieli, L. D. (2022) · Studies in computational intelligence · doi:10.1007/978-981-16-8488-3_6 · Cited by 20
Mutual match for semi-supervised online evolutive learning
Li, D., Zhu, X., & Song, L. (2022) · Applied Intelligence · doi:10.1007/s10489-022-03564-7 · Cited by 18
“It is also a better way to use GAN [23, 24] or other unsupervised feature representation learning [25– 27] methods to assist SSL systems.”
MethodFeature discretization-based deep clustering for thyroid ultrasound image feature extraction
Yu, R., Tian, Y., Gao, J., Liu, Z., Wei, X., Jiang, H., Huang, Y., & Li, X. (2022) · Computers in Biology and Medicine · doi:10.1016/j.compbiomed.2022.105600 · Cited by 17
Self-citationSpatial-Temporal Invariant Contrastive Learning for Remote Sensing Scene Classification
Huang, H., Mou, Z., Li, Y., Li, Q., Chen, J., & Li, H. (2022) · IEEE Geoscience and Remote Sensing Letters · doi:10.1109/lgrs.2022.3173419 · Cited by 20
You, Y., Chang, S., Yang, Z., & Sun, Q. (2023) · Frontiers in Neuroscience · doi:10.3389/fnins.2023.1167723 · Cited by 19
“org to supervised learning with limited annotation information (Jing and Tian, 2019).”
BackgroundTowards Inadequately Pre-trained Models in Transfer Learning
Deng, A., Li, X., Hu, D., Wang, T., Xiong, H., & Xu, C. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.01777 · Cited by 24
“…reached a consensus that supervised or unsupervised pre-training as a parameter initialization or even an important medium for representation learning on existing large datasets is beneficial[12, 22] for general downstream tasks[46, 43, 47, 60, 17, 6, 5, 16, 8, 79] or specific ones[68, 67, 72, 32].”
BackgroundPrediction of Crime in Neighbourhoods of New York City using Spatial Data Analysis
Almuhanna, A. A., Alrehili, M. M., Alsubhi, S. H., & Syed, L. (2021) · International Conference on Artificial Intelligence and Data Analytics (CAIDA) · doi:10.1109/caida51941.2021.9425120 · Cited by 17
“Therefore, after the data pre-processing, in order to classify 25 different types of crimes several machine learning algorithms were applied in order to compare their results including, Support Vector Machine (SVM) [25], Random Forest (RF) [26], and XGboost [27]classifiers.”
MethodLiu, J., Cao, L., Akın, O., & Tian, Y. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1907.11704 · Cited by 17
“In computer vision research, several frameworks (Doersch et al., 2015; Doersch and Zisserman, 2017; Mundhenk et al., 2018; Jing and Tian, 2019) have been proposed to learn image and video feature representation based on data transformation.”
BackgroundSelf-citationStrength from Weakness: Fast Learning Using Weak SupervisionPreprint
Robinson, J., Jegelka, S., & Sra, S. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2002.08483 · Cited by 34
“Conversely, weakly supervised learning can be viewed as a special case of self-supervision where the pretext task is selected from some naturally occurring label source [26].”
BackgroundBillones, R. K. C., Lauresta, D. A., Dellosa, J. T., Bong, Y., Stergioulas, L. K., & Yunus, S. (2025) · Technologies · doi:10.3390/technologies13090421 · Cited by 19
“Human-labeled or crowdsourced data for supervised learning [11,16,19,27,56,67,71,72,76,93].”
Zero-shot learning with self-supervision by shuffling semantic embeddings
Kim, H., Lee, J., & Byun, H. (2021) · Neurocomputing · doi:10.1016/j.neucom.2021.01.037 · Cited by 16
E3 Outlier: a Self-Supervised Framework for Unsupervised Deep Outlier Detection
Wang, S., Zeng, Y., Yu, G. W., Cheng, Z., Liu, X., Zhou, S., Zhu, E., Kloft, M., Yin, J., & Liao, Q. (2022) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2022.3188763 · Cited by 22
“…effective pre-training technique or auxiliary task to improve the performance of high-level downstream tasks, self-supervised learning has been explored in many application scenarios, such as image classification, segmentation, object detection, action recognition [62] and anomaly detection [57].”
A self-supervised temporal temperature prediction method based on dilated contrastive learning
Lei, Y., Chen, X., Xie, Y., & Cen, L. (2022) · Journal of Process Control · doi:10.1016/j.jprocont.2022.11.005 · Cited by 16
“In the recent several years, self-supervised learning has been the hottest topic in the sub-domains of machine learning [18, 30, 36, 39].”
Yang, X., Cao, W., Lu, Y., & Zhou, Y. (2022) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2022.3225663 · Cited by 17
“Self-supervised learning (SSL) techniques have recently attracted much significant attention from researchers for their ability to learn the model without requiring human cost samples [23], [24], [25], [26].”
BackgroundHierarchical Clustering and Refinement for Generalized Multi-Camera Person Tracking
Li, Z., Wang, R., Li, H., Wei, B., Shi, Y., Ling, H., Chen, J., Liu, B., Li, Z., & Zheng, H. (2023) · IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw59228.2023.00584 · Cited by 19
Movies2Scenes: Using Movie Metadata to Learn Scene Representation
Chen, S., Liu, C.-H., Hao, X., Nie, X., Arap, M., & Hamid, R. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.00632 · Cited by 31
Zhang, R., Haihong, E., & Song, M. (2024) · Chinese Journal of Electronics · doi:10.23919/cje.2022.00.396 · Cited by 21
Unsupervised mutual transformer learning for multi-gigapixel Whole Slide Image classification
Javed, S., Mahmood, A., Qaiser, T., Werghi, N., & Rajpoot, N. (2024) · Medical Image Analysis · doi:10.1016/j.media.2024.103203 · Cited by 21
Srivatsan, B., Sridharan, N. V., Aravinth, S., Sugumaran, V., Dhanraj, J. A., Solomon, J. M., & Vaidhyanathan, R. M. (2024) · Journal of low frequency noise, vibration and active control · doi:10.1177/14613484241273652 · Cited by 17
“32 The mathematical operation performed in a convolution layer can be described in equation (1 Furthermore, each convolution operation is accompanied by an activation function that introduces non-linearity into the model.”
Li, H., Chen, B., Chen, J., Li, S., He, F., & Hu, Y. (2024) · Crop Protection · doi:10.1016/j.cropro.2024.106981 · Cited by 19
Wu, Y., Li, X., Li, J., Yang, K., Zhu, P., & Zhang, S. (2024) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3664647.3681710 · Cited by 45
“Self-supervised learning aims to learn visual representations from unlabeled images by solving pretext tasks [15].”
Optimizing Stroke Detection Using Evidential Networks and Uncertainty-Based Refinement
Akbarifar, F., Dukelow, S. P., Jin, A. Y., Mousavi, P., & Scott, S. H. (2025) · IEEE Transactions on Neural Systems and Rehabilitation Engineering · doi:10.1109/tnsre.2025.3531768 · Cited by 17
Lei, W., Xu, W., Gu, R., Fu, H., Zhang, S., Zhang, S., & Wang, G. (2021) · Lecture notes in computer science · doi:10.1007/978-3-030-87196-3_15 · Cited by 29
“To reducing the demand on human annotations, several techniques such as weakly-supervised- [15], semi-supervised- [20], self-supervised- [9,21,24] and oneshot learning (OSL) [2, 5] attracted increasing attentions.”
BackgroundBeyond Learning: Back to Geometric Essence of Visual Odometry via Fusion-Based Paradigm
Jin, Y., Wei, N., Fu, M., Liu, H., & Zhang, W. (2021) · IEEE Transactions on Instrumentation and Measurement · doi:10.1109/tim.2021.3121483 · Cited by 17
AutoGeoLabel: Automated Label Generation for Geospatial Machine Learning
Albrecht, C. M., Marianno, F. J., & Klein, L. J. (2021) · IEEE International Conference on Big Data (BigData) · doi:10.1109/bigdata52589.2021.9672060 · Cited by 16
“demonstrated the generation of expressive feature vectors without the need for any labels [51].”
BackgroundSelf-Supervised Contrastive Representation Learning in Computer Vision
Baştanlar, Y., & Orhan, S. (2022) · Artificial intelligence · doi:10.5772/intechopen.104785 · Cited by 20
Gupta, A., Sabirsh, A., Wählby, C., & Sintorn, I. (2022) · IEEE Journal of Biomedical and Health Informatics · doi:10.1109/jbhi.2022.3177602 · Cited by 16
“Recently, the emergence of self-supervised learning techniques ([9], [10]) has shown great promise in reducing the requirement of massive amounts of labeled data.”
BackgroundGeneralization Memorization Machine with Zero Empirical Risk for Classification
Wang, Z., Bai, L., & Shao, Y. (2024) · Pattern Recognition · doi:10.1016/j.patcog.2024.110469 · Cited by 17
“More importantly, it is difficult to apply the generalization-memorization kernel into other non-kernel machine learning models [3], [10]–[12] by this path.”
MethodTaming Self-Supervised Learning for Presentation Attack Detection: De-Folding and De-Mixing
Kong, Z., Zhang, W., Liu, F., Luo, W., Liu, H., Shen, L., & Ramachandra, R. (2023) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2023.3243229 · Cited by 18
“Self-supervised learning refers to learning methods in which CNNs are trained with automatically generated labels and then transferred to other computer vision tasks [42], [43], [44], [45].”
MethodOccluded Facial Expression Recognition Using Self-supervised Learning
Wang, J., Ding, H., & Wang, S. (2023) · Lecture notes in computer science · doi:10.1007/978-3-031-26316-3_8 · Cited by 16
Unsupervised representation learning of spontaneous MEG data with nonlinear ICA
Zhu, Y., Parviainen, T. M., Heinilä, E., Parkkonen, L. T., & Hyvärinen, A. (2023) · NeuroImage · doi:10.1016/j.neuroimage.2023.120142 · Cited by 18
Contrastive Learning of Person-Independent Representations for Facial Action Unit Detection
Li, Y., & Shan, S. (2023) · IEEE Transactions on Image Processing · doi:10.1109/tip.2023.3279978 · Cited by 22
“Recently, self-supervised representation learning approaches based on contrastive learning paradigm in the latent space have shown great promise, achieving state-of-the-art results [30], [46], [47], [48], [49], [50], [51].”
BackgroundKhowaja, S. A., Khuwaja, P., Dharejo, F. A., Raza, S., Lee, I. H., Naqvi, R. A., & Dev, K. (2023) · Information Fusion · doi:10.1016/j.inffus.2023.102044 · Cited by 19
Imran, A.-A.-Z., Pal, D., Patel, B. N., & Wang, A. (2021) · IEEE International Symposium on Biomedical Imaging (ISBI) · doi:10.1109/isbi48211.2021.9434044 · Cited by 16
“Self-supervision is a form of unsupervised learning in which a model is pre-trained followed by fine-tuning on the data from the same domain [4].”
BackgroundSelf-supervised monocular depth estimation via multiple bilateral consistency
Lu, Z., & Chen, Y. (2025) · Multimedia Systems · doi:10.1007/s00530-025-01700-0 · Cited by 17
GRANet: a graph residual attention network for gene regulatory network inference
Zhou, J., Gong, N., Hu, Y., Yu, H. Z., Wang, G., & Wu, H. (2025) · Briefings in Bioinformatics · doi:10.1093/bib/bbaf349 · Cited by 16
“Additionally, incorporating self-supervised learning methods could reduce reliance on large amounts of labeled data, further enhancing the model’s generalization capabilities [50].”
Advancing intelligent geography: Current status, innovations, and future prospects
Su, F., Yan, F., Wu, W., Fu, D., Cao, Y., Lyne, V. D., Meadows, M. E., Yao, L., Wang, J., Huang, Y., et al. (2025) · Geography and sustainability · doi:10.1016/j.geosus.2025.100375 · Cited by 27
Sánchez-Ferrera, A., Calvo, B., & Lozano, J. A. (2025) · ACM Computing Surveys · doi:10.1145/3770575 · Cited by 27
“The self-supervised pretext task, also known as proxy task, involves predicting specific attributes or characteristics of data that are already known.”
How Well Do Self-Supervised Models Transfer to Medical Imaging?
Anton, J., Castelli, L., Chan, M. F., Outters, M., Tang, W. H., Cheung, V., Shukla, P., Walambe, R. A., & Kotecha, K. V. (2022) · Journal of Imaging · doi:10.3390/jimaging8120320 · Cited by 21
“The learned features are then optimised for a task downstream, finetuning the features with supervised-learning on a smaller, target task-specific dataset [15].”
MethodOpen Self-Supervised Features for Remote-Sensing Image Scene Classification Using Very Few Samples
Qiu, C., Yu, A., Yi, X., Guan, N., Shi, D., & Tong, X. (2022) · IEEE Geoscience and Remote Sensing Letters · doi:10.1109/lgrs.2022.3228518 · Cited by 21
“As a special type of unsupervised learning, self-supervised learning (SSL) exploits self-designed proxy tasks to train models, and has produced results that are sometimes better than correlated supervised methods [10], [11].”
3D-FGAN: A 3D stochastic reconstruction method of digital cores
Zhang, T., Shen, T., Dong, Y., & Du, Y. (2023) · Geoenergy Science and Engineering · doi:10.1016/j.geoen.2023.212590 · Cited by 16
EEG-Oriented Self-Supervised Learning With Triple Information Pathways Network
Ko, W., Jeong, S., Song, S.-K., & Suk, H. (2024) · IEEE Transactions on Cybernetics · doi:10.1109/tcyb.2024.3410844 · Cited by 17
“Moreover, self-supervised learning has drawn attention by making it possible to optimize deep neural networks that use unlabeled samples [28].”
BackgroundRodrigues, C. N., Nunes, I. M., Pereira, M. B., Oliveira, H., & dos Santos, J. A. (2024) · Computers & Graphics · doi:10.1016/j.cag.2024.104014 · Cited by 15
Dual-decoder data decoupling training for semi-supervised medical image segmentation
Wang, B., Huang, T., Yang, S., Yang, Y., Zhai, J., & Zhang, X. (2024) · Biomedical Signal Processing and Control · doi:10.1016/j.bspc.2024.106984 · Cited by 15
Wheat disease recognition method based on the SC-ConvNeXt network model
Dong, T., Ma, X., Huang, B., Zhong, W., Han, Q., Wu, Q., & Tang, Y. X. (2024) · Scientific Reports · doi:10.1038/s41598-024-83636-5 · Cited by 15
Jannat, F.-E., Gholami, S., Alam, M. N., & Tabkhi, H. (2025) · Frontiers in Big Data · doi:10.3389/fdata.2025.1609124 · Cited by 15
Less is More: Sample Selection and Label Conditioning Improve Skin Lesion Segmentation
Ribeiro, V., Avila, S., & Junior, E. A. D. V. (2020) · IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw50498.2020.00377 · Cited by 17
“Researchers have attempted circumventing the need for data, with techniques like data augmentation [5, 34], the creation of artificial data from generative models [6], or even the use of self-supervision, which allows performing part of the training of supervised models without labels [21].”
Decoupled self-supervised label augmentation for fully-supervised image classification
Gao, W., Wu, M., Lam, S.-K., Xia, Q., & Zou, J. (2021) · Knowledge-Based Systems · doi:10.1016/j.knosys.2021.107605 · Cited by 15
Learning Visual Localization of a Quadrotor Using Its Noise as Self-Supervision
Nava, M., Paolillo, A., Guzzi, J., Gambardella, L. M., & Giusti, A. (2022) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2022.3143565 · Cited by 17
“The intuition is that auxiliary tasks force the model to recognize patterns in the input that are similar to those that must be recognized to solve the task of interest, and thus aid in learning meaningful intermediate representations [2].”
BackgroundLD-Net: A Lightweight Network for Real-Time Self-Supervised Monocular Depth Estimation
Xiong, M., Zhang, Z., Zhang, T., & Xiong, H. (2022) · IEEE Signal Processing Letters · doi:10.1109/lsp.2022.3160656 · Cited by 15
Cui, Y., Liu, F., Liu, X., Li, L., & Qian, X. (2022) · Remote Sensing · doi:10.3390/rs14102451 · Cited by 16
“As a subset of unsupervised learning methods, self-supervised learning methods avoid the extensive cost of collecting and annotating large-scale datasets [10], which leverages input data itself as supervision and benefits almost all types of downstream tasks [2].”
Metadata
- Authors
- Jing, L., & Tian, Y.
- Affiliations
- The Graduate Center, CUNY (US)
- Venue
- IEEE Transactions on Pattern Analysis and Machine Intelligence
- Published
- 2021-11-01
- Volume
- 43
- Issue
- 11
- Pages
- 4037-4058
- ISSN
- 0162-8828 (ISSN-L)
- 2160-9292
- 1939-3539
- OpenAlex
- W3023371261
- Cited by
- 2,036
- References
- 206
- Citations by year
- 2026: 160 · 2025: 283 · 2024: 354 · 2023: 405 · 2022: 377 · 2021: 305
- FWCI
- 119.51
- Type
- journal article
- Language
- en
- Open access
- green
- arxiv.org
- Licence
- Locations
- IEEE Transactions on Pattern Analysis and Machine Intelligence (publishedVersion)
- PubMed (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Funders
- National Science Foundation (IIS-1400802)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Domain Adaptation and Few-Shot Learning
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Image and Video Retrieval Techniques
- Keywords
- self-supervised learning, visual feature learning, imaging features, video features, deep learning architectures, unlabeled data, audio features, 3D data, benchmark dataset, evaluation metrics
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.