Self-Supervised Visual Feature Learning With Deep Neural Networks: A Survey
Open accessThe DOI matches its Crossref record; 204 of the 206 references it cites were found in a registry or an index.
Abstract
Large-scale labeled data are generally required to train deep neural networks in order to obtain better performance in visual feature learning from images or videos for computer vision applications. To avoid extensive cost of collecting and annotating large-scale datasets, as a subset of unsupervised learning methods, self-supervised learning methods are proposed to learn general image and video features from large-scale unlabeled data without using any human-annotated labels. This paper provides an extensive review of deep learning-based self-supervised general visual feature learning methods from images or videos. First, the motivation, general pipeline, and terminologies of this field are described. Then the common deep neural network architectures that used for self-supervised learning are summarized. Next, the schema and evaluation metrics of self-supervised learning methods are reviewed followed by the commonly used datasets for images, videos, audios, and 3D data, as well as the existing self-supervised visual feature learning methods. Finally, quantitative performance comparisons of the reviewed methods on benchmark datasets are summarized and discussed for both image and video feature learning. At last, this paper is concluded and lists a set of promising future directions for self-supervised visual feature learning.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Jing and Tian [18] summarize the self-supervised feature learning methods on image and video data, and four categories are discussed: generation-based, context-based, free semantic label-based, and cross modal-based, where cross-modal-based methods construct learning task using RGB frame sequence an optical flow sequence, which are unique features in the video.”
From Self-Supervised Learning for Time Series Analysis: Taxonomy, Progress, and Prospects (2024) · cites this paper as [18]
In the citing paper
“Another hot research sub-area is self-supervised learning, which learns discriminative visual representation by solving predefined pretext tasks [194].”
From Empowering things with intelligence: a survey of the progress, challenges, and opportunities in artificial intelligence of things (2021) · cites this paper as [194]
Citations
Cites
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474
Very Deep Convolutional Networks for Large-Scale Image Recognition
Simonyan, K., & Zisserman, A. (2014) · arXiv · doi:10.48550/arxiv.1409.1556 · Cited by 114,148 · cited 10×
“Figure is reproduced based on VGG [9].”
BackgroundMethodInfluentialImageNet classification with deep convolutional neural networks
Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017) · Communications of the ACM · doi:10.1145/3065386 · Cited by 112,155 · cited 8×
“Trained with accurate human-annotated labels, the supervised learning methods obtained break-through results on different computer vision applications [1], [4], [8], [16].”
MethodInfluentialHochreiter, S., & Schmidhuber, J. (1997) · Neural Computation · doi:10.1162/neco.1997.9.8.1735 · Cited by 102,318 · cited 1×
“Compared to standard RNN [72], long short term memory (LSTM) uses memory cells to store, modify, and access internal states, to better model the long-term temporal relationships within video frames [73].”
BackgroundFaster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks
Ren, S., He, K., Girshick, R., & Sun, J. (2017) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2016.2577031 · Cited by 73,472 · cited 2×
“DUE to the powerful ability to learn different levels of general visual features, deep neural networks have been used as the basic structure to many computer vision applications such as object detection [1], [2], [3], semantic segmentation [4], [5], [6], image captioning [7], etc.”
BackgroundMethodFaster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks
Ren, S., He, K., Girshick, R., & Sun, J. (2015) · arXiv · doi:10.48550/arxiv.1506.01497 · Cited by 70,812 · cited 2×
“DUE to the powerful ability to learn different levels of general visual features, deep neural networks have been used as the basic structure to many computer vision applications such as object detection [1], [2], [3], semantic segmentation [4], [5], [6], image captioning [7], etc.”
BackgroundMethodImageNet: A large-scale hierarchical image database
Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., & Fei-Fei, L. (2009) · 2009 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2009.5206848 · Cited by 65,333 · cited 5×
“• ImageNet: The ImageNet dataset [13] contains 1.”
BackgroundInfluentialGradient-based learning applied to document recognition
Lecun, Y., Bottou, L., Bengio, Y., & Haffner, P. (1998) · Proceedings of the IEEE · doi:10.1109/5.726791 · Cited by 58,805 · cited 1×
“• MNIST: The MNIST is a dataset of handwritten digits consisting of 70, 000 images while 60, 000 images belong to training set and the rest 10, 000 images are for testing [110].”
BackgroundDistinctive Image Features from Scale-Invariant Keypoints
Lowe, D. G. (2004) · International Journal of Computer Vision · doi:10.1023/b:visi.0000029664.99615.94 · Cited by 55,589
You Only Look Once: Unified, Real-Time Object Detection
Redmon, J., Divvala, S., Girshick, R., & Farhadi, A. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.91 · Cited by 53,803 · cited 1×
“Recently, many datasets such as MSCOCO [99] and OpenImage [14] have been proposed for object detection and many ConvNet-based models [1], [2], [3], [100], [101], [102], [103], [104] have been proposed and obtained great performance.”
Background
Cited by
Items 101–200 of 1,005
Foundation Models for Remote Sensing and Earth Observation: A survey
Xiao, A., Xuan, W., Wang, J., Huang, J., Tao, D., Lu, S., & Yokoya, N. (2025) · IEEE Geoscience and Remote Sensing Magazine · doi:10.1109/mgrs.2025.3576766 · Cited by 257
“Inspired by advancements in computer vision (CV) [3], [4], SSL has made significant strides in RS, with two primary approaches leading the way: contrastive learning and masked image modeling (MIM) .”
Weakly supervised machine learning
Ren, Z., Wang, S., & Zhang, Y. (2023) · CAAI Transactions on Intelligence Technology · doi:10.1049/cit2.12216 · Cited by 163
Bayoudh, K. (2024) · Information Fusion · doi:10.1016/j.inffus.2023.102217 · Cited by 159
Review on self-supervised image recognition using deep neural networks
Ohri, K., & Kumar, M. (2021) · Knowledge-Based Systems · doi:10.1016/j.knosys.2021.107090 · Cited by 179
Self-Supervised Spatiotemporal Feature Learning via Video Rotation Prediction
Jing, L., Yang, X., Liu, J., & Tian, Y. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1811.11387 · Cited by 119
Self-citationAn overview of deep learning methods for multimodal medical data mining
Behrad, F., & Saniee Abadeh, M. (2022) · Expert Systems with Applications · doi:10.1016/j.eswa.2022.117006 · Cited by 159
Hansen, S., Gautam, S., Jenssen, R., & Kampffmeyer, M. (2022) · Medical Image Analysis · doi:10.1016/j.media.2022.102385 · Cited by 167
Sub-Graph Contrast for Scalable Self-Supervised Graph Representation Learning
Jiao, Y., Xiong, Y., Zhang, J., Zhang, Y., Zhang, T., & Zhu, Y. (2020) · Proceedings · doi:10.1109/icdm50108.2020.00031 · Cited by 230
“Self-supervised learning [14] has recently emerged as a promising approach to overcome tlle dilemma of lacking available supervision.”
BackgroundMultimodal Emotion Recognition With Transformer-Based Self Supervised Feature Fusion
Siriwardhana, S., Kaluarachchi, T., Billinghurst, M., & Nanayakkara, S. (2020) · IEEE Access · doi:10.1109/access.2020.3026823 · Cited by 158
“SSL has become a prominent representation learning paradigm in both Natural Language Processing (NLP) and Computer Vision (CV) communities [15], [20], [26].”
BackgroundResultSignBERT+: Hand-Model-Aware Self-Supervised Pre-Training for Sign Language Understanding
Hu, H., Zhao, W., Zhou, W., & Li, H. (2023) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2023.3269220 · Cited by 231
“However, given the labeling cost, more and more works turn to self-supervised learning from a large volume of unlabeled data, which is readily available from the Web [39], [40].”
BackgroundVision Transformers in Image Restoration: A Survey
Ali, A. M., Benjdira, B., Koubâa, A., El‐Shafai, W., Khan, Z., & Boulila, W. (2023) · Sensors · doi:10.3390/s23052385 · Cited by 188
“For more information and a comprehensive survey of SSL technology, readers can refer to references [32,33].”
BackgroundImage Forgery Detection Using Deep Learning by Recompressing Images
Ali, S. S., Ganapathi, I. I., Vu, N., Ali, S. D., Saxena, N., & Werghi, N. (2022) · Electronics · doi:10.3390/electronics11030403 · Cited by 178
“Researchers have proposed a variety of methods for detecting the presence of image forgeries [7–9].”
MethodAutomatic sleep staging of EEG signals: recent development, challenges, and future directions
Phan, H., & Mikkelsen, K. B. (2022) · Physiological Measurement · doi:10.1088/1361-6579/ac6049 · Cited by 203
SCARF: Self-Supervised Contrastive Learning using Random Feature CorruptionPreprint
Bahri, D., Jiang, H., Tay, Y., & Metzler, D. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2106.15147 · Cited by 205
“A number of self-supervised learning techniques have been proposed in computer vision (Zhai et al., 2019; Tung et al., 2017; Jing & Tian, 2020).”
Self-Supervised Domain Adaptation for Computer Vision Tasks
Xu, J., Xiao, L., & Lopez, A. M. (2019) · IEEE Access · doi:10.1109/access.2019.2949697 · Cited by 154
“Self-supervised visual representation learning An extensive review of deep learning-based self-supervised general visual feature learning methods from images or videos is provided in [26].”
BackgroundSayyad, S., Kumar, S., Bongale, A. M., Kamat, P. V., Patil, S., & Kotecha, K. V. (2021) · IEEE Access · doi:10.1109/access.2021.3101284 · Cited by 175
“However, many scopes to use self-supervised learning [196] and self-supervised contrastive learning [197] algorithms are fine-tuned on limited data.”
BackgroundDeep Learning for Time-Series Prediction in IIoT: Progress, Challenges, and Prospects
Ren, L., Jia, Z., Laili, Y., & Huang, D. (2023) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2023.3291371 · Cited by 184
“Contrastive learning is another viable solution [135].”
CLMLF:A Contrastive Learning and Multi-Layer Fusion Method for Multimodal Sentiment Detection
Li, Z., Xu, B. H., Zhu, C., & Zhao, T. (2022) · Findings of the Association for Computational Linguistics: NAACL · doi:10.18653/v1/2022.findings-naacl.175 · Cited by 216
Self-Supervised Multimodal Learning: A Survey
Zong, Y., Aodha, O. M., & Hospedales, T. M. (2024) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2024.3429301 · Cited by 221
“Recently, self-supervised learning (SSL) [9], [10] has beguntoalleviatethis issuebygeneratingsupervisionfrom readily available unannotated data.”
3DSAM-adapter: Holistic adaptation of SAM from 2D to 3D for promptable tumor segmentation
Gong, S., Zhong, Y., Ma, W., Li, J., Wang, Z., Zhang, J., Heng, P.-A., & Dou, Q. (2024) · Medical Image Analysis · doi:10.1016/j.media.2024.103324 · Cited by 156
“…foundation models have seen significant interest in computer vision, benefit from pre-training techniques comprising self-supervised learning [25, 26], contrastive learning [27, 28], language-vision pre-training [29, 30], etc. Recently, SAM [4] pre-trained on over 11M images stands out as a…”
BackgroundPseudo-Supervised Deep Subspace Clustering
Lv, J., Kang, Z., Lu, X., & Xu, Z. (2021) · IEEE Transactions on Image Processing · doi:10.1109/tip.2021.3079800 · Cited by 167
“Self-supervised learning [54], [55] generally needs to design a pretext task, where a target objective can be computed without”
BackgroundSelf-supervised learning for videos: A survey
Schiappa, M. C., Rawat, Y. S., & Shah, M. (2023) · ACM Computing Surveys · doi:10.1145/3577925 · Cited by 155
“[2021], Jing and Tian [2020]. Contrastive self-supervised learning is covered in Jaiswal et al. [2021] focusing on the image domain. Multiple categories were covered in Jing and Tian [2020] including generation-based, context-based, free-semantic labeled base, and cross-modal based (i.e. optical flow). While both thes…”
BackgroundMethodInfluentialAuxiliary signal-guided knowledge encoder-decoder for medical report generation
Li, M., Liu, R., Wang, F., Chang, X., & Liang, X. (2023) · World Wide Web · doi:10.1007/s11280-022-01013-6 · Cited by 146
“d learning to loosen the requirements of training data. The core of self-supervised learning involves the design of various proxy tasks that provide auxiliary signals for training deep neural networks[12]. Furthermore, auxiliary signals are widely applied as the basic structure for image analysis. Adopting auxiliary s…”
BackgroundXiao, Z., Tong, H., Qu, R., Xing, H., Luo, S., Zhu, Z., Song, F., & Feng, L. (2023) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2023.3344294 · Cited by 172
“Experimental results show that compared with a number of state-of-the-art semi-supervised and supervised algorithms, the proposed CapMatch achieves decent performance”
Self-supervised anomaly detection in computer vision and beyond: A survey and outlook
Hojjati, H., Ho, T. K. K., & Armanfard, N. (2024) · Neural Networks · doi:10.1016/j.neunet.2024.106106 · Cited by 174
Hyperspectral Imagery Classification Based on Contrastive Learning
Hou, S., Shi, H., Cao, X., Zhang, X., & Jiao, L. (2021) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2021.3139099 · Cited by 162
“The key to self-supervised learning is how to design pretext tasks to automatically generate labels for data [37].”
BackgroundSARATR-X: Toward Building a Foundation Model for SAR Target Recognition
Li, W., Yang, W., Hou, Y., Liu, L., Liu, Y., & Li, X. (2025) · IEEE Transactions on Image Processing · doi:10.1109/tip.2025.3531988 · Cited by 185
“Self-supervised learning (SSL) [7], [36]–[38] can be used to mitigate label inefficiencies by exploring supervision in the data directly, thereby reducing the reliance on expensive expert labeling while efficiently scaling the data and models.”
LaPred: Lane-Aware Prediction of Multi-Modal Future Trajectories of Dynamic Agents
Kim, B., Park, S. H., Lee, S.-H., Khoshimjonov, E., Kum, D., Kim, J., Kim, J. S., & Choi, J. W. (2021) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr46437.2021.01440 · Cited by 177
Lai, J., Tan, H., Wang, J., Ji, L., Guo, J., Han, B., Shi, Y., Feng, Q., & Yang, W. (2023) · Nature Communications · doi:10.1038/s41467-023-39472-8 · Cited by 123
“In general, when the amount of annotated data is small, a network initialized with pretrained weights can achieve better classi-fi cation performance than a randomly initialized network 7 .”
3D Self-Supervised Methods for Medical Imaging
Taleb, A., Loetzsch, W., Danz, N., Severin, J., Gaertner, T., Bergner, B., & Lippert, C. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2006.03829 · Cited by 119
“Several methods in this line of research have been developed recently, and they found applications in numerous fields [13].”
BackgroundMethodTrends in Integration of Vision and Language Research: A Survey of Tasks, Datasets, and Methods
Mogadala, A., Kalimuthu, M., & Klakow, D. (2021) · Journal of Artificial Intelligence Research · doi:10.1613/jair.1.11688 · Cited by 47
“Independent from NLP, CV has achieved prominent improvements in tasks such as visual content classification (He et al., 2016), object detection (Redmon & Farhadi, 2017) and segmentation (He et al., 2017) etc., using large annotated datasets or self-supervision (Jing & Tian, 2019).”
MethodWang, Y., Wang, Y., Cai, J., Lee, T. K., Miao, C., & Wang, Z. J. (2022) · Medical Image Analysis · doi:10.1016/j.media.2022.102693 · Cited by 159
Li, Y., Wang, H., & Sun, M. (2023) · Reliability Engineering & System Safety · doi:10.1016/j.ress.2023.109850 · Cited by 180
Exponential Moving Average Normalization for Self-supervised and Semi-supervised Learning
Cai, Z., Ravichandran, A., Maji, S., Fowlkes, C. C., Tu, Z., & Soatto, S. (2021) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr46437.2021.00026 · Cited by 192
“An extensive survey of self-supervised learning can be found in [27].”
BackgroundAdaptive Memory Networks With Self-Supervised Learning for Unsupervised Anomaly Detection
Zhang, Y., Wang, J., Chen, Y., Yu, H., & Qin, T. (2022) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2021.3139916 · Cited by 166
“Assuming that the instances are consistent before and after basic data augmentations (i.e., normal data and abnormal data are still distinguishable after the same augmentations) [30], we design feature transformations on the original data for self-supervision.”
BackgroundMethodLu, W., Tao, C., Li, H., Qi, J., & Li, Y. (2022) · Remote Sensing of Environment · doi:10.1016/j.rse.2021.112830 · Cited by 149
Nearest Neighbor-Based Contrastive Learning for Hyperspectral and LiDAR Data Classification
Wang, M., Gao, F., Dong, J., Li, H.-C., & Du, Q. (2023) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2023.3236154 · Cited by 139
“Self-supervised learning mines the inherent attributes and semantics of large-scale unlabeled data to obtain beneficial semantic representations, and it does not require manually annotated data [23].”
BackgroundPointCutMix: Regularization strategy for point cloud classification
Zhang, J., Chen, L., Ouyang, B., Liu, B., Zhu, J., Chen, Y., Meng, Y., & Wu, D. (2022) · Neurocomputing · doi:10.1016/j.neucom.2022.07.049 · Cited by 163
Self-Supervised Learning With Adaptive Distillation for Hyperspectral Image Classification
Yue, J., Fang, L., Rahmani, H., & Ghamisi, P. (2021) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2021.3057768 · Cited by 151
“In other words, the supervised information used in SSL does not need to be obtained by manual annotation, but the algorithm automatically constructs the supervised information from unsupervised data to train the model [47], [48].”
MethodAudio self-supervised learning: A survey
Liu, S., Mallol-Ragolta, A., Parada‐Cabaleiro, E., Qian, K., Jing, X., Kathan, A., Hu, B., & Schuller, B. W. (2022) · Patterns · doi:10.1016/j.patter.2022.100616 · Cited by 188
“Besides, generative adversarial networks (GANs) are also seen as a kind of SSL framework in some works.(8,11,17) For instance, the generator creates data from a random vector by taking the real data as training targets.”
Structure-Preserving Image Super-Resolution
Ma, C., Rao, Y., Lu, J., & Zhou, J. (2021) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2021.3114428 · Cited by 146
Contrastive learning based self-supervised time-series analysis
Pöppelbaum, J., Chadha, G. S., & Schwung, A. (2022) · Applied Soft Computing · doi:10.1016/j.asoc.2021.108397 · Cited by 131
How Useful Is Self-Supervised Pretraining for Visual Tasks?
Newell, A., & Deng, J. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.00737 · Cited by 117
“The past several years have seen a wide variety of meth-ods proposed for visual self-supervision [22, 11, 14, 4, 3, 50].”
BackgroundSelf-Supervised Transformer for Sparse and Irregularly Sampled Multivariate Clinical Time-Series
Tipirneni, S., & Reddy, C. K. (2022) · ACM Transactions on Knowledge Discovery from Data · doi:10.1145/3516367 · Cited by 130
“Though this technique has shown great performance boosts with image [15] and text [9, 31] data, its application to time-series data has been limited.”
BackgroundKaleido-BERT: Vision-Language Pre-training on Fashion Domain
Zhuge, M., Gao, D., Fan, D.-P., Jin, L., Chen, B., Zhou, H., Qiu, M., & Shao, L. (2021) · 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr46437.2021.01246 · Cited by 118
“JPS [29,51] has been demonstrated to be suitable for self-supervised representation learning.”
BackgroundMethodPIP-Net: Patch-Based Intuitive Prototypes for Interpretable Image Classification
Nauta, M., Schlötterer, J., van Keulen, M., & Seifert, C. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.00269 · Cited by 227
“In line with other selfsupervised learning methods [12], we create a positive pair x′,x′′ by applying different data augmentations to input image x.”
BackgroundMethodInvestigating and Mitigating Degree-Related Biases in Graph Convoltuional Networks
Tang, X., Yao, H., Sun, Y., Wang, Y., Tang, J., Aggarwal, C. C., Mitra, P., & Wang, S. (2020) · ACM International Conference on Information and Knowledge Management (CIKM) · doi:10.1145/3340531.3411872 · Cited by 201
“h automatically generated labels, has become a successful approach in computer vision and natural language processing for unsupervised pretraining and for addressing the issue of lacking labeled data [14]. For example, pretext tasks such as Image Inpainting [21] and Image Jigsaw Puzzle [20] are widely adopted in compu…”
BackgroundVariations in Variational Autoencoders - A Comparative Evaluation
Wei, R., Garcia, C., El-Sayed, A., Peterson, V., & Mahmood, A. (2020) · IEEE Access · doi:10.1109/access.2020.3018151 · Cited by 145
“Second, basic visual skills are learned in a long process that runs through the first few years of life [61].”
BackgroundLi, J., Dong, Z., Lu, S., Wang, S., Yan, W., Ma, Y., Liu, Y., Huang, C., & Fu, X. (2022) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2022.3174895 · Cited by 255
“Self-super-vised learning refers to a learning method that uses automatically generated labels to train the network explicitly [68].”
CMID: A Unified Self-Supervised Learning Framework for Remote Sensing Image Understanding
Muhtar, D., Zhang, X., Xiao, P., Li, Z., & Gu, F. (2023) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2023.3268232 · Cited by 143
Qin, R., & Liu, T. (2022) · Remote Sensing · doi:10.3390/rs14030646 · Cited by 130
“, ResNet) to minimize the feature distance of positive pair samples and maximize the feature distance for negative samples [154,155].”
BackgroundSiamese Image Modeling for Self-Supervised Vision Representation Learning
Tao, C., Zhu, X., Su, W., Huang, G. M., Li, B., Zhou, J., Qiao, Y., Wang, X., & Dai, J. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.00212 · Cited by 147
“Self-supervised learning (SSL) has been the pursued target in the vision domain for a long time [27].”
BackgroundRODNet: Radar Object Detection using Cross-Modal Supervision
Wang, Y., Jiang, Z., Gao, X., Hwang, J., Xing, G., & Liu, H. (2021) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv48630.2021.00055 · Cited by 150
Self-supervised driven consistency training for annotation efficient histopathology image analysis
Srinidhi, C. L., Kim, S. W., Chen, F.-D., & Martel, A. L. (2022) · Medical Image Analysis · doi:10.1016/j.media.2021.102256 · Cited by 116
“These methods have recently demonstrated promising results on various computer vision (Jing and Tian, 2020; Laine and Aila, 2016; Sohn et al., 2020) and medical image analysis tasks (Chen et al.”
BackgroundMethodTao, C., Qi, J., Lu, W., Wang, H., & Li, H. (2022) · IEEE Geoscience and Remote Sensing Letters · doi:10.1109/lgrs.2020.3038420 · Cited by 115
“Most recently, a new trend is observed in machine learning, which is learning representations by self-supervised learning (SSL) methods without any additional annotation cost [8].”
BackgroundSelf-Supervised Remote Sensing Feature Learning: Learning Paradigms, Challenges, and Future Works
Tao, C., Qi, J., Guo, M., Zhu, Q., & Li, H. (2023) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2023.3276853 · Cited by 123
“Studies [14], [38], [40], [60], [61] have shown that the models trained on large-scale and diverse datasets have two advantages: 1) the learned parameters provide a good starting point, so models can converge faster when they are trained on other tasks and 2) such models usually have learned rich features, which help…”
BackgroundCommunication Beyond Transmitting Bits: Semantics-Guided Source and Channel Coding
Dai, J., Zhang, P., Niu, K., Wang, S., Si, Z., & Qin, X. (2022) · IEEE Wireless Communications · doi:10.1109/mwc.017.2100705 · Cited by 115
“representation or compression [3], [4], which are not relevant to wireless communication problems.”
Wang, H., Liu, Z., Ge, Y., & Peng, D. (2021) · Knowledge-Based Systems · doi:10.1016/j.knosys.2021.107978 · Cited by 124
Turay, T., & Vladimirova, T. (2022) · IEEE Access · doi:10.1109/access.2022.3147495 · Cited by 132
“Hybrid approaches, the use of which has become quite prevalent in the last few years [81], could be subdivided into three groups: semi-supervised learning [81], self-supervised learning [360], and self-taught learning [361].”
BackgroundEmbodied Intelligence: A Synergy of Morphology, Action, Perception and Learning
Liu, H., Guo, D., & Cangelosi, A. (2025) · ACM Computing Surveys · doi:10.1145/3717059 · Cited by 200
“Constructing pseudo-labels for unlabeled samples by various means can achieve good results in the feature learning stage [86].”
ColloSSL: Collaborative Self-Supervised Learning for Human Activity Recognition
Jain, Y., Tang, C. I., Min, C., Kawsar, F., & Mathur, A. (2022) · Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies · doi:10.1145/3517246 · Cited by 113
“One of the most exciting paradigm in this direction has been self-supervised learning (SSL) [26], where the core idea is to leverage the inherent structure present in the (unlabeled) input data to derive a supervisory signal.”
BackgroundMixing up contrastive learning: Self-supervised representation learning for time series
Wickstrøm, K., Kampffmeyer, M., Mikalsen, K. Ø., & Jenssen, R. (2022) · Pattern Recognition Letters · doi:10.1016/j.patrec.2022.02.007 · Cited by 113
“Learning useful representations is an active area of research in machine learning (Bengio et al., 2013; Jing and Tian, 2019), with encouraging results in recent works on image representation learning (Chen et al., 2020; He et al., 2020; Grill et al., called mixup (Zhang et al., 2018).”
ResultYang, C., Xiao, C., Westover, M. B., & Sun, J. (2023) · JMIR AI · doi:10.2196/46769 · Cited by 163
“While image [31,32], video [33], language [34,35], and speech [36] representations have benefited from contrastive learning, research on learning physiological signals has been limited [37,38].”
BackgroundDeep learning for blocking in entity matching: a design space exploration
Thirumuruganathan, S., Li, H., Tang, N., Ouzzani, M., Govind, Y., Paulsen, D., Fung, G., & Doan, A. (2021) · Proceedings of the VLDB Endowment · doi:10.14778/3476249.3476294 · Cited by 151
Li, X., Shi, D., Diao, X., & Xu, H. (2021) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2021.3109268 · Cited by 130
“The core idea of SCL is to train networks to maximum agreement of different views of the same images while minimizing the agreement of views from different images [32], [34].”
BackgroundMethodCroCo v2: Improved Cross-view Completion Pre-training for Stereo Matching and Optical Flow
Weinzaepfel, P., Lucas, T., Leroy, V., Cabon, Y., Arora, V., Brégier, R., Csurka, G., Antsfeld, L., Chidlovskii, B., & Revaud, J. (2023) · 2023 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.01647 · Cited by 110
“The success of instance discrimination [12, 13, 16, 28, 30] has drawn a lot of attention to self-supervised learning in computer vision [37].”
BackgroundProposalContrast: Unsupervised Pre-training for LiDAR-Based 3D Object Detection
Yin, J., Zhou, D., Zhang, L., Fang, J., Xu, C., Shen, J., & Wang, W. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-19842-7_2 · Cited by 143
RSPNet: Relative Speed Perception for Unsupervised Video Representation Learning
Chen, P., Huang, D. Y., He, D., Long, X., Zeng, R., Wen, S., Tan, M., & Gan, C. (2021) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v35i2.16189 · Cited by 140
Deep transfer learning for image classification: a survey
Plested, J., Phiri, M., & Gedeon, T. (2026) · Artificial Intelligence Review · doi:10.1007/s10462-026-11491-z · Cited by 126
Self-Supervised Pre-Training for Transformer-Based Person Re-IdentificationPreprint
Luo, H., Wang, P., Xu, Y. T., Ding, F., Zhou, Y., Wang, F., Li, H., & Jin, R. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2111.12084 · Cited by 142
“Self-supervised learning (SSL) methods are proposed to learn discriminative features from large-scale unlabeled data without any human-annotated labels [23].”
BackgroundMulti-task contrastive learning for automatic CT and X-ray diagnosis of COVID-19
Li, J., Zhao, G., Tao, Y., Zhai, P., Chen, H., He, H., & Cai, T. (2021) · Pattern Recognition · doi:10.1016/j.patcog.2021.107848 · Cited by 116
“Different from typical multi-task models, CMT-CNN requires no additional super-visions from related tasks, but seeks for annotation-free performance improvement based on the self-supervised learning [15–17] .”
FedGL: Federated graph learning framework with global self-supervision
Chen, C., Xu, Z., Hu, W., Zheng, Z., & Zhang, J. (2023) · Information Sciences · doi:10.1016/j.ins.2023.119976 · Cited by 118
Deep learning in image-based phenotypic drug discovery
Krentzel, D., Shorte, S. L., & Zimmer, C. (2023) · Trends in Cell Biology · doi:10.1016/j.tcb.2022.11.011 · Cited by 114
Machine learning in business process management: A systematic literature review
Weinzierl, S., Zilker, S., Dunzer, S., & Matzner, M. (2024) · Expert Systems with Applications · doi:10.1016/j.eswa.2024.124181 · Cited by 122
Wen, Z., Liu, Z., Zhang, S., & Pan, Q. (2021) · IEEE Transactions on Image Processing · doi:10.1109/tip.2021.3104179 · Cited by 102
Image segmentation review: Theoretical background and recent advances
Brar, K. K., Goyal, B., Dogra, A., Mustafa, M. A., Majumdar, R., Alkhayyat, A. H., & Kukreja, V. (2024) · Information Fusion · doi:10.1016/j.inffus.2024.102608 · Cited by 114
Deep Representation Learning in Speech Processing: Challenges, Recent Advances, and Future Trends
Latif, S., Rana, R. K., Khalifa, S., Jurdak, R., Qadir, J., & Schuller, B. W. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2001.00378 · Cited by 66
“Most importantly, automatically learnt feature representation is in most cases more flexible and powerful and can be applied to any data science problem in the fields of vision processing [75], text processing [76], and speech processing [77].”
BackgroundPredicting What You Already Know Helps: Provable Self-Supervised Learning
Lee, J. D., Lei, Q., Saunshi, N., & Zhuo, J. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2008.01064 · Cited by 45
“Self-supervised learning revitalizes machine learning models in computer vision, language modeling, and control problems (see reference therein (Jing & Tian, 2020; Kolesnikov et al., 2019; Devlin et al., 2018; Wang & Gupta, 2015; Jang et al., 2018)).”
BackgroundContrastive Learning with Adversarial ExamplesPreprint
Ho, C.-H., & Vasconcelos, N. M. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2010.12050 · Cited by 163
“Self-supervised learning (SSL) [25] aims to alleviate this limitation, by leveraging unlabeled data to define surrogate tasks that can be used for network training.”
BackgroundMorales, E. F., Murrieta-Cid, R., Becerra, I., & Esquivel-Basaldua, M. A. (2021) · Intelligent Service Robotics · doi:10.1007/s11370-021-00398-z · Cited by 101
“Summarizing, there are many works [6,55,72,76,97,102, 131,186] whose main application corresponds to computer vision problems; nonetheless, they provide important tools to resolve robotics tasks.”
BackgroundProposal Learning for Semi-Supervised Object Detection
Tang, P., Ramaiah, C., Wang, Y., Xu, R., & Xiong, C. (2021) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv48630.2021.00234 · Cited by 122
“Please see the recent survey [22] for more self-supervised learning approaches.”
BackgroundA survey on machine learning from few samples
Lu, J., Gong, P., Ye, J., Zhang, J., & Zhang, C. (2023) · Pattern Recognition · doi:10.1016/j.patcog.2023.109480 · Cited by 100
“Self-supervised learning becomes a popular technique nowadays, especially in the field of vision [180], [181], [182], [183], [184], [185], [186], [187], [188], [189] and language [190], [191], [192], [193], which aims to learn semantically meaningful representations using only the inherent structural information conta…”
BackgroundA Survey of Self-Supervised and Few-Shot Object Detection
Huang, G., Laradji, I., Vazquez, D., Lacoste-Julien, S., & Rodriguez, P. (2022) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2022.3199617 · Cited by 100
“Jing and Tian [53] present a survey on self-supervised”
BackgroundMachine Learning: A Review of Learning TypesPreprint
Sah, S. (2020) · Preprints.org · doi:10.20944/preprints202007.0230.v1 · Cited by 128
“Further reading (Jing & Tian, 2020).”
Dong, X., Lin, J., Lu, S., Huang, X., Wang, H., & Li, Y. (2022) · Surveys in Geophysics · doi:10.1007/s10712-022-09702-7 · Cited by 104
“In general, we utilize supervised, unsupervised, self-supervised, semi-supervised methods (Jing and Tian 2021; Schmarje et al.”
MethodAutoFi: Toward Automatic Wi-Fi Human Sensing via Geometric Self-Supervised Learning
Yang, J., Chen, X., Zou, H., Wang, D., & Xie, L. (2022) · IEEE Internet of Things Journal · doi:10.1109/jiot.2022.3228820 · Cited by 122
“Prevailing self-supervised learning methods employ handcrafted auxiliary tasks or contrastive learning [30].”
Consistency-Guided Differential Decoding for Enhancing Semi-Supervised Medical Image Segmentation
Zeng, Q., Xie, Y., Lu, Z., Lu, M., Zhang, J., & Xia, Y. (2024) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2024.3429340 · Cited by 118
Werther, M., Odermatt, D., Simis, S. G. H., Gurlin, D., Lehmann, M. K., Kutser, T., Gupana, R. S., Varley, A., Hunter, P., Tyler, A. N., et al. (2022) · Remote Sensing of Environment · doi:10.1016/j.rse.2022.113295 · Cited by 96
Self-supervised contrastive learning on agricultural images
Güldenring, R., & Nalpantidis, L. (2021) · Computers and Electronics in Agriculture · doi:10.1016/j.compag.2021.106510 · Cited by 103
“(Jing and Tian, 2021).”
Hijji, M., Iqbal, R., Pandey, A. K., Doctor, F., Karyotis, C., Rajeh, W., Alshehri, A., & Aradah, F. (2023) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2023.3235151 · Cited by 101
“Later, sensory data and imagery data were augmented to increase the size of the data set and achieve better accuracy [15].”
BackgroundMethodNetworking Systems for Video Anomaly Detection: A Tutorial and Survey
Liu, J., Liu, Y., Lin, J., Li, J., Cao, L., Sun, P., Hu, B., Song, L., Boukerche, A., & Leung, V. C. M. (2025) · ACM Computing Surveys · doi:10.1145/3729222 · Cited by 124
“A taxonomy of these methods is presented in Table 2,”
GEFA: Early Fusion Approach in Drug-Target Affinity Prediction
Nguyen, T. M., Nguyen, T., Le, T. M., & Tran, T. (2021) · IEEE Transactions on Computational Biology and Bioinformatics · doi:10.1109/tcbb.2021.3094217 · Cited by 101
Attention-based sequence to sequence model for machine remaining useful life prediction
Ragab, M., Chen, Z., Wu, M., Kwoh, C. K., Yan, R., & Li, X. (2021) · Neurocomputing · doi:10.1016/j.neucom.2021.09.022 · Cited by 101
Yuan, Y., Lin, L., Liu, Q., Hang, R., & Zhou, Z.-G. (2021) · International Journal of Applied Earth Observation and Geoinformation · doi:10.1016/j.jag.2021.102651 · Cited by 123
Masked Auto-Encoding Spectral–Spatial Transformer for Hyperspectral Image Classification
Ibañez, D., Fernández-Beltrán, R., Pla, F., & Yokoya, N. (2022) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2022.3217892 · Cited by 108
“Self-supervised learning as a semisupervision technique has been explored for a long time [56], [57], and later in many fields [58], [59] [60].”
BackgroundSelf-Supervised Graph Representation Learning via Global Context PredictionPreprint
Peng, Z., Dong, Y., Luo, M., Wu, X.-M., & Zheng, Q. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2003.01604 · Cited by 103
LogME: Practical Assessment of Pre-trained Models for Transfer LearningPreprint
You, K., Liu, Y., Wang, J., & Long, M. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2102.11005 · Cited by 339
“How to design better contrastive pre-training strategies is still under active research (Tian et al., 2020).”
BackgroundZhang, L., & Zhu, Z. (2019) · International Conference on 3D Vision (3DV) · doi:10.1109/3dv.2019.00051 · Cited by 98
“The unsupervised learning methods for image feature learning have obtained great success and the performance of unsupervised learning methods sometimes come very close to supervised methods [5, 13].”
Barragán-Montero, A., Bibal, A., Dastarac, M. H., Draguet, C., Valdés, G., Nguyen, D., Willems, S., Vandewinckele, L., Holmström, M., Löfman, F., et al. (2022) · Physics in Medicine & Biology · doi:10.1088/1361-6560/ac678a · Cited by 84
A survey on unsupervised learning for wearable sensor-based activity recognition
Ige, A. O., & Noor, M. H. M. (2022) · Applied Soft Computing · doi:10.1016/j.asoc.2022.109363 · Cited by 99
Metadata
- Authors
- Jing, L., & Tian, Y.
- Affiliations
- The Graduate Center, CUNY (US)
- Venue
- IEEE Transactions on Pattern Analysis and Machine Intelligence
- Published
- 2021-11-01
- Volume
- 43
- Issue
- 11
- Pages
- 4037-4058
- ISSN
- 0162-8828 (ISSN-L)
- 2160-9292
- 1939-3539
- OpenAlex
- W3023371261
- Cited by
- 2,036
- References
- 206
- Citations by year
- 2026: 160 · 2025: 283 · 2024: 354 · 2023: 405 · 2022: 377 · 2021: 305
- FWCI
- 119.51
- Type
- journal article
- Language
- en
- Open access
- green
- arxiv.org
- Licence
- Locations
- IEEE Transactions on Pattern Analysis and Machine Intelligence (publishedVersion)
- PubMed (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Funders
- National Science Foundation (IIS-1400802)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Domain Adaptation and Few-Shot Learning
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Image and Video Retrieval Techniques
- Keywords
- self-supervised learning, visual feature learning, imaging features, video features, deep learning architectures, unlabeled data, audio features, 3D data, benchmark dataset, evaluation metrics
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.