Self-Supervised Visual Feature Learning With Deep Neural Networks: A Survey
Open accessThe DOI matches its Crossref record; 204 of the 206 references it cites were found in a registry or an index.
Abstract
Large-scale labeled data are generally required to train deep neural networks in order to obtain better performance in visual feature learning from images or videos for computer vision applications. To avoid extensive cost of collecting and annotating large-scale datasets, as a subset of unsupervised learning methods, self-supervised learning methods are proposed to learn general image and video features from large-scale unlabeled data without using any human-annotated labels. This paper provides an extensive review of deep learning-based self-supervised general visual feature learning methods from images or videos. First, the motivation, general pipeline, and terminologies of this field are described. Then the common deep neural network architectures that used for self-supervised learning are summarized. Next, the schema and evaluation metrics of self-supervised learning methods are reviewed followed by the commonly used datasets for images, videos, audios, and 3D data, as well as the existing self-supervised visual feature learning methods. Finally, quantitative performance comparisons of the reviewed methods on benchmark datasets are summarized and discussed for both image and video feature learning. At last, this paper is concluded and lists a set of promising future directions for self-supervised visual feature learning.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Jing and Tian [18] summarize the self-supervised feature learning methods on image and video data, and four categories are discussed: generation-based, context-based, free semantic label-based, and cross modal-based, where cross-modal-based methods construct learning task using RGB frame sequence an optical flow sequence, which are unique features in the video.”
From Self-Supervised Learning for Time Series Analysis: Taxonomy, Progress, and Prospects (2024) · cites this paper as [18]
In the citing paper
“Another hot research sub-area is self-supervised learning, which learns discriminative visual representation by solving predefined pretext tasks [194].”
From Empowering things with intelligence: a survey of the progress, challenges, and opportunities in artificial intelligence of things (2021) · cites this paper as [194]
Citations
Cites
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474
Very Deep Convolutional Networks for Large-Scale Image Recognition
Simonyan, K., & Zisserman, A. (2014) · arXiv · doi:10.48550/arxiv.1409.1556 · Cited by 114,148 · cited 10×
“Figure is reproduced based on VGG [9].”
BackgroundMethodInfluentialImageNet classification with deep convolutional neural networks
Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017) · Communications of the ACM · doi:10.1145/3065386 · Cited by 112,155 · cited 8×
“Trained with accurate human-annotated labels, the supervised learning methods obtained break-through results on different computer vision applications [1], [4], [8], [16].”
MethodInfluentialHochreiter, S., & Schmidhuber, J. (1997) · Neural Computation · doi:10.1162/neco.1997.9.8.1735 · Cited by 102,318 · cited 1×
“Compared to standard RNN [72], long short term memory (LSTM) uses memory cells to store, modify, and access internal states, to better model the long-term temporal relationships within video frames [73].”
BackgroundFaster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks
Ren, S., He, K., Girshick, R., & Sun, J. (2017) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2016.2577031 · Cited by 73,472 · cited 2×
“DUE to the powerful ability to learn different levels of general visual features, deep neural networks have been used as the basic structure to many computer vision applications such as object detection [1], [2], [3], semantic segmentation [4], [5], [6], image captioning [7], etc.”
BackgroundMethodFaster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks
Ren, S., He, K., Girshick, R., & Sun, J. (2015) · arXiv · doi:10.48550/arxiv.1506.01497 · Cited by 70,812 · cited 2×
“DUE to the powerful ability to learn different levels of general visual features, deep neural networks have been used as the basic structure to many computer vision applications such as object detection [1], [2], [3], semantic segmentation [4], [5], [6], image captioning [7], etc.”
BackgroundMethodImageNet: A large-scale hierarchical image database
Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., & Fei-Fei, L. (2009) · 2009 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2009.5206848 · Cited by 65,333 · cited 5×
“• ImageNet: The ImageNet dataset [13] contains 1.”
BackgroundInfluentialGradient-based learning applied to document recognition
Lecun, Y., Bottou, L., Bengio, Y., & Haffner, P. (1998) · Proceedings of the IEEE · doi:10.1109/5.726791 · Cited by 58,805 · cited 1×
“• MNIST: The MNIST is a dataset of handwritten digits consisting of 70, 000 images while 60, 000 images belong to training set and the rest 10, 000 images are for testing [110].”
BackgroundDistinctive Image Features from Scale-Invariant Keypoints
Lowe, D. G. (2004) · International Journal of Computer Vision · doi:10.1023/b:visi.0000029664.99615.94 · Cited by 55,589
You Only Look Once: Unified, Real-Time Object Detection
Redmon, J., Divvala, S., Girshick, R., & Farhadi, A. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.91 · Cited by 53,803 · cited 1×
“Recently, many datasets such as MSCOCO [99] and OpenImage [14] have been proposed for object detection and many ConvNet-based models [1], [2], [3], [100], [101], [102], [103], [104] have been proposed and obtained great performance.”
Background
Cited by
Items 801–900 of 1,005
KTN: Knowledge Transfer Network for Learning Multiperson 2D-3D Correspondences
Wang, X., Gao, L., Zhou, Y., Song, J., & Wang, M. (2022) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2022.3181604 · Cited by 18
“Early works [28]– [32] use metric-learning approaches to learn feature representations of new concepts with limited annotation examples.”
BackgroundLiCaS3: A Simple LiDAR–Camera Self-Supervised Synchronization Method
Yuan, K., Li, D., Abdelfattah, M., & Wang, Z. J. (2022) · IEEE Transactions on Robotics · doi:10.1109/tro.2022.3167455 · Cited by 21
“It is worth mentioning that self-supervised learning on cross-modal retrieval [21] is still a very challenging problem in machine learning.”
BackgroundExploring Self-supervised Learning for Radio Signal Recognition
Yun, X., & Zhou, X. (2021) · IEEE International Conference on High Performance Computing and Communications; IEEE International Conference on Smart City; IEEE International Conference on Data Science and Systems (HPCC/SmartCity/DSS) · doi:10.1109/hpcc-dss-smartcity-dependsys53884.2021.00365 · Cited by 16
Liu, J., Cao, L., Akın, O., & Tian, Y. (2022) · Frontiers in Radiology · doi:10.3389/fradi.2022.1041518 · Cited by 15
“Therefore, in computer vision, some researchers proposed self-or un-supervised learning methods to learn feature representations without requiring manual data annotations (24 – 28).”
BackgroundSelf-citationWang, X., Yin, K., Ouyang, Q., Wen, X., Zhang, S., Zhang, W., Cao, L., Han, J., Jin, X., & Pei, D. (2022) · Proceedings/Proceedings - International Symposium on Software Reliability Engineering · doi:10.1109/issre55969.2022.00043 · Cited by 20
“Self-supervised learning is a suitable tool to solve this problem by creating self-defined pseudo labels as supervision and learning the representations of the data [23].”
Formula-Driven Supervised Learning in Computer Vision: A Literature Survey
Hafiz, A. M., Hassaballah, M., & Binbusayyis, A. (2023) · Applied Sciences · doi:10.3390/app13020723 · Cited by 19
A Noising-Denoising Framework for Point Cloud Upsampling via Normalizing Flows
Hu, X., Wei, X., & Sun, J. (2023) · Pattern Recognition · doi:10.1016/j.patcog.2023.109569 · Cited by 15
Federated Meta-Learning for Few-Shot Fault Diagnosis With Representation Encoding
Cui, J., Li, J., Mei, Z., Wei, K., Wei, S., Ding, M., Chen, W., & Guo, S. (2023) · IEEE Transactions on Instrumentation and Measurement · doi:10.1109/tim.2023.3328074 · Cited by 18
“…framework for few-shot FD. REFML harnesses federated meta-learning (FML) and draws inspiration from representation learning for capturing discriminative feature representations [29], [30], [31], [32], [33] where W represents the parameters of the global model, and U is the number of total clients.”
BackgroundKnowledge-guided pre-training and fine-tuning: Video representation learning for action recognition
Wang, G., Zhou, Y., He, Z., Lu, K., Feng, Y., Liu, Z., & Wang, G. (2023) · Neurocomputing · doi:10.1016/j.neucom.2023.127136 · Cited by 16
Molecular Contrastive Pretraining with Collaborative Featurizations
Zhu, Y., Chen, D., Du, Y., Wang, Y., Liu, Q., & Wu, S. (2024) · Journal of Chemical Information and Modeling · doi:10.1021/acs.jcim.3c01468 · Cited by 17
“A typical framework pretrains the encoder model with training objectives over large-scale unlabeled datasets and then fine-tunes the learned model on labeled downstream tasks.”
PIFiA: self-supervised approach for protein functional annotation from single-cell imaging data
Razdaibiedina, A., Brechalov, A. V., Friesen, H., Ušaj, M. M., Masinas, M. P. D., Suresh, H. G., Wang, K., Boone, C. M., Ba, J. L., & Andrews, B. (2024) · Molecular Systems Biology · doi:10.1038/s44320-024-00029-6 · Cited by 18
“An emerging alternative to supervised methods for biological image analysis involves self-supervised approaches, which do not require manually assigned categories during training (Chen et al, 2020; Jenni and Favaro, 2018; Jing and Tian, 2020).”
Moving SLAM: Fully Unsupervised Deep Learning in Non-Rigid Scenes
Xu, D., Vedaldi, A., & Henriques, J. F. (2021) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros51168.2021.9636075 · Cited by 19
“Self-supervised learning [4, 20] has gathered significant attention recently, since it promises to achieve unsupervised learning by reusing standard elements from supervised learning (e.g. architectures, loss functions), in relatively intuitive configurations.”
BackgroundLeveraging Metadata in Representation Learning With Georeferenced Seafloor Imagery
Yamada, T., Massot‐Campos, M., Prügel‐Bennett, A., Williams, S. B., Pizarro, O. R., & Thornton, B. (2021) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2021.3101881 · Cited by 15
“…first image’s horizontal location neighbourhood, and another n ∗ / 3 images are selected from the second image’s depth neighbourhood, and the final n ∗ / 3 images are randomly selected from the whole dataset in accordance with the principles of triplet loss contrastive learning demonstrated in [22].”
MethodUnderstand and Improve Contrastive Learning Methods for Visual Representation: A ReviewPreprint
Liu, R. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2106.03259 · Cited by 22
“[25] provides a comprehensive review of self-supervised learning methods for visual representation.”
BackgroundLabel-Free Explainability for Unsupervised ModelsPreprint
Crabbé, J., & van der Schaar, M. (2022) · arXiv (Cornell University) · doi:10.48550/arxiv.2203.01928 · Cited by 18
“When this is the case, self-supervised learning preconises to leverage unlabelled data to solve an unsupervised pretext tasks such as predicting the missing part of a masked image (Jing & Tian, 2021).”
MethodPOPAR: Patch Order Prediction and Appearance Recovery for Self-supervised Medical Image Analysis
Pang, J., Haghighi, F., Ma, D., Islam, N. U., Taher, M. R. H., Gotway, M. B., & Liang, J. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-16852-9_8 · Cited by 19
Sun, H., & Li, M. (2023) · Science China Information Sciences · doi:10.1007/s11432-021-3535-2 · Cited by 16
Temporal Graph Representation Learning with Adaptive Augmentation Contrastive
Chen, H., Jiao, P., Tang, H., & Wu, H. (2023) · Lecture notes in computer science · doi:10.1007/978-3-031-43415-0_40 · Cited by 20
“Inspired by recent advancements of CL in computer vision [12] and natural language processing [18] domains, some research has been conducted to apply CL to graph data.”
MethodSelf-FI: Self-Supervised Learning for Disease Diagnosis in Fundus Images
Nguyen, T. D., Le, D.-T., Bum, J., Kim, S.-H., Song, S. J., & Choo, H. (2023) · Bioengineering · doi:10.3390/bioengineering10091089 · Cited by 15
“This approach harnesses the inherent structure and context intrinsic to the data to learn meaningful representations [4].”
BackgroundNisar, M. A., Shirahama, K., Irshad, M. T., Huang, X., & Grzegorzek, M. (2023) · Sensors · doi:10.3390/s23198234 · Cited by 17
“Deep learning, also known as machine learning with deep neural networks (DNNs), is one of the most popular feature-learning methods [7].”
BackgroundSelf-supervised learning for clustering of wireless spectrum activity
Milosheski, L., Cerar, G., Bertalanič, B., Fortuna, C., & Mohorčič, M. (2023) · Computer Communications · doi:10.1016/j.comcom.2023.10.009 · Cited by 14
“Given the advent of large datasets which are expensive or practically impossible to label, self-supervised learning (SSL) [13], as another intermediate learning approach, is becoming an important alternative that is particularly suitable to reduce the data labelling cost and leverage the unlabelled…”
BackgroundLearning Representations for Multilead Electrocardiograms From Morphology-Rhythm Contrast
Liu, W., Zhang, H., Chang, S., Wang, H., He, J. L., & Huang, Q. (2024) · IEEE Transactions on Instrumentation and Measurement · doi:10.1109/tim.2024.3369152 · Cited by 21
“As a major category of SSRL, contrastive learning has shown its effectiveness in computer vision recently [17].”
BackgroundChen, Z., Liu, S., Liu, S., Chang, Y., Chen, J., Feng, G., & He, S. (2024) · Knowledge-Based Systems · doi:10.1016/j.knosys.2024.111922 · Cited by 14
Few-Shot MS and PAN Joint Classification With Improved Cross-Source Contrastive Learning
Zhu, H., Guo, P., Hou, B., Li, X., Jiao, C., Ren, B., Jiao, L., & Wang, S. (2024) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2024.3416298 · Cited by 14
“…the significant volume of unlabeled data prevalent in real-world scenarios, researchers in machine learning have recently delved into exploring self-supervised learning [16], [17], [18], [19], [20], which aims to maximize the utilization of unlabeled data to address the “few-shot dilemma.””
BackgroundA Real-Time Grasping Detection Network Architecture for Various Grasping Scenarios
Gao, H., Zhao, J., Hu, J., & Sun, C. (2024) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2024.3419180 · Cited by 15
“The data-driven method first involves the construction of a knowledge repository, during the offline phase, which maps perceptual information (such as RGBD images, point clouds, and tactile data) to grasp poses [8].”
Wang, L., Pan, Q., Su, D., & Huang, S. (2024) · Canadian Geotechnical Journal · doi:10.1139/cgj-2024-0191 · Cited by 14
Purma, V., Srinath, S., Srirangarajan, S., Kakkar, A., & Prathosh, A. P. (2024) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2024.3453492 · Cited by 18
“Jing et al. [46] provide a detailed review of deep learning-based self-supervised general visual feature learning methods from images.”
Rashid, H., Benbouzid, M., Amirat, Y., Berghout, T., Titah-Benbouzid, H., & Mamoune, A. (2024) · Engineering Applications of Artificial Intelligence · doi:10.1016/j.engappai.2024.109316 · Cited by 15
Precise occlusion-aware and feature-level reconstruction for occluded person re-identification
Shu, X., Li, H., Wei, W. L., Qiao, R., Li, N., Ruan, W., Su, H., Wang, B., Chen, S., & Zhou, J. (2024) · Neurocomputing · doi:10.1016/j.neucom.2024.128919 · Cited by 18
Ye, Y., Dai, J., Zhou, L., Duan, K., Tao, R., Li, W., & Hong, D. (2025) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2025.3542868 · Cited by 29
“A. Contrastive Learning SSL [20], [34], [35] originated from natural language processing (NLP).”
Self-Supervised Graph Masked Autoencoders for Hyperspectral Image Classification
Hu, Z., Tu, B., Liu, B., He, Y., Li, J., & Plaza, A. (2025) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2025.3555967 · Cited by 20
“…were conducted on the software platforms PyTorch and PyTorch-Geometric. and invariant feature representations through SLL, contrastive learning, or pretraining on large-scale datasets, enhancing generalization ability while alleviating gradient problems in DNNs and improving robustness, [49].”
BotSSCL: Social Bot Detection with Self-Supervised Contrastive Learning
Akhtar, M. M., Bhuiyan, N. S., Masood, R., Ikram, M., & Kanhere, S. S. (2025) · Online Social Networks and Media · doi:10.1016/j.osnem.2025.100318 · Cited by 21
Random features Hopfield networks generalize retrieval to previously unseen examples
Kalaj, S., Lauditi, C., Perugini, G., Lucibello, C., Malatesta, E. M., & Negri, M. (2025) · Physica A Statistical Mechanics and its Applications · doi:10.1016/j.physa.2025.130946 · Cited by 24
Zulueta-Coarasa, T., Jug, F., Mathur, A., Moore, J., Muñoz‐Barrutia, A., Anita, L., Babalola, K., Bankhead, P., Paul‐Gilloteaux, P., Gogoberidze, N., et al. (2025) · Nature Methods · doi:10.1038/s41592-025-02835-8 · Cited by 16
“Beyond that, the MIFA guidelines would benefit any annotated dataset of biological images, not only those intended for AI applications.”
Fan, L., Huang, J., Di, D., Su, A., Song, T., Pagnucco, M., & Song, Y. (2025) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51701.2025.01989 · Cited by 40
“Contrastive learning (CL) [25, 34] aims to learn task-agnostic feature representations by aligning similar pairs while separating dissimilar pairs in the feature space.”
Seeking an Optimal Approach for Computer-Aided Pulmonary Embolism Detection
Islam, N. U., Gehlot, S., Zhou, Z., Gotway, M. B., & Liang, J. (2021) · Lecture notes in computer science · doi:10.1007/978-3-030-87589-3_71 · Cited by 19
FLUID: Few-Shot Self-Supervised Image Deraining
Rai, S. N., Saluja, R., Arora, C., Balasubramanian, V. N., Subramanian, A., & Jawahar, C. V. (2022) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv51458.2022.00049 · Cited by 15
“Self-Supervised Learning: Self-supervised learning [16] refers to the learning of visual features from the unlabeled dataset.”
BackgroundMethodMultiscale Human Activity Recognition and Anticipation Network
Xing, Y., Golodetz, S., Everitt, A., Markham, A., & Trigoni, N. (2022) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2022.3175480 · Cited by 14
“D ConvNets, a common input window with 16, 32, or 64 frames is usually selected [31], [54], [71].”
BackgroundResearch on Chengdu Ma Goat Recognition Based on Computer Vison
Pu, J., Yu, C., Chen, X., Zhang, Y. L., Yang, X., & Li, J. (2022) · Animals · doi:10.3390/ani12141746 · Cited by 14
“To reduce the substantial cost of collecting and labeling large-scale datasets, selfsupervised learning, an important component of unsupervised learning methods, attempts to learn general image and video features from large-scale unlabeled data without using any manually labeled data [8].”
BackgroundMethodInfluentialLearning low‐dose CT degradation from unpaired data with flow‐based model
Liu, X., Liang, X., Deng, L., Tan, S., & Xie, Y. (2022) · Medical Physics · doi:10.1002/mp.15886 · Cited by 14
Long Short-Term Memory Neural Equalizer
Wang, Z., Xu, Z., He, J., Delingette, H., & Fan, J. (2023) · IEEE Transactions on Signal and Power Integrity · doi:10.1109/tsipi.2023.3242855 · Cited by 15
“supervised/unsupervised learning [42], [43], [44], there is a possibility to reduce the efforts for collecting the training dataset by applying those self-adaptive learning approaches to signal”
BackgroundMultilabel Image Classification using the CNN and DC-CNN Model on Pascal VOC 2012 Dataset
Juyal, P., & Kundaliya, A. (2023) · International Conference on Sustainable Computing and Smart Systems (ICSCSS) · doi:10.1109/icscss57650.2023.10169541 · Cited by 14
Wang, Q., Fang, W. W., & Xiong, N. N. (2023) · IEEE Internet of Things Journal · doi:10.1109/jiot.2023.3293506 · Cited by 17
“Inspired by the work in [17], GM is trained in a self-supervised way with the binary cross-entropy loss L BCE ( · ) , as follows: where the loss is calculated between the predicted output of GM and the pseudo label g jt [42].”
MethodFrom lazy to rich to exclusive task representations in neural networks and neural codes
Farrell, M., Recanatesi, S., & Shea‐Brown, E. (2023) · Current Opinion in Neurobiology · doi:10.1016/j.conb.2023.102780 · Cited by 20
Multi-scale pseudo labeling for unsupervised deep edge detection
Zhou, C., Yuan, C., Wang, H., Li, L., Oehmcke, S., Liu, J., & Peng, J. (2023) · Knowledge-Based Systems · doi:10.1016/j.knosys.2023.111057 · Cited by 16
Contrastive Transformer Hashing for Compact Video Representation
Shen, X., Zhou, Y., Yuan, Y., Yang, X., Lan, L., & Zheng, Y. (2023) · IEEE Transactions on Image Processing · doi:10.1109/tip.2023.3326994 · Cited by 16
“To alleviate the annotation bottleneck, self-supervised learning (SSL) [25] is proposed to learn visual features from unlabeled images or videos without any annotation.”
MethodConversational Query Rewriting with Self-Supervised Learning
Liu, H., Chen, M., Wu, Y., He, X., & Zhou, B. (2021) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp39728.2021.9413557 · Cited by 14
“While Computer Vision is making amazing progress on SSL only in the last few years [16], SSL has been a trend in NLP research recently.”
BackgroundExtending Momentum Contrast With Cross Similarity Consistency Regularization
Seyfi, M., Banitalebi-Dehkordi, A., & Zhang, Y. (2022) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2022.3169145 · Cited by 15
“To overcome the costly and laborious data 38 annotation, many self-supervised/unsupervised representation 39 learning methods have been developed to extract visual seman40 tics of the large-scale image datasets, without the burden of 41 labeling them [4], [5].”
MethodLiu, W., Guo, Q., Gao, X., Chang, S., Wang, H., He, J. L., & Huang, Q. (2022) · IEEE Internet of Things Journal · doi:10.1109/jiot.2022.3188771 · Cited by 14
“It is worth noting that the downstream supervised tasks are just proxies to evaluate the quality of the unsupervised feature learning [31].”
BackgroundExploring Self-Supervised Regularization for Supervised and Semi-Supervised LearningPreprint
Tran, P. V. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1906.10343 · Cited by 23
“research directions include domain adaptation [45], zero- and few-shot learning [41], active and continual learning [29], representation learning for data augmentation [32], self-supervised learning [16], and multi-task learning [33]. In this work, we examine the task of semi-supervised learning (SSL) [4], within the…”
BackgroundImage2Point: 3D Point-Cloud Understanding with Pretrained 2D ConvNets.Preprint
Xu, C., Yang, S., Zhai, B., Wu, B., Yue, X., Zhan, W., Vajda, P., Keutzer, K., & Tomizuka, M. (2021) · arXiv (Cornell University) · Cited by 14
Large model enhanced computational ghost imaging
Chen, Y., An, H., Sun, Z., Tian, T., Chen, M., Spielmann, C., & Li, X. (2025) · Science China Technological Sciences · doi:10.1007/s11431-025-3062-3 · Cited by 14
“To reduce dataset acquisition costs and enhance model generalization across different scenarios, self-supervised methods [34–36] have been introduced to train models without the need for labeled data.”
Wen, T., Wang, J., Lu, R., Tan, S., Li, P., Yao, X., Liu, H., Yi, Z., Li, L., Liu, S., et al. (2023) · European Journal of Medicinal Chemistry · doi:10.1016/j.ejmech.2023.115199 · Cited by 14
Visual DNA: Representing and Comparing Images Using Distributions of Neuron Activations
Ramtoula, B., Gadd, M., Newman, P. M., & Martini, D. D. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.01069 · Cited by 18
“Pre-trained networks on large datasets also provide generally useful representations [24], which are often fine-tuned for specific applications.”
BackgroundDual-modality image feature fusion network for gastric precancerous lesions classification
Wang, J., Zhang, B., Wang, Y., Zhou, C., Zou, D., Vonsky, M. S., Mitrofanova, L. B., & Li, Q. (2023) · Biomedical Signal Processing and Control · doi:10.1016/j.bspc.2023.105516 · Cited by 13
Image Dehazing Assessment: A Real-World Dataset and a Haze Density-Aware Criteria
Chen, J., Yang, G., Wang, S., Wang, D., & Liao, X. (2023) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2023.3345141 · Cited by 13
Wang, Y., Ni, D., Huang, Z., & Chen, P. (2024) · Expert Systems with Applications · doi:10.1016/j.eswa.2024.123601 · Cited by 13
Dissecting AI-based mutation prediction in lung adenocarcinoma: A comprehensive real-world study
Dernbach, G., Kazdal, D. N., Ruff, L., Alber, M., Romanovsky, E., Schallenberg, S., Christopoulos, P., Weis, C., Muley, T., Schneider, M. A., et al. (2024) · European Journal of Cancer · doi:10.1016/j.ejca.2024.114292 · Cited by 14
Human-in-the-loop: Using classifier decision boundary maps to improve pseudo labels
Benato, B. C., Grosu, C., Falcão, A. X., & Telea, A. C. (2024) · Computers & Graphics · doi:10.1016/j.cag.2024.104062 · Cited by 14
“The confidence of the apprentice model was included in the loss function to mitigate such problems [ 5,37].”
Merveille, F. F. R., Jia, B., Xu, Z., & Fred, B. (2024) · Sensors · doi:10.3390/s24217034 · Cited by 17
ProST: Prompt Future Snapshot on Dynamic Graphs for Spatio-Temporal Prediction
Xia, K., Lin, L., Wang, S., Zhang, Q., Wang, S., & He, T. (2025) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/3690624.3709273 · Cited by 14
Ahmed, I., Ahmad, M., Chehri, A., & Jeon, G. (2025) · Information Fusion · doi:10.1016/j.inffus.2025.103527 · Cited by 14
End-to-end novel visual categories learning via auxiliary self-supervision
Qing, Y., Zeng, Y., Cao, Q. P., & Huang, G.-B. (2021) · Neural Networks · doi:10.1016/j.neunet.2021.02.015 · Cited by 18
“Self-supervised learning has demonstrated very prominent results in the field of unsupervised feature learning [15].”
BackgroundIterative Pseudo-Labeling with Deep Feature Annotation and Confidence-Based Sampling
Benato, B. C., Telea, A. C., & Falcão, A. X. (2021) · SIBGRAPI Conference on Graphics, Patterns and Images (SIBGRAPI) · doi:10.1109/sibgrapi54419.2021.00034 · Cited by 18
Software Reliability Assessment Using Hybrid Neuro-Fuzzy Model
Gandhi, P., Khan, M. Z., Sharma, R. K., Alhazmi, O. H., Bhatia, S., & Chakraborty, C. (2021) · Computer Systems Science and Engineering · doi:10.32604/csse.2022.019943 · Cited by 13
“The integration of these two paradigms overcomes the limitations of each technique’s isolated concept [14].”
BackgroundLi, M., Chen, D., Liu, S., & Liu, F. (2021) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2021.3133760 · Cited by 13
“However, in view of the success of recent feature metric methods for self-supervised problem [30], [31], unlike common semisupervised methods, our work is based on feature similarity rather than output.”
MethodVisual Domain Adaptation in the Deep Learning Era
Csurka, G., Hospedales, T. M., Salzmann, M., & Tommasi, T. (2022) · Synthesis lectures on computer vision · doi:10.2200/s01169ed1v01y202202cov020 · Cited by 21
Dan, T., Huang, Z., Cai, H., Lyday, R. G., Laurienti, P. J., & Wu, G. (2022) · Human Brain Mapping · doi:10.1002/hbm.25897 · Cited by 15
Evaluation Methods for Representation Learning: A Survey
Nozawa, K., & Sato, I. (2022) · International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2022/776 · Cited by 20
“To do so, unsupervised representation learning studies reviewed in [ Jing and Tian, 2019 ] proposed a novel unsupervised objective function that is called “pretext task” to train h without supervised signals.”
BackgroundSousa, M., Vieira, P. A., Queluz, M. P., & Rodrigues, A. (2022) · IEEE Access · doi:10.1109/access.2022.3193486 · Cited by 14
Ding, Y., Xie, W., Wong, K. K. L., & Liao, Z. (2022) · Computer Methods and Programs in Biomedicine · doi:10.1016/j.cmpb.2022.107049 · Cited by 13
Wei, X., Niu, X.-K., Zhang, X.-S., & Li, Y. (2022) · IEEE International Conference on Big Data (BigData) · doi:10.1109/bigdata55660.2022.10020283 · Cited by 16
Match Cutting: Finding Cuts with Smooth Visual Transitions
Chen, B., Ziai, A., Tucker, R. S., & Xie, Y. (2023) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv56688.2023.00215 · Cited by 18
“Video Representation Learning Self-supervised methods have dominated much of the progress in multi-modal media understanding in recent years [76, 47, 24, 37].”
BackgroundWan, L., Jing, Q., Sun, Z., Zhang, C., Li, Z., & Chen, Y. (2023) · IEEE Transactions on Information Forensics and Security · doi:10.1109/tifs.2023.3273911 · Cited by 14
“of a much larger order of magnitude than our pre-training dataset SYSU-MM01, so that it can offer more accurate batch statistics and stable gradient flow in early training [41].”
Yan, Z., Liu, H., Tao, L., Ma, J., & Cheng, Y. (2023) · Aerospace · doi:10.3390/aerospace10080681 · Cited by 14
Lu, X., Liu, X., Xiao, Z., Zhang, S., Huang, J. J., Yang, C. F., & Liu, S. (2023) · Computers in Biology and Medicine · doi:10.1016/j.compbiomed.2023.107337 · Cited by 17
Word self-update contrastive adversarial networks for text-to-image synthesis
Xiao, J., Sun, Y., & Bi, X. (2023) · Neural Networks · doi:10.1016/j.neunet.2023.08.038 · Cited by 13
Enhancing Person Re-Identification Performance Through In Vivo Learning
Huang, Y., Huang, Y., Zhang, Z., Wu, Q., Zhong, Y., & Wang, L. (2023) · IEEE Transactions on Image Processing · doi:10.1109/tip.2023.3341762 · Cited by 15
“A. Self-Supervised Visual Learning SSL investigates learning representations from unlabeled data which contain semantic or structural information that can be utilized [16].”
BackgroundRevealing the Denoising Principle of Zero-Shot N2N-Based Algorithm from 1D Spectrum to 2D Image
Luo, S.-H., Pan, S., Chen, G., Xie, Y., Ren, B. W., Liu, G., & Tian, Z. (2024) · Analytical Chemistry · doi:10.1021/acs.analchem.3c04608 · Cited by 14
Selective HuBERT: Self-Supervised Pre-Training for Target Speaker in Clean and Mixture Speech
Lin, J., Ge, M., Wang, W., Li, H., & Feng, M. (2024) · IEEE Signal Processing Letters · doi:10.1109/lsp.2024.3383794 · Cited by 17
“S ELF-SUPERVISED pre-training has gained much popularity among the researchers in recent years [1], [2], [3]. pre-training By pre-training the models on large amounts of unannotated speech data, not only is the cost of annotation saved, but the models also generate universal speech representations…”
BackgroundPlebe, A., & Lio, M. D. (2020) · IEEE Access · doi:10.1109/access.2020.3028185 · Cited by 17
“Unlike unsupervised learning, self-supervision is not motivated by biological plausibility; it is instead a way around the ever-present issue of manual data labeling in large datasets of images [44], [45].”
Iterative, Deep Synthetic Aperture Sonar Image Segmentation
Sun, Y.-C., Gerg, I. D., & Monga, V. (2022) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2022.3162420 · Cited by 18
“1) Self-Supervised Methods: For self-supervised learning, pretext tasks [31] are pre-designed for networks to solve, and some characteristics of the data set are utilized to generate the pretext task goals (i.”
MethodComparing Learning Methodologies for Self-Supervised Audio-Visual Representation Learning
Terbouche, H., Schoneveld, L., Benson, O., & Othmani, A. (2022) · IEEE Access · doi:10.1109/access.2022.3164745 · Cited by 13
“They are generally not useful by themselves and they are used to extract semantic representations [2].”
BackgroundTowards Global Video Scene Segmentation with Context-Aware Transformer
Yang, Y., Huang, Y., Guo, W.-L., Xu, B., & Xia, D. (2023) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v37i3.25426 · Cited by 41
“The pseudolabels are automatically created without requiring labeled data (Jing and Tian 2021).”
BackgroundRyali, C. K., Schwab, D. J., & Morcos, A. S. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2103.12719 · Cited by 17
“We do not make a formal distinction between self-/unsupervised learning (but see Jing and Tian (2020)) and broadly discuss related work.”
BackgroundSelf-Supervised Learning for Fine-Grained Image ClassificationPreprint
Breiki, F. A., Ridzuan, M., & Grandhe, R. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2107.13973 · Cited by 13
“Pretext tasks like generation-based, context-based, free semantic label-based and cross modal-based have been used to learn discriminatory features [17, 23, 18].”
BackgroundMethodZhang, G., Zhan, Q., Gao, Q., Mao, K., Yang, P., Gao, Y., Wang, L., Song, B., Chen, Y., Bian, Y., et al. (2024) · Computers in Biology and Medicine · doi:10.1016/j.compbiomed.2024.107989 · Cited by 13
Cross-domain self-supervised few-shot learning via multiple crops with teacher-student network
Wang, G., Wang, Y., Zhang, J., Wang, X., & Pan, Z. (2024) · Engineering Applications of Artificial Intelligence · doi:10.1016/j.engappai.2024.107892 · Cited by 13
Liu, C., Tang, L., Zhang, K., & Xu, X. (2024) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2024.3388103 · Cited by 15
“…predicting accuracy with obvious functional correlations and certain influencing factors, such as kernel learning methods [9], [10], decision trees [11], ensemble learning [12], neural networks [13], [14], DL models [15], [16], [17], and multiobjective evolutionary learning models [18], [19], [20].”
Zhang, Y., Lin, L., Huang, Y., Wang, X., Hsieh, S., Gadekallu, T. R., & Piran, M. J. (2024) · IEEE Internet of Things Journal · doi:10.1109/jiot.2024.3398023 · Cited by 13
“Although these methods mainly target data in the Euclidean domain, anomaly detection of graph-structured data from outside the Euclidean space is also receiving extensive attention in recent years [19].”
Chen, Y., Dang, X., Zhu, D., Huang, Y., & Qin, K. (2024) · Transactions in GIS · doi:10.1111/tgis.13190 · Cited by 12
“When the quantity and quality of these samples are inadequate, the predictive performance of these models often falls short of expectations (Jing & Tian, 2020 ).”
Gong, H., Wang, Z., Huang, S., & Wang, J. (2024) · Biomedical Signal Processing and Control · doi:10.1016/j.bspc.2024.106572 · Cited by 12
Graph Convolutional Network With Self-Supervised Learning for Brain Disease Classification
Wang, G., Chu, Y., Wang, Q., Zhang, L., Qiao, L., & Liu, M. (2024) · IEEE Transactions on Computational Biology and Bioinformatics · doi:10.1109/tcbb.2024.3422152 · Cited by 12
“SSL [27], as a special form of unsupervised learning, is generally used to learn more generalized representations from the available data by solving well-designed pretext tasks.”
Self-Supervised Marine Organism Detection From Underwater Images
Li, J., Yang, W., Qiao, S., Gu, Z., Zheng, B., & Zheng, H. (2024) · IEEE Journal of Oceanic Engineering · doi:10.1109/joe.2024.3455565 · Cited by 13
Opti-Acoustic Semantic SLAM with Unknown Objects in Underwater Environments
Singh, K., Hong, J., Rypkema, N. R., & Leonard, J. J. (2024) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros58592.2024.10802819 · Cited by 19
“Models leveraging unsupervised learning techniques excel in extracting visual features from unseen images [45].”
Towards open-world recognition: Critical problems and challenges
Wang, K., Li, Z., Yang, C., Dong, W., & Chen, J. (2025) · Engineering Applications of Artificial Intelligence · doi:10.1016/j.engappai.2025.110042 · Cited by 12
Chen, F., Zhu, Z., Sun, C., & Xia, L. (2025) · Applied Acoustics · doi:10.1016/j.apacoust.2025.110593 · Cited by 13
Khotimah, W. N., Bennamoun, M., Boussaïd, F., Xu, L., & Sohel, F. A. (2025) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2025.3541263 · Cited by 12
“Self-supervised learning (SSL) offers an alternative by pretraining models on pretext tasks without human annotations [13].”
No-Reference Video Quality Assessment Using Distortion Learning and Temporal Attention
Kossi, K. K., Coulombe, S., Desrosiers, C., & Gagnon, G. (2022) · IEEE Access · doi:10.1109/access.2022.3167446 · Cited by 13
“As in self-supervised representation learning approaches [41], this branch is trained with easy-to-obtain labels, i.e. images corrupted with different types of distortion, to learn a representation in a pre-training phase.”
BackgroundGong, R., Wang, L., Wang, J., Ge, B., Yu, H., & Shi, J. (2022) · Computers in Biology and Medicine · doi:10.1016/j.compbiomed.2022.105641 · Cited by 13
Metadata
- Authors
- Jing, L., & Tian, Y.
- Affiliations
- The Graduate Center, CUNY (US)
- Venue
- IEEE Transactions on Pattern Analysis and Machine Intelligence
- Published
- 2021-11-01
- Volume
- 43
- Issue
- 11
- Pages
- 4037-4058
- ISSN
- 0162-8828 (ISSN-L)
- 2160-9292
- 1939-3539
- OpenAlex
- W3023371261
- Cited by
- 2,036
- References
- 206
- Citations by year
- 2026: 160 · 2025: 283 · 2024: 354 · 2023: 405 · 2022: 377 · 2021: 305
- FWCI
- 119.51
- Type
- journal article
- Language
- en
- Open access
- green
- arxiv.org
- Licence
- Locations
- IEEE Transactions on Pattern Analysis and Machine Intelligence (publishedVersion)
- PubMed (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Funders
- National Science Foundation (IIS-1400802)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Domain Adaptation and Few-Shot Learning
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Image and Video Retrieval Techniques
- Keywords
- self-supervised learning, visual feature learning, imaging features, video features, deep learning architectures, unlabeled data, audio features, 3D data, benchmark dataset, evaluation metrics
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.