Self-Supervised Representation Learning: Introduction, advances, and challenges
Open accessThe DOI matches its Crossref record; 111 of the 125 references it cites were found in a registry or an index, and 1 are not checked yet.
Abstract
Self-supervised representation learning (SSRL) methods aim to provide powerful, deep feature learning without the requirement of large annotated data sets, thus alleviating the annotation bottleneck—one of the main barriers to the practical deployment of deep learning today. These techniques have advanced rapidly in recent years, with their efficacy approaching and sometimes surpassing fully supervised pretraining alternatives across a variety of data modalities, including image, video, sound, text, and graphs. This article introduces this vibrant area, including key concepts, the four main families of approaches and associated state-of-the-art techniques, and how self-supervised methods are applied to diverse modalities of data. We further discuss practical considerations including workflows, representation transferability, and computational cost. Finally, we survey major open challenges in the field, that provide fertile ground for future work.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source doesn’t support the claim as written.
In the citing paper
“The various self-supervised pretexts are divided into five broad families: transformation prediction, masked prediction, instance discrimination, clustering, and contrastive instance discrimination.”
From Self-Supervised Learning for Time Series Analysis: Taxonomy, Progress, and Prospects (2024) · cites this paper as [28]
Checked by a reasoning model — may be wrong.
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“Contrastive learning [56-58] is in fact a training procedure used in recent artificial intelligence models as a way to efficiently learn semantic representations.”
From How Adversarial REM Dreams May Facilitate Creativity, and Why We Become Aware of Them (2024) · cites this paper as [58]
The cited source says what the claim attributes to it.
In the citing paper
“For example, Ericsson et al. [28] provides a very comprehensive review of pretext tasks for multiple modalities, including image, video, text, audio, time series, and graph.”
From Self-Supervised Learning for Time Series Analysis: Taxonomy, Progress, and Prospects (2024) · cites this paper as [28]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474 · cited 4×
“State of the art architectures usually start with CNN representation encoding hθ(·), with ResNet [1] being widely used, before appending task-specific”
InfluentialVaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., & Polosukhin, I. (2017) · arXiv · doi:10.48550/arxiv.1706.03762 · Cited by 120,404 · cited 3×
“long been used to model the context while recent works have moved to Transformer-based architectures with self-attention [2], which allow longer range connections to be made across words in a sentence, but require more data for training.”
Exploiting Generative AI to Scale up Intelligent Tutoring Systems
Jakubův, J., Chvalovský, K., Goertzel, Z., Kaliszyk, C., Olšák, M., Piotrowski, B., Schulz, S., Suda, M., & Urban, J. (2023) · DROPS (Schloss Dagstuhl – Leibniz Center for Informatics) · doi:10.4230/lipics.itp.2023.19 · Cited by 76,658
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2018) · arXiv · doi:10.48550/arxiv.1810.04805 · Cited by 73,173 · cited 9×
“Self-supervised strategies have been leveraged successfully to improve sample efficiency of learning across a variety of modalities, from image [5]–[7], video [8], [9], speech [10], [11], text [12], [13], and graphs [14], [15].”
InfluentialImageNet: A large-scale hierarchical image database
Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., & Fei-Fei, L. (2009) · 2009 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2009.5206848 · Cited by 65,333 · cited 2×
“As in much of computer vision, ImageNet [35] is the most typical source data set for self-supervised pretraining [5]–[7], [40], consisting of 1.28 million training images across 1,000 object categories, with the most commonly used resolution at 224 × 224.”
InfluentialBERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2019) · Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) · doi:10.18653/v1/n19-1423 · Cited by 54,325 · cited 9×
“Self-supervised strategies have been leveraged successfully to improve sample efficiency of learning across a variety of modalities, from image [5]–[7], video [8], [9], speech [10], [11], text [12], [13], and graphs [14], [15].”
InfluentialAn Image is Worth 16x16 Words: Transformers for Image Recognition at Scale
Dosovitskiy, A., Beyer, L., Kolesnikov, A., Weissenborn, D., Zhai, X., Unterthiner, T., Dehghani, M., Minderer, M., Heigold, G., Gelly, S., et al. (2020) · arXiv · doi:10.48550/arxiv.2010.11929 · Cited by 47,472 · cited 1×
“One notable version is a vision transformer [58], which is increasingly being used by recent self-supervised methods on image data [43].”
The Elements of Statistical Learning
Hastie, T., Tibshirani, R., & Friedman, J. (2009) · Springer Series in Statistics · doi:10.1007/978-0-387-84858-7 · Cited by 25,970 · cited 2×
“These range from classic shallow approaches such as Gaussian mixtures [21] to deep methods such as variational autoencoders (VAEs) and generative adversarial networks (GANs) [18].”
Semi-supervised classification with graph convolutional networks
Kipf, T. N., & Welling, M. (2016) · arXiv · doi:10.48550/arxiv.1609.02907 · Cited by 24,853 · cited 1×
“Graph NNs (GNNs) [89] have advanced all these tasks significantly, especially where massive-labeled data are available.”
Distributed Representations of Words and Phrases and their Compositionality
Mikolov, T., Sutskever, I., Chen, K., Corrado, G., & Dean, J. (2013) · arXiv · doi:10.48550/arxiv.1310.4546 · Cited by 24,533 · cited 6×
“With data annotation being a major bottleneck, NLP was the first discipline to make major and successful use of self-supervision [22].”
Influential
Research works printed but not linked: 4
- 0, Youtube-8m A large and diverse labeled video dataset for video understanding research
- gokaslan, 2019, OpenWebText Corpus
- radford, 2019, Language Models are Unsupervised Multitask Learners
- goodfellow, 2016, Deep Learning
Cited by
Items 301–400 of 641
Self-Supervised Learning for Speaker Recognition: A study and review
Lepage, T., & Dehak, R. (2025) · Speech Communication · doi:10.1016/j.specom.2025.103333 · Cited by 4
“Given the growing interest in SSL, several reviews have been published covering the technology [60, 32, 61], as well as its applications in NLP [62, 63, 64], CV [65], and SP [66, 67].”
A Survey on Deep Text Hashing: Efficient Semantic Text Retrieval with Binary RepresentationPreprint
He, L., Huang, Z., Cheng, Y. J., Li, R., Zhang, Z., Zhang, K., Li, Z., Liu, Q., & Chen, E. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2510.27232 · Cited by 4
CLA: Latent Alignment for Online Continual Self-Supervised LearningPreprint
Cignoni, G., Cossu, A., Gomez-Villa, A., van de Weijer, J., & Carta, A. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2507.10434 · Cited by 4
MKCRec: Meta-relation guided Knowledge Coupling for Paper Recommendation
Zhang, C., Zhang, Z., Huang, J., Liu, Y., Jin, D., Xiao, X., & Shen, Z. (2025) · ACM Transactions on Information Systems · doi:10.1145/3715101 · Cited by 4
Audio Analysis for Multimodal Learning
Guzhov, A. (2025) · Publication Server of Kaiserslautern University of Technology (Kaiserslautern University of Technology) · doi:10.26204/kluedo/8804 · Cited by 5
Ma, Y., Zhou, X., Li, X., Liang, L., & Jin, S. (2025) · IEEE Transactions on Cognitive Communications and Networking · doi:10.1109/tccn.2025.3592931 · Cited by 4
“Inspired by the demand for achieving low computational complexity in resource allocation for RIS-assisted downlink MISO-OFDMA wireless systems, this paper investigates an RIS-assisted MISO-OFDMA downlink communication sys-tem.”
Knowledge accumulation in continually learned representations and the issue of feature forgetting
Hess, T., Verwimp, E., van de Ven, G. M., & Tuytelaars, T. (2024) · Lirias (KU Leuven) · Cited by 4
Graph Contrastive Representation Learning with Input-Aware and Cluster-Aware Regularization
Li, J., Li, B., Zhang, Q., Chen, X., Huang, X., Guo, L., & Fu, Y.-G. (2023) · Lecture notes in computer science · doi:10.1007/978-3-031-43415-0_39 · Cited by 4
Yachamaneni, T., Kotadiya, U., & Arora, A. S. (2025) · International Journal of Artificial Intelligence Data Science and Machine Learning · doi:10.63282/3050-9262.ijaidsml-v6i1p118 · Cited by 4
Self-supervised representation learning for cloud detection using Sentinel-2 images
Gbodjo, Y. J. E., Hughes, L. H., Molinier, M., Tuia, D., & Li, J. (2025) · Remote Sensing of Environment · doi:10.1016/j.rse.2025.115205 · Cited by 4
Nirmaladevi, K., Revathi, K., Mohan, K. B. K., Jayapradha, J., & Kumar, T. V. V. P. (2024) · International Journal of Intelligent Systems and Applications in Engineering · Cited by 4
EEG-SSL: A Framework for Self-Supervised Learning on EEG
Truong, D., Khalid, M. A., & Delorme, A. (2024) · IEEE International Conference on Bioinformatics and Biomedicine (BIBM) · doi:10.1109/bibm62325.2024.10822136 · Cited by 4
How Adversarial REM Dreams May Facilitate Creativity, and Why We Become Aware of Them
Deperrois, N., Petrovici, M., Jordan, J., Huber, L., & Senn, W. (2024) · Clinical and Translational Neuroscience · doi:10.3390/ctn8020021 · Cited by 3
Febrinanto, F. G., Moore, K., Thapa, C., Ma, J., & Saikrishna, V. (2025) · Expert Systems with Applications · doi:10.1016/j.eswa.2025.131056 · Cited by 3
Impact of Pre-training Datasets on Human Activity Recognition with Contrastive Predictive Coding
da Silva, B. E. R., Napoli, O. O., Delgado, J. V., Rocha, A. R., Boccato, L., & Borin, E. (2025) · Lecture notes in computer science · doi:10.1007/978-3-031-79035-5_21 · Cited by 3
A study on self-supervised sketch-based image retrieval on unpaired data (S3BIR)
Campos, W., Saavedra, J. M., & Stears, C. (2025) · Neural Computing and Applications · doi:10.1007/s00521-025-11142-4 · Cited by 3
Self-Supervised Scribble-based Segmentation of Single Cells in Biofilms
Bommanapally, V., Subramaniam, M., Talluri, S. N. L., Gadhamshetty, V. R., & Jones, J. (2023) · IEEE International Conference on Bioinformatics and Biomedicine (BIBM) · doi:10.1109/bibm58861.2023.10385729 · Cited by 3
“Various pretext tasks include learning by augmentation such as rotation prediction, image inpainting, jigsaw puzzle, and colorization [1].”
BackgroundLearning beyond sensations: how dreams organize neuronal representationsPreprint
Deperrois, N., Petrovici, M. A., Senn, W., & Jordan, J. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2308.01830 · Cited by 3
“The authors thus conclude that creative dreams are a key ingredient for the acquisition of semantic latent representations.”
BackgroundMethodWahidi, M., Rizk, A., & Imad, R. (2025) · IEEE Access · doi:10.1109/access.2025.3589514 · Cited by 3
“Fortunately, new learning paradigms such as unsupervised [105], semi-supervised [106], self-supervised [107], and reinforcement learning (RL) [7] have emerged as promising solutions to this bottleneck.”
Multimodal transformers for emotion recognition
Vazquez-Rodriguez, J. (2023) · theses.fr (ABES) · doi:10.70675/a1927be2zdba6z405azbe6fz6856529b46bb · Cited by 3
Bai, F., Wei, R., Bai, X., Jin, D., Ye, X., Lü, L., Yan, K., & Meng, M. Q. ‐. (2025) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2025.3558861 · Cited by 3
Deng, Y., Wu, X., Gao, X., Li, W., & Zhang, D. (2025) · Journal of Neural Engineering · doi:10.1088/1741-2552/ae0070 · Cited by 3
ViewMix: Augmentation for Robust Representation in Self-Supervised Learning
Das, A., & Zhong, X. R. (2024) · IEEE Access · doi:10.1109/access.2024.3353133 · Cited by 3
“Since the joint embedding architectures experience aggressive forms of image transformations, the models become invariant [12] to certain distortions.”
BackgroundReinforcement Learning with Decoupled State Representation for Robot Manipulations
Dong, K., Zeng, Y., Wang, K., Luo, Y., Wang, Y., Cheng, E., Sun, Z., Zhang, Q., & Song, B. (2024) · Neural Processing Letters · doi:10.1007/s11063-024-11650-9 · Cited by 3
Yao, Y., Chen, H., Wang, J., & Wang, Y. (2025) · Microorganisms · doi:10.3390/microorganisms13071635 · Cited by 3
“Self-supervised learning [23] has emerged as a promising alternative, offering the ability to extract effective representations from unlabeled data.”
Nasersharif, B., & Azad, M. (2023) · doi:10.1109/icee59167.2023.10334799 · Cited by 3
“SSL methods have been extensively used in a variety of modalities such as image, video, speech, text, and graphs [4].”
MethodSrivast, D., Singh, V., Song, L., & Kochersberger, K. B. (2024) · Research Square · doi:10.21203/rs.3.rs-3865180/v1 · Cited by 3
SS-CPGAN: Self-Supervised Cut-and-Pasting Generative Adversarial Network for Object Segmentation
Chaturvedi, K., Braytee, A., Li, J., & Prasad, M. (2023) · Sensors · doi:10.3390/s23073649 · Cited by 3
“In this way, the networks learn good feature representations with the help of pseudo labels created by the pretext tasks [27].”
BackgroundKim, T. H., Krichen, M., Ojo, S., Sampedro, G. A., & Alamro, M. A. (2024) · Frontiers in Computational Neuroscience · doi:10.3389/fncom.2024.1414462 · Cited by 3
“SS-DRPL has become a popular and interesting method in machine learning over the past few years (Ericsson et al., 2022).”
MethodEnvironment-Aware Knowledge Distillation for Improved Resource-Constrained Edge Speech Recognition
Pimentel, A., Guimarães, H. R., Avila, A. R., & Falk, T. H. (2023) · Applied Sciences · doi:10.3390/app132312571 · Cited by 3
Generation of Synthetic Echocardiograms Using Video Diffusion Models
Pellicer, A. O., Yadav, A. K. S., Bhagtani, K., Xiang, Z., Pizlo, Z., Gradus‐Pizlo, I., & Delp, E. J. (2024) · IEEE Southwest Symposium on Image Analysis and Interpretation (SSIAI) · doi:10.1109/ssiai59505.2024.10508643 · Cited by 3
“Self-supervised deep learning methods learn feature representations from unlabeled data by using pretext tasks such as reconstruction of input data from the learned feature representations [5].”
MethodIslam, F., Nabi, M. M., Ball, J. E., & Goodin, C. T. (2024) · IEEE Access · doi:10.1109/access.2024.3457685 · Cited by 3
“The basic difference between SSL with unsupervised learning is that the SSL method creates its own labels from the input data [47].”
MethodNext-Gen AIGC: a review of multimodal foundation models for text-to-media innovations
Jin, C., Fan, J., Huang, J., Fu, J., Mei, T., Yuan, L., & Luo, J. (2026) · Frontiers of Computer Science · doi:10.1007/s11704-025-51171-9 · Cited by 3
SimEXT: Self-supervised Representation Learning for Extreme Values in Time Series
Galib, A. H., Tan, P., & Luo, L. (2023) · Proceedings · doi:10.1109/icdm58522.2023.00119 · Cited by 3
“Existing SSL approaches, although promising for extracting meaningful time series representations [4], [5], often prioritize common patterns over extreme values.”
BackgroundHassan, S. U., Adil, M., Khan, N. A., Khan, M. K., & Jin, Z. (2026) · IEEE Internet of Things Journal · doi:10.1109/jiot.2026.3695869 · Cited by 3
“2) Self-Supervised and Physics-Guided Learning for Label E ffi ciency and Robustness: Self-supervised learning has emerged as a promising direction for mmWave health monitoring because labeled radar datasets remain small, fragmented, and expensive to annotate [79].”
Self-supervised representation learning on gene expression data
Dradjat, K., Hamidi, M., Bartet, P., & Hanczar, B. (2025) · Bioinformatics · doi:10.1093/bioinformatics/btaf533 · Cited by 3
“These advances are largely due to recent self-supervised methods, which construct a new data representation space with a pre-training step, making prediction tasks easier (Ericsson et al. 2022).”
Accelerated green material and solvent discovery with chemistry- and physics-guided generative AI
Al-Sakkari, E. G., Ragab, A., Benali, M., Ajao, O., Boffito, D. C., & Dagdougui, H. (2026) · Artificial Intelligence Chemistry · doi:10.1016/j.aichem.2025.100106 · Cited by 3
ACL-ECG: Anatomy-Aware Contrastive Learning for Multi-Lead Electrocardiograms
Liu, W., Wu, Z., & Yuan, Z. (2026) · Sensors · doi:10.3390/s26031080 · Cited by 3
“To comprehensively assess the quality of the learned representations, two standard transfer learning evaluation protocols are adopted: linear probing and fine-tuning [34].”
Research on Image Representation Learning Method Based on Self-Supervised Learning
Zhang, J. (2026) · International Scientific Technical and Economic Research · doi:10.71451/istaer2616 · Cited by 3
“As a method that mines potential structural information from data without manual annotation, self-supervised learning has gradually shown great potential in image representation learning [3] , [4] , [5] .”
Transfer or Self-Supervised? Bridging the Performance Gap in Medical ImagingPreprint
Zhao, Z., Alzubaidi, L., Zhang, J., Duan, Y., Naseem, U., & Gu, Y. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2407.05592 · Cited by 3
“…aspects such as the suitability of different architectures, the relevance of source and target domains, the exploration of various pretext tasks, analysis of the impact of the size of the pre-training dataset, and evaluation of the model capacity on the performance of downstream tasks [5, 6].”
Score Combining for Contrastive OOD DetectionPreprint
Reehorst, E. T., & Schniter, P. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2501.12204 · Cited by 3
Zhang, K., Gao, Q. Z., OGORZAŁEK, M. J., Liu, H., & Lü, J. (2025) · International Journal of Robust and Nonlinear Control · doi:10.1002/rnc.7908 · Cited by 3
Representation learning for a generalized, quantitative comparison of complex model outputsPreprint
Cess, C. G., & Finley, S. D. (2022) · arXiv (Cornell University) · doi:10.48550/arxiv.2208.06530 · Cited by 3
Tackling Low-Resource ECG Classification with Self-supervised Learning
da Silva, R. B. P., & Silva, D. F. (2025) · Lecture notes in computer science · doi:10.1007/978-3-031-79038-6_17 · Cited by 2
da Luz, G. P. C. P., Napoli, O. O., Delgado, J. V., Rocha, A. R., Boccato, L., & Borin, E. (2025) · Lecture notes in computer science · doi:10.1007/978-3-031-79035-5_6 · Cited by 2
Applications of Self-Supervised Learning to Biomedical Signals: where are we nowPreprint
Pup, F. D., & Atzori, M. (2023) · doi:10.36227/techrxiv.22567021.v2 · Cited by 2
Edge AI on Constrained Devices: A Survey of Deployment, Optimization, and Lifecycle Management
Passarotto, M., Abdesslem, F. B., Eriksson, J., & Voigt, T. (2026) · ACM Transactions on Embedded Computing Systems · doi:10.1145/3827611 · Cited by 2
Bostancioglu, A., Daş, B., Kaya, M. O., Dağdögen, H. A., & Daş, R. (2026) · Expert Systems with Applications · doi:10.1016/j.eswa.2026.132678 · Cited by 2
The Promise of Self-Supervised Learning for Dental Caries
Vinh, T. Q., & Byeon, H. (2023) · International Journal of Advanced Computer Science and Applications · doi:10.14569/ijacsa.2023.0140907 · Cited by 2
“Self-supervised representation learning [6] is a type of machine learning that learns to represent data without being explicitly labeled.”
BackgroundWeng, X., Song, H., Lin, Y., Zhang, X., Liu, B., Wu, Y., & Yang, J. (2024) · Information Sciences · doi:10.1016/j.ins.2024.120108 · Cited by 2
From Pixels to Waveforms: Evaluating Pre-trained Image Models for Few-Shot Audio Classification
Heggan, C., Hospedales, T., Budgett, S., & Yaghoobi, M. (2024) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn60899.2024.10650537 · Cited by 2
“Self-supervised pre-training coupled with linear readout or fine-tuning has become increasingly popular due to the increased effectiveness of modern self-supervision and increased dataset scale [8].”
BackgroundSelf-citationGMCL: Graph Mask Contrastive Learning for Self-Supervised Graph Representation Learning
Xu, L., Pan, Z., Chen, H., & Zhou, T. (2024) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn60899.2024.10650219 · Cited by 2
“Self-Supervised Learning (SSL) stands as a crucial subfield within the realm of machine learning, aiming to address the challenge of acquiring labels in supervised learning by learning from unlabeled data and autonomously generating labels or tasks [1].”
BackgroundSelf-supervised 3D Skeleton Completion for Vascular Structures
Ren, J., Li, Z., Cheng, W., Zou, Z., Park, K., Pan, Y., & Ling, H. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-72120-5_54 · Cited by 2
“Recently, self-supervised learning (SSL) [9,32] addresses the scarcity of ground truth data through a set of training schemes, garnering increasing attention [7].”
BackgroundSuperCoT-X: Masked Hyperspectral Image Modeling With Diverse Superpixel-Level Contrastive Tokenizer
Liang, M., Wu, W., Shen, H., Yu, L., Yu, X., & Jiao, L. (2025) · IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing · doi:10.1109/jstars.2025.3586617 · Cited by 2
Torrents, J., Costa, T., & de Polavieja, G. G. (2025) · eLife · doi:10.7554/elife.107602.2 · Cited by 2
Morandé, S. (2023) · Qeios · doi:10.32388/znkj97 · Cited by 2
Kataoka, A., Nagano, Y., & Oizumi, M. (2025) · Frontiers in Computational Neuroscience · doi:10.3389/fncom.2025.1613291 · Cited by 2
“Recent studies have demonstrated the effectiveness of self-supervised learning as a powerful method for representation learning (Arora et al., 2019; Medina et al., 2020; Chen and He, 2020; Newell and Deng, 2020; Ericsson et al., 2022; Nozawa and Sato, 2021; Shi et al., 2021; Wang et al., 2021; Bao…”
Deep learning processing of remotely sensed multi-spectral images
Restaino, R., Carbone, A., & Vivone, G. (2025) · Elsevier eBooks · doi:10.1016/b978-0-44-326484-9.00012-9 · Cited by 2
Yifan, 陈. C., Sun, Z., & Xuelong, 李. L. (2025) · Laser & Optoelectronics Progress · doi:10.3788/lop251007 · Cited by 2
Balestriero, R., & LeCun, Y. (2026) · IEEE Signal Processing Magazine · doi:10.1109/msp.2026.3659059 · Cited by 2
A deep feature refinement self-supervised learning algorithm for medical image annotation
Zhang, J., Li, D., Wu, Y., Zhao, Z., Wang, Y., Li, Y., & Zhang, B. (2025) · Applied Intelligence · doi:10.1007/s10489-025-06737-2 · Cited by 2
Galib, A. H., McDonald, A., Tan, P., & Luo, L. (2023) · International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2023/414 · Cited by 2
“Self-supervised learning [Ericsson et al. , 2022; Jaiswal et al. , 2020 ] offers a possible solution to remedy this problem by extracting useful features from the available time series and using the information for missing value imputation.”
BackgroundBallas, A., Papapanagiotou, V., & Diou, C. (2024) · IEEE Access · doi:10.1109/access.2024.3519297 · Cited by 2
“Self-supervised representation learning (SSL) [28] was initially proposed for mitigating the limit posed by the requirement of large annotated datasets for practical and production-ready deep learning systems.”
Kaczmarek, E., Szeto, J., Nichyporuk, B., & Arbel, T. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-16128-4_1 · Cited by 2
Efficiency-oriented approaches for self-supervised speech representation learning
Lugo, L., & Vielzeuf, V. (2023) · International Journal of Speech Technology · doi:10.1007/s10772-024-10121-9 · Cited by 2
Evain, S. V. (2024) · theses.fr (ABES) · doi:10.70675/1ee1ec61za8e2z465azbd08z950ac2a14d7f · Cited by 2
Multiple representation contrastive self-supervised learning for pulmonary nodule detection✰
Torki, A., Adibi, P., & Kashani, H. B. (2024) · Knowledge-Based Systems · doi:10.1016/j.knosys.2024.112307 · Cited by 2
Zhang, M., Gu, X., Qi, J., Zhang, Z., Yang, H., Xu, J., Peng, C., & Li, H. (2024) · Remote Sensing · doi:10.3390/rs16071293 · Cited by 2
“The basic pipeline of the second TL paradigm is ‘pre-training + fine-tuning’ [30].”
BackgroundExploring Sensor Modalities to Capture User Behaviors for Reading Detection
Islam, M. R., Vargo, A. W., Iwata, M., Iwamura, M., & Kise, K. (2022) · IEICE Transactions on Information and Systems · doi:10.1587/transinf.2020zdl0003 · Cited by 2
Individualized, self-supervised deep learning for blood glucose predictionPreprint
Fuest, J., Tacke, M., Ullman, L., & Washington, P. Y. (2023) · medRxiv · doi:10.1101/2023.08.19.23294318 · Cited by 2
“The degree to which self-supervised learning enhances performance is heavily dependent on the features the model learns during self-supervised training [34].”
BackgroundWang, R., Cheng, Y., & Wang, X. (2025) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2025.3556838 · Cited by 2
“Self-supervised representation learning in RL. Inspired by the significant advancements of self-supervised representation learning in computer vision [28], an alternative category of approaches focuses on incorporating auxiliary representation tasks in conjunction with the RL objective [29].”
Dey, M., & Rana, S. P. (2025) · Energies · doi:10.3390/en18215611 · Cited by 2
Multimodal Deep Learning and Foundation Models for Early Detection and Forecasting of Plant Diseases
Manda, T., Huang, T., Ding, Y., Dai, S., Yang, L., & Dai, T. (2026) · Plants · doi:10.3390/plants15172564 · Cited by 2
“…refers to a class of representation learning methods in which a model is trained on pretext tasks derived from the structure of unlabeled data itself such as predicting masked regions or maximizing agreement between augmented views of the same image without requiring human-annotated labels [29].”
Representation learning of single-cell time-series with deep variational autoencodersPreprint
Fraisse, A., Oyarzún, D. A., & Karoui, M. E. (2025) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2025.09.22.677729 · Cited by 2
Understanding Ice Crystal Habit Diversity with Self-Supervised LearningPreprint
Ko, J., Govindarajan, H., Lindsten, F., Przybylo, V. M., Sulia, K., van Lier‐Walqui, M., & Lamb, K. D. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2509.07688 · Cited by 2
“Self-supervised learning (SSL) is an effective approach to learn informative representations of unlabeled data [13, 14].”
KINLI: Time Series Forecasting for Monitoring Poultry Health in Complex Pen Environments
Pack, C. I., Zeiser, T., Beecks, C., & Lutz, T. (2025) · Animals · doi:10.3390/ani15213180 · Cited by 2
“This led to research on the creation of time series foundational models [35–38].”
Representation-based ECG signal prediction forneural networks pre-training
Ставиченко, С. Г., & Filatova, A. Y. (2025) · Herald of Advanced Information Technology · doi:10.15276/hait.08.2025.7 · Cited by 2
Khanh, G. L., Cong, T. D., & Lan, H. H. T. (2025) · The Transport and Communications Science Journal · doi:10.47869/tcsj.77.1.10 · Cited by 2
“Self-supervised learning (SSL) addresses this bottleneck by enabling models to learn meaningful latent representations directly from raw, unlabeled data through pretext tasks such as masking, reconstruction, and contrastive learning [9].”
Li, K., Yang, S., Gao, C., & Marsic, I. (2023) · International Workshop on Semantic Evaluation (SemEval) · doi:10.18653/v1/2023.semeval-1.204 · Cited by 2
“Vision language pre-training aims to learn the semantic correspondence between vision and language modalities through self-supervised learning (Li et al., 2022b; Ericsson et al., 2022).”
BackgroundJiang, Y., Miao, Q., Dewei, W. U., He, J., & Zhao, C. (2026) · Drones · doi:10.3390/drones10050339 · Cited by 2
Hu, Y., Duan, X., Chen, Y., & Zhao, Z. (2024) · Information Technology And Control · doi:10.5755/j01.itc.53.3.34856 · Cited by 2
“This has laid a solid foundation for future research [6].”
BackgroundGlobal Context Enhanced Anomaly Detection of Cyber Attacks via Decoupled Graph Neural NetworksPreprint
Hafez, A. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2409.15304 · Cited by 2
Centerpoints Are All You Need in Overhead ImageryPreprint
Inder, J. M., Lowell, M., & Maltenfort, A. J. (2022) · arXiv (Cornell University) · doi:10.48550/arxiv.2210.01857 · Cited by 2
Auto Labelling of Vessel Trajectories for Maritime Downstream Tasks
Amoke, D. A., Li, Y.-C., & Naqvi, S. M. (2025) · Lecture notes in networks and systems · doi:10.1007/978-981-96-6438-2_38 · Cited by 2
SimCDL: A Simple Framework for Contrastive Dictionary Learning
Ilie-Ablachim, D. C., & Dumitrescu, B. (2024) · Applied Sciences · doi:10.3390/app142210082 · Cited by 2
Secure Visual Data Processing via Federated Learning
Santos, P. N. P., Carvalho, T., Magalhães, F., & Antunes, L. F. (2025) · International Conference on Information Systems Security and Privacy (ICISSP) · doi:10.5220/0013183000003899 · Cited by 2
“Recent advances emphasize self-supervised learning, which scales well with unlabeled data (Ericsson et al., 2022).”
Context-Aware Augmentation for Contrastive Self-Supervised Representation Learning
Sepanj, H., & Fieguth, P. (2024) · Journal of Computational Vision and Imaging Systems · doi:10.15353/jcvis.v9i1.10001 · Cited by 2
Scene-Centric Unsupervised Video Panoptic SegmentationPreprint
Reich, C., Hahn, O., Araslanov, N., Leal-Taixé, L., Rupprecht, C., Cremers, D., & Roth, S. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2606.04925 · Cited by 2
“Self-supervised representation learning aims to learn expressive and transferable visual representations from unlabeled data [30].”
ICE-T: Interactions-aware Cross-column Contrastive Embedding for Heterogeneous Tabular Datasets
Tokár, T., & Sanner, S. (2025) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v39i20.35385 · Cited by 2
Piano performance beat assessment: integrating transformer with multimodal feature learning
Deng, J. (2025) · International Journal of Information and Communication Technology · doi:10.1504/ijict.2025.149992 · Cited by 2
Usmonova, S., Meliyevna, S. F., Shukurov, B., Shadiyeva, G., Nazarova, D., Abduganieva, S., & Isakov, B. N. (2026) · International Academic Journal of Science and Engineering · doi:10.71086/iajse/v13i2/iajse1365 · Cited by 2
Guo, E., & Zuo, Q. (2025) · Measurement Science and Technology · doi:10.1088/1361-6501/ade322 · Cited by 2
Building More Efficient AI Models through Unsupervised Representation Learning
Mishra, S., Bandi, S., Komandla, V., & Konidala, S. (2024) · International Journal of Artificial Intelligence Data Science and Machine Learning · doi:10.63282/3050-9262.ijaidsml-v5i3p112 · Cited by 2
Domingues, R. K., & Cordioli, J. A. (2024) · Annual Conference of the PHM Society · doi:10.36001/phmconf.2024.v16i1.4183 · Cited by 2
“This transfer of learning can be done in two ways: either by retraining only the part of the model responsible for producing the expected result for the target task or by retraining the entire model (Ericsson, Gouk, Loy, & Hospedales, 2022).”
Rì, F. A. D., Stefani, D., Turchet, L., & Conci, N. (2025) · Institutional Research Information System (Università degli Studi di Trento) · Cited by 2
Ward, T. B., Wang, X., McFarland, B. J., Ahamed, M. A., Nozad, S., Arshad, T., Nebbache, H., Chen, J., & Imran, A.-A.-Z. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2506.21006 · Cited by 2
Muna, U. M., Shawon, M. M. H., Jobayer, M., Akter, S., Hasan, M. R., & Alam, M. G. R. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2510.10719 · Cited by 2
“To overcome this data-supervision bottleneck, Self-Supervised Learning (SSL) has worked as a powerful alternative and effective paradigm, leveraging unlabeled data to learn robust and generalizable representations [21, 22].”
Improved 2D Hand Trajectory Prediction with Multi-View Consistency
Ma, J., Zhang, E., Xu, J., Chen, X., & Wang, H. (2025) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros60139.2025.11246827 · Cited by 1
“The first 500 epochs only involve reducing L diff k ( ˆ G sub , G sub ) as a pretraining stage, which is inspired by self-supervised representation learning techniques [29], [34], [35].”
VETE: improving visual embeddings through text descriptions for eCommerce search engines
Martínez, G., Saavedra, J. M., & Murrugarra-Llerena, N. (2023) · Multimedia Tools and Applications · doi:10.1007/s11042-023-14595-8 · Cited by 1
Feature representation learning for human activity recognition
Khaertdinov, B. (2024) · Research Publications (Maastricht University) · doi:10.26481/dis.20240618bk · Cited by 1
Metadata
- Authors
- Ericsson, L., Gouk, H., Loy, C. C., & Hospedales, T. M.
- Affiliations
- University of Edinburgh (GB)
- Nanyang Technological University (SG)
- Venue
- IEEE Signal Processing Magazine
- Published
- 2022-05
- Volume
- 39
- Issue
- 3
- Pages
- 42-62
- ISSN
- 1053-5888 (ISSN-L)
- 1558-0792
- OpenAlex
- W3207924272
- Cited by
- 433
- References
- 125
- Citations by year
- 2026: 89 · 2025: 105 · 2024: 132 · 2023: 86 · 2022: 20
- FWCI
- 28.22
- Type
- journal article
- Language
- en
- Open access
- green
- arxiv.org
- Licence
- Locations
- IEEE Signal Processing Magazine (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- Edinburgh Research Explorer (University of Edinburgh) (submittedVersion, cc-by-nd)
- DR-NTU (Nanyang Technological University) (submittedVersion)
- arXiv (Cornell University)
- Funders
- Engineering and Physical Sciences Research Council (EP/S000631/1, EP/L016427/1, EP/R026173/1)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Domain Adaptation and Few-Shot Learning
- Physical Sciences › Computer Science › Artificial Intelligence › Advanced Graph Neural Networks
- Physical Sciences › Computer Science › Artificial Intelligence › Machine Learning and Algorithms
- Keywords
- self-supervised representation learning, pre-training, deep feature learning, image representation, video representation, sound representation, text representation, graph representation, computational cost, open problems
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.