Self-Supervised Representation Learning: Introduction, advances, and challenges
Open accessThe DOI matches its Crossref record; 111 of the 125 references it cites were found in a registry or an index, and 1 are not checked yet.
Abstract
Self-supervised representation learning (SSRL) methods aim to provide powerful, deep feature learning without the requirement of large annotated data sets, thus alleviating the annotation bottleneck—one of the main barriers to the practical deployment of deep learning today. These techniques have advanced rapidly in recent years, with their efficacy approaching and sometimes surpassing fully supervised pretraining alternatives across a variety of data modalities, including image, video, sound, text, and graphs. This article introduces this vibrant area, including key concepts, the four main families of approaches and associated state-of-the-art techniques, and how self-supervised methods are applied to diverse modalities of data. We further discuss practical considerations including workflows, representation transferability, and computational cost. Finally, we survey major open challenges in the field, that provide fertile ground for future work.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source doesn’t support the claim as written.
In the citing paper
“The various self-supervised pretexts are divided into five broad families: transformation prediction, masked prediction, instance discrimination, clustering, and contrastive instance discrimination.”
From Self-Supervised Learning for Time Series Analysis: Taxonomy, Progress, and Prospects (2024) · cites this paper as [28]
Checked by a reasoning model — may be wrong.
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“Contrastive learning [56-58] is in fact a training procedure used in recent artificial intelligence models as a way to efficiently learn semantic representations.”
From How Adversarial REM Dreams May Facilitate Creativity, and Why We Become Aware of Them (2024) · cites this paper as [58]
The cited source says what the claim attributes to it.
In the citing paper
“For example, Ericsson et al. [28] provides a very comprehensive review of pretext tasks for multiple modalities, including image, video, text, audio, time series, and graph.”
From Self-Supervised Learning for Time Series Analysis: Taxonomy, Progress, and Prospects (2024) · cites this paper as [28]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474 · cited 4×
“State of the art architectures usually start with CNN representation encoding hθ(·), with ResNet [1] being widely used, before appending task-specific”
InfluentialVaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., & Polosukhin, I. (2017) · arXiv · doi:10.48550/arxiv.1706.03762 · Cited by 120,404 · cited 3×
“long been used to model the context while recent works have moved to Transformer-based architectures with self-attention [2], which allow longer range connections to be made across words in a sentence, but require more data for training.”
Exploiting Generative AI to Scale up Intelligent Tutoring Systems
Jakubův, J., Chvalovský, K., Goertzel, Z., Kaliszyk, C., Olšák, M., Piotrowski, B., Schulz, S., Suda, M., & Urban, J. (2023) · DROPS (Schloss Dagstuhl – Leibniz Center for Informatics) · doi:10.4230/lipics.itp.2023.19 · Cited by 76,658
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2018) · arXiv · doi:10.48550/arxiv.1810.04805 · Cited by 73,173 · cited 9×
“Self-supervised strategies have been leveraged successfully to improve sample efficiency of learning across a variety of modalities, from image [5]–[7], video [8], [9], speech [10], [11], text [12], [13], and graphs [14], [15].”
InfluentialImageNet: A large-scale hierarchical image database
Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., & Fei-Fei, L. (2009) · 2009 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2009.5206848 · Cited by 65,333 · cited 2×
“As in much of computer vision, ImageNet [35] is the most typical source data set for self-supervised pretraining [5]–[7], [40], consisting of 1.28 million training images across 1,000 object categories, with the most commonly used resolution at 224 × 224.”
InfluentialBERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2019) · Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) · doi:10.18653/v1/n19-1423 · Cited by 54,325 · cited 9×
“Self-supervised strategies have been leveraged successfully to improve sample efficiency of learning across a variety of modalities, from image [5]–[7], video [8], [9], speech [10], [11], text [12], [13], and graphs [14], [15].”
InfluentialAn Image is Worth 16x16 Words: Transformers for Image Recognition at Scale
Dosovitskiy, A., Beyer, L., Kolesnikov, A., Weissenborn, D., Zhai, X., Unterthiner, T., Dehghani, M., Minderer, M., Heigold, G., Gelly, S., et al. (2020) · arXiv · doi:10.48550/arxiv.2010.11929 · Cited by 47,472 · cited 1×
“One notable version is a vision transformer [58], which is increasingly being used by recent self-supervised methods on image data [43].”
The Elements of Statistical Learning
Hastie, T., Tibshirani, R., & Friedman, J. (2009) · Springer Series in Statistics · doi:10.1007/978-0-387-84858-7 · Cited by 25,970 · cited 2×
“These range from classic shallow approaches such as Gaussian mixtures [21] to deep methods such as variational autoencoders (VAEs) and generative adversarial networks (GANs) [18].”
Semi-supervised classification with graph convolutional networks
Kipf, T. N., & Welling, M. (2016) · arXiv · doi:10.48550/arxiv.1609.02907 · Cited by 24,853 · cited 1×
“Graph NNs (GNNs) [89] have advanced all these tasks significantly, especially where massive-labeled data are available.”
Distributed Representations of Words and Phrases and their Compositionality
Mikolov, T., Sutskever, I., Chen, K., Corrado, G., & Dean, J. (2013) · arXiv · doi:10.48550/arxiv.1310.4546 · Cited by 24,533 · cited 6×
“With data annotation being a major bottleneck, NLP was the first discipline to make major and successful use of self-supervision [22].”
Influential
Research works printed but not linked: 4
- 0, Youtube-8m A large and diverse labeled video dataset for video understanding research
- gokaslan, 2019, OpenWebText Corpus
- radford, 2019, Language Models are Unsupervised Multitask Learners
- goodfellow, 2016, Deep Learning
Cited by
Items 401–500 of 641
Semi-Supervised Drivable Road Segmentation with Expanded Feature Cross-Consistency
Ma, S., & Song, C. (2023) · Applied Sciences · doi:10.3390/app132112036 · Cited by 1
“As deep learning becomes mainstream, methods that can balance low data annotation and higher accuracy are deserving of our attention, such as deep transfer learning [5], domain adaptation [37], self-supervised learning [6,7], and semi-supervised learning [8].”
BackgroundAN OVERVIEW OF MULTIMODAL LEARNING: CONCEPTS, CHALLENGES, APPLICATIONS AND DATASETS
Ateş, A. H., & Yurtsever, U. (2026) · Konya Journal of Engineering Sciences · doi:10.36306/konjes.1716813 · Cited by 1
Bilgisayarlı Görüde Öz-Denetimli Öğrenme Yöntemleri Üzerine Bir İnceleme
Alasu, S., & Talu, M. F. (2024) · Düzce Üniversitesi Bilim ve Teknoloji Dergisi · doi:10.29130/dubited.1201292 · Cited by 1
Tassi, S. C., Polyzos, K. D., Dimos, S. S., Polyzos, D. K., Fotiadis, D. I., & Sakellarios, A. I. (2025) · IEEE Journal of Biomedical and Health Informatics · doi:10.1109/jbhi.2025.3609142 · Cited by 1
“Inlightofthese challenges, self-supervised learning (Self-SL) has emerged as a transformative strategy that relies solely on unlabeled data, making it particularly appealing in practical settings where labeled data is scarce and expensive to acquire [16], [17].”
Ohri, K., Sukheja, D., Kumar, P., Arora, S., & Kumar, M. (2026) · IEEE Access · doi:10.1109/access.2026.3722626 · Cited by 1
“Ericsson et al. [11] survey self-supervised representation learning in IEEE Signal Processing Magazine, with an emphasis on evaluation protocols and general challenges.”
Performance of Small Language Model Pretraining on FABRIC: An Empirical Study
Rao, P. S. (2026) · IEEE International Conference on Data Engineering Workshops (ICDEW) · doi:10.1109/icdew71238.2026.00019 · Cited by 1
“Large language models (LLMs) (a.k.a. foundation models), which are trained on a large corpus of unlabeled data via self-supervised learning [5], have become the bedrock of generative AI.”
Transformers for Payment Transactions Via Periodic Positional Encoding and Contrastive Learning
Yamashita, M., Kosan, M., Zhao, Y., Liu, C., Agrawal, S., & Chetia, C. (2025) · IEEE International Conference on Big Data (BigData) · doi:10.1109/bigdata66926.2025.11402594 · Cited by 1
“The key to their success lies in carefully designed pretraining objectives such as masked language modeling (MLM) and next token prediction, which enable models to learn rich representations from raw text [6].”
A Survey on Self-Supervised Learning in Cybersecurity: Network Intrusion and Malware Detection
Almaraz-Rivera, J. G., Cantoral-Ceballos, J. A., & Botero, J. F. (2026) · Machine Learning and Knowledge Extraction · doi:10.3390/make8050121 · Cited by 1
“In the state of the art, we can find surveys from 2021 that explain the potential of this novel learning paradigm for videos, images, audio, text, and graphs [2,5,10,12,21,22].”
InfluentialHow Adversarial Rem-Dreams May Facilitate Creativity, and Why We Become Aware of ThemPreprint
Deperrois, N., Petrovici, M. A., Jordan, J., Huber, L. S., & Senn, W. (2024) · Preprints.org · doi:10.20944/preprints202403.0684.v2 · Cited by 1
Deep learning for change detection in 3D point clouds
de Gélis, I. (2023) · theses.fr (ABES) · doi:10.70675/9ca6d4f1z61b8z4b99zaa0dz1d47d0b86cc0 · Cited by 1
Dash, B. M. (2024) · theses.fr (ABES) · doi:10.70675/83ceb975z9781z42e0zaac9z041c66a1a139 · Cited by 1
Neural networks and deep learning: part II
Theodoridis, S. (2025) · Elsevier eBooks · doi:10.1016/b978-0-44-329238-5.00025-1 · Cited by 1
Kim, J., Kim, S., Seo, S., Kim, B., Mun, D., Lee, H. J., & Hwang, S. (2025) · Applied Artificial Intelligence · doi:10.1080/08839514.2025.2462389 · Cited by 1
Fang, L., Ding, S., Feng, H., Yu, J., Tang, L., & Ghamisi, P. (2024) · CAAI Transactions on Intelligence Technology · doi:10.1049/cit2.12399 · Cited by 1
Torrents, J., Costa, T., & de Polavieja, G. G. (2026) · eLife · doi:10.7554/elife.107602.3 · Cited by 1
CLIB: Contrastive learning of ignoring background for underwater fish image classification
Yan, Q., Du, X., Li, C., & Tian, X. (2024) · Frontiers in Neurorobotics · doi:10.3389/fnbot.2024.1423848 · Cited by 1
Goyo, M. A., & Hidalgo, M. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-76607-7_10 · Cited by 1
A Comparative Study of Generative Adversarial Networks for Generating Car Damaged ImagesPreprint
Kyu, P. M., & Woraratpanya, K. (2023) · Research Square · doi:10.21203/rs.3.rs-2509412/v1 · Cited by 1
A Comprehensive Survey of Wireless Foundation Models for AI-Native 6G NetworksPreprint
Khan, N., Sbeihi, B. A., Alshehhi, M., & Saeed, N. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2608.14694 · Cited by 1
“2, current learning paradigms can be broadly grouped into self-supervised learning, contrastive representation learning, masked modeling, and multimodal pre-training [16], [49].”
Wang, P., Majumdar, A., Liu, W.-T., Kang, J., Guðnason, J., Chen, Y., Liu, I.-J., Lee, K.-Y., Chen, T.-T., Lee, H., et al. (2026) · Expert Systems with Applications · doi:10.1016/j.eswa.2026.133852 · Cited by 1
Automated diagnosis of cancer bone metastases using multi-view contrastive learning
Jian, X., Lin, Q., Zeng, X., Cao, Y., Man, Z., Liu, C., Cai, Z., Gui, C., Ji, J., & Huang, X. (2025) · Physica Scripta · doi:10.1088/1402-4896/ade91a · Cited by 1
Machine Learning In Design Engineering and Manufacturing
Toro, J. V. (2023) · Linköping studies in science and technology. Thesis · doi:10.3384/9789180754569 · Cited by 1
Lip and Speech Synchronization using Supervised Contrastive Learning and Cross-Modal Attention
Varshney, M., Mukherji, M., Senthil Raja, G., Ganesh, A., & Banerjee, K. (2024) · International Conference on Automatic Face and Gesture Recognition · doi:10.1109/fg59268.2024.10581985 · Cited by 1
“The supervised contrastive learning method was introduced by Khosla et. al. in [12] where the paper proposed to use multiple positive samples instead of one positive sample in triplet loss and self-supervised contrastive learning [8], [23], [16], [25].”
MethodSelf-supervised CondenseNet for feature learning to increase the accuracy in image classification
Darvish-Motevali, M., Sohrabi, M. K., & Roshdi, I. (2024) · Multimedia Tools and Applications · doi:10.1007/s11042-024-18477-5 · Cited by 1
Altaf, F., Islam, S. M. S., Janjua, N. K., & Akhtar, N. (2023) · Lecture notes in computer science · doi:10.1007/978-3-031-25825-1_9 · Cited by 1
Torrents, J., Costa, T., & de Polavieja, G. G. (2025) · eLife · doi:10.7554/elife.107602.1 · Cited by 1
Cardiac Classification with Multi-Scale Convolutional Neural Network From Paper ECGPreprint
Cheng, X., Jiang, Y. F., & Peng, G. (2025) · medRxiv · doi:10.1101/2025.10.05.25337357 · Cited by 1
“Several deep learning architectures, including a convolutional neural network (CNN) [17], a long short-term memory (LSTM) network [39], and a self-supervised learning (SSL) model based on autoencoders [40], are proposed and trained on the CPEIC cardiac dataset [32].”
JPSSL: SAR Terrain Classification Based on Jigsaw Puzzles and FC-CRF
Ren, Z., Lu, Y., Hou, B., Li, W., & Sha, F. (2024) · Remote Sensing · doi:10.3390/rs16091635 · Cited by 1
“The development and challenges of self-supervised learning are introduced in [25].”
Li, T., & Feng, L. (2024) · Frontiers in Ecology and Evolution · doi:10.3389/fevo.2023.1267259 · Cited by 1
“Model optimization and deep learning are critical directions to promote the coordinated development of rural ecological construction and carbon neutrality Ericsson et al. (2022).”
Selecting Unlabeled Data for Tabular Self-Supervised Learning
Stevanoska, S., Dost, K., Villalón, C. L. C., & Džeroski, S. (2025) · Lecture notes in computer science · doi:10.1007/978-3-032-05461-6_16 · Cited by 1
Safari, F., Tree, J. J., & Vafaee, F. (2025) · Briefings in Bioinformatics · doi:10.1093/bib/bbaf701 · Cited by 1
A Survey of Molecular Representation Learning: From Single Modalities to Foundation ModelsPreprint
Harnik, Y., Specktor, E., Cohen, Y., Milo, A., & Pakman, A. (2026) · ChemRxiv · doi:10.26434/chemrxiv-2025-vxsvs/v2 · Cited by 1
A Survey of Molecular Representation Learning: From Single Modalities to Foundation ModelsPreprint
Harnik, Y., Specktor, E., Cohen, Y., Milo, A., & Pakman, A. (2025) · ChemRxiv · doi:10.26434/chemrxiv-2025-vxsvs · Cited by 1
Reducing Labeled Data Dependency with Self-Supervised Learning: Advances and applications
Tyagi, H., Bhardwaj, M. K., Bipin, Tyagi, S., Singh, P., & Himanshu (2026) · Lecture notes in networks and systems · doi:10.1007/978-3-032-30005-8_20 · Cited by 1
CIM: Interpretable model with consistency representation and prompt learning
Dong, L., Li, Z., Chen, L., Zheng, C., Peng, X., Li, Y., & Cui, X. (2026) · Information Processing & Management · doi:10.1016/j.ipm.2026.104881 · Cited by 1
Lüken, M., Garcia‐Bernardo, J., Deb, S., Hafner, F., & Khosla, M. (2026) · Scientific Reports · doi:10.1038/s41598-026-56690-4 · Cited by 1
“To make accurate predictions, deep learning models typically encode input data into latent numerical representations—so-called embeddings [10, 11].”
Pedro, A. P. S. (2026) · UEF eRepo (University of Eastern Finland) · Cited by 1
Ek, S. (2024) · theses.fr (ABES) · doi:10.70675/a47253c1zf05az480aza992zc137c06fbc2f · Cited by 1
Barchid, S. (2023) · theses.fr (ABES) · doi:10.70675/0eeab9b1z60afz4c09z9289z9107c824f423 · Cited by 1
Improving Joint Embedding Predictive Architecture with Diffusion NoisePreprint
Qiu, Y., Zhu, R., & Chen, Y.-C. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2507.15216 · Cited by 1
“Besides, the methodology of diffusion model is adding noise and then denoising, which is consistent with the philosophy of SSL: corruption first and then reconstruction [2, 28, 13].”
Stuhr, B. (2023) · Tesis Doctorals en Xarxa (Consorci de Serveis Universitaris de Catalunya) · doi:10.48550/arxiv.2312.00101 · Cited by 1
Jamali, A. (2023) · HARVEST (University of Saskatchewan) · Cited by 1
A Systematic Analysis of In-Domain and Out-of-Domain Strategies for Self-Supervised Learning in HAR
Silva, B. E. R. D., Soto, D. H. P., Rocha, A. D. R., Boccato, L., & Borin, E. (2026) · IEEE Access · doi:10.1109/access.2026.3719080 · Cited by 1
Zhu, J., An, L., Zhu, H., Jiang, F., Liu, S., Wei, B., & Yi, C. (2026) · IEEE Transactions on Neural Systems and Rehabilitation Engineering · doi:10.1109/tnsre.2026.3698236 · Cited by 1
“Therecent success of deep learning in representation learning [33], [34] has motivated relevant works in extracting dFC representations.”
Semi-Supervised Contrastive Learning for Label-Efficient Electrocardiogram Noise Detection
Rahman, S., & Karmakar, C. K. (2026) · IEEE Transactions on Instrumentation and Measurement · doi:10.1109/tim.2026.3664602 · Cited by 1
“To address the limitations associated with labeled data dependency, self-supervised learning (SSL) has gained significant attention as a paradigm that enables e ff ective representation learning by leveraging unlabeled data through the formulation of proxy tasks or pretext objectives [15].”
Sharing to learn and learning to share : Fitting together metalearning and multi-task learning
Upadhyay, R. (2023) · DiVA (Luleå University of Technology) · Cited by 1
Curriculum-guided dynamic division strategy for graph contrastive learning
Lin, Y., Zhang, Q., Li, J., Gong, X.-T., & Fu, Y.-G. (2024) · Knowledge-Based Systems · doi:10.1016/j.knosys.2024.112163 · Cited by 1
CRC-RL: A Novel Visual Feature Representation Architecture for Unsupervised Reinforcement LearningPreprint
Jain, D., Majumder, A., Dutta, S., & Kumar, S. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2301.13473 · Cited by 1
“The feature representations are learned using self-supervised methods which are increasingly becoming popular as they obviate the need for manually generated labeled datasets thereby simplifying the practical deployment of deep learning models [18].”
MethodZhang, T., & Zhang, D. (2024) · International Conference on Image Processing, Intelligent Control and Computer Engineering (IPICE) · doi:10.1145/3691016.3691032 · Cited by 1
“Furthermore, CNN architectures have seen significant advancements in computer vision tasks [ 4 , 8 - 10 ] : an attention mechanism is introduced in CNNs, which allows the network to focus on different areas when processing images [ 4 ] .”
Learning temporal embeddings from electronic health records of chronic kidney disease patientsPreprint
Kumar, A., Cypko, M. A., & Amft, O. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2601.18675 · Cited by 1
Phonetic-based Ranking for Improved Pseudo-Labeling in Low-Resource ASR
Matassoni, M., Gretter, R., Falavigna, D., Nabih, M., Brutti, A., Negri, M., Cettolo, M., Gaido, M., Papi, S., & Bentivogli, L. (2026) · Language Resources and Evaluation Conference (LREC) · doi:10.63317/338dnb8n7e85 · Cited by 1
Anatomy of Conflict: A Tool for Understanding Conflict Dynamics from Social Media Data
Sun, D., Li, J., Lyu, Y., Liu, X., Youn, C., Zhan, J., & Abdelzaher, T. (2025) · International Conference on Smart Applications, Communications and Networking (SmartNets) · doi:10.1109/smartnets65254.2025.11106782 · Cited by 1
“Modern techniques for representation learning are generally self-supervised [13], meaning that they do not require data labels.”
Mun, H., Yang, T., Nam, Y., & Choi, C. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2607.06948 · Cited by 1
“…point cloud models, such as Point-BERT (Yu et al., 2022), Point-MAE (Pang et al., 2022), and MaskSurf (Y. Zhang et al., 2022), have been shown to outperform supervised learning models on downstream shape classification and part segmentation tasks (Ericsson et al., 2022; Zeng et al., 2023).”
A Supervised Contrastive Learning Pretrain-Finetune Approach for Time SeriesPreprint
Tran, T. H., Nguyen, L. M., Yeo, K., Nguyen, N. H., & Vaculín, R. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2311.12290 · Cited by 1
“In recent years, there has been significant advancements in self-supervised representation learning [Ericsson et al., 2022, Jaiswal et al., 2020, Misra and Maaten, 2020] with applications on time series [Kiyasseh et al., 2021, Tonekaboni et al., 2021, Franceschi et al., 2019, Eldele et al., 2021,…”
Nahar, S., Rahman, M. M., Rahman, M. M., Rahman, M. M. H., Ullah, M. S., & Hossain, M. S. (2023) · World Journal of Advanced Engineering Technology and Sciences · doi:10.30574/wjaets.2023.10.1.0279 · Cited by 1
Self-trained Panoptic SegmentationPreprint
Verma, S. (2023) · arXiv (Cornell University) · doi:10.13140/rg.2.2.35112.14081 · Cited by 1
Efficient Distribution Matching of Representations via Noise-Injected Deep InfoMax
Butakov, I. D., Semenenko, A., Tolmachev, A., Gladkov, A., Munkhoeva, M., & Frolov, A. (2024) · International Conference on Learning Representations · doi:10.48550/arxiv.2410.06993 · Cited by 1
“Existing approaches to representation learning can be divided into three categories (Ericsson et al., 2022): supervised (requires labeled data), self-supervised and unsupervised (no labeling is required).”
BackgroundWang, X., Liu, C., Mu, R., Chen, Y., Gong, D., Yang, Q., & Liu, Q. (2025) · Current Medical Imaging Formerly Current Medical Imaging Reviews · doi:10.2174/0115734056374967250704090646 · Cited by 1
SARL: Spatially-Aware Self-Supervised Representation Learning for Visuo-Tactile Perception
Khurana, G., Wei, L., & Zhang, D. (2026) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra57385.2026.11696245 · Cited by 1
“Self-supervised representation learning (SSL) [10] has become central in robotics by enabling models to learn from interaction data without manual annotation, improving sample efficiency and generalisation [11].”
Mask-embedded transformer for English text recognition and correction
Sang, H. (2025) · International Journal of Information and Communication Technology · doi:10.1504/ijict.2025.10073376 · Cited by 1
Zhou, A., Zhu, C., & Wang, X. (2025) · Discover Applied Sciences · doi:10.1007/s42452-025-08089-5 · Cited by 1
“…the challenges of self-supervised learning and pointed out that the inconsistency with downstream task objectives is a performance bottleneck, which directly leads to the optimization conflict between self-supervised pre-training and reinforcement learning strategy objectives in this study [17].”
Zou, Y., Liang, T., Jafri, S. R. U. N., Xu, Z., Zhou, L., & Xiong, B. (2026) · Sensors · doi:10.3390/s26010308 · Cited by 1
“Self-supervised representation learning has been explored to pretrain point cloud encoders using geometric consistency [41], contrastive objectives [42], or reconstruction-based tasks [43], providing transferable features for downstream segmentation with limited labeled data.”
Wang, Y., Yao, Q., Sun, J., Zhang, Y., & Feng, L. (2026) · Applied Soft Computing · doi:10.1016/j.asoc.2026.115382 · Cited by 1
Leveraging Weak annotations for Deep learning tasks on Biofilm Images
Bommanapally, V., Ashaduzzman, M., Subramaniam, M., Talluri, S. N. L., & Gadhamshetty, V. R. (2022) · IEEE International Conference on Bioinformatics and Biomedicine (BIBM) · doi:10.1109/bibm55620.2022.9995289 · Cited by 1
“Existing studies show that SSL techniques have been efficient in extracting better representations from the data leading to better performance in any downstream task like classification, detection or segmentation [2].”
MethodTriplet network with autoencoder for remote sensing image scene recognition and classification
hu, F., dai, Y., & wu, L. (2024) · International Conference on Image, Signal Processing and Pattern Recognition (ISPP) · doi:10.1117/12.3034132 · Cited by 1
COINS: Counting Cones Using Inpainting Based Self-supervised Learning
Bommanapally, V., Akhavanrezayat, A., Nguyen, Q. D., & Subramaniam, M. (2024) · Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC) · doi:10.1109/embc53108.2024.10782790 · Cited by 1
“SSL techniques design pretext tasks that are suitable for the target downstream tasks to extract features from the unlabeled data [14].”
Cluster Specific Representation LearningPreprint
Sabanayagam, M., Al-Dabooni, O., & Esser, P. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2412.03471 · Cited by 1
Rare-Aware Autoencoding: Reconstructing Spatially Imbalanced DataPreprint
Garcia, A., van Gemert, J. V., Brinks, D., & Tomen, N. (2026) · arXiv.org · doi:10.48550/arxiv.2604.02031 · Cited by 2
“Reasons for variance in the spatial location can be traced back to acquisition difficulties [16,38,48], either for a single source [17], or a continuous stream such as video [37,46].”
Artificial neural networks in magnetic resonance relaxometry
Parasram, T., & Xiao, D. (2024) · Annual reports on NMR spectroscopy · doi:10.1016/bs.arnmr.2024.05.003
Zhang, B., Wang, Z., Wang, S. T., Qin, C., Xia, P., & Liu, C. (2026) · Advanced Engineering Informatics · doi:10.1016/j.aei.2026.105140
Machine Learning and Pattern Recognition in Affective Computing
Cabada, R. Z., López, H. M. C., & Escalante, H. J. (2023) · Springer eBooks · doi:10.1007/978-3-031-32542-7_2
Self-Supervised Representation Learning for Human Activity Recognition Using Inertial Sensor Data
Khatun, M. M., Ahsan, M. S., Al Amin, M., Eva, A. A., Sayeema, A., & Firdaus, S. (2025) · 2025 International Conference on Quantum Photonics, Artificial Intelligence, and Networking (QPAIN) · doi:10.1109/qpain66474.2025.11171865
“Contrastive learning techniques, in particular, have achieved remarkable success in computer vision and natural language processing by maximizing agreement between augmented views of the same sample while minimizing agreement across different samples [16].”
Self-Supervised CNN for Video Double Compression Detection via Artifact Prediction
Zhang, R., Wang, J., Luo, J., & Zhou, Y. (2026) · Lecture notes in computer science · doi:10.1007/978-981-95-9284-5_21
Lu, H.-E., Koivisto, D., Lou, Y., Zeng, Z., Yu, T., Wang, J., Meng, X., Nowikow, C., Wilson, R., Kumbhare, D. A., et al. (2026) · medRxiv · doi:10.64898/2026.04.07.26350333
Exploring Data-Oriented Strategies for Training ECG Self-supervised Models
Silva, R. D. C., Medronha, A., & Silva, D. F. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-15990-8_15
Tin, D., Gharakheili, H. H., & Batista, G. (2026) · Lecture notes in computer science · doi:10.1007/978-981-92-1926-1_6
“It relies on data augmentation and aims to map different augmented views of input samples into a feature space, where similar views (positive pairs) are pulled closer together, and dissimilar ones (negative pairs) are pushed apart [8].”
Latent-consistency-constrained self-supervised representation stabilisation
Wang, C. (2026) · Cyber-Physical Systems · doi:10.1080/23335777.2026.2696420
Chu, Y., & Zhou, Y. (2026) · International Conference on Power Electronics and Electric Drives (PEED) · doi:10.1109/peed69047.2026.00055
“…learning has been widely recognized as a general strategy to reduce annotation dependence by constructing surrogate objectives from unlabeled data, which is particularly aligned with industrial electronics where normal-operation data are abundant while fault labels are limited and expensive [2].”
A Self-Supervised Auxiliary Learning Approach for Effective Depression Screening
Hickey, K., Flores, R., Shrestha, A., & Rundensteiner, E. (2025) · International Conference on Machine Learning and Applications · doi:10.1109/icmla66185.2025.00168
“By compelling the model to reconstruct these masked segments, we employ self-supervision to foster the learning of richer representations of facial dynamics [13].”
How Human-Centered Is AI-Aided Learning in Education?
Prezenski, S. (2026) · Springer eBooks · doi:10.1007/978-3-032-26816-7_21
How Rapid-Eye-Movement Dreams May Facilitate Learning and CreativityPreprint
Deperrois, N., Petrovici, M. A., Jordan, J., Huber, L. S., & Senn, W. (2024) · Preprints.org · doi:10.20944/preprints202403.0684.v1
Gbodjo, Y. J. E., & Ienco, D. (2026) · SSRN Electronic Journal · doi:10.2139/ssrn.6890917
Loc-Aware: Integrating Geographic Metadata into Contrastive Learning
Carvalho, J., Rambour, C., Breloy, A., & Castillo-Navarro, J. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-39385-2_38
Zuo, Z., Zhuang, C., Gao, P., Qin, J., Feng, H., & Sebe, N. (2026) · International Journal of Computer Vision · doi:10.1007/s11263-026-03008-6 · Cited by 1
LogLessen: A Representation Learning Approach to Log Compression via Variational Autoencoders
Gan, X., Chen, N., & Wang, W. (2026) · Lecture notes in computer science · doi:10.1007/978-981-92-3410-3_9
Li, T., Gu, C., & Zhao, L. (2026) · Applied Sciences · doi:10.3390/app16199824
Benchmarking Object Detection and a Novel Self-Supervised Framework for Weed Detection in Soybean
Srivastava, D., Singh, V., Wamanse, R., Li, S., Kochersberger, K. B., Virk, S. S., & Yadav, P. (2026) · Remote Sensing · doi:10.3390/rs18111720
Towards Unsupervised Domain-Specific Open-World RecognitionPreprint
Alfarisy, G. A. F., Malik, O. A., & Ong, W.-H. (2024) · SSRN Electronic Journal · doi:10.2139/ssrn.4719921
Towards Unsupervised Domain-Specific Open-World RecognitionPreprint
Alfarisy, G. A. F., Malik, O. A., & Ong, W.-H. (2024) · SSRN Electronic Journal · doi:10.2139/ssrn.4854548
Fu, Y.-G., Chen, X., Xu, S., Li, J., Zhang, Q., Meng, W., & Liu, G. (2025) · Knowledge-Based Systems · doi:10.1016/j.knosys.2025.114048
Masuyama, N., Fujii, R., Kinoshita, T., & Nojima, Y. (2025) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn64981.2025.11227370
“Many studies have shown the superiority of SSL for extracting representations of a dataset in several domains such as images, videos, natural language processing, and graphs [2].”
HMSDD: Hybrid Multi-Scale Deepfake Detection through Self-Supervised Learning and Frequency Analysis
Prakash, P., & Jaffino, G. (2026) · IEEE International Conference on Emerging Computing and Intelligent Technologies (ICoECIT) · doi:10.1109/icoecit68303.2026.11497562
“A capable model for learning generalizable representations has emerged by leveraging self-supervised learning (SSL) [5].”
Domain Adaptation in HTR through Self-Supervised Visual Pretraining
N, A. D. R., Janugam, P., Sangubotla, R., Pasha, S. J., & Rohith, R. (2026) · International Conference on Trends in Electronics and Informatics (ICOEI) · doi:10.1109/icoei68323.2026.11541710
“Stepped self-supervised learning (SSL), which is considered to be one of the promising solutions that allows our models to learn meaningful features directly from data [15,16].”
Statistical Contrastive Learning for Spatio-Temporal Anomaly Detection
Zhen, Z., Chen, Y., & Gel, Y. R. (2026) · Data Science in Science · doi:10.1080/26941899.2026.2648108
Wu, Y., Liu, B., Li, C., & Kong, Y. (2026) · Mathematical Geosciences · doi:10.1007/s11004-026-10318-5
Kim, J., & Kwon, G. (2026) · Electronics Letters · doi:10.1049/ell2.70658
Liang, T., Du, L., & Lan, G. (2026) · Ad Hoc Networks · doi:10.1016/j.adhoc.2026.104406
Encrypted Network Traffic Classification Using Generative and Contrastive Self-Supervised Learning
Towhid, M. S., & Shahriar, N. (2026) · IEEE Transactions on Networking · doi:10.1109/ton.2026.3737411
“Usually, a self-supervised learning algorithm has two stages [10], [11], [12], [18].”
Multimodal Motion Tracking: A Review
Aidoo, E., Custance, O., Tenagyei, E. K., Dai, C., Chen, M.-S., & Xu, Z.-J. (2026) · IEEE Sensors Journal · doi:10.1109/jsen.2026.3701620
“Self-Supervised Pretraining: Exploiting intrinsic data structures to learn representations without explicit labels, thereby facilitating e ffi cient use of unlabeled data [231].”
Metadata
- Authors
- Ericsson, L., Gouk, H., Loy, C. C., & Hospedales, T. M.
- Affiliations
- University of Edinburgh (GB)
- Nanyang Technological University (SG)
- Venue
- IEEE Signal Processing Magazine
- Published
- 2022-05
- Volume
- 39
- Issue
- 3
- Pages
- 42-62
- ISSN
- 1053-5888 (ISSN-L)
- 1558-0792
- OpenAlex
- W3207924272
- Cited by
- 433
- References
- 125
- Citations by year
- 2026: 89 · 2025: 105 · 2024: 132 · 2023: 86 · 2022: 20
- FWCI
- 28.22
- Type
- journal article
- Language
- en
- Open access
- green
- arxiv.org
- Licence
- Locations
- IEEE Signal Processing Magazine (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- Edinburgh Research Explorer (University of Edinburgh) (submittedVersion, cc-by-nd)
- DR-NTU (Nanyang Technological University) (submittedVersion)
- arXiv (Cornell University)
- Funders
- Engineering and Physical Sciences Research Council (EP/S000631/1, EP/L016427/1, EP/R026173/1)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Domain Adaptation and Few-Shot Learning
- Physical Sciences › Computer Science › Artificial Intelligence › Advanced Graph Neural Networks
- Physical Sciences › Computer Science › Artificial Intelligence › Machine Learning and Algorithms
- Keywords
- self-supervised representation learning, pre-training, deep feature learning, image representation, video representation, sound representation, text representation, graph representation, computational cost, open problems
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.