Contrastive Representation Learning: A Framework and Review
Open accessThe DOI matches its Crossref record; 147 of the 162 references it cites were found in a registry or an index, and 15 are not checked yet.
Abstract
Contrastive Learning has recently received interest due to its success in self-supervised representation learning in the computer vision domain. However, the origins of Contrastive Learning date as far back as the 1990s and its development has spanned across many fields and domains including Metric Learning and natural language processing. In this paper, we provide a comprehensive literature review and we propose a general Contrastive Representation Learning framework that simplifies and unifies many different contrastive learning methods. We also provide a taxonomy for each of the components of contrastive learning in order to summarise it and distinguish it from other forms of machine learning. We then discuss the inductive biases which are present in any contrastive learning system and we analyse our framework under different views from various sub-fields of Machine Learning. Examples of how contrastive learning has been applied in computer vision, natural language processing, audio processing, and others, as well as in Reinforcement Learning are also presented. Finally, we discuss the challenges and some of the most promising future research directions ahead.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“According to the suggestions of Le-Khac et al. [29], the main methods can be divided into the following categories: multisensory signals, data augmentation, localglobal consistency, and temporal consistency.”
From Self-Supervised Learning for Time Series Analysis: Taxonomy, Progress, and Prospects (2024) · cites this paper as [29]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Contrastive learning [56-58] is in fact a training procedure used in recent artificial intelligence models as a way to efficiently learn semantic representations.”
From How Adversarial REM Dreams May Facilitate Creativity, and Why We Become Aware of Them (2024) · cites this paper as [56]
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“As summarized in [29], commonly used contrastive loss functions generally include scoring functions (cosine similarity), energy-based margin functions (pair loss and triplet loss), probabilistic NCE-based functions, and mutual information based functions.”
From Self-Supervised Learning for Time Series Analysis: Taxonomy, Progress, and Prospects (2024) · cites this paper as [29]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474 · cited 2×
“In SimCLRv2 [17], empirical experiments show that the output of the second layer of a 3-layer MLP projection head is a better representation for transfer learning than the output of the base ResNet [42] encoder.”
BackgroundMethodVaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., & Polosukhin, I. (2017) · arXiv · doi:10.48550/arxiv.1706.03762 · Cited by 120,404 · cited 3×
“Recently, the Bidirectional Encoder Representation from Transformer (BERT) [24] model learns bidirectional word representations using the Transformer architecture’s decoder [103] and demonstrated great performance for transfer learning in multiple downstream tasks.”
MethodHochreiter, S., & Schmidhuber, J. (1997) · Neural Computation · doi:10.1162/neco.1997.9.8.1735 · Cited by 102,318 · cited 1×
“Even though the BERT [24] text encoder and the Faster RCNN [87] object detection model are trained separately and not updated by the contrastive loss, the LSTM cells [48] and a linear project head can still map between words and object representations.”
MethodExploiting Generative AI to Scale up Intelligent Tutoring Systems
Jakubův, J., Chvalovský, K., Goertzel, Z., Kaliszyk, C., Olšák, M., Piotrowski, B., Schulz, S., Suda, M., & Urban, J. (2023) · DROPS (Schloss Dagstuhl – Leibniz Center for Informatics) · doi:10.4230/lipics.itp.2023.19 · Cited by 76,658
Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks
Ren, S., He, K., Girshick, R., & Sun, J. (2017) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2016.2577031 · Cited by 73,472
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2018) · arXiv · doi:10.48550/arxiv.1810.04805 · Cited by 73,173 · cited 5×
“The discretised speech representation can then be used directly as a representation for other models that expect discrete input such as BERT [24].”
MethodInfluentialFaster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks
Ren, S., He, K., Girshick, R., & Sun, J. (2015) · arXiv · doi:10.48550/arxiv.1506.01497 · Cited by 70,812
ImageNet: A large-scale hierarchical image database
Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., & Fei-Fei, L. (2009) · 2009 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2009.5206848 · Cited by 65,333
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2019) · Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) · doi:10.18653/v1/n19-1423 · Cited by 54,325 · cited 5×
“The discretised speech representation can then be used directly as a representation for other models that expect discrete input such as BERT [24].”
MethodInfluentialEfficient Estimation of Word Representations in Vector Space
Mikolov, T., Chen, K., Corrado, G., & Dean, J. (2013) · arXiv · doi:10.48550/arxiv.1301.3781 · Cited by 26,568
Research works printed but not linked: 1
- stratos, 2019, Noise contrastive estimation
Cited by
Items 201–300 of 948
Enhancing bibliographic reference parsing with contrastive learning and prompt learning
Yin, Z., & Wang, S. (2024) · Engineering Applications of Artificial Intelligence · doi:10.1016/j.engappai.2024.108548 · Cited by 17
Soft Contrastive Sequential Recommendation
Zhang, Y., Wang, Z., Yu, W., Hu, L., Jiang, P., Gai, K., & Chen, X. (2024) · ACM Transactions on Information Systems · doi:10.1145/3665325 · Cited by 20
“CL is a paradigm which learns an effective feature representation for discriminative models by positive pairs and negative pairs.”
Fine-tuning protein embeddings for functional similarity evaluation
Dickson, A. M., & Mofrad, M. R. K. (2024) · Bioinformatics · doi:10.1093/bioinformatics/btae445 · Cited by 16
Rodrigues, C. N., Nunes, I. M., Pereira, M. B., Oliveira, H., & dos Santos, J. A. (2024) · Computers & Graphics · doi:10.1016/j.cag.2024.104014 · Cited by 15
Wu, J., Ma, J., Xi, H., Li, J., & Zhu, J. (2024) · Neural Networks · doi:10.1016/j.neunet.2024.106914 · Cited by 16
Yuan, Y., Lin, L., Xin, Q., Zhou, Z.-G., & Liu, Q. (2025) · IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing · doi:10.1109/jstars.2025.3527017 · Cited by 20
“It also suggests incorporating these techniques with advanced semi-supervised learning [66] and self-supervised learning [67] frameworks, where data augmentation serves as a key component.”
Farmanesh, A., Sanchis, R. G., & Ordieres-Meré, J. B. (2025) · Sensors · doi:10.3390/s25103048 · Cited by 17
“CL learns an embedding space where similar samples are mapped close together while dissimilar samples are pushed apart, enabling effective feature learning even with limited labeled data [48,49].”
Machine Learning-Based Analysis of Free-Text Keystroke Dynamics
Chang, H.-C., Li, J., & Stamp, M. (2022) · Advances in information security · doi:10.1007/978-3-030-97087-1_14 · Cited by 15
Moebel, E., & Kervrann, C. (2022) · Computer Methods and Programs in Biomedicine · doi:10.1016/j.cmpb.2022.107017 · Cited by 16
SMiLE: Schema-augmented Multi-level Contrastive Learning for Knowledge Graph Link Prediction
Peng, M., Liu, B., Xie, Q., Xu, W., Wang, H., & Peng, M. (2022) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2022.findings-emnlp.307 · Cited by 20
Jin, J., Song, Y., Kan, D., Zhang, B., Lyu, Y., Zhang, J., & Lu, H. (2024) · Information Processing & Management · doi:10.1016/j.ipm.2024.103673 · Cited by 14
Kim, D. H., Kim, H., Jung, Y., Kim, S., & Song, B. C. (2024) · Neurocomputing · doi:10.1016/j.neucom.2024.127588 · Cited by 17
Uncertainty-Aware Contrastive Learning for semi-supervised named entity recognition
Yang, K., Yang, Z., Zhao, S., Yang, Z., Zhang, S., & Chen, H. (2024) · Knowledge-Based Systems · doi:10.1016/j.knosys.2024.111762 · Cited by 17
Zhu, L., Wang, Y., Hu, Y., Su, X., & Fu, K. (2024) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2024.3417421 · Cited by 24
“In contrast, contrastive learning [20] enables the model to learn more hidden distinguishing features, which helps similar samples approach each other more closely and dissimilar samples move away.”
BackgroundTSFF: A Triple-Stream Feature Fusion Method for Ethereum Phishing Scam Detection
Hou, W., Cui, B., Chen, Y., Li, R., & Song, W. (2024) · IEEE Internet of Things Journal · doi:10.1109/jiot.2024.3473771 · Cited by 14
“Contrastive learning can learn richer feature representations by contrasting the similarity and differences between sequences [19].”
Evaluating few-shot and contrastive learning methods for code clone detection
Khajezade, M., Fard, F. H., & Shehata, M. (2024) · Empirical Software Engineering · doi:10.1007/s10664-024-10441-z · Cited by 15
“A code clone detection algorithm based on deep learning aims to find a function Φ to compare the representation of two code fragments and assign a label to each pair based on their similarity.”
Meng, S., Su, X., Sun, G., Li, M., & Xue, F. (2025) · Computers Environment and Urban Systems · doi:10.1016/j.compenvurbsys.2025.102255 · Cited by 18
Contrastive Active Learning Under Class Distribution Mismatch
Du, P., Chen, H., Zhao, S., Chai, S., Chen, H., & Li, C. (2022) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2022.3188807 · Cited by 20
“Several contrastive techniques, such as SimCLR [28] and Momentum Contrast (MoCo) [36], can learn representations at a category-level [26], [30], [31], [37], which are instrumental in the downstream classification task.”
MethodChen, J., & Stouffs, R. M. F. A. (2022) · Computer-Aided Civil and Infrastructure Engineering · doi:10.1111/mice.12908 · Cited by 14
“Embed-ding specialized inductive biases that mirror certain aspects of information processing can lead to structured models supporting dynamic construct and novel inferences (Le-Khac et al., 2020).”
BackgroundDeep-learned event variables for collider phenomenology
Kim, D., Kong, K., Matchev, K. T., Park, M., & Shyamsundar, P. (2023) · Physical review. D/Physical review. D. · doi:10.1103/physrevd.107.l031904 · Cited by 14
Momentum Contrast for Unsupervised Visual Representation Learning
Sowe, E. A. (2025) · Journal of Advances in Civil and Mechanical Engineering · doi:10.64030/3067-2457.02.01.02 · Cited by 14
Machine learning for enzyme catalytic activity: current progress and future horizons
Qiu, S., Saeed, H., Leonard, W., Li, F., & Yang, A. (2026) · Briefings in Bioinformatics · doi:10.1093/bib/bbag002 · Cited by 15
“Subsequently, FusionESP [42] used embeddings generated by ESM-2 [43] for enzymes and MoLFormer [44] for substrates to train a contrastive learning model [45] in place of FCNN, and reached a higher classification accuracy than ProSmith on the same test set, 94.77% versus 94.2%.”
InfluentialReal-Time Cattle Interaction Recognition via Triple-stream Network
Yang, Y., Komatsu, M., Ohkawa, T., & Oyama, K. (2022) · International Conference on Machine Learning and Applications · doi:10.1109/icmla55696.2022.00016 · Cited by 14
Si, J., Jia, W., & Jiang, H. (2024) · IEEE Access · doi:10.1109/access.2024.3366185 · Cited by 14
“Learning comparative relationships between samples improves model generalization and robustness, enabling better handling of complex data and tasks [32], [33].”
BackgroundPhysics-Guided Dual Self-Supervised Learning for Structure-Based Material Property Prediction
Fu, N., Wei, L., & Hu, J. (2024) · The Journal of Physical Chemistry Letters · doi:10.1021/acs.jpclett.4c00100 · Cited by 17
“Intuitively, contrastive learning can be considered as learning by comparing [31], which pushes the network to implicitly learn to extract the essential features or representations of the data that do not vary with the instance transformations or augmentations.”
BackgroundContrastive learning for enhancing feature extraction in anticancer peptides
Lee, B., & Shin, D. (2024) · Briefings in Bioinformatics · doi:10.1093/bib/bbae220 · Cited by 16
“Contrastive learning, a self-supervised representation learning, operates by pairing datasets to closely align features of positive pairs while distancing those of negative pairs, subsequently learning meaningful representations in data [30].”
BackgroundMethodAn autoencoder-based self-supervised learning for multimodal sentiment analysis
Feng, W., Wang, X., Cao, D., & Lin, D. (2024) · Information Sciences · doi:10.1016/j.ins.2024.120682 · Cited by 13
Triple contrastive learning representation boosting for supervised multiclass tasks
Li, X., Liu, Z., & Liu, S. (2024) · Information Processing & Management · doi:10.1016/j.ipm.2024.104011 · Cited by 13
Ryu, S. Y., Choi, J. Y., & Yoo, T. K. (2025) · Medical & Biological Engineering & Computing · doi:10.1007/s11517-025-03353-7 · Cited by 13
“SimCLR’s contrastive learning framework enabled the model to learn robust, generalized representations from unlabeled RAO images by maximizing similarity between augmented views of the same image (positive pairs) and minimizing similarity with different images (negative pairs) [40].”
Underwater Distortion Target Recognition Network (UDTRNet) via Enhanced Image Features
Cai, L., Chen, C., & Chai, H. (2021) · Computational Intelligence and Neuroscience · doi:10.1155/2021/4193625 · Cited by 13
“Le-Khac et al. [4] summarized the existing literature on contrast learning and proposed a generalized framework for contrast representation learning. framework simplifies and unifies many different contrast learning algorithms and addresses the application of the contrast learning framework to the…”
BackgroundSemi-supervised rotation-invariant representation learning for wafer map pattern analysis
Kang, H., & Kang, S. (2023) · Engineering Applications of Artificial Intelligence · doi:10.1016/j.engappai.2023.105864 · Cited by 13
Liu, Y., Shi, J., Zhao, C., & Zhang, C. (2023) · Digital Health · doi:10.1177/20552076231188852 · Cited by 13
“CL learns the feature representation of the sample so that the feature representation is as close as possible.(48) Inspired by previous work,(49) CL is incorporated into the summarization model.”
MethodSun, P., Wang, Y., Zhang, M., Wu, C., Fang, Y., Zhu, H. C., Fang, Y., & Wang, M. (2024) · International World Wide Web Conference (WWW) · doi:10.1145/3589335.3648297 · Cited by 17
“Self-supervised learning has been introduced to enhance recommender model training [18, 20, 37, 49].”
MethodMotamed, S., Paudel, D. P., & Gool, L. V. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-72633-0_7 · Cited by 27
“Inspired by success of contrastive losses for representation learning [24,28,36,54], we employ an InfoNCE-based [36] loss to learn the embeddings CPT in a contrastive setting.”
MethodLi, H., Wang, J., Li, Z., Cecil, K. M., Altaye, M., Dillman, J. R., Parikh, N. A., & He, L. (2024) · NeuroImage · doi:10.1016/j.neuroimage.2024.120579 · Cited by 16
“Particularly, VPT infants with sub-scores less or equal to 85 are more likely to develop long-term neurodevelopmental (cognitive, language and motor) impairments.”
Optimizing Novelty of Top-k Recommendations using Large Language Models and Reinforcement Learning
Sharma, A., Li, H., Li, X., & Jiao, J. (2024) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/3637528.3671618 · Cited by 18
van Dijk, R., Arévalo, J., Babadi, M., Carpenter, A. E., & Singh, S. (2024) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1012547 · Cited by 14
“Contrastive learning is a method where data points corresponding to the same entity – for example the same sample – are brought closer together while others are pushed apart in a feature space [20].”
DiffNMR: diffusion models for nuclear magnetic resonance spectra elucidation
Yang, Q., Wu, B., Liu, X., Chen, B., Li, W., Long, G., Chen, X., & Xiao, M. (2026) · Materials Futures · doi:10.1088/2752-5724/ae301f · Cited by 19
“This alignment is achieved via contrastive learning [40–42], where the NMR encoder minimizes the distance between the latent representations of matching spectrum-molecule pairs while maximizing it for non-matching pairs.”
Tran, M., Wagner, S. J., Boxberg, M., & Peng, T. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-16434-7_10 · Cited by 15
So Different Yet So Alike! Constrained Unsupervised Text Style Transfer
Kashyap, A. R., Hazarika, D., Kan, M., Zimmermann, R., & Poria, S. (2022) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2022.acl-long.32 · Cited by 20
Volumetric supervised contrastive learning for seismic semantic segmentation
Kokilepersaud, K., Prabhushankar, M., & AlRegib, G. (2022) · Second International Meeting for Applied Geoscience & Energy · doi:10.1190/image2022-3751735.1 · Cited by 12
Spectrum Sensing Algorithm Based on Self-Supervised Contrast Learning
Li, X., Zhao, Z., Zhang, Y., Zheng, S., & Dai, S. (2023) · Electronics · doi:10.3390/electronics12061317 · Cited by 12
Wen, W., Liu, Y., Bai, Y., & Meng, Q. (2023) · Proceedings of the Institution of Mechanical Engineers Part F Journal of Rail and Rapid Transit · doi:10.1177/09544097231165093 · Cited by 12
“29 The higher-order features of the data can be effectively extracted by this method.”
MethodEnd-to-End Mispronunciation Detection and Diagnosis Using Transfer Learning
Peng, L., Gao, Y., Bao, R., Li, Y., & Zhang, J. (2023) · Applied Sciences · doi:10.3390/app13116793 · Cited by 12
Zhang, Y., Huang, X., & Zhang, Z. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.01267 · Cited by 17
PolSAR Image Classification Based on Multi-Modal Contrastive Fully Convolutional Network
Hua, W., Wang, Y., Yang, S., & Jin, X. (2024) · Remote Sensing · doi:10.3390/rs16020296 · Cited by 12
Multi-view representation learning for tabular data integration using inter-feature relationships
Tripathi, S., Fritz, B. A., Abdelhack, M., Avidan, M. S., Chen, Y., & King, C. R. (2024) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2024.104602 · Cited by 13
“In recent years, the dominating approach to co-embedding has been contrastive learning (CL), a framework where different views of the same information or different augmentations of the same datasets are used to learn latent representation [22].”
Semantic Pose Verification for Outdoor Visual Localization with Self-supervised Contrastive Learning
Orhan, S., Guerrero, J. J., & Baştanlar, Y. (2022) · IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw56347.2022.00444 · Cited by 15
Learning Contextual Tag Embeddings for Cross-Modal Alignment of Audio and Tags
Favory, X., Drossos, K., Virtanen, T. I., & Serra, X. (2021) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp39728.2021.9414638 · Cited by 12
Evaluation of Contrastive Learning with Various Code Representations for Code Clone DetectionPreprint
Zubkov, M., Spirin, E., Bogomolov, E., & Bryksin, T. (2022) · SSRN Electronic Journal · doi:10.2139/ssrn.4159812 · Cited by 14
Miao, Q., Wang, P., Zhao, H., Li, Z., Qi, Y., Mao, J., Li, M., & Tang, G. (2024) · Natural Resources Research · doi:10.1007/s11053-024-10335-3 · Cited by 11
“Self-supervised contrastive learning is an innovative approach in the field of machine learning that aims to learn data representations from raw data (Le-Khac et al., 2020).”
Context Is All You Need: A Hybrid Attention-Based Method for Detecting Code Design Patterns
Houichime, T., & Amrani, Y. E. (2025) · IEEE Access · doi:10.1109/access.2025.3525849 · Cited by 17
Wafer Defect Classification Algorithm With Label Embedding Using Contrastive Learning
Hwang, J., Ha, S., & Kim, D. (2025) · IEEE Access · doi:10.1109/access.2025.3527491 · Cited by 13
Swallow: A Transfer-Robust Website Fingerprinting Attack via Consistent Feature Learning
Shen, M., Wu, J., Ai, J., Li, Q. L., Ren, C., Xu, K., & Zhu, L. (2025) · ACM SIGSAC Conference on Computer and Communications Security (CCS) · doi:10.1145/3719027.3744795 · Cited by 17
“These attacks employ metric learning [32] or contrastive learning [11, 17] to fine-tune a well-trained classifier with a few traffic traces collected in a new network condition.”
AtmoDist: Self-supervised representation learning for atmospheric dynamics
Hoffmann, S., & Lessig, C. (2023) · Environmental Data Science · doi:10.1017/eds.2023.1 · Cited by 17
“Since our work employs a pretext task, we will not discuss these methods but refer to Le-Khac et al. (2020) for an overview.”
MethodGeneralizable Pancreas Segmentation via a Dual Self-Supervised Learning Framework
Li, J., Zhu, H., Chen, T., & Qian, X. (2023) · IEEE Journal of Biomedical and Health Informatics · doi:10.1109/jbhi.2023.3294278 · Cited by 12
Self-supervised ensembled learning for autism spectrum classification
Gaur, M., Chaturvedi, K., Vishwakarma, D. K., Ramasamy, S., & Prasad, M. (2023) · Research in Autism · doi:10.1016/j.rasd.2023.102223 · Cited by 13
Zhou, Y., Wang, Y., Yan, N., Feng, L., Chen, Y., Wu, T., Gao, J., Zhang, X., & Zhu, W. (2023) · Remote Sensing · doi:10.3390/rs15205009 · Cited by 11
HyperKon: A Self-Supervised Contrastive Network for Hyperspectral Image Analysis
Ayuba, D. L., Guillemaut, J., Martí-Cardona, B., & Méndez, O. (2024) · Remote Sensing · doi:10.3390/rs16183399 · Cited by 13
Hierarchical Multi-Task Learning for Diagram Question Answering with Multi-Modal Transformer
Yuan, Z., Xiao, P., Wu, X. Y., & Xu, C. (2021) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3474085.3475255 · Cited by 11
Quantifying Emotional Similarity in Speech
Harvill, J. B., Leem, S.-G., Abdelwahab, M., Lotfian, R., & Busso, C. (2021) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2021.3127390 · Cited by 13
“Learning a similarity space has been actively researched in the field of contrastive learning [30].”
BackgroundComputational protein design with data‐driven approaches: Recent developments and perspectives
Liu, H., & Chen, Q. (2022) · Wiley Interdisciplinary Reviews Computational Molecular Science · doi:10.1002/wcms.1646 · Cited by 14
SSSD: Self-Supervised Self Distillation
Chen, W.-C., & Chu, W.-T. (2023) · IEEE Workshop/Winter Conference on Applications of Computer Vision · doi:10.1109/wacv56688.2023.00279 · Cited by 11
“More literature surveys on self-supervised learning or contrastive learning can be found in [15] and [16].”
BackgroundPDSum: Prototype-driven Continuous Summarization of Evolving Multi-document Sets Stream
Yoon, S., Chan, H. P., & Han, J. (2023) · International World Wide Web Conference (WWW) · doi:10.1145/3543507.3583371 · Cited by 15
“Note that the regularized contrastive loss is uniquely designed for EMDS, different from typical contrastive losses [11, 19, 37].”
BackgroundDeep Unsupervised Representation Learning for Feature-Informed EEG Domain Extraction
Ng, H. W., & Guan, C. (2023) · IEEE Transactions on Neural Systems and Rehabilitation Engineering · doi:10.1109/tnsre.2023.3339179 · Cited by 12
“Thus far, fewer work has been done on representation learning on spatiotemporal signals [21] compared to more well established datasets such as images [13], [22], [23] or natural language [23], [24].”
BackgroundGraph-Induced Contrastive Learning for Intra-Camera Supervised Person Re-Identification
Wang, M., Lai, B., Huang, J., Gong, X., & Hua, X. (2021) · IEEE Access · doi:10.1109/access.2021.3055266 · Cited by 11
“Contrastive learning has been successfully applied to unsupervised representation learning [19]–[21], [32], [33] in recent years.”
Semi-Supervised Learning via Weight-aware Distillation under Class Distribution Mismatch
Du, P., Zhao, S., Sheng, Z., Li, C., & Chen, H. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.01504 · Cited by 10
“The teacher model could produce closely aligned representations for instances from the same categories and maximize the mutual information among them [2, 18, 20, 25].”
BackgroundAdaptive control for circulating cooling water system using deep reinforcement learning
Xu, J., Li, H., & Zhang, Q. (2024) · PLoS ONE · doi:10.1371/journal.pone.0307767 · Cited by 10
“The reference step input signals for the flow (m^3/h) and temperature (˚C) are set to [550, 650] and [20, 30], respectively, to achieve robustness to significant setpoint changes in the system.”
BackgroundModalChorus: Visual Probing and Alignment of Multi-Modal Embeddings via Modal Fusion Map
Ye, Y., Xiao, S., Zeng, X., & Zeng, W. (2024) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2024.3456387 · Cited by 10
“Instead, in the field of AI, representation learning of single and multi-modal embeddings has been playing a pivotal role for various downstream tasks [41,55].”
BackgroundRetrieval-Augmented Dialogue Knowledge Aggregation for expressive conversational speech synthesis
Liu, R., Jia, Z., Bao, F., & Li, H. (2025) · Information Fusion · doi:10.1016/j.inffus.2025.102948 · Cited by 20
Hierarchical multi-granular multi-label contrastive learning
Li, H., Fang, M., Li, X., Chen, B., & Wang, G. (2025) · Pattern Recognition · doi:10.1016/j.patcog.2025.111567 · Cited by 10
Li, R., Li, Z., Kofi, B. A., Sun, J., He, Y., & Jiao, M. (2026) · Chemosensors · doi:10.3390/chemosensors14060143 · Cited by 10
Analysis of Vibrations and Currents for Broken Rotor Bar Detection in Three-phase Induction Motors
Taghiyarrenani, Z., & Berenji, A. (2022) · PHM Society European Conference · doi:10.36001/phme.2022.v7i1.3332 · Cited by 10
Gong, J., Chen, Z., Ma, C., Xiao, Z., Wang, H., Tang, G., Liu, L., Xu, S., Long, B., & Jiang, Y. (2023) · IEEE International Conference on Data Engineering (ICDE) · doi:10.1109/icde55515.2023.00247 · Cited by 14
“Recently, contrastive learning methods [35] have achieved remarkable successes on learning useful representations in a self-supervised manner.”
BackgroundForecasting the future clinical events of a patient through contrastive learning
Zhang, Z., Yan, C., Zhang, X., Nyemba, S., & Malin, B. (2022) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocac086 · Cited by 14
Consistency-Contrast Learning for Conceptual Coding
Chang, J., Zhang, J., Xu, Y., Li, J., Ma, S., & Gao, W. (2022) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3503161.3547928 · Cited by 11
“In this paper, we propose Consistency-Contrast Learning (CCL), which adapts the philosophy of contrastive learning and introduces consistency-contrast regularization, aiming at aligning the structure of texture representation space and source image space.”
BackgroundInfluentialFaithful and Consistent Graph Neural Network Explanations with Rationale Alignment
Zhao, T., Luo, D., Zhang, X. D., & Wang, S. (2023) · ACM Transactions on Intelligent Systems and Technology · doi:10.1145/3616542 · Cited by 13
A multi-view mask contrastive learning graph convolutional neural network for age estimation
Zhang, Y., Shou, Y., Meng, T., Ai, W., & Li, K. (2024) · Knowledge and Information Systems · doi:10.1007/s10115-024-02193-5 · Cited by 16
“In this section, the self-supervised methods are first introduced, including masked image autoen-coders and contrastive learning.”
Do Generative Models Know Disentanglement? Contrastive Learning is All You NeedPreprint
Ren, X., Yang, T., Wang, Y., & Zeng, W. (2021) · arXiv (Cornell University) · Cited by 11
SINCERE: Supervised Information Noise-Contrastive Estimation REvisitedPreprint
Feeney, P. J., & Hughes, M. C. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2309.14277 · Cited by 13
“…in embedding space z ′ t (Wu et al. 2018; Chen et al. 2020a; Chen, Xie, and He 2021). z ′ t can be produced by embedding a data augmented copy of x t (Le-Khac, Healy, and Smeaton 2020; Chen et al. 2020a), embedding x t via older (Wu et al. 2018) or averaged embedding function parameters (Chen,…”
BackgroundGANs Conditioning Methods: A SurveyPreprint
Bourou, A., Mezger, V., & Genovesio, A. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2408.15640 · Cited by 12
Learning beyond sensations: How dreams organize neuronal representations
Deperrois, N., Petrovici, M. A., Senn, W., & Jordan, J. (2024) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2023.105508 · Cited by 9
Kazi, S., Khoja, S., & Daud, A. (2025) · Computer Science Review · doi:10.1016/j.cosrev.2025.100756 · Cited by 9
Stimulate the Potential of Robots via Competition
Huang, K., Guo, D., Zhang, X., Ji, X., & Liu, H. (2024) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra57147.2024.10611581 · Cited by 9
“Our idea shares fundamental similarities with the approach of adversarial learning and contrastive learning [34], [35].”
BackgroundGe, Y., Zhang, H., Shi, J., Luo, D., Chang, S., He, J. W., Huang, Q., & Wang, H. (2025) · Knowledge-Based Systems · doi:10.1016/j.knosys.2025.113090 · Cited by 11
Chen, C., Li, F., Chen, H., & Lin, Y. (2025) · Knowledge-Based Systems · doi:10.1016/j.knosys.2025.113215 · Cited by 9
“In the domain of graph learning, the effectiveness of contrastive learning is widely recognized [21].”
BackgroundFerreira, J., Darabi, R., Sousa, A. J., Brueckner, F. P., Reis, L. P., Reis, A. R., Tavares, J. M. R. S., & Sousa, J. (2025) · Journal of Intelligent Manufacturing · doi:10.1007/s10845-025-02614-4 · Cited by 14
AimTS: Augmented Series and Image Contrastive Learning for Time Series Classification
Chen, Y., Huang, S., Cheng, Y., Chen, P., Rao, Z., Shu, Y., Yang, B., Pan, L., & Guo, C. (2025) · IEEE International Conference on Data Engineering (ICDE) · doi:10.1109/icde65448.2025.00149 · Cited by 20
Disentangled Representation Learning for Robust Brainprint Recognition
Zheng, C., Zhu, Q., Fei, L., Li, S., Zhai, X., Zhang, D. D., & Zhang, D. (2025) · IEEE Transactions on Information Forensics and Security · doi:10.1109/tifs.2025.3602266 · Cited by 12
“Contrastive learning (CL) is a popular representation learning method, which aims at making similar instances closer and dissimilar instances far from each other in the latent space [30], [31].”
Yang, Y., Teo, H. H., King, Y. J., Zhang, Y., Wang, G., & Qu, X. (2025) · Scientific Reports · doi:10.1038/s41598-025-28302-0 · Cited by 11
“The pretraining loss is the mean squared error (MSE) between the reconstructed and original spectrograms 18,20 .”
Connecting the Dots in Self-Supervised Learning: A Brief Survey for Beginners
Fang, P., Li, X., Yang, Y., Zhang, S., Kang, Q.-Y., Li, X., & Lan, Z. (2022) · Journal of Computer Science and Technology · doi:10.1007/s11390-022-2158-x · Cited by 11
“A uni(cid:12)ed framework using contrastive learning as the objective for representations is surveyed in [104].”
BackgroundMethodTowards Using Generative AI for Facilitating Image Creation in Spatial Augmented Reality
Hu, Y. '., Hu, W. S., & Quigley, A. J. (2023) · IEEE International Symposium on Mixed and Augmented Reality Adjunct (ISMAR-Adjunct) · doi:10.1109/ismar-adjunct60411.2023.00095 · Cited by 12
“Addressing these challenges, we introduce contrastive learning [13].”
BackgroundInter-structure and intra-semantics graph contrastive learning for disease prediction
Kang, Y., Zheng, J., Yang, M., & An, N. (2024) · Knowledge-Based Systems · doi:10.1016/j.knosys.2024.112059 · Cited by 12
Fine-grained adaptive contrastive learning for unsupervised feature extraction
Yin, T., Wang, J., Zhao, Y., Wang, S., Ma, Y., & Liu, M. (2024) · Neurocomputing · doi:10.1016/j.neucom.2024.129014 · Cited by 9
Zhang, H., Li, X., Ramelli, F., David, R. O., Pasquier, J. T., & Henneberger, J. (2024) · Atmospheric measurement techniques · doi:10.5194/amt-17-7109-2024 · Cited by 14
S2SNet: A Pretrained Neural Network for Superconductivity Discovery
Liu, P., Yang, K., Zhang, J., & Xu, R. (2022) · International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2022/708 · Cited by 9
“Contrastive learning is commonly used for pre-training in CV [Le-Khac et al., 2020].”
MethodNetwork Comparison with Interpretable Contrastive Network Representation Learning
Fujiwara, T., Zhao, J. F., Chen, F., Yu, Y., & Ma, K. (2022) · Journal of Data Science Statistics and Visualisation · doi:10.52933/jdssv.v2i5.56 · Cited by 14
“One aims to learn an embedding from similar and dissimilar pairs of input samples so that similar samples are placed close together in the embedding space, and vice versa (Le-Khac et al. 2020; Jaiswal et al. 2021).”
BackgroundDual Contrastive Learning for Efficient Static Feature Representation in Sequential Recommendations
Li, P., Que, M., & Tuzhilin, A. (2023) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2023.3289469 · Cited by 14
“Contrastive learning mechanism [39] is a powerful method thateffectivelyseparatespositivesamplesfromnegativesamplesthroughtheself-supervisedlearningprocess[40].”
MethodDeep contrastive learning for predicting cancer prognosis using gene expression values
Sun, A., Franzmann, E. J., Chen, Z., & Cai, X. (2024) · Briefings in Bioinformatics · doi:10.1093/bib/bbae544 · Cited by 12
Vural, O., Hamdi, S. M., & Boubrahimi, S. F. (2024) · International Conference on Machine Learning and Applications · doi:10.1109/icmla61862.2024.00163 · Cited by 9
“…in discriminative models or reconstructing input samples as in generative models, data representations are learned through comparison between data points in the input space by mapping similar data points close while increasing the distance between dissimilar data points in the embedding space [4].”
Metadata
- Authors
- Le-Khac, P. H., Healy, G., & Smeaton, A. F.
- Affiliations
- Dublin City University (IE)
- Venue
- IEEE Access
- Published
- 2020
- Volume
- 8
- Pages
- 193907-193934
- ISSN
- 2169-3536 (ISSN-L)
- OpenAlex
- W3096655658
- Cited by
- 862
- References
- 144
- Citations by year
- 2026: 111 · 2025: 197 · 2024: 212 · 2023: 157 · 2022: 113 · 2021: 71 · 2020: 1
- FWCI
- 33.61
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Locations
- IEEE Access (publishedVersion, cc-by)
- arXiv (Cornell University) (acceptedVersion)
- Dublin City University Open Access Institutional Repository (Dublin City University) (submittedVersion, cc-by)
- Arrow@dit (Dublin Institute of Technology) (submittedVersion, cc-by)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- arXiv (Cornell University) (cc-by)
- Funders
- Science Foundation Ireland through the Science Foundation Ireland (SFI) Centre for Research Training in Machine Learning (18/CRT/6183)
- Insight Centre for Data Analytics (SFI/12/RC/2289_P2)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Domain Adaptation and Few-Shot Learning
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Physical Sciences › Computer Science › Artificial Intelligence › Machine Learning and Algorithms
- Keywords
- contrastive learning, representation learning, self-supervised learning, metric learning, inductive bias, computer vision, natural language processing, audio signal processing, reinforcement learning
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.