Contrastive Representation Learning: A Framework and Review
Open accessThe DOI matches its Crossref record; 147 of the 162 references it cites were found in a registry or an index, and 15 are not checked yet.
Abstract
Contrastive Learning has recently received interest due to its success in self-supervised representation learning in the computer vision domain. However, the origins of Contrastive Learning date as far back as the 1990s and its development has spanned across many fields and domains including Metric Learning and natural language processing. In this paper, we provide a comprehensive literature review and we propose a general Contrastive Representation Learning framework that simplifies and unifies many different contrastive learning methods. We also provide a taxonomy for each of the components of contrastive learning in order to summarise it and distinguish it from other forms of machine learning. We then discuss the inductive biases which are present in any contrastive learning system and we analyse our framework under different views from various sub-fields of Machine Learning. Examples of how contrastive learning has been applied in computer vision, natural language processing, audio processing, and others, as well as in Reinforcement Learning are also presented. Finally, we discuss the challenges and some of the most promising future research directions ahead.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“According to the suggestions of Le-Khac et al. [29], the main methods can be divided into the following categories: multisensory signals, data augmentation, localglobal consistency, and temporal consistency.”
From Self-Supervised Learning for Time Series Analysis: Taxonomy, Progress, and Prospects (2024) · cites this paper as [29]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Contrastive learning [56-58] is in fact a training procedure used in recent artificial intelligence models as a way to efficiently learn semantic representations.”
From How Adversarial REM Dreams May Facilitate Creativity, and Why We Become Aware of Them (2024) · cites this paper as [56]
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“As summarized in [29], commonly used contrastive loss functions generally include scoring functions (cosine similarity), energy-based margin functions (pair loss and triplet loss), probabilistic NCE-based functions, and mutual information based functions.”
From Self-Supervised Learning for Time Series Analysis: Taxonomy, Progress, and Prospects (2024) · cites this paper as [29]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474 · cited 2×
“In SimCLRv2 [17], empirical experiments show that the output of the second layer of a 3-layer MLP projection head is a better representation for transfer learning than the output of the base ResNet [42] encoder.”
BackgroundMethodVaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., & Polosukhin, I. (2017) · arXiv · doi:10.48550/arxiv.1706.03762 · Cited by 120,404 · cited 3×
“Recently, the Bidirectional Encoder Representation from Transformer (BERT) [24] model learns bidirectional word representations using the Transformer architecture’s decoder [103] and demonstrated great performance for transfer learning in multiple downstream tasks.”
MethodHochreiter, S., & Schmidhuber, J. (1997) · Neural Computation · doi:10.1162/neco.1997.9.8.1735 · Cited by 102,318 · cited 1×
“Even though the BERT [24] text encoder and the Faster RCNN [87] object detection model are trained separately and not updated by the contrastive loss, the LSTM cells [48] and a linear project head can still map between words and object representations.”
MethodExploiting Generative AI to Scale up Intelligent Tutoring Systems
Jakubův, J., Chvalovský, K., Goertzel, Z., Kaliszyk, C., Olšák, M., Piotrowski, B., Schulz, S., Suda, M., & Urban, J. (2023) · DROPS (Schloss Dagstuhl – Leibniz Center for Informatics) · doi:10.4230/lipics.itp.2023.19 · Cited by 76,658
Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks
Ren, S., He, K., Girshick, R., & Sun, J. (2017) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2016.2577031 · Cited by 73,472
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2018) · arXiv · doi:10.48550/arxiv.1810.04805 · Cited by 73,173 · cited 5×
“The discretised speech representation can then be used directly as a representation for other models that expect discrete input such as BERT [24].”
MethodInfluentialFaster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks
Ren, S., He, K., Girshick, R., & Sun, J. (2015) · arXiv · doi:10.48550/arxiv.1506.01497 · Cited by 70,812
ImageNet: A large-scale hierarchical image database
Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., & Fei-Fei, L. (2009) · 2009 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2009.5206848 · Cited by 65,333
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2019) · Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) · doi:10.18653/v1/n19-1423 · Cited by 54,325 · cited 5×
“The discretised speech representation can then be used directly as a representation for other models that expect discrete input such as BERT [24].”
MethodInfluentialEfficient Estimation of Word Representations in Vector Space
Mikolov, T., Chen, K., Corrado, G., & Dean, J. (2013) · arXiv · doi:10.48550/arxiv.1301.3781 · Cited by 26,568
Research works printed but not linked: 1
- stratos, 2019, Noise contrastive estimation
Cited by
Items 301–400 of 948
Self-Supervised Learning from Automatically Separated Sound Scenes
Fonseca, E., Jansen, A., Ellis, D. P. W., Wisdom, S., Tagliasacchi, M., Hershey, J. R., Plakal, M., Hershey, S., Moore, R. C., & Serra, X. (2021) · IEEE Workshop on Applications of Signal Processing to Audio and Acoustics (WASPAA) · doi:10.1109/waspaa52581.2021.9632739 · Cited by 18
“For general-purpose audio, including a variety of environmental sounds beyond speech, the majority of works are based on contrastive learning [14, 15, 16, 17, 18, 19], where a representation is learned by comparing pairs of examples selected by some semantically-correlated notion of similarity [20].”
BackgroundMethodInfluentialKang, G., Ren, H., Chen, W., Liu, J., Cao, B., & Xu, Y. (2024) · IEEE Transactions on Network and Service Management · doi:10.1109/tnsm.2024.3515103 · Cited by 8
“Contrastive learning encourages features from different views to come closer in the embedding space, thereby achieving self-distinguishing feature representation in an unsupervised manner and supplementing supervised tasks with unlabeled data [35].”
Rao, Y., Cao, M., & Ye, M. (2025) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2025.3568519 · Cited by 9
“Multimodal contrastive learning [46], [47] is particularly effective at align feature prepresentations.”
Interpreting human sleep activity through neural contrastive learning
Chen, Z., Zheng, H., Zhou, J., Zheng, L., Lin, P., Wang, H., Busche, M. A., Behrens, T. E. J., Dolan, R. J., & Liu, Y. (2026) · Neuron · doi:10.1016/j.neuron.2026.03.028 · Cited by 8
Self-organizing maps on “what-where” codes towards fully unsupervised classification
Sa-Couto, L., & Wichert, A. (2023) · Biological Cybernetics · doi:10.1007/s00422-023-00963-y · Cited by 8
From Pre-Training to Meta-Learning: A Journey in Low-Resource-Language Representation Learning
Zaikis, D., & Vlahavas, I. P. (2023) · IEEE Access · doi:10.1109/access.2023.3326337 · Cited by 8
“In recent years, the advent of deep learning and neural network architectures [1] has revolutionized the field, allowing for the development of more powerful and adaptive language representation models [2].”
BackgroundA Rewiring Contrastive Patch PerformerMixer Framework for Graph Representation Learning
Sun, Z., Harit, A., Cristea, A. I., Wang, J., & Lió, P. (2023) · IEEE International Conference on Big Data (BigData) · doi:10.1109/bigdata59044.2023.10386951 · Cited by 13
“Contrastive learning is becoming prominent for pre-trained methods, which aims to learn embeddings of objects by comparing different samples instead of learning the mapping function between input and labels [39]–[41].”
BackgroundUnsupervised Point Cloud Co-Part Segmentation via Co-Attended Superpoint Generation and Aggregation
Umam, A., Yang, C.-K., Chuang, J., & Lin, Y. (2024) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2024.3371294 · Cited by 8
“The attention mechanism can be viewed as a technique that dynamically weights the input data based on their importance for feature extraction [60], while contrastive learning can be regarded as learning by comparing similar and dissimilar data [61].”
BackgroundXu, R., Samat, A., Zhu, E., Li, E., & Li, W. (2024) · Remote Sensing · doi:10.3390/rs16111974 · Cited by 10
Class Hierarchy Aware Contrastive Feature Learning for Multigranularity SAR Target Recognition
Wen, Z., Wang, Z., Zhang, J., Lv, Y., & Wu, Q. (2024) · IEEE Transactions on Aerospace and Electronic Systems · doi:10.1109/taes.2024.3421171 · Cited by 8
Deep Loss Convexification for Learning Iterative Models
Zhang, Z., Shao, Y., Zhang, Y., Lin, F., Zhang, H. K., & Rundensteiner, E. A. (2024) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2024.3509860 · Cited by 16
“A good survey can be found in [48].”
Li, S., Zhou, G., Chen, J., Sun, Y., Huang, N., & Peng, S. (2024) · Neurocomputing · doi:10.1016/j.neucom.2024.129232 · Cited by 12
Contrastive Learning for Network Intrusion Detection: A Comprehensive Survey
Tan, X., Cheng, J., Li, H., & Yang, Y. (2024) · International Conference on Computer, Internet of Things and Smart City (CIoTSC) · doi:10.1145/3731867.3731894 · Cited by 10
“Le-Khac et al. [24] comprehensively reviewed contrastive representation learning and put forward a unified framework, which consolidated and simplified various contrastive learning methods.”
Knowledge Representation Learning with Contrastive Completion Coding
Ouyang, B., Huang, W., Chen, R., Tan, Z., Liu, Y., Sun, M., & Zhu, J. (2021) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2021.findings-emnlp.263 · Cited by 10
“Contrastive losses measure the distance, or similarity, between representations in the latent space, which is one of the key differences between contrastive learning methods and other representation learning approaches (Le-Khac et al., 2020).”
BackgroundTowards Better Detection of Biased Language with Scarce, Noisy, and Biased Annotations
Li, Z., Lu, Z., & Yin, M. (2022) · Proceedings of the AAAI/ACM Conference on AI Ethics and Society · doi:10.1145/3514094.3534142 · Cited by 14
“Self-supervised contrastive learning has been widely applied in computer vision [4, 23, 34, 35, 50], where data augmentation is utilized to allow the model to learn compact and discriminative features.”
MethodReinforced Contrastive Graph Neural Networks (RCGNN) for Anomaly Detection
Sun, Z., Su, J., Jeon, D., Velasquez, A., Song, H., & Niu, S. (2022) · IEEE International Performance, Computing, and Communications Conference · doi:10.1109/ipccc55026.2022.9894334 · Cited by 8
“The contrastive learning framework is often used in unsupervised and self-2022 supervised learning [17].”
MethodPhunruangsakao, C., Achanccaray, D., Izumi, S., & Hayashibe, M. (2022) · Frontiers in Human Neuroscience · doi:10.3389/fnhum.2022.1032724 · Cited by 8
“The core idea of contrastive representation learning is to project feature representations such that samples from the same class are grouped together, whereas samples from different classes are projected further apart (Khac et al., 2020).”
BackgroundContrastive Learning of Features between Images and LiDAR
Jiang, P., & Saripalli, S. (2022) · IEEE International Conference on Automation Science and Engineering (CASE) · doi:10.1109/case49997.2022.9926589 · Cited by 12
“Several contrastive learning loss functions have proposed [9], [10].”
Two-phase self-supervised pretraining for object re-identification
Ma, H., Li, X., Yuan, X., & Zhao, C. (2022) · Knowledge-Based Systems · doi:10.1016/j.knosys.2022.110220 · Cited by 10
A Generic Self-Supervised Framework of Learning Invariant Discriminative Features
Ntelemis, F., Jin, Y., & Thomas, S. A. (2023) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2023.3265607 · Cited by 10
“Invariant representations mean that the learned representations (e.g., features) are insensitive to some transformations in the input, for instance, rotation, shift, and translation of the object to be recognized in image data [21].”
BackgroundLiu, X., Li, J., Mu, J., Yang, M., Xu, R., & Wang, B. (2023) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v37i11.26560 · Cited by 14
Embeddings Jurídico: Representações Orientadas à Linguagem Jurídica Brasileira
do Carmo, F. A., Serejo, F., Jacob, A. F. L., Santana, E. É. C., & Lobato, F. M. F. (2023) · Workshop de Computação Aplicada em Governo Eletrônico (WCGE) · doi:10.5753/wcge.2023.230786 · Cited by 13
MERLIN-Seg: Self-supervised despeckling for label-efficient semantic segmentation
Dalsasso, E., Rambour, C., Trouvé, N., & Thome, N. (2024) · Computer Vision and Image Understanding · doi:10.1016/j.cviu.2024.103940 · Cited by 9
Semi-Supervised End-To-End Contrastive Learning For Time Series Classification
Cai, H., & Zhang, X. (2026) · Neural Processing Letters · doi:10.1007/s11063-026-11869-8 · Cited by 9
A Generative Approach to Person Reidentification
Asperti, A., Fiorilla, S., & Orsini, L. (2024) · Sensors · doi:10.3390/s24041240 · Cited by 9
Online continual learning through unsupervised mutual information maximization
Hihn, H., & Braun, D. A. (2024) · Neurocomputing · doi:10.1016/j.neucom.2024.127422 · Cited by 16
Fine-Grained Engine Fault Sound Event Detection Using Multimodal Signals
Fedorishin, D., Forte, L., Schneider, P., Setlur, S., & Govindaraju, V. (2024) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp48485.2024.10448485 · Cited by 8
“In literature, contrastive losses have been shown to create strong and discriminative embedding spaces for a wide variety of tasks [23].”
BackgroundChowdhury, S., Hamerly, G., & McGarrity, M. E. (2024) · IEEE/CVF Winter Conference on Applications of Computer Vision Workshops (WACVW) · doi:10.1109/wacvw60836.2024.00097 · Cited by 12
“Due to the representation learning capability, contrastive learning is especially effective at learning from limited labeled data [47, 72, 76] and has been effective in different downstream tasks in Computer Vision [15,40,42,64].”
BackgroundActNetFormer: Transformer-ResNet Hybrid Method for Semi-supervised Action Recognition in Videos
Dass, S. D. S., Barua, H. B., Krishnasamy, G., Paramesran, R., & Phan, R. C. ‐. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-78354-8_22 · Cited by 13
Contrastive learning and subtyping of post-COVID-19 lung computed tomography images
Li, F., Zhang, X., Comellas, A. P., Hoffman, E. A., Yang, T., & Lin, C. (2022) · Frontiers in Physiology · doi:10.3389/fphys.2022.999263 · Cited by 8
“That is, by contrasting between images of positive and negative pairs, representations of positive pairs will be attracted together while representations of negative pairs will be repelled far apart (Le-Khac et al., 2020).”
BackgroundAssessing Consensus of Developers' Views on Code ReadabilityPreprint
Sergeyuk, A., Lvova, O., Titov, S., Serova, A., Bagirov, F., & Bryksin, T. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2407.03790 · Cited by 11
Enhanced side information fusion framework for sequential recommendation
Su, Z.-A., Zhang, J., Fang, Z., & Gao, Y. (2024) · International Journal of Machine Learning and Cybernetics · doi:10.1007/s13042-024-02328-8 · Cited by 10
“InfoNCE loss is a popular contrastive loss function recently used in self-supervised learning L fj [17].”
MGKGR: Multimodal Semantic Fusion for Geographic Knowledge Graph Representation
Zhang, J., Chen, R., Li, S., Li, T., & Yao, H. L. (2024) · Algorithms · doi:10.3390/a17120593 · Cited by 7
Navigating Through Whole Slide Images With Hierarchy, Multi-Object, and Multi-Scale Data
Tran, M., Wagner, S., Weichert, W., Matek, C., Boxberg, M., & Peng, T. (2025) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2025.3532728 · Cited by 12
“Self-supervised and semi-supervised learning can deal with such scenarios and have shown promising results in both computer vision [8] and computational pathology [9].”
Identifying Illicit Activities in Blockchain Transaction Graph Networks
Adam, T., & Babič, F. (2025) · Electronics · doi:10.3390/electronics14234599 · Cited by 7
Liu, T., Tang, K., Zang, Y., Li, A., Chen, X., Yang, P., Xia, L., Zhang, Y., Qiu, B., & Chen, Y. (2026) · ISPRS Journal of Photogrammetry and Remote Sensing · doi:10.1016/j.isprsjprs.2026.02.039 · Cited by 7
Auh, J., Cho, C.-S., & Kim, S. (2024) · ETRI Journal · doi:10.4218/etrij.2023-0285 · Cited by 7
Hierarchical Knowledge Stripping for Multimodal Sentiment Analysis
Xiong, A., Zeng, Y., & Hu, H. (2024) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2024.3456117 · Cited by 11
“In recent years, contrastive learning has garnered significant attention due to its remarkable success in self-supervised representation learning within the field of computer vision [52].”
PCF-Lift: Panoptic Lifting by Probabilistic Contrastive Fusion
Zhu, R., Qiu, S., Wu, Q., Hui, K.-H., Heng, P., & Fu, C. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-72627-9_6 · Cited by 21
“Specifically, the contrastive loss used in [7,27,34,45] tends to increase the predicted feature similarities of predefined positive pairs, while decreasing the similarities of negative pairs for training.”
Le, T.-D., Macabiau, C., Albert, K., Jouvet, P. A., & Noumeir, R. (2024) · IEEE Access · doi:10.1109/access.2024.3488595 · Cited by 8
“Contrastive learning is an SSL technique that teaches a model to distinguish between similar and dissimilar data points [30].”
Towards Robust Event-Based Networks for Nighttime via Unpaired Day-to-Night Event Translation
Jeong, Y., Cho, H., & Yoon, K. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-72855-6_17 · Cited by 13
A temporal graph-based contrastive approach for financial time series forecasting
Barazandeh, I., Haratizadeh, S., & Sermpinis, G. (2025) · Engineering Applications of Artificial Intelligence · doi:10.1016/j.engappai.2025.110834 · Cited by 8
SynCSE: syntax graph-based contrastive learning of sentence embeddings
Kim, Y., Oh, D., & Huang, H. H. (2024) · Expert Systems with Applications · doi:10.1016/j.eswa.2025.128047 · Cited by 9
Madokoro, H., & Nix, S. (2025) · Sensors · doi:10.3390/s25134053 · Cited by 7
“The features of CLIP [73] include not only multimodal learning [82] but also contrastive learning [83] and zero-shot learning [84].”
Distribution-aware contrastive learning for domain adaptation in 3D LiDAR segmentation
Mendili, L. E., Daniel, S., & Badard, T. (2025) · Computer Vision and Image Understanding · doi:10.1016/j.cviu.2025.104438 · Cited by 8
Yan, Q., Li, X., Cui, J., Rong, J., Zhang, J., Gao, P., Xu, Y., Qiu, F., & Zuo, C. (2025) · Briefings in Bioinformatics · doi:10.1093/bib/bbaf317 · Cited by 7
Xu, P. J., Kan, S. X., Jin, J., Zhang, Z. J., Gu, Y. X., Zhang, B., & Zhou, Y. L. (2025) · Neurocomputing · doi:10.1016/j.neucom.2025.131817 · Cited by 9
Label Refinement via Contrastive Learning for Distantly-Supervised Named Entity Recognition
Ying, H., Luo, S., Dang, T., & Sheng, Y. (2022) · Findings of the Association for Computational Linguistics: NAACL · doi:10.18653/v1/2022.findings-naacl.203 · Cited by 7
Pose Uncertainty Aware Movement Synchrony Estimation via Spatial-Temporal Graph Transformer
Li, J., Bhat, A. N., & Barmaki, R. (2022) · INTERNATIONAL CONFERENCE ON MULTIMODAL INTERACTION · doi:10.1145/3536221.3556627 · Cited by 8
“We further applied contrastive learning in pretraining, a machine learning technique to learn by comparing between different samples instead of learning from each data sample individually [34].”
MethodExploiting the Distortion-Semantic Interaction in Fisheye Data
Kokilepersaud, K., Prabhushankar, M., Yarici, Y., AlRegib, G., & Parchami, A. (2023) · IEEE Open Journal of Signal Processing · doi:10.1109/ojsp.2023.3272281 · Cited by 10
“We then take advantage of contrastive learning approaches [17] in order to explicitly enforce a model to learn a representation that reflects both the distortion and semantic characteristic of the object through a weighted contrastive loss αLC + (1 − α)LDC .”
MethodDing, H., Wang, S., Xie, Z., Li, M., & Ma, L. (2023) · Computers & Graphics · doi:10.1016/j.cag.2023.07.025 · Cited by 9
Huang, W., Zhu, G., Huang, Q., Chen, Z., Chen, J., & Li, J. (2023) · International Conference on Advanced Robotics and Mechatronics (ICARM) · doi:10.1109/icarm58088.2023.10218404 · Cited by 7
Liu, Z., Zhang, F., Cheng, L., Deng, H., Yang, X., Zhang, Z., & Zhou, C.-C. (2026) · Computers and Electronics in Agriculture · doi:10.1016/j.compag.2026.112387 · Cited by 10
Self-Supervised Contrastive Learning In Spiking Neural Networks
Bahariasl, Y., & Kheradpisheh, S. R. (2024) · Iranian Conference on Machine Vision and Image Processing · doi:10.1109/mvip62238.2024.10491173 · Cited by 7
“One stand-out SSL model is SimCLR (Simple Contrastive Learning) [11], the main architecture of SimCLR is shown in Fig 1.”
MethodFCSSL: fusion enhanced contrastive self-supervised learning method for parallel MRI reconstruction
Ding, P., Duan, J., Xue, L., & Liu, Y. (2024) · Physics in Medicine and Biology · doi:10.1088/1361-6560/ad6d28 · Cited by 8
Contrastive Learning-Based Attribute Extraction Method for Enhanced Terrain Classification
Liu, X., Chen, H., & Chen, H. (2024) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra57147.2024.10611271 · Cited by 7
“In this paper, we propose a terrain attribute extraction method based on contrastive learning, which has proven effective in mining the intrinsic attributes of data [23].”
MethodSpectral-Aware Augmentation for Enhanced Graph Representation Learning
Yang, K., Han, H., Jin, W., & Liu, H. (2024) · ACM International Conference on Information and Knowledge Management (CIKM) · doi:10.1145/3627673.3679762 · Cited by 9
Semantics-Aware Image Aesthetics Assessment using Tag Matching and Contrastive Ranking
Yang, Z., Li, L., Chen, P., Wu, J., & Dong, W. (2024) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3664647.3680972 · Cited by 7
“Early IAA models utilized hand-crafted features and simple machine learning to classify images into high and low aesthetic categories, based on photography rules.”
Audio–visual self-supervised representation learning: A survey
Alsuwat, M. S., Al-Shareef, S., & Alghamdi, M. (2025) · Neurocomputing · doi:10.1016/j.neucom.2025.129750 · Cited by 10
Automatic water-level class estimation from repeated crowd-based photos of streams
Wang, Z., Seibert, J., van Meerveld, I., Lyu, H., & Zhang, C. (2023) · Hydrological Sciences Journal · doi:10.1080/02626667.2023.2240312 · Cited by 7
“Unlike the traditional method of extracting water-level information from photos based on reference object segmentation or water segmentation, the applied method enables the extracted feature of photo pairs to be implicitly measured based on a comparative representation learning framework (Le-Khac et al. 2020).”
MethodReducing Training Sample Memorization in GANs by Training with Memorization RejectionPreprint
Bai, A., Hsieh, C., Kan, W., & Lin, H.-T. (2022) · arXiv (Cornell University) · doi:10.48550/arxiv.2210.12231 · Cited by 7
Contrastive Meta-Learning for Partially Observable Few-Shot LearningPreprint
Jelley, A., Storkey, A., Antoniou, A., & Devlin, S. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2301.13136 · Cited by 9
“In computer vision and natural language applications these views typically consist of different augmentations of data items, which are carefully crafted to preserve semantic features, and thereby act as an inductive bias to encourage the contrastive learner to retain these consistent features (Le-Khac et al., 2020).”
BackgroundTransformer-based anomaly detection in P-LEO constellations: A dynamic graph approach
Indaco, M., & Guzzetti, D. (2024) · Acta Astronautica · doi:10.1016/j.actaastro.2024.02.019 · Cited by 6
FedSC: Federated Learning with Semantic-Aware Collaboration
Wang, H., Li, H., Chen, H., Yan, J. H., Shi, J., & Shen, J. (2025) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/3711896.3736957 · Cited by 11
Explainable Contrastive Learning for KL Grading Classification in Knee Osteoarthritis
Gao, F., Littlefield, N. G., Myers, N. T., Yates, A. J., Weiss, K. R., Plate, J. F., Tafti, A. P., & Amirian, S. (2025) · Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC) · doi:10.1109/embc58623.2025.11252884 · Cited by 6
Context-Based URL Classification for Open Access Datasets and Software in Scholarly Documents
Salsabil, L., Obadage, R. R., Banerjee, B., Abeysinghe, Y., Alam, S., Färber, M., Ingram, W. A., Fox, E. A., & Wu, J. (2025) · ACM/IEEE Joint Conference on Digital Libraries · doi:10.1109/jcdl67857.2025.00031 · Cited by 6
“After confirming a match, we classified the URL as an “Author Provided Dataset.””
Beyond Shannon: Semantic Information Theory and Methodology
Zhang, P., Niu, K., Liang, Z., Wang, C., Wu, J., Liu, Y., Xu, W., Ma, N., Xu, X., & Zhang, R. (2026) · IEEE Transactions on Network Science and Engineering · doi:10.1109/tnse.2026.3676901 · Cited by 13
“In particular, deep autoencoders and related representation-learning architectures [80], [81], [82], [83] learn compact latent representations that support semantic extraction and compression.”
Gao, X., Jamil, N., & Ramli, M. I. (2024) · Applied Sciences · doi:10.3390/app142311093 · Cited by 7
Yousef, I., Shah, S. L., & Gopaluni, R. B. (2025) · Engineering Applications of Artificial Intelligence · doi:10.1016/j.engappai.2025.110914 · Cited by 6
Abed-Esfahani, P., Darwin, B. C., Howard, D., Wang, N., Kim, E., Lerch, J. P., & French, L. (2022) · PLoS ONE · doi:10.1371/journal.pone.0262717 · Cited by 6
Rahul, M. R., & Chiddarwar, S. S. (2023) · International Journal of Machine Learning and Cybernetics · doi:10.1007/s13042-023-01891-w · Cited by 6
“Numerous techniques, including data augmentation, adversarial training [27], and contrastive learning [28], are used in some of the most recent works, such as [29].”
MethodZhang, H., Liu, W., Li, Z., Shi, J., Chang, S., Wang, H., He, J. L., & Huang, Q. (2023) · IEEE Internet of Things Journal · doi:10.1109/jiot.2023.3336915 · Cited by 7
“Representations are learned by maximizing agreement of the encoded features between two differently augmented views of the same data examples, while simultaneously minimizing those generated from different images [9].”
BackgroundMethodApplication of deep learning in fMRI-based human brain parcellation: a review
Li, Y., Chen, X., Ling, Q., He, Z., & Liu, A. (2023) · Measurement Science and Technology · doi:10.1088/1361-6501/ad14e2 · Cited by 7
“For another crucial evaluation, it is suggested to regard reproducibility as a goal to establish contrastive learning frameworks [86] so that similar parcellations within individuals while distinct parcellations across individuals.”
Information bottleneck fusion for deep multi-view clustering
Hu, J., Yang, C., Huang, K., Wang, H., Peng, B., & Li, T. (2024) · Knowledge-Based Systems · doi:10.1016/j.knosys.2024.111551 · Cited by 6
A Robust Framework for Distributional Shift Detection Under Sample-Bias
Torpmann-Hagen, B., Riegler, M. A., Halvorsen, P., & Johansen, D. (2024) · IEEE Access · doi:10.1109/access.2024.3393296 · Cited by 7
Domain-Collaborative Contrastive Learning for Hyperspectral Image Classification
Luo, H., Qiao, X., Xu, Y., Zhong, S., & Gong, C. (2024) · IEEE Geoscience and Remote Sensing Letters · doi:10.1109/lgrs.2024.3425482 · Cited by 6
“Contrastive learning (CL) is a self-supervised technique that enhances representations by contrasting positive and negative examples to bring similar ones closer and push dissimilar ones apart [9], [10].”
BackgroundGDMol: Generative Double‐Masking Self‐Supervised Learning for Molecular Property Prediction
Liu, Y., Fan, Q., Xu, C., Ning, X., Wang, Y., Liu, Y., Xie, Y., Zhang, Y., Chen, Y., & Liu, H. (2024) · Molecular Informatics · doi:10.1002/minf.202400146 · Cited by 6
“However, most of these models use contrastive learning on domain-similar activity data, which are not strictly unlabelled [12].”
Unsupervised Canine Emotion Recognition Using Momentum Contrast
Bhave, A., Hafner, A., Bhave, A., & Gloor, P. A. (2024) · Sensors · doi:10.3390/s24227324 · Cited by 6
“A detailed taxonomy of contrastive learning methods was conducted by Le-Khac et al. [13], differentiating among supervised, semi-supervised and unsupervised contrastive learning techniques.”
Yasutomi, S., & Tanaka, T. (2025) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2025.3543383 · Cited by 6
“Further, a self-supervised learning method called contrastive learning (CL) has recently gained popularity [5].”
Spike-Representation of EEG Signals for Performance Enhancement of Brain-Computer Interfaces
Singanamalla, S. K. R., & Lin, C. (2022) · Frontiers in Neuroscience · doi:10.3389/fnins.2022.792318 · Cited by 6
Semantic Information in Contrastive Learning
Quan, S., Hirano, M., & Yamakawa, Y. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.00523 · Cited by 6
“The seminal idea of contrastive learning dates back at least to the 1990s [2, 3, 31], but it has become prominent in recent years thanks to the large pretrained models in the fields of NLP and CV [31].”
BackgroundSet Representation Learning with Generalized Sliced-Wasserstein EmbeddingsPreprint
Naderializadeh, N., Kolouri, S., Comer, J. F., Andrews, R. W., & Hoffmann, H. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2103.03892 · Cited by 6
“, 2019a), leveraging the recently developed contrastive learning losses (Le-Khac et al., 2020).”
BackgroundMethodInfluentialExpanding Frozen Vision-Language Models without Retraining: Towards Improved Robot PerceptionPreprint
Tavassoli, R., Amani, M., & Akhavian, R. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2308.16493 · Cited by 7
“Contrastive learning, a subfield of unsupervised learning, works by learning a representation of its inputs such that similar inputs result in similar vectors and dissimilar inputs yield dissimilar vectors [25].”
BackgroundSheng, T., & Huber, M. (2025) · Sensors · doi:10.3390/s25134032 · Cited by 5
“Common weakly supervised strategies include multiple instance learning using group-level labels [28], label propagation from a small labeled subset [29], and contrastive methods with pairwise constraints [30,31] (e.g., must-link, cannot-link).”
View-Based Knowledge-Augmented Multimodal Semantic Understanding for Optical Remote Sensing Images
Zhu, L., Su, X., Tang, J., Hu, Y., & Wang, Y. (2025) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2025.3532349 · Cited by 6
“Contrastive learning [61], a principal method in self-supervised learning, learns effective representations by minimizing the distance between similar samples and maximizing the distance between disparate samples.”
Integrating self-supervised learning with vision transformers for glaucoma detection
Liao, C., Todo, Y., & Tang, Z. (2025) · Journal of Electronic Imaging · doi:10.1117/1.jei.34.2.023016 · Cited by 5
“36 Although methods such as SimCLR, 35 MoCo, 37 and BYOL 38 have demonstrated strong capabilities in learning robust feature representations and high-level semantic features through contrastive learning, they are primarily designed for general feature extraction and may not fully address certain…”
A contrastive clustering loss function increases class-balanced in time series classification
Wu, C., Cheng, X., & Wang, H. H. (2025) · Expert Systems with Applications · doi:10.1016/j.eswa.2025.127493 · Cited by 9
Foreground natural anomaly synthesis for attention guided anomaly detection
Xiang, X., Liu, M., Zhang, S., Wei, P., & Chen, B. (2025) · Neurocomputing · doi:10.1016/j.neucom.2025.130282 · Cited by 6
Cao, Z., Chen, S., & Li, J. (2025) · IEEE Signal Processing Letters · doi:10.1109/lsp.2025.3577922 · Cited by 6
Enhanced Spatiotemporal Consistency for Image-to-LiDAR Data Pretraining
Xu, X., Kong, L., Shuai, H., Zhang, W., Pan, L., Chen, K., Liu, Z., & Liu, Q. (2025) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2025.3640589 · Cited by 5
“By designing effective pretraining objectives, models can extract meaningful features from raw, unlabeled data, reducing dependence on large-scale annotations while improving downstream performance [11].”
Chen, Z. S., Lu, B., & Wu, W. (2025) · IEEE Signal Processing Magazine · doi:10.1109/msp.2025.3595320 · Cited by 6
“Contrastive representation learning is a framework for learning by sample comparison in a supervised, self-supervised, or un-supervised manner [28].”
A New Data Augmentation Model for Fault Diagnosis of Transformer Windings Under Scarce Fault Data
Wang, S. H., Zhu, B., Peng, H., Liu, J., & Wang, S. (2026) · IET Electric Power Applications · doi:10.1049/elp2.70140 · Cited by 5
Zero-shot forecasting of streamflow using time series foundation models: are we there yet?
Sun, A. Y., & Sun, A. A. (2026) · Machine Learning Earth · doi:10.1088/3049-4753/ae4982 · Cited by 7
Generating Compositional Color Representations from Text
Maheshwari, P., Jain, N., Vaddamanu, P., Raut, D., Vaishay, S., & Vinay, V. (2021) · ACM International Conference on Information and Knowledge Management (CIKM) · doi:10.1145/3459637.3482346 · Cited by 5
“as computer vision and natural language processing [6, 8, 20, 23].”
A Semi‐supervised Classification Method of Parasites Using Contrastive Learning
Ren, Y., Jiang, H., Zhu, H., Tian, Y., & Hu, J. (2021) · IEEJ Transactions on Electrical and Electronic Engineering · doi:10.1002/tee.23525 · Cited by 5
“By defining the similarity and dissimilarity distribution, the desired invariant/covariant properties of the learned representation are specified [17].”
BackgroundSelf-supervised scientific document recommendation based on contrastive learning
Tan, S., Zhang, T., Zhao, S. G., & Zhang, Y. (2023) · Scientometrics · doi:10.1007/s11192-023-04782-7 · Cited by 6
“In recent years, research on contrast learning has become a significant advancement in self-supervised representation learning (Le-Khac et al., 2020).”
BackgroundGeorgas, A., Georgas, K., & Hristoforou, E. V. (2023) · Micromachines · doi:10.3390/mi14081518 · Cited by 5
Li, Z., Lloyd, S., Beckman, M., & Passonneau, R. J. (2023) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2023.findings-emnlp.254 · Cited by 5
“Contrastive Learning (CL) aims to improve representation learning by making similar data instances closer and separating dissimilar ones (Le-Khac et al., 2020).”
BackgroundSemi-supervised learning for concrete defect segmentation from images
Wang, W., Su, C., Han, G., & Hu, S. (2024) · Structural Health Monitoring · doi:10.1177/14759217231217097 · Cited by 4
“37,38 Contrastive learning can be utilized to enhance the traditional cross-entropy loss by discovering the global semantic relationship between pixels across all the training images, thereby producing a structured semantic feature relationship.”
Manticore: An Unsupervised Intrusion Detection System Based on Contrastive Learning in 5G Networks
Yuan, L., Sun, J., Zhuang, S., Liu, Y., Geng, L., Zou, J., Xin, P., Huang, W., & Ma, W. (2024) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp48485.2024.10447814 · Cited by 6
“Thirdly, to improve the detection capability, some researchers explore contrastive learning in wired networks [14, 15, 16, 17], which has been widely applied in diverse domains [18, 19, 20, 21].”
Background
Metadata
- Authors
- Le-Khac, P. H., Healy, G., & Smeaton, A. F.
- Affiliations
- Dublin City University (IE)
- Venue
- IEEE Access
- Published
- 2020
- Volume
- 8
- Pages
- 193907-193934
- ISSN
- 2169-3536 (ISSN-L)
- OpenAlex
- W3096655658
- Cited by
- 862
- References
- 144
- Citations by year
- 2026: 111 · 2025: 197 · 2024: 212 · 2023: 157 · 2022: 113 · 2021: 71 · 2020: 1
- FWCI
- 33.61
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Locations
- IEEE Access (publishedVersion, cc-by)
- arXiv (Cornell University) (acceptedVersion)
- Dublin City University Open Access Institutional Repository (Dublin City University) (submittedVersion, cc-by)
- Arrow@dit (Dublin Institute of Technology) (submittedVersion, cc-by)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- arXiv (Cornell University) (cc-by)
- Funders
- Science Foundation Ireland through the Science Foundation Ireland (SFI) Centre for Research Training in Machine Learning (18/CRT/6183)
- Insight Centre for Data Analytics (SFI/12/RC/2289_P2)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Domain Adaptation and Few-Shot Learning
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Physical Sciences › Computer Science › Artificial Intelligence › Machine Learning and Algorithms
- Keywords
- contrastive learning, representation learning, self-supervised learning, metric learning, inductive bias, computer vision, natural language processing, audio signal processing, reinforcement learning
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.