The DOI matches its DataCite record; 53 of the 53 references it cites were found in a registry or an index.
Representation Learning with Contrastive Predictive Coding
Open accessAbstract
While supervised learning has enabled great progress in many applications, unsupervised learning has not seen such widespread adoption, and remains an important and challenging endeavor for artificial intelligence. In this work, we propose a universal unsupervised learning approach to extract useful representations from high-dimensional data, which we call Contrastive Predictive Coding. The key insight of our model is to learn such representations by predicting the future in latent space by using powerful autoregressive models. We use a probabilistic contrastive loss which induces the latent space to capture information that is maximally useful to predict future samples. It also makes the model tractable by using negative sampling. While most prior work has focused on evaluating representations for a particular modality, we demonstrate that our approach is able to learn useful representations achieving strong performance on four distinct domains: speech, images, text and reinforcement learning in 3D environments.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“For example, Contrastive Predictive Coding (CPC) [80,143] utilizes a contrastive future prediction task, and it was found that predicting one or two timesteps in the future is (too) easy and results in poor performance, whereas predicting multiple timesteps into the future is much more complex, and leads to very effective representations.”
From Past, Present, and Future of Sensor-based Human Activity Recognition Using Wearables: A Surveying Tutorial on a Still Challenging Task (2025) · cites this paper as [143]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Using this contrastive loss, self-supervised networks learn to make the representations similar for positive pairs, and dissimilar for negative pairs, and hence are able to cluster similar (or positive) samples together and push apart dissimilar (or negative) samples.”
From Detecting galaxy tidal features using self-supervised representation learning (2024) · cites this paper as Oord et al. 2018
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“It means that InfoNCE loss maximizes the agreement of two different augmented sequences' representations from the same sequence while minimizing the agreement with other negative sequences.”
From Contrastive Learning from Extremely Augmented Skeleton Sequences for Self-Supervised Action Recognition (2022) · cites this paper as Oord, Li, and Vinyals 2018
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“We introduce the Contrastive Predictive Coding (CPC) framework to human activity recognition, which captures the long-term temporal structure of sensor data streams.”
From Contrastive Predictive Coding for Human Activity Recognition (2021) · cites this paper as [58]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“As mentioned in [58], either 𝑐 𝑡 (the output of 𝑔 𝑒𝑛𝑐 ) or 𝑧 𝑡 (the output of 𝑔 𝑎𝑟 ) can be utilized as the representation.”
From Contrastive Predictive Coding for Human Activity Recognition (2021) · cites this paper as [58]
Experimental — accuracy not yet measured
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“The framework for data augmentation and batch formation plays a critical role in multi-view similarity learning [12,16,44,115], where we start by following the popular SimCLR [16] method.”
From MARs: Multi-view Attention Regularizations for Patch-based Feature Recognition of Space Terrain (2024) · cites this paper as [16]
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“We note that this is a common setting for self-supervised contrastive learners with NCE objectives (Gutmann & Hyvärinen, 2010;Oord et al., 2018), including SimCLR (Chen et al., 2020) and SimCSE (Gao et al., 2021).”
From The Platonic Representation Hypothesis (2024) · cites this paper as Oord et al., 2018
The cited source says what the claim attributes to it.
In the citing paper
“The key insight behind this approach is that predicting just the next future timestep involves exploiting the local smoothness of the signal, whereas predicting multiple subsequent timesteps requires inferring the global structure of time-series data.”
From Contrastive Predictive Coding for Human Activity Recognition (2021) · cites this paper as [58]
The cited source says what the claim attributes to it.
In the citing paper
“Haresamudram et al. [80] adapted Contrastive Predictive Coding (CPC [143]) for HAR, emphasizing the importance of temporal structure in sensor data representation.”
From Past, Present, and Future of Sensor-based Human Activity Recognition Using Wearables: A Surveying Tutorial on a Still Challenging Task (2025) · cites this paper as [143]
The cited source says what the claim attributes to it.
In the citing paper
“Contrastive predictive coding (CPC) proposed by [109] provides a contrastive learning framework to perform the prediction task using InfoNCE loss.”
From Self-Supervised Learning for Time Series Analysis: Taxonomy, Progress, and Prospects (2024) · cites this paper as [109]
The cited source says what the claim attributes to it.
In the citing paper
“It can be seen that CPC does not directly predict future observations x t + k . Instead, it tries to preserve the mutual information of c t and x t + k .”
From Self-Supervised Learning for Time Series Analysis: Taxonomy, Progress, and Prospects (2024) · cites this paper as [109]
The cited source says what the claim attributes to it.
In the citing paper
“This contrastive loss, or InfoNCE loss (Oord et al. 2018), is minimised when the similarity is high for positive pairs and low for negative pairs.”
From Detecting galaxy tidal features using self-supervised representation learning (2024) · cites this paper as Oord et al. 2018
The cited source says what the claim attributes to it.
In the citing paper
“To evaluate the performance of our TS-TCC model, we follow the standard linear benchmarking evaluation scheme [Oord et al. , 2018; Chen et al. , 2020]. Particularly, we train a linear classifier (single MLP layer) on top of a frozen self-supervised pretrained encoder model.”
From Time-Series Representation Learning via Temporal and Contextual Contrasting (2021) · cites this paper as Oord et al. , 2018
The cited source says what the claim attributes to it.
In the citing paper
“Contrastive-based methods usually use a contrastive similarity metric, such as InfoNCE loss [109].”
From Self-Supervised Learning for Time Series Analysis: Taxonomy, Progress, and Prospects (2024) · cites this paper as [109]
The cited source says what the claim attributes to it.
In the citing paper
“This results in representations that encode the high-level information between temporally separated parts of the time-series signal [58], thereby leading to improved downstream recognition performance.”
From Contrastive Predictive Coding for Human Activity Recognition (2021) · cites this paper as [58]
The cited source says what the claim attributes to it.
In the citing paper
“Contrastive Predictive Coding (CPC) [58], which is the focus of this work, performs contrastive learning over multiple future timestep predictions.”
From Contrastive Predictive Coding for Human Activity Recognition (2021) · cites this paper as [58]
The cited source says what the claim attributes to it.
In the citing paper
“This framework has resulted in excellent representation learning performance across domains such as computer vision [12, 33, 35], natural language [49], speech recognition [6, 58] and time-series data [39].”
From Contrastive Predictive Coding for Human Activity Recognition (2021) · cites this paper as [58]
The cited source says what the claim attributes to it.
In the citing paper
“If extra context from the past is useful for classification, then 𝑐 𝑡 can be used; else 𝑧 𝑡 may be more effective.”
From Contrastive Predictive Coding for Human Activity Recognition (2021) · cites this paper as [58]
The check wasn’t confident either way, so it gives no verdict — never disputed.
In the citing paper
“SimCLR aims to maximize the learned representation similarity between augmented views of the same input.”
From MARs: Multi-view Attention Regularizations for Patch-based Feature Recognition of Space Terrain (2024) · cites this paper as [16]
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., & Polosukhin, I. (2017) · arXiv · doi:10.48550/arxiv.1706.03762 · Cited by 120,404 · cited 1×
“More recent advancements in autoregressive modeling such as masked convolutional architectures [18, 19] or self-attention networks [20] could help improve results further.”
BackgroundImageNet classification with deep convolutional neural networks
Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017) · Communications of the ACM · doi:10.1145/3065386 · Cited by 112,155 · cited 1×
“These techniques made manually specified features largely redundant and have greatly improved state-of-the-art in several real-world applications [1, 2, 3].”
MethodAdam: A Method for Stochastic Optimization
Kingma, D. P., & Ba, J. (2014) · arXiv · doi:10.48550/arxiv.1412.6980 · Cited by 98,946 · cited 1×
“We use the Adam optimizer [32] with a learning rate of 2e-4, and use 8 GPUs each with a minibatch of 8 examples from which the negative samples in the contrastive loss are drawn.”
MethodImageNet Large Scale Visual Recognition Challenge
Russakovsky, O., Deng, J., Su, H., Krause, J., Satheesh, S., Ma, S., Huang, Z., Karpathy, A., Khosla, A., Bernstein, M., et al. (2015) · International Journal of Computer Vision · doi:10.1007/s11263-015-0816-y · Cited by 39,380 · cited 1×
“In our visual representation experiments we use the ILSVRC ImageNet competition dataset [34].”
MethodBatch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
Ioffe, S., & Szegedy, C. (2015) · arXiv · doi:10.48550/arxiv.1502.03167 · Cited by 27,818 · cited 1×
“We did not use Batch-Norm [38].”
Efficient Estimation of Word Representations in Vector Space
Mikolov, T., Chen, K., Corrado, G., & Dean, J. (2013) · arXiv · doi:10.48550/arxiv.1301.3781 · Cited by 26,568 · cited 2×
“In Word2Vec [9] neighbouring words are predicted using a contrastive loss.”
BackgroundLearning Phrase Representations using RNN Encoder–Decoder for Statistical Machine Translation
Cho, K., van Merrienboer, B., Gulcehre, C., Bahdanau, D., Bougares, F., Schwenk, H., & Bengio, Y. (2014) · Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.3115/v1/d14-1179 · Cited by 25,545
Sequence to Sequence Learning with Neural Networks
Sutskever, I., Vinyals, O., & Le, Q. V. (2014) · arXiv · doi:10.48550/arxiv.1409.3215 · Cited by 17,766 · cited 1×
“These techniques made manually specified features largely redundant and have greatly improved state-of-the-art in several real-world applications [1, 2, 3].”
BackgroundSelf-citationFaceNet: A unified embedding for face recognition and clustering
Schroff, F., Kalenichenko, D., & Philbin, J. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298682 · Cited by 15,267 · cited 1×
“For example, the techniques proposed by [21, 22, 23] were based on triplet losses using a max-margin approach to separate positive from negative examples.”
MethodLayer NormalizationPreprint
Ba, J., Kiros, J., & Hinton, G. E. (2016) · arXiv.org · doi:10.48550/arxiv.1607.06450 · Cited by 13,762 · cited 1×
“[41] is the same skip-thought model with layer normalization trained for 1M iterations. evaluate with 10-fold cross-validation for MR, CR, Subj, MPQA and use the train/test split for TREC.”
Method
Research works printed but not linked: 4
- [12][12] Michael Gutmann and Aapo Hyvärinen. Noise-contrastive estimation: A new estimation principle for unnormalized statistical models. In Proceedings of the Thirteenth International Conference on Artificial Intelligence and Statistics , pages 297-304, 2010. -
- [31][31] Daniel Povey, Arnab Ghoshal, Gilles Boulianne, Lukas Burget, Ondrej Glembek, Nagendra Goel, Mirko Hannemann, Petr Motlicek, Yanmin Qian, Petr Schwarz, et al. The kaldi speech recognition toolkit. In IEEE 2011 workshop on automatic speech recognition and understanding . IEEE Signal Processing Society, 2011. -
- [33][33] Laurens van der Maaten and Geoffrey Hinton. Visualizing data using t-sne. Journal of machine learning research , 9(Nov):2579-2605, 2008. -
- [48][48] Sida Wang and Christopher D. Manning. Baselines and bigrams: Simple, good sentiment and topic classification. In Proceedings of the 50th Annual Meeting of the Association for Computational Linguistics: Short Papers - Volume 2 , ACL '12, pages 90-94, 2012. -
Cited by
Sorted by citations
Items 801–900 of 1,066
Joint Representation Learning and Novel Category Discovery on Single- and Multi-modal Data
Jia, X., Han, K., Zhu, Y., & Green, B. (2021) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00065 · Cited by 64
Li, S., Liu, F., Jiao, L., Chen, P., & Li, L. (2022) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2022.3185638 · Cited by 59
“…puzzle [69], counting [70], rotation [23], instance discrimination [71], [72], etc. Contrast self-supervised learning is a breakthrough direction, which achieves the most advanced performance, such as InfoNCE [73], MoCo [74], SimCLR [24], SimSiam [75], and Bootstrap Your Own Latent (BYOL) [76].”
BackgroundMCRN: A Multi-source Cross-modal Retrieval Network for remote sensing
Yuan, Z., Zhang, W., Tian, C., Mao, Y., Zhou, R., Wang, H., Fu, K., & Sun, X. (2022) · International Journal of Applied Earth Observation and Geoinformation · doi:10.1016/j.jag.2022.103071 · Cited by 61
Multi-modal Alignment using Representation Codebook
Duan, J., Chen, L., Tran, S., Yang, J., Xu, Y., Zeng, B., & Chilimbi, T. M. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.01520 · Cited by 71
Self-Supervised Vision Transformers for Joint SAR-Optical Representation Learning
Wang, Y., Albrecht, C. M., & Zhu, X. X. (2022) · IEEE International Geoscience and Remote Sensing Symposium (IGARSS) · doi:10.1109/igarss46834.2022.9883983 · Cited by 82
GRLC: Graph Representation Learning With Constraints
Peng, L., Mo, Y., Xu, J., Shen, J., Shi, X., Li, X., Shen, H. T., & Zhu, X. (2023) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2022.3230979 · Cited by 63
“In real applications, uni-modal losses (e.g., the mean squared error and the cross-entropy loss) are inefficient for learning discriminative representations as they focus on every detail of the reconstruction process [6].”
BackgroundMethodOpen-Vocabulary Point-Cloud Object Detection without 3D Annotation
Lu, Y., Xu, C., Wei, X., Xie, X., Tomizuka, M., Keutzer, K., & Zhang, S. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.00121 · Cited by 63
“And the contrastive loss [18] is given by”
BackgroundToward Label-Efficient Emotion and Sentiment Analysis
Zhao, S., Hong, X., Yang, J., Zhao, Y., & Ding, G. (2023) · Proceedings of the IEEE · doi:10.1109/jproc.2023.3309299 · Cited by 62
“During unsupervised contrastive pretraining, the unlabeled texts/images are clustered in the latent space, forming fairly good decision boundaries between different classes [82].”
BackgroundRobust Optical and SAR Image Matching Using Attention-Enhanced Structural Features
Ye, Y., Yang, C., Gong, G., Yang, P., Quan, D., & Li, J. (2024) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2024.3366247 · Cited by 62
Artificial intelligence in metal forming
Cao, J., Bambach, M., Merklein, M., Mozaffar, M., & Xue, T. (2024) · CIRP Annals · doi:10.1016/j.cirp.2024.04.102 · Cited by 66
DeepMDP: Learning Continuous Latent Space Models for Representation LearningPreprint
Gelada, C., Kumar, S., Buckman, J., Nachum, O., & Bellemare, M. G. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1906.02736 · Cited by 118
Cluster Contrast for Unsupervised Person Re-IdentificationPreprint
Dai, Z., Wang, G., Yuan, W., et al. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2103.11568 · Cited by 59
DualNet: Continual Learning, Fast and Slow
Pham, Q., Liu, C., & Hoi, S. (2021) · arXiv · doi:10.48550/arxiv.2110.00175 · Cited by 84
LatentCLR: A Contrastive Learning Approach for Unsupervised Discovery of Interpretable Directions
Yüksel, O. K., Simsar, E., Er, E. G., & Yanardag, P. (2021) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.01400 · Cited by 54
Spoken Moments: Learning Joint Audio-Visual Representations from Video Descriptions
Monfort, M., Jin, S., Liu, A. H., Harwath, D. F., Feris, R., Glass, J. R., & Oliva, A. (2021) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr46437.2021.01463 · Cited by 63
I2V-GAN: Unpaired Infrared-to-Visible Video Translation
Li, S., Han, B., Yu, Z., Liu, C. H., Chen, K., & Wang, S. (2021) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3474085.3475445 · Cited by 58
CALDA: Improving Multi-Source Time Series Domain Adaptation With Contrastive Adversarial Learning
Wilson, G., Doppa, J. R., & Cook, D. J. (2023) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2023.3298346 · Cited by 67
Selvaraj, P., Gokul, N. C., Kumar, P., & Khapra, M. M. (2022) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2022.acl-long.150 · Cited by 74
SEGA: Semantic Guided Attention on Visual Prototype for Few-Shot Learning
Yang, F., Wang, R., & Chen, X. (2022) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv51458.2022.00165 · Cited by 63
Efficient Adapter Transfer of Self-Supervised Speech Models for Automatic Speech Recognition
Thomas, B., Kessler, S., & Karout, S. A. (2022) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp43922.2022.9746223 · Cited by 68
Interinstance and Intratemporal Self-Supervised Learning With Few Labeled Data for Fault Diagnosis
Hu, C., Wu, J., Sun, C., Yan, R., & Chen, X. (2022) · IEEE Transactions on Industrial Informatics · doi:10.1109/tii.2022.3183601 · Cited by 59
“For example, CPC [21] extracted unsupervised features for speech recognition by maximizing the similarity between audio and its context segments.”
MethodNear-Optimal Representation Learning for Hierarchical Reinforcement LearningPreprint
Nachum, O., Gu, S., Lee, H., & Levine, S. (2018) · International Conference on Learning Representations · doi:10.48550/arxiv.1810.01257 · Cited by 239
Meta reinforcement learning as task inference
Humplik, J., Galashov, A., Hasenclever, L., Ortega, P. A., Teh, Y. W., & Heess, N. (2019) · arXiv · doi:10.48550/arxiv.1905.06424 · Cited by 82
A Survey of Deep Learning for Scientific DiscoveryPreprint
Raghu, M., & Schmidt, E. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2003.11755 · Cited by 221
DeCoAR 2.0: Deep Contextualized Acoustic Representations with Vector QuantizationPreprint
Ling, S., & Liu, Y. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2012.06659 · Cited by 60
Applying Wav2vec2.0 to Speech Recognition in Various Low-resource LanguagesPreprint
Cheng, Y., Wang, J., Cheng, N., Zhou, S., & Xu, B. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2012.12121 · Cited by 83
DiVA: Diverse Visual Feature Aggregation for Deep Metric Learning
Milbich, T., Roth, K., Bharadhwaj, H., Sinha, S., Bengio, Y., Ommer, B., & Cohen, J. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-58598-3_35 · Cited by 60
CIL: Contrastive Instance Learning Framework for Distantly Supervised Relation Extraction
Chen, T., Shi, H., Tang, S., Chen, Z., Wu, F. Q., & Zhuang, Y. (2021) · doi:10.18653/v1/2021.acl-long.483 · Cited by 66
Self-Supervised Correlation Learning for Cross-Modal Retrieval
Liu, Y., Wu, J., Qu, L., Gan, T., Yin, J., & Nie, L. (2022) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2022.3152086 · Cited by 57
Simple and Effective Zero-shot Cross-lingual Phoneme Recognition
Xu, Q., Baevski, A., & Auli, M. (2022) · Interspeech (USB) · doi:10.21437/interspeech.2022-60 · Cited by 115
Deep Contrastive One-Class Time Series Anomaly Detection
Wang, R., Liu, C., Mou, X., Gao, K., Guo, X., Pin, L., Wo, T., & Liu, X. (2023) · Society for Industrial and Applied Mathematics eBooks · doi:10.1137/1.9781611977653.ch78 · Cited by 68
A Review of Predictive and Contrastive Self-supervised Learning for Medical Images
Wang, W.-C., Ahn, E., Feng, D. D., & Kim, J. (2023) · Machine Intelligence Research · doi:10.1007/s11633-022-1406-4 · Cited by 64
“Based on the ideas of NCE, CPC proposed InfoNCE and its relationship with MI.”
BackgroundMethodInfluentialWang, D., Deng, Y., Yin, Z., Shum, H., & Wang, B. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.01724 · Cited by 63
“We enforce the InfoNCE loss [43] following [77] to maximize the similarity between positive pairs and minimize the similarity between negative pairs:”
MethodBinding Touch to Everything: Learning Unified Multimodal Tactile Representations
Yang, F., Feng, C., Chen, Z., Park, H., Wang, D., Dou, Y., Zeng, Z., Chen, X., Gangopadhyay, S., Owens, A., et al. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.02488 · Cited by 98
“We optimize this objective using InfoNCE loss [93] to match touches to correct images: where τ is a temperature hyperparameter [120] and C is feature dimension.”
MethodPredictively encoded graph convolutional network for noise-robust skeleton-based action recognition
Yoon, Y., Yu, J., & Jeon, M. (2021) · Applied Intelligence · doi:10.1007/s10489-021-02487-z · Cited by 56
Unsupervised Neural Network Models of the Ventral Visual StreamPreprint
Zhuang, C., Yan, S., Nayebi, A., et al. (2020) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2020.06.16.155556 · Cited by 57
Self-Supervised Sim-to-Real Adaptation for Visual Robotic Manipulation
Jeong, R., Aytar, Y., Khosid, D., Zhou, Y., Kay, J., Lampe, T., Bousmalis, K., & Nori, F. (2020) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra40945.2020.9197326 · Cited by 60
CLCC: Contrastive Learning for Color Constancy
Lo, Y.-C., Chang, C.-C., Chiu, H.-C., Huang, Y., Chen, C.-P., Chang, Y.-L., & Jou, K. (2021) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr46437.2021.00796 · Cited by 63
Self-Supervised Learning on 3D Point Clouds by Learning Discrete Generative Models
Eckart, B., Yuan, W., Liu, C., & Kautz, J. (2021) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr46437.2021.00815 · Cited by 56
Fu, H., Zhou, S., Yang, Q., Tang, J., Liu, G., Liu, K., & Li, X. (2021) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v35i14.17518 · Cited by 58
Clustering by Maximizing Mutual Information Across Views
Do, K., Tran, T., & Venkatesh, S. (2021) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00978 · Cited by 64
Wav2vec-Switch: Contrastive Learning from Original-Noisy Speech Pairs for Robust Speech Recognition
Wang, Y., Li, J., Wang, H., Qian, Y., Wang, C., & Wu, Y. (2022) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp43922.2022.9746929 · Cited by 55
Uncertainty-Guided Pixel Contrastive Learning for Semi-Supervised Medical Image Segmentation
Wang, T., Lu, J., Lai, Z., Wen, J., & Kong, H. (2022) · International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2022/201 · Cited by 60
“We use the popular InfoNCE [van den Oord et al. , 2018] loss function to calculate the contrastive loss.”
MethodContrastive Domain Adaptation for Time-Series Via Temporal Mixup
Eldele, E., Ragab, M., Chen, Z., Wu, M., Kwoh, C. K., & Li, X. (2023) · IEEE Transactions on Artificial Intelligence · doi:10.1109/tai.2023.3293473 · Cited by 66
“As inspired by [41], we leverage the probability vectors in the InfoNCE loss [42] to maximize the similarity between each domain and its corresponding intermediate view.”
MethodFine-Grained Classification with Noisy Labels
Wei, Q., Feng, L., Sun, H., Wang, R., Guo, C., & Yin, Y. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.01121 · Cited by 59
“In CL, a widely used loss function for measuring the similarity of each data point is InfoNCE [30] and can be summarised as LINFO = − log exp (q · q̂/τ) exp (q · q̂/τ) + ∑D d=1 exp (q · kd/τ) , where τ is a hyper-parameter for temperature scaling.”
MethodLearning Gait Representation From Massive Unlabelled Walking Videos: A Benchmark
Fan, C., Hou, S., Wang, J., Huang, Y., & Yu, S. (2023) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2023.3312419 · Cited by 61
“There are many pretext tasks usually employed in contrastive framework, e.g., context auto-encoding in [56] and contrastive multiview coding in [57].”
MethodLiang, B., Gui, L., He, Y., Cambria, E., & Xu, R. (2024) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2024.3380375 · Cited by 66
“Supervised contrastive learning is proposed by [42], which is built on the contrastive self-supervised literature [40], [50], [55], [56] and allows us to effectively leverage the label information of the data.”
BackgroundSelf-supervised Learning for Video Correspondence FlowPreprint
Lai, Z., & Xie, W. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1905.00875 · Cited by 104
Are all negatives created equal in contrastive instance discrimination?Preprint
Cai, T., Frankle, J., Schwab, D. J., & Morcos, A. S. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2010.06682 · Cited by 79
Semi-Supervised Time Series Classification by Temporal Relation Prediction
Fan, H., Zhang, F., Wang, R., Huang, X., & Li, Z. (2021) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp39728.2021.9413883 · Cited by 64
Representation Learning via Global Temporal Alignment and Cycle-Consistency
Hadji, I., Derpanis, K. G., & Jepson, A. D. (2021) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr46437.2021.01092 · Cited by 60
Yellapragada, B., Hornauer, S., Snyder, K. R., Yu, S. X., & Yiu, G. C. (2021) · Ophthalmology Retina · doi:10.1016/j.oret.2021.06.010 · Cited by 56
Graph Communal Contrastive Learning
Li, B., Jing, B., & Tong, H. (2022) · International World Wide Web Conference (WWW) · doi:10.1145/3485447.3512208 · Cited by 60
Hierarchical Self-Supervised Learning for 3D Tooth Segmentation in Intra-Oral Mesh Scans
Liu, Z., He, X., Wang, H., Xiong, H., Zhang, Y., Wang, G., Hao, J., Feng, Y., Zhu, F., & Hu, H. R. (2022) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2022.3222388 · Cited by 55
“Contrastive learning is a typical self-supervised learning approach that has recently achieved great success and become a milestone in the field of visual representation learning [14], [16], [34].”
BackgroundMethodInstaFormer: Instance-Aware Image-to-Image Translation with Transformer
Kim, S.-H., Baek, J., Park, J., Kim, G., & Kim, S. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.01778 · Cited by 62
Toward Effective Image Manipulation Detection With Proposal Contrastive Learning
Zeng, Y., Zhao, B., Qiu, S., Dai, T., & Xia, S. (2023) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2023.3247607 · Cited by 64
“Then, following the InfoNCE loss [32], the proposed PCL loss is defined as”
MethodMicroAST: Towards Super-fast Ultra-Resolution Arbitrary Style Transfer
Wang, Z., Zhao, L., Zuo, Z., et al. (2023) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v37i3.25374 · Cited by 55
Contrastive Video Question Answering via Video Graph Transformer
Xiao, J., Zhou, P., Yao, A., Li, Y., Hong, R., Yan, S., & Chua, T. (2023) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2023.3292266 · Cited by 73
“L is the loss value, λ is a trade-off parameter and L ∗ ( x, x + , x − ) is given by the InfoNCE loss [70] ⎡ ⎛ ⎞ ⎤ where x , x + , x − denote the placeholders for the inputs of the anchor, positive and negative samples respectively. s VGT denotes the dot-product value of the two inputs in the…”
BackgroundSelf-citationCheng, D., Zhang, L., Bu, C., Wu, H., & Song, A. (2023) · Knowledge-Based Systems · doi:10.1016/j.knosys.2023.110789 · Cited by 55
Liu, Z., Wen, C., Su, Z., Liu, S., Sun, J., Kong, W., & Yang, Z. (2023) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2023.3294636 · Cited by 58
“Oord et al. [47] proposed a probabilistic contrastive loss named InfoNCE, which induces the latent space to capture information that is maximally useful for predicting future samples.”
BackgroundRao, H., & Miao, C. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.02118 · Cited by 63
“A contrastive predictive coding (CPC) approach based on the context auto-encoding and probabilistic contrastive loss [27] is proposed to learn different-domain representations.”
MethodPiCO+: Contrastive Label Disambiguation for Robust Partial Label Learning
Wang, H., Xiao, R., Li, Y., Feng, L., Niu, G., Chen, G., & Zhao, J. (2023) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2023.3342650 · Cited by 88
Self-citationVerbs in Action: Improving verb understanding in video-language models
Momeni, L., Caron, M., Nagrani, A., Zisserman, A., & Schmid, C. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.01428 · Cited by 58
“[70] Aäron van den Oord, Yazhe Li, and Oriol Vinyals.”
BackgroundMethodPseudo Labeling Methods for Semi-Supervised Semantic Segmentation: A Review and Future Perspectives
Ran, L., Li, Y., Liang, G., & Zhang, Y. (2024) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2024.3508768 · Cited by 66
Self-citationDivide and Contrast: Self-supervised Learning from Uncurated Data
Tian, Y., Hénaff, O. J., & van den Oord, A. (2021) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00991 · Cited by 69
Self-citationVi2CLR: Video and Image for Visual Contrastive Learning of Representation
Diba, A., Sharma, V., Safdari, R., Lotfi, D., Sarfraz, M. S., Stiefelhagen, R., & Van Gool, L. (2021) · 2021 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00153 · Cited by 53
Fine-grained Angular Contrastive Learning with Coarse Labels
Bukchin, G., Schwartz, E., Saenko, K., Shahar, O., Feris, R., Giryes, R., & Karlinsky, L. (2021) · 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr46437.2021.00862 · Cited by 55
Multilingual Multimodal Pre-training for Zero-Shot Cross-Lingual Transfer of Vision-Language Models
Huang, P.-Y., Patrick, M., Hu, J., Neubig, G., Metze, F., & Hauptmann, A. G. (2021) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/2021.naacl-main.195 · Cited by 68
Contrast and Order Representations for Video Self-supervised Learning
Hu, K., Shao, J., Liu, Y., Raj, B., Savvides, M., & Shen, Z. (2021) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00784 · Cited by 56
Video Contrastive Learning with Global Context
Kuang, H., Zhu, Y., Zhang, Z., Li, X., Tighe, J., Schwertfeger, S., Stachniss, C., & Li, M. (2021) · IEEE International Conference on Computer Vision Workshops · doi:10.1109/iccvw54120.2021.00358 · Cited by 58
Lu, A. X., Lu, A. X., Pritišanac, I., Zarin, T., Forman-Kay, J. D., & Moses, A. M. (2022) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1010238 · Cited by 56
Image Coding for Machines with Omnipotent Feature Learning
Feng, R., Jin, X., Guo, Z., Feng, R., Gao, Y., He, T., Zhang, Z., Sun, S., & Chen, Z. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-19836-6_29 · Cited by 62
Probabilistic Representations for Video Contrastive Learning
Park, J., Lee, J., Kim, I.-J., & Sohn, K. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.01430 · Cited by 59
Wang, Y., Wu, M., Li, X., Xie, L., & Chen, Z. (2023) · IEEE Transactions on Artificial Intelligence · doi:10.1109/tai.2023.3241896 · Cited by 56
“For HAR, we compare with SupEncoder [4], CPC [37], and TS-TCC [4], whose results are adopted from [4].”
MethodPrototype Matching Learning for Incomplete Multi-View Clustering
Yuan, H., Sun, Y., Zhou, F., et al. (2025) · IEEE Transactions on Image Processing · doi:10.1109/tip.2025.3529378 · Cited by 59
“Conventional contrastive learning methods, such as InfoNCE [33] and CCE [34] tend to overly focus on false positive noise pairs within prototypes during training, leading to overfitting.”
Decomposition-based multi-scale transformer framework for time series anomaly detection
Zhang, W., & Luo, C. (2025) · Neural Networks · doi:10.1016/j.neunet.2025.107399 · Cited by 61
“Traditional contrastive learning needs to manually generate positive and negative samples, then lessen the gap between samples in the same categories and magnify the distance between samples in different categories, such as InfoNCE (Oord et al., 2018), RINCE (Hoffmann et al., 2022), ReCo (Lin et al., 2023).”
Contrastive Self-Supervised Pre-Training for Video Quality Assessment
Chen, P., Li, L., Wu, J., Dong, W., & Shi, G. (2021) · IEEE Transactions on Image Processing · doi:10.1109/tip.2021.3130536 · Cited by 56
Bootstrapped Representation Learning on GraphsPreprint
Thakoor, S., Tallec, C., Azar, M. G., Munos, R., Veličković, P., & Valko, M. (2021) · arXiv (Cornell University) · Cited by 118
ReduNet: A White-box Deep Network from the Principle of Maximizing Rate ReductionPreprint
Chan, K. H. R., Yu, Y., You, C., Qi, H., Wright, J. N., & Ma, Y. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2105.10446 · Cited by 77
Deep Self-Supervised Representation Learning for Free-Hand Sketch
Xu, P., Song, Z., Yin, Q., Song, Y.-Z., & Wang, L. (2021) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2020.3003048 · Cited by 53
SPLAT: Speech-Language Joint Pre-Training for Spoken Language Understanding
Chung, Y.-A., Zhu, C., & Zeng, M. (2021) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/2021.naacl-main.152 · Cited by 66
Contrastive Unsupervised Learning for Speech Emotion Recognition
Li, M., Yang, B., Levy, J., Stolcke, A., Rozgić, V., Matsoukas, S., Papayiannis, C., Bone, D., & Wang, C. (2021) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp39728.2021.9413910 · Cited by 60
Towards Compact Single Image Super-Resolution via Contrastive Self-distillation
Wang, Y., Lin, S., Qu, Y., et al. (2021) · doi:10.24963/ijcai.2021/155 · Cited by 53
Dual Contrastive Loss and Attention for GANs
Yu, N., Liu, G., Dündar, A., Tao, A., Catanzaro, B., Davis, L. S., & Fritz, M. (2021) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00666 · Cited by 58
A Leap among Quantum Computing and Quantum Neural Networks: A Survey
Massoli, F. V., Vadicamo, L., Amato, G. M., & Falchi, F. (2022) · ACM Computing Surveys · doi:10.1145/3529756 · Cited by 78
Semantic Feature Extraction for Generalized Zero-Shot Learning
Kim, J., Shim, K., & Shim, B. (2022) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v36i1.20002 · Cited by 52
Contrastive Graph Convolutional Networks with adaptive augmentation for text classification
Yang, Y., Miao, R., Wang, Y., & Wang, X. (2022) · Information Processing & Management · doi:10.1016/j.ipm.2022.102946 · Cited by 55
Generative Spoken Dialogue Language Modeling
Nguyen, T. A., Kharitonov, E., Copet, J., Adi, Y., Hsu, W.-N., Elkahky, A. M., Tomasello, P., Algayres, R., Sagot, B., Mohamed, A., et al. (2023) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00545 · Cited by 97
Improving the Generalization of MAML in Few-Shot Classification via Bi-Level Constraint
Shao, Y., Wu, W., You, X., Gao, C., & Sang, N. (2022) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2022.3232717 · Cited by 53
“Contrastive learning is a fundamental technique in self-supervised representation learning [39], [40], [41], [42], [43], [44].”
BackgroundMethodPart Aware Contrastive Learning for Self-Supervised Action Recognition
Hua, Y., Wu, W., Zheng, C., Lu, A., Liu, M., Chen, C., & Wu, S. (2023) · International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2023/95 · Cited by 59
Wang, Z., & Ma, C. (2023) · IEEE International Conference on Computer Vision Workshops · doi:10.1109/iccvw60793.2023.00094 · Cited by 58
“Contrastive Learning: Contrastive learning has emerged as a powerful tool for learning robust and discriminative feature representations, and it is a form of selfsupervised learning [30].”
BackgroundSSiT: Saliency-Guided Self-Supervised Image Transformer for Diabetic Retinopathy Grading
Huang, Y., Lyu, J., Cheng, P., Tam, R. C., & Tang, X. (2024) · IEEE Journal of Biomedical and Health Informatics · doi:10.1109/jbhi.2024.3362878 · Cited by 67
“In this work, the InfoNCE loss [42] is adopted as the objective function wherein positive sample o + k denotes features from the same source image as o q , while negative sample set { o − k } denotes features from other images in the same training batch. τ is a temperature parameter [43]…”
BackgroundDetecting and Grounding Multi-Modal Media Manipulation and Beyond
Shao, R., Wu, T., Wu, J., Nie, L., & Liu, Z. (2024) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2024.3367749 · Cited by 53
“Following InfoNCE loss [53], we formulate image-to-text contrastive loss by:”
MethodDoes Negative Sampling Matter? a Review With Insights Into its Theory and Applications
Yang, Z., Ming, D., Huang, T., Cen, Y., Song, J., Xu, B., Dong, Y., & Tang, J. (2024) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2024.3371473 · Cited by 65
“In-batch negative sampling [48], [156] samples negatives from the current mini-batch.”
BackgroundPixel-Superpixel Contrastive Learning and Pseudo-Label Correction for Hyperspectral Image Clustering
Guan, R., Li, Z., Li, X., & Tang, C. (2024) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp48485.2024.10447080 · Cited by 57
“The loss function generally uses the InfoNCE loss [21], and the loss function definition formula for the positive example pair ( z i , z j ) is where sim ( z i , z j ) is a similarity function that calculates the similarity between z i and the positive sample z j , z k , k ∈ [1 , K ] is the…”
MethodLu, S., Zhu, Z., Zhang, Y., & Yao, Y. (2025) · Engineering Applications of Artificial Intelligence · doi:10.1016/j.engappai.2025.111575 · Cited by 54
Self-supervised object detection from audio-visual correspondence
Afouras, T., Asano, Y. M., Fagan, F., Vedaldi, A., & Metze, F. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.01032 · Cited by 56
Lai, C.-I., Chuang, Y.-S., Lee, H.-Y., Li, S.-W. D., & Glass, J. R. (2021) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp39728.2021.9414922 · Cited by 60
Contrastive Pre-Training of GNNs on Heterogeneous Graphs
Jiang, X., Lu, Y., Fang, Y., & Shi, C. (2021) · ACM International Conference on Information and Knowledge Management (CIKM) · doi:10.1145/3459637.3482332 · Cited by 57
A Survey on Contrastive Self-Supervised Learning
Jaiswal, A., Babu, A. R., Zadeh, M. Z., Banerjee, D., & Makedon, F. (2020) · Technologies · doi:10.3390/technologies9010002 · Cited by 154
Metadata
- Authors
- Oord, A. V. D., Li, Y., & Vinyals, O.
- Venue
- arXiv
- Publisher
- arXiv
- Published
- 2018
- ISSN
- 2331-8422 (ISSN-L)
- arXiv
- 1807.03748
- OpenAlex
- W4297808394
- Cited by
- 6,242
- References
- 50
- Citations by year
- 2026: 77 · 2025: 660 · 2024: 1288 · 2023: 1444 · 2022: 1043 · 2021: 1173 · 2020: 454 · 2019: 88 · 2018: 15
- Type
- posted content
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Domain Adaptation and Few-Shot Learning
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Physical Sciences › Computer Science › Artificial Intelligence › Advanced Graph Neural Networks
- Keywords
- contrastive predictive coding, unsupervised feature learning, autoregressive models, negative sampling, high-dimensional data, speech representation, image representation, text representation
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
