The DOI matches its DataCite record; 53 of the 53 references it cites were found in a registry or an index.
Representation Learning with Contrastive Predictive Coding
Open accessAbstract
While supervised learning has enabled great progress in many applications, unsupervised learning has not seen such widespread adoption, and remains an important and challenging endeavor for artificial intelligence. In this work, we propose a universal unsupervised learning approach to extract useful representations from high-dimensional data, which we call Contrastive Predictive Coding. The key insight of our model is to learn such representations by predicting the future in latent space by using powerful autoregressive models. We use a probabilistic contrastive loss which induces the latent space to capture information that is maximally useful to predict future samples. It also makes the model tractable by using negative sampling. While most prior work has focused on evaluating representations for a particular modality, we demonstrate that our approach is able to learn useful representations achieving strong performance on four distinct domains: speech, images, text and reinforcement learning in 3D environments.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“For example, Contrastive Predictive Coding (CPC) [80,143] utilizes a contrastive future prediction task, and it was found that predicting one or two timesteps in the future is (too) easy and results in poor performance, whereas predicting multiple timesteps into the future is much more complex, and leads to very effective representations.”
From Past, Present, and Future of Sensor-based Human Activity Recognition Using Wearables: A Surveying Tutorial on a Still Challenging Task (2025) · cites this paper as [143]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Using this contrastive loss, self-supervised networks learn to make the representations similar for positive pairs, and dissimilar for negative pairs, and hence are able to cluster similar (or positive) samples together and push apart dissimilar (or negative) samples.”
From Detecting galaxy tidal features using self-supervised representation learning (2024) · cites this paper as Oord et al. 2018
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“It means that InfoNCE loss maximizes the agreement of two different augmented sequences' representations from the same sequence while minimizing the agreement with other negative sequences.”
From Contrastive Learning from Extremely Augmented Skeleton Sequences for Self-Supervised Action Recognition (2022) · cites this paper as Oord, Li, and Vinyals 2018
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“We introduce the Contrastive Predictive Coding (CPC) framework to human activity recognition, which captures the long-term temporal structure of sensor data streams.”
From Contrastive Predictive Coding for Human Activity Recognition (2021) · cites this paper as [58]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“As mentioned in [58], either 𝑐 𝑡 (the output of 𝑔 𝑒𝑛𝑐 ) or 𝑧 𝑡 (the output of 𝑔 𝑎𝑟 ) can be utilized as the representation.”
From Contrastive Predictive Coding for Human Activity Recognition (2021) · cites this paper as [58]
Experimental — accuracy not yet measured
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“The framework for data augmentation and batch formation plays a critical role in multi-view similarity learning [12,16,44,115], where we start by following the popular SimCLR [16] method.”
From MARs: Multi-view Attention Regularizations for Patch-based Feature Recognition of Space Terrain (2024) · cites this paper as [16]
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“We note that this is a common setting for self-supervised contrastive learners with NCE objectives (Gutmann & Hyvärinen, 2010;Oord et al., 2018), including SimCLR (Chen et al., 2020) and SimCSE (Gao et al., 2021).”
From The Platonic Representation Hypothesis (2024) · cites this paper as Oord et al., 2018
The cited source says what the claim attributes to it.
In the citing paper
“The key insight behind this approach is that predicting just the next future timestep involves exploiting the local smoothness of the signal, whereas predicting multiple subsequent timesteps requires inferring the global structure of time-series data.”
From Contrastive Predictive Coding for Human Activity Recognition (2021) · cites this paper as [58]
The cited source says what the claim attributes to it.
In the citing paper
“Haresamudram et al. [80] adapted Contrastive Predictive Coding (CPC [143]) for HAR, emphasizing the importance of temporal structure in sensor data representation.”
From Past, Present, and Future of Sensor-based Human Activity Recognition Using Wearables: A Surveying Tutorial on a Still Challenging Task (2025) · cites this paper as [143]
The cited source says what the claim attributes to it.
In the citing paper
“Contrastive predictive coding (CPC) proposed by [109] provides a contrastive learning framework to perform the prediction task using InfoNCE loss.”
From Self-Supervised Learning for Time Series Analysis: Taxonomy, Progress, and Prospects (2024) · cites this paper as [109]
The cited source says what the claim attributes to it.
In the citing paper
“It can be seen that CPC does not directly predict future observations x t + k . Instead, it tries to preserve the mutual information of c t and x t + k .”
From Self-Supervised Learning for Time Series Analysis: Taxonomy, Progress, and Prospects (2024) · cites this paper as [109]
The cited source says what the claim attributes to it.
In the citing paper
“This contrastive loss, or InfoNCE loss (Oord et al. 2018), is minimised when the similarity is high for positive pairs and low for negative pairs.”
From Detecting galaxy tidal features using self-supervised representation learning (2024) · cites this paper as Oord et al. 2018
The cited source says what the claim attributes to it.
In the citing paper
“To evaluate the performance of our TS-TCC model, we follow the standard linear benchmarking evaluation scheme [Oord et al. , 2018; Chen et al. , 2020]. Particularly, we train a linear classifier (single MLP layer) on top of a frozen self-supervised pretrained encoder model.”
From Time-Series Representation Learning via Temporal and Contextual Contrasting (2021) · cites this paper as Oord et al. , 2018
The cited source says what the claim attributes to it.
In the citing paper
“Contrastive-based methods usually use a contrastive similarity metric, such as InfoNCE loss [109].”
From Self-Supervised Learning for Time Series Analysis: Taxonomy, Progress, and Prospects (2024) · cites this paper as [109]
The cited source says what the claim attributes to it.
In the citing paper
“This results in representations that encode the high-level information between temporally separated parts of the time-series signal [58], thereby leading to improved downstream recognition performance.”
From Contrastive Predictive Coding for Human Activity Recognition (2021) · cites this paper as [58]
The cited source says what the claim attributes to it.
In the citing paper
“Contrastive Predictive Coding (CPC) [58], which is the focus of this work, performs contrastive learning over multiple future timestep predictions.”
From Contrastive Predictive Coding for Human Activity Recognition (2021) · cites this paper as [58]
The cited source says what the claim attributes to it.
In the citing paper
“This framework has resulted in excellent representation learning performance across domains such as computer vision [12, 33, 35], natural language [49], speech recognition [6, 58] and time-series data [39].”
From Contrastive Predictive Coding for Human Activity Recognition (2021) · cites this paper as [58]
The cited source says what the claim attributes to it.
In the citing paper
“If extra context from the past is useful for classification, then 𝑐 𝑡 can be used; else 𝑧 𝑡 may be more effective.”
From Contrastive Predictive Coding for Human Activity Recognition (2021) · cites this paper as [58]
The check wasn’t confident either way, so it gives no verdict — never disputed.
In the citing paper
“SimCLR aims to maximize the learned representation similarity between augmented views of the same input.”
From MARs: Multi-view Attention Regularizations for Patch-based Feature Recognition of Space Terrain (2024) · cites this paper as [16]
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., & Polosukhin, I. (2017) · arXiv · doi:10.48550/arxiv.1706.03762 · Cited by 120,404 · cited 1×
“More recent advancements in autoregressive modeling such as masked convolutional architectures [18, 19] or self-attention networks [20] could help improve results further.”
BackgroundImageNet classification with deep convolutional neural networks
Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017) · Communications of the ACM · doi:10.1145/3065386 · Cited by 112,155 · cited 1×
“These techniques made manually specified features largely redundant and have greatly improved state-of-the-art in several real-world applications [1, 2, 3].”
MethodAdam: A Method for Stochastic Optimization
Kingma, D. P., & Ba, J. (2014) · arXiv · doi:10.48550/arxiv.1412.6980 · Cited by 98,946 · cited 1×
“We use the Adam optimizer [32] with a learning rate of 2e-4, and use 8 GPUs each with a minibatch of 8 examples from which the negative samples in the contrastive loss are drawn.”
MethodImageNet Large Scale Visual Recognition Challenge
Russakovsky, O., Deng, J., Su, H., Krause, J., Satheesh, S., Ma, S., Huang, Z., Karpathy, A., Khosla, A., Bernstein, M., et al. (2015) · International Journal of Computer Vision · doi:10.1007/s11263-015-0816-y · Cited by 39,380 · cited 1×
“In our visual representation experiments we use the ILSVRC ImageNet competition dataset [34].”
MethodBatch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
Ioffe, S., & Szegedy, C. (2015) · arXiv · doi:10.48550/arxiv.1502.03167 · Cited by 27,818 · cited 1×
“We did not use Batch-Norm [38].”
Efficient Estimation of Word Representations in Vector Space
Mikolov, T., Chen, K., Corrado, G., & Dean, J. (2013) · arXiv · doi:10.48550/arxiv.1301.3781 · Cited by 26,568 · cited 2×
“In Word2Vec [9] neighbouring words are predicted using a contrastive loss.”
BackgroundLearning Phrase Representations using RNN Encoder–Decoder for Statistical Machine Translation
Cho, K., van Merrienboer, B., Gulcehre, C., Bahdanau, D., Bougares, F., Schwenk, H., & Bengio, Y. (2014) · Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.3115/v1/d14-1179 · Cited by 25,545
Sequence to Sequence Learning with Neural Networks
Sutskever, I., Vinyals, O., & Le, Q. V. (2014) · arXiv · doi:10.48550/arxiv.1409.3215 · Cited by 17,766 · cited 1×
“These techniques made manually specified features largely redundant and have greatly improved state-of-the-art in several real-world applications [1, 2, 3].”
BackgroundSelf-citationFaceNet: A unified embedding for face recognition and clustering
Schroff, F., Kalenichenko, D., & Philbin, J. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298682 · Cited by 15,267 · cited 1×
“For example, the techniques proposed by [21, 22, 23] were based on triplet losses using a max-margin approach to separate positive from negative examples.”
MethodLayer NormalizationPreprint
Ba, J., Kiros, J., & Hinton, G. E. (2016) · arXiv.org · doi:10.48550/arxiv.1607.06450 · Cited by 13,762 · cited 1×
“[41] is the same skip-thought model with layer normalization trained for 1M iterations. evaluate with 10-fold cross-validation for MR, CR, Subj, MPQA and use the train/test split for TREC.”
Method
Research works printed but not linked: 4
- [12][12] Michael Gutmann and Aapo Hyvärinen. Noise-contrastive estimation: A new estimation principle for unnormalized statistical models. In Proceedings of the Thirteenth International Conference on Artificial Intelligence and Statistics , pages 297-304, 2010. -
- [31][31] Daniel Povey, Arnab Ghoshal, Gilles Boulianne, Lukas Burget, Ondrej Glembek, Nagendra Goel, Mirko Hannemann, Petr Motlicek, Yanmin Qian, Petr Schwarz, et al. The kaldi speech recognition toolkit. In IEEE 2011 workshop on automatic speech recognition and understanding . IEEE Signal Processing Society, 2011. -
- [33][33] Laurens van der Maaten and Geoffrey Hinton. Visualizing data using t-sne. Journal of machine learning research , 9(Nov):2579-2605, 2008. -
- [48][48] Sida Wang and Christopher D. Manning. Baselines and bigrams: Simple, good sentiment and topic classification. In Proceedings of the 50th Annual Meeting of the Association for Computational Linguistics: Short Papers - Volume 2 , ACL '12, pages 90-94, 2012. -
Cited by
Sorted by citations
Items 901–1,000 of 1,066
SUPERB: Speech processing Universal PERformance Benchmark
Yang, S.-W., Chi, P.-H., Chuang, Y.-S., Lai, C.-I. J., Lakhotia, K., Lin, Y. Y., Liu, A. T., Shi, J., Chang, X., Lin, G.-T., et al. (2021) · arXiv · doi:10.48550/arxiv.2105.01051 · Cited by 52
Uncertainty-Aware Clustering for Unsupervised Domain Adaptive Object Re-Identification
Wang, P., Ding, C., Tan, W., Gong, M., Jia, K., & Tao, D. (2022) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2022.3149629 · Cited by 55
Wang, H., Ahn, E., & Kim, J. (2022) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v36i2.20143 · Cited by 54
ViKiNG: Vision-Based Kilometer-Scale Navigation with Geographic Hints
Shah, D., & Levine, S. (2022) · Robotics: Science and Systems · doi:10.15607/rss.2022.xviii.019 · Cited by 54
Mix and Localize: Localizing Sound Sources in Mixtures
Hu, X., Chen, Z., & Owens, A. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.01023 · Cited by 53
Contrastive Learning for Legal Judgment Prediction
Zhang, H., Dou, Z., Zhu, Y., & Wen, J.-R. (2023) · ACM Transactions on Information Systems · doi:10.1145/3580489 · Cited by 57
“It has been widely applied in computer vision [2, 9, 11, 27], natural language processing [6, 32], and information retrieval [40, 41], due to its high eiciency in leveraging the training data without the need for annotation.”
BackgroundLearning Audio-Visual Source Localization via False Negative Aware Contrastive Learning
Sun, W., Zhang, J., Wang, J., Liu, Z., Zhong, Y., Feng, T., Guo, Y., Zhang, Y., & Barnes, N. M. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.00621 · Cited by 55
“As such, the Noise Contrastive Estimation (NCE) loss [24, 30] is used to perform instance discrimination by pushing closer the distance between a positive audio-image pair, while pulling away any negative pairs.”
BackgroundMethodInfluentialSelf-supervised learning for point cloud data: A survey
Zeng, C., Wang, W., Nguyen, A., Xiao, J., & Yue, Y. (2023) · Expert Systems with Applications · doi:10.1016/j.eswa.2023.121354 · Cited by 60
Knowledge-Aided Momentum Contrastive Learning for Remote-Sensing Image Text Retrieval
Ji, Z., Meng, C., Zhang, Y., Pang, Y., & Li, X. (2023) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2023.3332317 · Cited by 52
SMART: Syntax-Calibrated Multi-Aspect Relation Transformer for Change Captioning
Tu, Y., Li, L., Su, L., et al. (2024) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2024.3365104 · Cited by 52
“Next, we introduce the InfoNCE loss [44] to optimize their cross-view contrastive alignment, i.e., pulling semantically close pairs of ˜ x k,bcls and ˆ x k,acls together and pushing away non-related pairs where “sim” is the dot-product function to measure the similarity between two features. τ is…”
MethodFu, D., Liu, J., Zhong, H., Zhang, X., & Zhang, F. (2024) · Knowledge-Based Systems · doi:10.1016/j.knosys.2024.111846 · Cited by 56
Spatial–Spectral Graph Contrastive Clustering With Hard Sample Mining for Hyperspectral Images
Guan, R., Tu, W., Li, Z., et al. (2024) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2024.3464648 · Cited by 52
Speech Recognition and Multi-Speaker Diarization of Long Conversations
Mao, H. H., Li, S. X., McAuley, J., & Cottrell, G. W. (2020) · Interspeech (USB) · doi:10.21437/interspeech.2020-3039 · Cited by 58
Symbolic pregression: Discovering physical laws from distorted video
Udrescu, S., & Tegmark, M. (2021) · Physical review. E · doi:10.1103/physreve.103.043307 · Cited by 56
CDPAM: Contrastive Learning for Perceptual Audio Similarity
Manocha, P., Jin, Z., Zhang, R., & Finkelstein, A. (2021) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp39728.2021.9413711 · Cited by 56
Self-Regulated Learning for Egocentric Video Activity Anticipation
Qi, Z., Wang, S., Su, C., Su, L., Huang, Q., & Tian, Q. C. (2021) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2021.3059923 · Cited by 57
Text-Free Prosody-Aware Generative Spoken Language Modeling
Kharitonov, E., Lee, A., Polyak, A., Adi, Y., Copet, J., Lakhotia, K., Nguyen, T. A., Rivière, M., Mohamed, A., Emmanuel Dupoux, A., et al. (2022) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2022.acl-long.593 · Cited by 53
Learning Dynamic Belief Graphs to Generalize on Text-Based GamesPreprint
Adhikari, A., Yuan, X., Côté, M.-A., Zelinka, M., Rondeau, M.-A., Laroche, R., Poupart, P., Tang, J., Trischler, A., & Hamilton, W. L. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2002.09127 · Cited by 62
Prototypical contrastive learning of unsupervised representationsPreprint
Li, J. (2024) · arXiv (Cornell University) · doi:10.57702/qpm0hks1 · Cited by 52
i-Mix: A Domain-Agnostic Strategy for Contrastive Representation LearningPreprint
Lee, K., Zhu, Y., Sohn, K., Li, C., Shin, J., & Lee, H. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2010.08887 · Cited by 82
Negative Selection by Clustering for Contrastive Learning in Human Activity Recognition
Wang, J., Zhu, T., Chen, L. L., Ning, H., & Wan, Y. (2023) · IEEE Internet of Things Journal · doi:10.1109/jiot.2023.3239945 · Cited by 52
Shot Contrastive Self-Supervised Learning for Scene Boundary Detection
Chen, S., Nie, X., Fan, D., Zhang, D., Bhat, V., & Hamid, R. (2021) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr46437.2021.00967 · Cited by 62
Intra-Inter Subject Self-Supervised Learning for Multivariate Cardiac Signals
Xiang, L., Ng, D., Hong, S., & Feng, M. (2022) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v36i4.20376 · Cited by 54
Chen, Z., Wei, J., & Li, R. (2022) · International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2022/117 · Cited by 51
A Survey of Traversability Estimation for Mobile Robots
Sevastopoulos, C., & Konstantopoulos, S. (2022) · IEEE Access · doi:10.1109/access.2022.3202545 · Cited by 55
Pinheiro, G. A., Silva, J. L. F. D., & Quiles, M. G. (2022) · Journal of Chemical Information and Modeling · doi:10.1021/acs.jcim.2c00521 · Cited by 54
Supervised contrastive learning for recommendation
Yang, C. T., Zou, J., Wu, J., Hong-bing, X., & Fan, S. (2022) · Knowledge-Based Systems · doi:10.1016/j.knosys.2022.109973 · Cited by 53
Towards Online Domain Adaptive Object Detection
Vibashan, V. S., Oza, P., & Patel, V. M. (2023) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv56688.2023.00055 · Cited by 56
Trends and developments in automatic speech recognition research
O’Shaughnessy, D. D. (2023) · Computer Speech & Language · doi:10.1016/j.csl.2023.101538 · Cited by 72
Shilpa, S., Kashyap, G., & Sunoj, R. B. (2023) · The Journal of Physical Chemistry A · doi:10.1021/acs.jpca.3c04779 · Cited by 54
Building3D: An Urban-Scale Dataset and Benchmarks for Learning Roof Structures from Point Clouds
Wang, R., Huang, S., & Yang, H. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.01837 · Cited by 55
“Started from self-supervised learning on 2D images, SimCLR [28] and CPC [29,30] have reached top performance on image classification benchmarks.”
MethodTowards Large-Scale 3D Representation Learning with Multi-Dataset Point Prompt Training
Wu, X., Tian, Z., Wen, X., Peng, B., Liu, X., Yu, K., & Zhao, H. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.01849 · Cited by 72
“To achieve this goal, we employ the InfoNCE [65] as alignment criteria and restrict negative samples to the specific dataset category space as shown in Fig.”
MethodInfluentialFighting Class Imbalance with Contrastive Learning
Marrakchi, Y., Makansi, O., & Brox, T. (2021) · Lecture notes in computer science · doi:10.1007/978-3-030-87199-4_44 · Cited by 61
Detecting Outliers with Foreign Patch Interpolation
Tan, J., Hou, B., Batten, J., Qiu, H., & Kainz, B. (2022) · The Journal of Machine Learning for Biomedical Imaging · doi:10.59275/j.melba.2022-e651 · Cited by 49
Hero: Hierarchical Encoder for Video+Language Omni-representation Pre-trainingPreprint
Li, L., Chen, Y.-C., Cheng, Y., Gan, Z., Yu, L., & Liu, J.-J. (2020) · Conference on Empirical Methods in Natural Language Processing · doi:10.48550/arxiv.2005.00200 · Cited by 576
An Improved Inter-Intra Contrastive Learning Framework on Self-Supervised Video Representation
Tao, L., Wang, X., & Yamasaki, T. (2022) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2022.3141051 · Cited by 50
Utilizing Self-Supervised Representations for MOS Prediction
Tseng, W.-C., Huang, C., Kao, W.-T., Lin, Y. Y., & Lee, H.-Y. (2021) · Interspeech (USB) · doi:10.21437/interspeech.2021-2013 · Cited by 54
Efficient and Accurate Conversion of Spiking Neural Network with Burst Spikes
Li, Y., & Zeng, Y. (2022) · International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2022/345 · Cited by 77
Self-citationTopic-Aware Contrastive Learning for Abstractive Dialogue Summarization
Liu, J., Zou, Y., Zhang, H., Chen, H., Ding, Z., Yuan, C., & Wang, X. (2021) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2021.findings-emnlp.106 · Cited by 60
SCS-Co: Self-Consistent Style Contrastive Learning for Image Harmonization
Hang, Y., Xia, B., Yang, W., & Liao, Q. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.01909 · Cited by 51
ESA: External Space Attention Aggregation for Image-Text Retrieval
Zhu, H., Zhang, C., Wei, Y., Huang, S., & Zhao, Y. (2023) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2023.3253548 · Cited by 49
“samples, the InfoNCE loss [50] considers all negative samples within mini-batch, while the hardest negative triplet loss (HTL) [13] considers the most challenging negative sample”
BackgroundA Robust Shape-Aware Rib Fracture Detection and Segmentation Framework With Contrastive Learning
Cao, Z., Xu, L., Chen, D. Z., Gao, H., & Wu, J. (2023) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2023.3263074 · Cited by 53
Contrastive Learning Models for Sentence Representations
Xu, L., Xie, H., Li, Z., Wang, F. L., Wang, W., & Li, Q. (2023) · ACM Transactions on Intelligent Systems and Technology · doi:10.1145/3593590 · Cited by 50
“Mutual information (MI) loss is another frequently used loss function in representation learning, and MI loss-based models learn representations by maximizing the MI between the inputs and their hidden vector representations [80, 101].”
BackgroundInfluentialLin, Y., Qu, Z., Chen, H., Gao, Z., Li, Y., Xia, L., Ma, K., Zheng, Y., & Cheng, K.-T. T. (2023) · Medical Image Analysis · doi:10.1016/j.media.2023.102933 · Cited by 56
Self-citationUnderstanding and Constructing Latent Modality Structures in Multi-Modal Representation Learning
Jiang, Q., Chen, C., Zhao, H., Chen, L., Ping, Q., Tran, S. D., Xu, Y., Zeng, B., & Chilimbi, T. M. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.00740 · Cited by 57
“CLIP [40, 48, 52], ALIGN [25]), and rely on contrastive loss [6, 20, 44] to align multiple modalities.”
MethodImproving Table Structure Recognition with Visual-Alignment Sequential Coordinate Modeling
Huang, Y., Lü, N., Chen, D., et al. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.01071 · Cited by 50
“An In-foNCE [48] loss is employed between f nci and all visual representation { g nci } Ki =1 in the image.”
MethodSelf-citationCoVR: Learning Composed Video Retrieval from Web Video Captions
Ventura, L., Yang, A., Schmid, C., & Varol, G. (2024) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v38i6.28334 · Cited by 62
MCL: Multimodal Contrastive Learning for Deepfake Detection
Liu, X., Yu, Y., Li, X., & Zhao, Y. (2023) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2023.3312738 · Cited by 59
“Inspired by the InfoNCE [63], a contrastive loss L ct that brings the authenticity label into the loss formulation is designed.”
MethodMulti-Grained Radiology Report Generation With Sentence-Level Image-Language Contrastive Learning
Liu, A., Guo, Y., Yong, J., & Xu, F. (2024) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2024.3372638 · Cited by 55
“Formally, the contrastive loss for a batch of size B is based on the InfoNCE loss [38] and represented as : where R i and I i denote the i-th report and image, S denotes a sentence in the i-th report, and τ is a trainable parameter.”
BackgroundWang, Y., Chen, X., Zhao, E., Zhao, C., Song, M., & Yu, C. (2024) · IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing · doi:10.1109/jstars.2024.3387985 · Cited by 53
Unsupervised Deep Representation Learning and Few-Shot Classification of PolSAR Images
Zhang, L., Zhang, S., Zou, B., & Dong, H. (2020) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2020.3043191 · Cited by 52
FairFil: Contrastive Neural Debiasing Method for Pretrained Text EncodersPreprint
Cheng, P., Hao, W., Yuan, S., Si, S., & Carin, L. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2103.06413 · Cited by 93
Object DGCNN: 3D Object Detection using Dynamic GraphsPreprint
Wang, Y., & Solomon, J. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2110.06923 · Cited by 51
Learnable Locality-Sensitive Hashing for Video Anomaly Detection
Lu, Y., Cao, C., Zhang, Y., & Zhang, Y. (2022) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2022.3205348 · Cited by 50
SLUE: New Benchmark Tasks For Spoken Language Understanding Evaluation on Natural Speech
Shon, S., Pasad, A., Wu, F., Brusco, P., Artzi, Y., Livescu, K., & Han, K. J. (2022) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp43922.2022.9746137 · Cited by 60
Wang, F., Zhao, Z., Zhai, Z., Guo, Y., Xi, H., Wang, S., & Chen, X. (2022) · Reliability Engineering & System Safety · doi:10.1016/j.ress.2022.108897 · Cited by 48
Liu, T., Zhang, C., Lam, K., & Kong, J. (2022) · IEEE Transactions on Information Forensics and Security · doi:10.1109/tifs.2022.3216479 · Cited by 50
Disentangled Representations Learning for Multi-target Cross-domain Recommendation
Guo, X., Li, S., Guo, N., et al. (2022) · ACM Transactions on Information Systems · doi:10.1145/3572835 · Cited by 49
“Inspired by the adversarial learning, several literatures using self-supervised data augmentation techniques also achieve the disentangled representation learning, by designing diferent contrastive objectives over structural sequential data [44, 53] and nonstructural graphical data [45].”
MethodZhang, H., Liu, W., Shi, J., Chang, S., Wang, H., He, J. L., & Huang, Q. (2022) · IEEE Transactions on Instrumentation and Measurement · doi:10.1109/tim.2022.3228267 · Cited by 61
U²-Former: Nested U-Shaped Transformer for Image Restoration via Multi-View Contrastive Learning
Feng, X., Ji, H., Pei, W., Li, J., Lu, G., & Zhang, D. D. (2023) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2023.3286405 · Cited by 79
“Thus, to maximize the mutual information between corresponding patches, we employ the noise contrastive estimation(NCE) framework [56], [57], and then the binary classification is set up : exp ( e p i · e pos j ) exp ( e p i · e pos j ) + where N i , N j , and N k are the number of patches for…”
MethodI’m Me, We’re Us, and I’m Us: Tri-directional Contrastive Learning on Hypergraphs
Lee, D., & Shin, K. (2023) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v37i7.26019 · Cited by 55
CoCoSoDa: Effective Contrastive Learning for Code Search
Shi, E., Wang, Y., Gu, W., Du, L., Zhang, H., Han, S., Zhang, D., & Sun, H. (2023) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1109/icse48619.2023.00185 · Cited by 90
Filtering, Distillation, and Hard Negatives for Vision-Language Pre-Training
Radenović, F., Dubey, A., Kadian, A., Mihaylov, T., Vandenhende, S., Patel, Y., Wen, Y., Ramanathan, V., & Mahajan, D. K. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.00673 · Cited by 56
“Several lines of work have studied the shortcomings of the original InfoNCE objective [55], specifically, the selection and importance of negative samples.”
BackgroundMethodInfluentialShi, Y., Xu, H., Zhang, Y., Qi, Z., & Wang, D. (2023) · IEEE Transactions on Cognitive Communications and Networking · doi:10.1109/tccn.2023.3318414 · Cited by 55
“Contrastive self-supervised learning methods train an encoder to encode x into an explicit vector z to measure similarity between different samples, such as PIRL [33], CPC [34], and SimCLR [35].”
MethodEnhancing Cross-View Geo-Localization With Domain Alignment and Scene Consistency
Xia, P., Wan, Y., Zheng, Z., Zhang, Y., & Deng, J. (2024) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2024.3443510 · Cited by 69
“The aforementioned works are based on discriminative models, while [27] introduces a generative model-based contrastive representation learning method.”
A survey of graph neural networks and their industrial applications
Lu, H., Wang, L., Ma, X., Cheng, J., & Zhou, M. (2024) · Neurocomputing · doi:10.1016/j.neucom.2024.128761 · Cited by 66
Arango-Argoty, G., Bikiel, D. E., Sun, G. J., Kipkogei, E., Smith, K. M., Pro, S. C., Choe, E. Y., & Jacob, E. (2025) · Cancer Cell · doi:10.1016/j.ccell.2025.03.029 · Cited by 61
Towards Unsupervised Speech Recognition and Synthesis with Quantized Speech Representation Learning
Liu, A. H., Tu, T., Lee, H.-Y., & Lee, L.-S. (2020) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp40776.2020.9053571 · Cited by 45
slimIPL: Language-Model-Free Iterative Pseudo-Labeling
Likhomanenko, T., Xu, Q., Kahn, J., Synnaeve, G., & Collobert, R. (2021) · Interspeech (USB) · doi:10.21437/interspeech.2021-740 · Cited by 46
3D Human Shape and Pose from a Single Low-Resolution Image with Self-Supervised Learning
Xu, X., Chen, H., Moreno-Noguer, F., Jeni, L. A., & la Torre, F. D. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-58545-7_17 · Cited by 45
On Compositions of Transformations in Contrastive Self-Supervised Learning
Patrick, M., Asano, Y. M., Kuznetsova, P., Fong, R., Henriques, J. F., Zweig, G., & Vedaldi, A. (2021) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00944 · Cited by 65
Avqvc: One-Shot Voice Conversion By Vector Quantization With Applying Contrastive Learning
Tang, H., Zhang, X., Wang, J., Cheng, N., & Xiao, J. (2022) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp43922.2022.9746369 · Cited by 51
MixCo: Mix-up Contrastive Learning for Visual RepresentationPreprint
Kim, S., Lee, G., Bae, S., & Yun, S.-Y. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2010.06300 · Cited by 85
From Canonical Correlation Analysis to Self-supervised Graph Neural NetworksPreprint
Zhang, H., Wu, Q., Yan, J., Wipf, D., & Yu, P. S. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2106.12484 · Cited by 138
SLAM: A Unified Encoder for Speech and Language Modeling via Speech-Text Joint Pre-TrainingPreprint
Bapna, A., Chung, Y.-A., Wu, N., Gulati, A., Ye, J., Clark, J. H., Johnson, M., Riesa, J., Conneau, A., & Zhang, Y. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2110.10329 · Cited by 95
Point-BERT: Pre-training 3D Point Cloud Transformers with Masked Point ModelingPreprint
Yu, X., Tang, L., Rao, Y., et al. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2111.14819 · Cited by 49
MuLan: A Joint Embedding of Music Audio and Natural Language
Huang, Q., Jansen, A., Lee, J., Ganti, R., Li, J. Y., & Ellis, D. P. W. (2022) · arXiv · doi:10.48550/arxiv.2208.12415 · Cited by 49
On Exposing the Challenging Long Tail in Future Prediction of Traffic Actors
Makansi, O., Çiçek, Ö., Marrakchi, Y., & Brox, T. (2021) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.01290 · Cited by 56
Negative sampling strategies for contrastive self-supervised learning of graph representations
Hafidi, H., Ghogho, M., Ciblat, P., & Swami, A. (2021) · Signal Processing · doi:10.1016/j.sigpro.2021.108310 · Cited by 47
Self-Supervised RF Signal Representation Learning for NextG Signal Classification With Deep Learning
Davaslıoğlu, K., Boztaş, S., Ertem, M. C., Sagduyu, Y. E., & Ayanoğlu, E. (2022) · IEEE Wireless Communications Letters · doi:10.1109/lwc.2022.3217292 · Cited by 57
“MoCo-v3 uses the Information Noise-Contrastive Estimation (InfoNCE) loss [20] that is given by”
MethodRobust Cross-Modal Representation Learning with Progressive Self-Distillation
Andonian, A., Chen, S., & Hamid, R. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.01594 · Cited by 57
Improving Medical Vision-Language Contrastive Pretraining With Semantics-Aware Triage
Liu, B., Lu, D., Wei, D., Wu, X., Wang, Y., Zhang, Y., & Zheng, Y. (2023) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2023.3294980 · Cited by 56
“One of the representative paradigm is to pretrain the network by maximizing the mutual information between medical images and their corresponding reports with contrastive loss [38] to learn more useful visual representations.”
BackgroundTowards All-in-One Pre-Training via Maximizing Multi-Modal Mutual Information
Su, W., Zhu, X., Tao, C., Lu, L., Li, B., Huang, G. M., Qiao, Y., Wang, X., Zhou, J., & Dai, J. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.01525 · Cited by 58
Unpaired Image-to-Image Translation with Shortest Path Regularization
Xie, S., Xu, Y., Gong, M., et al. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.00981 · Cited by 55
“Then it employs the infoNCE loss [57] to associate corresponding patches and disassoci-ate them from others.”
MethodWang, F., Wu, Z., Zhao, Z., Zhai, Z., Wang, C., & Chen, X. (2024) · Reliability Engineering & System Safety · doi:10.1016/j.ress.2024.110325 · Cited by 54
Learning fine-grained representation with token-level alignment for multimodal sentiment analysis
Li, X., Zhang, H., Dong, Z., Cheng, X., Liu, Y., & Zhang, X. (2024) · Expert Systems with Applications · doi:10.1016/j.eswa.2024.126274 · Cited by 57
Multimodal foundation models for material property prediction and discovery
Moro, V., Loh, C., Dangovski, R., Ghorashi, A., Ma, A., Chen, Z., Kim, S., Lu, P. Y., Christensen, T., & Marin Soljacic, A. (2025) · Newton · doi:10.1016/j.newton.2025.100016 · Cited by 69
Self-Supervised Representation Learning From Multi-Domain Data
Feng, Z., Xu, C., & Tao, D. (2019) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00334 · Cited by 48
Delving into Inter-Image Invariance for Unsupervised Visual Representations
Xie, J., Zhan, X., Liu, Z., Ong, Y.-S., & Loy, C. C. (2022) · International Journal of Computer Vision · doi:10.1007/s11263-022-01681-x · Cited by 59
Text-Free Image-to-Speech Synthesis Using Learned Segmental Units
Hsu, W.-N., Harwath, D. F., Miller, T., Song, C., & Glass, J. R. (2021) · doi:10.18653/v1/2021.acl-long.411 · Cited by 65
Pretraining Techniques for Sequence-to-Sequence Voice Conversion
Huang, W.-C., Hayashi, T., Wu, Y.-C., Kameoka, H., & Toda, T. (2021) · IEEE/ACM Transactions on Audio Speech and Language Processing · doi:10.1109/taslp.2021.3049336 · Cited by 46
Chikontwe, P., Luna, M., Kang, M., Hong, K. S., Ahn, J. H., & Park, S. H. (2021) · Medical Image Analysis · doi:10.1016/j.media.2021.102105 · Cited by 54
Self-supervised human mobility learning for next location prediction and trajectory classification
Zhou, F., Dai, Y., Gao, Q., Wang, P., & Zhong, T. (2021) · Knowledge-Based Systems · doi:10.1016/j.knosys.2021.107214 · Cited by 52
Domain Contrast for Domain Adaptive Object Detection
Liu, F., Zhang, X., Wan, F., Ji, X., & Ye, Q. (2021) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2021.3091620 · Cited by 49
Modulated Contrast for Versatile Image Synthesis
Zhan, F., Zhang, J., Yu, Y., Wu, R., & Lu, S. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.01774 · Cited by 56
Video Representation Learning by Dense Predictive CodingPreprint
Han, T., Xie, W., & Zisserman, A. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1909.04656 · Cited by 48
On Mutual Information in Contrastive Learning for Visual RepresentationsPreprint
Wu, M., Zhuang, C., Mosse, M., Yamins, D. L. K., & Goodman, N. D. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2005.13149 · Cited by 55
Transductive Information Maximization For Few-Shot LearningPreprint
Boudiaf, M. A., Masud, Z. I., Rony, J., et al. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2008.11297 · Cited by 48
Mavromatis, C., & Karypis, G. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2009.06946 · Cited by 72
MoCo-CXR: MoCo Pretraining Improves Representation and Transferability of Chest X-ray ModelsPreprint
Sowrirajan, H., Yang, J., Ng, A. Y., & Rajpurkar, P. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2010.05352 · Cited by 61
Metadata
- Authors
- Oord, A. V. D., Li, Y., & Vinyals, O.
- Venue
- arXiv
- Publisher
- arXiv
- Published
- 2018
- ISSN
- 2331-8422 (ISSN-L)
- arXiv
- 1807.03748
- OpenAlex
- W4297808394
- Cited by
- 6,242
- References
- 50
- Citations by year
- 2026: 77 · 2025: 660 · 2024: 1288 · 2023: 1444 · 2022: 1043 · 2021: 1173 · 2020: 454 · 2019: 88 · 2018: 15
- Type
- posted content
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Domain Adaptation and Few-Shot Learning
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Physical Sciences › Computer Science › Artificial Intelligence › Advanced Graph Neural Networks
- Keywords
- contrastive predictive coding, unsupervised feature learning, autoregressive models, negative sampling, high-dimensional data, speech representation, image representation, text representation
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
