The DOI matches its DataCite record; 53 of the 53 references it cites were found in a registry or an index.
Representation Learning with Contrastive Predictive Coding
Open accessAbstract
While supervised learning has enabled great progress in many applications, unsupervised learning has not seen such widespread adoption, and remains an important and challenging endeavor for artificial intelligence. In this work, we propose a universal unsupervised learning approach to extract useful representations from high-dimensional data, which we call Contrastive Predictive Coding. The key insight of our model is to learn such representations by predicting the future in latent space by using powerful autoregressive models. We use a probabilistic contrastive loss which induces the latent space to capture information that is maximally useful to predict future samples. It also makes the model tractable by using negative sampling. While most prior work has focused on evaluating representations for a particular modality, we demonstrate that our approach is able to learn useful representations achieving strong performance on four distinct domains: speech, images, text and reinforcement learning in 3D environments.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“For example, Contrastive Predictive Coding (CPC) [80,143] utilizes a contrastive future prediction task, and it was found that predicting one or two timesteps in the future is (too) easy and results in poor performance, whereas predicting multiple timesteps into the future is much more complex, and leads to very effective representations.”
From Past, Present, and Future of Sensor-based Human Activity Recognition Using Wearables: A Surveying Tutorial on a Still Challenging Task (2025) · cites this paper as [143]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Using this contrastive loss, self-supervised networks learn to make the representations similar for positive pairs, and dissimilar for negative pairs, and hence are able to cluster similar (or positive) samples together and push apart dissimilar (or negative) samples.”
From Detecting galaxy tidal features using self-supervised representation learning (2024) · cites this paper as Oord et al. 2018
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“It means that InfoNCE loss maximizes the agreement of two different augmented sequences' representations from the same sequence while minimizing the agreement with other negative sequences.”
From Contrastive Learning from Extremely Augmented Skeleton Sequences for Self-Supervised Action Recognition (2022) · cites this paper as Oord, Li, and Vinyals 2018
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“We introduce the Contrastive Predictive Coding (CPC) framework to human activity recognition, which captures the long-term temporal structure of sensor data streams.”
From Contrastive Predictive Coding for Human Activity Recognition (2021) · cites this paper as [58]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“As mentioned in [58], either 𝑐 𝑡 (the output of 𝑔 𝑒𝑛𝑐 ) or 𝑧 𝑡 (the output of 𝑔 𝑎𝑟 ) can be utilized as the representation.”
From Contrastive Predictive Coding for Human Activity Recognition (2021) · cites this paper as [58]
Experimental — accuracy not yet measured
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“The framework for data augmentation and batch formation plays a critical role in multi-view similarity learning [12,16,44,115], where we start by following the popular SimCLR [16] method.”
From MARs: Multi-view Attention Regularizations for Patch-based Feature Recognition of Space Terrain (2024) · cites this paper as [16]
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“We note that this is a common setting for self-supervised contrastive learners with NCE objectives (Gutmann & Hyvärinen, 2010;Oord et al., 2018), including SimCLR (Chen et al., 2020) and SimCSE (Gao et al., 2021).”
From The Platonic Representation Hypothesis (2024) · cites this paper as Oord et al., 2018
The cited source says what the claim attributes to it.
In the citing paper
“The key insight behind this approach is that predicting just the next future timestep involves exploiting the local smoothness of the signal, whereas predicting multiple subsequent timesteps requires inferring the global structure of time-series data.”
From Contrastive Predictive Coding for Human Activity Recognition (2021) · cites this paper as [58]
The cited source says what the claim attributes to it.
In the citing paper
“Haresamudram et al. [80] adapted Contrastive Predictive Coding (CPC [143]) for HAR, emphasizing the importance of temporal structure in sensor data representation.”
From Past, Present, and Future of Sensor-based Human Activity Recognition Using Wearables: A Surveying Tutorial on a Still Challenging Task (2025) · cites this paper as [143]
The cited source says what the claim attributes to it.
In the citing paper
“Contrastive predictive coding (CPC) proposed by [109] provides a contrastive learning framework to perform the prediction task using InfoNCE loss.”
From Self-Supervised Learning for Time Series Analysis: Taxonomy, Progress, and Prospects (2024) · cites this paper as [109]
The cited source says what the claim attributes to it.
In the citing paper
“It can be seen that CPC does not directly predict future observations x t + k . Instead, it tries to preserve the mutual information of c t and x t + k .”
From Self-Supervised Learning for Time Series Analysis: Taxonomy, Progress, and Prospects (2024) · cites this paper as [109]
The cited source says what the claim attributes to it.
In the citing paper
“This contrastive loss, or InfoNCE loss (Oord et al. 2018), is minimised when the similarity is high for positive pairs and low for negative pairs.”
From Detecting galaxy tidal features using self-supervised representation learning (2024) · cites this paper as Oord et al. 2018
The cited source says what the claim attributes to it.
In the citing paper
“To evaluate the performance of our TS-TCC model, we follow the standard linear benchmarking evaluation scheme [Oord et al. , 2018; Chen et al. , 2020]. Particularly, we train a linear classifier (single MLP layer) on top of a frozen self-supervised pretrained encoder model.”
From Time-Series Representation Learning via Temporal and Contextual Contrasting (2021) · cites this paper as Oord et al. , 2018
The cited source says what the claim attributes to it.
In the citing paper
“Contrastive-based methods usually use a contrastive similarity metric, such as InfoNCE loss [109].”
From Self-Supervised Learning for Time Series Analysis: Taxonomy, Progress, and Prospects (2024) · cites this paper as [109]
The cited source says what the claim attributes to it.
In the citing paper
“This results in representations that encode the high-level information between temporally separated parts of the time-series signal [58], thereby leading to improved downstream recognition performance.”
From Contrastive Predictive Coding for Human Activity Recognition (2021) · cites this paper as [58]
The cited source says what the claim attributes to it.
In the citing paper
“Contrastive Predictive Coding (CPC) [58], which is the focus of this work, performs contrastive learning over multiple future timestep predictions.”
From Contrastive Predictive Coding for Human Activity Recognition (2021) · cites this paper as [58]
The cited source says what the claim attributes to it.
In the citing paper
“This framework has resulted in excellent representation learning performance across domains such as computer vision [12, 33, 35], natural language [49], speech recognition [6, 58] and time-series data [39].”
From Contrastive Predictive Coding for Human Activity Recognition (2021) · cites this paper as [58]
The cited source says what the claim attributes to it.
In the citing paper
“If extra context from the past is useful for classification, then 𝑐 𝑡 can be used; else 𝑧 𝑡 may be more effective.”
From Contrastive Predictive Coding for Human Activity Recognition (2021) · cites this paper as [58]
The check wasn’t confident either way, so it gives no verdict — never disputed.
In the citing paper
“SimCLR aims to maximize the learned representation similarity between augmented views of the same input.”
From MARs: Multi-view Attention Regularizations for Patch-based Feature Recognition of Space Terrain (2024) · cites this paper as [16]
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., & Polosukhin, I. (2017) · arXiv · doi:10.48550/arxiv.1706.03762 · Cited by 120,404 · cited 1×
“More recent advancements in autoregressive modeling such as masked convolutional architectures [18, 19] or self-attention networks [20] could help improve results further.”
BackgroundImageNet classification with deep convolutional neural networks
Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017) · Communications of the ACM · doi:10.1145/3065386 · Cited by 112,155 · cited 1×
“These techniques made manually specified features largely redundant and have greatly improved state-of-the-art in several real-world applications [1, 2, 3].”
MethodAdam: A Method for Stochastic Optimization
Kingma, D. P., & Ba, J. (2014) · arXiv · doi:10.48550/arxiv.1412.6980 · Cited by 98,946 · cited 1×
“We use the Adam optimizer [32] with a learning rate of 2e-4, and use 8 GPUs each with a minibatch of 8 examples from which the negative samples in the contrastive loss are drawn.”
MethodImageNet Large Scale Visual Recognition Challenge
Russakovsky, O., Deng, J., Su, H., Krause, J., Satheesh, S., Ma, S., Huang, Z., Karpathy, A., Khosla, A., Bernstein, M., et al. (2015) · International Journal of Computer Vision · doi:10.1007/s11263-015-0816-y · Cited by 39,380 · cited 1×
“In our visual representation experiments we use the ILSVRC ImageNet competition dataset [34].”
MethodBatch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
Ioffe, S., & Szegedy, C. (2015) · arXiv · doi:10.48550/arxiv.1502.03167 · Cited by 27,818 · cited 1×
“We did not use Batch-Norm [38].”
Efficient Estimation of Word Representations in Vector Space
Mikolov, T., Chen, K., Corrado, G., & Dean, J. (2013) · arXiv · doi:10.48550/arxiv.1301.3781 · Cited by 26,568 · cited 2×
“In Word2Vec [9] neighbouring words are predicted using a contrastive loss.”
BackgroundLearning Phrase Representations using RNN Encoder–Decoder for Statistical Machine Translation
Cho, K., van Merrienboer, B., Gulcehre, C., Bahdanau, D., Bougares, F., Schwenk, H., & Bengio, Y. (2014) · Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.3115/v1/d14-1179 · Cited by 25,545
Sequence to Sequence Learning with Neural Networks
Sutskever, I., Vinyals, O., & Le, Q. V. (2014) · arXiv · doi:10.48550/arxiv.1409.3215 · Cited by 17,766 · cited 1×
“These techniques made manually specified features largely redundant and have greatly improved state-of-the-art in several real-world applications [1, 2, 3].”
BackgroundSelf-citationFaceNet: A unified embedding for face recognition and clustering
Schroff, F., Kalenichenko, D., & Philbin, J. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298682 · Cited by 15,267 · cited 1×
“For example, the techniques proposed by [21, 22, 23] were based on triplet losses using a max-margin approach to separate positive from negative examples.”
MethodLayer NormalizationPreprint
Ba, J., Kiros, J., & Hinton, G. E. (2016) · arXiv.org · doi:10.48550/arxiv.1607.06450 · Cited by 13,762 · cited 1×
“[41] is the same skip-thought model with layer normalization trained for 1M iterations. evaluate with 10-fold cross-validation for MR, CR, Subj, MPQA and use the train/test split for TREC.”
Method
Research works printed but not linked: 4
- [12][12] Michael Gutmann and Aapo Hyvärinen. Noise-contrastive estimation: A new estimation principle for unnormalized statistical models. In Proceedings of the Thirteenth International Conference on Artificial Intelligence and Statistics , pages 297-304, 2010. -
- [31][31] Daniel Povey, Arnab Ghoshal, Gilles Boulianne, Lukas Burget, Ondrej Glembek, Nagendra Goel, Mirko Hannemann, Petr Motlicek, Yanmin Qian, Petr Schwarz, et al. The kaldi speech recognition toolkit. In IEEE 2011 workshop on automatic speech recognition and understanding . IEEE Signal Processing Society, 2011. -
- [33][33] Laurens van der Maaten and Geoffrey Hinton. Visualizing data using t-sne. Journal of machine learning research , 9(Nov):2579-2605, 2008. -
- [48][48] Sida Wang and Christopher D. Manning. Baselines and bigrams: Simple, good sentiment and topic classification. In Proceedings of the 50th Annual Meeting of the Association for Computational Linguistics: Short Papers - Volume 2 , ACL '12, pages 90-94, 2012. -
Cited by
Sorted by citations
Items 601–700 of 1,066
Hallucination Augmented Contrastive Learning for Multimodal Large Language Model
Jiang, C., Xu, H., Dong, M., Chen, J., Ye, W., Yan, M., Ye, Q., Zhang, J., Huang, F., & Zhang, S. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.02553 · Cited by 140
MMA: Multi-Modal Adapter for Vision-Language Models
Yang, L., Zhang, R., Wang, Y., & Xie, X. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.02249 · Cited by 108
“For example, CLIP [50] and ALIGN [30] are trained with contrastive loss [47] from approximately 400 million and 1 billion image-text pairs, respectively.”
BackgroundMemory-augmented Dense Predictive Coding for Video Representation LearningPreprint
Han, T., Xie, W., & Zisserman, A. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2008.01065 · Cited by 80
Spatiotemporal Contrastive Video Representation LearningPreprint
Qian, R., Meng, T., Gong, B., et al. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2008.03800 · Cited by 80
Self-supervised Representation Learning for Astronomical Images
Hayat, M. A., Stein, G., Harrington, P., Lukić, Z., & Mustafa, M. (2021) · The Astrophysical Journal Letters · doi:10.3847/2041-8213/abf2c7 · Cited by 79
Putting An End to End-to-End: Gradient-Isolated Learning of Representations
Löwe, S., O'Connor, P., & Veeling, B. S. (2019) · UvA-DARE (University of Amsterdam) · doi:10.48550/arxiv.1905.11786 · Cited by 77
Learning Robust and Multilingual Speech Representations
Kawakami, K., Wang, L., Dyer, C., Blunsom, P., & van den Oord, A. (2020) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2020.findings-emnlp.106 · Cited by 102
Self-citationMean Shift for Self-Supervised Learning
Koohpayegani, S. A., Tejankar, A., & Pirsiavash, H. (2021) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.01016 · Cited by 79
Qu, L., Liu, S., Liu, X., Wang, M., & Song, Z. (2022) · Physics in Medicine and Biology · doi:10.1088/1361-6560/ac910a · Cited by 89
“Contrast predictive coding ( CPC ) ( Van den Oord et al 2018 ) is an early contrastive self-supervised method applied to natural image processing whose goal is to maximize the mutual information between patches ( positive samples ) from the same image and minimize the mutual information between…”
Temporal Feature Alignment and Mutual Information Maximization for Video-Based Human Pose Estimation
Liu, Z., Feng, R., Chen, H., Wu, S., Gao, Y., Gao, Y., & Wang, X. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.01073 · Cited by 85
Contrastive Learning Enhanced Intrusion Detection
Yue, Y., Chen, X., Han, Z., Zeng, X., & Zhu, Y. (2022) · IEEE Transactions on Network and Service Management · doi:10.1109/tnsm.2022.3218843 · Cited by 87
“Van den Oord et al. [35], Hnaff [36], took the context relationship as the prediction task and proposed the contrastive prediction coding model.”
BackgroundTransformer Fusion and Pixel-Level Contrastive Learning for RGB-D Salient Object Detection
Wu, J., Hao, F., Liang, W., & Xu, J. (2023) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2023.3275308 · Cited by 80
Self-Supervised Time Series Representation Learning via Cross Reconstruction Transformer
Zhang, W., Yang, L. Q., Geng, S., & Hong, S. (2023) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2023.3292066 · Cited by 106
Self-Supervised Deep Convolutional Neural Network for Chest X-Ray Classification
Gazda, M., Plavka, J., Gazda, J., & Drotár, P. (2021) · IEEE Access · doi:10.1109/access.2021.3125324 · Cited by 76
Zhou, Y., Xu, H., Zhang, W., Gao, B., & Heng, P. (2021) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00695 · Cited by 80
Fusing topology contexts and logical rules in language models for knowledge graph completion
Lin, Q., Mao, R., Liu, J., Xu, F., & Cambria, E. (2022) · Information Fusion · doi:10.1016/j.inffus.2022.09.020 · Cited by 76
Retrieval Augmented Classification for Long-Tail Visual Recognition
Long, A., Yin, W., Ajanthan, T., Nguyen, D.-V., Purkait, P., Garg, R., Blair, A., Shen, C., & van den Hengel, A. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.00683 · Cited by 97
Learning Robust Representations via Multi-View Information BottleneckPreprint
Federici, M., Dutta, A., Forré, P. D., Kushman, N., & Akata, Z. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2002.07017 · Cited by 223
Non-Autoregressive Predictive Coding for Learning Speech Representations from Local Dependencies
Liu, A. H., Chung, Y.-A., & Glass, J. R. (2021) · Interspeech (USB) · doi:10.21437/interspeech.2021-349 · Cited by 93
Cross, L., Cockburn, J., Yue, Y., & O’Doherty, J. P. (2020) · Neuron · doi:10.1016/j.neuron.2020.11.021 · Cited by 77
Contrastive Positive Sample Propagation Along the Audio-Visual Event Line
Zhou, J., Guo, D., & Wang, M. (2022) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2022.3223688 · Cited by 90
“of contrastive learning turns out to be an effective solution that is widely used in self-supervised learning [56], [57], [58], [59], [60] and various weakly supervised tasks [61],”
BackgroundContrastive Trajectory Similarity Learning with Dual-Feature Attention
Chang, Y., Qi, J., Liang, Y., & Tanin, E. (2023) · IEEE International Conference on Data Engineering (ICDE) · doi:10.1109/icde55515.2023.00224 · Cited by 96
“Following previous contrastive learning models, we use the InfoNCE [44] loss for model training.”
MethodInfluentialText Is MASS: Modeling as Stochastic Embedding for Text-Video Retrieval
Wang, J., Wang, P., Sun, G., et al. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.01566 · Cited by 77
ContIG: Self-supervised Multimodal Contrastive Learning for Medical Imaging with Genetics
Taleb, A., Kirchler, M., Monti, R., & Lippert, C. (2022) · 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52688.2022.02024 · Cited by 83
Unsupervised Automatic Speech Recognition: A review
Aldarmaki, H., Ullah, A., Ram, S., & Zaki, N. (2022) · Speech Communication · doi:10.1016/j.specom.2022.02.005 · Cited by 92
Exploring Task Difficulty for Few-Shot Relation Extraction
Han, J., Cheng, B. H., & Lu, W. (2021) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2021.emnlp-main.204 · Cited by 84
GPL: Generative Pseudo Labeling for Unsupervised Domain Adaptation of Dense Retrieval
Wang, K., Thakur, N., Reimers, N., & Gurevych, I. (2022) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/2022.naacl-main.168 · Cited by 83
On Learning Contrastive Representations for Learning with Noisy Labels
Li, Y., Liu, S., She, Q., McLeod, A. I., & Wang, B. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.01618 · Cited by 89
Self-citationMA-GCL: Model Augmentation Tricks for Graph Contrastive Learning
Gong, X., Yang, C. H., & Shi, C. (2023) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v37i4.25547 · Cited by 79
Zhang, J., Zhang, S., Shen, X., Lukasiewicz, T., & Xu, Z.-H. (2023) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2023.3290356 · Cited by 72
“For example, CPC [30], [31], which utilizes the idea of contrastive learning in the latent space, predicts the embedding of the next or adjacent sample.”
BackgroundMethodLocal Aggregation for Unsupervised Learning of Visual EmbeddingsPreprint
Zhuang, C., Zhai, A., & Yamins, D. L. K. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1903.12355 · Cited by 76
Autoregressive Unsupervised Image Segmentation
Ouali, Y., Hudelot, C., & Tami, M. (2020) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-58571-6_9 · Cited by 75
Self-Supervised Representation Learning for Wafer Bin Map Defect Pattern Classification
Kahng, H., & Kim, S. B. (2020) · IEEE Transactions on Semiconductor Manufacturing · doi:10.1109/tsm.2020.3038165 · Cited by 73
On Generative Spoken Language Modeling from Raw Audio
Lakhotia, K., Kharitonov, E., Hsu, W.-N., Adi, Y., Polyak, A., Bolte, B., Nguyen, T.-A., Copet, J., Baevski, A., Mohamed, A., et al. (2021) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00430 · Cited by 212
Self-Supervised 3D Hand Pose Estimation from monocular RGB via Contrastive Learning
Spurr, A., Dahiya, A., Wang, X., Zhang, X., & Hilliges, O. (2021) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.01104 · Cited by 82
OBoW: Online Bag-of-Visual-Words Generation for Self-Supervised Learning
Gidaris, S., Bursuc, A., Puy, G., Nikos Komodakis, A., Cord, M., & Pérez, P. (2021) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr46437.2021.00676 · Cited by 86
Combined scaling for zero-shot transfer learning
Pham, H., Dai, Z., Ghiasi, G., Kawaguchi, K., Liu, H., Yu, A. W., Yu, J., Chen, Y.-T., Luong, M.-T., Wu, Y., et al. (2023) · Neurocomputing · doi:10.1016/j.neucom.2023.126658 · Cited by 85
Sensor Data Augmentation by Resampling in Contrastive Learning for Human Activity Recognition
Wang, J., Zhu, T., Gan, J., Chen, L., Ning, H., & Wan, Y. (2022) · IEEE Sensors Journal · doi:10.1109/jsen.2022.3214198 · Cited by 82
Cross-modal Contrastive Learning for Speech Translation
Ye, R., Wang, M., & Li, L. (2022) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/2022.naacl-main.376 · Cited by 86
A survey on artificial intelligence in histopathology image analysis
Abdelsamea, M. M., Zidan, U., Senousy, Z., Gaber, M. M., Rakha, E. A., & Ilyas, M. (2022) · Wiley Interdisciplinary Reviews Data Mining and Knowledge Discovery · doi:10.1002/widm.1474 · Cited by 84
“Thus, the main direction in self-supervised learning has become what is called Contras-tive Learning (van den Oord et al., 2018).”
BackgroundA Closer Look at Few-shot Image Generation
Zhao, Y., Ding, H., Huang, H., & Cheung, N. M. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.00893 · Cited by 80
Selfie: Self-supervised Pretraining for Image EmbeddingPreprint
Trinh, T. H., Luong, M.-T., & Le, Q. V. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1906.02940 · Cited by 122
InfoXLM: An Information-Theoretic Framework for Cross-Lingual Language Model Pre-TrainingPreprint
Chi, Z., Dong, L., Wei, F., et al. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2007.07834 · Cited by 75
Self-Supervised Contrastive Learning for Unsupervised Phoneme Segmentation
Kreuk, F., Keshet, J., & Adi, Y. (2020) · Interspeech (USB) · doi:10.21437/interspeech.2020-2398 · Cited by 82
Contrastive Multimodal Fusion with TupleInfoNCE
Liu, Y., Fan, Q., Zhang, S., Dong, H., Funkhouser, T. A., & Yi, L. (2021) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00079 · Cited by 77
Improving Contrastive Learning by Visualizing Feature Transformation
Zhu, R., Zhao, B., Liu, J., Sun, Z., & Chen, C. W. (2021) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.01014 · Cited by 78
GATE: Graph CCA for Temporal Self-Supervised Learning for Label-Efficient fMRI Analysis
Peng, L., Wang, N., Xu, J., Zhu, X., & Li, X. (2023) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2022.3201974 · Cited by 75
A Practical Contrastive Learning Framework for Single-Image Super-Resolution
Wu, G., Jiang, J., & Liu, X. (2023) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2023.3290038 · Cited by 80
Huang, J., Yang, X., Zhou, F., et al. (2023) · Computer-Aided Civil and Infrastructure Engineering · doi:10.1111/mice.13042 · Cited by 74
“According to the contrastive loss function, that is, info noise comparative estimation proposed by Oord et al. (2018), the goal of network training is to keep q close to k + and away from any k − .”
BackgroundLiu, Z., Kang, X., & Ren, F. (2023) · IEEE/ACM Transactions on Audio Speech and Language Processing · doi:10.1109/taslp.2023.3282092 · Cited by 76
SELFormer: molecular representation learning via SELFIES language models
Yüksel, A., Ulusoy, E., Ünlü, A., & Doğan, T. (2023) · Machine Learning Science and Technology · doi:10.1088/2632-2153/acdb30 · Cited by 103
Detecting and Grounding Multi-Modal Media Manipulation
Shao, R., Wu, T., & Liu, Z. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.00667 · Cited by 98
“Following InfoNCE loss [31], we formulate image-to-text contrastive loss by: where τ is a temperature hyper-parameter, T − = { T − 1 , ..., T − K } is a set of negative text samples that are not matched to I as well as that belong to manipulated image-text pairs .”
MethodOmniMAE: Single Model Masked Pretraining on Images and Videos
Girdhar, R., El-Nouby, A., Singh, M., Alwala, K. V., Joulin, A., & Misra, I. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.01003 · Cited by 101
Sun, L., Lian, Z., Liu, B., & Tao, J. (2024) · Information Fusion · doi:10.1016/j.inffus.2024.102382 · Cited by 92
“After that, we utilize symmetric InfoNCE loss [62] for HCMCL.”
MethodSubject-Aware Contrastive Learning for Biosignals
Cheng, J. Y., Goh, H., Dogrusoz, K., Tuzel, O., & Azemi, E. (2020) · arXiv · doi:10.48550/arxiv.2007.04871 · Cited by 74
Longitudinal self-supervised learning
Zhao, Q., Liu, Z., Adeli, E., & Pohl, K. M. (2021) · Medical Image Analysis · doi:10.1016/j.media.2021.102051 · Cited by 71
XLM-E: Cross-lingual Language Model Pre-training via ELECTRA
Chi, Z., Huang, S., Dong, L., Ma, S., Zheng, B., Singhal, S., Bajaj, P., Song, X., Mao, X.-L., Huang, H., et al. (2022) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2022.acl-long.427 · Cited by 85
Un-mix: Rethinking Image Mixtures for Unsupervised Visual Representation Learning
Shen, Z., Liu, Z., Liu, Z., Savvides, M., Darrell, T., & Xing, E. P. (2022) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v36i2.20119 · Cited by 74
USCFormer: Unified Transformer With Semantically Contrastive Learning for Image Dehazing
Wang, Y., Xiong, J., Yan, X., & Wei, M. (2023) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2023.3277709 · Cited by 69
Hard Patches Mining for Masked Image Modeling
Wang, H., Song, K., Fan, J., Wang, Y., Xie, J., & Zhang, Z. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.01000 · Cited by 83
“Aiming at learning from data without any annotations, self-supervised learning (SSL) approaches have raised significant interest in computer vision, and how to design an appropriate pretext task becomes the crux [12, 41, 58, 71].”
BackgroundPRIOR: Prototype Representation Joint Learning from Medical Images and Reports
Cheng, P., Lin, L., Lyu, J., Huang, Y., Luo, W., & Tang, X. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.01953 · Cited by 93
“Specifically, we employ the InfoNCE loss [35] to estimate the lower bound of the mutual information.”
MethodLearning Invariant Representations for Reinforcement Learning without Reconstruction
Zhang, A., McAllister, R., Calandra, R., Gal, Y., & Levine, S. (2020) · arXiv · doi:10.48550/arxiv.2006.10742 · Cited by 165
Distilling Audio-Visual Knowledge by Compositional Contrastive Learning
Chen, Y., Xian, Y., Koepke, A. S., Shan, Y., & Akata, Z. (2021) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr46437.2021.00694 · Cited by 80
A Systematic Survey of Regularization and Normalization in GANs
Li, Z., Usman, M., Tao, R., Xia, P., Wang, C., Chen, H., & Li, B. (2022) · ACM Computing Surveys · doi:10.1145/3569928 · Cited by 80
Robust Contrastive Learning against Noisy Views
Chuang, C.-Y., Hjelm, R. D., Wang, X., Vineet, V., Joshi, N., Torralba, A. B., Jegelka, S., & Song, Y. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.01617 · Cited by 90
Facial Video-Based Remote Physiological Measurement via Self-Supervised Learning
Yue, Z.-J., Shi, M.-J., & Ding, S. (2022) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2023.3298650 · Cited by 77
“…Contrastive Loss: Since we do not have ground truth PPG signals for training, we use the popular contrastive self-supervised learning [34], [60] to pull close rPPG signals ( Y p = { y p } ) from positive samples while pushing them away Authorized licensed use limited to the terms of the…”
MethodOpen Vocabulary Semantic Segmentation with Patch Aligned Contrastive Learning
Mukhoti, J., Lin, T., Poursaeed, O., Wang, R., Shah, A., Torr, P. H. S., & Lim, S.-N. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.01860 · Cited by 90
“Grounded in Mutual Information: To understand how our compatibility function φ̂(x,y) works, we go back to the relation of the InfoNCE [38] loss with mutual information (MI).”
BackgroundMethodInfluentialNon-Contrastive Unsupervised Learning of Physiological Signals from Video
Speth, J., Vance, N., Flynn, P. J., & Czajka, A. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.01390 · Cited by 80
“[50] incorporated the previously ignored resampling factor for a soft modification to the InfoNCE loss [33] that scales the desired similarity between pairs by their relative sampling rate.”
Best of Both Worlds: Multimodal Contrastive Learning with Tabular and Imaging Data
Hager, P., Menten, M. J., & Rueckert, D. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.02291 · Cited by 96
“In contrast to the original InfoNCE [43] loss and following CLIP, we only contrast projections between modalities, never within one modality.”
MethodMultimodal Hate Speech Detection in Memes Using Contrastive Language-Image Pre-Training
Arya, G., Hasan, M. K., Bagwari, A., Safie, N., Islam, S., Ahmed, F. R. A., De, A., Khan, M. A., & Ghazal, T. M. (2024) · IEEE Access · doi:10.1109/access.2024.3361322 · Cited by 79
Pretraining with Contrastive Sentence Objectives Improves Discourse Performance of Language Models
Iter, D., Guu, K., Lansing, L., & Jurafsky, D. (2020) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2020.acl-main.439 · Cited by 66
GCCAD: Graph Contrastive Learning for Anomaly Detection
Chen, B., Zhang, J., Zhang, X., Dong, Y., Song, J., Zhang, P., Xu, K., Kharlamov, E., & Tang, J. (2022) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2022.3200459 · Cited by 76
Temporal Alignment Networks for Long-term Video
Han, T., Xie, W., & Zisserman, A. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.00292 · Cited by 72
Video-Text as Game Players: Hierarchical Banzhaf Interaction for Cross-Modal Representation Learning
Jin, P., Huang, J., Xiong, P., Tian, S., Liu, C., Ji, X., Li, Y., & Chen, J. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.00244 · Cited by 75
“Then the cross-modal contrastive loss [48] can be formulated as: where B is the batch size and τ is the temperature hyper-parameter.”
BackgroundSelf-citationUSER: Unified Semantic Enhancement With Momentum Contrast for Image-Text Retrieval
Zhang, Y., Ji, Z., Wang, D., Pang, Y., & Li, X. (2024) · IEEE Transactions on Image Processing · doi:10.1109/tip.2023.3348297 · Cited by 78
“For instance, Liu et al. [36] proposed a Hierarchical Transformer model for video-text retrieval task, which performs cross-modal contrastive matching and employs the InfoNCE loss [35] as its training objective.”
BackgroundMethodDiffusionRet: Generative Text-Video Retrieval with Diffusion Model
Jin, P., Li, H., Cheng, Z., Li, K., Ji, X., Liu, C., Li, Y., & Chen, J. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.00234 · Cited by 100
“To better understand the diffusion process, we show the additional visualization of the diffusion process in Fig.”
BackgroundMethodInfluentialSelf-citationRobust Pre-Training by Adversarial Contrastive LearningPreprint
Jiang, Z., Chen, T., Chen, T., & Wang, Z. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2010.13337 · Cited by 132
SEED: Self-supervised Distillation For Visual RepresentationPreprint
Fang, Z., Wang, J., Wang, L., Zhang, L., Yang, Y., & Liu, Z. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2101.04731 · Cited by 134
Rethinking Self-supervised Correspondence Learning: A Video Frame-level Similarity Perspective
Xu, J., & Wang, X. (2021) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00992 · Cited by 71
Contrastive Out-of-Distribution Detection for Pretrained Transformers
Zhou, W., Liu, F., & Chen, M. (2021) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2021.emnlp-main.84 · Cited by 99
Ye, H., Li, X., Yao, Y., & Tong, H. (2022) · ACM Transactions on Information Systems · doi:10.1145/3568396 · Cited by 73
“Therefore, we adopt the contrastive learning as an auxiliary task, and follow the InfoNCE loss [43] to optimize a lower bound of Mutual Information, which can be written as follows:”
MethodSupervised Contrastive Learning for RFF Identification With Limited Samples
Peng, Y., Hou, C., Zhang, Y., Lin, Y., Gui, G., Gacanin, H., Mao, S., & Adachi, F. (2023) · IEEE Internet of Things Journal · doi:10.1109/jiot.2023.3272628 · Cited by 72
“The supervised contrastive loss (SCL) in [47] can be regarded as an improved form of triplet loss, which is also a typical metric loss function.”
InfluentialZhang, J., Liu, Y., Ding, G., Tang, B., & Chen, Y. (2024) · IEEE Transactions on Information Forensics and Security · doi:10.1109/tifs.2024.3427361 · Cited by 74
Beyer, L., Hénaff, O. J., Kolesnikov, A., Zhai, X., & Oord, A. V. D. (2020) · arXiv · doi:10.48550/arxiv.2006.07159 · Cited by 176
Self-citationMasked Contrastive Representation Learning for Reinforcement Learning
Zhu, J., Xia, Y., Wu, L., Deng, J., Zhou, W., Qin, T., Liu, T., & Li, H. (2022) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2022.3176413 · Cited by 76
Chen, T., Frankle, J., Chang, S., Liu, S., Zhang, Y., Carbin, M., & Wang, Z. (2021) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr46437.2021.01604 · Cited by 101
Gu, R., Wang, G., Lu, J., Zhang, J., Lei, W., Chen, Y., Liao, W., Zhang, S., Li, K., Metaxas, D., et al. (2023) · Medical Image Analysis · doi:10.1016/j.media.2023.102904 · Cited by 80
OVTrack: Open-Vocabulary Multiple Object Tracking
Li, S., Fischer, T., Ke, L., Ding, H., Danelljan, M., & Yu, F. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.00539 · Cited by 82
“Inspired by recent progress in self-supervised representation learning [8, 24, 47], Fischer et al.”
ProtoCLIP: Prototypical Contrastive Language Image Pretraining
Chen, D., Zhao, W., Liu, F., Yang, Z., Zheng, S., Tan, Y., & Zhou, E. (2023) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2023.3335859 · Cited by 82
“It optimizes the information noise contrastive estimation (InfoNCE) objective [2] during pretraining, but how this simple objective derives meaningful image–text representations is not well studied.”
Sun, Z., & Li, X. (2024) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2024.3367910 · Cited by 71
“Contrastive learning, a widely adopted self-supervised learning technique in computer vision tasks, empowers deep learning models to map high-dimensional images or videos into lower-dimensional feature embeddings without the need for labeled data [39], [40], [41], [42], [43], [44], [45], [46], [47].”
BackgroundSelf-Supervised Learning with Data Augmentations Provably Isolates Content from StylePreprint
von Kügelgen, J., Sharma, Y., Gresele, L., Brendel, W., Schölkopf, B., Besserve, M., & Locatello, F. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2106.04619 · Cited by 104
Radulescu, A., Shin, Y. S., & Niv, Y. (2021) · Annual Review of Neuroscience · doi:10.1146/annurev-neuro-092920-120559 · Cited by 63
Unsupervised Representation Learning by Predicting Random Distances
Hu, W., Pang, G., Shen, C., & Ma, C. (2020) · International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2020/408 · Cited by 72
Temporal Context Aggregation for Video Retrieval with Contrastive Learning
Shao, J., Wen, X., Zhao, B., & Xue, X. (2021) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv48630.2021.00331 · Cited by 76
Dual Contradistinctive Generative Autoencoder
Parmar, G., Li, D., Lee, K., & Tu, Z. (2021) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr46437.2021.00088 · Cited by 71
Prototypical Contrast and Reverse Prediction: Unsupervised Skeleton Based Action Recognition
Xu, S., Rao, H., Hu, X., Cheng, J., & Hu, B. (2020) · IEEE transactions on multimedia · doi:10.1109/tmm.2021.3129616 · Cited by 77
Unified Speech-Text Pre-training for Speech Translation and Recognition
Tang, Y., Gong, H., Dong, N., Wang, C., Hsu, W.-N., Gu, J., Baevski, A., Li, X., Mohamed, A., Auli, M., et al. (2022) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2022.acl-long.105 · Cited by 74
MGDCF: Distance Learning via Markov Graph Diffusion for Neural Collaborative Filtering
Hu, J., Hooi, B., Qian, S., Fang, Q., & Xu, C. (2024) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2023.3348537 · Cited by 79
Wang, W., Li, C., Li, A., Li, F., Chen, J., & Zhang, T. (2023) · Knowledge-Based Systems · doi:10.1016/j.knosys.2023.110692 · Cited by 69
Zhang, J., Lin, L., & Liu, J. (2023) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v37i3.25451 · Cited by 71
Metadata
- Authors
- Oord, A. V. D., Li, Y., & Vinyals, O.
- Venue
- arXiv
- Publisher
- arXiv
- Published
- 2018
- ISSN
- 2331-8422 (ISSN-L)
- arXiv
- 1807.03748
- OpenAlex
- W4297808394
- Cited by
- 6,242
- References
- 50
- Citations by year
- 2026: 77 · 2025: 660 · 2024: 1288 · 2023: 1444 · 2022: 1043 · 2021: 1173 · 2020: 454 · 2019: 88 · 2018: 15
- Type
- posted content
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Domain Adaptation and Few-Shot Learning
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Physical Sciences › Computer Science › Artificial Intelligence › Advanced Graph Neural Networks
- Keywords
- contrastive predictive coding, unsupervised feature learning, autoregressive models, negative sampling, high-dimensional data, speech representation, image representation, text representation
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
