The DOI matches its DataCite record; 53 of the 53 references it cites were found in a registry or an index.
Representation Learning with Contrastive Predictive Coding
Open accessAbstract
While supervised learning has enabled great progress in many applications, unsupervised learning has not seen such widespread adoption, and remains an important and challenging endeavor for artificial intelligence. In this work, we propose a universal unsupervised learning approach to extract useful representations from high-dimensional data, which we call Contrastive Predictive Coding. The key insight of our model is to learn such representations by predicting the future in latent space by using powerful autoregressive models. We use a probabilistic contrastive loss which induces the latent space to capture information that is maximally useful to predict future samples. It also makes the model tractable by using negative sampling. While most prior work has focused on evaluating representations for a particular modality, we demonstrate that our approach is able to learn useful representations achieving strong performance on four distinct domains: speech, images, text and reinforcement learning in 3D environments.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“For example, Contrastive Predictive Coding (CPC) [80,143] utilizes a contrastive future prediction task, and it was found that predicting one or two timesteps in the future is (too) easy and results in poor performance, whereas predicting multiple timesteps into the future is much more complex, and leads to very effective representations.”
From Past, Present, and Future of Sensor-based Human Activity Recognition Using Wearables: A Surveying Tutorial on a Still Challenging Task (2025) · cites this paper as [143]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Using this contrastive loss, self-supervised networks learn to make the representations similar for positive pairs, and dissimilar for negative pairs, and hence are able to cluster similar (or positive) samples together and push apart dissimilar (or negative) samples.”
From Detecting galaxy tidal features using self-supervised representation learning (2024) · cites this paper as Oord et al. 2018
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“It means that InfoNCE loss maximizes the agreement of two different augmented sequences' representations from the same sequence while minimizing the agreement with other negative sequences.”
From Contrastive Learning from Extremely Augmented Skeleton Sequences for Self-Supervised Action Recognition (2022) · cites this paper as Oord, Li, and Vinyals 2018
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“We introduce the Contrastive Predictive Coding (CPC) framework to human activity recognition, which captures the long-term temporal structure of sensor data streams.”
From Contrastive Predictive Coding for Human Activity Recognition (2021) · cites this paper as [58]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“As mentioned in [58], either 𝑐 𝑡 (the output of 𝑔 𝑒𝑛𝑐 ) or 𝑧 𝑡 (the output of 𝑔 𝑎𝑟 ) can be utilized as the representation.”
From Contrastive Predictive Coding for Human Activity Recognition (2021) · cites this paper as [58]
Experimental — accuracy not yet measured
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“The framework for data augmentation and batch formation plays a critical role in multi-view similarity learning [12,16,44,115], where we start by following the popular SimCLR [16] method.”
From MARs: Multi-view Attention Regularizations for Patch-based Feature Recognition of Space Terrain (2024) · cites this paper as [16]
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“We note that this is a common setting for self-supervised contrastive learners with NCE objectives (Gutmann & Hyvärinen, 2010;Oord et al., 2018), including SimCLR (Chen et al., 2020) and SimCSE (Gao et al., 2021).”
From The Platonic Representation Hypothesis (2024) · cites this paper as Oord et al., 2018
The cited source says what the claim attributes to it.
In the citing paper
“The key insight behind this approach is that predicting just the next future timestep involves exploiting the local smoothness of the signal, whereas predicting multiple subsequent timesteps requires inferring the global structure of time-series data.”
From Contrastive Predictive Coding for Human Activity Recognition (2021) · cites this paper as [58]
The cited source says what the claim attributes to it.
In the citing paper
“Haresamudram et al. [80] adapted Contrastive Predictive Coding (CPC [143]) for HAR, emphasizing the importance of temporal structure in sensor data representation.”
From Past, Present, and Future of Sensor-based Human Activity Recognition Using Wearables: A Surveying Tutorial on a Still Challenging Task (2025) · cites this paper as [143]
The cited source says what the claim attributes to it.
In the citing paper
“Contrastive predictive coding (CPC) proposed by [109] provides a contrastive learning framework to perform the prediction task using InfoNCE loss.”
From Self-Supervised Learning for Time Series Analysis: Taxonomy, Progress, and Prospects (2024) · cites this paper as [109]
The cited source says what the claim attributes to it.
In the citing paper
“It can be seen that CPC does not directly predict future observations x t + k . Instead, it tries to preserve the mutual information of c t and x t + k .”
From Self-Supervised Learning for Time Series Analysis: Taxonomy, Progress, and Prospects (2024) · cites this paper as [109]
The cited source says what the claim attributes to it.
In the citing paper
“This contrastive loss, or InfoNCE loss (Oord et al. 2018), is minimised when the similarity is high for positive pairs and low for negative pairs.”
From Detecting galaxy tidal features using self-supervised representation learning (2024) · cites this paper as Oord et al. 2018
The cited source says what the claim attributes to it.
In the citing paper
“To evaluate the performance of our TS-TCC model, we follow the standard linear benchmarking evaluation scheme [Oord et al. , 2018; Chen et al. , 2020]. Particularly, we train a linear classifier (single MLP layer) on top of a frozen self-supervised pretrained encoder model.”
From Time-Series Representation Learning via Temporal and Contextual Contrasting (2021) · cites this paper as Oord et al. , 2018
The cited source says what the claim attributes to it.
In the citing paper
“Contrastive-based methods usually use a contrastive similarity metric, such as InfoNCE loss [109].”
From Self-Supervised Learning for Time Series Analysis: Taxonomy, Progress, and Prospects (2024) · cites this paper as [109]
The cited source says what the claim attributes to it.
In the citing paper
“This results in representations that encode the high-level information between temporally separated parts of the time-series signal [58], thereby leading to improved downstream recognition performance.”
From Contrastive Predictive Coding for Human Activity Recognition (2021) · cites this paper as [58]
The cited source says what the claim attributes to it.
In the citing paper
“Contrastive Predictive Coding (CPC) [58], which is the focus of this work, performs contrastive learning over multiple future timestep predictions.”
From Contrastive Predictive Coding for Human Activity Recognition (2021) · cites this paper as [58]
The cited source says what the claim attributes to it.
In the citing paper
“This framework has resulted in excellent representation learning performance across domains such as computer vision [12, 33, 35], natural language [49], speech recognition [6, 58] and time-series data [39].”
From Contrastive Predictive Coding for Human Activity Recognition (2021) · cites this paper as [58]
The cited source says what the claim attributes to it.
In the citing paper
“If extra context from the past is useful for classification, then 𝑐 𝑡 can be used; else 𝑧 𝑡 may be more effective.”
From Contrastive Predictive Coding for Human Activity Recognition (2021) · cites this paper as [58]
The check wasn’t confident either way, so it gives no verdict — never disputed.
In the citing paper
“SimCLR aims to maximize the learned representation similarity between augmented views of the same input.”
From MARs: Multi-view Attention Regularizations for Patch-based Feature Recognition of Space Terrain (2024) · cites this paper as [16]
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., & Polosukhin, I. (2017) · arXiv · doi:10.48550/arxiv.1706.03762 · Cited by 120,404 · cited 1×
“More recent advancements in autoregressive modeling such as masked convolutional architectures [18, 19] or self-attention networks [20] could help improve results further.”
BackgroundImageNet classification with deep convolutional neural networks
Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017) · Communications of the ACM · doi:10.1145/3065386 · Cited by 112,155 · cited 1×
“These techniques made manually specified features largely redundant and have greatly improved state-of-the-art in several real-world applications [1, 2, 3].”
MethodAdam: A Method for Stochastic Optimization
Kingma, D. P., & Ba, J. (2014) · arXiv · doi:10.48550/arxiv.1412.6980 · Cited by 98,946 · cited 1×
“We use the Adam optimizer [32] with a learning rate of 2e-4, and use 8 GPUs each with a minibatch of 8 examples from which the negative samples in the contrastive loss are drawn.”
MethodImageNet Large Scale Visual Recognition Challenge
Russakovsky, O., Deng, J., Su, H., Krause, J., Satheesh, S., Ma, S., Huang, Z., Karpathy, A., Khosla, A., Bernstein, M., et al. (2015) · International Journal of Computer Vision · doi:10.1007/s11263-015-0816-y · Cited by 39,380 · cited 1×
“In our visual representation experiments we use the ILSVRC ImageNet competition dataset [34].”
MethodBatch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
Ioffe, S., & Szegedy, C. (2015) · arXiv · doi:10.48550/arxiv.1502.03167 · Cited by 27,818 · cited 1×
“We did not use Batch-Norm [38].”
Efficient Estimation of Word Representations in Vector Space
Mikolov, T., Chen, K., Corrado, G., & Dean, J. (2013) · arXiv · doi:10.48550/arxiv.1301.3781 · Cited by 26,568 · cited 2×
“In Word2Vec [9] neighbouring words are predicted using a contrastive loss.”
BackgroundLearning Phrase Representations using RNN Encoder–Decoder for Statistical Machine Translation
Cho, K., van Merrienboer, B., Gulcehre, C., Bahdanau, D., Bougares, F., Schwenk, H., & Bengio, Y. (2014) · Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.3115/v1/d14-1179 · Cited by 25,545
Sequence to Sequence Learning with Neural Networks
Sutskever, I., Vinyals, O., & Le, Q. V. (2014) · arXiv · doi:10.48550/arxiv.1409.3215 · Cited by 17,766 · cited 1×
“These techniques made manually specified features largely redundant and have greatly improved state-of-the-art in several real-world applications [1, 2, 3].”
BackgroundSelf-citationFaceNet: A unified embedding for face recognition and clustering
Schroff, F., Kalenichenko, D., & Philbin, J. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298682 · Cited by 15,267 · cited 1×
“For example, the techniques proposed by [21, 22, 23] were based on triplet losses using a max-margin approach to separate positive from negative examples.”
MethodLayer NormalizationPreprint
Ba, J., Kiros, J., & Hinton, G. E. (2016) · arXiv.org · doi:10.48550/arxiv.1607.06450 · Cited by 13,762 · cited 1×
“[41] is the same skip-thought model with layer normalization trained for 1M iterations. evaluate with 10-fold cross-validation for MR, CR, Subj, MPQA and use the train/test split for TREC.”
Method
Research works printed but not linked: 4
- [12][12] Michael Gutmann and Aapo Hyvärinen. Noise-contrastive estimation: A new estimation principle for unnormalized statistical models. In Proceedings of the Thirteenth International Conference on Artificial Intelligence and Statistics , pages 297-304, 2010. -
- [31][31] Daniel Povey, Arnab Ghoshal, Gilles Boulianne, Lukas Burget, Ondrej Glembek, Nagendra Goel, Mirko Hannemann, Petr Motlicek, Yanmin Qian, Petr Schwarz, et al. The kaldi speech recognition toolkit. In IEEE 2011 workshop on automatic speech recognition and understanding . IEEE Signal Processing Society, 2011. -
- [33][33] Laurens van der Maaten and Geoffrey Hinton. Visualizing data using t-sne. Journal of machine learning research , 9(Nov):2579-2605, 2008. -
- [48][48] Sida Wang and Christopher D. Manning. Baselines and bigrams: Simple, good sentiment and topic classification. In Proceedings of the 50th Annual Meeting of the Association for Computational Linguistics: Short Papers - Volume 2 , ACL '12, pages 90-94, 2012. -
Cited by
Sorted by citations
Items 401–500 of 1,066
Social NCE: Contrastive Learning of Socially-aware Motion Representations
Liu, Y., Yan, Q., & Alahi, A. (2021) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.01484 · Cited by 118
Self-Supervised Learning of Remote Sensing Scene Representations Using Contrastive Multiview Coding
Stojnić, V., & Risojević, V. (2021) · IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw53098.2021.00129 · Cited by 125
Prototypical Graph Contrastive Learning
Lin, S., Liu, C., Zhou, P., Hu, Z.-Y., Wang, S., Zhao, R., Zheng, Y., Lin, L., Xing, E. P., & Liang, X. (2022) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2022.3191086 · Cited by 125
Wang, D., Deng, L., Yeung, Y. T., Chen, X. D., Liu, X., & Meng, H. M. L. (2021) · Interspeech (USB) · doi:10.21437/interspeech.2021-283 · Cited by 165
Multi-modal Self-Supervision from Generalized Data TransformationsPreprint
Patrick, M., Asano, Y. M., Кузнецова, П., Fong, R., Henriques, J. F., Zweig, G., & Vedaldi, A. (2021) · arXiv (Cornell University) · Cited by 117
Semantic Models for the First-Stage Retrieval: A Comprehensive Review
Guo, J., Cai, Y., Fan, Y., Sun, F., Zhang, R., & Cheng, X. (2022) · ACM Transactions on Information Systems · doi:10.1145/3486250 · Cited by 153
CODER: Knowledge-infused cross-lingual medical term embedding for term normalization
Yuan, Z. P., Zhao, Z., Sun, H., Li, J., Wang, F., & Yu, S. (2022) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2021.103983 · Cited by 122
Learning from Noisy Data with Robust Representation Learning
Li, J., Xiong, C., & Hoi, S. C. H. (2021) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00935 · Cited by 122
A Lightweight Multi-Scale Crossmodal Text-Image Retrieval Method in Remote Sensing
Yuan, Z., Zhang, W., Rong, X., Li, X., Chen, J., Wang, H., Fu, K., & Sun, X. (2021) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2021.3124252 · Cited by 117
Haugaard, R. L., & Buch, A. G. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.00663 · Cited by 141
Simple Unsupervised Graph Representation Learning
Mo, Y., Peng, L., Xu, J., Shi, X., & Zhu, X. (2022) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v36i7.20748 · Cited by 136
“For instance, Contrastive Predictive Coding (CPC) (van den Oord, Li, and Vinyals 2018) contrasts a summary of ordered local features to predict a local feature in the future while Deep InfoMax (DIM) (Devon et al. 2019) simultaneously contrasts a single global feature with all local features.”
BackgroundLi, M., Zhang, L., Cui, L., Bai, L., Li, Z., & Wu, X. (2023) · Pattern Recognition · doi:10.1016/j.patcog.2023.109874 · Cited by 118
MAGE: MAsked Generative Encoder to Unify Representation Learning and Image Synthesis
Li, T., Chang, H., Mishra, S. K., Zhang, H., Katabi, D., & Krishnan, D. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.00213 · Cited by 156
“Contrastive learning [8, 10, 44] has proven to be a competitive and systematic method to learn effective representations without human supervision, getting performance very close to that of supervised pre-training.”
BackgroundInstance Localization for Self-supervised Detection Pretraining
Yang, C., Wu, Z., Zhou, B., & Lin, S. C.-F. (2021) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr46437.2021.00398 · Cited by 126
Twin Contrastive Learning with Noisy Labels
Huang, Z., Zhang, J., & Shan, H. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.01122 · Cited by 135
Ma, C., & Wang, Z. (2024) · Knowledge-Based Systems · doi:10.1016/j.knosys.2024.112203 · Cited by 124
“Despite the promising performance of CNN-, ViT-, and Mamba-based segmentation networks, these networks typically require large, well-annotated datasets that are often difficult to obtain owing to high annotation costs.”
BackgroundA Comprehensive Study of Deep Video Action RecognitionPreprint
Zhu, Y., Li, X., Liu, C., Zolfaghari, M., Xiong, Y., Wu, C., Zhang, Z., Tighe, J., Manmatha, R., & Li, M. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2012.06567 · Cited by 212
Distilhubert: Speech Representation Learning by Layer-Wise Distillation of Hidden-Unit Bert
Chang, H.-J., Yang, S.-W., & Lee, H.-Y. (2022) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp43922.2022.9747490 · Cited by 125
Siriwardhana, S., Reis, A., Weerasekera, R., & Nanayakkara, S. C. (2020) · Interspeech (USB) · doi:10.21437/interspeech.2020-1212 · Cited by 137
Li, C., Tang, T., Wang, G., Peng, J., Wang, B., Liang, X., & Chang, X. (2021) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.01206 · Cited by 111
TimeCLR: A self-supervised contrastive learning framework for univariate time series representation
Yang, X., Zhang, Z., & Cui, R. (2022) · Knowledge-Based Systems · doi:10.1016/j.knosys.2022.108606 · Cited by 114
Represent, Compare, and Learn: A Similarity-Aware Framework for Class-Agnostic Counting
Shi, M., Lü, H., Chen, F., Liu, C., & Cao, Z. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.00931 · Cited by 126
Audio self-supervised learning: A survey
Liu, S., Mallol-Ragolta, A., Parada‐Cabaleiro, E., Qian, K., Jing, X., Kathan, A., Hu, B., & Schuller, B. W. (2022) · Patterns · doi:10.1016/j.patter.2022.100616 · Cited by 134
Hyperbolic Vision Transformers: Combining Improvements in Metric Learning
Ermolov, A., Mirvakhabova, L., Khrulkov, V., Sebe, N., & Oseledets, I. V. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.00726 · Cited by 139
Supervised Contrastive Learning
Khosla, P., Teterwak, P., Wang, C., Sarna, A., Tian, Y., Isola, P., Maschinot, A., Liu, C., & Krishnan, D. (2020) · arXiv · doi:10.48550/arxiv.2004.11362 · Cited by 3,569
Frozen Pretrained Transformers as Universal Computation Engines
Lu, K., Grover, A., Abbeel, P., & Mordatch, I. (2022) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v36i7.20729 · Cited by 113
“Linear probing is a common technique for evaluating the embeddings learned by an un-supervised model (Donahue, Kr¨ahenb¨uhl, and Darrell 2016; Oord, Li, and Vinyals 2018; Chen et al. 2020), which is reasonable when you finetune on the same modality as the pre-trained one.”
MethodA Comparison of Discrete and Soft Speech Units for Improved Voice Conversion
van Niekerk, B., Carbonneau, M., Zaïdi, J., Baas, M., Seuté, H., & Kamper, H. (2022) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp43922.2022.9746484 · Cited by 112
Self-supervised Vision Transformers for Land-cover Segmentation and Classification
Scheibenreif, L., Hanna, J., Mommert, M., & Borth, D. (2022) · IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw56347.2022.00148 · Cited by 131
Denoising Pretraining for Semantic Segmentation
Brempong, E. A., Kornblith, S., Chen, T., Parmar, N., Minderer, M., & Norouzi, M. (2022) · 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw56347.2022.00462 · Cited by 120
Contrastive Cross-Domain Sequential Recommendation
Cao, J., Cong, X., Sheng, J., Liu, T., & Wang, B. (2022) · ACM International Conference on Information and Knowledge Management (CIKM) · doi:10.1145/3511808.3557262 · Cited by 137
“In past years, several classical works have proposed to utilize contrastive learning for computer vision, neural language processing, and graph data, such as DIM [13], CPC [39] and DGI [40].”
BackgroundImage-to-Lidar Self-Supervised Distillation for Autonomous Driving Data
Sautier, C., Puy, G., Gidaris, S., Boulch, A., Bursuc, A., & Marlet, R. (2022) · Computer Vision and Pattern Recognition · doi:10.1109/cvpr52688.2022.00966 · Cited by 188
Space-Time Correspondence as a Contrastive Random WalkPreprint
Jabri, A., Owens, A., & Efros, A. A. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2006.14613 · Cited by 176
Visual Language Pretrained Multiple Instance Zero-Shot Transfer for Histopathology Images
Lu, M. Y., Chen, B., Zhang, A., Williamson, D. F. K., Chen, R. J., Ding, T., Le, L. P., Chuang, Y.-S., & Mahmood, F. (2023) · 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52729.2023.01893 · Cited by 122
QS-Attn: Query-Selected Attention for Contrastive Learning in I2I Translation
Hu, X., Zhou, X., Huang, Q., Shi, Z., Li, S., & Li, Q. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.01775 · Cited by 124
WildNet: Learning Domain Generalized Semantic Segmentation from the Wild
Lee, S., Seong, H., Lee, S., & Kim, E. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.00970 · Cited by 126
Self-supervised Video Representation Learning Using Inter-intra Contrastive Framework
Li, T., Wang, X., & Yamasaki, T. (2020) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3394171.3413694 · Cited by 108
Complementary Relation Contrastive Distillation
Zhu, J., Tang, S., Chen, D., Yu, S., Liu, Y., Rong, M., Yang, A., & Wang, X. (2021) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr46437.2021.00914 · Cited by 120
Seeing What You Said: Talking Face Generation Guided by a Lip Reading Expert
Wang, J., Qian, X., Zhang, M., Tan, R. T., & Li, H. (2023) · 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52729.2023.01408 · Cited by 116
“To achieve such a goal, we select infoNCE [42].”
BackgroundSelf-Supervised Learning of Point Clouds via Orientation Estimation
Poursaeed, O., Jiang, T., Qiao, H., Xu, N., & Kim, V. G. (2020) · International Conference on 3D Vision (3DV) · doi:10.1109/3dv50981.2020.00112 · Cited by 113
Multi-View Intent Disentangle Graph Networks for Bundle Recommendation
Zhao, S., Wei, W., Zou, D., et al. (2022) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v36i4.20359 · Cited by 106
Knowledge Enhancement for Contrastive Multi-Behavior Recommendation
Xuan, H., Liu, Y., Li, B., & Yin, H. (2023) · ACM International Conference on Web Search and Data Mining (WSDM) · doi:10.1145/3539597.3570386 · Cited by 113
“According to work [15], we define a self-supervised contrastive loss based on InfoNCE loss as follows:”
MethodFew-Shot Learning Meets Transformer: Unified Query-Support Transformers for Few-Shot Classification
Wang, X., Wang, X., Jiang, B., & Luo, B. (2023) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2023.3282777 · Cited by 111
“First, for the sample-Former module, we specifically introduce a contrastive loss as suggested in work [72], [73], which encourages the positive query-support sample pairs with same label (i.e., Y s = Y q ) to be closing and the negative query-support sample pairs with different labels (i.e., Y s…”
MethodEgo-Exo4D: Understanding Skilled Human Activity from First- and Third-Person Perspectives
Grauman, K. L., Westbury, A., Torresani, L., Kitani, K., Malik, J., Afouras, T., Ashutosh, K., Baiyya, V. B., Bansal, S., Boote, B., et al. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.01834 · Cited by 267
A Mutual Information Maximization Perspective of Language Representation LearningPreprint
Kong, L., d’Autume, C. D. M., Wang, L., Yu, L., Dai, Z., & Yogatama, D. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1910.08350 · Cited by 129
3D Infomax improves GNNs for Molecular Property PredictionPreprint
Stärk, H., Beaini, D., Corso, G., Tossou, P., Dallago, C., Günnemann, S., & Lió, P. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2110.04126 · Cited by 124
AVLnet: Learning Audio-Visual Language Representations from Instructional Videos
Rouditchenko, A., Boggust, A., Harwath, D. F., Chen, B., Joshi, D., Thomas, S., Audhkhasi, K., Kuehne, H., Panda, R., Feris, R., et al. (2021) · Interspeech (USB) · doi:10.21437/interspeech.2021-1312 · Cited by 144
CLEVE: Contrastive Pre-training for Event Extraction
Wang, Z., Wang, X., Han, X., Lin, Y., Hou, L., Liu, Z., Li, P., Li, J., & Zhou, J. (2021) · Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/2021.acl-long.491 · Cited by 126
Ma, Y., He, Y., Zhang, A., Wang, X., & Chua, T. (2022) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/3534678.3539229 · Cited by 126
Online Knowledge Distillation via Mutual Contrastive Learning for Visual Recognition
Yang, C., An, Z., Zhou, H., Zhuang, F., Xu, Y., & Zhang, Q. (2023) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2023.3257878 · Cited by 119
Selective Structured State-Spaces for Long-Form Video Understanding
Wang, J., Zhu, W., Wang, P., Yu, X., Liu, L. L., Omar, M., & Hamid, R. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.00618 · Cited by 131
“Similarly, the LSMCL adoptes similar objective as the InfoNCE [51]: where ρ is the temperature hyperparameter.”
MethodSurgicalSAM: Efficient Class Promptable Surgical Instrument Segmentation
Yue, W., Zhang, J., Hu, K. Y., Xia, Y., Luo, J., & Wang, Z. (2024) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v38i7.28514 · Cited by 117
Learning Speaker Representations with Mutual Information
Ravanelli, M., & Bengio, Y. (2019) · Interspeech (USB) · doi:10.21437/interspeech.2019-2380 · Cited by 90
Kang, J. S., Fernández-Beltrán, R., Duan, P., Liu, S., & Plaza, A. (2020) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2020.3007029 · Cited by 109
Predictive Coding: a Theoretical and Experimental Review
Millidge, B., Seth, A., & Buckley, C. L. (2021) · arXiv · doi:10.48550/arxiv.2107.12979 · Cited by 106
When Does Contrastive Visual Representation Learning Work?
Cole, E., Yang, X., Wilber, K., Aodha, O. M., & Belongie, S. (2022) · 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52688.2022.01434 · Cited by 105
Self-Supervised Learning for Few-Shot Image Classification
Chen, D., Chen, Y., Li, Y., Mao, F., He, Y., & Xue, H. (2021) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp39728.2021.9413783 · Cited by 136
Self-citationBaek, J., Matsui, Y., & Aizawa, K. (2021) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr46437.2021.00313 · Cited by 109
Efficient Non-local Contrastive Attention for Image Super-resolution
Xia, B., Hang, Y., Tian, Y., Yang, W., Liao, Q., & Zhou, J. N. (2022) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v36i3.20179 · Cited by 115
Remote photoplethysmography for heart rate measurement: A review
Xiao, H., Liu, T., Sun, Y., Li, Y., Zhao, S., & Avolio, A. P. (2023) · Biomedical Signal Processing and Control · doi:10.1016/j.bspc.2023.105608 · Cited by 120
Self-citationLearning and Evaluating Representations for Deep One-class ClassificationPreprint
Sohn, K., Li, C., Yoon, J., Jin, M., & Pfister, T. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2011.02578 · Cited by 162
SeCo: Exploring Sequence Supervision for Unsupervised Representation Learning
Yao, T., Zhang, Y., Qiu, Z., Pan, Y., & Mei, T. (2021) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v35i12.17274 · Cited by 106
Contrastive Cross-domain Recommendation in Matching
Xie, R., Liu, Q., Wang, L., Liu, S., Zhang, B., & Lin, L. (2022) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/3534678.3539125 · Cited by 112
Li, M., Xie, Y., Shen, Y., Ke, B., Qiao, R., Ren, B., Lin, S., & Ma, L. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.01451 · Cited by 113
CMID: A Unified Self-Supervised Learning Framework for Remote Sensing Image Understanding
Muhtar, D., Zhang, X., Xiao, P., Li, Z., & Gu, F. (2023) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2023.3268232 · Cited by 122
“Finally, we apply infoNCE loss [44] to align their representations by discriminating them from other images”
MethodAssessing the State of Self-Supervised Human Activity Recognition Using Wearables
Haresamudram, H., Essa, I., & Plötz, T. (2022) · Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies · doi:10.1145/3550299 · Cited by 107
Nearest Neighbor Matching for Deep Clustering
Dang, Z., Deng, C. D., Yang, X., Wei, K., & Huang, H. (2021) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr46437.2021.01348 · Cited by 120
CLIP-Art: Contrastive Pre-training for Fine-Grained Art Classification
Conde, M. V., & Turgutlu, K. (2021) · IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw53098.2021.00444 · Cited by 111
Voice-Face Homogeneity Tells Deepfake
Cheng, H. H., Guo, Y., Wang, T., Li, Q., Chang, X., & Nie, L. (2023) · ACM Transactions on Multimedia Computing Communications and Applications · doi:10.1145/3625231 · Cited by 136
TMR: Text-to-Motion Retrieval Using Contrastive 3D Human Motion Synthesis
Petrovich, M., Black, M. J., & Varol, G. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.00870 · Cited by 139
“In contrast to Guo et al. [15], who consider one random negative per batch (and use a margin loss), we adopt the more recent formulation of InfoNCE [34], which was proven effective in other work [7, 26, 39].”
BackgroundMethodInfluentialSelf-Supervised Remote Sensing Feature Learning: Learning Paradigms, Challenges, and Future Works
Tao, C., Qi, J., Guo, M., Zhu, Q., & Li, H. (2023) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2023.3276853 · Cited by 116
“[147], [148], [149], [150], [151], [152], [153], [154] takes different augmented views of the same image as positive samples and the views of any other images as negative”
BackgroundRevisiting Classifier: Transferring Vision-Language Models for Video Recognition
Wu, W., Sun, Z., & Ouyang, W. (2023) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v37i3.25386 · Cited by 120
“2 is not the rigorous form of the Noise-Contrastive Estimation (NCE) loss proposed in (Van den Oord, Li, and Vinyals 2018).”
BackgroundImproving Transformer-based Speech Recognition Using Unsupervised Pre-trainingPreprint
Jiang, D., Lei, X., Li, W., Luo, N., Hu, Y., Zou, W., & Li, X. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1910.09932 · Cited by 94
Effectiveness of Self-Supervised Pre-Training for ASR
Baevski, A., & Mohamed, A. (2020) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp40776.2020.9054224 · Cited by 110
Rao, H., Wang, S., Hu, X., Tan, M., Guo, Y., Cheng, J. S., Liu, X., & Hu, B. (2021) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2021.3092833 · Cited by 104
Self-Supervised Pretraining Improves Self-Supervised Pretraining
Reed, C., Yue, X., Nrusimha, A., Ebrahimi, S., Vijaykumar, V., Mao, R., Li, B., Zhang, S., Guillory, D., Metzger, S. L., et al. (2022) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv51458.2022.00112 · Cited by 116
Generalization in Reinforcement Learning by Soft Data Augmentation
Hansen, N., & Wang, X. (2021) · 2021 IEEE International Conference on Robotics and Automation (ICRA) · doi:10.1109/icra48506.2021.9561103 · Cited by 111
Time Series Contrastive Learning with Information-Aware Augmentations
Luo, D., Cheng, W. D., Wang, Y., Xu, D., Ni, J., Yu, W., Zhang, X., Liu, Y., Chen, Y., Chen, H., et al. (2023) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v37i4.25575 · Cited by 113
Rethinking Domain Generalization for Face Anti-spoofing: Separability and Alignment
Sun, Y., Liu, Y., Liu, X., Li, Y., & Chu, W. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.02353 · Cited by 111
Self-citationDomain Generalization for Medical Image Analysis: A Review
Yoon, J. S., Oh, K., Shin, Y., Mazurowski, M. A., & Suk, H. (2024) · Proceedings of the IEEE · doi:10.1109/jproc.2024.3507831 · Cited by 153
“A typical contrastive learning loss function [109] is defined as where sim( · , · ) is a function for cosine similarity, x i and x j are positive pairs, k ∈ a ( i ) are the indexes of selected negative samples, and x i and x k are the negative pairs.”
Self-Supervised Video Forensics by Audio-Visual Anomaly Detection
Feng, C., Chen, Z., & Owens, A. (2023) · 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52729.2023.01011 · Cited by 101
Learning Representations by Predicting Bags of Visual Words
Gidaris, S., Bursuc, A., Komodakis, N., Pérez, P., & Cord, M. (2020) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr42600.2020.00696 · Cited by 101
Semi-Supervised Action Recognition with Temporal Contrastive Learning
Singh, A., Chakraborty, O., Varshney, A., Panda, R., Feris, R., Saenko, K., & Das, A. (2021) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr46437.2021.01025 · Cited by 99
Rich features for perceptual quality assessment of UGC videos
Wang, Y., Ke, J., Talebi, H., Yim, J. G., Birkbeck, N., Adsumilli, B., Milanfar, P., & Yang, F. (2021) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr46437.2021.01323 · Cited by 105
Zhou, H.-Y., Lu, C., Yang, S., Han, X., & Yu, Y. (2021) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00348 · Cited by 105
On Feature Decorrelation in Self-Supervised Learning
Hua, T., Wang, W., Xue, Z., Ren, S., Wang, Y., & Zhao, H. (2021) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00946 · Cited by 137
Global and Local Mixture Consistency Cumulative Learning for Long-tailed Visual Recognitions
Du, F., Yang, P., Jia, Q., Nan, F., Chen, X., & Yang, Y. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.01518 · Cited by 114
Self-Supervised Multimodal Learning: A Survey
Zong, Y., Aodha, O. M., & Hospedales, T. M. (2024) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2024.3429301 · Cited by 165
“For instance, Contrastive Forward Modeling [158] combines contrastive predictive coding [159] with predicting future state to maximize the mutual information between predictions and positives.”
Decoupling the Role of Data, Attention, and Losses in Multimodal Transformers
Hendricks, L. A., Mellor, J., Schneider, R., Alayrac, J.-B., & Nematzadeh, A. (2021) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00385 · Cited by 100
A Survey of Self-Supervised and Few-Shot Object Detection
Huang, G., Laradji, I., Vazquez, D., Lacoste-Julien, S., & Rodriguez, P. (2022) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2022.3199617 · Cited by 100
Effectiveness of self-supervised pre-training for speech recognitionPreprint
Baevski, A., Auli, M., & Mohamed, A. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1911.03912 · Cited by 142
Beyond the feedforward sweep: feedback computations in the visual cortex
Kreiman, G., & Serre, T. (2020) · Annals of the New York Academy of Sciences · doi:10.1111/nyas.14320 · Cited by 96
Federated Self-Supervised Learning of Multisensor Representations for Embedded Intelligence
Saeed, A., Salim, F. D., Özçelebi, T., & Lukkien, J. J. (2024) · Figshare · Cited by 92
RSPNet: Relative Speed Perception for Unsupervised Video Representation Learning
Chen, P., Huang, D. Y., He, D., Long, X., Zeng, R., Wen, S., Tan, M., & Gan, C. (2021) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v35i2.16189 · Cited by 108
Adversarial and Contrastive Variational Autoencoder for Sequential Recommendation
Xie, Z., Liu, C., Zhang, Y., et al. (2021) · doi:10.1145/3442381.3449873 · Cited by 98
Zhang, Y., Feng, K., Ji, J., Yu, K., Ren, Z., & Liu, Z. (2022) · IEEE/ASME Transactions on Mechatronics · doi:10.1109/tmech.2022.3218771 · Cited by 99
“The MI between the last prediction token and the input sample can be estimated by [44]”
BackgroundInstance Relation Graph Guided Source-Free Domain Adaptive Object Detection
Vibashan, V. S., Oza, P., & Patel, V. M. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.00343 · Cited by 120
Vector-Quantized Autoregressive Predictive Coding
Chung, Y.-A., Tang, H., & Glass, J. (2020) · Interspeech 2020 · doi:10.21437/interspeech.2020-1228 · Cited by 97
Solving Mixed-Modal Jigsaw Puzzle for Fine-Grained Sketch-Based Image Retrieval
Pang, K., Yang, Y., Hospedales, T. M., Xiang, T., & Song, Y.-Z. (2020) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr42600.2020.01036 · Cited by 95
Foreground-Action Consistency Network for Weakly Supervised Temporal Action Localization
Huang, L., Wang, L., & Li, H. (2021) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00790 · Cited by 94
SELF-LABELLING VIA SIMULTANEOUS CLUSTERING AND REPRESENTATION LEARNINGPreprint
Asano, Y. M., Rupprecht, C., & Vedaldi, A. (2019) · Oxford University Research Archive (ORA) (University of Oxford) · doi:10.48550/arxiv.1911.05371 · Cited by 523
Metadata
- Authors
- Oord, A. V. D., Li, Y., & Vinyals, O.
- Venue
- arXiv
- Publisher
- arXiv
- Published
- 2018
- ISSN
- 2331-8422 (ISSN-L)
- arXiv
- 1807.03748
- OpenAlex
- W4297808394
- Cited by
- 6,242
- References
- 50
- Citations by year
- 2026: 77 · 2025: 660 · 2024: 1288 · 2023: 1444 · 2022: 1043 · 2021: 1173 · 2020: 454 · 2019: 88 · 2018: 15
- Type
- posted content
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Domain Adaptation and Few-Shot Learning
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Physical Sciences › Computer Science › Artificial Intelligence › Advanced Graph Neural Networks
- Keywords
- contrastive predictive coding, unsupervised feature learning, autoregressive models, negative sampling, high-dimensional data, speech representation, image representation, text representation
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
