Contrasting Contrastive Self-Supervised Representation Learning Pipelines
Open accessAbstract
In the past few years, we have witnessed remarkable breakthroughs in self-supervised representation learning. Despite the success and adoption of representations learned through this paradigm, much is yet to be understood about how different training methods and datasets influence performance on downstream tasks. In this paper, we analyze contrastive approaches as one of the most successful and popular variants of self-supervised representation learning. We perform this analysis from the perspective of the training algorithms, pre-training datasets and end tasks. We examine over 700 training experiments including 30 encoders, 4 pre-training datasets and 20 diverse downstream tasks. Our experiments address various questions regarding the performance of self-supervised models compared to their supervised counterparts, current benchmarks used for evaluation, and the effect of the pre-training data on end task performance. Our Visual Representation Benchmark (ViRB) is available at: https://github.com/allenai/virb.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Performance on the ImageNet and downstream datasets is not always consistent due to the inconstancy of data distribution, data domain, and task objective.
“However, recent research [112], [113] suggested that performance on the ImageNet and downstream datasets is not always consistent due to the inconstancy of data distribution, data domain, and task objective.”
From Large-Scale Unsupervised Semantic Segmentation (2023) · cites this paper as [113]
Cites
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474 · see citing works
ImageNet classification with deep convolutional neural networks
Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017) · Communications of the ACM · doi:10.1145/3065386 · Cited by 112,155 · see citing works
U-Net: Convolutional Networks for Biomedical Image Segmentation
Ronneberger, O., Fischer, P., & Brox, T. (2015) · Lecture notes in computer science · doi:10.1007/978-3-319-24574-4_28 · Cited by 99,404 · see citing works
Very Deep Convolutional Networks for Large-Scale Image Recognition
Simonyan, K., & Zisserman, A. (2014) · arXiv · doi:10.48550/arxiv.1409.1556 · Cited by 74,248 · see citing works
ImageNet: A large-scale hierarchical image database
Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., & Fei-Fei, L. (2009) · 2009 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2009.5206848 · Cited by 65,333 · see citing works
PyTorch: An Imperative Style, High-Performance Deep Learning Library
Paszke, A., Gross, S., Massa, F., Lerer, A., Bradbury, J., Chanan, G., Killeen, T., Lin, Z., Gimelshein, N., Antiga, L., et al. (2019) · arXiv · doi:10.48550/arxiv.1912.01703 · Cited by 31,473 · see citing works
Learning Multiple Layers of Features from Tiny Images
Krizhevsky, A. (2024) · doi:10.57702/zp44cu3g · Cited by 23,890 · see citing works
Encoder-Decoder with Atrous Separable Convolution for Semantic Image Segmentation
Chen, L.-C., Zhu, Y., Papandreou, G., Schroff, F., & Adam, H. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-01234-2_49 · Cited by 20,054 · see citing works
Are we ready for autonomous driving? The KITTI vision benchmark suite
Geiger, A., Lenz, P., & Urtasun, R. (2012) · 2012 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2012.6248074 · Cited by 14,897 · see citing works
Momentum Contrast for Unsupervised Visual Representation Learning
He, K., Fan, H., Wu, Y., Xie, S., & Girshick, R. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.00975 · Cited by 13,347 · see citing works
The Cityscapes Dataset for Semantic Urban Scene Understanding
Cordts, M., Omran, M., Ramos, S., Rehfeld, T., Enzweiler, M., Benenson, R., Franke, U., Roth, S., & Schiele, B. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.350 · Cited by 12,433 · see citing works
A simple framework for contrastive learning of visual representations
Chen, T., Kornblith, S., Norouzi, M., & Hinton, G. (2020) · arXiv · doi:10.48550/arxiv.2002.05709 · Cited by 10,857 · see citing works
Shi, W., Caballero, J., Huszar, F., Totz, J., Aitken, A. P., Bishop, R., Rueckert, D., & Wang, Z. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.207 · Cited by 7,570 · see citing works
nuScenes: A Multimodal Dataset for Autonomous Driving
Caesar, H., Bankiti, V., Lang, A. H., Vora, S., Liong, V. E., Xu, Q., Krishnan, A., Pan, Y., Baldan, G., & Beijbom, O. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.01164 · Cited by 7,152 · see citing works
Representation Learning with Contrastive Predictive Coding
Oord, A. V. D., Li, Y., & Vinyals, O. (2018) · arXiv · doi:10.48550/arxiv.1807.03748 · Cited by 6,242 · see citing works
Indoor segmentation and support inference from rgbd images
Silberman, N., Hoiem, D., Kohli, P., & Fergus, R. (2012) · Lecture Notes in Computer Science · doi:10.1007/978-3-642-33715-4_54 · Cited by 5,760 · see citing works
Context Encoders: Feature Learning by Inpainting
Pathak, D., Krahenbuhl, P., Donahue, J., Darrell, T., & Efros, A. A. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.278 · Cited by 5,691 · see citing works
Places: A 10 Million Image Database for Scene Recognition
Zhou, B., Lapedriza, A., Khosla, A., Oliva, A., & Torralba, A. (2018) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2017.2723009 · Cited by 4,042 · see citing works
Bootstrap Your Own Latent: A New Approach to Self-Supervised Learning
Grill, J.-B., Strub, F., Altché, F., Tallec, C., Richemond, P. H., Buchatskaya, E., Doersch, C., Pires, B. A., Guo, Z. D., Azar, M. G., et al. (2020) · arXiv · doi:10.48550/arxiv.2006.07733 · Cited by 3,846 · see citing works
Zhang, R., Isola, P., & Efros, A. A. (2016) · European Conference on Computer Vision · doi:10.1007/978-3-319-46487-9_40 · Cited by 3,803 · see citing works
Exploring Simple Siamese Representation Learning
Chen, X., & He, K. (2021) · 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr46437.2021.01549 · Cited by 3,725 · see citing works
The Kinetics Human Action Video DatasetPreprint
Kay, W., Carreira, J., Simonyan, K., Zhang, B., Hillier, C., Vijayanarasimhan, S., Viola, F., Green, T., Back, T., Natsev, P., et al. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1705.06950 · Cited by 3,711 · see citing works
SUN database: Large-scale scene recognition from abbey to zoo
Xiao, J., Hays, J., Ehinger, K. A., Oliva, A., & Torralba, A. (2010) · 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2010.5539970 · Cited by 3,283 · see citing works
Unsupervised Learning of Visual Representations by Solving Jigsaw Puzzles
Noroozi, M., & Favaro, P. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-46466-4_5 · Cited by 2,775 · see citing works
Unsupervised Visual Representation Learning by Context Prediction
Doersch, C., Gupta, A., & Efros, A. A. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.167 · Cited by 2,629 · see citing works
Improved Baselines with Momentum Contrastive LearningPreprint
Chen, X., Fan, H., Girshick, R., & He, K. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2003.04297 · Cited by 2,579 · see citing works
Fei-Fei, L., Fergus, R., & Perona, P. (2005) · IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvpr.2004.383 · Cited by 2,513 · see citing works
Unsupervised Learning of Visual Features by Contrasting Cluster Assignments
Caron, M., Misra, I., Mairal, J., Goyal, P., Bojanowski, P., & Joulin, A. (2020) · arXiv · doi:10.48550/arxiv.2006.09882 · Cited by 2,499 · see citing works
Unsupervised Representation Learning by Predicting Image Rotations
Gidaris, S., Singh, P., & Komodakis, N. (2018) · arXiv · doi:10.48550/arxiv.1803.07728 · Cited by 2,477 · see citing works
Describing Textures in the Wild
Cimpoi, M., Maji, S., Kokkinos, I., Mohamed, S., & Vedaldi, A. (2014) · 2014 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2014.461 · Cited by 2,438 · see citing works
Learning deep representations by mutual information estimation and maximization
Hjelm, R. D., Fedorov, A., Lavoie-Marchildon, S., Grewal, K., Bachman, P., Trischler, A., & Bengio, Y. (2018) · arXiv · doi:10.48550/arxiv.1808.06670 · Cited by 1,991 · see citing works
Adversarial Feature LearningPreprint
Donahue, J., Krähenbühl, P., & Darrell, T. (2016) · International Conference on Learning Representations · doi:10.48550/arxiv.1605.09782 · Cited by 1,982 · see citing works
EuroSAT: A Novel Dataset and Deep Learning Benchmark for Land Use and Land Cover Classification
Helber, P., Bischke, B., Dengel, A., & Borth, D. (2019) · IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing · doi:10.1109/jstars.2019.2918242 · Cited by 1,951 · see citing works
CLEVR: A Diagnostic Dataset for Compositional Language and Elementary Visual Reasoning
Johnson, J., Hariharan, B., van der Maaten, L., Fei-Fei, L., Zitnick, C. L., & Girshick, R. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.215 · Cited by 1,781 · see citing works
The kinetics human action video dataset
Kay, W. (2024) · TIB Data Manager · doi:10.57702/jtuxnt5m · Cited by 1,646 · see citing works
Parkhi, O., Vedaldi, A., Zisserman, A., & Jawahar, C. V. (2012) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2012.6248092 · Cited by 1,563 · see citing works
Big Self-Supervised Models are Strong Semi-Supervised LearnersPreprint
Chen, T., Kornblith, S., Swersky, K., Norouzi, M., & Hinton, G. E. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2006.10029 · Cited by 1,480 · see citing works
Self-Supervised Learning of Pretext-Invariant Representations
Misra, I., & van der Maaten, L. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.00674 · Cited by 1,435 · see citing works
Do Better ImageNet Models Transfer Better?
Kornblith, S., Shlens, J., & Le, Q. V. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00277 · Cited by 1,325 · see citing works
Exploring the Limits of Weakly Supervised Pretraining
Mahajan, D., Girshick, R., Ramanathan, V., He, K., Paluri, M., Li, Y., Bharambe, A., & van der Maaten, L. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-01216-8_12 · Cited by 1,224 · see citing works
Large-Scale Long-Tailed Recognition in an Open World
Liu, Z., Miao, Z., Zhan, X., Wang, J., Gong, B., & Yu, S. X. (2019) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2019.00264 · Cited by 1,169 · see citing works
A simple framework for contrastive learning of visual representations
Chen, T. (2024) · TIB Data Manager · doi:10.57702/iuuvgtaz · Cited by 1,140 · see citing works
Taskonomy: Disentangling Task Transfer Learning
Zamir, A. R., Sax, A., Shen, W., Guibas, L., Malik, J., & Savarese, S. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00391 · Cited by 1,128 · see citing works
Unsupervised Learning of Visual Representations Using Videos
Wang, X., & Gupta, A. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.320 · Cited by 1,005 · see citing works
Do ImageNet Classifiers Generalize to ImageNet?
Recht, B., Roelofs, R., Schmidt, L., & Shankar, V. (2019) · arXiv · doi:10.48550/arxiv.1902.10811 · Cited by 790 · see citing works
Self-citationWhat makes for good views for contrastive learning
Tian, Y., Sun, C., Poole, B., Krishnan, D., Schmid, C., & Isola, P. (2020) · arXiv · doi:10.48550/arxiv.2005.10243 · Cited by 773 · see citing works
Split-Brain Autoencoders: Unsupervised Learning by Cross-Channel Prediction
Zhang, R., Isola, P., & Efros, A. A. (2016) · Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2017.76 · Cited by 724 · see citing works
Similarity of Neural Network Representations Revisited
Kornblith, S., Norouzi, M., Lee, H., & Hinton, G. (2019) · arXiv · doi:10.48550/arxiv.1905.00414 · Cited by 724 · see citing works
Agrawal, P., Carreira, J., & Malik, J. (2015) · IEEE International Conference on Computer Vision · doi:10.1109/iccv.2015.13 · Cited by 639 · see citing works
AI2-THOR: An Interactive 3D Environment for Visual AI
Kolve, E., Mottaghi, R., Han, W., VanderBilt, E., Weihs, L., Herrasti, A., Deitke, M., Ehsani, K., Gordon, D., Zhu, Y., et al. (2017) · arXiv · doi:10.48550/arxiv.1712.05474 · Cited by 578 · see citing works
Self-citationLearning Features by Watching Objects Move
Pathak, D., Girshick, R. B., Dollár, P., Darrell, T., & Hariharan, B. (2016) · Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2017.638 · Cited by 549 · see citing works
Anticipating Visual Representations from Unlabeled Video
Vondrick, C., Pirsiavash, H., & Torralba, A. B. (2016) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2016.18 · Cited by 538 · see citing works
Lending A Hand: Detecting Hands and Recognizing Activities in Complex Egocentric Interactions
Bambach, S., Lee, S., Crandall, D., & Yu, C. (2015) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.226 · Cited by 420 · see citing works
Underspecification Presents Challenges for Credibility in Modern Machine Learning
D'Amour, A., Heller, K., Moldovan, D., Adlam, B., Alipanahi, B., Beutel, A., Chen, C., Deaton, J., Eisenstein, J., Hoffman, M. D., et al. (2020) · arXiv · doi:10.48550/arxiv.2011.03395 · Cited by 398 · see citing works
Representation Learning by Learning to Count
Noroozi, M., Pirsiavash, H., & Favaro, P. (2017) · IEEE International Conference on Computer Vision · doi:10.1109/iccv.2017.628 · Cited by 379 · see citing works
Learning Image Representations Tied to Ego-Motion
Jayaraman, D., & Grauman, K. L. (2015) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.166 · Cited by 229 · see citing works
Measuring Robustness to Natural Distribution Shifts in Image Classification
Taori, R., Dave, A., Shankar, V., Carlini, N., Recht, B., & Schmidt, L. (2020) · arXiv · doi:10.48550/arxiv.2007.00644 · Cited by 206 · see citing works
Self-citationSelf-supervised Pretraining of Visual Features in the Wild
Goyal, P., Caron, M., Lefaudeux, B., Xu, M., Wang, P., Pai, V., Singh, M., Liptchinsky, V., Misra, I., Joulin, A., et al. (2021) · arXiv · doi:10.48550/arxiv.2103.01988 · Cited by 141 · see citing works
Demystifying Contrastive Self-Supervised Learning: Invariances, Augmentations and Dataset BiasesPreprint
Purushwalkam, S., & Gupta, A. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2007.13916 · Cited by 137 · see citing works
How Useful Is Self-Supervised Pretraining for Visual Tasks?
Newell, A., & Deng, J. (2020) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr42600.2020.00737 · Cited by 122 · see citing works
A critical analysis of self-supervision, or what we can learn from a single image
Asano, Y. M., Rupprecht, C., & Vedaldi, A. (2019) · Oxford University Research Archive (ORA) (University of Oxford) · doi:10.48550/arxiv.1904.13132 · Cited by 115 · see citing works
What makes instance discrimination good for transfer learning?
Zhao, N., Wu, Z., Lau, R. W. H., & Lin, S. (2020) · arXiv · doi:10.48550/arxiv.2006.06606 · Cited by 115 · see citing works
Self-Supervised Pretraining Improves Self-Supervised Pretraining
Reed, C., Yue, X., Nrusimha, A., Ebrahimi, S., Vijaykumar, V., Mao, R., Li, B., Zhang, S., Guillory, D., Metzger, S. L., et al. (2022) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv51458.2022.00112 · Cited by 105 · see citing works
When Does Contrastive Visual Representation Learning Work?
Cole, E., Yang, X., Wilber, K., Aodha, O. M., & Belongie, S. (2022) · 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52688.2022.01434 · Cited by 105 · see citing works
The Visual Task Adaptation BenchmarkPreprint
Zhai, X., Puigcerver, J., Kolesnikov, A., Ruyssen, P., Riquelme, C., Lučić, M., Djolonga, J., Pinto, A. S., Neumann, M., Dosovitskiy, A., et al. (2019) · arXiv (Cornell University) · Cited by 42 · see citing works
Concept Generalization in Visual Representation Learning
Sarıyıldız, M. B., Kalantidis, Y., Larlus, D., & Alahari, K. (2021) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00949 · Cited by 33 · see citing works
A Task-Oriented Approach for Cost-Sensitive Recognition
Mottaghi, R., Hajishirzi, H., & Farhadi, A. (2016) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2016.242 · Cited by 2 · see citing works
Self-citation
Flagged first, then by citations
7 of the 74 references couldn’t be matched to a known paper.
Cited by
Sorted by citations
Machine learning bridges omics sciences and plant breeding
Yan, J., & Wang, X. (2022) · Trends in Plant Science · doi:10.1016/j.tplants.2022.08.018 · Cited by 182 · see citing works
When Does Contrastive Visual Representation Learning Work?
Cole, E., Yang, X., Wilber, K., Aodha, O. M., & Belongie, S. (2022) · 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52688.2022.01434 · Cited by 105 · see citing works
Large-Scale Unsupervised Semantic Segmentation
Gao, S., Li, Z.-Y., Yang, M.-H., Cheng, M.-M., Han, J., & Torr, P. (2023) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2022.3218275 · Cited by 88 · see citing works
A Review of Predictive and Contrastive Self-supervised Learning for Medical Images
Wang, W.-C., Ahn, E., Feng, D.-W., & Kim, J. (2023) · Machine Intelligence Research · doi:10.1007/s11633-022-1406-4 · Cited by 59 · see citing works
Zhang, T., Gao, P., Dong, H., Zhuang, Y., Wang, G., Zhang, W., & Chen, H. (2022) · Remote Sensing · doi:10.3390/rs14225675 · Cited by 49 · see citing works
Unsupervised Domain Generalization by Learning a Bridge Across Domains
Harary, S., Schwartz, E., Arbelle, A., Staar, P., Abu-Hussein, S., Amrani, E., Herzig, R., Alfassy, A., Giryes, R., Kuehne, H., et al. (2021) · Computer Vision and Pattern Recognition · doi:10.1109/cvpr52688.2022.00521 · Cited by 56 · see citing works
Teaching Matters: Investigating the Role of Supervision in Vision Transformers
Walmer, M., Suri, S., Gupta, K., & Shrivastava, A. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.00723 · Cited by 26 · see citing works
Learning to Align Sequential Actions in the Wild
Liu, W., Tekin, B., Coskun, H., Vineet, V., Fua, P., & Pollefeys, M. (2021) · Computer Vision and Pattern Recognition · doi:10.1109/cvpr52688.2022.00222 · Cited by 37 · see citing works
Improving fine-tuning of self-supervised models with Contrastive Initialization
Pan, H., Guo, Y. F., Deng, Q., Yang, H., Chen, J., & Chen, Y. (2022) · Neural Networks · doi:10.1016/j.neunet.2022.12.012 · Cited by 22 · see citing works
Peng, G., Lei, Y., Li, H., Wu, D. J., Wang, J., & Liu, F. (2021) · IEEE Access · doi:10.1109/access.2021.3132301 · Cited by 21 · see citing works
Scale-Adaptive Feature Aggregation for Efficient Space-Time Video Super-Resolution
Huang, Z., Huang, A., Hu, X., Chen, H., Xu, J., & Zhou, S. (2024) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv57701.2024.00418 · Cited by 21 · see citing works
Benchmarking Omni-Vision Representation Through the Lens of Visual Realms
Zhang, Y., Yin, Z., Shao, J., & Liu, Z. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-20071-7_35 · Cited by 19 · see citing works
How Severe Is Benchmark-Sensitivity in Video Self-supervised Learning?
Thoker, F. M., Doughty, H., Bagad, P., & Snoek, C. G. M. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-19830-4_36 · Cited by 18 · see citing works
Benchmarking Self-Supervised Contrastive Learning Methods for Image-Based Plant Phenotyping
Ogidi, F., Eramian, M., & Stavness, I. (2023) · Plant Phenomics · doi:10.34133/plantphenomics.0037 · Cited by 16 · see citing works
Gwilliam, M., & Shrivastava, A. (2022) · Computer Vision and Pattern Recognition · doi:10.1109/cvpr52688.2022.00942 · Cited by 26 · see citing works
Soft Contrastive Sequential Recommendation
Zhang, Y., Wang, Z., Yu, W., Hu, L., Jiang, P., Gai, K., & Chen, X. (2024) · ACM Transactions on Information Systems · doi:10.1145/3665325 · Cited by 10 · see citing works
A coded knowledge distillation framework for image classification based on adaptive JPEG encoding
Salamah, A. H., Hamidi, S. M., & Yang, E. (2024) · Pattern Recognition · doi:10.1016/j.patcog.2024.110966 · Cited by 9 · see citing works
Pang, K., Liu, Y., Zhou, S., Liao, Y., Yin, Z., Zhao, L., & Chen, H. (2024) · Foods · doi:10.3390/foods13223598 · Cited by 6 · see citing works
Gao, X., Jamil, N., & Ramli, M. I. (2024) · Applied Sciences · doi:10.3390/app142311093 · Cited by 6 · see citing works
Carneiro, G. A., Aubry, T., Cunha, A. M. T. D. S., Radeva, P. I., & Sousa, J. J. (2025) · Computers and Electronics in Agriculture · doi:10.1016/j.compag.2025.111134 · Cited by 6 · see citing works
On Pretraining Data Diversity for Self-Supervised Learning
Hammoud, H. A. A. K., Das, T., Pizzati, F., Torr, P. H. S., Bibi, A., & Ghanem, B. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-72992-8_4 · Cited by 8 · see citing works
Unsupervised Facial Action Representation Learning by Temporal Prediction
Wang, C., & Wang, Z. (2022) · Frontiers in Neurorobotics · doi:10.3389/fnbot.2022.851847 · Cited by 3 · see citing works
Self-supervised learning for early detection of neurodegenerative diseases with small data
Jiang, H. (2023) · doi:10.32657/10356/166402 · Cited by 3 · see citing works
Witte, A. V., Lange, S., & Lins, C. (2024) · Industrial Artificial Intelligence · doi:10.1007/s44244-024-00020-y · Cited by 2 · see citing works
Patch-wise self-supervised visual representation learning: a fine-grained approach
Javidani, A., Sadeghi, M. A., & Araabi, B. N. (2025) · Signal Image and Video Processing · doi:10.1007/s11760-025-04020-y · Cited by 2 · see citing works
Pre-training for Action Recognition with Automatically Generated Fractal Datasets
Svyezhentsev, D., Retsinas, G., & Maragos, P. A. (2025) · International Journal of Computer Vision · doi:10.1007/s11263-025-02420-8 · Cited by 1 · see citing works
Self-supervised Pretraining of Vision Transformers for Earth Observation
Fuller, A. (2023) · doi:10.22215/etd/2023-15793 · Cited by 1 · see citing works
Buatthaisong, N., Slijepcevic, I. V., Scaife, A. M. M., Bowles, M., Hopkins, A., Mohan, D., Shabala, S. S., & Wong, O. I. (2025) · Monthly Notices of the Royal Astronomical Society · doi:10.1093/mnras/staf1942
- Authors
- Kotar, K., Ilharco, G., Schmidt, L., Ehsani, K., & Mottaghi, R.
- Affiliations
- Allen Institute for Artificial Intelligence (US)
- University of Washington (US)
- Venue
- 2021 IEEE/CVF International Conference on Computer Vision (ICCV)
- Publisher
- IEEE
- Published
- 2021-10
- Pages
- 9929-9939
- OpenAlex
- W3139236157
- Cited by
- 28
- References
- 74
- Citations by year
- 2025: 4 · 2024: 7 · 2023: 5 · 2022: 11 · 2021: 1
- FWCI
- 2.77
- Type
- conference paper
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- IEEE/CVF International Conference on Computer Vision (ICCV) (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Domain Adaptation and Few-Shot Learning
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Multimodal Machine Learning Applications
- Keywords
- pre-training dataset, downstream task performance, encoder comparison, benchmark evaluation
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
