Unsupervised Representation Learning by Predicting Image Rotations
Open accessAbstract
Over the last years, deep convolutional neural networks (ConvNets) have transformed the field of computer vision thanks to their unparalleled capacity to learn high level semantic image features. However, in order to successfully learn those features, they usually require massive amounts of manually labeled data, which is both expensive and impractical to scale. Therefore, unsupervised semantic feature learning, i.e., learning without requiring manual annotation effort, is of crucial importance in order to successfully harvest the vast amount of visual data that are available today. In our work we propose to learn image features by training ConvNets to recognize the 2d rotation that is applied to the image that it gets as input. We demonstrate both qualitatively and quantitatively that this apparently simple task actually provides a very powerful supervisory signal for semantic feature learning. We exhaustively evaluate our method in various unsupervised feature learning benchmarks and we exhibit in all of them state-of-the-art performance. Specifically, our results on those benchmarks demonstrate dramatic improvements w.r.t. prior state-of-the-art approaches in unsupervised representation learning and thus significantly close the gap with supervised feature learning. For instance, in PASCAL VOC 2007 detection task our unsupervised pre-trained AlexNet model achieves the state-of-the-art (among unsupervised methods) mAP of 54.4% that is only 2.4 points lower from the supervised case. We get similarly striking results when we transfer our unsupervised learned features on various other tasks, such as ImageNet classification, PASCAL classification, PASCAL segmentation, and CIFAR-10 classification. The code and models of our paper will be published on: https://github.com/gidariss/FeatureLearningRotNet .
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Pretext tasks can limit the generality of the learned representations.
“However, the pretext tasks can limit the generality of the learned representations.”
From Time-Series Representation Learning via Temporal and Contextual Contrasting (2021) · cites this paper as Gidaris et al., 2018
The Introduction says representations learned through earlier self-supervised pretext tasks had not matched supervised representations in performance, which supports a qualified concern about their effectiveness. It also calls them good alternatives for transfer to other vision tasks; the paper does not establish that the pretext tasks themselves limit the representations’ generality. No page number is available in our copy. (LLM, conf=0.72)
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Gradient-based learning applied to document recognition
Lecun, Y., Bottou, L., Bengio, Y., & Haffner, P. (1998) · Proceedings of the IEEE · doi:10.1109/5.726791 · Cited by 58,805 · cited 1× · see citing works
“In recent years, the widespread adoption of deep convolutional neural networks (LeCun et al., 1998) (ConvNets) in computer vision, has lead to a tremendous progress in the field.”
BackgroundFully convolutional networks for semantic segmentationPreprint
Long, J., Shelhamer, E., & Darrell, T. (2014) · Computer Vision and Pattern Recognition · doi:10.48550/arxiv.1605.06211 · Cited by 43,601 · cited 1× · see citing works
“For instance, the image features learned by ConvNets in this supervised manner have achieved excellent results when they are transferred to other vision tasks, such as object detection (Girshick, 2015), semantic segmentation (Long et al., 2015), or image captioning (Karpathy & Fei-Fei, 2015).”
BackgroundImageNet Large Scale Visual Recognition Challenge
Russakovsky, O., Deng, J., Su, H., Krause, J., Satheesh, S., Ma, S., Huang, Z., Karpathy, A., Khosla, A., Bernstein, M., et al. (2015) · International Journal of Computer Vision · doi:10.1007/s11263-015-0816-y · Cited by 39,380 · cited 10× · see citing works
“Specifically, our self-supervised approach manages to drastically improve the state-of-the-art results on unsupervised feature learning for ImageNet classification, PASCAL classification, PASCAL detection, PASCAL segmentation, and CIFAR-10 classification, surpassing prior approaches by a significant margin and thus drastic…”
BackgroundMethodInfluentialFast R-CNNPreprint
Girshick, R. B. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1504.08083 · Cited by 29,416 · cited 1× · see citing works
“For instance, the image features learned by ConvNets in this supervised manner have achieved excellent results when they are transferred to other vision tasks, such as object detection (Girshick, 2015), semantic segmentation (Long et al., 2015), or image captioning (Karpathy & Fei-Fei, 2015).”
BackgroundUnsupervised representation learning with deep convolutional generative adversarial networks
Radford, A., Metz, L., & Chintala, S. (2015) · arXiv · doi:10.48550/arxiv.1511.06434 · Cited by 12,235 · cited 1× · see citing works
“…(Dosovitskiy et al., 2014; Liao et al., 2016; Yang et al., 2016), reconstruction based methods (Bengio et al., 2007; Huang et al., 2007; Masci et al., 2011), and methods that involve learning generative probabilistic models Goodfellow et al. (2014); Donahue et al. (2016); Radford et al. (2015).”
MethodDeep Visual-Semantic Alignments for Generating Image DescriptionsPreprint
Karpathy, A., & Fei-Fei, L. (2014) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.48550/arxiv.1412.2306 · Cited by 6,303 · see citing works
Context Encoders: Feature Learning by Inpainting
Pathak, D., Krahenbuhl, P., Donahue, J., Darrell, T., & Efros, A. A. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.278 · Cited by 5,691 · cited 3× · see citing works
“Specifically, we first train a RotNet model on the training images of the ImageNet dataset and then we evaluate the performance of the self-supervised features on the image Context (Doersch et al., 2015) 19.7 26.7 31.9 32.7 30.9 Context Encoders (Pathak et al., 2016b) 18.2 23.2 23.4 21.9 18.4 Colorization (Zhang et al.,…”
InfluentialDeep visual-semantic alignments for generating image descriptions
Karpathy, A., & Fei-Fei, L. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298932 · Cited by 4,902 · see citing works
Greedy layer-wise training of deep networks
Bengio, Y., Lamblin, P., Popovici, D., & Larochelle, H. (2007) · Advances in Neural Information Processing Systems 19 · doi:10.7551/mitpress/7503.003.0024 · Cited by 4,841 · cited 2× · see citing works
“…feature learning are clustering based methods (Dosovitskiy et al., 2014; Liao et al., 2016; Yang et al., 2016), reconstruction based methods (Bengio et al., 2007; Huang et al., 2007; Masci et al., 2011), and methods that involve learning generative probabilistic models Goodfellow et al.…”
Zhang, R., Isola, P., & Efros, A. A. (2016) · European Conference on Computer Vision · doi:10.1007/978-3-319-46487-9_40 · Cited by 3,803 · cited 9× · see citing works
“…59.7 Random from (Noroozi & Favaro, 2016) 27.1 12.0 Tracking Wang & Gupta (2015) 38.8 29.8 Context (Doersch et al., 2015) 45.6 30.4 Colorization (Zhang et al., 2016a) 40.7 35.2 Jigsaw Puzzles (Noroozi & Favaro, 2016) 45.3 34.6 BIGAN (Donahue et al., 2016) 41.9 32.2 NAT (Bojanowski & Joulin,…”
BackgroundInfluentialUnsupervised Learning of Visual Representations by Solving Jigsaw Puzzles
Noroozi, M., & Favaro, P. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-46466-4_5 · Cited by 2,775 · cited 9× · see citing works
“ImageNet labels from (Bojanowski & Joulin, 2017) 59.7 59.7 Random from (Noroozi & Favaro, 2016) 27.1 12.0 Tracking Wang & Gupta (2015) 38.8 29.8 Context (Doersch et al., 2015) 45.6 30.4 Colorization (Zhang et al., 2016a) 40.7 35.2 Jigsaw Puzzles (Noroozi & Favaro, 2016) 45.3 34.6 BIGAN (Donahue et…”
BackgroundMethodInfluentialUnsupervised Visual Representation Learning by Context Prediction
Doersch, C., Gupta, A., & Efros, A. A. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.167 · Cited by 2,629 · cited 6× · see citing works
“…on other vision tasks, such as object recognition, object detection, and semantic segmentation (Zhang et al., 2016a; Larsson et al., 2016; Zhang et al., 2016b; Larsson et al., 2017; Doersch et al., 2015; Noroozi & Favaro, 2016; Noroozi et al., 2017; Pathak et al., 2016a; Doersch & Zisserman, 2017).”
BackgroundMethodInfluentialAccurate, Large Minibatch SGD: Training ImageNet in 1 Hour
Goyal, P., Dollár, P., Girshick, R., Noordhuis, P., Wesolowski, L., Kyrola, A., Tulloch, A., Jia, Y., & He, K. (2017) · arXiv · doi:10.48550/arxiv.1706.02677 · Cited by 2,294 · cited 1× · see citing works
“…our AlexNet model trains in around 2 days using a single Titan X GPU), and can trivially adopt the efficient parallelization schemes devised for supervised learning (Goyal et al., 2017), making it an ideal candidate for unsupervised learning on internet-scale data (i.e., billions of images).”
MethodStacked Convolutional Auto-Encoders for Hierarchical Feature Extraction
Masci, J., Meier, U., Ciresan, D., & Schmidhuber, J. (2011) · International Conference on Artificial Neural Networks · doi:10.1007/978-3-642-21735-7_7 · Cited by 2,185 · cited 1× · see citing works
“…methods (Dosovitskiy et al., 2014; Liao et al., 2016; Yang et al., 2016), reconstruction based methods (Bengio et al., 2007; Huang et al., 2007; Masci et al., 2011), and methods that involve learning generative probabilistic models Goodfellow et al. (2014); Donahue et al. (2016); Radford et…”
MethodAdversarial Feature LearningPreprint
Donahue, J., Krähenbühl, P., & Darrell, T. (2016) · International Conference on Learning Representations · doi:10.48550/arxiv.1605.09782 · Cited by 1,982 · cited 5× · see citing works
“…23.4 21.9 18.4 Colorization (Zhang et al., 2016a) 16.0 25.7 29.6 30.3 29.7 Jigsaw Puzzles (Noroozi & Favaro, 2016) 23.0 31.9 35.0 34.2 29.3 BIGAN (Donahue et al., 2016) 22.0 28.7 31.8 31.3 29.7 Split-Brain (Zhang et al., 2016b) 21.3 30.7 34.0 34.1 32.5 Counting (Noroozi et al., 2017) 23…”
MethodInfluentialUnsupervised Learning of Invariant Feature Hierarchies with Applications to Object Recognition
Ranzato, M., Huang, F., Boureau, Y.-L., & LeCun, Y. (2007) · 2007 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2007.383157 · Cited by 1,186 · cited 2× · see citing works
“…are clustering based methods (Dosovitskiy et al., 2014; Liao et al., 2016; Yang et al., 2016), reconstruction based methods (Bengio et al., 2007; Huang et al., 2007; Masci et al., 2011), and methods that involve learning generative probabilistic models Goodfellow et al. (2014); Donahue et al.…”
Discriminative Unsupervised Feature Learning with Convolutional Neural NetworksPreprint
Dosovitskiy, A., Springenberg, J., Riedmiller, M. A., & Brox, T. (2014) · Neural Information Processing Systems · doi:10.48550/arxiv.1406.6909 · Cited by 1,080 · cited 3× · see citing works
“Dosovitskiy et al. (2014) train a ConvNet model to yield representations that are discriminative between images and at the same time invariant on geometric and chromatic transformations.”
BackgroundMethodLearning Representations for Automatic Colorization
Larsson, G., Maire, M., & Shakhnarovich, G. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-46493-0_35 · Cited by 946 · cited 2× · see citing works
“For example, in order to learn features, Zhang et al. (2016a) and Larsson et al. (2016) train ConvNets to colorize gray scale images, Doersch et al. (2015) and Noroozi & Favaro (2016) predict the relative position of image patches, and Agrawal et al. (2015) predict the egomotion (i.e., self-motion)…”
BackgroundJoint Unsupervised Learning of Deep Representations and Image Clusters
Yang, J., Parikh, D., & Batra, D. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.556 · Cited by 879 · cited 2× · see citing works
“Other successful cases of unsupervised feature learning are clustering based methods (Dosovitskiy et al., 2014; Liao et al., 2016; Yang et al., 2016), reconstruction based methods (Bengio et al.”
Split-Brain Autoencoders: Unsupervised Learning by Cross-Channel Prediction
Zhang, R., Isola, P., & Efros, A. A. (2016) · Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2017.76 · Cited by 724 · cited 3× · see citing works
“Specifically, we first train a RotNet model on the training images of the ImageNet dataset and then we evaluate the performance of the self-supervised features on the image Context (Doersch et al., 2015) 19.7 26.7 31.9 32.7 30.9 Context Encoders (Pathak et al., 2016b) 18.2 23.2 23.4 21.9 18.4 Colorization (Zhang et al.,…”
BackgroundInfluentialMulti-task Self-Supervised Visual Learning
Doersch, C., & Zisserman, A. (2017) · IEEE International Conference on Computer Vision · doi:10.1109/iccv.2017.226 · Cited by 682 · cited 1× · see citing works
“…on other vision tasks, such as object recognition, object detection, and semantic segmentation (Zhang et al., 2016a; Larsson et al., 2016; Zhang et al., 2016b; Larsson et al., 2017; Doersch et al., 2015; Noroozi & Favaro, 2016; Noroozi et al., 2017; Pathak et al., 2016a; Doersch & Zisserman, 2017).”
BackgroundAgrawal, P., Carreira, J., & Malik, J. (2015) · IEEE International Conference on Computer Vision · doi:10.1109/iccv.2015.13 · Cited by 639 · cited 3× · see citing works
“…(2016a) and Larsson et al. (2016) train ConvNets to colorize gray scale images, Doersch et al. (2015) and Noroozi & Favaro (2016) predict the relative position of image patches, and Agrawal et al. (2015) predict the egomotion (i.e., self-motion) of a moving vehicle between two consecutive frames.”
BackgroundMethodLearning Features by Watching Objects Move
Pathak, D., Girshick, R. B., Dollár, P., Darrell, T., & Hariharan, B. (2016) · Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2017.638 · Cited by 549 · cited 3× · see citing works
“Specifically, we first train a RotNet model on the training images of the ImageNet dataset and then we evaluate the performance of the self-supervised features on the image Context (Doersch et al., 2015) 19.7 26.7 31.9 32.7 30.9 Context Encoders (Pathak et al., 2016b) 18.2 23.2 23.4 21.9 18.4 Colorization (Zhang et al.,…”
BackgroundInfluentialColorization as a Proxy Task for Visual Understanding
Larsson, G., Maire, M., & Shakhnarovich, G. (2017) · Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2017.96 · Cited by 527 · cited 1× · see citing works
“…on other vision tasks, such as object recognition, object detection, and semantic segmentation (Zhang et al., 2016a; Larsson et al., 2016; Zhang et al., 2016b; Larsson et al., 2017; Doersch et al., 2015; Noroozi & Favaro, 2016; Noroozi et al., 2017; Pathak et al., 2016a; Doersch & Zisserman, 2017).”
BackgroundRepresentation Learning by Learning to Count
Noroozi, M., Pirsiavash, H., & Favaro, P. (2017) · IEEE International Conference on Computer Vision · doi:10.1109/iccv.2017.628 · Cited by 379 · cited 3× · see citing works
“Specifically, we first train a RotNet model on the training images of the ImageNet dataset and then we evaluate the performance of the self-supervised features on the image Context (Doersch et al., 2015) 19.7 26.7 31.9 32.7 30.9 Context Encoders (Pathak et al., 2016b) 18.2 23.2 23.4 21.9 18.4 Colorization (Zhang et al.,…”
BackgroundInfluentialDeep roto-translation scattering for object classification
Oyallon, E., & Mallat, S. (2014) · Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2015.7298904 · Cited by 304 · see citing works
Lin, M., Chen, Q., & Yan, S. (2013) · arXiv · doi:10.48550/arxiv.1312.4400 · Cited by 252 · cited 1× · see citing works
“In our CIFAR-10 experiments we implement the RotNet models with Network-In-Network (NIN) architectures (Lin et al., 2013).”
MethodUnsupervised Learning by Predicting Noise
Bojanowski, P., & Joulin, A. (2017) · arXiv · doi:10.48550/arxiv.1704.05310 · Cited by 228 · cited 3× · see citing works
“ImageNet labels from (Bojanowski & Joulin, 2017) 59.7 59.7 Random from (Noroozi & Favaro, 2016) 27.1 12.0 Tracking Wang & Gupta (2015) 38.8 29.8 Context (Doersch et al., 2015) 45.6 30.4 Colorization (Zhang et al., 2016a) 40.7 35.2 Jigsaw Puzzles (Noroozi & Favaro, 2016) 45.3 34.6 BIGAN (Donahue et…”
InfluentialData-dependent Initializations of Convolutional Neural NetworksPreprint
Krähenbühl, P., Doersch, C., Donahue, J., & Darrell, T. (2015) · International Conference on Learning Representations · doi:10.48550/arxiv.1511.06856 · Cited by 211 · see citing works
Scaling the Scattering Transform: Deep Hybrid Networks
Oyallon, E., Belilovsky, E., & Zagoruyko, S. (2017) · IEEE International Conference on Computer Vision · doi:10.1109/iccv.2017.599 · Cited by 176 · see citing works
Flagged first, then by citations
5 of the 35 references couldn’t be matched to a known paper.
Research works printed but not linked: 6
- M. Everingham, L. Van Gool, C. K. I. Williams, J. Winn, and A. Zisserman. The pascal visual object classes (voc) challenge. International Journal of Computer Vision, 88(2):303-338, June 2010.
- Ross Girshick. Fast r-cnn. In Proceedings of the IEEE international conference on computer vision, pp. 1440-1448, 2015.
- Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. Generative adversarial nets. In Advances in neural infor- mation processing systems, pp. 2672-2680, 2014.
- Renjie Liao, Alex Schwing, Richard Zemel, and Raquel Urtasun. Learning deep parsimonious representations. In Advances in Neural Information Processing Systems, pp. 5076-5084, 2016.
- Xiaolong Wang and Abhinav Gupta. Unsupervised learning of visual representations using videos. In Proceedings of the IEEE International Conference on Computer Vision, pp. 2794-2802, 2015.
- Bolei Zhou, Agata Lapedriza, Jianxiong Xiao, Antonio Torralba, and Aude Oliva. Learning deep features for scene recognition using places database. In Z. Ghahramani, M. Welling, C. Cortes, N. D. Lawrence, and K. Q. Weinberger (eds.), Advances in Neural Information Processing Sys- tems 27, pp. 487-495.
Cited by
Sorted by citations
Momentum Contrast for Unsupervised Visual Representation Learning
He, K., Fan, H., Wu, Y., Xie, S., & Girshick, R. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.00975 · Cited by 13,347 · see citing works
Masked Autoencoders Are Scalable Vision Learners
He, K., Chen, X., Xie, S., Li, Y., Dollar, P., & Girshick, R. (2022) · 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52688.2022.01553 · Cited by 8,489 · see citing works
“Self-supervised learning approaches have seen significant interest in computer vision, often focusing on different pre-text tasks for pre-training [15, 61, 42, 70, 45, 17].”
BackgroundSimMIM: a Simple Framework for Masked Image Modeling
Xie, Z., Zhang, Z., Cao, Y., Lin, Y., Bao, J., Yao, Z., Dai, Q., & Hu, H. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.00943 · Cited by 1,348 · see citing works
“Other self-supervised learning approaches During the last two decades, there have been numerous pretext tasks to learn visual representation in a self-supervised way: grayscale image colorization [57], jigsaw puzzle solving [34], split-brain auto-encoding [58], rotation prediction [17], learning to cluster [4].”
BackgroundUNITER: UNiversal Image-TExt Representation Learning
Chen, Y.-C., Li, L., Yu, L., El Kholy, A., Ahmed, F., Gan, Z., Cheng, Y., & Liu, J. (2020) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-58577-8_7 · Cited by 2,055 · see citing works
Self-Supervised Learning of Pretext-Invariant Representations
Misra, I., & van der Maaten, L. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.00674 · Cited by 1,435 · see citing works
iBOT: Image BERT Pre-Training with Online TokenizerPreprint
Zhou, J., Wei, C., Wang, H., Shen, W., Xie, C., Yuille, A., & Kong, T. (2021) · arXiv.org · doi:10.48550/arxiv.2111.07832 · Cited by 1,275 · see citing works
“Other methods rely on handcrafted pretext tasks and assume the image representation should instead be aware of image augmentation by solving image jigsaw puzzle (Noroozi & Favaro, 2016; Wei et al., 2019), predicting rotation (Komodakis & Gidaris, 2018) or relative position (Doersch et al., 2015).”
MethodDINOv2: Learning Robust Visual Features without Supervision
Oquab, M., Darcet, T., Moutakanni, T., Vo, H., Szafraniec, M., Khalidov, V., Fernandez, P., Haziza, D., Massa, F., El-Nouby, A., et al. (2023) · arXiv · doi:10.48550/arxiv.2304.07193 · Cited by 1,078 · see citing works
“Many other pretext tasks were introduced based on, for example, re-colorizing images (Zhang et al., 2016), predicting transformations (Gidaris et al., 2018), inpainting (Pathak et al., 2016) or patch re-ordering (Noroozi & Favaro, 2016; Misra & Maaten, 2020).”
BackgroundMethodCutPaste: Self-Supervised Learning for Anomaly Detection and Localization
Li, C.-L., Sohn, K., Yoon, J., & Pfister, T. (2021) · 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr46437.2021.00954 · Cited by 1,107 · see citing works
SPot-the-Difference Self-Supervised Pre-training for Anomaly Detection and Segmentation
Zou, Y., Jeong, J., Pemula, L., Zhang, D.-Q., & Dabeer, O. (2022) · European Conference on Computer Vision · doi:10.48550/arxiv.2207.14315 · Cited by 886
“image colorization [46], rotation prediction [20], jigsaw puzzles [29].”
BackgroundSelf-Supervised Pre-Training of Swin Transformers for 3D Medical Image Analysis
Tang, Y., Yang, D., Li, W., Roth, H. R., Landman, B., Xu, D., Nath, V., & Hatamizadeh, A. (2022) · 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52688.2022.02007 · Cited by 849 · see citing works
Efficient Test-Time Model Adaptation without Forgetting
Niu, S.-C., Wu, J.-X., Zhang, Y.-F., Chen, Y.-F., Zheng, S., Zhao, P.-L., & Tan, M. (2022) · International Conference on Machine Learning · doi:10.48550/arxiv.2204.02610 · Cited by 794
“Given a test sample, TTT first fine-tunes the model via rotation classification (Gidaris et al., 2018) and then makes a prediction using the updated model.”
MethodDeep neural network models for computational histopathology: A survey
Srinidhi, C. L., Ciga, O., & Martel, A. L. (2021) · Medical Image Analysis · doi:10.1016/j.media.2020.101813 · Cited by 749 · see citing works
Transformer-based unsupervised contrastive learning for histopathological image classification
Wang, X., Yang, S., Zhang, J., Wang, M., Zhang, J., Yang, W., Huang, J., & Han, X. (2022) · Medical Image Analysis · doi:10.1016/j.media.2022.102559 · Cited by 751 · see citing works
A Comprehensive Survey on Test-Time Adaptation Under Distribution Shifts
Liang, J., He, R., & Tan, T.-P. (2023) · International Journal of Computer Vision · doi:10.1007/s11263-024-02181-w · Cited by 693 · see citing works
“, rotation prediction [87] and solving jigsaw puzzles [51]).”
InfluentialRevisiting Self-Supervised Visual Representation Learning
Kolesnikov, A., Zhai, X., & Beyer, L. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00202 · Cited by 675 · see citing works
A Comprehensive Survey of Image Augmentation Techniques for Deep Learning
Xu, M., Yoon, S., Fuentes, A., & Park, D. S. (2023) · Pattern Recognition · doi:10.1016/j.patcog.2023.109347 · Cited by 685 · see citing works
“First, a pretext task can be created via image augmentation, such as predicting the degrees of rotation [105] and relative positions of image patches [106].”
BackgroundA Survey on Self-Supervised Learning: Algorithms, Applications, and Future Trends
Gui, J., Chen, T., Zhang, J., Cao, Q., Sun, Z., Luo, H., & Tao, D. (2024) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2024.3415112 · Cited by 622 · see citing works
“…The SS-GAN [144] is defined by combining the objective functions of GANs with the concept of rotation [7] L where V ( G, D ) represents the objective function of GANs as given in (18), and r ∼ R refers to a rotation selected from a set of possible rotations, similar to the concept presented in [7].”
BackgroundMethodInfluentialTowards Stable Test-Time Adaptation in Dynamic Wild World
Niu, S.-C., Wu, J.-X., Zhang, Y.-F., Wen, Z.-Q., Chen, Y.-F., Zhao, P.-L., & Tan, M. (2023) · International Conference on Learning Representations · doi:10.48550/arxiv.2302.12400 · Cited by 662
“(2020) first propose the TTT pipeline, in which at training phase a model is trained on source Dtrain via both cross-entropy LCE and self-supervised rotation prediction (Gidaris et al., 2018) LS :”
BackgroundBig Self-Supervised Models Advance Medical Image Classification
Azizi, S., Mustafa, B., Ryan, F., Beaver, Z., Freyberg, J., Deaton, J., Loh, A., Karthikesalingam, A., Kornblith, S., Chen, T., et al. (2021) · 2021 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00346 · Cited by 685 · see citing works
ScaleNet: An Unsupervised Representation Learning Method for Limited Information
Huang, H.-L., & Roozbahani, M. M. (2023) · German Conference on Pattern Recognition · doi:10.1007/978-3-030-92659-5_11 · Cited by 651 · see citing works
“The ScaleNet is examined by training on the ImageNet dataset similar to other self-supervised learning methods [14,37,38].”
BackgroundMethodResultInfluentialTime-Series Representation Learning via Temporal and Contextual Contrasting
Eldele, E., Ragab, M., Chen, Z., Wu, M., Kwoh, C. K., Li, X., & Guan, C. (2021) · Proceedings of the Thirtieth International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2021/324 · Cited by 696 · see citing works
Dense Contrastive Learning for Self-Supervised Visual Pre-Training
Wang, X., Zhang, R., Shen, C., Kong, T., & Li, L. (2021) · 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr46437.2021.00304 · Cited by 660 · see citing works
Contrastive Learning for Sequential Recommendation
Xie, X., Sun, F., Liu, Z., Wu, S., Gao, J., Zhang, J., Ding, B., & Cui, B. (2022) · IEEE International Conference on Data Engineering (ICDE) · doi:10.1109/icde53745.2022.00099 · Cited by 810 · see citing works
“In the CV community, various self-supervised learning pretext tasks are proposed, including predicting image rotations [13] and relative patch locations [10], solving jigsaw puzzles [34], and colorization problems [49].”
BackgroundExploring Cross-Image Pixel Contrast for Semantic Segmentation
Wang, W., Zhou, T., Yu, F., Dai, J., Konukoglu, E., & Gool, L. V. (2021) · 2021 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00721 · Cited by 604 · see citing works
Masked Feature Prediction for Self-Supervised Visual Pre-Training
Wei, C., Fan, H., Xie, S., Wu, C.-Y., Yuille, A., & Feichtenhofer, C. (2022) · 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52688.2022.01426 · Cited by 587 · see citing works
“Self-supervised learning aims to learn from unlabeled visual data by a pre-text task that is constructed by image/patch operations ( e.g ., [7, 26, 36, 65, 91, 98]) and spatiotemporal operations ( e.g ., [35, 38, 64, 68, 90]).”
BackgroundFine-Grained Image Analysis With Deep Learning: A Survey
Wei, X.-S., Song, Y.-Z., Aodha, O. M., Wu, J.-X., Peng, Y.-X., Tang, J.-H., Yang, J., & Belongie, S. (2021) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2021.3126648 · Cited by 514 · see citing works
Patch SVDD: Patch-Level SVDD for Anomaly Detection and Segmentation
Yi, J., & Yoon, S. (2021) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-69544-6_23 · Cited by 551 · see citing works
Huang, S.-C., Pareek, A., Jensen, M., Lungren, M. P., Yeung, S., & Chaudhari, A. S. (2023) · npj Digital Medicine · doi:10.1038/s41746-023-00811-0 · Cited by 479 · see citing works
“Examples of innate relationship for visual inputs include predicting image rotation angle 24 , solving jigsaw puzzles of an image 25 , or determining the relative positions of image patches 26 .”
Contrastive Test-Time Adaptation
Chen, D., Wang, D., Darrell, T., & Ebrahimi, S. (2022) · Computer Vision and Pattern Recognition · doi:10.1109/cvpr52688.2022.00039 · Cited by 452 · see citing works
DeSTSeg: Segmentation Guided Denoising Student-Teacher for Anomaly Detection
Zhang, X., Li, S., Li, X., Huang, P.-C., Shan, J., & Chen, T. (2022) · Computer Vision and Pattern Recognition · doi:10.1109/cvpr52729.2023.00381 · Cited by 408 · see citing works
“Classical anomaly simulation strategies, such as rotation [12] and cutout [11], do not perform well in detecting fine-grained anomalous patterns [17].”
BackgroundRobust fine-tuning of zero-shot models
Wortsman, M., Ilharco, G., Kim, J. W., Li, M., Kornblith, S., Roelofs, R., Lopes, R. G., Hajishirzi, H., Farhadi, A., Namkoong, H., et al. (2022) · 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52688.2022.00780 · Cited by 428 · see citing works
Prototype Augmentation and Self-Supervision for Incremental Learning
Zhu, F., Zhang, X.-Y., Wang, C., Yin, F., & Liu, C.-L. (2021) · 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr46437.2021.00581 · Cited by 427 · see citing works
With a Little Help from My Friends: Nearest-Neighbor Contrastive Learning of Visual Representations
Dwibedi, D., Aytar, Y., Tompson, J., Sermanet, P., & Zisserman, A. (2021) · 2021 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00945 · Cited by 404 · see citing works
Real-World Robot Learning with Masked Visual Pre-trainingPreprint
Radosavovic, I., Xiao, T.-T., James, S., Abbeel, P., Malik, J., & Darrell, T. (2022) · Conference on Robot Learning · doi:10.48550/arxiv.2210.03109 · Cited by 389 · see citing works
“These approaches either learn pretext tasks that distinguish views from 73 the augmentations [34, 35, 36, 37], or learn to be invariant to the augmentations by modeling sim74 ilarities [38, 39, 40, 41].”
BackgroundSub-Image Anomaly Detection with Deep Pyramid Correspondences
Cohen, N., & Hoshen, Y. (2020) · arXiv · doi:10.48550/arxiv.2005.02357 · Cited by 372 · see citing works
Self-Supervised Learning in Remote Sensing: A review
Wang, Y., Albrecht, C. M., Braham, N. A. A., Mou, L., & Zhu, X. X. (2022) · IEEE Geoscience and Remote Sensing Magazine · doi:10.1109/mgrs.2022.3198244 · Cited by 364 · see citing works
“Among others, predicting rotation angles [49, 128] and image inpainting [64, 129] are popular approaches.”
BackgroundMethodXie, Z., Lin, Y., Zhang, Z., Cao, Y., Lin, S., & Hu, H. (2021) · 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr46437.2021.01641 · Cited by 370 · see citing works
Unsupervised Semantic Segmentation by Distilling Feature CorrespondencesPreprint
Hamilton, M., Zhang, Z., Hariharan, B., Snavely, N., & Freeman, W. (2022) · International Conference on Learning Representations · doi:10.48550/arxiv.2203.08414 · Cited by 355 · see citing works
“, 2017), rotation prediction (Gidaris et al., 2018), and most recently, contrastive learning over multiple augmentations (Hjelm et al.”
BackgroundMasked Visual Pre-training for Motor ControlPreprint
Xiao, T.-T., Radosavovic, I., Darrell, T., & Malik, J. (2022) · arXiv.org · doi:10.48550/arxiv.2203.06173 · Cited by 346 · see citing works
“The approaches often rely on pretext tasks for pre-training (Doersch et al., 2015; Wang & Gupta, 2015; Noroozi & Favaro, 2016; Zhang et al., 2016; Pathak et al., 2016; Komodakis & Gidaris, 2018).”
MethodMulti-task Self-Supervised Learning for Human Activity Detection
Saeed, A., Ozcelebi, T., & Lukkien, J. (2019) · Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies · doi:10.1145/3328932 · Cited by 346 · see citing works
Self-Supervised GANs via Auxiliary Rotation Loss
Chen, T., Zhai, X., Ritter, M., Lucic, M., & Houlsby, N. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.01243 · Cited by 328 · see citing works
Augmentation-Free Self-Supervised Learning on Graphs
Lee, N., Lee, J., & Park, C. (2021) · AAAI Conference on Artificial Intelligence · doi:10.48550/arxiv.2112.02472 · Cited by 319
MAGE: MAsked Generative Encoder to Unify Representation Learning and Image Synthesis
Li, T., Chang, H., Mishra, S. K., Zhang, H., Katabi, D., & Krishnan, D. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.00213 · Cited by 122 · see citing works
“Such tasks include solving jigsaw puzzles [43], restoring a missing patch [45], or predicting image rotation [22].”
BackgroundDeep Clustering: A Comprehensive Survey
Ren, Y.-Z., Pu, J., Yang, Z., Xu, J., Li, G., Pu, X.-R., Yu, P. S., & He, L. (2022) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2024.3403155 · Cited by 306 · see citing works
“Such tasks can be predicting the patch context [117], inpainting patches [118], colorizing images [119], solving jigsaw puzzles [120], and predicting rotations [121], and so on.”
Self-Supervised Learning by Cross-Modal Audio-Video Clustering
Alwassel, H., Mahajan, D., Korbar, B., Torresani, L., Ghanem, B., & Tran, D. (2019) · arXiv · doi:10.48550/arxiv.1911.12667 · Cited by 298 · see citing works
Self-supervised learning: A succinct review
Rani, V., Nabi, S. T., Kumar, M., Mittal, A., & Kumar, K. (2023) · Archives of Computational Methods in Engineering · doi:10.1007/s11831-023-09884-2 · Cited by 295 · see citing works
“[47] proposed a ConvNets-based method for recognizing 2D rotations in images.”
MethodSelf-Supervised Learning for Time Series Analysis: Taxonomy, Progress, and Prospects
Zhang, K., Wen, Q., Zhang, C., Cai, R., Jin, M., Liu, Y., Zhang, J. Y., Liang, Y., Pang, G., Song, D., et al. (2024) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2024.3387317 · Cited by 295 · see citing works
“With the help of well-designed pretext tasks, SSL has recently achieved great success in the domains of Computer Vision (CV) [7], [8], [9], [10] and Natural Language Processing (NLP) [11], [12].”
MethodTest-Time Training with Masked Autoencoders
Gandelsman, Y., Sun, Y., Chen, X., & Efros, A. A. (2022) · Neural Information Processing Systems · doi:10.48550/arxiv.2209.07522 · Cited by 293
“The self-supervised pretext task employed by [41] is rotation prediction [15]: rotate each input in the image plane by a multiple of 90 degrees, and predict the angle as a four-way classification problem.”
Self supervised contrastive learning for digital histopathology
Ciga, O., Xu, T., & Martel, A. L. (2022) · Machine Learning with Applications · doi:10.1016/j.mlwa.2021.100198 · Cited by 294 · see citing works
Wang, T., Huang, J., Zhang, H., & Sun, Q. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.01077 · Cited by 291 · see citing works
General Facial Representation Learning in a Visual-Linguistic Manner
Zheng, Y., Yang, H., Zhang, T., Bao, J., Chen, D., Huang, Y., Yuan, L., Chen, D., Zeng, M., & Wen, F. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.01814 · Cited by 193 · see citing works
Targeted Supervised Contrastive Learning for Long-Tailed Recognition
Li, T., Cao, P., Yuan, Y., Fan, L., Yang, Y.-Z., Feris, R., Indyk, P., & Katabi, D. (2021) · Computer Vision and Pattern Recognition · doi:10.1109/cvpr52688.2022.00679 · Cited by 286 · see citing works
“Recent years have witnessed a steady progress on self-supervised representation learning [10, 13, 31, 32].”
BackgroundYang, D., Xu, Z., Li, W., Myronenko, A., Roth, H. R., Harmon, S., Xu, S., Turkbey, B., Turkbey, E., Wang, X., et al. (2021) · Medical Image Analysis · doi:10.1016/j.media.2021.101992 · Cited by 279 · see citing works
Unsupervised Pre-Training of Image Features on Non-Curated Data
Caron, M., Bojanowski, P., Mairal, J., & Joulin, A. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00305 · Cited by 258 · see citing works
Disentangled Representation Learning
Wang, X., Chen, H., Tang, S., Wu, Z., & Zhu, W.-W. (2022) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2024.3420937 · Cited by 266 · see citing works
“[96] propose a self-supervised semantic feature learning method through predicting rotated images with ConvNet model to achieve comparable performances with supervised methods.”
MethodMasked Discrimination for Self-Supervised Learning on Point Clouds
Liu, H., Cai, M., & Lee, Y. J. (2022) · European Conference on Computer Vision · doi:10.48550/arxiv.2203.11183 · Cited by 264
“Traditional methods typically rely on pretext tasks, such as image rotation prediction [18], image colorization [67], and solving jigsaw puzzles [35].”
MethodContrastive Meta Learning with Behavior Multiplicity for Recommendation
Wei, W., Huang, C., Xia, L., Xu, Y., Zhao, J.-S., & Yin, D.-W. (2022) · Web Search and Data Mining · doi:10.1145/3488560.3498527 · Cited by 251
“, rotation [13], color distortion [5]) are used to generate negative instances.”
MethodRobust and data-efficient generalization of self-supervised machine learning for diagnostic imaging
Azizi, S., Culp, L., Freyberg, J., Mustafa, B., Baur, S., Kornblith, S., Chen, T., Tomasev, N., Mitrović, J., Strachan, P., et al. (2023) · Nature Biomedical Engineering · doi:10.1038/s41551-023-01049-7 · Cited by 250 · see citing works
“Preliminary works in self-supervised representation learning focused on the problem of learning representations without labels such that a small linear classifier network operating on these representations could achieve high classification accuracy 33,35,164,165 .”
BackgroundTemporal Attention Unit: Towards Efficient Spatiotemporal Predictive Learning
Tan, C., Gao, Z., Wu, L., Xu, Y., Xia, J., Li, S., & Li, S. Z. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.01800 · Cited by 237 · see citing works
Guo, T.-Y., Liu, H., Chen, Z., Liu, M.-Y., Wang, T., & Ding, R.-W. (2021) · AAAI Conference on Artificial Intelligence · doi:10.48550/arxiv.2112.03590 · Cited by 241
“Some contrastive learning methods (Zhang, Isola, and Efros 2016; Pathak et al. 2016; Gidaris, Singh, and Komodakis 2018) focus on designing various novel pretext tasks to find the pattern information hidden in the unlabeled data.”
BackgroundSemi-Supervised and Unsupervised Deep Visual Learning: A Survey
Chen, Y., Mancini, M., Zhu, X., & Akata, Z. (2022) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2022.3201576 · Cited by 235 · see citing works
“The objective of an instance-level pretext task can be written as: where L unsup ( · ) can be various loss functions (e.g., cross-entropy loss [101]) that learn a mapping from a transformed input image Φ z ( x ) to a discrete category or a configuration of the applied transformation z .”
BackgroundMethodInfluential3D Human Action Representation Learning via Cross-View Consistency Pursuit
Li, L., Wang, M., Ni, B., Wang, H., Yang, J., & Zhang, W. (2021) · 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr46437.2021.00471 · Cited by 236 · see citing works
“, jigsaw puzzles [35, 36, 54], colorization [63], predicting rotation [8, 59].”
BackgroundUnsupervised Semantic Segmentation by Contrasting Object Mask Proposals
Van Gansbeke, W., Vandenhende, S., Georgoulis, S., & Van Gool, L. (2021) · 2021 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00990 · Cited by 231 · see citing works
BEVT: BERT Pretraining of Video Transformers
Wang, R., Chen, D., Wu, Z., Chen, Y., Dai, X., Liu, M., Jiang, Y.-G., Zhou, L., & Yuan, L. (2022) · 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52688.2022.01432 · Cited by 234 · see citing works
“The main idea is to design surrogate tasks including inpainting [44], colorization [66, 67], jigsaw predictions [42], rotation predictions [25], etc ., as a form of supervisory signals in lieu of manual labels.”
BackgroundSelf-Supervised Representation Learning by Rotation Feature Decoupling
Feng, Z., Xu, C., & Tao, D. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.01061 · Cited by 225 · see citing works
A Complete Survey on Generative AI (AIGC): Is ChatGPT from GPT-4 to GPT-5 All You Need?Preprint
Zhang, C.-N., Zhang, C.-S., Zheng, S., Qiao, Y., Li, C.-H., Zhang, M.-C., Dam, S. K., Thwal, C. M., Tun, Y. L., Huy, L. L., et al. (2023) · arXiv.org · doi:10.48550/arxiv.2303.11717 · Cited by 225 · see citing works
“Early works focused on designing various pretext tasks like jigsaw puzzles [303] or predicting rotation [121].”
BackgroundSelf-Supervised Contrastive Representation Learning for Semi-Supervised Time-Series Classification
Eldele, E., Ragab, M., Chen, Z., Wu, M., Kwoh, C.-K., Li, X., & Guan, C. (2023) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2023.3308189 · Cited by 224 · see citing works
Chaitanya, K., Erdil, E., Karani, N., & Konukoglu, E. (2021) · Medical Image Anal. · doi:10.48550/arxiv.2112.09645 · Cited by 219
“Some examples of such tasks include: predicting the rotation applied on the input image [27], in-painting the missing pixel values [28], context restoration [29], and many more [7], [30], [31], [32].”
BackgroundRetCCL: Clustering-guided contrastive learning for whole-slide image retrieval
Wang, X., Du, Y., Yang, S., Zhang, J., Wang, M., Zhang, J., Yang, W., Huang, J., & Han, X. (2023) · Medical Image Analysis · doi:10.1016/j.media.2022.102645 · Cited by 218 · see citing works
Melas-Kyriazi, L., Rupprecht, C., Laina, I., & Vedaldi, A. (2022) · Computer Vision and Pattern Recognition · doi:10.1109/cvpr52688.2022.00818 · Cited by 211 · see citing works
“Early research in this area was based on solving pretext tasks such as rotation, jigsaw puzzles, colorization, and inpainting [24, 27, 39, 59, 61, 62, 95, 96].”
BackgroundWang, J., Jiao, J., Bao, L., He, S., Liu, Y., & Liu, W. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00413 · Cited by 210 · see citing works
Parametric Classification for Generalized Category Discovery: A Baseline Study
Wen, X., Zhao, B., & Qi, X. (2022) · IEEE International Conference on Computer Vision · doi:10.1109/iccv51070.2023.01521 · Cited by 206 · see citing works
“The former is often achieved by using self-supervised learning methods [22, 53, 18, 24, 9, 55] to improve the generalization ability of features to novel categories.”
BackgroundAdversarial Robustness: From Self-Supervised Pre-Training to Fine-Tuning
Chen, T., Liu, S., Chang, S., Cheng, Y., Amini, L., & Wang, Z. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.00078 · Cited by 205 · see citing works
The Norm Must Go On: Dynamic Unsupervised Domain Adaptation by Normalization
Mirza, M. J., Micorek, J., Possegger, H., & Bischof, H. (2021) · Computer Vision and Pattern Recognition · doi:10.1109/cvpr52688.2022.01435 · Cited by 205 · see citing works
“For training we use random cropping and horizontal flipping as augmentations.”
MethodVision Transformers for Image Classification: A Comparative Survey
Wang, Y., Deng, Y., Zheng, Y.-J., Chattopadhyay, P., & Wang, L.-P. (2025) · Technologies · doi:10.3390/technologies13010032 · Cited by 202 · see citing works
“In training, ViT adopts a pre-training and fine-tuning strategy [32–34].”
CroCo: Self-Supervised Pre-training for 3D Vision Tasks by Cross-View Completion
Weinzaepfel, P., Leroy, V., Lucas, T., Br'egier, R., Cabon, Y., Arora, V., Antsfeld, L., Chidlovskii, B., Csurka, G., & Revaud, J. (2022) · Neural Information Processing Systems · doi:10.48550/arxiv.2210.10716 · Cited by 200
“the color, the orientation or the ordering of a sequence of patches obtained from the image that the model then learns to recover [28, 34, 57, 58].”
BackgroundA survey on deep learning for skin lesion segmentation
Mirikharaji, Z., Abhishek, K., Bissoto, A. E., Barata, C., Avila, S., Junior, E. A. D. V., Celebi, M. E., & Hamarneh, G. (2023) · Medical Image Analysis · doi:10.1016/j.media.2023.102863 · Cited by 205 · see citing works
Balestriero, R., & LeCun, Y. (2022) · Neural Information Processing Systems · doi:10.52202/068431-1934 · Cited by 197
“…G is often constructed by augmenting X through data-augmentations known to preserve input semantics [Kanazawa et al., 2016, Novotny et al., 2018, Gidaris et al., 2018] e.g. horizontal flip for an image, although recent methods have went away from DA by using videos from which consecutive frames…”
MethodSignBERT+: Hand-Model-Aware Self-Supervised Pre-Training for Sign Language Understanding
Hu, H., Zhao, W., Zhou, W., & Li, H. (2023) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2023.3269220 · Cited by 154 · see citing works
“self-supervised pre-training [41], [42], [43], [44], [45], [46], [47].”
BackgroundImage-to-Lidar Self-Supervised Distillation for Autonomous Driving Data
Sautier, C., Puy, G., Gidaris, S., Boulch, A., Bursuc, A., & Marlet, R. (2022) · Computer Vision and Pattern Recognition · doi:10.1109/cvpr52688.2022.00966 · Cited by 188 · see citing works
“Several approaches have been proposed for representation learning on the image domain [11, 16,23,46,47,49,70].”
BackgroundSelf-citationGMSS: Graph-Based Multi-Task Self-Supervised Learning for EEG Emotion Recognition
Li, Y., Chen, J., Li, F., Fu, B.-X., Wu, H., Ji, Y., Zhou, Y., Niu, Y., Shi, G.-M., & Zheng, W.-M. (2022) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2022.3170428 · Cited by 185 · see citing works
“[46] applied a 2D rotation to the image to construct the pretext task and then predicted the rotation angle to enable the model to learn the position, type, and posture of objects in the image.”
MethodHansen, S., Gautam, S., Jenssen, R., & Kampffmeyer, M. C. (2022) · Medical Image Anal. · doi:10.1016/j.media.2022.102385 · Cited by 185 · see citing works
What to Hide from Your Students: Attention-Guided Masked Image Modeling
Kakogeorgiou, I., Gidaris, S., Psomas, B., Avrithis, Y., Bursuc, A., Karantzalos, K., & Komodakis, N. (2022) · European Conference on Computer Vision · doi:10.1007/978-3-031-20056-4_18 · Cited by 183 · see citing works
“the prediction of patch orderings [48] or rotation angles [22].”
BackgroundSelf-citationReview on self-supervised image recognition using deep neural networks
Ohri, K., & Kumar, M. (2021) · Knowledge-Based Systems · doi:10.1016/j.knosys.2021.107090 · Cited by 179 · see citing works
A Cookbook of Self-Supervised Learning
Balestriero, R., Ibrahim, M., Sobal, V., Morcos, A., Shekhar, S., Goldstein, T., Bordes, F., Bardes, A., Mialon, G., Tian, Y., et al. (2023) · arXiv · doi:10.48550/arxiv.2304.12210 · Cited by 182 · see citing works
“Another line of work tries to retain style information about the augmentations to improve downstream performance on tasks requiring style information such as color by predicting style information [Xiao et al., 2020, Dangovski et al., 2021, Gidaris et al., 2018, Scherr et al., 2022].”
BackgroundGuo, T., Liu, H., Chen, Z., Liu, M., Wang, T., & Ding, R. (2022) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v36i1.19957 · Cited by 181 · see citing works
“Some contrastive learning methods (Zhang, Isola, and Efros 2016; Pathak et al. 2016; Gidaris, Singh, and Komodakis 2018) focus on designing various novel pretext tasks to find the pattern information hidden in the unlabeled data.”
BackgroundLeveraging Real Talking Faces via Self-Supervision for Robust Forgery Detection
Haliassos, A., Mira, R., Petridis, S., & Pantic, M. (2022) · 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52688.2022.01453 · Cited by 180 · see citing works
“, predicting rotations [44] or solving jigsaw puzzles [82], can improve generalisation on the main task at hand [13,43,53].”
Segment Any Point Cloud Sequences by Distilling Vision Foundation ModelsPreprint
Liu, Y.-C., Kong, L., Cen, J., Chen, R.-N., Zhang, W., Pan, L., Chen, K., & Liu, Z.-W. (2023) · Neural Information Processing Systems · doi:10.48550/arxiv.2306.09347 · Cited by 175 · see citing works
“…[84, 77, 83, 14, 97, 45, 110, 29, 91] or indoor scenes [38, 15, 22, 108, 10, 39, 57, 58, 109] by either pretext task learning [21, 70, 71, 112, 30], contrastive learning [11, 32, 12, 34, 27, 13, 46, 98, 104] or mask modeling [105, 31, 24, 61], where the scale and diversity are much lower than…”
BackgroundLearning Representation for Clustering Via Prototype Scattering and Positive Sampling
Huang, Z., Chen, J., Zhang, J.-P., & Shan, H. (2021) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2022.3216454 · Cited by 172 · see citing works
Back to the Feature: Classical 3D Features are (Almost) All You Need for 3D Anomaly Detection
Horwitz, E., & Hoshen, Y. (2022) · 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw59228.2023.00298 · Cited by 169 · see citing works
“[22] that extend RotNet [17] and CSI [41] who extend contrastive methods [10, 20, 21].”
BackgroundGuiding Pseudo-labels with Uncertainty Estimation for Source-free Unsupervised Domain Adaptation
Litrico, M., Bue, A. D., & Morerio, P. (2023) · Computer Vision and Pattern Recognition · doi:10.1109/cvpr52729.2023.00738 · Cited by 167 · see citing works
Naive-Student: Leveraging Semi-Supervised Learning in Video Sequences for Urban Scene Segmentation
Chen, L.-C., Lopes, R. G., Cheng, B., Collins, M. D., Cubuk, E. D., Zoph, B., Adam, H., & Shlens, J. (2020) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-58545-7_40 · Cited by 165 · see citing works
Reinforcement Learning with Action-Free Pre-Training from VideosPreprint
Seo, Y., Lee, K., James, S., & Abbeel, P. (2022) · International Conference on Machine Learning · doi:10.48550/arxiv.2203.13880 · Cited by 163 · see citing works
“Various representation learning methods, including reconstruction (He et al., 2021), rotation (Gidaris et al., 2018), solving zigsaw puzzles (Noroozi & Favaro, 2016), and contrastive learning (He et al., 2020; Chen et al., 2020), have reduced the gap with supervised pre-training with labels.”
BackgroundV-JEPA 2.1: Unlocking Dense Features in Video Self-Supervised Learning
Mur-Labadia, L., Muckley, M., Bar, A., Assran, M., Sinha, K., Rabbat, M. G., LeCun, Y., Ballas, N., & Bardes, A. (2026) · European Conference on Computer Vision · doi:10.48550/arxiv.2603.14482 · Cited by 160
“…of image patches (Doersch et al., 2015), reordering shuffled patches (Noroozi and Favaro, 2016; Misra and Maaten, 2020), inpainting missing regions (Pathak et al., 2016), re-colorizing grayscale images (Zhang et al., 2016), or predicting applied image transformations (Gidaris et al., 2018).”
Self-Supervised Learning of Object Parts for Semantic Segmentation
Ziegler, A., & Asano, Y. M. (2022) · Computer Vision and Pattern Recognition · doi:10.1109/cvpr52688.2022.01410 · Cited by 160 · see citing works
“Instance-level objectives include predicting augmentations applied to an image [15, 22, 31, 38, 39, 54, 55] or learning to discriminate between images [2, 7, 17, 25, 53], often done by the use of contrastive losses [24].”
BackgroundUVCGAN: UNet Vision Transformer cycle-consistent GAN for unpaired image-to-image translation
Torbunov, D., Huang, Y., Yu, H.-W., Huang, J., Yoo, S., Lin, M., Viren, B., & Ren, Y. (2022) · IEEE Workshop/Winter Conference on Applications of Computer Vision · doi:10.1109/wacv56688.2023.00077 · Cited by 158 · see citing works
Entropy is not Enough for Test-Time Adaptation: From the Perspective of Disentangled Factors
Lee, J., Jung, D., Lee, S., Park, J., Shin, J., Hwang, U., & Yoon, S. (2024) · International Conference on Learning Representations · doi:10.48550/arxiv.2403.07366 · Cited by 157
“Depending on the choice of the proxy, TTT may incorporate objectives such as rotation prediction (Gidaris et al., 2018) or contrastive-based losses (Liu et al., 2021b; Bartler et al., 2022).”
MethodFreeSOLO: Learning to Segment Objects without Annotations
Wang, X., Yu, Z.-D., Mello, S. D., Kautz, J., Anandkumar, A., Shen, C.-H., & Álvarez, J. (2022) · Computer Vision and Pattern Recognition · doi:10.1109/cvpr52688.2022.01378 · Cited by 157 · see citing works
“ing [31], jigsaw puzzles [32] and orientation discrimination [33].”
BackgroundTowards the Generalization of Contrastive Self-Supervised LearningPreprint
Huang, W., Yi, M.-Y., Zhao, X.-Y., & Jiang, Z.-H. (2021) · International Conference on Learning Representations · doi:10.48550/arxiv.2111.00743 · Cited by 156 · see citing works
“, 2017; Donahue and Simonyan, 2019), pretext based (Zhang et al., 2016; Pathak et al., 2016; Noroozi and Favaro, 2016; Gidaris et al., 2018; Lee et al., 2020; Teng and Huang, 2021) and contrastive based (Hjelm et al.”
BackgroundSelf-supervised learning for videos: A survey
Schiappa, M. C., Rawat, Y. S., & Shah, M. (2023) · ACM Computing Surveys · doi:10.1145/3577925 · Cited by 155 · see citing works
“SSL has been found to be effective for different domains, such as images [39, 86, 91, 131, 143, 189], natural language [84, 143, 177], graphs [113], and point-clouds [56, 73, 153].”
Background
- Authors
- Gidaris, S., Singh, P., & Komodakis, N.
- Affiliations
- École nationale des ponts et chaussées (FR)
- Venue
- arXiv
- Publisher
- arXiv
- Published
- 2018
- arXiv
- 1803.07728
- OpenAlex
- W2785325870
- Cited by
- 2,477
- References
- 35
- Citations by year
- 2026: 7 · 2025: 99 · 2024: 298 · 2023: 433 · 2022: 443 · 2021: 678 · 2020: 398 · 2019: 109 · 2018: 12
- FWCI
- 122.22
- Type
- conference paper
- Language
- en
- Open access
- green
- enpc.hal.science
- Locations
- HAL (submittedVersion)
- arXiv (Cornell University) (submittedVersion)
- HAL (submittedVersion, other-oa)
- arXiv (Cornell University)
- Funders
- Nvidia
- Agence Nationale de la Recherche
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Domain Adaptation and Few-Shot Learning
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Image and Video Retrieval Techniques
- Keywords
- unsupervised feature learning, semantic feature learning, convolutional neural networks, feature transfer, object detection, image classification, image segmentation, AlexNet
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
