The DOI matches its Crossref record; 59 of the 59 references it cites were found in a registry or an index; 4 of the 37 sources its claims cite were checked for support.
Self-supervised Learning: A Succinct Review
Open accessAbstract
Machine learning has made significant advances in the field of image processing. The foundation of this success is supervised learning, which necessitates annotated labels generated by humans and hence learns from labelled data, whereas unsupervised learning learns from unlabeled data. Self-supervised learning (SSL) is a type of un-supervised learning that helps in the performance of downstream computer vision tasks such as object detection, image comprehension, image segmentation, and so on. It can develop generic artificial intelligence systems at a low cost using unstructured and unlabeled data. The authors of this review article have presented detailed literature on self-supervised learning as well as its applications in different domains. The primary goal of this review article is to demonstrate how images learn from their visual features using self-supervised approaches. The authors have also discussed various terms used in self-supervised learning as well as different types of learning, such as contrastive learning, transfer learning, and so on. This review article describes in detail the pipeline of self-supervised learning, including its two main phases: pretext and downstream tasks. The authors have shed light on various challenges encountered while working on self-supervised learning at the end of the article.
Overview
AI snapshot
Written from the paper’s full text.
TL;DR
This review summarizes self-supervised learning (SSL), covering learning paradigms, pretext and downstream tasks, contrastive versus non-contrastive methods, applications, datasets, and challenges. It argues SSL can exploit unlabeled data to learn useful representations for downstream vision, NLP, and healthcare tasks, while noting difficulties in designing pretext tasks and applying SSL to unstructured or multi-object domains.
Key findings
- SSL is presented as a bridge between supervised and unsupervised learning: it auto-generates labels from unlabeled data and uses two main phases, pretext tasks and downstream tasks, for representation learning.
- The review distinguishes contrastive learning, which uses positive and negative samples, from non-contrastive learning, which uses only positive sample pairs; Table 1 states contrastive learning is prone to dimensional collapse and uses complex neural networks, while non-contrastive learning has lesser dimensional collapse and simpler networks.
- Applications are reviewed across NLP, healthcare, and computer vision. Cited examples include BERT/T5, ERNIE 2.0, and ALBERT in NLP; Jamaludin et al. using 1016 subjects for spinal MRI; Kwasigroch et al. using 2000 training, 150 validation, and 600 testing images for skin lesion classification; Ghesu et al. using 100 million medical images; and Nguyen et al. using StructSeg with CT images of 60 lung cancer patients and RSNA with 17,079 images.
- The critical analysis states that SSL performance improves as dataset size grows, but SSL struggles when images contain multiple objects, as in self-driving scenes, and when images lack structure, as in satellite and medical images, where relative patch prediction and jigsaw puzzles are described as useless.
Methods
- Design: a succinct narrative literature review; no new experiments or primary empirical study are reported.
- Data/sample: synthesizes prior SSL literature and lists datasets used in computer vision, medical imaging, and NLP, including CIFAR-10 with 60,000 images, PASCAL VOC 2012 with 11,530 images, MS COCO with 328,000 images, and Kinetics 400 with 400 video clips per human action.
- Main analysis: qualitative organization and discussion of learning algorithms, SSL tasks, SSL learning techniques, applications, taxonomy, datasets, critical analysis, and future directions.
Limitations
- Stated in the paper: SSL pre-trained models often rely on ImageNet-style images with a single dominant object; multi-object scenes are hard to distinguish; satellite and medical images lack structure, making relative patch prediction and jigsaw puzzles ineffective; different augmentation methods are needed for less structured datasets; and designing a useful pretext task is the most difficult aspect of SSL.
- Evident from the text: as a succinct narrative review, it does not describe a systematic search or selection protocol, risk-of-bias assessment, or quantitative meta-analysis.
- Evident from the text: many numerical details are reported from cited secondary studies rather than independently verified by the review authors.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
31 Not checked yet · 29 Own interpretation · 14 No basis given · 3 Supported · 1 Supported so far
Every source the paper cites for this claim was checked and supports it.
In the paper
“HAR has enabled the field to address one of its most pressing issues: using unlabeled data to build reliable recognition systems with only a few labeled training samples [30].”
§ Self-supervised Learning: A Succinct Review > Self-Supervised Learning (SSL) > Self-Supervised Learning (SSL) Tasks
What the sources say
- Supportedcites [30] Haresamudram H, Essa I, Plötz T (2022) Assessing the state of self-supervised human activity recognition using wearables. arXiv preprint arXiv:2202.12938 source
The Abstract says self-supervised learning in wearables-based HAR has opened opportunities to use unlabeled data to derive reliable recognition systems when only small amounts of labeled training data are available. This directly supports the attributed claim. (LLM, conf=0.94)
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“After reviewing a large amount of literature, we discovered that as the size of the dataset grows, so does the system's performance.”
§ Self-supervised Learning: A Succinct Review > Critical analysis
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: unclear — It is unclear why the paper says it holds.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Self-Supervised Learning (SSL) is a solution to the aforementioned issues that has emerged as one of the most promising techniques that does not necessitate any expensive manual annotations.”
§ Self-supervised Learning: A Succinct Review > Introduction
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
In the paper
“Manually labeling data is a time-consuming and labor-intensive process [1].”
- Also stated in: § Motivation Behind the Work · plain
“Labeling each data point manually or with data labelling software is a costly and time-consuming task.”
§ Self-supervised Learning: A Succinct Review > Introduction
What the sources say
- Not checked yetcites [1] Albelwi S (2022) Survey on self-supervised learning: auxiliary pretext tasks and contrastive learning methods in imaging. Entropy 24(4):551. https://doi.org/10.3390/e24040551 source
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Also stated in: § Motivation Behind the Work · plain
In the paper
“Doersch et al. [18] proposed using SSL to predict a missing patch. They chose a random patch and predicted the relative position of an image's second and central patches, as shown in Fig. 9b.”
§ Self-supervised Learning: A Succinct Review > Self-Supervised Learning (SSL) > Self-Supervised Learning (SSL) Tasks
What the sources say
- Not checked yetcites [18] Doersch C, Gupta A, Efros AA (2015) Unsupervised visual representation learning by context prediction. In: 2015 IEEE International Conference on Computer Vision (ICCV). https://doi.org/10.1109/iccv.2015.167 source
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
In the paper
“When a low-capacity model, such as AlexNet, is used for pre-training, little improvement can be obtained when compared to the Resnet50 family [26].”
§ Self-supervised Learning: A Succinct Review > Self-Supervised Learning (SSL) > Types of SSL Learning Techniques
What the sources say
- Not checked yetcites [26] Krizhevsky A, Sutskever I, Hinton GE (2017) ImageNet classification with deep convolutional Neural Networks. Commun ACM 60(6):84–90. https://doi.org/10.1145/3065386 source
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
In the paper
“Devgon et al. [21] developed a self-supervised learning model to estimate an object's rotation between the desired rotation and the current rotation. They used a trained model to estimate the rotation between two depth images.”
§ Self-supervised Learning: A Succinct Review > Self-Supervised Learning (SSL) > Self-Supervised Learning (SSL) Tasks
What the sources say
- Not checked yetcites [21] Devgon S, Ichnowski J, Balakrishna A, Zhang H, Goldberg K (2020). Orienting novel 3D objects using self-supervised learning of rotation transforms. In: 2020 IEEE 16th International Conference on Automation Science and Engineering (CASE), pp 453–1460 source
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
In the paper
“Estimating the rotation is a task requiring instance discrimination [20].”
§ Self-supervised Learning: A Succinct Review > Self-Supervised Learning (SSL) > Self-Supervised Learning (SSL) Tasks
What the sources say
- Not checked yetcites [20] Feng Z, Xu C, Tao D (2019) Self-supervised representation learning by rotation feature decoupling. In: 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). https://doi.org/10.1109/cvpr.2019.01061 source
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
In the paper
“They demonstrated that self-supervised learning can perform well with large amounts of data without the need for annotations.”
§ Self-supervised Learning: A Succinct Review > Self-Supervised Learning (SSL) > Applications of SSL and Its State of the Art Work
What the sources say
- Not checked yetcites Spathis et al. [45] Spathis D, Perez-Pozuelo I, Marques-Fernandez L, Mascolo C (2022) Breaking away from labels: the promise of self-supervised machine learning in intelligent health. Patterns 3(2):1410. https://doi.org/10.1016/j.patter.2021.100410 source
boosteddirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“When dealing with less structured datasets containing medical and satellite images, a different augmentation method is required than when dealing with datasets for natural language processing.”
§ Self-supervised Learning: A Succinct Review > Critical analysis
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Rests on an assumption: Satellite and medical images lack structure, making context finding extremely difficult.
In the paper
“They demonstrated in their proposed method that SSL can perform better even when only a small number of images are labeled.”
§ Self-supervised Learning: A Succinct Review > Self-Supervised Learning (SSL) > Applications of SSL and Its State of the Art Work
What the sources say
- Not checked yetcites Kwasigroch et al. [43] Kwasigroch A, Grochowski M, Mikołajczyk A (2020) Self-supervised learning to increase the performance of skin lesion classification. Electronics 9(11):1930. https://doi.org/10.3390/electronics9111930 source
boosteddirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
In the paper
“By sensing the channel, SSL can estimate the traffic behavior.”
§ Self-supervised Learning: A Succinct Review > Self-Supervised Learning (SSL) > Applications of SSL and Its State of the Art Work
What the sources say
- Not checked yetcites Isravel et al. [25] Isravel DP, Silas S, Rajsingh EB (2021) Self-supervised learning approaches for traffic engineering in software-defined networks. In: Advances in Intelligent Systems and Computing, pp 511–522. https://doi.org/10.1007/978-981-33-6984-9_41 source
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“Because SSL can process large data sets without relying on labels, an incomplete sentence in the NLP domain can be completed with a few words.”
§ Self-supervised Learning: A Succinct Review > Critical analysis
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
In the paper
“The fairness of self-supervised learning is affected by model fine-tuning [58].”
§ Self-supervised Learning: A Succinct Review > Self-Supervised Learning (SSL) > Taxonomy of SSL
What the sources say
- Not checked yetcites [58] Ramapuram J, Busbridge D, Webb R (2021) Evaluating the fairness of fine-tuning strategies in self-supervised learning. arXiv preprint arXiv:2110.00538
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Probably not a substantial claim. 51
- It may state two separate claims as one. 21
Every source the paper cites for this claim was checked and supports it.
In the paper
“Transfer learning refers to the transfer of knowledge from one domain to another [15].”
§ Self-supervised Learning: A Succinct Review > Learning Algorithms > Transfer Learning
What the sources say
- Supportedcites [15] Weiss K, Khoshgoftaar TM, Wang D (2016) A survey of transfer learning. J Big Data 3:9. https://doi.org/10.1186/s40537-016-0043-6 source
The “Definitions of transfer learning” section defines transfer learning as using information from a source domain and task to improve a target predictive function. It also describes transferring information through instances, features, parameters, or relationships between domains, supporting the citing sentence’s broad paraphrase. (LLM, conf=0.92)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
- Supported
The sources with a verdict support this claim; the rest weren’t checked yet, have no full text we could read, or their verdicts aren’t rated yet.
In the paper
“The auxiliary pretext is primarily used to fill in missing parts of an image [6], convert it to gray scale [7], predict hidden parts, and many other tasks.”
§ Self-supervised Learning: A Succinct Review > Introduction
What the sources say
- Supportedcites [6] Pathak D, Krähenbühl P, Donahue J, Darrell T, Efros AA (2016) Context encoders: feature learning by inpainting. IEEE Conf Comput Vis Pattern Recogn 2016:2536–2544. https://doi.org/10.1109/CVPR.2016.278 source
The citation to this paper is attached to filling in missing parts of an image, not to converting images to grayscale. The Introduction describes training a context encoder to predict missing pixel values, and the Conclusion identifies its task as semantic inpainting. (LLM, conf=0.94)
- Not checked yetcites [7] Larsson G, Maire M, Shakhnarovich G (2016) Learning representations for automatic colorization. In: Computer Vision—ECCV 2016, pp 577–593. https://doi.org/10.1007/978-3-319-46493-0_35 source
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.
- Supported
Every source the paper cites for this claim was checked and supports it.
In the paper
“Li et al. [24] solved Jigsaw puzzles using the GAN model. They created a multitask pipeline for solving unpaired image jigsaw puzzles.”
§ Self-supervised Learning: A Succinct Review > Self-Supervised Learning (SSL) > Self-Supervised Learning (SSL) Tasks
What the sources say
- Supportedcites [24] Li R, Liu S, Wang G, Liu G, Zeng B (2022) Jigsawgan: auxiliary learning for solving jigsaw puzzles with generative adversarial networks. IEEE Trans Image Process 31:513–524. https://doi.org/10.1109/tip.2021.3120052 source
The paper’s opening passage (p. 1) explicitly describes JigsawGAN as a GAN-based method for solving jigsaw puzzles with unpaired images and says it uses a multi-task pipeline. The Introduction (p. 1) identifies its classification and GAN branches. (LLM, conf=0.95)
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
- Supported
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“Self-supervised approaches have dominated supervised learning.”
§ Self-supervised Learning: A Succinct Review > Conclusion and Future Directions
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. The paper may give no reason why it holds.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“When compared to supervised models, SSL has achieved remarkable success in the computer vision, NLP, and healthcare domains.”
§ Self-supervised Learning: A Succinct Review > Critical analysis
plaincorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. The paper may give no reason why it holds.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“Self-supervised learning approaches have been shown to be effective in difficult downstream tasks such as image classification, object detection, image segmentation, and other tasks with little labeled input.”
§ Self-supervised Learning: A Succinct Review > Conclusion and Future Directions
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. The paper may give no reason why it holds.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The basic idea behind using transfer learning is to transfer the information contained in a trained model on a task with a large amount of data to an objective task with less data.”
§ Self-supervised Learning: A Succinct Review > Learning Algorithms > Transfer Learning
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
None of the sources cited for this claim have been checked yet.
In the paper
“Chen et al. [35, 36] used a hybrid SSL approach to regularize the training of a text classification task.”
§ Self-supervised Learning: A Succinct Review > Self-Supervised Learning (SSL) > Applications of SSL and Its State of the Art Work
What the sources say
- Not checked yetcites [35] Chen T, Liu S, Chang S, Cheng Y, Amini L, Wang Z (2020a) Adversarial robustness: from self-supervised pre-training to fine-tuning. In: 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). https://doi.org/10.1109/cvpr42600.2020.00078 source
- Not checked yetcites [36] Chen T, Kornblith, Norouzi M, Hinton G (2020b) A simple framework for contrastive learning of visual representations. arXiv preprint arXiv:2002. 05709
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The major categories of SSL learning are Contrastive learning and non-contrastive learning.”
§ Self-supervised Learning: A Succinct Review > Self-Supervised Learning (SSL) > Types of SSL Learning Techniques
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. The paper may give no reason why it holds.
In the paper
“You and Wang [31] proposed using a view-enhanced jigsaw puzzle (VEJP) to recognize 3D human actions. VEJP captures multi-view data and forces the encoder to obtain view independent high-level features from the human skeleton.”
§ Self-supervised Learning: A Succinct Review > Self-Supervised Learning (SSL) > Self-Supervised Learning (SSL) Tasks
What the sources say
- Not checked yetcites [31] You W, Wang X (2022) View enhanced jigsaw puzzle for self-supervised feature learning in 3D human action recognition. IEEE Access 10:36385–36396. https://doi.org/10.1109/access.2022.3165040 source
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Pretext tasks, also known as secondary tasks, allow the model to learn useful feature representation information that is used to complete downstream tasks.”
§ Self-supervised Learning: A Succinct Review > Self-Supervised Learning (SSL) > Self-Supervised Learning (SSL) Tasks
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“These algorithms do not guarantee that including only unlabeled data will improve prediction performance because unlabeled data is only useful when it contains information useful for label prediction.”
§ Self-supervised Learning: A Succinct Review > Learning Algorithms > Semi-supervised Learning
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Image classification is used as a downstream task to estimate the quality of an image's features.”
§ Self-supervised Learning: A Succinct Review > Self-Supervised Learning (SSL) > Self-Supervised Learning (SSL) Tasks
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. The paper may give no reason why it holds.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“In contrastive learning one sample from the training data is used as an anchor, its augmented form is labeled as a positive sample, and the remaining examples in the training batch are labeled as negative samples.”
§ Self-supervised Learning: A Succinct Review > Self-Supervised Learning (SSL) > Types of SSL Learning Techniques
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The ConvNet model serves as the foundation of SSL, and its architecture influences the quality of visual representations learned through the pretext task.”
§ Self-supervised Learning: A Succinct Review > Self-Supervised Learning (SSL) > Types of SSL Learning Techniques
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
In the paper
“SSL differs from unsupervised learning in that it requires labels but does not require human labeling [2].”
§ Self-supervised Learning: A Succinct Review > Introduction
What the sources say
- Not checked yetcites [2] Ohri K, Kumar M (2021) Review on self-supervised image recognition using deep neural networks. Knowl-Based Syst 224:7090. https://doi.org/10.1016/j.knosys.2021.107090 source
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
In the paper
“Lan et al. [39] proposed a two-parameters reduction technique called ALBERT (A lite BERT) that use fewer parameters than traditional BERT.”
§ Self-supervised Learning: A Succinct Review > Self-Supervised Learning (SSL) > Applications of SSL and Its State of the Art Work
What the sources say
- Not checked yetcites [39] Lan Z, Chen M, Goodman S, Gimpel K, Sharma P, Soricut R (2019) Albert: a lite bert for self-supervised learning of language representations. arXiv preprint arXiv:1909.11942.
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The SSL model trains itself using one part of the input data to learn the other part of the input data.”
§ Self-supervised Learning: A Succinct Review > Self-Supervised Learning (SSL)
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim.
In the paper
“Down streaming can be accomplished in two ways: fine-tuning or using a linear classifier [25].”
§ Self-supervised Learning: A Succinct Review > Self-Supervised Learning (SSL) > Self-Supervised Learning (SSL) Tasks
What the sources say
- Not checked yetcites [25] Isravel DP, Silas S, Rajsingh EB (2021) Self-supervised learning approaches for traffic engineering in software-defined networks. In: Advances in Intelligent Systems and Computing, pp 511–522. https://doi.org/10.1007/978-981-33-6984-9_41 source
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
In the paper
“Task adaptive pre-training (TAPT) and domain adaptive pre-training (DAPT) have been proposed by Gururangan et al. [37]. TAPT continues to train it on the target task's training dataset, after pre-training RoBERTa on large-scale corpora. DAPT is still pre-training RoBERTa on datasets with minor domain differences from datasets in target tasks.”
§ Self-supervised Learning: A Succinct Review > Self-Supervised Learning (SSL) > Applications of SSL and Its State of the Art Work
What the sources say
- Not checked yetcites [37] Gururangan S, Marasović A, Swayamdipta S, Lo K, Beltagy I, Downey D, Smith NA (2020) Don’t stop pretraining: adapt language models to domains and tasks. In: Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics. https://doi.org/10.18653/v1/2020.acl-main.740 source
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“HAR is frequently used to evaluate the quality of video features learned using self-supervised methods.”
§ Self-supervised Learning: A Succinct Review > Self-Supervised Learning (SSL) > Self-Supervised Learning (SSL) Tasks
plaincorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. The paper may give no reason why it holds.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Designing an appropriate pretext task necessitates domain knowledge.”
§ Self-supervised Learning: A Succinct Review > Self-Supervised Learning (SSL) > Self-Supervised Learning (SSL) Tasks
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. The paper may give no reason why it holds.
In the paper
“Solving a jigsaw puzzle requires not only knowledge of a single patch, but also knowledge of the relationships between different patches of the same image, as shown in Fig. 9d [22].”
§ Self-supervised Learning: A Succinct Review > Self-Supervised Learning (SSL) > Self-Supervised Learning (SSL) Tasks
What the sources say
- Not checked yetcites [22] Noroozi M, Favaro P (2016) Unsupervised learning of visual representations by solving jigsaw puzzles. In: European Conference on Computer Vision, pp 69–84. Springer, Cham. source
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
In the paper
“the goal of incremental learning is to learn new knowledge from new samples [14] and solve new tasks on a continuous basis without forgetting previous tasks by using new data.”
§ Self-supervised Learning: A Succinct Review > Learning Algorithms > Increment Learning (IL)
What the sources say
- Not checked yetcites [14] Luo Y, Yin L, Bai W, Mao K (2020) An appraisal of incremental learning methods. Entropy (Basel, Switzerland) 22(11):1190. https://doi.org/10.3390/e22111190 source
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.
In the paper
“Instagram is an example of weakly-supervised learning, in which users' hash tags are used to label data [2].”
§ Self-supervised Learning: A Succinct Review > Learning Algorithms > Weakly-Supervised Learning
What the sources say
- Not checked yetcites [2] Ohri K, Kumar M (2021) Review on self-supervised image recognition using deep neural networks. Knowl-Based Syst 224:7090. https://doi.org/10.1016/j.knosys.2021.107090 source
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
In the paper
“The goal of contrastive learning is to bring semantically related samples closer together while keeping dissimilar samples separate [32].”
§ Self-supervised Learning: A Succinct Review > Self-Supervised Learning (SSL) > Types of SSL Learning Techniques
What the sources say
- Not checked yetcites [32] Bhattacharjee A, Karami M, Liu H (2022) Text transformations in contrastive self-supervised learning: a review. arXiv preprint arXiv:2203.12000 source
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The SSL algorithm has the ability to auto-generate the labels for un-labelled data, which converts the un-supervised model to a supervised model.”
§ Self-supervised Learning: A Succinct Review > Self-Supervised Learning (SSL)
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. The paper may give no reason why it holds. It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“SSL's most fruitful results in the field of NLP are BERT and T5.”
§ Self-supervised Learning: A Succinct Review > Self-Supervised Learning (SSL) > Applications of SSL and Its State of the Art Work
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. The paper may give no reason why it holds.
In the paper
“Sun et al. [38] proposed ERNIE 2.0, a framework for learning multiple tasks in incremental increments. They created several tasks and trained the ERNIE 2.0 model to extract lexical, syntactic, and semantic information from the training dataset. They used General Language Understanding Evaluation (GLUE) to assess the model's performance.”
§ Self-supervised Learning: A Succinct Review > Self-Supervised Learning (SSL) > Applications of SSL and Its State of the Art Work
What the sources say
- Not checked yetcites [38] Sun Y, Wang S, Li Y, Feng S, Tian H, Wu H, Wang H (2020) Ernie 2.0: a continual pre-training framework for language understanding. Proc AAAI Conf Artifi Intell 34(05):8968–8975. https://doi.org/10.1609/aaai.v34i05.6428 source
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.
In the paper
“In contrastive learning, three types of encoders are used: image encoders, momentum encoders, and dictionaries [33].”
§ Self-supervised Learning: A Succinct Review > Self-Supervised Learning (SSL) > Types of SSL Learning Techniques
What the sources say
- Not checked yetcites [33] He K, Fan H, Wu F, Xie S, Girshick R (2020) Momentum contrast for unsupervised visual representation learning. IEEE/CVF Conf Comput Vis Pattern Recogn 2020:9726–9735. https://doi.org/10.1109/CVPR42600.2020.00975 source
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Contrastive learning, on the other hand, distinguishes between augmented image features.”
§ Self-supervised Learning: A Succinct Review > Introduction
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. The paper may give no reason why it holds.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“BYOL and SimSiam proved that using these operations, the non-contrastive learning model does not suffer representation collapse.”
§ Self-supervised Learning: A Succinct Review > Self-Supervised Learning (SSL) > Types of SSL Learning Techniques
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. The paper may give no reason why it holds.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The difference between the SSL-Reg technique and TAPT and DAPT is that SSL-Reg uses a self-supervised task (for example, mask token prediction) to regularize RoBERTa fine-tuning, whereas TAPT and DAPT use separate tasks.”
§ Self-supervised Learning: A Succinct Review > Self-Supervised Learning (SSL) > Applications of SSL and Its State of the Art Work
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“FCN is a watershed moment in semantic segmentation because it employs a fully convolutional network to solve the problem.”
§ Self-supervised Learning: A Succinct Review > Self-Supervised Learning (SSL) > Self-Supervised Learning (SSL) Tasks
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. The paper may give no reason why it holds.
In the paper
“SSL employs two main concepts: auxiliary pretext tasks and contrastive learning [4].”
§ Self-supervised Learning: A Succinct Review > Introduction
What the sources say
- Not checked yetcites [4] Tao L, Wang X, Yamasaki T (2022) An improved inter-intra contrastive learning framework on self-supervised video representation. IEEE Trans Circ Syst Video Technol. https://doi.org/10.1109/tcsvt.2022.3141051 source
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
In the paper
“Zhou et al. [34] used the SSL regularization technique in NLP for text classification. They defined text classification as a key concept in NLP. Authors provided training texts that were encoded using a text encoder as input. In the encoded text, the authors defined two tasks: prediction and self-supervision.”
§ Self-supervised Learning: A Succinct Review > Self-Supervised Learning (SSL) > Applications of SSL and Its State of the Art Work
What the sources say
- Not checked yetcites [34] Zhou M, Li Z, Xie P (2021) Self-supervised regularization for text classification. Trans Assoc Comput Linguist 9:641–656. https://doi.org/10.1162/tacl_a_00389 source
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“In pretext tasks, the hidden portion of data is predicted using the visible portion.”
§ Self-supervised Learning: A Succinct Review > Self-Supervised Learning (SSL) > Self-Supervised Learning (SSL) Tasks
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“SSL tasks are divided into two categories: pretext and downstream tasks.”
§ Self-supervised Learning: A Succinct Review > Self-Supervised Learning (SSL) > Self-Supervised Learning (SSL) Tasks
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
In the paper
“The pretext task can be applied to any type of data, such as images, audio, video, and so on [5].”
§ Self-supervised Learning: A Succinct Review > Self-Supervised Learning (SSL) > Self-Supervised Learning (SSL) Tasks
What the sources say
- Not checked yetcites [5] Jaiswal A, Babu AR, Zadeh MZ, Banerjee D, Makedon F (2020) A survey on contrastive self-supervised learning. Technologies 9(1):2. https://doi.org/10.3390/technologies9010002 source
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Data augmentation can alter an image's visual appearance without changing its semantics.”
§ Self-supervised Learning: A Succinct Review > Self-Supervised Learning (SSL) > Types of SSL Learning Techniques
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. The paper may give no reason why it holds.
In the paper
“Supervised learning necessitates manual labelling of data, which slows the system's performance [8].”
§ Self-supervised Learning: A Succinct Review > Learning Algorithms > Supervised Learning
What the sources say
- Not checked yetcites [8] Alloghani M, Al-Jumeily D, Mustafina J, Hussain A, Aljaaf AJ (2019) A systematic review on supervised and unsupervised machine learning algorithms for Data Science. In: Unsupervised and Semi-Supervised Learning, pp 3–21. https://doi.org/10.1007/978-3-030-22475-2_1 source
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“SSL is considered to be the bridge between supervised learning and unsupervised learning.”
§ Self-supervised Learning: A Succinct Review > Self-Supervised Learning (SSL)
plaincorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. The paper may give no reason why it holds.
In the paper
“In pretext tasks, pseudo labels are used for representations that were generated automatically by taking into account the attributes of the dataset. These pseudo labels are then used for classification, detection, and segmentation [5].”
§ Self-supervised Learning: A Succinct Review > Introduction
What the sources say
- Not checked yetcites [5] Jaiswal A, Babu AR, Zadeh MZ, Banerjee D, Makedon F (2020) A survey on contrastive self-supervised learning. Technologies 9(1):2. https://doi.org/10.3390/technologies9010002 source
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Contrastive learning is used to learn a representation or feature space that attracts and repels representations from similar images.”
§ Self-supervised Learning: A Succinct Review > Self-Supervised Learning (SSL) > Types of SSL Learning Techniques
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“SSL has become a hot research topic for researchers since 2018, when Google introduced the natural language processing (NLP) model.”
§ Self-supervised Learning: A Succinct Review > Self-Supervised Learning (SSL) > Applications of SSL and Its State of the Art Work
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. The paper may give no reason why it holds.
In the paper
“Semi-supervised learning uses both labelled and unlabeled data to perform specific tasks [9].”
§ Self-supervised Learning: A Succinct Review > Learning Algorithms > Semi-supervised Learning
What the sources say
- Not checked yetcites [9] Engelen JEV, Hoos HH (2019) A survey on semi-supervised learning. Mach Learn 109(2):373–440. https://doi.org/10.1007/s10994-019-05855-6 source
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“This is referred to as catastrophic forgetting (CF), and it is the primary issue with incremental learning.”
§ Self-supervised Learning: A Succinct Review > Learning Algorithms > Increment Learning (IL)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
In the paper
“Weak supervision is classified into three types: incomplete supervision, inexact supervision, and inaccurate supervision [10].”
§ Self-supervised Learning: A Succinct Review > Learning Algorithms > Weakly-Supervised Learning
What the sources say
- Not checked yetcites [10] Zhou ZH (2017) A brief introduction to weakly supervised learning. Natl Sci Rev 5(1):44–53. https://doi.org/10.1093/nsr/nwx106 source
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Non-contrastive learning techniques only depends on positive sample pairs which means that the training data contains only related representations”
§ Self-supervised Learning: A Succinct Review > Self-Supervised Learning (SSL) > Types of SSL Learning Techniques
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
In the paper
“The findings explain in their article that SSL has the ability to solve the problems caused by supervised learning.”
§ Self-supervised Learning: A Succinct Review > Self-Supervised Learning (SSL) > Applications of SSL and Its State of the Art Work
What the sources say
- Not checked yetcites Chowdhury et al. [40] Chowdhury A, Rosenthal J, Waring J, Umeton R (2021) Applying self-supervised learning to medicine: review of the State of the art and medical implementations. Informatics 8(3):59. https://doi.org/10.3390/informatics8030059 source
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“SSL has been widely used in computer vision applications such as object detection, image classification, graph classification, visual question answering, and so on.”
§ Self-supervised Learning: A Succinct Review > Self-Supervised Learning (SSL) > Applications of SSL and Its State of the Art Work
plaincorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“If multiple data modalities from other sensors are available, the constraints between them can be used to train networks to learn features.”
§ Self-supervised Learning: A Succinct Review > Conclusion and Future Directions
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. The paper may give no reason why it holds.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“Only a few studies have examined learning multiple pretext tasks for self-supervised feature learning.”
§ Self-supervised Learning: A Succinct Review > Conclusion and Future Directions
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. The paper may give no reason why it holds.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“Various pretext tasks generate different supervision signals, which can help the network learn more typical characteristics.”
§ Self-supervised Learning: A Succinct Review > Conclusion and Future Directions
plainconditional- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. The paper may give no reason why it holds.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“In most existing self-supervised visual feature learning algorithms, ConvNet is trained to solve one pretext task.”
§ Self-supervised Learning: A Succinct Review > Conclusion and Future Directions
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. The paper may give no reason why it holds.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“To achieve the highest level of performance, downstream tasks in SSL use fine-tuned models.”
§ Self-supervised Learning: A Succinct Review > Self-Supervised Learning (SSL) > Taxonomy of SSL
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. The paper may give no reason why it holds.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“Self-supervised learning has attracted many researchers for its excellent data efficiency. In this method, fewer labels and smaller samples are used to learn more by incorporating a neural network.”
§ Self-supervised Learning: A Succinct Review > Self-Supervised Learning (SSL) > Taxonomy of SSL
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Later words can be completed by understanding the semantics of the previous sentence.”
§ Self-supervised Learning: A Succinct Review > Critical analysis
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. The paper may give no reason why it holds.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“The majority of self-supervised visual feature learning approaches currently available are focused on learning features for a single modality.”
§ Self-supervised Learning: A Succinct Review > Conclusion and Future Directions
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. The paper may give no reason why it holds.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“These datasets are collected for self-supervised training without the use of human-annotated labels.”
§ Self-supervised Learning: A Succinct Review > Sources of Datasets
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
In the paper
“Object detection necessitates dense labeling, which is a time-consuming process,the FSOD method attempts to recognize previously unseen object classes based on a few labels.”
§ Self-supervised Learning: A Succinct Review > Self-Supervised Learning (SSL) > Applications of SSL and Its State of the Art Work
What the sources say
- Not checked yetcites Huang et al. [48] Huang G, Laradji I, Vazquez D, Lacoste-Julien S, Rodriguez P (2021) A survey of self-supervised and few-shot object detection. arXiv preprint arXiv:2110.14711 source
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“As a result, approaches like relative patch prediction and jigsaw puzzles are useless when dealing with such images.”
§ Self-supervised Learning: A Succinct Review > Critical analysis
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: unclear — It is unclear why the paper says it holds.
It may state two separate claims as one.
- Rests on an assumption: Satellite and medical images lack structure, making context finding extremely difficult.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“When combined with other learning methods, SSL can achieve greater success.”
§ Self-supervised Learning: A Succinct Review > Conclusion and Future Directions
plainconditional- Asserted: passes — The authors put it forward as true.
- Checkable: fails — It may be too vague to check against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. It may be too vague to check against evidence. The paper’s conclusions may not depend on it. The paper may give no reason why it holds.
In the paper
“The data curation process must be automated to reduce manual input, and the number of labels required for good performance must be reduced [57].”
§ Self-supervised Learning: A Succinct Review > Self-Supervised Learning (SSL) > Taxonomy of SSL
What the sources say
- Not checked yetcites [57] Taherkhani F, Dabouei A, Soleymani S, Dawson J, Nasrabadi NM (2021) Self-supervised Wasserstein pseudo-labeling for semi-supervised image classification. In: 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp 12262–12272. https://doi.org/10.1109/CVPR46437.2021.01209. source
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.
The paper relies on this without support.
Unlabeled data is abundant and easily available.
“Unlabeled data, on the other hand, is available in plenty and easily.”
The paper relies on this without support.
Most artificial intelligence techniques require labelled datasets to make predictions.
“Most artificial intelligence techniques require labelled datasets to make predictions.”
The paper relies on this without support.
A large dataset is required to train machines to predict.
“A large dataset is required to train machines to predict.”
The paper relies on this without support.
Humans can learn faster than machines by observing and sensing their surroundings.
“One important distinction between humans and machines is that humans can learn faster than machines by observing and sensing their surroundings.”
The paper relies on this without support.
Machines can take hours or even days to simulate.
“Machines, on the other hand, can take hours or even days to simulate.”
The paper relies on this without support.
The quality of features increases as the number of layers in a network increases.
“The quality of features increases as the number of layers in a network increases.”
The paper relies on this without support.
When the domain gap between self-supervised pre-training and the downstream task is smaller, performance on the downstream task is usually better.
“When the domain gap between self-supervised pre-training and the downstream task is smaller, performance on the downstream task is usually better.”
The paper relies on this without support.
The vast majority of self-supervised pre-trained models, such as those trained on ImageNet, are trained on images with a single dominant object.
“The vast majority of self-supervised pre-trained models, such as those in the ImageNet dataset, are trained on images with a single dominant object.”
The paper relies on this without support.
Scenes in applications such as self-driving cars contain several items, making it difficult to distinguish between two similar scenes.
“The scene in applications such as self-driving cars contains several items, making it difficult to distinguish between two similar scenes.”
The paper relies on this without support.
Satellite and medical images lack structure, making context finding extremely difficult.
“Finding context in satellite and medical images is extremely difficult due to their lack of structure.”
The paper relies on this without support.
Unlabeled data is freely available in vast amounts.
“They use the vast amounts of unlabeled data that are freely available.”
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Advances in NLU have been crucial in improving AI's proficiency in interpreting and analyzing language across a spectrum of contexts, ranging from straightforward conversational exchanges to intricate textual data [190], [192], [193].”
From From Google Gemini to OpenAI Q* (Q-Star): A Survey on Reshaping the Generative Artificial Intelligence (AI) Research Landscape (2025) · cites this paper as [193]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
Each contradiction between two passages listed here was confirmed by an independent check.
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
“HMDB51 | 6849 | 51 each category contains 101 clips”
§ Self-supervised Learning: A Succinct Review > Sources of Datasets
Reported 6849 · recomputed 5151
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ Self-supervised Learning: A Succinct Review > Self-Supervised Learning (SSL) > Applications of SSL and Its State of the Art Work
“Chen et al. [35, 36] used a hybrid SSL approach to regularize the training of a text classification task.”
§ Self-supervised Learning: A Succinct Review > Self-Supervised Learning (SSL) > Applications of SSL and Its State of the Art Work
“Chen et al. [35, 36] | Machine learning | Propsed Contrastive learning framework for visual representation | ImageNet”
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ Self-supervised Learning: A Succinct Review > Self-Supervised Learning (SSL) > Applications of SSL and Its State of the Art Work
“PASCAL VOC 2007(20,044 testing images and 19,808 testing images)”
§ Self-supervised Learning: A Succinct Review > Sources of Datasets
“PASCAL VOC 2007 | 9963 images | 20 | Computer vision”
Citations
Cites
ImageNet classification with deep convolutional neural networks
Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017) · Communications of the ACM · doi:10.1145/3065386 · Cited by 112,155
You Only Look Once: Unified, Real-Time Object Detection
Redmon, J., Divvala, S., Girshick, R., & Farhadi, A. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.91 · Cited by 53,803
Rich Feature Hierarchies for Accurate Object Detection and Semantic Segmentation
Girshick, R., Donahue, J., Darrell, T., & Malik, J. (2014) · 2014 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2014.81 · Cited by 32,684
Pan, S. J., & Yang, Q. (2010) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2009.191 · Cited by 23,387
Momentum Contrast for Unsupervised Visual Representation Learning
He, K., Fan, H., Wu, Y., Xie, S., & Girshick, R. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.00975 · Cited by 13,347
Weiss, K., Khoshgoftaar, T. M., & Wang, D. (2016) · Journal of Big Data · doi:10.1186/s40537-016-0043-6 · Cited by 6,260
Context Encoders: Feature Learning by Inpainting
Pathak, D., Krahenbuhl, P., Donahue, J., Darrell, T., & Efros, A. A. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.278 · Cited by 5,691
ALBERT: A Lite BERT for Self-supervised Learning of Language Representations
Lan, Z., Chen, M., Goodman, S., Gimpel, K., Sharma, P., & Soricut, R. (2019) · arXiv · doi:10.48550/arxiv.1909.11942 · Cited by 4,308
Zhang, R., Isola, P., & Efros, A. A. (2016) · Lecture notes in computer science · doi:10.1007/978-3-319-46487-9_40 · Cited by 3,417
Unsupervised Learning of Visual Representations by Solving Jigsaw Puzzles
Noroozi, M., & Favaro, P. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-46466-4_5 · Cited by 2,775
Research works printed but not linked: 2
- Orhan, AE, Gupta VV, Lake BM (2007) Self-supervised learning through the eues of a child 2020, arXiv e-prints, arXiv-2007
- Chen T, Kornblith, Norouzi M, Hinton G (2020b) A simple framework for contrastive learning of visual representations. arXiv preprint arXiv:2002. 05709
Cited by
Sorted by citations
Zhao, Z., Alzubaidi, L., Zhang, J., Duan, Y., & Gu, Y. (2023) · Expert Systems with Applications · doi:10.1016/j.eswa.2023.122807 · Cited by 486
Vision-language models for medical report generation and visual question answering: a review
Hartsock, I., & Rasool, G. (2024) · Frontiers in Artificial Intelligence · doi:10.3389/frai.2024.1430984 · Cited by 298
Ten deep learning techniques to address small data problems with remote sensing
Safonova, A. N., Ghazaryan, G., Stiller, S., Main-Knorn, M., Nendel, C., & Ryo, M. (2023) · International Journal of Applied Earth Observation and Geoinformation · doi:10.1016/j.jag.2023.103569 · Cited by 242
A Machine Learning-Oriented Survey on Tiny Machine Learning
Capogrosso, L., Cunico, F., Cheng, D. S., Fummi, F., & Cristani, M. (2024) · IEEE Access · doi:10.1109/access.2024.3365349 · Cited by 209
Artificial intelligence−powered electrochemical sensor: Recent advances, challenges, and prospects
Nashruddin, S. N. A. B. M., Salleh, F. H. M., Yunus, R. M., & Zaman, H. B. (2024) · Heliyon · doi:10.1016/j.heliyon.2024.e37964 · Cited by 181
Metadata
- Authors
- Rani, V., Nabi, S. T., Kumar, M., Mittal, A., & Kumar, K.
- Affiliations
- Maharaja Ranjit Singh Punjab Technical University (IN)
- Panjab University (IN)
- Corresponding author
- Kumar, Munish
- Venue
- Archives of Computational Methods in Engineering
- Published
- 2023-01-20
- Volume
- 30
- Issue
- 4
- Pages
- 2761-2775
- ISSN
- 1134-3060 (ISSN-L)
- 1886-1784
- PMC
- PMC9857922
- OpenAlex
- W4317542939
- Semantic Scholar
- 0a1c21e12d9dea630ae10459a8ca8a25dc8bfe8c
- Cited by
- 295
- References
- 52
- Citations by year
- 2026: 77 · 2025: 133 · 2024: 71 · 2023: 14
- FWCI
- 6.17
- Type
- journal article
- Language
- en
- Open access
- bronze
- doi.org
- Licence
- Locations
- Archives of Computational Methods in Engineering (publishedVersion)
- PubMed Central (publishedVersion)
- PubMed (publishedVersion)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Domain Adaptation and Few-Shot Learning
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
- Physical Sciences › Computer Science › Artificial Intelligence › Machine Learning and Algorithms
- Keywords
- self-supervised learning, contrastive learning, transfer learning, pretext tasks, downstream tasks, unsupervised learning, image learning, visual features, computer vision, object detection, image segmentation, image understanding
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
A language model drew the claims from the paper on 2026-10-08, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-10-08 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Cited sources are checked with a language model’s help (gpt-6-sol).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
