Deep Semi-Supervised Learning
Abstract
Convolutional neural networks (CNNs) attain state-of-the-art performance on various classification tasks assuming a sufficiently large number of labeled training examples. Unfortunately, curating sufficiently large labeled training dataset requires human involvement, which is expensive and time consuming. Semi-supervised methods can alleviate this problem by utilizing a limited number of labeled data in conjunction with sufficiently large unlabeled data to construct a classification model. Self-training techniques are among the earliest semi-supervised methods proposed to enhance learning by utilizing unlabeled data. In this paper, we propose a deep semi-supervised learning (DSSL) self-training method that utilizes the strengths of both supervised and unsupervised learning within a single model. We measure the efficacy of the proposed method on semi-supervised visual object classification tasks using the datasets CIFAR-10, CIFAR-100, STL-10, MNIST, and SVHN. The experiments show that DSSL surpasses semi-supervised state-of-the-art methods for most of the aforementioned datasets.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
System training for semi-supervised learning combines small quantities of a fully labelled dataset with large quantities of unlabelled data.
“System training for semi-supervised learning system combines small quantities of a fully labelled dataset with large quantities of unlabelled data [147].”
From Review of Wearable Devices and Data Collection Considerations for Connected Health (2021) · cites this paper as [147]
Cites
ImageNet classification with deep convolutional neural networks
Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017) · Communications of the ACM · doi:10.1145/3065386 · Cited by 112,155 · see citing works
Gradient-based learning applied to document recognition
Lecun, Y., Bottou, L., Bengio, Y., & Haffner, P. (1998) · Proceedings of the IEEE · doi:10.1109/5.726791 · Cited by 58,805 · see citing works
ImageNet Large Scale Visual Recognition Challenge
Russakovsky, O., Deng, J., Su, H., Krause, J., Satheesh, S., Ma, S., Huang, Z., Karpathy, A., Khosla, A., Bernstein, M., et al. (2015) · International Journal of Computer Vision · doi:10.1007/s11263-015-0816-y · Cited by 39,380 · see citing works
Learning Multiple Layers of Features from Tiny Images
Krizhevsky, A. (2024) · doi:10.57702/zp44cu3g · Cited by 23,890 · see citing works
Reducing the dimensionality of data with neural networks
Hinton, G. E., & Salakhutdinov, R. R. (2006) · Science · doi:10.1126/science.1127647 · Cited by 20,928 · see citing works
Zagoruyko, S., & Komodakis, N. (2016) · Procedings of the British Machine Vision Conference 2016 · doi:10.5244/c.30.87 · Cited by 7,244 · see citing works
Reading digits in natural images with unsupervised feature learning
Netzer, Y. (2024) · doi:10.57702/e7wwo0zk · Cited by 4,261 · see citing works
Semi-Supervised Learning Literature Survey
Zhu, X. (2005) · Cited by 3,828 · see citing works
Multi-column deep neural networks for image classification
Ciresan, D. C., Meier, U., & Schmidhuber, J. (2012) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2012.6248110 · Cited by 3,760 · see citing works
An analysis of single-layer networks in unsupervised feature learning
Coates, A., Ng, A. Y., & Lee, H. (2011) · Cited by 2,760 · see citing works
(2013) · Cited by 2,056 · see citing works
Temporal Ensembling for Semi-Supervised LearningPreprint
Laine, S., & Aila, T. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1610.02242 · Cited by 1,910 · see citing works
Wide Residual NetworksPreprint
Zagoruyko, S., & Komodakis, N. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1605.07146 · Cited by 1,850 · see citing works
Big data deep learning: Challenges and perspectives
Chen, X.-W., & Lin, X. (2014) · IEEE Access · doi:10.1109/access.2014.2325029 · Cited by 1,255 · see citing works
Self-citationAnalyzing the effectiveness and applicability of co-training
Nigam, K., & Ghani, R. (2000) · ACM International Conference on Information and Knowledge Management (CIKM) · doi:10.1145/354756.354805 · Cited by 1,022 · see citing works
Semi-Supervised Learning with Ladder NetworksPreprint
Rasmus, A., Harri Valpola, B., Honkala, M., Berglund, M., & Raiko, T. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1507.02672 · Cited by 941 · see citing works
Regularization With Stochastic Transformations and Perturbations for Deep Semi-Supervised LearningPreprint
Sajjadi, M. S. M., Javanmardi, M., & Taşdizen, T. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1606.04586 · Cited by 874 · see citing works
Semi-Supervised Self-Training of Object Detection Models
Rosenberg, C., Hebert, M., & Schneiderman, H. (2005) · 2005 Seventh IEEE Workshops on Applications of Computer Vision (WACV/MOTION'05) - Volume 1 · doi:10.1109/acvmot.2005.107 · Cited by 796 · see citing works
Unsupervised and Semi-supervised Learning with Categorical Generative Adversarial NetworksPreprint
Springenberg, J. T. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1511.06390 · Cited by 578 · see citing works
Reranking and self-training for parser adaptation
McClosky, D., Charniak, E., & Johnson, M. S. (2006) · doi:10.3115/1220175.1220218 · Cited by 237 · see citing works
Semi-supervised learning of compact document representations with deep networks
Ranzato, M., & Szummer, M. (2008) · International Conference on Machine Learning · doi:10.1145/1390156.1390256 · Cited by 215 · see citing works
Learning Deep Networks from Noisy Labels with Dropout Regularization
Jindal, I., Nokleby, M. S., & Chen, X. (2016) · Proceedings · doi:10.1109/icdm.2016.0121 · Cited by 189 · see citing works
Self-citationFrom neural PCA to deep unsupervised learning
Valpola, H. (2015) · Advances in Independent Component Analysis and Learning Machines · doi:10.1016/b978-0-12-802806-3.00008-7 · Cited by 181 · see citing works
Semi-Supervised Learning with Context-Conditional Generative Adversarial NetworksPreprint
Denton, E., Gross, S., & Fergus, R. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1611.06430 · Cited by 130 · see citing works
Multi-Prediction Deep Boltzmann Machines
Goodfellow, I., Mirza, M., Courville, A. C., & Bengio, Y. (2013) · neural information processing systems · Cited by 120 · see citing works
Unsupervised Learning of Discriminative Attributes and Visual Representations
Huang, C., Loy, C. C., & Tang, X. (2016) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2016.559 · Cited by 99 · see citing works
Mutual exclusivity loss for semi-supervised deep learning
Sajjadi, M. S. M., Javanmardi, M., & Taşdizen, T. (2016) · Proceedings - International Conference on Image Processing · doi:10.1109/icip.2016.7532690 · Cited by 81 · see citing works
Weight-averaged consistency targets improve semi-supervised deep learning results.Preprint
Tarvainen, A., & Valpola, H. (2017) · arXiv (Cornell University) · Cited by 61 · see citing works
Multi-channel multi-model feature learning for face recognition
Aslan, M. S., Hailat, Z., Alafif, T. K., & Chen, X. (2016) · Pattern Recognition Letters · doi:10.1016/j.patrec.2016.11.021 · Cited by 23 · see citing works
Self-citationDeep unsupervised learning through spatial contrastingPreprint
Hoffer, E., Hubara, I., & Ailon, N. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1610.00243 · Cited by 14 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
Review of Wearable Devices and Data Collection Considerations for Connected Health
Vijayan, V., Connolly, J. P., Condell, J., McKelvey, N., & Gardiner, P. (2021) · Sensors · doi:10.3390/s21165589 · Cited by 435 · see citing works
Vijayan, V., McKelvey, N., Condell, J., Gardiner, P., & Connolly, J. (2020) · 2020 31st Irish Signals and Systems Conference (ISSC) · doi:10.1109/issc49989.2020.9180174 · Cited by 3 · see citing works
- Authors
- Hailat, Z., Komarichev, A., & Chen, X.-W.
- Affiliations
- Wayne State University (US)
- ORCID
- Komarichev, Artem
- Venue
- 2018 24th International Conference on Pattern Recognition (ICPR)
- Publisher
- IEEE
- Published
- 2018-08
- Volume
- 10
- Pages
- 2154-2159
- ISSN
- 1041-3278 (ISSN-L)
- OpenAlex
- W2902356949
- Cited by
- 13
- References
- 30
- Citations by year
- 2024: 1 · 2023: 1 · 2022: 1 · 2021: 5 · 2020: 3 · 2019: 1 · 2018: 1
- FWCI
- 1.02
- Type
- conference paper
- Language
- en
- Open access
- closed
- Locations
- Proceedings - International Conference on Pattern Recognition/Proceedings/International Conference on Pattern Recognition (publishedVersion)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Machine Learning and Algorithms
- Physical Sciences › Computer Science › Artificial Intelligence › Domain Adaptation and Few-Shot Learning
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
- Keywords
- self-training, convolutional neural networks, unlabeled data, visual object classification, semi-supervised classification, CIFAR-10, CIFAR-100, STL-10, MNIST, SVHN
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
