The DOI matches its Crossref record; 51 of the 51 references it cites were found in a registry or an index.
Smooth Neighbors on Teacher Graphs for Semi-Supervised Learning
Open accessAbstract
The recently proposed self-ensembling methods have achieved promising results in deep semi-supervised learning, which penalize inconsistent predictions of unlabeled data under different perturbations. However, they only consider adding perturbations to each single data point, while ignoring the connections between data samples. In this paper, we propose a novel method, called Smooth Neighbors on Teacher Graphs (SNTG). In SNTG, a graph is constructed based on the predictions of the teacher model, i.e., the implicit self-ensemble of models. Then the graph serves as a similarity measure with respect to which the representations of "similar" neighboring points are learned to be smooth on the low-dimensional manifold. We achieve state-of-the-art results on semi-supervised learning benchmarks. The error rates are 9.89%, 3.99% for CIFAR-10 with 4000 labels, SVHN with 500 labels, respectively. In particular, the improvements are significant when the labels are fewer. For the non-augmented MNIST with only 20 labels, the error rate is reduced from previous 4.81% to 1.36%. Our method also shows robustness to noisy labels.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
ImageNet classification with deep convolutional neural networks
Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017) · Communications of the ACM · doi:10.1145/3065386 · Cited by 112,155
Adam: A Method for Stochastic Optimization
Kingma, D. P., & Ba, J. (2014) · arXiv · doi:10.48550/arxiv.1412.6980 · Cited by 98,946
(2008) · Journal of Machine Learning Research · Cited by 33,051
A Global Geometric Framework for Nonlinear Dimensionality Reduction
Tenenbaum, J. B., Silva, V. D., & Langford, J. C. (2000) · Science · doi:10.1126/science.290.5500.2319 · Cited by 13,585
TensorFlow: Large-Scale Machine Learning on Heterogeneous Distributed Systems
Abadi, M., Agarwal, A., Barham, P., Brevdo, E., Chen, Z., Citro, C., Corrado, G. S., Davis, A., Dean, J., Devin, M., et al. (2016) · arXiv · doi:10.48550/arxiv.1603.04467 · Cited by 9,408
Laplacian Eigenmaps for Dimensionality Reduction and Data Representation
Belkin, M., & Niyogi, P. (2003) · Neural Computation · doi:10.1162/089976603321780317 · Cited by 7,621
Improved Techniques for Training GANs
Salimans, T., Goodfellow, I., Zaremba, W., Cheung, V., Radford, A., & Chen, X. (2016) · arXiv · doi:10.48550/arxiv.1606.03498 · Cited by 6,965
Learning Deep Architectures for AI
Bengio, Y. (2009) · Foundations and Trends® in Machine Learning · doi:10.1561/2200000006 · Cited by 6,938
DeepFace: Closing the Gap to Human-Level Performance in Face Verification
Taigman, Y., Yang, M., Ranzato, M., & Wolf, L. (2014) · 2014 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2014.220 · Cited by 6,542
Improving neural networks by preventing co-adaptation of feature detectors
Hinton, G. E., Srivastava, N., Krizhevsky, A., Sutskever, I., & Salakhutdinov, R. R. (2012) · arXiv · doi:10.48550/arxiv.1207.0580 · Cited by 6,382
Large-Scale Machine Learning with Stochastic Gradient Descent
Bottou, L. (2010) · Proceedings of COMPSTAT'2010 · doi:10.1007/978-3-7908-2604-3_16 · Cited by 5,795
Combining labeled and unlabeled data with co-training
Blum, A., & Mitchell, T. (1998) · Proceedings of the eleventh annual conference on Computational learning theory · doi:10.1145/279943.279962 · Cited by 5,495
Dimensionality Reduction by Learning an Invariant Mapping
Hadsell, R., Chopra, S., & LeCun, Y. (2006) · 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition - Volume 2 (CVPR'06) · doi:10.1109/cvpr.2006.100 · Cited by 5,336
Learning Deep Architectures for AI
Bengio, Y. (2009) · now publishers, Inc. eBooks · doi:10.1561/9781601982957 · Cited by 4,941
CNN Features Off-the-Shelf: An Astounding Baseline for Recognition
Razavian, A. S., Azizpour, H., Sullivan, J., & Carlsson, S. (2014) · 2014 IEEE Conference on Computer Vision and Pattern Recognition Workshops · doi:10.1109/cvprw.2014.131 · Cited by 4,451
Tarvainen, A., & Valpola, H. (2017) · arXiv · doi:10.48550/arxiv.1703.01780 · Cited by 4,275
Learning a Similarity Metric Discriminatively, with Application to Face Verification
Chopra, S., Hadsell, R., & LeCun, Y. (2005) · 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05) · doi:10.1109/cvpr.2005.202 · Cited by 4,036
Learning with Local and Global Consistency
Zhou, D., Bousquet, O., Lal, T. N., Weston, J., & Schölkopf, B. (2003) · MPG.PuRe (Max Planck Society) · Cited by 3,864
Semi-Supervised Learning Literature Survey
Zhu, X. (2005) · Minds at UW (University of Wisconsin) · Cited by 3,844
Multi-column deep neural networks for image classification
Ciresan, D., Meier, U., & Schmidhuber, J. (2012) · 2012 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2012.6248110 · Cited by 3,720
(2006) · Cited by 3,624
Semi-supervised learning using Gaussian fields and harmonic functions
Zhu, X., Zoubin Ghahramani, A., & Lafferty, J. (2003) · Cited by 3,509
Virtual Adversarial Training: A Regularization Method for Supervised and Semi-Supervised Learning
Miyato, T., Maeda, S.-I., Koyama, M., & Ishii, S. (2019) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2018.2858821 · Cited by 2,843
Transductive Inference for Text Classification using Support Vector Machines
Joachims, T. (1999) · Cited by 2,723
SIGNATURE VERIFICATION USING A “SIAMESE” TIME DELAY NEURAL NETWORK
BROMLEY, J., BENTZ, J. W., BOTTOU, L., GUYON, I., LECUN, Y., MOORE, C., SÄCKINGER, E., & SHAH, R. (1994) · Series in Machine Perception and Artificial Intelligence · doi:10.1142/9789812797926_0003 · Cited by 2,350
Semi-supervised Learning by Entropy Minimization
Grandvalet, Y., & Bengio, Y. (2004) · Cited by 1,582
Learning from labeled and unlabeled data with label propagation
Zhu, X., & Ghahramani, Z. (2002) · Cited by 1,534
Semi-Supervised Learning with Deep Generative Models
Kingma, D. P., Rezende, D. J., Mohamed, S., & Welling, M. (2014) · UvA-DARE (University of Amsterdam) · doi:10.48550/arxiv.1406.5298 · Cited by 1,415
Temporal Ensembling for Semi-Supervised Learning
Laine, S., & Aila, T. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1610.02242 · Cited by 1,395
Semi-Supervised Learning (Chapelle, O. et al., Eds.; 2006) [Book reviews]
Chapelle, O., Scholkopf, B., & Zien, E. (2009) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.2009.2015974 · Cited by 1,066
Analyzing the effectiveness and applicability of co-training
Nigam, K., & Ghani, R. (2000) · Proceedings of the ninth international conference on Information and knowledge management · doi:10.1145/354756.354805 · Cited by 1,005
Weight Normalization: A Simple Reparameterization to Accelerate Training of Deep Neural Networks
Salimans, T., & Kingma, D. P. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1602.07868 · Cited by 883
Davison, M. L., & Sireci, S. G. (2000) · Handbook of Applied Multivariate Statistics and Mathematical Modeling · doi:10.1016/b978-012691360-6/50013-6 · Cited by 817
Semi-Supervised Self-Training of Object Detection Models
Rosenberg, C., Hebert, M., & Schneiderman, H. (2005) · 2005 Seventh IEEE Workshops on Applications of Computer Vision (WACV/MOTION'05) - Volume 1 · doi:10.1109/acvmot.2005.107 · Cited by 796
Semi-Supervised Classification by Low Density Separation
Chapelle, O., Zien, A., & Ghahramani, R. C. Z. (2005) · Cited by 765
Training Deep Neural Networks on Noisy Labels with Bootstrapping
Reed, S., Lee, H., Anguelov, D., Szegedy, C., Erhan, D., & Rabinovich, A. (2014) · arXiv (Cornell University) · doi:10.48550/arxiv.1412.6596 · Cited by 667
CNN Features off-the-shelf: an Astounding Baseline for Recognition
Razavian, A. S., Azizpour, H., Sullivan, J., & Carlsson, S. (2014) · arXiv · doi:10.48550/arxiv.1403.6382 · Cited by 607
Adversarially Learned Inference
Dumoulin, V., Belghazi, M. I., Poole, B., Mastropietro, O., Lamb, A., Arjovsky, M., & Courville, A. C. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1606.00704 · Cited by 579
Regularization With Stochastic Transformations and Perturbations for Deep Semi-Supervised Learning
Sajjadi, M. S. M., Javanmardi, M., & Taşdizen, T. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1606.04586 · Cited by 532
Semi-Supervised Learning with Ladder Networks
Rasmus, A., Harri Valpola, B., Honkala, M., Berglund, M., & Raiko, T. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1507.02672 · Cited by 527
Deep learning via semi-supervised embedding
Weston, J., Ratle, F., & Collobert, R. (2008) · Proceedings of the 25th international conference on Machine learning - ICML '08 · doi:10.1145/1390156.1390303 · Cited by 380
Principled Hybrids of Generative and Discriminative Models
Lasserre, J. A., Bishop, C. M., & Minka, T. P. (2006) · 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition - Volume 1 (CVPR'06) · doi:10.1109/cvpr.2006.227 · Cited by 335
Learning with Pseudo-Ensembles
Bachman, P., Alsharif, O., & Precup, D. (2014) · arXiv (Cornell University) · doi:10.48550/arxiv.1412.4864 · Cited by 329
Unsupervised and Semi-supervised Learning with Categorical Generative Adversarial Networks
Springenberg, J. T. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1511.06390 · Cited by 300
Unifying distillation and privileged information
Lopez-Paz, D., Bottou, L., Schölkopf, B., & Vapnik, V. (2015) · arXiv · doi:10.48550/arxiv.1511.03643 · Cited by 270
Good Semi-supervised Learning that Requires a Bad GAN
Dai, Z., Yang, Z., Yang, F., Cohen, W. W., & Salakhutdinov, R. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1705.09783 · Cited by 209
Triple Generative Adversarial Nets
Li, C., Xu, K., Zhu, J., & Zhang, B. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1703.02291 · Cited by 204
Self-citationThe Manifold Tangent Classifier
Rifai, S., Dauphin, Y., Vincent, P., Bengio, Y., & Muller, X. (2011) · Cited by 162
Dynamic Label Propagation for Semi-supervised Multi-class Multi-label Classification
Wang, B., Tu, Z., & Tsotsos, J. K. (2013) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2013.60 · Cited by 144
Efficient Graph-Based Semi-Supervised Learning of Structured Tagging Models
Subramanya, A., Petrov, S., & Pereira, F. M. B. (2010) · Cited by 112
Learning Deep Parsimonious Representations
Liao, R., Schwing, A. G., Zemel, R. S., & Urtasun, R. (2016) · neural information processing systems · Cited by 85
Cited by
A survey on semi-supervised learning
van Engelen, J. E., & Hoos, H. H. (2020) · Machine Learning · doi:10.1007/s10994-019-05855-6 · Cited by 2,666
Self-Training With Noisy Student Improves ImageNet Classification
Xie, Q., Luong, M.-T., Hovy, E., & Le, Q. V. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.01070 · Cited by 2,393
Unsupervised Data Augmentation for Consistency Training
Xie, Q., Dai, Z., Hovy, E., Luong, M.-T., & Le, Q. V. (2019) · arXiv · doi:10.48550/arxiv.1904.12848 · Cited by 1,517
Rethinking Pre-training and Self-training
Zoph, B., Ghiasi, G., Lin, T.-Y., Cui, Y., Liu, H., Cubuk, E. D., & Le, Q. V. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2006.06882 · Cited by 369
CoMatch: Semi-supervised Learning with Contrastive Graph Regularization
Li, J., Xiong, C., & Hoi, S. C. H. (2021) · 2021 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00934 · Cited by 284
Metadata
- Authors
- Luo, Y., Zhu, J., Li, M., Ren, Y., & Zhang, B.
- Venue
- 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition
- Publisher
- IEEE
- Published
- 2018-06
- Pages
- 8896-8905
- ISSN
- 1063-6919 (ISSN-L)
- OpenAlex
- W2963558289
- Cited by
- 264
- References
- 51
- FWCI
- 15.92
- Type
- conference paper
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.