The DOI matches its Crossref record; 54 of the 54 references it cites were found in a registry or an index.
Towards Making Unlabeled Data Never Hurt
Abstract
It is usually expected that learning performance can be improved by exploiting unlabeled data, particularly when the number of labeled data is limited. However, it has been reported that, in some cases existing semi-supervised learning approaches perform even worse than supervised ones which only use labeled data. For this reason, it is desirable to develop safe semi-supervised learning approaches that will not significantly reduce learning performance when unlabeled data are used. This paper focuses on improving the safeness of semi-supervised support vector machines (S3VMs). First, the S3VM-us approach is proposed. It employs a conservative strategy and uses only the unlabeled instances that are very likely to be helpful, while avoiding the use of highly risky ones. This approach improves safeness but its performance improvement using unlabeled data is often much smaller than S3VMs. In order to develop a safe and well-performing approach, we examine the fundamental assumption of S3VMs, i.e., low-density separation. Based on the observation that multiple good candidate low-density separators may be identified from training data, safe semi-supervised support vector machines (S4VMs) are here proposed. This approach uses multiple low-density separators to approximate the ground-truth decision boundary and maximizes the improvement in performance of inductive SVMs for any candidate separator. Under the assumption employed by S3VMs, it is here shown that S4VMs are provably safe and that the performance improvement using unlabeled data can be maximized. An out-of-sample extension of S4VMs is also presented. This extension allows S4VMs to make predictions on unseen instances. Our empirical study on a broad range of data shows that the overall performance of S4VMs is highly competitive with S3VMs, whereas in contrast to S3VMs which hurt performance significantly in many cases, S4VMs rarely perform worse than inductive SVMs.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Maximum Likelihood from Incomplete Data Via the EM Algorithm
Dempster, A. P., Laird, N. M., & Rubin, D. B. (1977) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/j.2517-6161.1977.tb01600.x · Cited by 50,444
Wu, Y. (1999) · Technometrics · doi:10.1080/00401706.1999.10485951 · Cited by 27,221
Algorithms for Clustering Data
Sarle, W. S. (1990) · Technometrics · doi:10.1080/00401706.1990.10484648 · Cited by 7,865
Algorithms for clustering data
Flury, B. (1989) · Journal of Statistical Planning and Inference · doi:10.1016/0378-3758(89)90033-5 · Cited by 6,432
Combining labeled and unlabeled data with co-training
Blum, A., & Mitchell, T. (1998) · Proceedings of the eleventh annual conference on Computational learning theory · doi:10.1145/279943.279962 · Cited by 5,495
(2006) · The MIT Press eBooks · doi:10.7551/mitpress/9780262033589.001.0001 · Cited by 4,280
Learning with Local and Global Consistency
Zhou, D., Bousquet, O., Lal, T. N., Weston, J., & Schölkopf, B. (2003) · MPG.PuRe (Max Planck Society) · Cited by 3,864
Semi-Supervised Learning Literature Survey
Zhu, X. (2005) · Minds at UW (University of Wisconsin) · Cited by 3,844
(2006) · Cited by 3,624
Semi-supervised learning using Gaussian fields and harmonic functions
Zhu, X., Zoubin Ghahramani, A., & Lafferty, J. (2003) · Cited by 3,509
Thermodynamical approach to the traveling salesman problem: An efficient simulation algorithm
Černý, V. (1985) · Journal of Optimization Theory and Applications · doi:10.1007/bf00940812 · Cited by 3,436
Simulated annealing: theory and applications.
(1988) · Mathematics and Computers in Simulation · doi:10.1016/0378-4754(88)90023-7 · Cited by 2,966
Text classification from labeled and unlabeled documents using EM
Nigam, K., Mccallum, A. K., Thrun, S., & Mitchell, T. (2000) · Machine Learning · doi:10.1023/a:1007692713085 · Cited by 2,736
Transductive Inference for Text Classification using Support Vector Machines
Joachims, T. (1999) · Cited by 2,723
Ensemble Methods: Foundations and Algorithms
Zhou, Z.-H. (2012) · doi:10.1201/b12207 · Cited by 2,534
Self-citationSimulated Annealing: Theory and Applications
van Laarhoven, P. J. M., & Aarts, E. H. L. (1987) · Springer eBooks · doi:10.1007/978-94-015-7744-1 · Cited by 2,385
Semi-supervised Learning by Entropy Minimization
Grandvalet, Y., & Bengio, Y. (2004) · neural information processing systems · Cited by 2,177
Ensemble Methods: Foundations and Algorithms
Zhou, Z. (2012) · Cited by 2,051
Self-citationOptimization by simulated annealing: Quantitative studies
Kirkpatrick, S. (1984) · Journal of Statistical Physics · doi:10.1007/bf01009452 · Cited by 1,805
Working Set Selection Using Second Order Information for Training Support Vector Machines
Fan, R.-E., Chen, P.-H., & Lin, C. (2005) · Journal of Machine Learning Research · doi:10.5555/1046920.1194907 · Cited by 1,403
Tri-training: exploiting unlabeled data using three classifiers
Zhou, Z.-H., & Li, M. (2005) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2005.186 · Cited by 1,171
Self-citationYuille, A. L., & Rangarajan, A. (2003) · Neural Computation · doi:10.1162/08997660360581958 · Cited by 1,151
Learning from Labeled and Unlabeled Data using Graph Mincuts
Blum, A. L., & Chawla, S. (2018) · doi:10.1184/r1/6606860 · Cited by 928
A dual coordinate descent method for large-scale linear SVM
Hsieh, C.-J., Chang, K.-W., Lin, C.-J., Keerthi, S. S., & Sundararajan, S. (2008) · Proceedings of the 25th international conference on Machine learning - ICML '08 · doi:10.1145/1390156.1390208 · Cited by 872
Semi-Supervised Support Vector Machines
Bennett, K. P., & Demiriz, A. (1998) · Cited by 826
Semi-Supervised Classification by Low Density Separation
Chapelle, O., Zien, A., & Ghahramani, R. C. Z. (2005) · Cited by 765
Large Graph Construction for Scalable Semi-Supervised Learning
Liu, W., He, J., & Chang, S. (2010) · Cited by 554
Collobert, R., Sinz, F. H., Weston, J., & Bottou, L. (2006) · doi:10.5555/1248547.1248609 · Cited by 466
Semi-supervised learning by disagreement
Zhou, Z.-H., & Li, M. (2010) · Knowledge and Information Systems · doi:10.1007/s10115-009-0209-z · Cited by 410
Self-citationAn analysis of diversity measures
Tang, E. K., Suganthan, P. N., & Yao, X. (2006) · Machine Learning · doi:10.1007/s10994-006-9449-2 · Cited by 404
Optimization Techniques for Semi-Supervised Support Vector Machines
Chapelle, O., Sindhwani, V., & Keerthi, S. S. (2008) · MPG.PuRe (Max Planck Society) · doi:10.5555/1390681.1390688 · Cited by 401
Convergence of alternating optimization
Bezdek, J. C., & Hathaway, R. J. (2003) · Neural, Parallel & Scientific Computations archive · doi:10.5555/964885.964886 · Cited by 389
A Mixture of Experts Classifier with Learning Based on Both Labelled and Unlabelled Data
Miller, D. J., & Uyar, H. S. (1996) · neural information processing systems · Cited by 292
Unlabeled data: Now it helps, now it doesn't
Singh, A., Nowak, R. D., & Zhu, X. (2008) · Cited by 210
Semi-supervised learning of mixture models
Cozman, F. G., Cohen, I. L., & Cirelo, M. C. (2003) · Cited by 201
Robust and Scalable Graph-Based Semisupervised Learning
Liu, W., Wang, J., & Chang, S.-F. (2012) · Proceedings of the IEEE · doi:10.1109/jproc.2012.2197809 · Cited by 196
Learning From Labeled And Unlabeled Data: An Empirical Study Across Techniques And Domains
Chawla, N. V., & Karakoulas, G. (2005) · Journal of Artificial Intelligence Research · doi:10.1613/jair.1509 · Cited by 185
A continuation method for semi-supervised SVMs
Chapelle, O., Chi, M., & Zien, A. (2006) · Proceedings of the 23rd international conference on Machine learning - ICML '06 · doi:10.1145/1143844.1143868 · Cited by 178
SETRED: Self-training with Editing
Li, M., & Zhou, Z.-H. (2005) · Lecture Notes in Computer Science · doi:10.1007/11430919_71 · Cited by 173
Self-citationUnsupervised and semi-supervised multi-class support vector machines
Xu, L., & Schuurmans, D. (2005) · Cited by 151
Semi-Supervised Learning via Regularized Boosting Working on Multiple Semi-Supervised Assumptions
Chen, K., & Wang, S. (2010) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2010.92 · Cited by 140
Learning from labeled and unlabeled data: An empirical study across techniques and domains
Chawla, N. V., & Karakoulas, G. (2005) · Cited by 129
Graph transduction via alternating minimization
Wang, J., Jebara, T., & Chang, S. (2008) · International Conference on Machine Learning · doi:10.1145/1390156.1390300 · Cited by 125
Branch and Bound for Semi-Supervised Support Vector Machines
Chapelle, O., Sindhwani, V., & Keerthi, S. S. (2007) · The MIT Press eBooks · doi:10.7551/mitpress/7503.003.0032 · Cited by 124
A discriminative model for semi-supervised learning
Balcan, M.-F., & Blum, A. L. (2010) · Journal of the ACM · doi:10.1145/1706591.1706599 · Cited by 119
Tighter and convex maximum margin clustering
Li, Y.-F., Tsang, I. W.-H., Kwok, J. T.-Y., et al. (2009) · Cited by 115
Self-citationConvex Methods for Transduction
Bie, T. D., & Cristianini, N. (2003) · Ghent University Academic Bibliography (Ghent University) · Cited by 112
Semi-supervised learning using label mean
Li, Y.-F., Kwok, J. T.-Y., & Zhou, Z. (2009) · International Conference on Machine Learning · doi:10.1145/1553374.1553456 · Cited by 112
Self-citationBootstrapping SVM active learning by incorporating unlabelled images for image retrieval
Wang, L., Chan, K. L., & Zhang, Z. (2003) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2003.1211412 · Cited by 96
Prototype vector machine for large scale semi-supervised learning
Zhang, K., Kwok, J. T.-Y., & Parvin, B. (2009) · International Conference on Machine Learning · doi:10.1145/1553374.1553531 · Cited by 87
Convex and Scalable Weakly Labeled SVMsPreprint
Li, Y.-F., Tsang, I. W.-H., Kwok, J. T.-Y., et al. (2013) · arXiv (Cornell University) · doi:10.5555/2567709.2567730 · Cited by 84
Self-citationMaximum margin clustering made practical
Zhang, K., Tsang, I. W.-H., & Kwok, J. T.-Y. (2007) · International Conference on Machine Learning · doi:10.1145/1273496.1273637 · Cited by 78
Improving Semi-Supervised Support Vector Machines Through Unlabeled Instances Selection
Li, Y.-F., & Zhou, Z. (2011) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v25i1.7920 · Cited by 69
Self-citationTransductive support vector machines and applications in bioinformatics for promoter recognition
Kasabov, N. K., & Pang, S. (2003) · International Conference on Neural Networks and Signal Processing · doi:10.1109/icnnsp.2003.1279199 · Cited by 60
Cited by
A survey on semi-supervised learning
van Engelen, J. E., & Hoos, H. H. (2020) · Machine Learning · doi:10.1007/s10994-019-05855-6 · Cited by 2,666
A brief introduction to weakly supervised learning
Zhou, Z.-H. (2018) · National Science Review · doi:10.1093/nsr/nwx106 · Cited by 1,899
Self-citationConcrete Problems in AI Safety
Amodei, D., Olah, C., Steinhardt, J., Christiano, P., Schulman, J., & Mané, D. (2016) · arXiv · doi:10.48550/arxiv.1606.06565 · Cited by 1,314
A survey of data fusion in smart city applications
Lau, B. P. L., Marakkalage, S. H., Zhou, Y., Hassan, N. U., Yuen, C., Zhang, M., & Tan, U.-X. (2019) · Information Fusion · doi:10.1016/j.inffus.2019.05.004 · Cited by 320
Multi-Modal Curriculum Learning for Semi-Supervised Image Classification
Gong, C., Tao, D., Maybank, S. J., Liu, W., Kang, G., & Yang, J. (2016) · IEEE Transactions on Image Processing · doi:10.1109/tip.2016.2563981 · Cited by 285
Metadata
- Authors
- Li, Y.-F., & Zhou, Z.-H.
- Venue
- IEEE Transactions on Pattern Analysis and Machine Intelligence
- Published
- 2015-01
- Volume
- 37
- Issue
- 1
- Pages
- 175-188
- ISSN
- 0162-8828 (ISSN-L)
- 2160-9292
- OpenAlex
- W2019288156
- Cited by
- 318
- References
- 54
- FWCI
- 29.71
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.