The DOI matches its Crossref record; 52 of the 52 references it cites were found in a registry or an index.
Classification with Noisy Labels by Importance Reweighting
Open accessAbstract
In this paper, we study a classification problem in which sample labels are randomly corrupted. In this scenario, there is an unobservable sample with noise-free labels. However, before being observed, the true labels are independently flipped with a probability ρ ∈ [0,0.5), and the random label noise can be class-conditional. Here, we address two fundamental problems raised by this scenario. The first is how to best use the abundant surrogate loss functions designed for the traditional classification problem when there is label noise. We prove that any surrogate loss function can be used for classification with noisy labels by using importance reweighting, with consistency assurance that the label noise does not ultimately hinder the search for the optimal classifier of the noise-free sample. The other is the open problem of how to obtain the noise rate ρ. We show that the rate is upper bounded by the conditional probability P(∧Y|X) of the noisy sample. Consequently, the rate can be estimated, because the upper bound can be easily reached in classification problems. Experimental results on synthetic and real datasets confirm the efficiency of our methods.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
The Nature of Statistical Learning Theory
Vapnik, V. N. (1995) · doi:10.1007/978-1-4757-2440-0 · Cited by 39,138
Regularization and Variable Selection Via the Elastic Net
Zou, H., & Hastie, T. (2005) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/j.1467-9868.2005.00503.x · Cited by 20,957
The Nature of Statistical Learning Theory
Vapnik, V. N. (2000) · Springer eBooks · doi:10.1007/978-1-4757-3264-1 · Cited by 10,696
(2006) · Cited by 3,624
Valiant, L. G. (1984) · Communications of the ACM · doi:10.1145/1968.1972 · Cited by 3,279
Smooth minimization of non-smooth functions
Nesterov, Y. (2005) · Mathematical Programming · doi:10.1007/s10107-004-0552-5 · Cited by 2,455
Rademacher and Gaussian Complexities: Risk Bounds and Structural Results
Bartlett, P. L., & Mendelson, S. (2001) · Lecture Notes in Computer Science · doi:10.1007/3-540-44581-1_15 · Cited by 1,949
Classification in the Presence of Label Noise: A Survey
Frenay, B., & Verleysen, M. (2014) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2013.2292894 · Cited by 1,777
Correntropy: Properties and Applications in Non-Gaussian Signal Processing
Liu, W., Pokharel, P. P., & Principe, J. C. (2007) · IEEE Transactions on Signal Processing · doi:10.1109/tsp.2007.896065 · Cited by 1,695
Correcting Sample Selection Bias by Unlabeled Data
Huang, J., Smola, A. J., Gretton, A., Borgwardt, K. M., & Schölkopf, B. (2007) · Advances in Neural Information Processing Systems 19 · doi:10.7551/mitpress/7503.003.0080 · Cited by 1,571
Neural network learning theoretical foundations
Anthony, M., & Bartlett, P. L. (2009) · Cited by 1,310
Foundations of Machine Learning
(2012) · Boosting · doi:10.7551/mitpress/8291.003.0006 · Cited by 1,074
Convexity, Classification, and Risk Bounds
Bartlett, P. L., Jordan, M. I., & McAuliffe, J. D. (2006) · Journal of the American Statistical Association · doi:10.1198/016214505000000907 · Cited by 1,037
Angluin, D., & Laird, P. (1988) · Machine Learning · doi:10.1007/bf00116829 · Cited by 827
Natarajan, N., Dhillon, I. S., Ravikumar, P., & Tewari, A. (2013) · Cited by 740
Direct importance estimation with model selection and its application to covariate shift adaptation
Sugiyama, M., Nakajima, S., Kashima, H., von Bünau, P., & Kawanabe, M. (2009) · Cited by 679
Maximum Correntropy Criterion for Robust Face Recognition
He, R., Zheng, W.-S., & Hu, B.-G. (2011) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2010.220 · Cited by 670
Efficient noise-tolerant learning from statistical queries
Kearns, M. (1998) · Journal of the ACM · doi:10.1145/293347.293351 · Cited by 665
Domain Adaptation Problems: A DASVM Classification Technique and a Circular Validation Strategy
Bruzzone, L., & Marconcini, M. (2010) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2009.57 · Cited by 591
Bartlett, P. L., Bousquet, O., & Mendelson, S. (2002) · Cited by 533
Covariate Shift by Kernel Mean Matching
Gretton, A., Smola, A., Huang, J., Schmittfull, M., Borgwardt, K., & Schölkopf, B. (2008) · Dataset Shift in Machine Learning · doi:10.7551/mitpress/9780262170055.003.0008 · Cited by 378
Bartlett, P. L., Bousquet, O., & Mendelson, S. (2005) · The Annals of Statistics · doi:10.1214/009053605000000282 · Cited by 372
Learning in the Presence of Malicious Errors
Kearns, M., & Li, M. (1993) · SIAM Journal on Computing · doi:10.1137/0222052 · Cited by 340
Support Vector Machines Under Adversarial Label Noise
Biggio, B., Nelson, B. A., & Laskov, P. (2011) · UNICA IRIS Institutional Research Information System (University of Cagliari) · Cited by 309
Gong, P., Zhang, C., Lu, Z., Huang, J. Z., & Ye, J. (2013) · PubMed · doi:10.48550/arxiv.1303.4434 · Cited by 272
Noise Tolerance Under Risk Minimization
Manwani, N., & Sastry, P. S. (2013) · IEEE Transactions on Cybernetics · doi:10.1109/tsmcb.2012.2223460 · Cited by 258
A Least-squares Approach to Direct Importance Estimation
KanamoriTakafumi, HidoShohei, & SugiyamaMasashi (2009) · Journal of Machine Learning Research · doi:10.5555/1577069.1755831 · Cited by 243
Trace Norm Regularization: Reformulations, Algorithms, and Multi-Task Learning
Pong, T. K., Tseng, P., Ji, S., & Ye, J. (2010) · SIAM Journal on Optimization · doi:10.1137/090763184 · Cited by 208
Semi-supervised novelty detection
Blanchard, G., Lee, G., & Scott, C. D. (2009) · doi:10.20347/wias.preprint.1471 · Cited by 186
A Two-Stage Weighting Framework for Multi-Source Domain Adaptation
Sun, Q., Chattopadhyay, R., Panchanathan, S., & Ye, J. (2011) · Cited by 176
Estimating a Kernel Fisher Discriminant in the Presence of Label Noise
Lawrence, N. D., & Schölkopf, B. (2001) · Cited by 168
Support Vector Machines are Universally Consistent
Steinwart, I. (2002) · Journal of Complexity · doi:10.1006/jcom.2002.0642 · Cited by 164
Relative Density-Ratio Estimation for Robust Distribution Comparison
Yamada, M., Suzuki, T., Kanamori, T., Hachiya, H., & Sugiyama, M. (2013) · Neural Computation · doi:10.1162/neco_a_00442 · Cited by 161
A polynomial-time algorithm for learning noisy linear threshold functions
Blum, A. L., FRIEZE, A. M., Kannan, R., & Vempala, S. (2002) · Annual Symposium on Foundations of Computer Science · doi:10.1109/sfcs.1996.548492 · Cited by 129
A Polynomial-Time Algorithm for Learning Noisy Linear Threshold Functions
Blum, A. L., FRIEZE, A. M., Kannan, R., & Vempala, S. (1998) · Algorithmica · doi:10.1007/pl00013833 · Cited by 98
Noise Tolerant Variants of the Perceptron Algorithm
Khardon, R., & Wachman, G. (2007) · Journal of Machine Learning Research · doi:10.5555/1248659.1248667 · Cited by 90
Learning linear threshold functions in the presence of classification noise
Bylander, T. (1994) · Annual Conference on Computational Learning Theory (COLT) · doi:10.1145/180139.181176 · Cited by 76
Bshouty, N. H., Eiron, N., & Kushilevitz, E. (2002) · Theoretical Computer Science · doi:10.1016/s0304-3975(01)00403-0 · Cited by 76
On the sample complexity of noise-tolerant learning
Aslam, J. A., & Decatur, S. E. (1996) · Information Processing Letters · doi:10.1016/0020-0190(96)00006-3 · Cited by 57
Learning Halfspaces with Malicious Noise
Klivans, A. R., Long, P. M., & Servedio, R. A. (2009) · Cited by 57
Learning noisy perceptrons by a perceptron in polynomial time
Cohen, E. (2002) · Annual Symposium on Foundations of Computer Science · doi:10.1109/sfcs.1997.646140 · Cited by 56
Bregman Divergences and Surrogates for Learning
Nock, R., & Nielsen, F. (2008) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2008.225 · Cited by 55
Crammer, K., & Lee, D. D. (2010) · Cited by 51
Cesa‐Bianchi, N., Shalev‐Shwartz, S., & Shamir, O. (2011) · IEEE Transactions on Information Theory · doi:10.1109/tit.2011.2164053 · Cited by 49
Learning Halfspaces with Malicious Noise
Klivans, A. R., Long, P. M., & Servedio, R. A. (2009) · Lecture notes in computer science · doi:10.1007/978-3-642-02927-1_51 · Cited by 36
Sample-efficient strategies for learning in the presence of noise
Cesa‐Bianchi, N., Dichterman, E., Fischer, P., Shamir, E., & Simon, H. U. (1999) · Journal of the ACM · doi:10.1145/324133.324221 · Cited by 34
Theoretical Analysis of Density Ratio Estimation
Kanamori, T., Suzuki, T., & Sugiyama, M. (2010) · IEICE Transactions on Fundamentals of Electronics Communications and Computer Sciences · doi:10.1587/transfun.e93.a.787 · Cited by 30
Multiple Kernel Learning from Noisy Labels by Stochastic ProgrammingPreprint
Yang, T., Mahdavi, M., Jin, R., et al. (2012) · arXiv (Cornell University) · doi:10.48550/arxiv.1206.4629 · Cited by 22
Learning large-margin halfspaces with more malicious noise
Long, P., & Servedio, R. A. (2011) · Cited by 16
Learning Kernel Perceptrons on Noisy Data Using Random Projections
Stempfel, G., & Ralaivola, L. (2007) · Lecture notes in computer science · doi:10.1007/978-3-540-75225-7_27 · Cited by 14
Bshouty, N. H., Eiron, N., & Kushilevitz, E. (1999) · Lecture notes in computer science · doi:10.1007/3-540-46769-6_17 · Cited by 12
Valiant, L. G. (2026) · Foundations of Computation and Machine Learning · doi:10.1145/3749740.3749746
Cited by
Structural Deep Network Embedding
Wang, D., Cui, P., & Zhu, W. (2016) · Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining · doi:10.1145/2939672.2939753 · Cited by 2,845
Making Deep Neural Networks Robust to Label Noise: A Loss Correction Approach
Patrini, G., Rozza, A., Menon, A. K., Nock, R., & Qu, L. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.240 · Cited by 1,399
Learning From Noisy Labels With Deep Neural Networks: A Survey
Song, H., Kim, M., Park, D., Shin, Y., & Lee, J.-G. (2023) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2022.3152527 · Cited by 1,308
Confident Learning: Estimating Uncertainty in Dataset Labels
Northcutt, C., Jiang, L., & Chuang, I. (2021) · Journal of Artificial Intelligence Research · doi:10.1613/jair.1.12125 · Cited by 795
Deep learning with noisy labels: Exploring techniques and remedies in medical image analysis
Karimi, D., Dou, H., Warfield, S. K., & Gholipour, A. (2020) · Medical Image Analysis · doi:10.1016/j.media.2020.101759 · Cited by 706
Metadata
- Authors
- Liu, T., & Tao, D.
- Venue
- IEEE Transactions on Pattern Analysis and Machine Intelligence
- Published
- 2016-03-01
- Volume
- 38
- Issue
- 3
- Pages
- 447-461
- ISSN
- 0162-8828 (ISSN-L)
- 2160-9292
- 1939-3539
- OpenAlex
- W1514928307
- Cited by
- 782
- References
- 52
- FWCI
- 56.62
- Type
- journal article
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.