The DOI matches its Crossref record; 38 of the 38 references it cites were found in a registry or an index.
Making Deep Neural Networks Robust to Label Noise: A Loss Correction Approach
Open accessAbstract
We present a theoretically grounded approach to train deep neural networks, including recurrent networks, subject to class-dependent label noise. We propose two procedures for loss correction that are agnostic to both application domain and network architecture. They simply amount to at most a matrix inversion and multiplication, provided that we know the probability of each class being corrupted into another. We further show how one can estimate these probabilities, adapting a recent technique for noise estimation to the multi-class setting, and thus providing an end-to-end framework. Extensive experiments on MNIST, IMDB, CIFAR-10, CIFAR-100 and a large scale dataset of clothing images employing a diversity of architectures - stacking dense, convolutional, pooling, dropout, batch normalization, word embedding, LSTM and residual layers - demonstrate the noise robustness of our proposals. Incidentally, we also prove that, when ReLU is the only non-linearity, the loss curvature is immune to class-dependent label noise.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474
ImageNet classification with deep convolutional neural networks
Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017) · Communications of the ACM · doi:10.1145/3065386 · Cited by 112,155
Hochreiter, S., & Schmidhuber, J. (1997) · Neural Computation · doi:10.1162/neco.1997.9.8.1735 · Cited by 102,318
Gradient-based learning applied to document recognition
Lecun, Y., Bottou, L., Bengio, Y., & Haffner, P. (1998) · Proceedings of the IEEE · doi:10.1109/5.726791 · Cited by 58,805
(2014) · Cited by 33,419
Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
Ioffe, S., & Szegedy, C. (2015) · arXiv · doi:10.48550/arxiv.1502.03167 · Cited by 27,818
Learning Multiple Layers of Features from Tiny Images
Krizhevsky, A. (2024) · doi:10.57702/zp44cu3g · Cited by 23,395
Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification
He, K., Zhang, X., Ren, S., & Sun, J. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.123 · Cited by 19,741
(2010) · Cited by 14,134
Identity mappings in deep residual networks
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-46493-0_38 · Cited by 10,741
Identity Mappings in Deep Residual Networks
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · arXiv · doi:10.48550/arxiv.1603.05027 · Cited by 10,665
(2010) · Cited by 8,261
Learning Word Vectors for Sentiment Analysis
Maas, A. L., Daly, R. E., Pham, P. T., Huang, D., Ng, A. Y., & Potts, C. (2011) · Cited by 3,751
Deep Networks with Stochastic Depth
Huang, G., Sun, Y., Liu, Z., Sedra, D., & Weinberger, K. Q. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-46493-0_39 · Cited by 2,104
Classification in the Presence of Label Noise: A Survey
Frenay, B., & Verleysen, M. (2014) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2013.2292894 · Cited by 1,777
(2015) · neural information processing systems · Cited by 1,305
Semi-supervised Sequence Learning
Dai, A. M., & Le, Q. V. (2015) · arXiv · doi:10.48550/arxiv.1511.01432 · Cited by 994
Learning from massive noisy labeled data for image classification
Xiao, T., Xia, T., Yang, Y., Huang, C., & Wang, X. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298885 · Cited by 907
Classification with Noisy Labels by Importance Reweighting
Liu, T., & Tao, D. (2016) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2015.2456899 · Cited by 782
Natarajan, N., Dhillon, I. S., Ravikumar, P., & Tewari, A. (2013) · Cited by 740
Training Deep Neural Networks on Noisy Labels with Bootstrapping
Reed, S., Lee, H., Anguelov, D., Szegedy, C., Erhan, D., & Rabinovich, A. (2014) · arXiv (Cornell University) · doi:10.48550/arxiv.1412.6596 · Cited by 667
Training Convolutional Networks with Noisy Labels
Sukhbaatar, S., Bruna, J., Paluri, M., Bourdev, L. D., & Fergus, R. (2014) · arXiv (Cornell University) · doi:10.48550/arxiv.1406.2080 · Cited by 443
Learning to Label Aerial Images from Noisy Data
Mnih, V., & Hinton, G. E. (2012) · Cited by 364
The Unreasonable Effectiveness of Noisy Data for Fine-Grained Recognition
Krause, J., Sapp, B., Howard, A., Zhou, H., Toshev, A., Duerig, T., Philbin, J., & Fei-Fei, L. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-46487-9_19 · Cited by 296
Learning Everything about Anything: Webly-Supervised Visual Concept Learning
Divvala, S. K., Farhadi, A., & Guestrin, C. (2014) · 2014 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2014.412 · Cited by 267
Making risk minimization tolerant to label noise
Ghosh, A., Manwani, N., & Sastry, P. S. (2015) · Neurocomputing · doi:10.1016/j.neucom.2014.09.081 · Cited by 202
Harvesting Image Databases from the Web
Schroff, F., Criminisi, A., & Zisserman, A. (2011) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2010.133 · Cited by 172
Random classification noise defeats all convex potential boosters
Long, P. M., & Servedio, R. A. (2010) · Machine Learning · doi:10.1007/s10994-009-5165-z · Cited by 171
On the design of loss functions for classification: theory, robustness to outliers, and savageboost
Masnadi-Shirazi, H., & Vasconcelos, N. M. (2008) · Cited by 167
Composite Binary LossesPreprint
Reid, M. D., & Williamson, R. C. (2009) · arXiv (Cornell University) · doi:10.48550/arxiv.0912.3301 · Cited by 149
Learning from Corrupted Binary Labels via Class-Probability Estimation
Menon, A. K., van Rooyen, B., Ong, C. S., & Williamson, B. (2015) · ANU Open Research (Australian National University) · Cited by 135
Self-citationMixture Proportion Estimation via Kernel Embedding of DistributionsPreprint
Ramaswamy, H. G., Scott, C. D., & Tewari, A. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1603.02501 · Cited by 94
Learning Object Categories From Internet Image Searches
Fergus, R., Fei-Fei, L., Perona, P., & Zisserman, A. (2010) · Proceedings of the IEEE · doi:10.1109/jproc.2010.2048990 · Cited by 83
Visual recognition by learning from web data: A weakly supervised domain generalization approach
Niu, L., Li, W., & Xu, D. (2015) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2015.7298894 · Cited by 80
Learning SVMs from Sloppily Labeled Data
Stempfel, G., & Ralaivola, L. (2009) · Lecture notes in computer science · doi:10.1007/978-3-642-04274-4_91 · Cited by 61
Class Proportion Estimation with Application to Multiclass Anomaly RejectionPreprint
Sanderson, T., & Scott, C. D. (2013) · arXiv (Cornell University) · doi:10.48550/arxiv.1306.5056 · Cited by 36
Learning from Binary Labels with Instance-Dependent CorruptionPreprint
Menon, A. K., van Rooyen, B., & Natarajan, N. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1605.00751 · Cited by 20
Self-citationMachine learning via transitions
van Rooyen, B. (2015) · ANU Open Research (Australian National University) · doi:10.25911/5d690a4044d0e · Cited by 2
Cited by
Generalized Cross Entropy Loss for Training Deep Neural Networks with Noisy Labels
Zhang, Z., & Sabuncu, M. R. (2018) · arXiv · doi:10.48550/arxiv.1805.07836 · Cited by 1,907
“Forward T and T̂ represent forward correction with the true and estimated confusion matrices, respectively [31].”
BackgroundMethodInfluentialTemporal Ensembling for Semi-Supervised Learning
Laine, S., & Aila, T. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1610.02242 · Cited by 1,395
Learning From Noisy Labels With Deep Neural Networks: A Survey
Song, H., Kim, M., Park, D., Shin, Y., & Lee, J.-G. (2023) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2022.3152527 · Cited by 1,308
Symmetric Cross Entropy for Robust Learning With Noisy Labels
Wang, Y., Ma, X., Chen, Z., Luo, Y., Yi, J., & Bailey, J. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00041 · Cited by 1,046
Robust Loss Functions under Label Noise for Deep Neural Networks
Ghosh, A., Kumar, H., & Sastry, P. S. (2017) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v31i1.10894 · Cited by 943
Metadata
- Authors
- Patrini, G., Rozza, A., Menon, A. K., Nock, R., & Qu, L.
- Venue
- 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)
- Publisher
- IEEE
- Published
- 2017-07
- Pages
- 2233-2241
- ISSN
- 1063-6919 (ISSN-L)
- OpenAlex
- W2964292098
- Cited by
- 1,399
- References
- 38
- FWCI
- 40.27
- Type
- conference paper
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.