The DOI matches its Crossref record; 77 of the 83 references it cites were found in a registry or an index.
A systematic study of the class imbalance problem in convolutional neural networks
Open accessAbstract
In this study, we systematically investigate the impact of class imbalance on classification performance of convolutional neural networks (CNNs) and compare frequently used methods to address the issue. Class imbalance is a common problem that has been comprehensively studied in classical machine learning, yet very limited systematic research is available in the context of deep learning. In our study, we use three benchmark datasets of increasing complexity, MNIST, CIFAR-10 and ImageNet, to investigate the effects of imbalance on classification and perform an extensive comparison of several methods to address the issue: oversampling, undersampling, two-phase training, and thresholding that compensates for prior class probabilities. Our main evaluation metric is area under the receiver operating characteristic curve (ROC AUC) adjusted to multi-class tasks since overall accuracy metric is associated with notable difficulties in the context of imbalanced data. Based on results from our experiments we conclude that (i) the effect of class imbalance on classification performance is detrimental; (ii) the method of addressing class imbalance that emerged as dominant in almost all analyzed scenarios was oversampling; (iii) oversampling should be applied to the level that completely eliminates the imbalance, whereas the optimal undersampling ratio depends on the extent of imbalance; (iv) as opposed to some classical machine learning models, oversampling does not cause overfitting of CNNs; (v) thresholding should be applied to compensate for prior class probabilities when overall number of properly classified cases is of interest.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“A memory of the past is needed to rectify past classes' scores.”
From A Comprehensive Study of Class Incremental Learning Algorithms for Visual Tasks (2020) · cites this paper as [12]
In the citing paper
“Here, we introduced a competitive method where classification bias in favor of new classes is reduced by using prior class probabilities [12].”
From A Comprehensive Study of Class Incremental Learning Algorithms for Visual Tasks (2020) · cites this paper as [12]
In the citing paper
“Finally, the highly-discussed dimension target class balance pays tribute to the technical requirements of ML140,141,144,150,159.”
From The METRIC-framework for assessing data quality for trustworthy AI in medicine: A systematic review (2024) · cites this paper as [141]
Citations
Cites
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474 · cited 1×
“And for all the cases, the classes chosen to be minority were the ones with the lowest label value.”
BackgroundVery Deep Convolutional Networks for Large-Scale Image Recognition
Simonyan, K., & Zisserman, A. (2014) · arXiv · doi:10.48550/arxiv.1409.1556 · Cited by 114,148 · cited 1×
“A standard CNN architecture comprises fully connected layers and a number of blocks consisting of convolution kernels, activation function layer and max pooling [22, 23, 24].”
MethodImageNet classification with deep convolutional neural networks
Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017) · Communications of the ACM · doi:10.1145/3065386 · Cited by 112,155
Scikit-learn: Machine Learning in Python
Pedregosa, F., Varoquaux, G., Gramfort, A., Michel, V., Thirion, B., Grisel, O., Blondel, M., Müller, A., Nothman, J., Louppe, G., et al. (2012) · arXiv · doi:10.48550/arxiv.1201.0490 · Cited by 71,016 · cited 1×
“We used a specific implementation of the ROC AUC available in scikit-learn python package [65].”
MethodGradient-based learning applied to document recognition
Lecun, Y., Bottou, L., Bengio, Y., & Haffner, P. (1998) · Proceedings of the IEEE · doi:10.1109/5.726791 · Cited by 58,805 · cited 2×
“Images The CNN model that we use for MNIST is the modern version of LeNet-5 [52].”
MethodImageNet Large Scale Visual Recognition Challenge
Russakovsky, O., Deng, J., Su, H., Krause, J., Satheesh, S., Ma, S., Huang, Z., Karpathy, A., Khosla, A., Bernstein, M., et al. (2015) · International Journal of Computer Vision · doi:10.1007/s11263-015-0816-y · Cited by 39,380 · cited 1×
“In our study, we used three benchmark datasets: MNIST [52], CIFAR-10 [53] and ImageNet Large Scale Visual Recognition Challenge (ILSVRC) 2012 [54].”
MethodSMOTE synthetic minority over-sampling technique
Chawla, N. V., Bowyer, K. W., Hall, L. O., & Kegelmeyer, W. P. (2002) · Journal of Artificial Intelligence Research · doi:10.1613/jair.953 · Cited by 32,865 · cited 3×
“For classical machine learning models it was shown that oversampling can cause overfitting, especially for minority classes [33].”
BackgroundMethodDelving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification
He, K., Zhang, X., Ren, S., & Sun, J. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.123 · Cited by 19,741 · cited 1×
“Network weights were initialized with Kaiming (also known as MSRA) initialization procedure [63].”
MethodVisualizing and understanding convolutional networks
Zeiler, M. D., & Fergus, R. (2014) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-10590-1_53 · Cited by 15,839 · cited 1×
“This property allows CNNs to learn hierarchical representations [2].”
BackgroundBatch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate ShiftPreprint
Ioffe, S. (2024) · arXiv (Cornell University) · doi:10.57702/o9raffed · Cited by 15,273
Cited by
Sorted by citations
Items 1,001–1,001 of 1,001
Differential Privacy Has Disparate Impact on Model Accuracy
Bagdasaryan, E., & Shmatikov, V. (2019) · arXiv · doi:10.48550/arxiv.1905.12101 · Cited by 219
“Other approaches to balancing accuracy across classes include oversampling [8], adversarial training [3] with a loss function that overweights the underrepresented group, cost-sensitive learning [9], and re-sampling [6].”
Background
Metadata
- Authors
- Buda, M., Maki, A., & Mazurowski, M. A.
- Affiliations
- Duke University (US)
- KTH Royal Institute of Technology (SE)
- Corresponding author
- Buda, Mateusz
- Venue
- Neural Networks
- Publisher
- Elsevier BV
- Published
- 2018-10
- Volume
- 106
- Pages
- 249-259
- ISSN
- 0893-6080 (ISSN-L)
- OpenAlex
- W2767106145
- Cited by
- 3,088
- References
- 83
- Citations by year
- 2026: 459 · 2025: 507 · 2024: 371 · 2023: 405 · 2022: 405 · 2021: 380 · 2020: 308 · 2019: 189 · 2018: 58
- FWCI
- 131.58
- Type
- journal article
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- Neural Networks (publishedVersion)
- PubMed (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- KTH Publication Database DiVA (KTH Royal Institute of Technology) (submittedVersion, other-oa)
- arXiv (Cornell University)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Imbalanced Data Classification Techniques
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
- Physical Sciences › Computer Science › Artificial Intelligence › Machine Learning and Data Classification
- Keywords
- class imbalance, convolutional neural networks, oversampling, undersampling, two-stage training, thresholding, area under the receiver operating characteristic, multi-class classification, overfitting, MNIST, CIFAR-10, ImageNet
- MeSH terms
- Machine Learning, Humans, Probability, ROC Curve, Neural Networks, Computer
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
