The DOI matches its Crossref record; 77 of the 83 references it cites were found in a registry or an index.
A systematic study of the class imbalance problem in convolutional neural networks
Open accessAbstract
In this study, we systematically investigate the impact of class imbalance on classification performance of convolutional neural networks (CNNs) and compare frequently used methods to address the issue. Class imbalance is a common problem that has been comprehensively studied in classical machine learning, yet very limited systematic research is available in the context of deep learning. In our study, we use three benchmark datasets of increasing complexity, MNIST, CIFAR-10 and ImageNet, to investigate the effects of imbalance on classification and perform an extensive comparison of several methods to address the issue: oversampling, undersampling, two-phase training, and thresholding that compensates for prior class probabilities. Our main evaluation metric is area under the receiver operating characteristic curve (ROC AUC) adjusted to multi-class tasks since overall accuracy metric is associated with notable difficulties in the context of imbalanced data. Based on results from our experiments we conclude that (i) the effect of class imbalance on classification performance is detrimental; (ii) the method of addressing class imbalance that emerged as dominant in almost all analyzed scenarios was oversampling; (iii) oversampling should be applied to the level that completely eliminates the imbalance, whereas the optimal undersampling ratio depends on the extent of imbalance; (iv) as opposed to some classical machine learning models, oversampling does not cause overfitting of CNNs; (v) thresholding should be applied to compensate for prior class probabilities when overall number of properly classified cases is of interest.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“A memory of the past is needed to rectify past classes' scores.”
From A Comprehensive Study of Class Incremental Learning Algorithms for Visual Tasks (2020) · cites this paper as [12]
In the citing paper
“Here, we introduced a competitive method where classification bias in favor of new classes is reduced by using prior class probabilities [12].”
From A Comprehensive Study of Class Incremental Learning Algorithms for Visual Tasks (2020) · cites this paper as [12]
In the citing paper
“Finally, the highly-discussed dimension target class balance pays tribute to the technical requirements of ML140,141,144,150,159.”
From The METRIC-framework for assessing data quality for trustworthy AI in medicine: A systematic review (2024) · cites this paper as [141]
Citations
Cites
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474 · cited 1×
“And for all the cases, the classes chosen to be minority were the ones with the lowest label value.”
BackgroundVery Deep Convolutional Networks for Large-Scale Image Recognition
Simonyan, K., & Zisserman, A. (2014) · arXiv · doi:10.48550/arxiv.1409.1556 · Cited by 114,148 · cited 1×
“A standard CNN architecture comprises fully connected layers and a number of blocks consisting of convolution kernels, activation function layer and max pooling [22, 23, 24].”
MethodImageNet classification with deep convolutional neural networks
Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017) · Communications of the ACM · doi:10.1145/3065386 · Cited by 112,155
Scikit-learn: Machine Learning in Python
Pedregosa, F., Varoquaux, G., Gramfort, A., Michel, V., Thirion, B., Grisel, O., Blondel, M., Müller, A., Nothman, J., Louppe, G., et al. (2012) · arXiv · doi:10.48550/arxiv.1201.0490 · Cited by 71,016 · cited 1×
“We used a specific implementation of the ROC AUC available in scikit-learn python package [65].”
MethodGradient-based learning applied to document recognition
Lecun, Y., Bottou, L., Bengio, Y., & Haffner, P. (1998) · Proceedings of the IEEE · doi:10.1109/5.726791 · Cited by 58,805 · cited 2×
“Images The CNN model that we use for MNIST is the modern version of LeNet-5 [52].”
MethodImageNet Large Scale Visual Recognition Challenge
Russakovsky, O., Deng, J., Su, H., Krause, J., Satheesh, S., Ma, S., Huang, Z., Karpathy, A., Khosla, A., Bernstein, M., et al. (2015) · International Journal of Computer Vision · doi:10.1007/s11263-015-0816-y · Cited by 39,380 · cited 1×
“In our study, we used three benchmark datasets: MNIST [52], CIFAR-10 [53] and ImageNet Large Scale Visual Recognition Challenge (ILSVRC) 2012 [54].”
MethodSMOTE synthetic minority over-sampling technique
Chawla, N. V., Bowyer, K. W., Hall, L. O., & Kegelmeyer, W. P. (2002) · Journal of Artificial Intelligence Research · doi:10.1613/jair.953 · Cited by 32,865 · cited 3×
“For classical machine learning models it was shown that oversampling can cause overfitting, especially for minority classes [33].”
BackgroundMethodDelving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification
He, K., Zhang, X., Ren, S., & Sun, J. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.123 · Cited by 19,741 · cited 1×
“Network weights were initialized with Kaiming (also known as MSRA) initialization procedure [63].”
MethodVisualizing and understanding convolutional networks
Zeiler, M. D., & Fergus, R. (2014) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-10590-1_53 · Cited by 15,839 · cited 1×
“This property allows CNNs to learn hierarchical representations [2].”
BackgroundBatch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate ShiftPreprint
Ioffe, S. (2024) · arXiv (Cornell University) · doi:10.57702/o9raffed · Cited by 15,273
Cited by
Sorted by citations
A survey on Image Data Augmentation for Deep Learning
Shorten, C., & Khoshgoftaar, T. M. (2019) · Journal of Big Data · doi:10.1186/s40537-019-0197-0 · Cited by 13,203
Class-Balanced Loss Based on Effective Number of Samples
Cui, Y., Jia, M., Lin, T.-Y., Song, Y., & Belongie, S. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00949 · Cited by 3,094
“CNNs trained on such data perform poorly for weakly represented classes [19, 15, 41, 4].”
BackgroundSurvey on deep learning with class imbalance
Johnson, J. M., & Khoshgoftaar, T. M. (2019) · Journal of Big Data · doi:10.1186/s40537-019-0192-5 · Cited by 3,032
CheXNet: Radiologist-Level Pneumonia Detection on Chest X-Rays with Deep LearningPreprint
Rajpurkar, P., Irvin, J., Zhu, K., Yang, B., Mehta, H., Duan, T., Ding, D. Y., Bagul, A., Langlotz, C. P., Shpanskaya, K. S., et al. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1711.05225 · Cited by 4,226
“We train the model using minibatches of size 16, where we oversample the minority (positive) class (Buda et al., 2017).”
MethodLearning Imbalanced Datasets with Label-Distribution-Aware Margin LossPreprint
Cao, K., Wei, C., Gaidon, A., Aréchiga, N., & Ma, T. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1906.07413 · Cited by 1,140
“, [46, 60, 5, 6] and references therein) and under-sampling the frequent classes (see, e.”
BackgroundMethodInfluentialBBN: Bilateral-Branch Network With Cumulative Learning for Long-Tailed Visual Recognition
Zhou, B., Cui, Q., Wei, X.-S., & Chen, Z.-M. (2020) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr42600.2020.00974 · Cited by 972
“, resampling [25, 1, 14, 1, 11, 2, 7, 21, 4] and cost-sensitive re-weighting [13, 28, 5, 23].”
MethodData imbalance in classification: Experimental evaluation
Thabtah, F., Hammoud, S., Kamalov, F., & Gonsalves, A. H. (2019) · Information Sciences · doi:10.1016/j.ins.2019.11.004 · Cited by 941
Augmentation for small object detection
Kisantal, M., Wojna, Z., Murawski, J., Naruniec, J., & Cho, K. (2019) · doi:10.5121/csit.2019.91713 · Cited by 903
“Oversampling We address the issue of relatively fewer images containing small objects by oversampling those images during training [4].”
BackgroundUçar, F., & Korkmaz, D. (2020) · Medical Hypotheses · doi:10.1016/j.mehy.2020.109761 · Cited by 768
Zhao, Z., Li, T., Wu, J., Sun, C., Wang, S., Yan, R., & Chen, X. (2020) · ISA Transactions · doi:10.1016/j.isatra.2020.08.010 · Cited by 723
“To address the problem of class imbalance, data-level methods and classifier-level methods can be used [54].”
MethodEnd-to-End Autonomous Driving: Challenges and Frontiers
Chen, L., Wu, P., Chitta, K., Jaeger, B., Geiger, A., & Li, H. (2024) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2024.3435937 · Cited by 826
“Various methods mitigate this issue with data processing, including over-sampling [246], [247], under-sampling [248], [249], and data augmentation [250], [251].”
MethodImbalance Problems in Object Detection: A Review
Öksüz, K., Çam, B. C., Kalkan, S., & Akbaş, E. (2020) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2020.2981890 · Cited by 628
“[139] showed that deep neural networks do not suffer from overfitting under oversampling and in fact, better performance can be achieved compared to undersampling.”
BackgroundMethodGupta, A., Anpalagan, A., Guan, L., & Khwaja, A. S. (2021) · Array · doi:10.1016/j.array.2021.100057 · Cited by 609
Mazurowski, M. A., Buda, M., Saha, A., & Bashir, M. R. (2019) · Journal of Magnetic Resonance Imaging · doi:10.1002/jmri.26534 · Cited by 554
“Some solutions have been proposed for this issue such as a higher rate of sampling of the examples from the minority class for training.(118) However, despite the solutions that have been proposed to address small dataset size”
BackgroundSelf-citationMeta-Weight-Net: Learning an Explicit Mapping For Sample WeightingPreprint
Shu, J., Xie, Q., Yi, L., Zhao, Q., Zhou, S., Xu, Z., & Meng, D. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1902.07379 · Cited by 637
Maintaining Discrimination and Fairness in Class Incremental Learning
Zhao, B., Xiao, X., Gan, G., Zhang, B., & Xia, S. (2020) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr42600.2020.01322 · Cited by 525
“Though class imbalance is an old topic and has attracted a lot of attention [4, 15, 17], multi-class imbalance learning is still an open problem [35].”
BackgroundDeep Learning for the Radiographic Detection of Periodontal Bone Loss
Krois, J., Ekert, T., Meinhold, L., Golla, T., Kharbot, B., Wittemeier, A., Dörfer, C. E., & Schwendicke, F. (2019) · Scientific Reports · doi:10.1038/s41598-019-44839-3 · Cited by 558
Long-tail learning via logit adjustmentPreprint
Menon, A. K., Jayasumana, S., Rawat, A. S., Jain, H., Veit, A., & Kumar, S. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2007.07314 · Cited by 677
“Real-world classification problems typically exhibit a long-tailed label distribution, wherein most labels are associated with only a few samples (Van Horn & Perona, 2017; Buda et al., 2017; Liu et al., 2019).”
BackgroundBalanced Meta-Softmax for Long-Tailed Visual RecognitionPreprint
Ren, J., Yu, C., sheng, S., Ma, X., Zhao, H., Yi, S., & Li, H. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2007.10740 · Cited by 449
“Specifically, re-sampling strategies [19, 5, 8, 10, 26, 2, 1] try to restore the true distributions from the imbalanced training data.”
BackgroundHidden stratification causes clinically meaningful failures in machine learning for medical imaging
Oakden-Rayner, L., Dunnmon, J., Carneiro, G., & Re, C. (2020) · Proceedings of the ACM Conference on Health, Inference, and Learning · doi:10.1145/3368555.3384468 · Cited by 411
“]. However, the cost of manual labelling is often prohibitive, appropriate augmentation transforms can be difficult to define, and oversampling an underperforming subset can cause degradation on others [14, 28, 4, 40]. As a result, medical imagery analysts have commonly begun either to use semi-automated labelling techn…”
BackgroundConvolutional Neural Networks for the Automatic Identification of Plant Diseases
Boulent, J., Foucher, S., Théau, J., & St-Charles, P.-L. (2019) · Frontiers in Plant Science · doi:10.3389/fpls.2019.00941 · Cited by 434
“The class imbalance problem affects both the convergence of the model and its ability to generalize (Buda et al., 2018).”
BackgroundIdentifying animal species in camera trap images using deep learning and citizen science
Willi, M., Pitman, R. T., Cardoso, A. W., Locke, C. M., Swanson, A., Boyer, A., Veldthuis, M., & Fortson, L. (2018) · Methods in Ecology and Evolution · doi:10.1111/2041-210x.13099 · Cited by 402
GraphSMOTE: Imbalanced Node Classification on Graphs with Graph Neural Networks
Zhao, T., Zhang, X., & Wang, S. (2021) · ACM International Conference on Web Search and Data Mining (WSDM) · doi:10.1145/3437963.3441720 · Cited by 432
“• Based on these observations, generally setting the oversampling scale set a value that can make the class balanced is a good choice, which is consistent with existing work for synthetic minority oversampling [6].”
BackgroundMethodA deep translation (GAN) based change detection network for optical and SAR remote sensing images
Li, X., Du, Z., Huang, Y., & Tan, Z. (2021) · ISPRS Journal of Photogrammetry and Remote Sensing · doi:10.1016/j.isprsjprs.2021.07.007 · Cited by 412
Sagawa, S., Koh, P. W., Hashimoto, T., & Liang, P. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1911.08731 · Cited by 1,178
“For label shifts, prior work has observed overfitting on rare labels and proposed algorithms to mitigate it (Buda et al., 2018; Cui et al., 2019; Cao et al., 2019).”
BackgroundResultInfluentialDeep learning in bioinformatics: Introduction, application, and perspective in the big data era
Li, Y., Huang, C., Ding, L., Li, Z., Pan, Y., & Gao, X. (2019) · Methods · doi:10.1016/j.ymeth.2019.04.008 · Cited by 401
Elreedy, D., Atiya, A. F., & Kamalov, F. (2023) · Machine Learning · doi:10.1007/s10994-022-06296-4 · Cited by 506
“…approaches for handling the class imbalance problem: the data level approach (Batista et al., 2004; Chawla et al., 2002; Guzmán-Ponce et al., 2021), the cost sensitive approach (Devi et al., 2022), and the algorithm level approach (Mullick et al., 2018; Buda et al., 2018; Ganaie et al., 2021).”
Deep Learning for the Radiographic Detection of Apical Lesions
Ekert, T., Krois, J., Meinhold, L., Elhennawy, K., Emara, R., Golla, T., & Schwendicke, F. (2019) · Journal of Endodontics · doi:10.1016/j.joen.2019.03.016 · Cited by 416
Dual-Sampling Attention Network for Diagnosis of COVID-19 From Community Acquired Pneumonia
Ouyang, X., Huo, J., Xia, L., Shan, F., Liu, J., Mo, Z., Yan, F., Ding, Z., Yang, Q., Song, B., et al. (2020) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2020.2995508 · Cited by 379
“To address this, some class resampling strategies have been proposed in the literature [45], [46], [47], [48], [49].”
BackgroundMethodZhang, H., Huang, L., Wu, C. Q., & Li, Z. (2020) · Computer Networks · doi:10.1016/j.comnet.2020.107315 · Cited by 362
Zhang, G., Wang, M., & Liu, K. (2019) · International Journal of Disaster Risk Science · doi:10.1007/s13753-019-00233-1 · Cited by 367
Fernando, K. R. M., & Tsokos, C. P. (2021) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2020.3047335 · Cited by 372
“Experiments conducted in [24] suggest that data sampling strategies have little effect on classification performance; however, results in [25] demonstrate that random oversampling leads to per-”
BackgroundA Comparative Performance Analysis of Data Resampling Methods on Imbalance Medical Data
Khushi, M., Shaukat, K., Alam, T. M., Hameed, I. A., Uddin, S., Luo, S., Yang, X., & Reyes, M. C. (2021) · IEEE Access · doi:10.1109/access.2021.3102399 · Cited by 335
Buda, M., Saha, A., & Mazurowski, M. A. (2019) · Computers in Biology and Medicine · doi:10.1016/j.compbiomed.2019.05.002 · Cited by 359
“Therefore, to account for this fact, we applied oversampling with data augmentation that was proved to help in training convolutional neural networks [23].”
MethodSelf-citationKellenberger, B., Marcos, D., & Tuia, D. (2018) · Remote Sensing of Environment · doi:10.1016/j.rse.2018.06.028 · Cited by 335
“While this has been shown to work well for other tasks (Buda et al., 2017), we empirically found it to cause the CNN to overfit to the small number of animal instances present in the training set.”
MethodThe class imbalance problem in deep learning
Ghosh, K., Bellinger, C., Corizzo, R., Branco, P., Krawczyk, B., & Japkowicz, N. (2022) · Machine Learning · doi:10.1007/s10994-022-06268-8 · Cited by 387
Contrastive Learning based Hybrid Networks for Long-Tailed Image Classification
Wang, P., Han, K., Wei, X.-S., Zhang, L., & Wang, L. (2021) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr46437.2021.00100 · Cited by 339
“Typical methods include data re-sampling [1, 8, 35, 4, 28], loss re-weighting [26, 7, 30, 33], margin modification [3], and data augmentation [19, 6, 29].”
BackgroundMethodImproving Calibration for Long-Tailed Recognition
Zhong, Z., Cui, J., Liu, S., & Jia, J. (2021) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr46437.2021.01622 · Cited by 339
“There are two groups of re-sampling strategies: over-sampling the tail-class images [28, 2, 3] and under-sampling the head-class images [13, 2].”
BackgroundDistribution Alignment: A Unified Framework for Long-tail Visual Recognition
Zhang, S., Li, Z., Yan, S., He, X., & Sun, J. (2021) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr46437.2021.00239 · Cited by 322
“The basic idea of resample-based methods is to over-sample the minority categories [4, 15] or to under-sample the frequent categories in the training process [10, 2].”
BackgroundMethodCReST: A Class-Rebalancing Self-Training Framework for Imbalanced Semi-Supervised Learning
Wei, C., Sohn, K., Mellina, C., Yuille, A., & Yang, F. (2021) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr46437.2021.01071 · Cited by 313
“Various solutions have been proposed to help alleviate bias, such as re-sampling [3, 4], re-”
BackgroundMethodParametric Contrastive Learning
Cui, J., Zhong, Z., Shu, L., Yu, B., & Jia, J. (2021) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00075 · Cited by 328
Effect of Dataset Size and Train/Test Split Ratios in QSAR/QSPR Multiclass Classification
Rácz, A., Bajusz, D., & Héberger, K. (2021) · Molecules · doi:10.3390/molecules26041111 · Cited by 360
“While oversampling can lead to overfitting in “classical” machine learning models, convolutional neural networks (CNNs) are less prone to overfitting [7].”
BackgroundIL2M: Class Incremental Learning With Dual Memory
Belouadah, E., & Popescu, A. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00067 · Cited by 302
“We will pursue work along the following lines: (1) test a constant complexity method such as IL2M for multitask IL to replicate real-life scenarios in which more diversified visual content is encountered, (2) enhance vanilla fine tuning by leveraging recent results which improve imbalanced learning [3] and make curric…”
BackgroundMethodPi, Y., Nath, N. D., & Behzadan, A. H. (2019) · Advanced Engineering Informatics · doi:10.1016/j.aei.2019.101009 · Cited by 301
“Considering crowdsourcing as a possible future direction of this work to scale up data collection by utilizing consumer-grade drones and low computational power, we favor pre-processing methods for data balancing which include oversampling the instances of minority classes, and under-sampling the instances of majority…”
MethodOn the class overlap problem in imbalanced data classification
Vuttipittayamongkol, P., Elyan, E., & Petrovski, A. (2020) · Knowledge-Based Systems · doi:10.1016/j.knosys.2020.106631 · Cited by 312
“Some other reviews focused on the issue of imbalanced data classification in specific contexts such as big data [24, 25], multi-class problem [24, 26] and neural networks [27, 28].”
BackgroundA Survey on Deep Learning-Based Change Detection from High-Resolution Remote Sensing Images
Jiang, H., Peng, M., Zhong, Y., Xie, H., Hao, Z., Lin, J.-M., Ma, X., & Hu, X. (2022) · Remote Sensing · doi:10.3390/rs14071552 · Cited by 313
Towards Fairness in Visual Recognition: Effective Strategies for Bias Mitigation
Wang, Z., Qinami, K., Karakozis, I. C., Genova, K., Nair, P., Hata, K., & Russakovsky, O. (2020) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr42600.2020.00894 · Cited by 296
“In relation to the former, [5, 31] study the effect of class-imbalance on learning, while [52] reveal the surprising phenomenon of bias amplification.”
BackgroundDistribution-Balanced Loss for Multi-label Classification in Long-Tailed Datasets
Wu, T., Huang, Q., Liu, Z., Wang, Y., & Lin, D. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-58548-8_10 · Cited by 289
Rethinking the Value of Labels for Improving Class-Imbalanced LearningPreprint
Yang, Y., & Xu, Z. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2006.07529 · Cited by 320
“This posts a major challenge for modern deep learning frameworks [2,5,10,20,53], where even with specialized techniques such as data re-sampling approaches [2,5,41] or class-balanced losses [7,13,26], significant performance drops still remain under extreme class imbalance.”
BackgroundMethodInfluentialTaherkhani, A., Cosma, G., & McGinnity, T. M. (2020) · Neurocomputing · doi:10.1016/j.neucom.2020.03.064 · Cited by 296
“Although class imbalance has been comprehensively studied in classical machine learning methods, it has received less attention in the context of deep learning [5].”
BackgroundMethodInfluentialNavigating the pitfalls of applying machine learning in genomics
Whalen, S., Schreiber, J. M., Noble, W. S., & Pollard, K. S. (2021) · Nature Reviews Genetics · doi:10.1038/s41576-021-00434-9 · Cited by 291
Addressing Class Imbalance in Federated Learning
Wang, L., Xu, S., Wang, X., & Zhu, Q. (2021) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v35i11.17219 · Cited by 306
A survey and analysis of intrusion detection models based on CSE-CIC-IDS2018 Big Data
Leevy, J. L., & Khoshgoftaar, T. M. (2020) · Journal Of Big Data · doi:10.1186/s40537-020-00382-x · Cited by 269
“We point out that the use of AUC as a robust, standalone metric for class imbalance has been demonstrated in several studies [95–97].”
ResultDisentangling Label Distribution for Long-tailed Visual Recognition
Hong, Y., Han, S., Choi, K., Seo, S., Kim, B., & Chang, B. (2021) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr46437.2021.00656 · Cited by 269
“Most long-tailed visual recognition methods can be divided into two strategies: modifying the data sampler to balance the class frequency during optimization [7, 21, 25, 8, 51], and modifying the class-wise weights of the classification loss to increase the importance of tail classes in terms of empirical risk minimiz…”
BackgroundMethodInfluentialFeature Space Augmentation for Long-Tailed Data
Chu, P., Bian, X., Liu, S., & Ling, H. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-58526-6_41 · Cited by 272
A survey on generative adversarial networks for imbalance problems in computer vision tasks
Sampath, V., Maurtua, I., Martín, J. J. A., & Gutierrez, A. (2021) · Journal Of Big Data · doi:10.1186/s40537-021-00414-0 · Cited by 288
“Class imbalance in the real-world image datasets is ubiquitous and can have an adverse effect on performance of ConvNets [14].”
BackgroundInfluentialA Multi-View Deep Learning Framework for EEG Seizure Detection
Yuan, Y., Xun, G., Jia, K., & Zhang, A. (2018) · IEEE Journal of Biomedical and Health Informatics · doi:10.1109/jbhi.2018.2871678 · Cited by 263
“Taking the advantage of the autoencoders, the proposed model is able to recognize instances rather than discriminating among classes, which has proved to work well for the class imbalance problem [75], [76].”
Balanced Contrastive Learning for Long-Tailed Visual Recognition
Zhu, J., Wang, Z., Chen, J., Chen, Y. P., & Jiang, Y. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.00678 · Cited by 265
“To tackle the problem of learning with imbalanced data, early methods mainly focus on re-sampling the training data [1, 2, 14, 33] or re-weighting the loss functions [9, 21, 45] to pay more attention to rare classes.”
MethodGenerative Adversarial Minority Oversampling
Mullick, S. S., Datta, S., & Das, S. (2019) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00178 · Cited by 248
“fairly persistent is such networks, severely affecting both the feature extraction as well as the classification process [48, 21, 49, 5, 22].”
BackgroundDeep Representation Learning on Long-Tailed Data: A Learnable Embedding Augmentation Perspective
Liu, J., Sun, Y., Han, C., Dou, Z., & Li, W. (2020) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr42600.2020.00304 · Cited by 244
“Recent works for feature learning on imbalanced data are mainly divided into three manners: re-sampling [1], reweighting [21], and data augmentation[3].”
BackgroundJozdani, S. E., Johnson, B. A., & Chen, D. (2019) · Remote Sensing · doi:10.3390/rs11141713 · Cited by 254
“Since the training data gathered for both the images were imbalanced, we applied the Synthetic Minority Oversampling Technique (SMOTE) [54] to artificially balance the training set, as this was found beneficial in past studies using imbalanced data [55,56] and machine learning classifiers [57].”
MethodAI meets UAVs: A survey on AI empowered UAV perception systems for precision agriculture
Su, J., Zhu, X., Li, S., & Chen, W. (2022) · Neurocomputing · doi:10.1016/j.neucom.2022.11.020 · Cited by 249
Westphal, E., & Seitz, H. (2021) · Additive manufacturing · doi:10.1016/j.addma.2021.101965 · Cited by 256
Hierarchical Incentive Mechanism Design for Federated Machine Learning in Mobile Networks
Lim, W. Y. B., Xiong, Z., Miao, C., Niyato, D. T., Yang, Q., Leung, C., & Poor, H. V. (2020) · IEEE Internet of Things Journal · doi:10.1109/jiot.2020.2985694 · Cited by 233
“In addition, the model owner may also not have a comprehensive coverage across data classes to build an effective model [6].”
BackgroundThölke, P., Mantilla-Ramos, Y.-J., Abdelhedi, H., Maschke, C., Dehgan, A., Harel, Y., Kemtur, A., Berrada, L. M., Sahraoui, M., Young, T., et al. (2023) · NeuroImage · doi:10.1016/j.neuroimage.2023.120253 · Cited by 274
Learning Memory-Augmented Unidirectional Metrics for Cross-modality Person Re-identification
Liu, J., Sun, Y., Zhu, F., Pei, H., Yang, Y., & Li, W. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.01876 · Cited by 242
Targeted Supervised Contrastive Learning for Long-Tailed Recognition
Li, T., Cao, P., Yuan, Y., Fan, L., Yang, Y., Feris, R., Indyk, P., & Katabi, D. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.00679 · Cited by 243
“Conventional methods have focused on designing class rebalancing paradigms through data re-sampling [1, 2, 5, 33] or adjusting the loss weights for different classes during training [3, 4, 9, 11, 20, 21].”
BackgroundMethodDelving into deep imbalanced regression
Yang, Y., Zha, K., Chen, Y.-C., Wang, H., & Katabi, D. (2021) · arXiv · doi:10.48550/arxiv.2102.09554 · Cited by 229
“Rather than preserving an ideal uniform distribution over each category, the data often exhibit skewed distributions with a long tail (Buda et al., 2018; Liu et al., 2019), where certain target values have significantly fewer observations.”
Park, S., Hong, Y., Heo, B., Yun, S., & Choi, J. Y. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.00676 · Cited by 230
“(1) No oversampling (vanilla); (2) Random oversampling (ROS) [47], that oversamples minority samples to balance the classes in the training data; (3) Remix [7], which oversamples minority classes by assigning higher weights to the minority labels when using Mixup [54]; (4) Feature space augmentation (FSA) [8].”
BackgroundInfluentialLuo, J., Huang, J., & Li, H. (2020) · Journal of Intelligent Manufacturing · doi:10.1007/s10845-020-01579-w · Cited by 223
Yan, P., Abdulkadir, A., Luley, P.-P., Rosenthal, M., Schatte, G. A., Grewe, B. F., & Stadelmann, T. (2024) · IEEE Access · doi:10.1109/access.2023.3349132 · Cited by 252
“Finally, a weighted loss can be used that balances the loss for the abnormal and normal class in supervised anomaly detection (Buda et al., 2018).”
BackgroundMethodMind the GAP: A balanced corpus of gendered ambiguous pronouns
Webster, K., Recasens, M., Axelrod, V., & Baldridge, J. (2018) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00240 · Cited by 223
“, 2017), and debiasing strategies include expanding and rebalancing data (Torralba and Efros, 2011; Buda, 2017; Ryu et al., 2017; Shankar et al., 2017), and balancing performance across subgroups (Dwork et al.”
Li, L., Chen, Y., Shen, Z., Zhang, X., Sang, J., Ding, Y., Yang, X., Li, ., Chen, M., Jin, C., et al. (2019) · Gastric Cancer · doi:10.1007/s10120-019-00992-2 · Cited by 219
“…between the two types of images might affect the convergence during the training phase and lead to poor generalization ability and model over-fit [24], we increased the sample size by following such image processing methods as image rotate transform, image mirror transform, image cut, image…”
MethodCS-ResNet: Cost-sensitive residual convolutional neural network for PCB cosmetic defect detection
Zhang, H., Jiang, L., & Li, C. (2021) · Expert Systems with Applications · doi:10.1016/j.eswa.2021.115673 · Cited by 213
R oot P ainter : deep learning segmentation of biological images with corrective annotation
Smith, A. G., Han, E., Petersen, J., Olsen, N. A. F., Giese, C., Athmann, M., Dresbøll, D. B., & Thorup‐Kristensen, K. (2022) · New Phytologist · doi:10.1111/nph.18387 · Cited by 225
“, 2019), or to correct for a class imbalance in the data where many examples exist of a majority class (Buda et al., 2018).”
BackgroundA Systematic Review on Imbalanced Learning Methods in Intelligent Fault Diagnosis
Ren, Z., Lin, T., Feng, K., Zhu, Y., Liu, Z., & Yan, K. L. (2023) · IEEE Transactions on Instrumentation and Measurement · doi:10.1109/tim.2023.3246470 · Cited by 222
“In other fields, researchers [33], [34], [340], [341] have analyzed the behavior of machine learning algorithms under imbalanced data and noted that the performance of algorithms under imbalanced data is related to several factors, such as the imbalance rate, sample set size, class mode, and task…”
BackgroundInfluentialDeep Learning in Data-Driven Pavement Image Analysis and Automated Distress Detection: A Review
Gopalakrishnan, K. (2018) · Data · doi:10.3390/data3030028 · Cited by 210
“This creates class imbalance, a problem that has been comprehensively studied in machine learning, but is beginning to be studied and addressed in the context of deep learning [55].”
BackgroundManagement of Thyroid Nodules Seen on US Images: Deep Learning May Match Performance of Radiologists
Buda, M., Wildman‐Tobriner, B., Hoang, J. K., Thayer, D., Tessler, F. N., Middleton, W. D., & Mazurowski, M. A. (2019) · Radiology · doi:10.1148/radiol.2019181343 · Cited by 214
Self-citationA novel lightweight deep convolutional neural network for early detection of oral cancer
Jubair, F., Al‐karadsheh, O., Malamos, D., Mahdi, S. A., Saad, Y., & Hassona, Y. M. (2021) · Oral Diseases · doi:10.1111/odi.13825 · Cited by 209
Influence-Balanced Loss for Imbalanced Visual Classification
Park, S., Lim, J., Jeon, Y., & Choi, J. Y. (2021) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00077 · Cited by 211
“This imbalance between classes can be problematic, since the model trained on such imbalanced data tends to overfit the dominant (majority) classes [18, 14, 4].”
BackgroundMethodBalanced MSE for Imbalanced Visual Regression
Ren, J., Zhang, M., Yu, C., & Liu, Z. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.00777 · Cited by 227
“Compared with imbalanced and long-tailed classification [4,12,24] that studies categorical labels, imbalanced regression focuses on continuous labels, which can be boundless, high-dimensional, and hence more challenging.”
BackgroundAnalyzing Overfitting Under Class Imbalance in Neural Networks for Image Segmentation
Li, Z., Kamnitsas, K., & Glocker, B. (2020) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2020.3046692 · Cited by 212
“Class imbalance, which has been the focus of previous works [4], [19], is a common issue in image classification and image segmentation.”
BackgroundImbalanced data preprocessing techniques for machine learning: a systematic mapping study
de Vargas, V. W., Aranda, J. A. S., Costa, R. D. S., Pereira, P., & Barbosa, J. L. V. (2022) · Knowledge and Information Systems · doi:10.1007/s10115-022-01772-8 · Cited by 202
“After learning with a multi-class CNN, the proposed techniques showed effective results in a wide range of IRs.”
BackgroundInfluentialDeep learning‐based methods for individual recognition in small birds
Ferreira, A. C., Silva, L. R., Renna, F., Brandl, H. B., Renoult, J. P., Farine, D. R., Covas, R., & Doutrelant, C. (2020) · Methods in Ecology and Evolution · doi:10.1111/2041-210x.13436 · Cited by 203
Detecting and interpreting myocardial infarction using fully convolutional neural networks
Strodthoff, N., & Strodthoff, C. (2018) · Physiological Measurement · doi:10.1088/1361-6579/aaf34d · Cited by 219
Li, Y., Martinis, S., & Wieland, M. (2019) · ISPRS Journal of Photogrammetry and Remote Sensing · doi:10.1016/j.isprsjprs.2019.04.014 · Cited by 201
Analyzing the impact of feature selection on the accuracy of heart disease prediction
Pathan, M. S., Nag, A., Pathan, M. M., & Dev, S. (2022) · Healthcare Analytics · doi:10.1016/j.health.2022.100060 · Cited by 200
“The unbalanced nature of the datasets leads to classification errors during the training of ML models [30].”
BackgroundAn improved SPEI drought forecasting approach using the long short-term memory neural network
Dikshit, A., Pradhan, B., & HUETE, A. R. (2021) · Journal of Environmental Management · doi:10.1016/j.jenvman.2021.111979 · Cited by 213
Korkmaz, D., & Açıkgöz, H. (2022) · Engineering Applications of Artificial Intelligence · doi:10.1016/j.engappai.2022.104959 · Cited by 206
Nex, F., Duarte, D., Tonolo, F. G., & Kerle, N. (2019) · Remote Sensing · doi:10.3390/rs11232765 · Cited by 194
“A balanced number of intact and damaged samples were extracted from each data category, preventing problems due to uneven samples in the different classes [49].”
MethodDeep Learning in Image Cytometry: A Review
Gupta, A., Harrison, P. J., Wieslander, H., Pielawski, N., Kartasalo, K., Partel, G., Solorzano, L., Suveer, A., Klemm, A. H., Spjuth, O., et al. (2018) · Cytometry Part A · doi:10.1002/cyto.a.23701 · Cited by 194
“A common problem for many datasets is class imbalance (109).”
BackgroundA Survey of Textual Emotion Recognition and Its Challenges
Deng, J., & Ren, F. (2021) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2021.3053275 · Cited by 214
“In [133], the authors think that data imbalance has a significant impact on the performance of neural network models and conclude that over-sampling is the most effective way to combat this problem.”
BackgroundSayed, G. I., Soliman, M., & Hassanien, A. E. (2021) · Computers in Biology and Medicine · doi:10.1016/j.compbiomed.2021.104712 · Cited by 188
The Devil Is in Classification: A Simple Framework for Long-Tail Instance Segmentation
Wang, T., Yu, L. Z., Kang, B., Li, J., Liew, J. H., Tang, S., Hoi, S. C. H., & Feng, J. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-58568-6_43 · Cited by 185
Language-Grounded Indoor 3D Semantic Segmentation in the Wild
Rozenberszki, D., Litany, O., & Dai, A. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-19827-4_8 · Cited by 207
MetaSAug: Meta Semantic Augmentation for Long-Tailed Visual Recognition
Li, S., Gong, K., Liu, C. H., Wang, Y., Qiao, F., & Cheng, X. (2021) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr46437.2021.00517 · Cited by 186
“Researchers propose to achieve a more balanced data distribution by over-sampling the minority classes [32, 5, 6] or under-sampling the majority classes [14, 19, 5].”
BackgroundSchwendicke, F., Elhennawy, K., Paris, S., Friebertshäuser, P., & Krois, J. (2019) · Journal of Dentistry · doi:10.1016/j.jdent.2019.103260 · Cited by 202
Identifying Autism Spectrum Disorder From Resting-State fMRI Using Deep Belief Network
Huang, Z.-A., Zhu, Z., Yau, C. H., & Tan, K. C. (2020) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2020.3007943 · Cited by 189
Li, D., Cong, A., & Guo, S. (2019) · Automation in Construction · doi:10.1016/j.autcon.2019.01.017 · Cited by 180
Gao, K., Su, J., Jiang, Z., Zeng, L., Feng, Z., Shen, H., Rong, P., Xu, X., Qin, J., Yang, Y., et al. (2020) · Medical Image Analysis · doi:10.1016/j.media.2020.101836 · Cited by 180
Metadata
- Authors
- Buda, M., Maki, A., & Mazurowski, M. A.
- Affiliations
- Duke University (US)
- KTH Royal Institute of Technology (SE)
- Corresponding author
- Buda, Mateusz
- Venue
- Neural Networks
- Publisher
- Elsevier BV
- Published
- 2018-10
- Volume
- 106
- Pages
- 249-259
- ISSN
- 0893-6080 (ISSN-L)
- OpenAlex
- W2767106145
- Cited by
- 3,088
- References
- 83
- Citations by year
- 2026: 459 · 2025: 507 · 2024: 371 · 2023: 405 · 2022: 405 · 2021: 380 · 2020: 308 · 2019: 189 · 2018: 58
- FWCI
- 131.58
- Type
- journal article
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- Neural Networks (publishedVersion)
- PubMed (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- KTH Publication Database DiVA (KTH Royal Institute of Technology) (submittedVersion, other-oa)
- arXiv (Cornell University)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Imbalanced Data Classification Techniques
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
- Physical Sciences › Computer Science › Artificial Intelligence › Machine Learning and Data Classification
- Keywords
- class imbalance, convolutional neural networks, oversampling, undersampling, two-stage training, thresholding, area under the receiver operating characteristic, multi-class classification, overfitting, MNIST, CIFAR-10, ImageNet
- MeSH terms
- Machine Learning, Humans, Probability, ROC Curve, Neural Networks, Computer
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
