Cited by 3,088Imbalanced Data Classification TechniquesAdvanced Neural Network ApplicationsMachine Learning and Data Classification

The DOI matches its Crossref record; 77 of the 83 references it cites were found in a registry or an index.

A systematic study of the class imbalance problem in convolutional neural networks

Open access

Buda, Mateusz, Maki, Atsuto, Mazurowski, Maciej A.·2018·Neural Networks

DOI 10.1016/j.neunet.2018.07.011

Open this paper in CoChat — free account Open PDF

Abstract

In this study, we systematically investigate the impact of class imbalance on classification performance of convolutional neural networks (CNNs) and compare frequently used methods to address the issue. Class imbalance is a common problem that has been comprehensively studied in classical machine learning, yet very limited systematic research is available in the context of deep learning. In our study, we use three benchmark datasets of increasing complexity, MNIST, CIFAR-10 and ImageNet, to investigate the effects of imbalance on classification and perform an extensive comparison of several methods to address the issue: oversampling, undersampling, two-phase training, and thresholding that compensates for prior class probabilities. Our main evaluation metric is area under the receiver operating characteristic curve (ROC AUC) adjusted to multi-class tasks since overall accuracy metric is associated with notable difficulties in the context of imbalanced data. Based on results from our experiments we conclude that (i) the effect of class imbalance on classification performance is detrimental; (ii) the method of addressing class imbalance that emerged as dominant in almost all analyzed scenarios was oversampling; (iii) oversampling should be applied to the level that completely eliminates the imbalance, whereas the optimal undersampling ratio depends on the extent of imbalance; (iv) as opposed to some classical machine learning models, oversampling does not cause overfitting of CNNs; (v) thresholding should be applied to compensate for prior class probabilities when overall number of properly classified cases is of interest.

Integrity signals

Registry

Crossref · 10.1016/j.neunet.2018.07.011

Cites

77 of 77 references verified0 citation errors · 0 not checkable

Statistics

Recomputing p-values needs the paper’s full text.

Claims the paper makes

We couldn’t retrieve this paper’s full text, so its claims and self-check weren’t run. The registry and citation checks still apply.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474 · cited 1×
“And for all the cases, the classes chosen to be minority were the ones with the lowest label value.”
Background
Very Deep Convolutional Networks for Large-Scale Image Recognition
Simonyan, K., & Zisserman, A. (2014) · arXiv · doi:10.48550/arxiv.1409.1556 · Cited by 114,148 · cited 1×
“A standard CNN architecture comprises fully connected layers and a number of blocks consisting of convolution kernels, activation function layer and max pooling [22, 23, 24].”
Method
ImageNet classification with deep convolutional neural networks
Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017) · Communications of the ACM · doi:10.1145/3065386 · Cited by 112,155
Scikit-learn: Machine Learning in Python
Pedregosa, F., Varoquaux, G., Gramfort, A., Michel, V., Thirion, B., Grisel, O., Blondel, M., Müller, A., Nothman, J., Louppe, G., et al. (2012) · arXiv · doi:10.48550/arxiv.1201.0490 · Cited by 71,016 · cited 1×
“We used a speciﬁc implementation of the ROC AUC available in scikit-learn python package [65].”
Method
Gradient-based learning applied to document recognition
Lecun, Y., Bottou, L., Bengio, Y., & Haffner, P. (1998) · Proceedings of the IEEE · doi:10.1109/5.726791 · Cited by 58,805 · cited 2×
“Images The CNN model that we use for MNIST is the modern version of LeNet-5 [52].”
Method
ImageNet Large Scale Visual Recognition Challenge
Russakovsky, O., Deng, J., Su, H., Krause, J., Satheesh, S., Ma, S., Huang, Z., Karpathy, A., Khosla, A., Bernstein, M., et al. (2015) · International Journal of Computer Vision · doi:10.1007/s11263-015-0816-y · Cited by 39,380 · cited 1×
“In our study, we used three benchmark datasets: MNIST [52], CIFAR-10 [53] and ImageNet Large Scale Visual Recognition Challenge (ILSVRC) 2012 [54].”
Method
SMOTE synthetic minority over-sampling technique
Chawla, N. V., Bowyer, K. W., Hall, L. O., & Kegelmeyer, W. P. (2002) · Journal of Artificial Intelligence Research · doi:10.1613/jair.953 · Cited by 32,865 · cited 3×
“For classical machine learning models it was shown that oversampling can cause overﬁtting, especially for minority classes [33].”
BackgroundMethod
Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification
He, K., Zhang, X., Ren, S., & Sun, J. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.123 · Cited by 19,741 · cited 1×
“Network weights were initialized with Kaiming (also known as MSRA) initialization procedure [63].”
Method
Visualizing and understanding convolutional networks
Zeiler, M. D., & Fergus, R. (2014) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-10590-1_53 · Cited by 15,839 · cited 1×
“This property allows CNNs to learn hierarchical representations [2].”
Background
Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate ShiftPreprint
Ioffe, S. (2024) · arXiv (Cornell University) · doi:10.57702/o9raffed · Cited by 15,273

Show all 83 references

Sorted by citations

Items 101–200 of 1,001

SCADA System Testbed for Cybersecurity Research Using Machine Learning Approach
Teixeira, M. A., Salman, T., Zolanvari, M., Jain, R. K., Meskin, N., & Samaka, M. (2018) · Future Internet · doi:10.3390/fi10080076 · Cited by 178
“However, this metric is not very reliable for evaluating the ML performance in the scenarios with imbalanced classes [26].”
Background
Effective Class-Imbalance Learning Based on SMOTE and Convolutional Neural Networks
Joloudari, J. H., Marefat, A., Nematollahi, M. A., Oyelere, S. S., & Hussain, S. (2023) · Applied Sciences · doi:10.3390/app13064006 · Cited by 249
Multi class SVM algorithm with active learning for network traffic classification
Dong, S. (2021) · Expert Systems with Applications · doi:10.1016/j.eswa.2021.114885 · Cited by 175
Adversarial Instance Augmentation for Building Change Detection in Remote Sensing Images
Chen, H., Li, W., & Shi, Z. (2021) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2021.3066802 · Cited by 172
A novel data augmentation approach to fault diagnosis with class-imbalance problem
Tian, J., Jiang, Y., Zhang, J., Luo, H., & Yin, S. (2023) · Reliability Engineering & System Safety · doi:10.1016/j.ress.2023.109832 · Cited by 170
The METRIC-framework for assessing data quality for trustworthy AI in medicine: A systematic review
Schwabe, D., Becker, K., Seyferth, M., Klaß, A., & Schaeffter, T. (2024) · npj Digital Medicine · doi:10.1038/s41746-024-01196-4 · Cited by 169
“Finally, the highly-discussed dimension target class balance pays tri-bute to the technical requirements of ML 140,141,144,150,159 .”
Background
FactSeg: Foreground Activation-Driven Small Object Semantic Segmentation in Large-Scale Remote Sensing Imagery
Ma, A., Wang, J., Zhong, Y., & Zheng, Z. (2021) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2021.3097148 · Cited by 169
“The methods of addressing the imbalanced sample problem in CNNs fall into four main approaches: 1) oversampling; 2) undersampling; 3) two-phase training; and 4) thresholding [34].”
Method
A Survey on Long-Tailed Visual Recognition
Yang, L., Jiang, H., Song, Q., & Guo, J. H. (2022) · International Journal of Computer Vision · doi:10.1007/s11263-022-01622-8 · Cited by 179
“method reduces the imbalance between the head classes and the tail classes by reducing the sample times of the head classes [11, 55, 58, 71], as shown in Fig.”
BackgroundMethod
Segmentation of roots in soil with U-Net
Smith, A. G., Petersen, J., Selvan, R., & Rasmussen, C. R. (2020) · Plant Methods · doi:10.1186/s13007-020-0563-0 · Cited by 172
Deep Learning: An Update for Radiologists
Cheng, P. M., Montagnon, E., Yamashita, R., Pan, I., Cadrin-Chênevert, A., Romero, F., Chartrand, G., Kadoury, S., & Tang, A. (2021) · Radiographics · doi:10.1148/rg.2021200210 · Cited by 179
“Common solutions to mitigate this imbalance include strongly weighting the minority class in the loss function (63) or oversampling the minority class to expose the model equally to all classes (64).”
Deep-Learning-Based Semantic Segmentation of Remote Sensing Images: A Survey
Huang, L., Jiang, B., Lv, S., Liu, Y., & Fu, Y. (2023) · IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing · doi:10.1109/jstars.2023.3335891 · Cited by 196
“The primary purpose of resampling is to make each category has the same number of samples [154].”
Background
Plant Leaf Disease Detection, Classification, and Diagnosis Using Computer Vision and Artificial Intelligence: A Review
Bhargava, A., Shukla, A., Goswami, O. P., Alsharif, M. H., Uthansakul, P., & Uthansakul, M. (2024) · IEEE Access · doi:10.1109/access.2024.3373001 · Cited by 188
“Therefore, FSL is the best possible feasible solu-tion for a confined number of samples [253], [254].”
Background
DKDFN: Domain Knowledge-Guided deep collaborative fusion network for multimodal unitemporal remote sensing land cover classification
Li, Y., Zhou, Y., Zhang, Y., Zhong, L., Wang, J., & Chen, J. (2022) · ISPRS Journal of Photogrammetry and Remote Sensing · doi:10.1016/j.isprsjprs.2022.02.013 · Cited by 170
Early Prediction of Diabetes Using an Ensemble of Machine Learning Models
Dutta, A., Hasan, M. K., Ahmad, M., Awal, M. A., Islam, M. A., Masud, M., & Meshref, H. (2022) · International Journal of Environmental Research and Public Health · doi:10.3390/ijerph191912378 · Cited by 178
“A class imbalance problem emerges when the frequency of one class (for example, cancer) can be 1000 times lower than that of another class (for example, healthy patient) [47].”
Background
AHCNet: An Application of Attention Mechanism and Hybrid Connection for Liver Tumor Segmentation in CT Volumes
Jiang, H., Shi, T., Bai, Z., & Huang, L. (2019) · IEEE Access · doi:10.1109/access.2019.2899608 · Cited by 163
“A large area of background usually leads to the learning process to get trapped in local minima of the loss function yielding a network with strong background bias prediction [41], so the selection of the loss function is very important.”
Background
Long- Tailed Recognition via Weight Balancing
Alshammari, S., Wang, Y.-X., Ramanan, D., & Kong, S. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.00677 · Cited by 169
“In the real open world, data tends to follow long-tailed distributions [8, 60, 84, 85].”
Background
Distributionally Robust Neural Networks
Sagawa, S., Koh, P. W., Hashimoto, T., & Liang, P. (2020) · International Conference on Learning Representations · Cited by 164
Classification of Imbalanced Data:Review of Methods and Applications
Kumar, P., Bhatnagar, R., Gaur, K., & Bhatnagar, A. (2021) · IOP Conference Series Materials Science and Engineering · doi:10.1088/1757-899x/1099/1/012077 · Cited by 180
Addressing class imbalance in deep learning for small lesion detection on medical images
Bria, A., Marrocco, C., & Tortorella, F. (2020) · Computers in Biology and Medicine · doi:10.1016/j.compbiomed.2020.103735 · Cited by 158
Incorporating textual information in customer churn prediction models based on a convolutional neural network
Caigny, A. D., Coussement, K., Bock, K. W. D., & Lessmann, S. (2019) · International Journal of Forecasting · doi:10.1016/j.ijforecast.2019.03.029 · Cited by 159
ResLT: Residual Learning for Long-tailed Recognition
Cui, J., Liu, S., Tian, Z., Zhong, Z., & Jia, J. (2022) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2022.3174892 · Cited by 156
“Re-sampling Strategy There are two groups of resampling strategies: over-sampling the tail-class images [29, 42, 2, 3] and under-sampling the head-class images [12, 18, 2].”
BackgroundMethod
FETCH: A deep-learning based classifier for fast transient classification
Agarwal, D., Aggarwal, K., Burke-Spolaor, S., Lorimer, D. R., & Garver-Daniels, N. (2020) · Monthly Notices of the Royal Astronomical Society · doi:10.1093/mnras/staa1856 · Cited by 148
Effects of sample size and network depth on a deep learning approach to species distribution modeling
Benkendorf, D. J., & Hawkins, C. P. (2020) · Ecological Informatics · doi:10.1016/j.ecoinf.2020.101137 · Cited by 144
Towards Resolving the Challenge of Long-tail Distribution in UAV Images for Object Detection
Yu, W., Yang, T., & Chen, C. (2021) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv48630.2021.00330 · Cited by 145
Machine learning approaches for pathologic diagnosis
Komura, D., & Ishikawa, S. (2019) · Archiv für Pathologische Anatomie und Physiologie und für Klinische Medicin · doi:10.1007/s00428-019-02594-w · Cited by 143
Deep learning for visual recognition and detection of aquatic animals: A review
Li, J., Xu, W., Deng, L., Xiao, Y., Han, Z., & Zheng, H. (2022) · Reviews in Aquaculture · doi:10.1111/raq.12726 · Cited by 148
Bag of Tricks for Long-Tailed Visual Recognition with Deep Convolutional Neural Networks
Zhang, Y., Wei, X.-S., Zhou, B., & Wu, J. (2021) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v35i4.16458 · Cited by 143
“For instance, re-sampling (Buda, Maki, and Mazurowski 2018; Japkowicz and Stephen 2002) and re-weighting (Mikolov et al. 2013; Cui et al. 2019) are two commonly used tricks to alleviate the imbalance of long-tailed distributions.”
Background
YOLC: You Only Look Clusters for Tiny Object Detection in Aerial Images
Liu, C., Gao, G., Huang, Z., Hu, Z., Liu, Q., & Wang, Y. (2024) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2024.3386928 · Cited by 160
“Several approaches, including re-sampling [60], [61], re-weighting [62], re-balancing [63], [64], and two-phase training paradigms [56], [58], have been proposed to address this issue.”
Background
Convolutional neural network for soil microplastic contamination screening using infrared spectroscopy
Ng, W., Minasny, B., & McBratney, A. B. (2019) · The Science of The Total Environment · doi:10.1016/j.scitotenv.2019.134723 · Cited by 137
Urinary Stone Detection on CT Images Using Deep Convolutional Neural Networks: Evaluation of Model Performance and Generalization
Parakh, A., Lee, H., Lee, J. H., Eisner, B. H., Sahani, D. V., & Do, S. (2019) · Radiology Artificial Intelligence · doi:10.1148/ryai.2019180066 · Cited by 146
A novel cellular automata model integrated with deep learning for dynamic spatio-temporal land use change simulation
Xing, W., Qian, Y., Guan, X., Yang, T., & Wu, H. (2020) · Computers & Geosciences · doi:10.1016/j.cageo.2020.104430 · Cited by 135
A multimodal hybrid parallel network intrusion detection model
Shi, S., Han, D., & Cui, M. (2023) · Connection Science · doi:10.1080/09540091.2023.2227780 · Cited by 141
“Moreover, data imbalance (Buda et al., 2018; Cai et al., 2022) is a serious problem in intrusion detection research based on DL and ML, which significantly affects the performance of intrusion detection algorithms.”
Background
Data Sampling Methods to Deal With the Big Data Multi-Class Imbalance Problem
Rendón, E., Alejo, R., Castorena, C., Isidro-Ortega, F. J., & Granda-Gutiérrez, E. E. (2020) · Applied Sciences · doi:10.3390/app10041276 · Cited by 146
DR|GRADUATE: uncertainty-aware deep learning-based diabetic retinopathy grading in eye fundus images
Araújo, T., Aresta, G., Mendonça, L., Penas, S., Maia, C., Carneiro, Â., Mendonça, A. M., & Campilho, A. (2020) · Medical Image Analysis · doi:10.1016/j.media.2020.101715 · Cited by 133
A deep learning algorithm for simulating autonomous driving considering prior knowledge and temporal information
Chen, S., Leng, Y., & Labi, S. (2019) · Computer-Aided Civil and Infrastructure Engineering · doi:10.1111/mice.12495 · Cited by 135
Predicting the Risk of Chronic Kidney Disease (CKD) Using Machine Learning Algorithm
Wang, W., Chakraborty, G., & Chakraborty, B. (2020) · Applied Sciences · doi:10.3390/app11010202 · Cited by 135
“The data imbalance problem usually can be solved by data level methods and model level methods [19].”
Method
A novel MRI segmentation method using CNN ‐based correction network for MRI ‐guided adaptive radiotherapy
Fu, Y., Mazur, T. R., Wu, X., Liu, S., Chang, X., Lu, Y., Li, H. H., Kim, H. J., Roach, M. C., Henke, L. E., et al. (2018) · Medical Physics · doi:10.1002/mp.13221 · Cited by 130
Unbalanced breast cancer data classification using novel fitness functions in genetic programming
Devarriya, D., Gulati, C., Mansharamani, V., Sakalle, A., & Bhardwaj, A. (2019) · Expert Systems with Applications · doi:10.1016/j.eswa.2019.112866 · Cited by 130
Endless Forams: >34,000 Modern Planktonic Foraminiferal Images for Taxonomic Training and Automated Species Recognition Using Convolutional Neural Networks
Hsiang, A. Y., Brombacher, A., Rillo, M. C., Vautravers, M. J., Conn, S., Lordsmith, S., Jentzen, A., Henehan, M. J., Metcalfe, B., Fenton, I. S., et al. (2019) · Paleoceanography and Paleoclimatology · doi:10.1029/2019pa003612 · Cited by 129
“We initially tested three CNNs: VGG16 (Simonyan & Zisserman, 2014), DenseNet121 (Huang et al., 2016), and InceptionV3 (Szegedy et al., 2015).”
BackgroundInfluential
PCPL: Predicate-Correlation Perception Learning for Unbiased Scene Graph Generation
Yan, S., Shen, C., Jin, Z., Huang, J., Jiang, R., Chen, Y., & Hua, X.-S. (2020) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3394171.3413722 · Cited by 129
“Existing methods can be categorized as re-sampling [1, 5, 11, 29, 42] and re-weighting[4, 14, 25].”
Method
Semi-Supervised Semantic Segmentation via Adaptive Equalization LearningPreprint
Hu, H., Wei, F., Hu, H., Ye, Q., Cui, J., & Wang, L. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2110.05474 · Cited by 130
“Moreover, we refer to the ideas of re-sampling [23, 24] and re-weighting [25, 26], which are designed for class imbalance problem.”
Method
Balanced-MixUp for Highly Imbalanced Medical Image Classification
Galdrán, A., Carneiro, G., & Ballester, M. Á. G. (2021) · Lecture notes in computer science · doi:10.1007/978-3-030-87240-3_31 · Cited by 138
Forecasting Different Types of Convective Weather: A Deep Learning Approach
Zhou, K., Zheng, Y., Ли, Б., Dong, W., & Zhang, X. (2019) · 气象学报：英文版 · doi:10.1007/s13351-019-8162-6 · Cited by 127
Synthetic data as an enabler for machine learning applications in medicine
Rajotte, J.-F., Bergen, R. V., Buckeridge, D. L., Emam, K. E., Ng, R. T., & Strome, E. (2022) · iScience · doi:10.1016/j.isci.2022.105331 · Cited by 143
“For example, the ubiquitous task of image classiﬁcation in deep learning results in poor performance when a model is trained on datasets with imbalanced classes (Buda et al., 2018), i.e. one or more classes are signiﬁcantly underrepresented in the dataset.”
Medicare fraud detection using neural networks
Johnson, J. M., & Khoshgoftaar, T. M. (2019) · Journal Of Big Data · doi:10.1186/s40537-019-0225-0 · Cited by 152
“[52], our results suggest that over-sampling until class imbalance is eliminated from the training data is best for neural networks.”
BackgroundResult
Assessment of Deep Convolutional Neural Networks for Road Surface Classification
Nolte, M., Kister, N., & Maurer, M. (2018) · IEEE International Conference on Intelligent Transportation Systems (ITSC) · doi:10.1109/itsc.2018.8569396 · Cited by 129
“One challenge for training neural networks for classification tasks is the class imbalance problem caused by over-represented classes (majority classes) and under-represented classes (minority classes) in a dataset: If single classes dominate a training set or single classes are only represented by a small number of s…”
Background
Protein Family-Specific Models Using Deep Neural Networks and Transfer Learning Improve Virtual Screening and Highlight the Need for More Data
Imrie, F., Bradley, A. R., van der Schaar, M., & Deane, C. M. (2018) · Journal of Chemical Information and Modeling · doi:10.1021/acs.jcim.8b00350 · Cited by 127
Deep Learning for Predicting Enhancing Lesions in Multiple Sclerosis from Noncontrast MRI
Narayana, P. A., Coronado, I., Sujit, S. J., Wolinsky, J. S., LUBLIN, F. D., & Gabr, R. E. (2019) · Radiology · doi:10.1148/radiol.2019191061 · Cited by 127
“This choice, effectively oversampling the minority class, minimizes problems related to class imbalance, which can be detrimental for classifier performance (34).”
Background
Waste classification using AutoEncoder network with integrated feature selection method in convolutional neural network models
Toğaçar, M., Ergen, B., & Cömert, Z. (2019) · Measurement · doi:10.1016/j.measurement.2019.107459 · Cited by 128
Machine Learning in Predictive Toxicology: Recent Applications and Future Directions for Classification Models
Wang, M. W. H., Goodman, J. M., & Allen, T. E. H. (2020) · Chemical Research in Toxicology · doi:10.1021/acs.chemrestox.0c00316 · Cited by 133
The balancing trick: Optimized sampling of imbalanced datasets—A brief survey of the recent State of the Art
Susan, S., & Kumar, A. (2020) · Engineering Reports · doi:10.1002/eng2.12298 · Cited by 142
“A very interesting study on the effect of class-imbalance on Convolutional Neural Networks in Reference 98 concludes that oversampling is the best solution for balancing the dataset and that application of undersampling depends on the degree of imbalance.”
Background
An Investigation of Why Overparameterization Exacerbates Spurious CorrelationsPreprint
Sagawa, S., Raghunathan, A., Koh, P. W., & Liang, P. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2005.04345 · Cited by 141
“Prior work on improving worst-group error focused on the underparameterized regime, with methods based on weighting/sampling (Shimodaira, 2000; Japkowicz & Stephen, 2002; Buda et al., 2018; Cui et al., 2019), distributionally robust optimization (DRO) (Ben-Tal et al.”
BackgroundMethodInfluential
Automatic sewer pipe defect semantic segmentation based on improved U-Net
Pan, G., Zheng, Y., Guo, S., & Lv, Y. (2020) · Automation in Construction · doi:10.1016/j.autcon.2020.103383 · Cited by 123
Classification of white blood cells using deep features obtained from Convolutional Neural Network models based on the combination of feature selection methods
Toğaçar, M., Ergen, B., & Cömert, Z. (2020) · Applied Soft Computing · doi:10.1016/j.asoc.2020.106810 · Cited by 121
Automated assessment reveals that the extinction risk of reptiles is widely underestimated across space and phylogeny
Caetano, G. H. D. O., Chapple, D. G., Grenyer, R., Raz, T., Rosenblatt, J., Tingley, R., Böhm, M., Meiri, S., & Roll, U. (2022) · PLoS Biology · doi:10.1371/journal.pbio.3001544 · Cited by 131
“In fact, Neural Networks are known to be sensitive to such imbalances [30], while XGBoost is considered more robust to them [29].”
Identification of Cotton Leaf Lesions Using Deep Learning Techniques
Caldeira, R. F., Santiago, W. E., & Teruel, B. (2021) · Sensors · doi:10.3390/s21093169 · Cited by 127
“One of the advantages of deep learning in the One of the advantages of deep learning in the classification of images is that it is able to deal with an unbalanced set of data via manipulation [68]. with an unbalanced set of data via manipulation [68].”
Background
A Data Set and Deep Learning Algorithm for the Detection of Masses and Architectural Distortions in Digital Breast Tomosynthesis Images
Buda, M., Saha, A., Walsh, R., Ghate, S. V., Li, N., Swiecicki, A., Lo, J. Y.-C., & Mazurowski, M. A. (2021) · JAMA Network Open · doi:10.1001/jamanetworkopen.2021.19100 · Cited by 137
“To provide an insight into the effects of different hyper-parameters on the performance, we performed a grid search over different network sizes and objectness loss functions that address the problem of class imbalance [17].”
Self-citation
Evaluation of a deep-learning model for multispectral remote sensing of land use and crop classification
Wang, L., Wang, J., Liu, Z., Zhu, J. J., & Qin, F. (2022) · The Crop Journal · doi:10.1016/j.cj.2022.01.009 · Cited by 129
MSCS-DeepLN: Evaluating lung nodule malignancy using multi-scale cost-sensitive neural networks
Xu, X., Wang, C., Guo, J., Gan, Y., Wang, J., Bai, H., Zhang, L., Li, W., & Zhang, Y. (2020) · Medical Image Analysis · doi:10.1016/j.media.2020.101772 · Cited by 115
Promises and perils of artificial intelligence in dentistry
Pethani, F. (2020) · Australian Dental Journal · doi:10.1111/adj.12812 · Cited by 129
Incremental Weighted Ensemble Broad Learning System for Imbalanced Data
Yang, K., Yu, Z., Chen, C. L. P., Cao, W., You, J., & Wong, H. (2021) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2021.3061428 · Cited by 114
“In [85], different strategies are investigated to address imbalance problem in clas-siﬁcation task of CNN, including RUS, ROS, and two-phase learning.”
Background
Nested Collaborative Learning for Long-Tailed Visual Recognition
Li, J., Tan, Z., Wan, J., Lei, Z., & Guo, G. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.00682 · Cited by 124
“The existing methods for long-tailed visual recognition can be roughly divided into three categories: class re-balancing [1, 11, 17, 24, 42, 51], multi-stage training [4,27] and multi-expert methods [2,33, 50,53,58].”
Method
Comprehensive analysis of clustering algorithms: exploring limitations and innovative solutions
Wani, A. A. (2024) · PeerJ Computer Science · doi:10.7717/peerj-cs.2286 · Cited by 168
“The reconstruction loss, often driven by the majority class, exacerbates this issue by neglecting the nuances of minority clusters ( Buda, Maki & Mazurowski, 2018 ).”
Background
Deep learning predictions of galaxy merger stage and the importance of observational realism
Bottrell, C., Hani, M. H., Teimoorinia, H., Ellison, S. L., Moreno, J., Torrey, P., Hayward, C. C., Thorp, M., Simard, L., & Hernquist, L. (2019) · Monthly Notices of the Royal Astronomical Society · doi:10.1093/mnras/stz2934 · Cited by 113
Blood Cell Images Segmentation using Deep Learning Semantic Segmentation
Tran, T., Kwon, O., Kwon, K., Lee, S., & Kang, K.-W. (2018) · IEEE International Conference on Electronics and Communication Engineering (ICECE) · doi:10.1109/icecome.2018.8644754 · Cited by 113
Deep Multi-Magnification Networks for multi-class breast cancer image segmentation
Ho, D. J., Yarlagadda, D. V. K., D’Alfonso, T. M., Hanna, M. G., Grabenstetter, A., Ntiamoah, P., Brogi, E., Tan, L. K., & Fuchs, T. J. (2021) · Computerized Medical Imaging and Graphics · doi:10.1016/j.compmedimag.2021.101866 · Cited by 116
“Class balancing is a prerequisite step for training CNNs for accurate performance [39].”
Background
The classification and localization of crack using lightweight convolutional neural network with CBAM
Chen, L., Yao, H., Fu, J., & Ng, C. (2022) · Engineering Structures · doi:10.1016/j.engstruct.2022.115291 · Cited by 119
The use of generative adversarial networks to alleviate class imbalance in tabular data: a survey
Sauber-Cole, R., & Khoshgoftaar, T. M. (2022) · Journal Of Big Data · doi:10.1186/s40537-022-00648-6 · Cited by 132
A Comprehensive Survey on Deep Clustering: Taxonomy, Challenges, and Future Directions
Zhou, S., Xu, H., Zheng, Z., Chen, J., Li, Z., Bu, J., Wu, J. X., Wang, X. Y., Zhu, W., & Ester, M. (2024) · ACM Computing Surveys · doi:10.1145/3689036 · Cited by 251
Deep learning guided image-based droplet sorting for on-demand selection and analysis of single cells and 3D cell cultures
Anagnostidis, V., Sherlock, B. E., Metz, J., Mair, P., Hollfelder, F., & Gielen, F. (2020) · Lab on a Chip · doi:10.1039/d0lc00055h · Cited by 116
Class-Imbalanced Deep Learning via a Class-Balanced Ensemble
Chen, Z., Duan, J., Kang, L., & Qiu, G. (2021) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2021.3071122 · Cited by 114
“Despite its fundamental importance, class imbalance in the context of deep representation learning has been under-researched [7], [8].”
BackgroundMethodResultInfluential
A Real-Time Collision Prediction Mechanism With Deep Learning for Intelligent Transportation System
Wang, X., Liu, J., Qiu, T., Mu, C., Chen, C., & Zhou, P. (2020) · IEEE Transactions on Vehicular Technology · doi:10.1109/tvt.2020.3003933 · Cited by 114
“Accordingtotheliterature[34], [35],oversamplingismoresuitableforalleviatingtheclass imbalanceissueofconvolutionalneuralnetworks.”
RSG: A Simple but Effective Module for Learning Imbalanced Datasets
Wang, J., Lukasiewicz, T., Hu, X., Cai, J., & Xu, Z. (2021) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr46437.2021.00378 · Cited by 114
“Resampling techniques include oversampling the minority classes [25, 2, 3, 1, 19] and undersampling the majority classes [2, 18, 10], which aims to balance the data distribution.”
BackgroundMethod
Classification of CITES-listed and other neotropical Meliaceae wood images using convolutional neural networks
Ravindran, P., Costa, A. D. C., Soares, R. K., & Wiedenhoeft, A. C. (2018) · Plant Methods · doi:10.1186/s13007-018-0292-9 · Cited by 108
“be effective for training CNNs in the presence of class imbalance [27].”
Background
Performance of a convolutional neural network derived from an ECG database in recognizing myocardial infarction
Makimoto, H., Höckmann, M., Lin, T., Gloeckner, D., Gerguri, S., Clasen, L., Schmidt, J., Assadi-Schmidt, A., Bejinariu, A., Müller, P., et al. (2020) · Scientific Reports · doi:10.1038/s41598-020-65105-x · Cited by 111
Dual Focal Loss to address class imbalance in semantic segmentation
Hossain, M. S., Betts, J. M., & Papliński, A. P. (2021) · Neurocomputing · doi:10.1016/j.neucom.2021.07.055 · Cited by 112
“Another approach is to replicate the data of minority classes, commonly known as oversampling [22].”
Background
deepDriver: Predicting Cancer Driver Genes Based on Somatic Mutations Using Deep Convolutional Neural Networks
Luo, P., Ding, Y., Lei, X., & Wu, F.-X. (2019) · Frontiers in Genetics · doi:10.3389/fgene.2019.00013 · Cited by 107
“Currently, two types of methods can be used to solve the imbalanced problem: data level methods and classiﬁer level methods (Buda et al., 2018).”
Method
Identifying Collapsed Buildings Using Post-Earthquake Satellite Imagery and Convolutional Neural Networks: A Case Study of the 2010 Haiti Earthquake
Ji, M., Liu, L., & Buchroithner, M. F. (2018) · Remote Sensing · doi:10.3390/rs10111689 · Cited by 116
“To deal with the imbalance problem in the convolutional neural network, seven approaches were compared in [48], including random over-sampling, random under-sampling, and thresholding with prior class probabilities.”
Method
Classification of pig calls produced from birth to slaughter according to their emotional valence and context of production
Briefer, E. F., Sypherd, C. C.-R., Linhart, P., Leliveld, L. M. C., de la Torre, M. P., Read, E. R., Guérin, C., Deiss, V., Monestier, C., Rasmussen, J. H., et al. (2022) · Scientific Reports · doi:10.1038/s41598-022-07174-8 · Cited by 107
“Though the neural network performs well here, it could likely be improved by as much as 10% by addressing the imbalance in context classes 23 .”
Background
SLC-GAN: An automated myocardial infarction detection model based on generative adversarial networks and convolutional neural networks with single-lead electrocardiogram synthesis
Li, W., Tang, Y. M., Yu, K. M., & To, S. (2022) · Information Sciences · doi:10.1016/j.ins.2021.12.083 · Cited by 106
Explainable diabetes classification using hybrid Bayesian-optimized TabNet architecture
Joseph, L. P., Joseph, E. A., & Prasad, R. (2022) · Computers in Biology and Medicine · doi:10.1016/j.compbiomed.2022.106178 · Cited by 110
Synthetic Examples Improve Generalization for Rare Classes
Beery, S., Liu, Y., Morris, D., Piavis, J., Kapoor, A., Meister, M., Joshi, N., & Perona, P. (2020) · 2020 IEEE Winter Conference on Applications of Computer Vision (WACV) · doi:10.1109/wacv45572.2020.9093570 · Cited by 103
“Imbalanced datasets often lead to bias in algorithm performance toward well-represented classes [23].”
Background
Deep Focus Parallel Convolutional Neural Network for Imbalanced Classification of Machinery Fault Diagnostics
Duan, A., Guo, L., Gao, H., Xiang-dong, W., & Dong, X. (2020) · IEEE Transactions on Instrumentation and Measurement · doi:10.1109/tim.2020.2998233 · Cited by 106
“As a result, features of the minority health conditions are learned inadequately, and leading to the degradation of performance [24], [25].”
Background
The Fingerprints of Fraud: Evidence from Mexico’s 1988 Presidential Election
Cantú, F. (2019) · American Political Science Review · doi:10.1017/s0003055419000285 · Cited by 105
“The first strategy makes sure that all classes in the training set are represented by a similar number of examples (Buda, Maki, and Mazurowski 2018).”
Background
Façade defects classification from imbalanced dataset using meta learning-based convolutional neural network
Guo, J., Wang, Q., Li, Y., & Liu, P. (2020) · Computer-Aided Civil and Infrastructure Engineering · doi:10.1111/mice.12578 · Cited by 103
“Undersampling is a relatively undesirable method, which sacrifices valuable data and has the potential to reduce the accuracy of the majority class (Buda, Maki, & Mazurowski, 2018).”
Method
Short-Term Traffic Flow Forecasting Method With M-B-LSTM Hybrid Network
Qu, Z., Li, H., Li, Z., & Tao, Z. (2020) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2020.3009725 · Cited by 105
Parallel Vision for Long-Tail Regularization: Initial Results From IVFC Autonomous Driving Testing
Wang, J., Wang, X., Shen, T., Wang, Y., Li, L., Tian, Y., Yu, H. L., Chen, L. F., Xin, J., Wu, X., et al. (2022) · IEEE Transactions on Intelligent Vehicles · doi:10.1109/tiv.2022.3145035 · Cited by 103
“Studies on long-tailed imbalanced data can be mainly classiﬁed into two categories: resampling [14]–[16] (including over-sampling and under-sampling) and cost-sensitive learning [17]–[20].”
Method
Detection of Prostate Cancer in Whole-Slide Images Through End-to-End Training With Image-Level Labels
Pinckaers, H., Bulten, W., van der Laak, J. A. W. M., & Litjens, G. J. S. (2021) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2021.3066295 · Cited by 110
“Equally, we oversampled negative cases to counter the imbalance in the dataset, instead of weighting [33].”
Background
CEGAN: Classification Enhancement Generative Adversarial Networks for unraveling data imbalance problems
Suh, S., Lee, H., Lukowicz, P., & Lee, Y. O. (2020) · Neural Networks · doi:10.1016/j.neunet.2020.10.004 · Cited by 102
Self Supervision to Distillation for Long-Tailed Visual Recognition
Li, T., Wang, L., & Wu, G. (2021) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00067 · Cited by 106
“Data re-sampling achieved class-balanced training by over-sampling tail classes [3, 13, 6, 33], or under-sampling head classes [10, 1], yet, they might incur generalization problems due to over-fit datascarce classes or under-fit data-abundant classes.”
Background
An approximation algorithm for graph partitioning via deterministic annealing neural network
Wu, Z., Karimi, H. R., & Dang, C. (2019) · Neural Networks · doi:10.1016/j.neunet.2019.05.010 · Cited by 101
Cost-Sensitive BERT for Generalisable Sentence Classification on Imbalanced Data
Madabushi, H. T., Kochkina, E., & Castelle, M. (2019) · Workshop on Natural Language Processing for Internet Freedom: Censorship, Disinformation, and Propaganda (NLP4IF) · doi:10.18653/v1/d19-5018 · Cited by 103
“As class imbalance is a widespread issue, multiple techniques have been developed that help alleviate it (Buda et al., 2018; Haixiang et al., 2017), by either adjusting the model (e.g. changing the performance metric) or changing the data (e.g. oversampling the minority class or undersampling the…”
Background
MPCE: A Maximum Probability Based Cross Entropy Loss Function for Neural Network Classification
Zhou, Y., Wang, X., Zhang, M.-C., Zhu, J.-L., Zheng, R., & Wu, Q.-T. (2019) · IEEE Access · doi:10.1109/access.2019.2946264 · Cited by 76
Distributional Robustness Loss for Long-tail Learning
Samuel, D., & Chechik, G. (2021) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00936 · Cited by 104
“Real-world data usually follows a long-tailed distribution, which causes models to favor head classes and overfit tail classes [5, 35].”
A Citizen Science Unmanned Aerial System Data Acquisition Protocol and Deep Learning Techniques for the Automatic Detection and Mapping of Marine Litter Concentrations in the Coastal Zone
Papakonstantinou, A., Batsaris, M., Spondylidis, S., & Topouzelis, K. (2021) · Drones · doi:10.3390/drones5010006 · Cited by 99
“To avoid the negative impact of the imbalanced class problem, we adopted the under-sampling method [66,67] to ensure that an equal number of samples was distributed between the 2 classes.”
Method
Deep Learning for Classification of Mini-UAVs Using Micro-Doppler Spectrograms in Cognitive Radar
Huizing, A. G., Heiligers, M., Dekker, B., de Wit, J. J. M., Cifola, L., & Harmanny, R. I. A. (2019) · IEEE Aerospace and Electronic Systems Magazine · doi:10.1109/maes.2019.2933972 · Cited by 106
“This imbalance can have a negative impact on the convergence of the classifier during training and the overall accuracy [20].”
Background
A lightweight deep learning algorithm for inspection of laser welding defects on safety vent of power battery
Yang, Y., Yang, R., Pan, L., Ma, J., Zhu, Y., Diao, T., & Zhang, L. (2020) · Computers in Industry · doi:10.1016/j.compind.2020.103306 · Cited by 105
Application of machine learning approaches for osteoporosis risk prediction in postmenopausal women
Shim, J., Kim, D. W., Ryu, K., Cho, E.-A., Ahn, J. H., Kim, J.-I., & Lee, S. H. (2020) · Archives of Osteoporosis · doi:10.1007/s11657-020-00802-8 · Cited by 106
“The AUROC is known as a strong indicator of performance for classifiers in imbalanced datasets [36, 37].”
Background
On Supervised Class-Imbalanced Learning: An Updated Perspective and Some Key Challenges
Das, S., Mullick, S. S., & Zelinka, I. (2022) · IEEE Transactions on Artificial Intelligence · doi:10.1109/tai.2022.3160658 · Cited by 109
“Even though deep-learning-based classiﬁers managed to provide a signiﬁcant improvement in performance over their traditional counterparts they are no less susceptible to the class imbalanceinthetrainingset[6],[16],[155].”
Background
Adopting deep learning methods for airborne RGB fluvial scene classificationPreprint
Carbonneau, P., Dugdale, S. J., Breckon, T. P., Dietrich, J., Fonstad, M. A., Miyamoto, H., & Woodget, A. S. (2020) · doi:10.1016/j.rse.2020.112107 · Cited by 107

Authors: Buda, M., Maki, A., & Mazurowski, M. A.
Affiliations: Duke University (US)
KTH Royal Institute of Technology (SE)
Corresponding author: Buda, Mateusz
ORCID: Buda, Mateusz
Maki, Atsuto
Mazurowski, Maciej A.
Venue: Neural Networks
Publisher: Elsevier BV
Published: 2018-10
Volume: 106
Pages: 249-259
ISSN: 0893-6080 (ISSN-L)
DOI: 10.1016/j.neunet.2018.07.011
OpenAlex: W2767106145
Cited by: 3,088
References: 83
Citations by year: 2026: 459 · 2025: 507 · 2024: 371 · 2023: 405 · 2022: 405 · 2021: 380 · 2020: 308 · 2019: 189 · 2018: 58
FWCI: 131.58
Type: journal article
Language: en
Open access: green
arxiv.org
Licence: www.elsevier.com/tdm/userlicense/1.0/ (tdm)
www.elsevier.com/legal/tdmrep-license (tdm)
Locations: Neural Networks (publishedVersion)
PubMed (publishedVersion)
arXiv (Cornell University) (submittedVersion)
KTH Publication Database DiVA (KTH Royal Institute of Technology) (submittedVersion, other-oa)
arXiv (Cornell University)
Topics: Physical Sciences › Computer Science › Artificial Intelligence › Imbalanced Data Classification Techniques
Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
Physical Sciences › Computer Science › Artificial Intelligence › Machine Learning and Data Classification
Keywords: class imbalance, convolutional neural networks, oversampling, undersampling, two-stage training, thresholding, area under the receiver operating characteristic, multi-class classification, overfitting, MNIST, CIFAR-10, ImageNet
MeSH terms: Machine Learning, Humans, Probability, ROC Curve, Neural Networks, Computer

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

A systematic study of the class imbalance problem in convolutional neural networks

Abstract

Overview

Integrity signals

Verification

Claims the paper makes

Claims other papers make about this one

Citations

Cites

Cited by

Related

Metadata

Record

About these checks