A systematic study of the class imbalance problem in convolutional neural networks
Open accessAbstract
In this study, we systematically investigate the impact of class imbalance on classification performance of convolutional neural networks (CNNs) and compare frequently used methods to address the issue. Class imbalance is a common problem that has been comprehensively studied in classical machine learning, yet very limited systematic research is available in the context of deep learning. In our study, we use three benchmark datasets of increasing complexity, MNIST, CIFAR-10 and ImageNet, to investigate the effects of imbalance on classification and perform an extensive comparison of several methods to address the issue: oversampling, undersampling, two-phase training, and thresholding that compensates for prior class probabilities. Our main evaluation metric is area under the receiver operating characteristic curve (ROC AUC) adjusted to multi-class tasks since overall accuracy metric is associated with notable difficulties in the context of imbalanced data. Based on results from our experiments we conclude that (i) the effect of class imbalance on classification performance is detrimental; (ii) the method of addressing class imbalance that emerged as dominant in almost all analyzed scenarios was oversampling; (iii) oversampling should be applied to the level that completely eliminates the imbalance, whereas the optimal undersampling ratio depends on the extent of imbalance; (iv) as opposed to some classical machine learning models, oversampling does not cause overfitting of CNNs; (v) thresholding should be applied to compensate for prior class probabilities when overall number of properly classified cases is of interest.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
FT th requires a memory of the past to rectify past classes' scores.
“A memory of the past is needed to rectify past classes' scores.”
From A Comprehensive Study of Class Incremental Learning Algorithms for Visual Tasks (2020) · cites this paper as [12]
The introduced method reduces the classification bias in favor of new classes by using prior class probabilities.
“Here, we introduced a competitive method where classification bias in favor of new classes is reduced by using prior class probabilities [12].”
From A Comprehensive Study of Class Incremental Learning Algorithms for Visual Tasks (2020) · cites this paper as [12]
The highly-discussed dimension target class balance pays tribute to the technical requirements of ML.
“Finally, the highly-discussed dimension target class balance pays tribute to the technical requirements of ML140,141,144,150,159.”
From The METRIC-framework for assessing data quality for trustworthy AI in medicine: A systematic review (2024) · cites this paper as [141]
Cites
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474 · cited 1× · see citing works
“And for all the cases, the classes chosen to be minority were the ones with the lowest label value.”
BackgroundImageNet classification with deep convolutional neural networks
Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017) · Communications of the ACM · doi:10.1145/3065386 · Cited by 112,155 · see citing works
Very Deep Convolutional Networks for Large-Scale Image Recognition
Simonyan, K., & Zisserman, A. (2014) · arXiv · doi:10.48550/arxiv.1409.1556 · Cited by 74,248 · cited 1× · see citing works
“A standard CNN architecture comprises fully connected layers and a number of blocks consisting of convolution kernels, activation function layer and max pooling [22, 23, 24].”
MethodScikit-learn: Machine Learning in Python
Pedregosa, F., Varoquaux, G., Gramfort, A., Michel, V., Thirion, B., Grisel, O., Blondel, M., Müller, A., Nothman, J., Louppe, G., et al. (2012) · arXiv · doi:10.48550/arxiv.1201.0490 · Cited by 71,016 · cited 1× · see citing works
“We used a specific implementation of the ROC AUC available in scikit-learn python package [65].”
MethodGradient-based learning applied to document recognition
Lecun, Y., Bottou, L., Bengio, Y., & Haffner, P. (1998) · Proceedings of the IEEE · doi:10.1109/5.726791 · Cited by 58,805 · cited 2× · see citing works
“Images The CNN model that we use for MNIST is the modern version of LeNet-5 [52].”
MethodImageNet Large Scale Visual Recognition Challenge
Russakovsky, O., Deng, J., Su, H., Krause, J., Satheesh, S., Ma, S., Huang, Z., Karpathy, A., Khosla, A., Bernstein, M., et al. (2015) · International Journal of Computer Vision · doi:10.1007/s11263-015-0816-y · Cited by 39,380 · cited 1× · see citing works
“In our study, we used three benchmark datasets: MNIST [52], CIFAR-10 [53] and ImageNet Large Scale Visual Recognition Challenge (ILSVRC) 2012 [54].”
MethodSMOTE synthetic minority over-sampling technique
Chawla, N. V., Bowyer, K. W., Hall, L. O., & Kegelmeyer, W. P. (2002) · Journal of Artificial Intelligence Research · doi:10.1613/jair.953 · Cited by 32,865 · cited 3× · see citing works
“For classical machine learning models it was shown that oversampling can cause overfitting, especially for minority classes [33].”
BackgroundMethodDelving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification
He, K., Zhang, X., Ren, S., & Sun, J. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.123 · Cited by 19,741 · cited 1× · see citing works
“Network weights were initialized with Kaiming (also known as MSRA) initialization procedure [63].”
MethodVisualizing and understanding convolutional networks
Zeiler, M. D., & Fergus, R. (2014) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-10590-1_53 · Cited by 15,839 · cited 1× · see citing works
“This property allows CNNs to learn hierarchical representations [2].”
BackgroundBatch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate ShiftPreprint
Ioffe, S. (2024) · arXiv (Cornell University) · doi:10.57702/o9raffed · Cited by 15,273 · see citing works
(2010) · Cited by 14,134 · see citing works
Proceedings of IEEE International Conference on Computer Vision
(1995) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.1995.466933 · Cited by 13,117 · see citing works
Advances in Neural Information Processing Systems 25
Zhang, Y., Sutton, C. A., Storkey, A., & Zoubin Ghahramani, A. (2012) · Cited by 12,653 · see citing works
Backpropagation Applied to Handwritten Zip Code Recognition
LeCun, Y., Boser, B., Denker, J. S., Henderson, D., Howard, R. E., Hubbard, W., & Jackel, L. D. (1989) · Neural Computation · doi:10.1162/neco.1989.1.4.541 · Cited by 11,891 · cited 1× · see citing works
“The standard CNN is built with fully connected layers and a number of blocks consisting of convolutions, activation function layer and max pooling [3, 4, 5].”
MethodCaffe: Convolutional Architecture for Fast Feature Embedding
Jia, Y., Shelhamer, E., Donahue, J., Karayev, S., Long, J., Girshick, R., Guadarrama, S., & Darrell, T. (2014) · Proceedings of the 22nd ACM international conference on Multimedia · doi:10.1145/2647868.2654889 · Cited by 11,077 · cited 1× · see citing works
“All experiments were implemented in the deep learning framework Caffe [55].”
MethodHe, H., & Garcia, E. A. (2009) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2008.239 · Cited by 10,359 · cited 1× · see citing works
“Methods for addressing class imbalance can be divided into two main categories [29].”
BackgroundThe use of the area under the ROC curve in the evaluation of machine learning algorithms
Bradley, A. P. (1997) · Pattern Recognition · doi:10.1016/s0031-3203(96)00142-2 · Cited by 7,217 · see citing works
Recent advances in convolutional neural networks
Gu, J., Wang, Z., Kuen, J., Ma, L., Shahroudy, A., Shuai, B., Liu, T., Wang, X., Wang, G., Cai, J., et al. (2018) · Pattern Recognition · doi:10.1016/j.patcog.2017.10.013 · Cited by 6,576 · see citing works
2009 IEEE Conference on Computer Vision and Pattern Recognition
(2009) · doi:10.1109/cvpr14067.2009 · Cited by 5,646 · see citing works
Caffe: Convolutional Architecture for Fast Feature EmbeddingPreprint
Jia, Y., Shelhamer, E., Donahue, J., Karayev, S., Long, J., Girshick, R., Guadarrama, S., & Darrell, T. (2014) · arXiv (Cornell University) · doi:10.48550/arxiv.1408.5093 · Cited by 4,748 · cited 1× · see citing works
“All experiments were implemented in the deep learning framework Caffe [55].”
MethodBorderline-SMOTE : A New Over-Sampling Method in Imbalanced Data Sets Learning
Han, H., Wang, W.-Y., & Mao, B.-H. (2005) · Lecture notes in computer science · doi:10.1007/11538059_91 · Cited by 4,331 · cited 1× · see citing works
“Some extensions of this technique were proposed, for example focusing only on examples near the boundary between classes [35].”
MethodStriving for Simplicity: The All Convolutional Net
Springenberg, J. T., Dosovitskiy, A., Brox, T., & Riedmiller, M. (2014) · arXiv · doi:10.48550/arxiv.1412.6806 · Cited by 3,447 · cited 2× · see citing works
“For CIFAR-10 experiments we use one of the best performing type of CNN model on this dataset, i.e. All-CNN [59].”
MethodThe class imbalance problem: A systematic study1
Japkowicz, N., & Stephen, S. (2002) · Intelligent Data Analysis · doi:10.3233/ida-2002-6504 · Cited by 3,295 · cited 3× · see citing works
“Na¨ıve version, called random majority undersampling, simply removes a random portion of examples from majority classes [17].”
BackgroundSUN database: Large-scale scene recognition from abbey to zoo
Xiao, J., Hays, J., Ehinger, K. A., Oliva, A., & Torralba, A. (2010) · 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2010.5539970 · Cited by 3,283 · cited 1× · see citing works
“There are plenty of examples in domains like computer vision [6, 7, 8, 9, 10], medical diagnosis [11, 12], fraud detection [13] and others [14, 15, 16] where this issue is highly significant and the frequency of one class (e.g., cancer) can be 1000 times less than another class (e.g., healthy…”
BackgroundBrain tumor segmentation with Deep Neural Networks
Havaei, M., Davy, A., Warde-Farley, D., Biard, A., Courville, A., Bengio, Y., Pal, C., Jodoin, P.-M., & Larochelle, H. (2017) · Medical Image Analysis · doi:10.1016/j.media.2016.05.004 · Cited by 3,278 · cited 2× · see citing works
“Recently introduced and successfully applied to CNN training for brain tumor segmentation, is two-phase training [28].”
BackgroundMethodExploratory Undersampling for Class-Imbalance Learning
Liu, X., Wu, J., & Zhou, Z. (2008) · IEEE Transactions on Systems Man and Cybernetics Part B (Cybernetics) · doi:10.1109/tsmcb.2008.2007853 · Cited by 2,477 · cited 1× · see citing works
“EasyEnsemble and BalanceCascade are methods that train a committee of classifiers on undersampled subsets [50].”
2013 IEEE Conference on Computer Vision and Pattern Recognition
(2013) · doi:10.1109/cvpr19808.2013 · Cited by 2,439 · see citing works
On the momentum term in gradient descent learning algorithms
Ning Qian, A. (1999) · Neural Networks · doi:10.1016/s0893-6080(98)00116-6 · Cited by 2,318 · cited 1× · see citing works
“9 [56].”
Learning from class-imbalanced data: Review of methods and applications
Guo, H.-X., Li, Y.-J., Shang, J., Gu, M.-Y., Huang, Y.-Y., & Gong, B. (2017) · Expert systems with applications · doi:10.1016/j.eswa.2016.12.035 · Cited by 2,212 · cited 5× · see citing works
“In the previous decade the number of publications on imbalance learning problem started growing rapidly [11, 16].”
BackgroundMethodResultInfluentialAddressing the Curse of Imbalanced Training Sets: One-Sided Selection.
Kubát, M., & Matwin, S. (1997) · Cited by 2,161 · see citing works
The foundations of cost-sensitive learning
Elkan, C. (2001) · Cited by 1,812 · see citing works
SMOTEBoost: Improving Prediction of the Minority Class in Boosting
Chawla, N. V., Lazarević, A. M., Hall, L., & Bowyer, K. W. (2003) · Lecture notes in computer science · doi:10.1007/978-3-540-39804-2_12 · Cited by 1,741 · cited 1× · see citing works
“SMOTEBoost, on the other hand, is a combination of boosting and SMOTE oversampling [51].”
BackgroundData Mining and Knowledge Discovery Handbook
Maimon, O., & Rokach, L. (2005) · Springer eBooks · doi:10.1007/b107408 · Cited by 1,559 · see citing works
2011 International Joint Conference on Neural Networks
Boubacar, H. A., Lecoeuche, S., & Maouche, S. (2010) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.2010.2063350 · Cited by 1,481 · see citing works
Deep learning for digital pathology image analysis: A comprehensive tutorial with selected use cases
Janowczyk, A. R., & Madabhushi, A. (2016) · Journal of Pathology Informatics · doi:10.4103/2153-3539.186902 · Cited by 1,355 · cited 1× · see citing works
“One of the most commonly used method in deep learning [16, 30, 31, 32].”
MethodTraining cost-sensitive neural networks with methods addressing the class imbalance problem
Zhou, Z., & Liu, X. (2006) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2006.17 · Cited by 1,354 · see citing works
Machine Learning for the Detection of Oil Spills in Satellite Radar Images
Kubát, M., Holte, R. C., & Matwin, S. (1998) · Machine Learning · doi:10.1023/a:1007452223027 · Cited by 1,255 · see citing works
Age and gender classification using convolutional neural networks
Levi, G., & Hassncer, T. (2015) · IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw.2015.7301352 · Cited by 1,179 · cited 1× · see citing works
“The most commonly used method in both classical machine learning and deep learning is oversampling [11, 35, 36, 37].”
MethodData Mining for Imbalanced Datasets: An Overview
Chawla, N. V. (2009) · Springer eBooks · doi:10.1007/978-0-387-09823-4_45 · Cited by 1,092 · cited 2× · see citing works
“However, it has some significant and long acknowledged limitations, particularly in the context of imbalanced datasets [19].”
BackgroundNeural Network Classifiers Estimate Bayesian a posteriori Probabilities
Richard, M. D., & Lippmann, R. P. (1991) · Neural Computation · doi:10.1162/neco.1991.3.4.461 · Cited by 995 · cited 2× · see citing works
“However, the most basic version simply compensates for prior class probabilities [43].”
BackgroundCost-Sensitive Learning of Deep Feature Representations From Imbalanced Data
Khan, S. H., Hayat, M., Bennamoun, M., Sohel, F. A., & Togneri, R. B. (2017) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2017.2732482 · Cited by 972 · cited 1× · see citing works
“Some previous studies showed results on cost sensitive learning of deep neural networks [24, 25, 26].”
ResultGoodfellow, I. J., Warde-Farley, D., Mirza, M., Courville, A., & Bengio, Y. (2013) · arXiv · doi:10.48550/arxiv.1302.4389 · Cited by 935 · cited 1× · see citing works
“There are exactly 5000 training and 1000 test examples in each class. We do not use any data augmentation but follow standard preprocessing comprising global contrast normalization and ZCA whitening [58]. For CIFAR-10 experiments we use one of the best performing type of CNN model on this dataset, i.e. All-CNN [59]. T…”
MethodC4.5, Class Imbalance, and Cost Sensitivity: Why Under-Sampling beats Over-Sampling
Drummond, C., & Holte, R. C. (2003) · Cited by 829 · see citing works
Neural Networks: Tricks of the Trade
Orr, G., & Müller, K.-R. (1998) · Lecture notes in computer science · doi:10.1007/3-540-49430-8 · Cited by 817 · see citing works
Mazurowski, M. A., Habas, P. A., Zurada, J. M., Lo, J. Y.-C., Baker, J. A., & Tourassi, G. D. (2007) · Neural Networks · doi:10.1016/j.neunet.2007.12.031 · Cited by 777 · cited 3× · see citing works
“It has been established that class imbalance can have significant detrimental effect on training of traditional classifiers [17] including multi-layer perceptrons [18].”
BackgroundMethodSelf-citationProceedings of the 22nd ACM international conference on Multimedia
(2014) · doi:10.1145/2647868 · Cited by 741 · see citing works
Class imbalances versus small disjuncts
Jo, T., & Japkowicz, N. (2004) · ACM SIGKDD Explorations Newsletter · doi:10.1145/1007730.1007737 · Cited by 687 · cited 1× · see citing works
“Cluster-based oversampling first clusters the dataset and then oversamples each cluster separately [36].”
BackgroundData mining for direct marketing: problems and solutions
Ling, C. X., & Cheng-hui, L. (1998) · Cited by 645 · see citing works
Learning from imbalanced data sets with boosting and data generation
Guo, H., & Viktor, H. L. (2004) · ACM SIGKDD Explorations Newsletter · doi:10.1145/1007730.1007736 · Cited by 606 · cited 1× · see citing works
“DataBoost-IM, on the other hand, identifies difficult examples with boosting preprocessing and uses them to generate synthetic data [37].”
MethodTree Induction for Probability-Based Ranking
Provost, F., & Domingos, P. (2003) · Machine Learning · doi:10.1023/a:1024099825458 · Cited by 541 · cited 1× · see citing works
“Since the basic version of ROC is only suitable for binary classification, we use a multi-class modification of it [67]”
MethodAUC: a statistically consistent and more discriminating measure than accuracy
Ling, C. X., Huang, J., & Zhang, H. (2003) · Cited by 501 · see citing works
Training deep neural networks on imbalanced data sets
Wang, S., Liu, W., Wu, J., Cao, L., Meng, Q., & Kennedy, P. J. (2016) · 2016 International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn.2016.7727770 · Cited by 491 · cited 1× · see citing works
“New kinds of loss function for neural networks training were also developed [27].”
BackgroundRelay Backpropagation for Effective Learning of Deep Convolutional Neural Networks
Shen, L., Lin, Z., & Huang, Q. (2016) · Lecture notes in computer science · doi:10.1007/978-3-319-46478-7_29 · Cited by 456 · cited 1× · see citing works
“An oversampling approach specific to neural networks optimized with stochastic gradient descent is class-aware sampling [38].”
BackgroundChan, P. K., & Stolfo, S. J. (1998) · Cited by 452 · see citing works
Learning When Data Sets are Imbalanced and When Costs are Unequal and Unknown
Maloof, M. A. (2003) · Cited by 406 · see citing works
A novelty detection approach to classification
Japkowicz, N., Myers, C. E., & Gluck, M. A. (1995) · Cited by 392 · see citing works
Automated annotation of coral reef survey images
Beijbom, O., Edmunds, P. J., Kline, D. I., Mitchell, B. G., & Kriegman, D. J. (2012) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2012.6247798 · Cited by 341 · cited 1× · see citing works
“There are plenty of examples in domains like computer vision [6, 7, 8, 9, 10], medical diagnosis [11, 12], fraud detection [13] and others [14, 15, 16] where this issue is highly significant and the frequency of one class (e.g., cancer) can be 1000 times less than another class (e.g., healthy…”
BackgroundCost-Sensitive Learning with Neural Networks.
Kukar, M., & Kononenko, I. (1998) · Cited by 244 · see citing works
Nonlinear Autoassociation Is Not Equivalent to PCA
Japkowicz, N., Hanson, S. J., & Gluck, M. A. (2000) · Neural Computation · doi:10.1162/089976600300015691 · Cited by 169 · cited 1× · see citing works
“Then, the classification of a new example is made based on a reconstruction error between the input and output patterns, e.g. absolute error, squared sum of errors, Euclidean or Mahalanobis distance [46, 47, 48].”
MethodImageNet pre-trained models with batch normalizationPreprint
Simon, M., Rodner, E., & Denzler, J. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1612.01452 · Cited by 157 · cited 1× · see citing works
“A model architecture employed for this dataset is ResNet-10 [60], i.e. a residual network [61] with batch normalization layers that are known to accelerate deep networks training [62].”
MethodWang, K., Makond, B., Chen, K.-H., & Wang, K.-M. (2013) · Applied Soft Computing · doi:10.1016/j.asoc.2013.09.014 · Cited by 140 · cited 1× · see citing works
“It has been shown that oversampling is effective, yet it can lead to overfitting [33, 34].”
Background(2004) · Cognitive technologies · doi:10.1007/978-3-642-18859-6 · Cited by 128 · see citing works
Improving Minority Class Prediction Using Case-Specific Feature Weights
Cardie, C., & Nowe, N. (1997) · Smith ScholarWorks (Smith College) · Cited by 109 · see citing works
Neural Network Classification and Prior Class Probabilities
Lawrence, S., Burns, I., Back, A. D., Tsoi, A. C., & Giles, C. L. (2012) · Lecture notes in computer science · doi:10.1007/978-3-642-35289-8_19 · Cited by 100 · see citing works
Classification and knowledge discovery in protein databases
Predrag Radivojac, A., Chawla, N. V., Dunker, A. K., & Zoran Obradovic, A. (2004) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2004.07.008 · Cited by 91 · see citing works
Johnson, B. A., Tateishi, R., & Hoan, N. T. (2013) · International Journal of Remote Sensing · doi:10.1080/01431161.2013.810825 · Cited by 89 · see citing works
The problem of bias in training data in regression problems in medical decision support
Namee, B. M., Cunningham, P., Byrne, S., & Corrigan, O. I. (2002) · Artificial Intelligence in Medicine · doi:10.1016/s0933-3657(01)00092-6 · Cited by 88 · cited 1× · see citing works
“There are plenty of examples in domains like computer vision [6, 7, 8, 9, 10], medical diagnosis [11, 12], fraud detection [13] and others [14, 15, 16] where this issue is highly significant and the frequency of one class (e.g., cancer) can be 1000 times less than another class (e.g., healthy…”
BackgroundOn the relation of performance to editing in nearest neighbor rules
Koplowitz, J., & Brown, T. A. (1981) · Pattern Recognition · doi:10.1016/0031-3203(81)90102-3 · Cited by 87 · cited 1× · see citing works
“A more general approach than undersampling is data decontamination that can involve relabeling of some examples [41, 42].”
BackgroundCost-aware Pre-training for Multiclass Cost-sensitive Deep LearningPreprint
Chung, Y.-A., Lin, H.-T., & Yang, S. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1511.09337 · Cited by 68 · cited 2× · see citing works
“Some previous studies showed results on cost sensitive learning of deep neural networks [24, 25, 26].”
ResultAn Approach to Imbalanced Data Sets Based on Changing Rule Strength
Grzymala‐Busse, J. W., Goodwin, L., Grzymała-Busse, W. J., & Zheng, X. (2004) · Cognitive technologies · doi:10.1007/978-3-642-18859-6_21 · Cited by 65 · cited 1× · see citing works
“There are plenty of examples in domains like computer vision [6, 7, 8, 9, 10], medical diagnosis [11, 12], fraud detection [13] and others [14, 15, 16] where this issue is highly significant and the frequency of one class (e.g., cancer) can be 1000 times less than another class (e.g., healthy…”
BackgroundDetection of concealed cars in complex cargo X-ray imagery using Deep Learning
Jaccard, N., Rogers, T. W., Morton, E. J., & Griffin, L. D. (2017) · Journal of X-Ray Science and Technology · doi:10.3233/xst-16199 · Cited by 63 · cited 1× · see citing works
“One of the most commonly used method in deep learning [16, 30, 31, 32].”
MethodThe Novelty Detection Approach for Different Degrees of Class Imbalance
Lee, H., & Cho, S. (2006) · Lecture notes in computer science · doi:10.1007/11893257_3 · Cited by 51 · cited 1× · see citing works
“This method has proved to work well for extremely high imbalance when classification problem turns info anomaly detection [49].”
MethodTowards Effective Classification of Imbalanced Data with Convolutional Neural Networks
Raj, V., Magg, S., & Wermter, S. (2016) · Lecture notes in computer science · doi:10.1007/978-3-319-46182-3_13 · Cited by 44 · cited 1× · see citing works
“Some previous studies showed results on cost sensitive learning of deep neural networks [24, 25, 26].”
ResultKing, I., Wang, J., Wang, D., & Chan, L. (2006) · Lecture notes in computer science · doi:10.1007/11893295 · Cited by 30 · see citing works
Restricted Decontamination for the Imbalanced Training Sample Problem
Barandela, R., Rangel, E., Sánchez, J. S., & Ferri, F. J. (2003) · Lecture notes in computer science · doi:10.1007/978-3-540-24586-5_52 · Cited by 27 · see citing works
The iNaturalist Species Classification and Detection DatasetPreprint
Horn, G. V., Aodha, O. M., Song, Y., Cui, Y., Sun, C., Shepard, A., Adam, H., Perona, P., & Belongie, S. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1707.06642 · Cited by 18 · see citing works
Novelty Detection Using Auto-Associative Neural Network
Sohn, H., Worden, K., & Farrar, C. R. (2001) · Dynamic Systems and Control · doi:10.1115/imece2001/dsc-24571 · Cited by 16 · cited 1× · see citing works
“absolute error, squared sum of errors, Euclidean or Mahalanobis distance [46, 47, 48].”
Background
Flagged first, then by citations
6 of the 83 references couldn’t be matched to a known paper.
Cited by
Sorted by citations
A survey on Image Data Augmentation for Deep Learning
Shorten, C., & Khoshgoftaar, T. M. (2019) · Journal of Big Data · doi:10.1186/s40537-019-0197-0 · Cited by 13,203 · see citing works
Survey on deep learning with class imbalance
Johnson, J. M., & Khoshgoftaar, T. M. (2019) · Journal of Big Data · doi:10.1186/s40537-019-0192-5 · Cited by 3,032 · see citing works
Class-Balanced Loss Based on Effective Number of Samples
Cui, Y., Jia, M., Lin, T.-Y., Song, Y., & Belongie, S. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00949 · Cited by 3,094 · see citing works
“CNNs trained on such data perform poorly for weakly represented classes [19, 15, 41, 4].”
BackgroundCheXNet: Radiologist-Level Pneumonia Detection on Chest X-Rays with Deep LearningPreprint
Rajpurkar, P., Irvin, J., Zhu, K., Yang, B., Mehta, H., Duan, T., Ding, D. Y., Bagul, A., Langlotz, C. P., Shpanskaya, K. S., et al. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1711.05225 · Cited by 2,283 · see citing works
“We train the model using minibatches of size 16, where we oversample the minority (positive) class (Buda et al., 2017).”
MethodLearning Imbalanced Datasets with Label-Distribution-Aware Margin LossPreprint
Cao, K., Wei, C., Gaidon, A., Aréchiga, N., & Ma, T. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1906.07413 · Cited by 1,080 · see citing works
“, [46, 60, 5, 6] and references therein) and under-sampling the frequent classes (see, e.”
BackgroundMethodInfluentialBBN: Bilateral-Branch Network With Cumulative Learning for Long-Tailed Visual Recognition
Zhou, B., Cui, Q., Wei, X.-S., & Chen, Z.-M. (2020) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr42600.2020.00974 · Cited by 949 · see citing works
“, resampling [25, 1, 14, 1, 11, 2, 7, 21, 4] and cost-sensitive re-weighting [13, 28, 5, 23].”
MethodData imbalance in classification: Experimental evaluation
Thabtah, F., Hammoud, S., Kamalov, F., & Gonsalves, A. H. (2019) · Information Sciences · doi:10.1016/j.ins.2019.11.004 · Cited by 857 · see citing works
Augmentation for small object detection
Kisantal, M., Wojna, Z., Murawski, J., Naruniec, J., & Cho, K. (2019) · doi:10.5121/csit.2019.91713 · Cited by 805 · see citing works
“Oversampling We address the issue of relatively fewer images containing small objects by oversampling those images during training [4].”
BackgroundUçar, F., & Korkmaz, D. (2020) · Medical Hypotheses · doi:10.1016/j.mehy.2020.109761 · Cited by 768 · see citing works
Zhao, Z., Li, T., Wu, J., Sun, C., Wang, S., Yan, R., & Chen, X. (2020) · ISA Transactions · doi:10.1016/j.isatra.2020.08.010 · Cited by 708 · see citing works
“To address the problem of class imbalance, data-level methods and classifier-level methods can be used [54].”
MethodEnd-to-End Autonomous Driving: Challenges and Frontiers
Chen, L., Wu, P., Chitta, K., Jaeger, B., Geiger, A., & Li, H. (2024) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2024.3435937 · Cited by 673 · see citing works
“Various methods mitigate this issue with data processing, including over-sampling [246], [247], under-sampling [248], [249], and data augmentation [250], [251].”
MethodImbalance Problems in Object Detection: A Review
Öksüz, K., Çam, B. C., Kalkan, S., & Akbaş, E. (2020) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2020.2981890 · Cited by 607 · see citing works
“[139] showed that deep neural networks do not suffer from overfitting under oversampling and in fact, better performance can be achieved compared to undersampling.”
BackgroundMethodGupta, A., Anpalagan, A., Guan, L., & Khwaja, A. S. (2021) · Array · doi:10.1016/j.array.2021.100057 · Cited by 571 · see citing works
Mazurowski, M. A., Buda, M., Saha, A., & Bashir, M. R. (2019) · Journal of Magnetic Resonance Imaging · doi:10.1002/jmri.26534 · Cited by 554 · see citing works
“Some solutions have been proposed for this issue such as a higher rate of sampling of the examples from the minority class for training.(118) However, despite the solutions that have been proposed to address small dataset size”
BackgroundSelf-citationMeta-Weight-Net: Learning an Explicit Mapping For Sample WeightingPreprint
Shu, J., Xie, Q., Yi, L., Zhao, Q., Zhou, S., Xu, Z., & Meng, D. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1902.07379 · Cited by 626 · see citing works
Maintaining Discrimination and Fairness in Class Incremental Learning
Zhao, B., Xiao, X., Gan, G., Zhang, B., & Xia, S. (2020) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr42600.2020.01322 · Cited by 525 · see citing works
“Though class imbalance is an old topic and has attracted a lot of attention [4, 15, 17], multi-class imbalance learning is still an open problem [35].”
BackgroundDeep Learning for the Radiographic Detection of Periodontal Bone Loss
Krois, J., Ekert, T., Meinhold, L., Golla, T., Kharbot, B., Wittemeier, A., Dörfer, C. E., & Schwendicke, F. (2019) · Scientific Reports · doi:10.1038/s41598-019-44839-3 · Cited by 538 · see citing works
Long-tail learning via logit adjustmentPreprint
Menon, A., Jayasumana, S., Rawat, A. S., Jain, H., Veit, A., & Kumar, S. (2020) · International Conference on Learning Representations · doi:10.48550/arxiv.2007.07314 · Cited by 1,166 · see citing works
“Real-world classification problems typically exhibit a long-tailed label distribution, wherein most labels are associated with only a few samples (Van Horn & Perona, 2017; Buda et al., 2017; Liu et al., 2019).”
BackgroundBalanced Meta-Softmax for Long-Tailed Visual RecognitionPreprint
Ren, J., Yu, C., sheng, S., Ma, X., Zhao, H., Yi, S., & Li, H. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2007.10740 · Cited by 439 · see citing works
“Specifically, re-sampling strategies [19, 5, 8, 10, 26, 2, 1] try to restore the true distributions from the imbalanced training data.”
BackgroundHidden stratification causes clinically meaningful failures in machine learning for medical imaging
Oakden-Rayner, L., Dunnmon, J., Carneiro, G., & Re, C. (2020) · Proceedings of the ACM Conference on Health, Inference, and Learning · doi:10.1145/3368555.3384468 · Cited by 411 · see citing works
“]. However, the cost of manual labelling is often prohibitive, appropriate augmentation transforms can be difficult to define, and oversampling an underperforming subset can cause degradation on others [14, 28, 4, 40]. As a result, medical imagery analysts have commonly begun either to use semi-automated labelling techn…”
BackgroundConvolutional Neural Networks for the Automatic Identification of Plant Diseases
Boulent, J., Foucher, S., Théau, J., & St-Charles, P.-L. (2019) · Frontiers in Plant Science · doi:10.3389/fpls.2019.00941 · Cited by 402 · see citing works
“The class imbalance problem affects both the convergence of the model and its ability to generalize (Buda et al., 2018).”
BackgroundIdentifying animal species in camera trap images using deep learning and citizen science
Willi, M., Pitman, R. T., Cardoso, A. W., Locke, C. M., Swanson, A., Boyer, A., Veldthuis, M., & Fortson, L. (2018) · Methods in Ecology and Evolution · doi:10.1111/2041-210x.13099 · Cited by 402 · see citing works
GraphSMOTE: Imbalanced Node Classification on Graphs with Graph Neural Networks
Zhao, T., Zhang, X., & Wang, S. (2021) · ACM International Conference on Web Search and Data Mining (WSDM) · doi:10.1145/3437963.3441720 · Cited by 410 · see citing works
“• Based on these observations, generally setting the oversampling scale set a value that can make the class balanced is a good choice, which is consistent with existing work for synthetic minority oversampling [6].”
BackgroundMethodA deep translation (GAN) based change detection network for optical and SAR remote sensing images
Li, X., Du, Z., Huang, Y., & Tan, Z. (2021) · ISPRS Journal of Photogrammetry and Remote Sensing · doi:10.1016/j.isprsjprs.2021.07.007 · Cited by 403 · see citing works
Sagawa, S., Koh, P. W., Hashimoto, T., & Liang, P. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1911.08731 · Cited by 563 · see citing works
“For label shifts, prior work has observed overfitting on rare labels and proposed algorithms to mitigate it (Buda et al., 2018; Cui et al., 2019; Cao et al., 2019).”
BackgroundResultInfluentialDeep learning in bioinformatics: Introduction, application, and perspective in the big data era
Li, Y., Huang, C., Ding, L., Li, Z., Pan, Y., & Gao, X. (2019) · Methods · doi:10.1016/j.ymeth.2019.04.008 · Cited by 401 · see citing works
Elreedy, D., Atiya, A. F., & Kamalov, F. (2023) · Machine Learning · doi:10.1007/s10994-022-06296-4 · Cited by 395 · see citing works
“…approaches for handling the class imbalance problem: the data level approach (Batista et al., 2004; Chawla et al., 2002; Guzmán-Ponce et al., 2021), the cost sensitive approach (Devi et al., 2022), and the algorithm level approach (Mullick et al., 2018; Buda et al., 2018; Ganaie et al., 2021).”
Deep Learning for the Radiographic Detection of Apical Lesions
Ekert, T., Krois, J., Meinhold, L., Elhennawy, K., Emara, R., Golla, T., & Schwendicke, F. (2019) · Journal of Endodontics · doi:10.1016/j.joen.2019.03.016 · Cited by 403 · see citing works
Dual-Sampling Attention Network for Diagnosis of COVID-19 From Community Acquired Pneumonia
Ouyang, X., Huo, J., Xia, L., Shan, F., Liu, J., Mo, Z., Yan, F., Ding, Z., Yang, Q., Song, B., et al. (2020) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2020.2995508 · Cited by 376 · see citing works
“To address this, some class resampling strategies have been proposed in the literature [45], [46], [47], [48], [49].”
BackgroundMethodZhang, H., Huang, L., Wu, C. Q., & Li, Z. (2020) · Computer Networks · doi:10.1016/j.comnet.2020.107315 · Cited by 346 · see citing works
Zhang, G., Wang, M., & Liu, K. (2019) · International Journal of Disaster Risk Science · doi:10.1007/s13753-019-00233-1 · Cited by 344 · see citing works
Fernando, K., & Tsokos, C. P. (2021) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2020.3047335 · Cited by 316 · see citing works
“Experiments conducted in [24] suggest that data sampling strategies have little effect on classification performance; however, results in [25] demonstrate that random oversampling leads to per-”
BackgroundA Comparative Performance Analysis of Data Resampling Methods on Imbalance Medical Data
Khushi, M., Shaukat, K., Alam, T. M., Hameed, I. A., Uddin, S., Luo, S., Yang, X., & Reyes, M. C. (2021) · IEEE Access · doi:10.1109/access.2021.3102399 · Cited by 335 · see citing works
Buda, M., Saha, A., & Mazurowski, M. A. (2019) · Computers in Biology and Medicine · doi:10.1016/j.compbiomed.2019.05.002 · Cited by 336 · see citing works
“Therefore, to account for this fact, we applied oversampling with data augmentation that was proved to help in training convolutional neural networks [23].”
MethodSelf-citationKellenberger, B., Marcos, D., & Tuia, D. (2018) · Remote Sensing of Environment · doi:10.1016/j.rse.2018.06.028 · Cited by 335 · see citing works
“While this has been shown to work well for other tasks (Buda et al., 2017), we empirically found it to cause the CNN to overfit to the small number of animal instances present in the training set.”
MethodThe class imbalance problem in deep learning
Ghosh, K., Bellinger, C., Corizzo, R., Branco, P., Krawczyk, B., & Japkowicz, N. (2022) · Machine Learning · doi:10.1007/s10994-022-06268-8 · Cited by 325 · see citing works
Contrastive Learning based Hybrid Networks for Long-Tailed Image Classification
Wang, P., Han, K., Wei, X.-S., Zhang, L., & Wang, L. (2021) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr46437.2021.00100 · Cited by 316 · see citing works
“Typical methods include data re-sampling [1, 8, 35, 4, 28], loss re-weighting [26, 7, 30, 33], margin modification [3], and data augmentation [19, 6, 29].”
BackgroundMethodImproving Calibration for Long-Tailed Recognition
Zhong, Z., Cui, J., Liu, S., & Jia, J. (2021) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr46437.2021.01622 · Cited by 316 · see citing works
“There are two groups of re-sampling strategies: over-sampling the tail-class images [28, 2, 3] and under-sampling the head-class images [13, 2].”
BackgroundDistribution Alignment: A Unified Framework for Long-tail Visual Recognition
Zhang, S., Li, Z., Yan, S., He, X., & Sun, J. (2021) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr46437.2021.00239 · Cited by 317 · see citing works
“The basic idea of resample-based methods is to over-sample the minority categories [4, 15] or to under-sample the frequent categories in the training process [10, 2].”
BackgroundMethodCReST: A Class-Rebalancing Self-Training Framework for Imbalanced Semi-Supervised Learning
Wei, C., Sohn, K., Mellina, C., Yuille, A., & Yang, F. (2021) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr46437.2021.01071 · Cited by 307 · see citing works
“Various solutions have been proposed to help alleviate bias, such as re-sampling [3, 4], re-”
BackgroundMethodParametric Contrastive Learning
Cui, J., Zhong, Z., Shu, L., Yu, B., & Jia, J. (2021) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00075 · Cited by 313 · see citing works
Effect of Dataset Size and Train/Test Split Ratios in QSAR/QSPR Multiclass Classification
Rácz, A., Bajusz, D., & Héberger, K. (2021) · Molecules · doi:10.3390/molecules26041111 · Cited by 309 · see citing works
“While oversampling can lead to overfitting in “classical” machine learning models, convolutional neural networks (CNNs) are less prone to overfitting [7].”
BackgroundIL2M: Class Incremental Learning With Dual Memory
Belouadah, E., & Popescu, A. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00067 · Cited by 302 · see citing works
“We will pursue work along the following lines: (1) test a constant complexity method such as IL2M for multitask IL to replicate real-life scenarios in which more diversified visual content is encountered, (2) enhance vanilla fine tuning by leveraging recent results which improve imbalanced learning [3] and make curric…”
BackgroundMethodPi, Y., Nath, N. D., & Behzadan, A. H. (2019) · Advanced Engineering Informatics · doi:10.1016/j.aei.2019.101009 · Cited by 301 · see citing works
“Considering crowdsourcing as a possible future direction of this work to scale up data collection by utilizing consumer-grade drones and low computational power, we favor pre-processing methods for data balancing which include oversampling the instances of minority classes, and under-sampling the instances of majority…”
MethodOn the class overlap problem in imbalanced data classification
Vuttipittayamongkol, P., Elyan, E., & Petrovski, A. (2020) · Knowledge-Based Systems · doi:10.1016/j.knosys.2020.106631 · Cited by 301 · see citing works
“Some other reviews focused on the issue of imbalanced data classification in specific contexts such as big data [24, 25], multi-class problem [24, 26] and neural networks [27, 28].”
BackgroundA Survey on Deep Learning-Based Change Detection from High-Resolution Remote Sensing Images
Jiang, H., Peng, M. C., Zhong, Y., Xie, H., Hao, Z., Lin, J.-M., Ma, X., & Hu, X. (2022) · Remote Sensing · doi:10.3390/rs14071552 · Cited by 295 · see citing works
Towards Fairness in Visual Recognition: Effective Strategies for Bias Mitigation
Wang, Z., Qinami, K., Karakozis, I. C., Genova, K., Nair, P., Hata, K., & Russakovsky, O. (2020) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr42600.2020.00894 · Cited by 284 · see citing works
“In relation to the former, [5, 31] study the effect of class-imbalance on learning, while [52] reveal the surprising phenomenon of bias amplification.”
BackgroundDistribution-Balanced Loss for Multi-label Classification in Long-Tailed Datasets
Wu, T., Huang, Q., Liu, Z., Wang, Y., & Lin, D. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-58548-8_10 · Cited by 284 · see citing works
Rethinking the Value of Labels for Improving Class-Imbalanced LearningPreprint
Yang, Y., & Xu, Z. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2006.07529 · Cited by 282 · see citing works
“This posts a major challenge for modern deep learning frameworks [2,5,10,20,53], where even with specialized techniques such as data re-sampling approaches [2,5,41] or class-balanced losses [7,13,26], significant performance drops still remain under extreme class imbalance.”
BackgroundMethodInfluentialTaherkhani, A., Cosma, G., & McGinnity, T. M. (2020) · Neurocomputing · doi:10.1016/j.neucom.2020.03.064 · Cited by 281 · see citing works
“Although class imbalance has been comprehensively studied in classical machine learning methods, it has received less attention in the context of deep learning [5].”
BackgroundMethodInfluentialNavigating the pitfalls of applying machine learning in genomics
Whalen, S., Schreiber, J. M., Noble, W. S., & Pollard, K. S. (2021) · Nature Reviews Genetics · doi:10.1038/s41576-021-00434-9 · Cited by 278 · see citing works
Addressing Class Imbalance in Federated Learning
Wang, L., Xu, S., Wang, X., & Zhu, Q. (2021) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v35i11.17219 · Cited by 273 · see citing works
A survey and analysis of intrusion detection models based on CSE-CIC-IDS2018 Big Data
Leevy, J. L., & Khoshgoftaar, T. M. (2020) · Journal Of Big Data · doi:10.1186/s40537-020-00382-x · Cited by 269 · see citing works
“We point out that the use of AUC as a robust, standalone metric for class imbalance has been demonstrated in several studies [95–97].”
ResultDisentangling Label Distribution for Long-tailed Visual Recognition
Hong, Y., Han, S., Choi, K., Seo, S., Kim, B., & Chang, B. (2021) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr46437.2021.00656 · Cited by 265 · see citing works
“Most long-tailed visual recognition methods can be divided into two strategies: modifying the data sampler to balance the class frequency during optimization [7, 21, 25, 8, 51], and modifying the class-wise weights of the classification loss to increase the importance of tail classes in terms of empirical risk minimiz…”
BackgroundMethodInfluentialFeature Space Augmentation for Long-Tailed Data
Chu, P., Bian, X., Liu, S., & Ling, H. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-58526-6_41 · Cited by 267 · see citing works
A survey on generative adversarial networks for imbalance problems in computer vision tasks
Sampath, V., Maurtua, I., Martín, J. J. A., & Gutierrez, A. (2021) · Journal Of Big Data · doi:10.1186/s40537-021-00414-0 · Cited by 275 · see citing works
“Class imbalance in the real-world image datasets is ubiquitous and can have an adverse effect on performance of ConvNets [14].”
BackgroundInfluentialA Multi-View Deep Learning Framework for EEG Seizure Detection
Yuan, Y., Xun, G., Jia, K., & Zhang, A. (2018) · IEEE Journal of Biomedical and Health Informatics · doi:10.1109/jbhi.2018.2871678 · Cited by 259 · see citing works
“Taking the advantage of the autoencoders, the proposed model is able to recognize instances rather than discriminating among classes, which has proved to work well for the class imbalance problem [75], [76].”
Balanced Contrastive Learning for Long-Tailed Visual Recognition
Zhu, J., Wang, Z., Chen, J., Chen, Y. P., & Jiang, Y. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.00678 · Cited by 246 · see citing works
“To tackle the problem of learning with imbalanced data, early methods mainly focus on re-sampling the training data [1, 2, 14, 33] or re-weighting the loss functions [9, 21, 45] to pay more attention to rare classes.”
MethodGenerative Adversarial Minority Oversampling
Mullick, S. S., Datta, S., & Das, S. (2019) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00178 · Cited by 244 · see citing works
“fairly persistent is such networks, severely affecting both the feature extraction as well as the classification process [48, 21, 49, 5, 22].”
BackgroundDeep Representation Learning on Long-Tailed Data: A Learnable Embedding Augmentation Perspective
Liu, J., Sun, Y., Han, C., Dou, Z., & Li, W. (2020) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr42600.2020.00304 · Cited by 241 · see citing works
“Recent works for feature learning on imbalanced data are mainly divided into three manners: re-sampling [1], reweighting [21], and data augmentation[3].”
BackgroundJozdani, S. E., Johnson, B. A., & Chen, D. (2019) · Remote Sensing · doi:10.3390/rs11141713 · Cited by 240 · see citing works
“Since the training data gathered for both the images were imbalanced, we applied the Synthetic Minority Oversampling Technique (SMOTE) [54] to artificially balance the training set, as this was found beneficial in past studies using imbalanced data [55,56] and machine learning classifiers [57].”
MethodAI meets UAVs: A survey on AI empowered UAV perception systems for precision agriculture
Su, J., Zhu, X., Li, S., & Chen, W. (2022) · Neurocomputing · doi:10.1016/j.neucom.2022.11.020 · Cited by 241 · see citing works
Westphal, E., & Seitz, H. (2021) · Additive manufacturing · doi:10.1016/j.addma.2021.101965 · Cited by 236 · see citing works
Hierarchical Incentive Mechanism Design for Federated Machine Learning in Mobile Networks
Lim, W. Y. B., Xiong, Z.-H., Miao, C., Niyato, D., Yang, Q., Leung, C., & Poor, H. (2020) · IEEE Internet of Things Journal · doi:10.1109/jiot.2020.2985694 · Cited by 181 · see citing works
“In addition, the model owner may also not have a comprehensive coverage across data classes to build an effective model [6].”
BackgroundThölke, P., Mantilla-Ramos, Y.-J., Abdelhedi, H., Maschke, C., Dehgan, A., Harel, Y., Kemtur, A., Berrada, L. M., Sahraoui, M., Young, T., et al. (2023) · NeuroImage · doi:10.1016/j.neuroimage.2023.120253 · Cited by 234 · see citing works
Learning Memory-Augmented Unidirectional Metrics for Cross-modality Person Re-identification
Liu, J., Sun, Y., Zhu, F., Pei, H., Yang, Y., & Li, W. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.01876 · Cited by 231 · see citing works
Targeted Supervised Contrastive Learning for Long-Tailed Recognition
Li, T., Cao, P., Yuan, Y., Fan, L., Yang, Y.-Z., Feris, R., Indyk, P., & Katabi, D. (2021) · Computer Vision and Pattern Recognition · doi:10.1109/cvpr52688.2022.00679 · Cited by 286 · see citing works
“Conventional methods have focused on designing class rebalancing paradigms through data re-sampling [1, 2, 5, 33] or adjusting the loss weights for different classes during training [3, 4, 9, 11, 20, 21].”
BackgroundMethodDelving into deep imbalanced regression
Yang, Y., Zha, K., Chen, Y.-C., Wang, H., & Katabi, D. (2021) · arXiv · doi:10.48550/arxiv.2102.09554 · Cited by 229 · see citing works
“Rather than preserving an ideal uniform distribution over each category, the data often exhibit skewed distributions with a long tail (Buda et al., 2018; Liu et al., 2019), where certain target values have significantly fewer observations.”
Park, S., Hong, Y., Heo, B., Yun, S., & Choi, J. Y. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.00676 · Cited by 222 · see citing works
“(1) No oversampling (vanilla); (2) Random oversampling (ROS) [47], that oversamples minority samples to balance the classes in the training data; (3) Remix [7], which oversamples minority classes by assigning higher weights to the minority labels when using Mixup [54]; (4) Feature space augmentation (FSA) [8].”
BackgroundInfluentialLuo, J., Huang, J., & Li, H. (2020) · Journal of Intelligent Manufacturing · doi:10.1007/s10845-020-01579-w · Cited by 219 · see citing works
Yan, P., Abdulkadir, A., Luley, P.-P., Rosenthal, M., Schatte, G. A., Grewe, B. F., & Stadelmann, T. (2024) · IEEE Access · doi:10.1109/access.2023.3349132 · Cited by 239 · see citing works
“Finally, a weighted loss can be used that balances the loss for the abnormal and normal class in supervised anomaly detection (Buda et al., 2018).”
BackgroundMethodMind the GAP: A balanced corpus of gendered ambiguous pronouns
Webster, K., Recasens, M., Axelrod, V., & Baldridge, J. (2018) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00240 · Cited by 223 · see citing works
“, 2017), and debiasing strategies include expanding and rebalancing data (Torralba and Efros, 2011; Buda, 2017; Ryu et al., 2017; Shankar et al., 2017), and balancing performance across subgroups (Dwork et al.”
Li, L., Chen, Y., Shen, Z., Zhang, X., Sang, J., Ding, Y., Yang, X., Li, ., Chen, M., Jin, C., et al. (2019) · Gastric Cancer · doi:10.1007/s10120-019-00992-2 · Cited by 219 · see citing works
“…between the two types of images might affect the convergence during the training phase and lead to poor generalization ability and model over-fit [24], we increased the sample size by following such image processing methods as image rotate transform, image mirror transform, image cut, image…”
MethodCS-ResNet: Cost-sensitive residual convolutional neural network for PCB cosmetic defect detection
Zhang, H., Jiang, L., & Li, C. (2021) · Expert Systems with Applications · doi:10.1016/j.eswa.2021.115673 · Cited by 213 · see citing works
R oot P ainter : deep learning segmentation of biological images with corrective annotation
Smith, A. G., Han, E., Petersen, J., Olsen, N. A. F., Giese, C., Athmann, M., Dresbøll, D. B., & Thorup‐Kristensen, K. (2022) · New Phytologist · doi:10.1111/nph.18387 · Cited by 213 · see citing works
“, 2019), or to correct for a class imbalance in the data where many examples exist of a majority class (Buda et al., 2018).”
BackgroundA Systematic Review on Imbalanced Learning Methods in Intelligent Fault Diagnosis
Ren, Z., Lin, T., Feng, K., Zhu, Y., Liu, Z., & Yan, K. L. (2023) · IEEE Transactions on Instrumentation and Measurement · doi:10.1109/tim.2023.3246470 · Cited by 214 · see citing works
“In other fields, researchers [33], [34], [340], [341] have analyzed the behavior of machine learning algorithms under imbalanced data and noted that the performance of algorithms under imbalanced data is related to several factors, such as the imbalance rate, sample set size, class mode, and task…”
BackgroundInfluentialDeep Learning in Data-Driven Pavement Image Analysis and Automated Distress Detection: A Review
Gopalakrishnan, K. (2018) · Data · doi:10.3390/data3030028 · Cited by 210 · see citing works
“This creates class imbalance, a problem that has been comprehensively studied in machine learning, but is beginning to be studied and addressed in the context of deep learning [55].”
BackgroundManagement of Thyroid Nodules Seen on US Images: Deep Learning May Match Performance of Radiologists
Buda, M., Wildman‐Tobriner, B., Hoang, J. K., Thayer, D., Tessler, F. N., Middleton, W. D., & Mazurowski, M. A. (2019) · Radiology · doi:10.1148/radiol.2019181343 · Cited by 210 · see citing works
Self-citationA novel lightweight deep convolutional neural network for early detection of oral cancer
Jubair, F., Al‐karadsheh, O., Malamos, D., Mahdi, S. A., Saad, Y., & Hassona, Y. M. (2021) · Oral Diseases · doi:10.1111/odi.13825 · Cited by 209 · see citing works
Influence-Balanced Loss for Imbalanced Visual Classification
Park, S., Lim, J., Jeon, Y., & Choi, J. Y. (2021) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00077 · Cited by 205 · see citing works
“This imbalance between classes can be problematic, since the model trained on such imbalanced data tends to overfit the dominant (majority) classes [18, 14, 4].”
BackgroundMethodBalanced MSE for Imbalanced Visual Regression
Ren, J., Zhang, M., Yu, C., & Liu, Z. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.00777 · Cited by 210 · see citing works
“Compared with imbalanced and long-tailed classification [4,12,24] that studies categorical labels, imbalanced regression focuses on continuous labels, which can be boundless, high-dimensional, and hence more challenging.”
BackgroundAnalyzing Overfitting Under Class Imbalance in Neural Networks for Image Segmentation
Li, Z., Kamnitsas, K., & Glocker, B. (2020) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2020.3046692 · Cited by 202 · see citing works
“Class imbalance, which has been the focus of previous works [4], [19], is a common issue in image classification and image segmentation.”
BackgroundImbalanced data preprocessing techniques for machine learning: a systematic mapping study
de Vargas, V. W., Aranda, J. A. S., Costa, R. D. S., Pereira, P., & Barbosa, J. L. V. (2022) · Knowledge and Information Systems · doi:10.1007/s10115-022-01772-8 · Cited by 202 · see citing works
“After learning with a multi-class CNN, the proposed techniques showed effective results in a wide range of IRs.”
BackgroundInfluentialDeep learning‐based methods for individual recognition in small birds
Ferreira, A. C., Silva, L. R., Renna, F., Brandl, H. B., Renoult, J. P., Farine, D. R., Covas, R., & Doutrelant, C. (2020) · Methods in Ecology and Evolution · doi:10.1111/2041-210x.13436 · Cited by 203 · see citing works
Detecting and interpreting myocardial infarction using fully convolutional neural networks
Strodthoff, N., & Strodthoff, C. (2018) · Physiological Measurement · doi:10.1088/1361-6579/aaf34d · Cited by 206 · see citing works
Li, Y., Martinis, S., & Wieland, M. (2019) · ISPRS Journal of Photogrammetry and Remote Sensing · doi:10.1016/j.isprsjprs.2019.04.014 · Cited by 201 · see citing works
Analyzing the impact of feature selection on the accuracy of heart disease prediction
Pathan, M. S., Nag, A., Pathan, M. M., & Dev, S. (2022) · Healthcare Analytics · doi:10.1016/j.health.2022.100060 · Cited by 200 · see citing works
“The unbalanced nature of the datasets leads to classification errors during the training of ML models [30].”
BackgroundAn improved SPEI drought forecasting approach using the long short-term memory neural network
Dikshit, A., Pradhan, B., & HUETE, A. R. (2021) · Journal of Environmental Management · doi:10.1016/j.jenvman.2021.111979 · Cited by 209 · see citing works
Korkmaz, D., & Açıkgöz, H. (2022) · Engineering Applications of Artificial Intelligence · doi:10.1016/j.engappai.2022.104959 · Cited by 197 · see citing works
Nex, F., Duarte, D., Tonolo, F. G., & Kerle, N. (2019) · Remote Sensing · doi:10.3390/rs11232765 · Cited by 194 · see citing works
“A balanced number of intact and damaged samples were extracted from each data category, preventing problems due to uneven samples in the different classes [49].”
MethodDeep Learning in Image Cytometry: A Review
Gupta, A., Harrison, P. J., Wieslander, H., Pielawski, N., Kartasalo, K., Partel, G., Solorzano, L., Suveer, A., Klemm, A. H., Spjuth, O., et al. (2018) · Cytometry Part A · doi:10.1002/cyto.a.23701 · Cited by 191 · see citing works
“A common problem for many datasets is class imbalance (109).”
BackgroundA Survey of Textual Emotion Recognition and Its Challenges
Deng, J., & Ren, F. (2021) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2021.3053275 · Cited by 201 · see citing works
“In [133], the authors think that data imbalance has a significant impact on the performance of neural network models and conclude that over-sampling is the most effective way to combat this problem.”
BackgroundSayed, G. I., Soliman, M., & Hassanien, A. E. (2021) · Computers in Biology and Medicine · doi:10.1016/j.compbiomed.2021.104712 · Cited by 188 · see citing works
The Devil Is in Classification: A Simple Framework for Long-Tail Instance Segmentation
Wang, T., Yu, L. Z., Kang, B., Li, J., Liew, J. H., Tang, S., Hoi, S. C. H., & Feng, J. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-58568-6_43 · Cited by 184 · see citing works
Language-Grounded Indoor 3D Semantic Segmentation in the Wild
Rozenberszki, D., Litany, O., & Dai, A. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-19827-4_8 · Cited by 185 · see citing works
MetaSAug: Meta Semantic Augmentation for Long-Tailed Visual Recognition
Li, S., Gong, K., Liu, C. H., Wang, Y., Qiao, F., & Cheng, X. (2021) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr46437.2021.00517 · Cited by 182 · see citing works
“Researchers propose to achieve a more balanced data distribution by over-sampling the minority classes [32, 5, 6] or under-sampling the majority classes [14, 19, 5].”
BackgroundSchwendicke, F., Elhennawy, K., Paris, S., Friebertshäuser, P., & Krois, J. (2019) · Journal of Dentistry · doi:10.1016/j.jdent.2019.103260 · Cited by 180 · see citing works
Identifying Autism Spectrum Disorder From Resting-State fMRI Using Deep Belief Network
Huang, Z.-A., Zhu, Z., Yau, C. H., & Tan, K. C. (2020) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2020.3007943 · Cited by 184 · see citing works
Li, D., Cong, A., & Guo, S. (2019) · Automation in Construction · doi:10.1016/j.autcon.2019.01.017 · Cited by 180 · see citing works
Gao, K., Su, J., Jiang, Z., Zeng, L., Feng, Z., Shen, H., Rong, P., Xu, X., Qin, J., Yang, Y., et al. (2020) · Medical Image Analysis · doi:10.1016/j.media.2020.101836 · Cited by 175 · see citing works
- Authors
- Buda, M., Maki, A., & Mazurowski, M. A.
- Affiliations
- Duke University (US)
- KTH Royal Institute of Technology (SE)
- Corresponding author
- Buda, Mateusz
- Venue
- Neural Networks
- Publisher
- Elsevier BV
- Published
- 2018-10
- Volume
- 106
- Pages
- 249-259
- ISSN
- 0893-6080 (ISSN-L)
- OpenAlex
- W2767106145
- Cited by
- 3,088
- References
- 83
- Citations by year
- 2026: 459 · 2025: 507 · 2024: 371 · 2023: 405 · 2022: 405 · 2021: 380 · 2020: 308 · 2019: 189 · 2018: 58 · 2017: 5
- FWCI
- 131.58
- Type
- journal article
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- Neural Networks (publishedVersion)
- PubMed (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- KTH Publication Database DiVA (KTH Royal Institute of Technology) (submittedVersion, other-oa)
- arXiv (Cornell University)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Imbalanced Data Classification Techniques
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
- Physical Sciences › Computer Science › Artificial Intelligence › Machine Learning and Data Classification
- Keywords
- class imbalance, convolutional neural networks, oversampling, undersampling, two-stage training, thresholding, area under the receiver operating characteristic, multi-class classification, overfitting, MNIST, CIFAR-10, ImageNet
- MeSH terms
- Machine Learning, Humans, Probability, ROC Curve, Neural Networks, Computer
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
