Training deep neural networks on imbalanced data sets
Abstract
Deep learning has become increasingly popular in both academic and industrial areas in the past years. Various domains including pattern recognition, computer vision, and natural language processing have witnessed the great power of deep networks. However, current studies on deep learning mainly focus on data sets with balanced class labels, while its performance on imbalanced data is not well examined. Imbalanced data sets exist widely in real world and they have been providing great challenges for classification tasks. In this paper, we focus on the problem of classification using deep network on imbalanced data sets. Specifically, a novel loss function called mean false error together with its improved version mean squared false error are proposed for the training of deep networks on imbalanced data sets. The proposed method can effectively capture classification errors from both majority class and minority class equally. Experiments and comparisons demonstrate the superiority of the proposed approach compared with conventional methods in classifying imbalanced data sets on deep neural networks.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
With loss function improving, deep neural networks exhibited excellent performance for classifying even extremely imbalanced data.
“with the loss function improving, deep neural networks (DNNs) exhibited excellent performance for classifying even extremely imbalanced data [113].”
From Machine Learning Based Toxicity Prediction: From Chemical Structural Description to Transcriptome Analysis (2018) · cites this paper as [113]
Cites
SMOTE synthetic minority over-sampling technique
Chawla, N. V., Bowyer, K. W., Hall, L. O., & Kegelmeyer, W. P. (2002) · Journal of Artificial Intelligence Research · doi:10.1613/jair.953 · Cited by 32,865 · see citing works
An introduction to ROC analysis
Fawcett, T. (2006) · Pattern Recognition Letters · doi:10.1016/j.patrec.2005.10.010 · Cited by 21,879 · see citing works
He, H., & Garcia, E. A. (2009) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2008.239 · Cited by 10,359 · see citing works
Chawla, N. V., Japkowicz, N., & Kotcz, A. (2004) · ACM SIGKDD Explorations Newsletter · doi:10.1145/1007730.1007733 · Cited by 2,074 · see citing works
The foundations of cost-sensitive learning
Elkan, C. (2001) · Cited by 1,812 · see citing works
Training cost-sensitive neural networks with methods addressing the class imbalance problem
Zhou, Z., & Liu, X. (2006) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2006.17 · Cited by 1,354 · see citing works
Domingos, P. (1999) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/312129.312220 · Cited by 1,307 · see citing works
Exploring Strategies for Training Deep Neural Networks
Larochelle, H., Bengio, Y., Louradour, J., & Lamblin, P. (2009) · doi:10.1145/1577069.1577070 · Cited by 1,109 · see citing works
Learning When Data Sets are Imbalanced and When Costs are Unequal and Unknown
Maloof, M. A. (2003) · Cited by 406 · see citing works
Cost-sensitive learning methods for imbalanced data
Thai-Nghe, N., Gantner, Z., & Schmidt-Thieme, L. (2010) · 2022 International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn.2010.5596486 · Cited by 358 · see citing works
Cost-Sensitive Learning with Neural Networks.
Kukar, M., & Kononenko, I. (1998) · Cited by 244 · see citing works
Bag Constrained Structure Pattern Mining for Multi-Graph Classification
Wu, J. X., Xingquan Zhu, A., Zhang, C., & Yu, P. S. (2014) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2013.2297923 · Cited by 168 · see citing works
Self-citationDynamic Sampling Approach to Training Neural Networks for Multiclass Imbalance Classification
Lin, M., Tang, K., & Yao, X. (2013) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2012.2228231 · Cited by 166 · see citing works
Boosting for Multi-Graph Classification
Wu, J. X., Pan, S., Xingquan Zhu, A., & Cai, Z. (2014) · IEEE Transactions on Cybernetics · doi:10.1109/tcyb.2014.2327111 · Cited by 108 · see citing works
Self-citationKernel-based SMOTE for SVM classification of imbalanced datasets
Mathew, J., Luo, M., Pang, C. K., & Chan, H. L. (2015) · Annual Conference of the IEEE Industrial Electronics Society (IECON) · doi:10.1109/iecon.2015.7392251 · Cited by 86 · see citing works
Forecasting of ozone episode days by cost-sensitive neural network methods
Tsai, C.-H., Chang, L., & Chiang, H.-C. (2009) · The Science of The Total Environment · doi:10.1016/j.scitotenv.2008.12.007 · Cited by 52 · see citing works
A Ranked Subspace Learning Method for Gene Expression Data Classification
He, H., & Shen, X. (2007) · International Conference on Artificial Intelligence · Cited by 38 · see citing works
Li, H., Li, J., Chang, P., et al. (2013) · International Journal of Hospitality Management · doi:10.1016/j.ijhm.2013.06.006 · Cited by 37 · see citing works
Cost-Sensitive Neural Network with ROC-Based Moving Threshold for Imbalanced Classification
Krawczyk, B., & Woźniak, M. (2015) · Lecture notes in computer science · doi:10.1007/978-3-319-24834-9_6 · Cited by 34 · see citing works
On compressing weighted time-evolving graphs
Liu, W., Kan, A., Chan, J., et al. (2012) · ACM International Conference on Information and Knowledge Management (CIKM) · doi:10.1145/2396761.2398630 · Cited by 30 · see citing works
Self-citationMining Labelled Tensors by Discovering both their Common and Discriminative Subspaces
Liu, W., Chan, J., Bailey, J. E., et al. (2013) · SIAM International Conference on Data Mining · doi:10.1137/1.9781611972832.68 · Cited by 23 · see citing works
Self-citation
Flagged first, then by citations
Cited by
Sorted by citations
A systematic study of the class imbalance problem in convolutional neural networks
Buda, M., Maki, A., & Mazurowski, M. A. (2018) · Neural Networks · doi:10.1016/j.neunet.2018.07.011 · Cited by 3,088 · see citing works
“New kinds of loss function for neural networks training were also developed [27].”
BackgroundSurvey on deep learning with class imbalance
Johnson, J. M., & Khoshgoftaar, T. M. (2019) · Journal of Big Data · doi:10.1186/s40537-019-0192-5 · Cited by 3,032 · see citing works
Wu, Y., & Wang, G. (2018) · International Journal of Molecular Sciences · doi:10.3390/ijms19082358 · Cited by 193 · see citing works
Fault diagnosis of high-speed train bogie by residual-squeeze net
Su, L., Ma, L., Qin, N., Huang, D., & Kemp, A. H. (2019) · IEEE Transactions on Industrial Informatics · doi:10.1109/tii.2019.2907373 · Cited by 102 · see citing works
Pham, T.-C., Luong, C.-M., Hoang, V.-D., & Doucet, A. (2021) · Scientific Reports · doi:10.1038/s41598-021-96707-8 · Cited by 101 · see citing works
- Authors
- Wang, S., Liu, W., Wu, J., Cao, L., Meng, Q., & Kennedy, P. J.
- Affiliations
- University of Technology Sydney (AU)
- Venue
- 2016 International Joint Conference on Neural Networks (IJCNN)
- Publisher
- IEEE
- Published
- 2016-07
- Pages
- 4368-4374
- ISSN
- 2161-4407 (ISSN-L)
- OpenAlex
- W2551429935
- Cited by
- 491
- References
- 21
- Citations by year
- 2026: 19 · 2025: 42 · 2024: 65 · 2023: 66 · 2022: 74 · 2021: 83 · 2020: 56 · 2019: 46 · 2018: 33 · 2017: 7
- FWCI
- 15.45
- Type
- conference paper
- Language
- en
- Open access
- closed
- Locations
- International Joint Conference on Neural Networks (IJCNN) (publishedVersion)
- UTS ePRESS (University of Technology Sydney) (submittedVersion)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Imbalanced Data Classification Techniques
- Physical Sciences › Computer Science › Artificial Intelligence › Machine Learning and Data Classification
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
- Keywords
- class imbalance, deep neural network, loss function design, minority class, majority class, classification
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
