The class imbalance problem: A systematic study1

Japkowicz, Nathalie; Stephen, Shaju

doi:10.3233/ida-2002-6504

Cited by 3,295Imbalanced Data Classification TechniquesMachine Learning and Data ClassificationData Stream Mining Techniques

The class imbalance problem: A systematic study1

Japkowicz, Nathalie, Stephen, Shaju·2002·Intelligent Data Analysis

DOI 10.3233/ida-2002-6504

Open this paper in CoChat — free account

Abstract

In machine learning problems, differences in prior class probabilities -- or class imbalances -- have been reported to hinder the performance of some standard classifiers, such as decision trees. This paper presents a systematic study aimed at answer

Integrity signals

Registry

Crossref · 10.3233/ida-2002-6504

Cites

18 of 18 references verified0 citation errors · 0 not checkable

Statistics

Recomputing p-values needs the paper’s full text.

Claims the paper makes

This paper’s full text isn’t openly available, so its claims and self-check weren’t run. The registry and citation checks still apply.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Naturalistic human activity datasets are often highly unbalanced.
“Naturalistic human activity datasets are often highly unbalanced.”
From Deep Convolutional and LSTM Recurrent Neural Networks for Multimodal Wearable Activity Recognition (2016) · cites this paper as [44]
This phenomenon stems from a classification bias that favours more prevalent classes, a bias that is amplified through successive iterations of learning.
“This phenomenon stems from a classification bias that favours more prevalent classes 164 , a bias that is amplified through successive iterations of learning 165 .”
From Machine culture (2023) · cites this paper as [165]

Addressing the Curse of Imbalanced Training Sets: One-Sided Selection.
Kubát, M., & Matwin, S. (1997) · Cited by 2,161 · see citing works
The foundations of cost-sensitive learning
Elkan, C. (2001) · Cited by 1,812 · see citing works
UCI Repository of Machine Learning Databases
Murphy, P. M. (1994) · Medical Entomology and Zoology · Cited by 1,803 · see citing works
MetaCost
Domingos, P. (1999) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/312129.312220 · Cited by 1,307 · see citing works
Machine Learning for the Detection of Oil Spills in Satellite Radar Images
Kubát, M., Holte, R. C., & Matwin, S. (1998) · Machine Learning · doi:10.1023/a:1007452223027 · Cited by 1,255 · see citing works
Heterogeneous Uncertainty Sampling for Supervised Learning
Lewis, D. D., & Catlett, J. (1994) · Elsevier eBooks · doi:10.1016/b978-1-55860-335-6.50026-x · Cited by 1,174 · see citing works
Adaptive Fraud Detection
Fawcett, T. E., & Provost, F. J. (1997) · Data Mining and Knowledge Discovery · doi:10.1023/a:1009700419189 · Cited by 889 · see citing works
Better Decisions through Science
Swets, J. A., Dawes, R. M., & Monahan, J. C. (2000) · Scientific American · doi:10.1038/scientificamerican1000-82 · Cited by 569 · see citing works
A novelty detection approach to classification
Japkowicz, N., Myers, C. E., & Gluck, M. A. (1995) · Cited by 392 · see citing works
Self-citation
The effect of class distribution on classifier learning: an empirical study
Weiss, G. M., & Provost, F. J. (2001) · doi:10.7282/t3-vpfw-sf95 · Cited by 357 · see citing works
Reducing Misclassification Costs
Pazzani, M. J., Merz, C. J., Murphy, P. M., Ali, K., Hume, T., & Brunk, C. A. (1994) · Elsevier eBooks · doi:10.1016/b978-1-55860-335-6.50034-9 · Cited by 343 · see citing works
Concept learning and the problem of small disjuncts
Holte, R. C., Acker, L., & Porter, B. (1989) · Cited by 342 · see citing works
Overfitting Avoidance as Bias
Schaffer, C. (1993) · Machine Learning · doi:10.1023/a:1022653209073 · Cited by 310 · see citing works
Supervised Versus Unsupervised Binary-Learning by Feedforward Neural Networks
Japkowicz, N. (2001) · Machine Learning · doi:10.1023/a:1007660820062 · Cited by 148 · see citing works
Self-citation
Concept-Learning in the Presence of Between-Class and Within-Class Imbalances
Japkowicz, N. (2001) · Lecture notes in computer science · doi:10.1007/3-540-45153-6_7 · Cited by 135 · see citing works
Self-citation
Learning with Rare Cases and Small Disjuncts
Weiss, G. M. (1995) · Elsevier eBooks · doi:10.1016/b978-1-55860-377-6.50075-x · Cited by 87 · see citing works
A Mixture-of-Experts Framework for Learning from Imbalanced Data Sets
Estabrooks, A., & Japkowicz, N. (2001) · Lecture notes in computer science · doi:10.1007/3-540-44816-0_4 · Cited by 86 · see citing works
Self-citation
Using Unsupervised Learning to Guide Resampling in Imbalanced Data Sets
Nickerson, A., Japkowicz, N., & Milios, E. (2001) · Cited by 66 · see citing works
Self-citation

Flagged first, then by citations

Sorted by citations

Learning from Imbalanced Data
He, H., & Garcia, E. A. (2009) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2008.239 · Cited by 10,359 · see citing works
The precision-recall plot is more informative than the ROC plot when evaluating binary classifiers on imbalanced datasets
Saito, T., & Rehmsmeier, M. (2015) · PLOS ONE · doi:10.1371/journal.pone.0118432 · Cited by 5,309 · see citing works
Ensemble learning: A survey
Sagi, O., & Rokach, L. (2018) · WIREs Data Mining and Knowledge Discovery · doi:10.1002/widm.1249 · Cited by 3,103 · see citing works
A systematic study of the class imbalance problem in convolutional neural networks
Buda, M., Maki, A., & Mazurowski, M. A. (2018) · Neural Networks · doi:10.1016/j.neunet.2018.07.011 · Cited by 3,088 · see citing works
“Na¨ıve version, called random majority undersampling, simply removes a random portion of examples from majority classes [17].”
Background
Class-Balanced Loss Based on Effective Number of Samples
Cui, Y., Jia, M., Lin, T.-Y., Song, Y., & Belongie, S. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00949 · Cited by 3,094 · see citing works
Deep Convolutional and LSTM Recurrent Neural Networks for Multimodal Wearable Activity Recognition
Ordóñez, F., & Roggen, D. (2016) · Sensors · doi:10.3390/s16010115 · Cited by 2,761 · see citing works
Learning from imbalanced data: open challenges and future directions
Krawczyk, B. (2016) · Progress in Artificial Intelligence · doi:10.1007/s13748-016-0094-0 · Cited by 2,459 · see citing works
The Balanced Accuracy and Its Posterior Distribution
Brodersen, K. H., Ong, C. S., Stephan, K. E., & Buhmann, J. M. (2010) · 2010 20th International Conference on Pattern Recognition · doi:10.1109/icpr.2010.764 · Cited by 1,816 · see citing works
Machine learning: a review of classification and combining techniques
Kotsiantis, S. B., Zaharakis, I. D., & Pintelas, P. E. (2006) · Artificial Intelligence Review · doi:10.1007/s10462-007-9052-3 · Cited by 1,582 · see citing works
A Systematic Literature Review on Fault Prediction Performance in Software Engineering
Hall, T., Beecham, S., Bowes, D., Gray, D., & Counsell, S. (2012) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2011.103 · Cited by 1,154 · see citing works
A Survey of Predictive Modeling on Imbalanced Domains
Branco, P., Torgo, L., & Ribeiro, R. P. (2017) · ACM Computing Surveys · doi:10.1145/2907070 · Cited by 1,129 · see citing works
Machine learning with big data: challenges and approaches.
L'Heureux, A., Grolinger, K., Elyamany, H. F., & Capretz, M. A. M. (2017) · IEEE Access · doi:10.1109/access.2017.2696365 · Cited by 1,023 · see citing works
Learning when training data are costly: the effect of class distribution on tree induction
Weiss, G. M., & Provost, F. (2003) · Journal of Artificial Intelligence Research · doi:10.1613/jair.1199 · Cited by 921 · see citing works
Calibrating Probability with Undersampling for Unbalanced Classification
Pozzolo, A. D., Caelen, O., Johnson, R. A., & Bontempi, G. (2015) · 2015 IEEE Symposium Series on Computational Intelligence · doi:10.1109/ssci.2015.33 · Cited by 667 · see citing works
MoodScope: building a mood sensor from smartphone usage patterns
LiKamWa, R., Liu, Y., Lane, N. D., & Zhong, L. (2013) · Proceeding of the 11th annual international conference on Mobile systems, applications, and services · doi:10.1145/2462456.2464449 · Cited by 444 · see citing works
How to develop machine learning models for healthcare
Chen, P.-H. C., Liu, Y., & Peng, L. (2019) · Nature Materials · doi:10.1038/s41563-019-0345-0 · Cited by 274 · see citing works
Predicting Perceived Stress Related to the Covid-19 Outbreak through Stable Psychological Traits and Machine Learning Models
Flesia, L., Monaro, M., Mazza, C., Fietta, V., Colicino, E., Segatto, B., & Roma, P. (2020) · Journal of Clinical Medicine · doi:10.3390/jcm9103350 · Cited by 291 · see citing works
UniKP: a unified framework for the prediction of enzyme kinetic parameters
Yu, H., Deng, H., He, J., Keasling, J. D., & Luo, X. (2023) · Nature Communications · doi:10.1038/s41467-023-44113-1 · Cited by 226 · see citing works
Privacy Risks of General-Purpose Language Models
Pan, X., Zhang, M., Ji, S., & Yang, M. (2020) · 2020 IEEE Symposium on Security and Privacy (SP) · doi:10.1109/sp40000.2020.00095 · Cited by 188 · see citing works
LIMU-BERT
Xu, H., Zhou, P., Tan, R., Li, M., & Shen, G. (2021) · Proceedings of the 19th ACM Conference on Embedded Networked Sensor Systems · doi:10.1145/3485730.3485937 · Cited by 150 · see citing works
Machine culture
Brinkmann, L., Baumann, F., Bonnefon, J.-F., Derex, M., Müller, T. F., Nussberger, A.-M., Czaplicka, A., Acerbi, A., Griffiths, T. L., Henrich, J., et al. (2023) · Nature Human Behaviour · doi:10.1038/s41562-023-01742-2 · Cited by 132 · see citing works
Wrist sensor-based tremor severity quantification in Parkinson's disease using convolutional neural network
Kim, H. B., Lee, W. W., Kim, A., Lee, H. J., Park, H. Y., Jeon, H. S., Kim, S. K., Jeon, B., & Park, K. S. (2018) · Computers in Biology and Medicine · doi:10.1016/j.compbiomed.2018.02.007 · Cited by 124 · see citing works
Identifying incipient dementia individuals using machine learning and amyloid imaging
Mathotaarachchi, S., Pascoal, T. A., Shin, M., Benedet, A. L., Kang, M. S., Beaudry, T., Fonov, V. S., Gauthier, S., & Rosa-Neto, P. (2017) · Neurobiology of Aging · doi:10.1016/j.neurobiolaging.2017.06.027 · Cited by 118 · see citing works
Online and Offline Domain Adaptation for Reducing BCI Calibration Effort
Wu, D. (2017) · IEEE Transactions on Human-Machine Systems · doi:10.1109/thms.2016.2608931 · Cited by 114 · see citing works
A Survey of Privacy Risks and Mitigation Strategies in the Artificial Intelligence Life Cycle
Shahriar, S., Allana, S., Hazratifard, S. M., & Dara, R. (2023) · IEEE Access · doi:10.1109/access.2023.3287195 · Cited by 114 · see citing works
Cortical thickness predicts the first onset of major depression in adolescence
Foland‐Ross, L. C., Sacchet, M. D., Prasad, G., Gilbert, B., Thompson, P. M., & Gotlib, I. H. (2015) · International Journal of Developmental Neuroscience · doi:10.1016/j.ijdevneu.2015.07.007 · Cited by 109 · see citing works
Explaining Outliers by Subspace Separability
Micenkova, B., Ng, R. T., Dang, X.-H., & Assent, I. (2013) · 2013 IEEE 13th International Conference on Data Mining · doi:10.1109/icdm.2013.132 · Cited by 104 · see citing works
A survey on clinical natural language processing in the United Kingdom from 2007 to 2022
Wu, H., Wang, M., Wu, J., Francis, F., Chang, Y.-H., Shavick, A., Dong, H., Poon, M. T. C., Fitzpatrick, N., Levine, A. P., et al. (2022) · npj Digital Medicine · doi:10.1038/s41746-022-00730-6 · Cited by 101 · see citing works
Machine Learning in Precision Agriculture: A Survey on Trends, Applications and Evaluations Over Two Decades
Condran, S., Bewong, M., Islam, M. Z., Maphosa, L., & Zheng, L. (2022) · IEEE Access · doi:10.1109/access.2022.3188649 · Cited by 99 · see citing works
Multilocus interactions at maternal tumor necrosis factor-α, tumor necrosis factor receptors, interleukin-6 and interleukin-6 receptor genes predict spontaneous preterm labor in European-American women
Menon, R., Velez, D. R., Simhan, H., Ryckman, K., Jiang, L., Thorsen, P., Vogel, I., Jacobsson, B., Merialdi, M., Williams, S. M., et al. (2006) · American Journal of Obstetrics and Gynecology · doi:10.1016/j.ajog.2006.03.059 · Cited by 91 · see citing works
Comparison of machine learning classifiers for influenza detection from emergency department free-text reports
López Pineda, A., Ye, Y., Visweswaran, S., Cooper, G. F., Wagner, M. M., & Tsui, F. (. (2015) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2015.08.019 · Cited by 88 · see citing works
Selecting Representative Data Sets
Borovicka, T., Jirina, M., Kordik, P., & Jiri, M. (2012) · Advances in Data Mining Knowledge Discovery and Applications · doi:10.5772/50787 · Cited by 94 · see citing works
Extractive text summarization system to aid data extraction from full text in systematic review development
Bui, D. D. A., Del Fiol, G., Hurdle, J. F., & Jonnalagadda, S. (2016) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2016.10.014 · Cited by 70 · see citing works
ACO Resampling: Enhancing the performance of oversampling methods for class imbalance classification
Li, M., Xiong, A., Wang, L., Deng, S., & Ye, J. (2020) · Knowledge-Based Systems · doi:10.1016/j.knosys.2020.105818 · Cited by 66 · see citing works
Resampling methods improve the predictive power of modeling in class-imbalanced datasets
Lee, P. (2014) · International Journal of Environmental Research and Public Health · doi:10.3390/ijerph110909776 · Cited by 62 · see citing works
Utilizing linguistically enhanced keystroke dynamics to predict typist cognition and demographics
Brizan, D. G., Goodkind, A., Koch, P., Balagani, K., Phoha, V. V., & Rosenberg, A. (2015) · International Journal of Human-Computer Studies · doi:10.1016/j.ijhcs.2015.04.005 · Cited by 54 · see citing works
PDF text classification to leverage information extraction from publication reports
Bui, D. D. A., Del Fiol, G., & Jonnalagadda, S. (2016) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2016.03.026 · Cited by 52 · see citing works
Class Imbalance Problem
Ling, C. X., & Sheng, V. S. (2011) · Encyclopedia of Machine Learning · doi:10.1007/978-0-387-30164-8_110 · Cited by 54 · see citing works
Predicting early dropout in online versus face-to-face guided self-help: A machine learning approach
Gonzalez Salas Duhne, P., Delgadillo, J., & Lutz, W. (2022) · Behaviour Research and Therapy · doi:10.1016/j.brat.2022.104200 · Cited by 38 · see citing works
Adding Stochastic Negative Examples into Machine Learning Improves Molecular Bioactivity Prediction
Cáceres, E. L., Mew, N. C., & Keiser, M. J. (2020) · Journal of Chemical Information and Modeling · doi:10.1021/acs.jcim.0c00565 · Cited by 33 · see citing works
Sample Subset Optimization for Classifying Imbalanced Biological Data
Yang, P., Zhang, Z., Zhou, B. B., & Zomaya, A. Y. (2011) · Lecture Notes in Computer Science · doi:10.1007/978-3-642-20847-8_28 · Cited by 31 · see citing works
Optimal selection of resampling methods for imbalanced data with high complexity
Kim, A., & Jung, I. (2023) · PLOS ONE · doi:10.1371/journal.pone.0288540 · Cited by 28 · see citing works
Uncertainty-Aware COVID-19 Detection from Imbalanced Sound Data
Xia, T., Han, J., Qendro, L., Dang, T., & Mascolo, C. (2021) · Interspeech 2021 · doi:10.21437/interspeech.2021-1320 · Cited by 20 · see citing works
Comparing GPT-based approaches in automated writing evaluation
Liu, Y., Lu, X., & Qi, H. (2025) · Assessing Writing · doi:10.1016/j.asw.2025.100961 · Cited by 19 · see citing works
Blood-Based Transcriptomic Biomarkers Are Predictive of Neurodegeneration Rather Than Alzheimer’s Disease
Shvetcov, A., Thomson, S., Spathos, J., Cho, A.-N., Wilkins, H. M., Andrews, S. J., Delerue, F., Couttas, T. A., Issar, J. K., Isik, F., et al. (2023) · International Journal of Molecular Sciences · doi:10.3390/ijms241915011 · Cited by 12 · see citing works
Using Pareto simulated annealing to address algorithmic bias in machine learning
Blanzeisky, W., & Cunningham, P. (2022) · The Knowledge Engineering Review · doi:10.1017/s0269888922000029 · Cited by 8 · see citing works
“It is well known that class imbalance in the training data can be accentuated in predictions (Japkowicz & Stephen, 2002).”
Entropy Guided Spectrum Based Bug Localization Using Statistical Language Model
Chakraborty, S., Li, Y., Irvine, M., Saha, R., & Ray, B. (2018) · arXiv · doi:10.48550/arxiv.1802.06947 · Cited by 7 · see citing works
“Such unbalanced data of buggy vs. non-buggy lines poses a threat to the efﬁciency of any classiﬁcation probelm [28, 12].”
Background
Classification with LSTM Networks in User Behaviour Analytics with Unbalanced Environment
Parshutin, S., Kirshners, A., Kornijenko, Y., Zabiniako, V., Gasparovica-Asite, M., & Rozkalns, A. (2021) · Automatic Control and Computer Sciences · doi:10.3103/s0146411621010077 · Cited by 7 · see citing works
Flagging incorrect nucleotide sequence reagents in biomedical papers: To what extent does the leading publication format impede automatic error detection?
Labbé, C., Cabanac, G., West, R. A., Gautier, T., Favier, B., & Byrne, J. A. (2020) · Scientometrics · doi:10.1007/s11192-020-03463-z · Cited by 6 · see citing works

DETERMINING QUALITY REQUIREMENTS AT THE UNIVERSITIES TO IMPROVE THE QUALITY OF EDUCATION
Mustafa, N. H., Kelesbayev, D., & Alibekova, Z. D. (2018) · The Journal of Economic Research & Business Administration · doi:10.26577/be-2018-2-2141
Susquehanna Chorale Spring Concert "Roots and Wings"
Hawkins, D. (2017)
Personality correlates and utilitarian judgments in the everyday context: Psychopathic traits and differential effects of empathy, social dominance orientation, and dehumanization beliefs
Takamatsu, R. (2019) · Personality and Individual Differences · doi:10.1016/j.paid.2019.03.029 · Cited by 22 · see citing works
Amount of altruistic punishment accounts for subsequent emotional gratification in participants with primary psychopathy
Masui, K., Iriguchi, S., Nomura, M., & Ura, M. (2011) · Personality and Individual Differences · doi:10.1016/j.paid.2011.07.006 · Cited by 14 · see citing works
The Information Used to Judge Supportiveness Depends on Whether the Judgment Reflects the Personality of Perceivers, the Objective Characteristics of Targets, or their Unique Relationships
Lakey, B., Lutz, C. J., & Scoboria, A. (2004) · Journal of Social and Clinical Psychology · doi:10.1521/jscp.23.6.817.54806 · Cited by 14 · see citing works

Authors: Japkowicz, N., & Stephen, S.
Affiliations: University of Ottawa (CA)
ORCID: Japkowicz, Nathalie
Venue: Intelligent Data Analysis
Publisher: SAGE Publications
Published: 2002-11-15
Volume: 6
Issue: 5
Pages: 429-449
ISSN: 1088-467X (ISSN-L)
1571-4128
DOI: 10.3233/ida-2002-6504
OpenAlex: W1941659294
Cited by: 3,295
References: 18
Citations by year: 2026: 181 · 2025: 229 · 2024: 217 · 2023: 267 · 2022: 252 · 2021: 265 · 2020: 253 · 2019: 226 · 2018: 184 · 2017: 134 · 2016: 114 · 2015: 116 · 2014: 135 · 2013: 105 · 2012: 113
FWCI: 20.06
Type: journal article
Language: en
Open access: closed
Locations: Intelligent Data Analysis (publishedVersion)
Topics: Physical Sciences › Computer Science › Artificial Intelligence › Imbalanced Data Classification Techniques
Physical Sciences › Computer Science › Artificial Intelligence › Machine Learning and Data Classification
Physical Sciences › Computer Science › Artificial Intelligence › Data Stream Mining Techniques
Keywords: class imbalance, classification performance, decision tree

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

The class imbalance problem: A systematic study1

Abstract

Integrity signals

Claims the paper makes

Claims other papers make about this one

Cites

Cited by

Related

Record

About these checks