The DOI matches its Crossref record; 45 of the 45 references it cites were found in a registry or an index.
Data imbalance in classification: Experimental evaluation
Abstract
The advent of Big Data has ushered a new era of scientific breakthroughs. One of the common issues that affects raw data is class imbalance problem which refers to imbalanced distribution of values of the response variable. This issue is present in fraud detection, network intrusion detection, medical diagnostics, and a number of other fields where negatively labeled instances significantly outnumber positively labeled instances. Modern machine learning techniques struggle to deal with imbalanced data by focusing on minimizing the error rate for the majority class while ignoring the minority class. The goal of our paper is demonstrate the effects of class imbalance on classification models. Concretely, we study the impact of varying class imbalance ratios on classifier accuracy. By highlighting the precise nature of the relationship between the degree of class imbalance and the corresponding effects on classifier performance we hope to help researchers to better tackle the problem. To this end, we carry out extensive experiments using 10-fold cross validation on a large number of datasets. In particular, we determine that the relationship between the class imbalance ratio and the accuracy is convex.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
SMOTE synthetic minority over-sampling technique
Chawla, N. V., Bowyer, K. W., Hall, L. O., & Kegelmeyer, W. P. (2002) · Journal of Artificial Intelligence Research · doi:10.1613/jair.953 · Cited by 32,865
A decision-theoretic generalization of on-line learning and an application to boosting
Freund, Y., & Schapire, R. E. (1997) · Journal of Computer and System Sciences · doi:10.1006/jcss.1997.1504 · Cited by 20,466
The WEKA data mining software: an update
Hall, M., Frank, E., Holmes, G., Pfahringer, B., Reutemann, P., & Witten, I. H. (2009) · ACM SIGKDD Explorations Newsletter · doi:10.1145/1656274.1656278 · Cited by 17,750
(1973) · CERN Document Server (European Organization for Nuclear Research) · Cited by 12,895
A Study of Cross-Validation and Bootstrap for Accuracy Estimation and Model Selection
Kohavi, R. (1995) · Zenodo · doi:10.5281/zenodo.19712698 · Cited by 10,909
He, H., & Garcia, E. A. (2009) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2008.239 · Cited by 10,359
Friedman, J., Hastie, T., & Tibshirani, R. (2000) · The Annals of Statistics · doi:10.1214/aos/1016218223 · Cited by 6,884
Proceedings of the 25th international conference on Machine learning
Cohen, W. W., McCallum, A., & Roweis, S. T. (2008) · Cited by 5,691
Pattern classification and scene analysis
Chien, Y. (1974) · IEEE Transactions on Automatic Control · doi:10.1109/tac.1974.1100577 · Cited by 3,398
The class imbalance problem: A systematic study1
Japkowicz, N., & Stephen, S. (2002) · Intelligent Data Analysis · doi:10.3233/ida-2002-6504 · Cited by 3,295
A systematic study of the class imbalance problem in convolutional neural networks
Buda, M., Maki, A., & Mazurowski, M. A. (2018) · Neural Networks · doi:10.1016/j.neunet.2018.07.011 · Cited by 3,088
Survey on deep learning with class imbalance
Johnson, J. M., & Khoshgoftaar, T. M. (2019) · Journal of Big Data · doi:10.1186/s40537-019-0192-5 · Cited by 3,032
Addressing the Curse of Imbalanced Training Sets: One-Sided Selection.
Kubát, M., & Matwin, S. (1997) · Cited by 2,230
Asymptotic Properties of Nearest Neighbor Rules Using Edited Data
Wilson, D. L. (1972) · IEEE Transactions on Systems, Man, and Cybernetics · doi:10.1109/tsmc.1972.4309137 · Cited by 2,130
CLASSIFICATION OF IMBALANCED DATA: A REVIEW
SUN, Y., WONG, A. K. C., & KAMEL, M. S. (2009) · International Journal of Pattern Recognition and Artificial Intelligence · doi:10.1142/s0218001409007326 · Cited by 1,692
Data Mining and Knowledge Discovery Handbook
(2005) · Springer eBooks · doi:10.1007/b107408 · Cited by 1,548
Data Mining for Imbalanced Datasets: An Overview
Chawla, N. V. (2009) · Data Mining and Knowledge Discovery Handbook · doi:10.1007/978-0-387-09823-4_45 · Cited by 1,082
(1976) · IEEE Transactions on Systems, Man, and Cybernetics · doi:10.1109/tsmc.1976.4309452 · Cited by 1,078
Allison, C., Auyeung, B., & Baron-Cohen, S. (2012) · Journal of the American Academy of Child & Adolescent Psychiatry · doi:10.1016/j.jaac.2011.11.003 · Cited by 1,016
A Multiple Resampling Method for Learning from Imbalanced Data Sets
Estabrooks, A., Jo, T., & Japkowicz, N. (2004) · Computational Intelligence · doi:10.1111/j.0824-7935.2004.t01-1-00228.x · Cited by 1,014
Handling imbalanced datasets: A review
Kotsiantis, S. B., Kanellopoulos, D. N., & Pintelas, P. (2006) · Cited by 920
Improving Identification of Difficult Small Classes by Balancing Class Distribution
Laurikkala, J. (2001) · Lecture Notes in Computer Science · doi:10.1007/3-540-48229-6_9 · Cited by 748
AdaCost: Misclassification Cost-Sensitive Boosting
Fan, W., Stolfo, S. J., Zhang, J., & Chan, P. K. (1999) · Cited by 612
Classification with class imbalance problem: A review
Ali, A., Shamsuddin, S. M., & Ralescu, A. (2015) · Cited by 511
On the Class Imbalance Problem
Guo, X., Yin, Y., Dong, C., Yang, G., & Zhou, G. (2008) · 2008 Fourth International Conference on Natural Computation · doi:10.1109/icnc.2008.871 · Cited by 495
Natural Language Processing and Text Mining
(2007) · Springer eBooks · doi:10.1007/978-1-84628-754-1 · Cited by 474
Class Imbalance Problem in Data Mining Review
Longadge, R., & Dongre, S. (2013) · arXiv (Cornell University) · doi:10.48550/arxiv.1305.1707 · Cited by 432
Proceedings of the Fourteenth International Conference on Machine Learning
Fisher, D. (1997) · Cited by 422
Machine learning in autistic spectrum disorder behavioral research: A review and ways forward
Thabtah, F. (2019) · Informatics for Health and Social Care · doi:10.1080/17538157.2017.1399132 · Cited by 292
Self-citationA new machine learning model based on induction of rules for autism detection
Thabtah, F., & Peebles, D. (2020) · Health Informatics Journal · doi:10.1177/1460458218824711 · Cited by 283
Self-citationFinding the Best Classification Threshold in Imbalanced Classification
Zou, Q., Xie, S., Lin, Z., Wu, M., & Ju, Y. (2016) · Big Data Research · doi:10.1016/j.bdr.2015.12.001 · Cited by 281
Proceedings of the Sixteenth International Conference on Machine Learning
Bratko, I., & Džeroski, S. (1999) · Cited by 255
An insight into imbalanced Big Data classification: outcomes and challenges
Fernández, A., del Río, S., Chawla, N. V., & Herrera, F. (2017) · Complex & Intelligent Systems · doi:10.1007/s40747-017-0037-9 · Cited by 230
A new computational intelligence approach to detect autistic features for autism screening
Thabtah, F., Kamalov, F., & Rajab, K. (2018) · International Journal of Medical Informatics · doi:10.1016/j.ijmedinf.2018.06.009 · Cited by 200
Self-citationClass imbalance revisited: a new experimental setup to assess the performance of treatment methods
Prati, R. C., Batista, G. E. A. P. A., & Silva, D. F. (2015) · Knowledge and Information Systems · doi:10.1007/s10115-014-0794-3 · Cited by 181
An accessible and efficient autism screening method for behavioural data and predictive analyses
Thabtah, F. (2018) · Health Informatics Journal · doi:10.1177/1460458218796636 · Cited by 174
Self-citationConsensus Clustering-Based Undersampling Approach to Imbalanced Learning
Onan, A. (2019) · Scientific Programming · doi:10.1155/2019/5901087 · Cited by 164
An empirical comparison of techniques for the class imbalance problem in churn prediction
Zhu, B., Baesens, B., & vanden Broucke, S. K. L. M. (2017) · Information Sciences · doi:10.1016/j.ins.2017.04.015 · Cited by 161
On predicting rare classes with SVM ensembles in scene classification
Yan, R., Liu, Y., Jin, R., & Hauptmann, A. G. (2003) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp.2003.1199097 · Cited by 133
2008 Fourth International Conference on Natural Computation
(2008) · doi:10.1109/icnc14457.2008 · Cited by 116
2017 8th International Conference on Information Technology (ICIT)
(2017) · doi:10.1109/icit41061.2017 · Cited by 75
A visualization cybersecurity method based on features' dissimilarity
AlShboul, R., Thabtah, F., Abdelhamid, N., & Al‐diabat, M. (2018) · Computers & Security · doi:10.1016/j.cose.2018.04.007 · Cited by 23
Self-citationHandling of Imbalanced Data in Text Classification: Category-Based Term Weights
Liu, Y., Loh, H. T., Youcef‐Toumi, K., & Tor, S. B. (2007) · Springer eBooks · doi:10.1007/978-1-84628-754-1_10 · Cited by 13
Enhancing prediction on imbalance data by thresholding technique with noise filtering
Radwan, A. M. (2017) · doi:10.1109/icitech.2017.8080033 · Cited by 13
Cited by
Elreedy, D., Atiya, A. F., & Kamalov, F. (2024) · Machine Learning · doi:10.1007/s10994-022-06296-4 · Cited by 388
Self-citationA Comparative Performance Analysis of Data Resampling Methods on Imbalance Medical Data
Khushi, M., Shaukat, K., Alam, T. M., Hameed, I. A., Uddin, S., Luo, S., Yang, X., & Reyes, M. C. (2021) · IEEE Access · doi:10.1109/access.2021.3102399 · Cited by 335
HCRNNIDS: Hybrid Convolutional Recurrent Neural Network-Based Network Intrusion Detection System
Khan, M. A. (2021) · Processes · doi:10.3390/pr9050834 · Cited by 329
Shaik, T., Tao, X., Li, Y., Dann, C., McDonald, J., Redmond, P., & Galligan, L. (2022) · IEEE Access · doi:10.1109/access.2022.3177752 · Cited by 286
Machine learning based approach to exam cheating detection
Kamalov, F., Sulieman, H., & Santandreu Calonge, D. (2021) · PLOS ONE · doi:10.1371/journal.pone.0254340 · Cited by 102
Self-citation
Metadata
- Authors
- Thabtah, F., Hammoud, S., Kamalov, F., & Gonsalves, A.
- Venue
- Information Sciences
- Publisher
- Elsevier BV
- Published
- 2020-03
- Volume
- 513
- Pages
- 429-441
- ISSN
- 0020-0255 (ISSN-L)
- OpenAlex
- W2989219518
- Cited by
- 850
- References
- 45
- FWCI
- 26.13
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.