The DOI matches its Crossref record; 2 of the 2 references it cites were found in a registry or an index.
A study of the behavior of several methods for balancing machine learning training data
Abstract
There are several aspects that might influence the performance achieved by existing learning systems. It has been reported that one of these aspects is related to class imbalance in which examples in training data belonging to one class heavily outnumber the examples in the other class. In this situation, which is found in real world data describing an infrequent but important event, the learning system may have difficulties to learn the concept related to the minority class. In this work we perform a broad experimental evaluation involving ten methods, three of them proposed by the authors, to deal with the class imbalance problem in thirteen UCI data sets. Our experiments provide evidence that class imbalance does not systematically hinder the performance of learning systems. In fact, the problem seems to be related to learning with too few minority class examples in the presence of other complicating factors, such as class overlapping. Two of our proposed methods deal with these conditions directly, allying a known over-sampling method with data cleaning methods in order to produce better-defined class clusters. Our comparative experiments show that, in general, over-sampling methods provide more accurate results than under-sampling methods considering the area under the ROC curve (AUC). This result seems to contradict results previously published in the literature. Two of our proposed methods, Smote + Tomek and Smote + ENN, presented very good results for data sets with a small number of positive examples. Moreover, Random over-sampling, a very simple over-sampling method, is very competitive to more complex over-sampling methods. Since the over-sampling methods provided very good performance results, we also measured the syntactic complexity of the decision trees induced from over-sampled data. Our results show that these trees are usually more complex then the ones induced from original data. Random over-sampling usually produced the smallest increase in the mean number of induced rules and Smote + ENN the smallest increase in the mean number of conditions per rule, when compared among the investigated over-sampling methods.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
SMOTE synthetic minority over-sampling technique
Chawla, N. V., Bowyer, K. W., Hall, L. O., & Kegelmeyer, W. P. (2002) · Journal of Artificial Intelligence Research · doi:10.1613/jair.953 · Cited by 32,865
The class imbalance problem: A systematic study1
Japkowicz, N., & Stephen, S. (2002) · Intelligent Data Analysis · doi:10.3233/ida-2002-6504 · Cited by 3,295
Cited by
He, H., & Garcia, E. A. (2009) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2008.239 · Cited by 10,359
Borderline-SMOTE : A New Over-Sampling Method in Imbalanced Data Sets Learning
Han, H., Wang, W.-Y., & Mao, B.-H. (2005) · Lecture notes in computer science · doi:10.1007/11538059_91 · Cited by 4,331
Improving imbalanced learning through a heuristic oversampling method based on k-means and SMOTE
Douzas, G., Bacao, F., & Last, F. (2018) · Information Sciences · doi:10.1016/j.ins.2018.06.056 · Cited by 1,277
A Systematic Literature Review on Fault Prediction Performance in Software Engineering
Hall, T., Beecham, S., Bowes, D., Gray, D., & Counsell, S. (2012) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2011.103 · Cited by 1,154
A Survey of Predictive Modeling on Imbalanced Domains
Branco, P., Torgo, L., & Ribeiro, R. P. (2017) · ACM Computing Surveys · doi:10.1145/2907070 · Cited by 1,129
One-Class Collaborative Filtering
Pan, R., Zhou, Y., Cao, B., Liu, N. N., Lukose, R., Scholz, M., & Yang, Q. (2008) · 2008 Eighth IEEE International Conference on Data Mining · doi:10.1109/icdm.2008.16 · Cited by 1,015
Clustering-based undersampling in class-imbalanced data
Lin, W.-C., Tsai, C.-F., Hu, Y.-H., & Jhang, J.-S. (2017) · Information Sciences · doi:10.1016/j.ins.2017.05.008 · Cited by 832
Fadlullah, Z. M., Tang, F., Mao, B., Kato, N., Akashi, O., Inoue, T., & Mizutani, K. (2017) · IEEE Communications Surveys & Tutorials · doi:10.1109/comst.2017.2707140 · Cited by 810
Malekloo, A., Ozer, E., AlHamaydeh, M., & Girolami, M. (2022) · Structural Health Monitoring · doi:10.1177/14759217211036880 · Cited by 607
Deep neural networks for bot detection
Kudugunta, S., & Ferrara, E. (2018) · Information Sciences · doi:10.1016/j.ins.2018.08.019 · Cited by 506
On the Class Imbalance Problem
Guo, X., Yin, Y., Dong, C., Yang, G., & Zhou, G. (2008) · 2008 Fourth International Conference on Natural Computation · doi:10.1109/icnc.2008.871 · Cited by 495
Network Intrusion Detection Combined Hybrid Sampling With Deep Hierarchical Network
Jiang, K., Wang, W., Wang, A., & Wu, H. (2020) · IEEE Access · doi:10.1109/access.2020.2973730 · Cited by 446
Goh, K. H., Wang, L., Yeow, A. Y. K., Poh, H., Li, K., Yeow, J. J. L., & Tan, G. Y. H. (2021) · Nature Communications · doi:10.1038/s41467-021-20910-4 · Cited by 415
Wang, K., Tian, J., Zheng, C., Yang, H., Ren, J., Liu, Y., Han, Q., & Zhang, Y. (2021) · Computers in Biology and Medicine · doi:10.1016/j.compbiomed.2021.104813 · Cited by 361
A Comparative Performance Analysis of Data Resampling Methods on Imbalance Medical Data
Khushi, M., Shaukat, K., Alam, T. M., Hameed, I. A., Uddin, S., Luo, S., Yang, X., & Reyes, M. C. (2021) · IEEE Access · doi:10.1109/access.2021.3102399 · Cited by 335
Amin, A., Anwar, S., Adnan, A., Nawaz, M., Howard, N., Qadir, J., Hawalah, A., & Hussain, A. (2016) · IEEE Access · doi:10.1109/access.2016.2619719 · Cited by 310
A hybrid machine learning approach to cerebral stroke prediction based on imbalanced medical dataset
Liu, T., Fan, W., & Wu, C. (2019) · Artificial Intelligence in Medicine · doi:10.1016/j.artmed.2019.101723 · Cited by 220
Data preprocessing in predictive data mining
Alexandropoulos, S.-A. N., Kotsiantis, S. B., & Vrahatis, M. N. (2019) · The Knowledge Engineering Review · doi:10.1017/s026988891800036x · Cited by 209
EXplainable Artificial Intelligence (XAI)—From Theory to Methods and Applications
Ortigossa, E. S., Gonçalves, T., & Nonato, L. G. (2024) · IEEE Access · doi:10.1109/access.2024.3409843 · Cited by 124
Cruz-Jesus, F., Castelli, M., Oliveira, T., Mendes, R., Nunes, C., Sa-Velho, M., & Rosa-Louro, A. (2020) · Heliyon · doi:10.1016/j.heliyon.2020.e04081 · Cited by 118
Goldstein, S. P., Evans, B. C., Flack, D., Juarascio, A., Manasse, S., Zhang, F., & Forman, E. M. (2017) · International Journal of Behavioral Medicine · doi:10.1007/s12529-016-9627-y · Cited by 112
Wan, X., Zhang, X., & Liu, L. (2021) · Applied Sciences · doi:10.3390/app11062606 · Cited by 90
Selecting Representative Data Sets
Borovicka, T., Jirina, M., Kordik, P., & Jiri, M. (2012) · Advances in Data Mining Knowledge Discovery and Applications · doi:10.5772/50787 · Cited by 94
Sha, L., Rakovic, M., Das, A., Gasevic, D., & Chen, G. (2022) · IEEE Transactions on Learning Technologies · doi:10.1109/tlt.2022.3196278 · Cited by 67
Pereira, F. D., Fonseca, S. C., Oliveira, E. H. T., Cristea, A. I., Bellhauser, H., Rodrigues, L., Oliveira, D. B. F., Isotani, S., & Carvalho, L. S. G. (2021) · IEEE Access · doi:10.1109/access.2021.3105956 · Cited by 63
ACO Resampling: Enhancing the performance of oversampling methods for class imbalance classification
Li, M., Xiong, A., Wang, L., Deng, S., & Ye, J. (2020) · Knowledge-Based Systems · doi:10.1016/j.knosys.2020.105818 · Cited by 66
Temporal feature aggregation with attention for insider threat detection from activity logs
Pal, P., Chattopadhyay, P., & Swarnkar, M. (2023) · Expert Systems with Applications · doi:10.1016/j.eswa.2023.119925 · Cited by 52
A deep learning model for Alzheimer’s disease diagnosis based on patient clinical records
Ávila-Jiménez, J. L., Cantón-Habas, V., Carrera-González, M. D. P., Rich-Ruiz, M., & Ventura, S. (2024) · Computers in Biology and Medicine · doi:10.1016/j.compbiomed.2023.107814 · Cited by 50
Sultana, M., Al-Jefri, M., & Lee, J. (2020) · JMIR mHealth and uHealth · doi:10.2196/17818 · Cited by 46
Stress, Schizophrenia, and Violence: A Machine Learning Approach
Kirchebner, J., Sonnweber, M., Nater, U. M., Günther, M., & Lau, S. (2022) · Journal of Interpersonal Violence · doi:10.1177/0886260520913641 · Cited by 34
Hybrid deep learning model using SPCAGAN augmentation for insider threat analysis
R.G., G., Sajjanhar, A., & Xiang, Y. (2024) · Expert Systems with Applications · doi:10.1016/j.eswa.2024.123533 · Cited by 28
Optimal selection of resampling methods for imbalanced data with high complexity
Kim, A., & Jung, I. (2023) · PLOS ONE · doi:10.1371/journal.pone.0288540 · Cited by 28
Randive, K., Mohan, R., & Sivakrishna, A. M. (2023) · Journal of Information Security and Applications · doi:10.1016/j.jisa.2023.103434 · Cited by 27
Human-in-the-loop AI in government: a case study
Benedikt, L., Joshi, C., Nolan, L., Henstra-Hill, R., Shaw, L., & Hook, S. (2020) · Proceedings of the 25th International Conference on Intelligent User Interfaces · doi:10.1145/3377325.3377489 · Cited by 24
Levy, T. J., Coppa, K., Cang, J., Barnaby, D. P., Paradis, M. D., Cohen, S. L., Makhnevich, A., van Klaveren, D., Kent, D. M., Davidson, K. W., et al. (2022) · Nature Communications · doi:10.1038/s41467-022-34646-2 · Cited by 14
“The fi ve imputed development cohorts were combined via concatenation for model development 63 and weighted so as not to arti fi cially increase the sample size, and the minority class (patients that died) was randomly oversampled with replacement 64,65 to correct for class imbalance.”
MethodZhang, H., Zheng, D., Wu, Q., Yan, N., Peng, H., Hu, Q., Peng, Y., Yan, Z., Shi, Z., Bao, C., et al. (2025) · Nature Communications · doi:10.1038/s41467-025-59797-w · Cited by 12
Entropy Guided Spectrum Based Bug Localization Using Statistical Language Model
Chakraborty, S., Li, Y., Irvine, M., Saha, R., & Ray, B. (2018) · arXiv · doi:10.48550/arxiv.1802.06947 · Cited by 7
“In contrast, to be unbiased to any of the metric, we considered all the data and applied state of the art random under-sampling[7] technique.”
MethodChoi, S., Kim, J., Lee, J. H., Lee, Y. N., & Lee, J. H. (2023) · Acta Dermato-Venereologica · doi:10.2340/actadv.v103.11953 · Cited by 5
Metadata
- Authors
- Batista, G. E. A. P. A., Prati, R. C., & Monard, M. C.
- Venue
- ACM SIGKDD Explorations Newsletter
- Published
- 2004-06
- Volume
- 6
- Issue
- 1
- Pages
- 20-29
- ISSN
- 1931-0145 (ISSN-L)
- 1931-0153
- OpenAlex
- W1993220166
- Cited by
- 4,203
- References
- 2
- FWCI
- 27.11
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.
