The DOI matches its Crossref record; 10 of the 10 references it cites were found in a registry or an index.
The impact of class imbalance in classification performance metrics based on the binary confusion matrix
Open accessAbstract
A major issue in the classification of class imbalanced datasets involves the determination of the most suitable performance metrics to be used. In previous work using several examples, it has been shown that imbalance can exert a major impact on the value and meaning of accuracy and on certain other well-known performance metrics. In this paper, our approach goes beyond simply studying case studies and develops a systematic analysis of this impact by simulating the results obtained using binary classifiers. A set of functions and numerical indicators are attained which enables the comparison of the behaviour of several performance metrics based on the binary confusion matrix when they are faced with imbalanced datasets. Throughout the paper, a new way to measure the imbalance is defined which surpasses the Imbalance Ratio used in previous studies. From the simulation results, several clusters of performance metrics have been identified that involve the use of Geometric Mean or Bookmaker Informedness as the best null-biased metrics if their focus on classification successes (dismissing the errors) presents no limitation for the specific application where they are used. However, if classification errors must also be considered, then the Matthews Correlation Coefficient arises as the best choice. Finally, a set of null-biased multi-perspective Class Balance Metrics is proposed which extends the concept of Class Balance Accuracy to other performance metrics.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
SMOTE synthetic minority over-sampling technique
Chawla, N. V., Bowyer, K. W., Hall, L. O., & Kegelmeyer, W. P. (2002) · Journal of Artificial Intelligence Research · doi:10.1613/jair.953 · Cited by 32,865
A systematic analysis of performance measures for classification tasks
Sokolova, M., & Lapalme, G. (2009) · Information Processing & Management · doi:10.1016/j.ipm.2009.03.002 · Cited by 7,515
Evaluation: from precision, recall and F-measure to ROC, informedness, markedness and correlation
Powers, D. M. W. (2020) · arXiv · doi:10.48550/arxiv.2010.16061 · Cited by 5,440
Comparison of the predicted and observed secondary structure of T4 phage lysozyme
Matthews, B. W. (1975) · Biochimica et Biophysica Acta (BBA) - Protein Structure · doi:10.1016/0005-2795(75)90109-9 · Cited by 5,399
A Review on Evaluation Metrics for Data Classification Evaluations
M, H., & M.N, S. (2015) · International Journal of Data Mining & Knowledge Management Process · doi:10.5121/ijdkp.2015.5201 · Cited by 2,693
Learning from imbalanced data: open challenges and future directions
Krawczyk, B. (2016) · Progress in Artificial Intelligence · doi:10.1007/s13748-016-0094-0 · Cited by 2,459
The Balanced Accuracy and Its Posterior Distribution
Brodersen, K. H., Ong, C. S., Stephan, K. E., & Buhmann, J. M. (2010) · 2010 20th International Conference on Pattern Recognition · doi:10.1109/icpr.2010.764 · Cited by 1,816
Optimal classifier for imbalanced data using Matthews Correlation Coefficient metric
Boughorbel, S., Jarray, F., & El-Anbari, M. (2017) · PLOS ONE · doi:10.1371/journal.pone.0177678 · Cited by 1,222
A Comparison of MCC and CEN Error Measures in Multi-Class Prediction
Jurman, G., Riccadonna, S., & Furlanello, C. (2012) · PLoS ONE · doi:10.1371/journal.pone.0041882 · Cited by 406
Amin, A., Anwar, S., Adnan, A., Nawaz, M., Howard, N., Qadir, J., Hawalah, A., & Hussain, A. (2016) · IEEE Access · doi:10.1109/access.2016.2619719 · Cited by 310
Cited by
Chicco, D., & Jurman, G. (2020) · BMC Genomics · doi:10.1186/s12864-019-6413-7 · Cited by 6,143
Chicco, D., Tötsch, N., & Jurman, G. (2021) · BioData Mining · doi:10.1186/s13040-021-00244-z · Cited by 893
Khan, A. A., Chaudhari, O., & Chandra, R. (2024) · Expert Systems with Applications · doi:10.1016/j.eswa.2023.122778 · Cited by 530
Cross validation for model selection: A review with examples from ecology
Yates, L. A., Aandahl, Z., Richards, S. A., & Brook, B. W. (2023) · Ecological Monographs · doi:10.1002/ecm.1557 · Cited by 459
Pick and Choose: A GNN-based Imbalanced Learning Approach for Fraud Detection
Liu, Y., Ao, X., Qin, Z., Chi, J., Feng, J., Yang, H., & He, Q. (2021) · Proceedings of the Web Conference 2021 · doi:10.1145/3442381.3449989 · Cited by 446
Metadata
- Authors
- Luque, A., Carrasco, A., Martín, A., & de las Heras, A.
- Venue
- Pattern Recognition
- Publisher
- Elsevier BV
- Published
- 2019-07
- Volume
- 91
- Pages
- 216-231
- ISSN
- 0031-3203 (ISSN-L)
- OpenAlex
- W2918408501
- Cited by
- 1,128
- References
- 10
- FWCI
- 52.03
- Type
- journal article
- Language
- en
- Open access
- hybrid
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: The impact of class imbalance in classification performance metrics based on the binary confusion matrix. https://app.cochat.ai/papers/10.1016/j.patcog.2019.02.023
- The paper
- Luque, A., Carrasco, A., Martín, A., & de las Heras, A. (2019). The impact of class imbalance in classification performance metrics based on the binary confusion matrix. Pattern Recognition. https://doi.org/10.1016/j.patcog.2019.02.023
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.