Classification assessment methods
Open accessAbstract
Classification techniques have been applied to many applications in various fields of sciences. There are several ways of evaluating classification algorithms. The analysis of such metrics and its significance must be interpreted correctly for evaluating different learning algorithms. Most of these measures are scalar metrics and some of them are graphical methods. This paper introduces a detailed overview of the classification assessment measures with the aim of providing the basics of these measures and to show how it works to serve as a comprehensive source for researchers who are interested in this field. This overview starts by highlighting the definition of the confusion matrix in binary and multi-class classification problems. Many classification measures are also explained in details, and the influence of balanced and imbalanced data on each metric is presented. An illustrative example is introduced to show (1) how to calculate these measures in binary and multi-class classification problems, and (2) the robustness of some measures against balanced and imbalanced data. Moreover, some graphical measures such as Receiver operating characteristics (ROC), Precision-Recall, and Detection error trade-off (DET) curves are presented with details. Additionally, in a step-by-step approach, different numerical examples are demonstrated to explain the preprocessing steps of plotting ROC, PR, and DET curves.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The LSTM model achieved an overall accuracy of 85% on the test set when trained solely on EEG data.
“Overall, the model demonstrated effective performance with 85% accuracy [39].”
From Multimodal Emotion Recognition: Emotion Classification Through the Integration of EEG and Facial Expressions (2025) · cites this paper as [39]
Section 2.2 defines accuracy as a classification metric, while Section 7 reports an experiment using an SVM on the Iris dataset. Searches for LSTM, EEG, and 85% accuracy found no passage reporting the attributed test-set result. (LLM, conf=0.76)
Experimental — accuracy not yet measuredIn disease diagnosis tasks using artificial intelligence technology, performance evaluation metrics are commonly calculated based on the confusion matrix for binary classification tasks.
“In disease diagnosis tasks using artificial intelligence technology, performance evaluation metrics are commonly calculated based on the confusion matrix for binary classification tasks [69], which include four types of classifications: True Positive (TP), False Positive (FP), True Negative (TN), and False Negative (FN).”
From A Comprehensive Review on Synergy of Multi-Modal Data and AI Technologies in Medical Diagnosis (2024) · cites this paper as [69]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
A systematic analysis of performance measures for classification tasks
Sokolova, M., & Lapalme, G. (2009) · Information Processing & Management · doi:10.1016/j.ipm.2009.03.002 · Cited by 7,515 · see citing works
Comparison of the predicted and observed secondary structure of T4 phage lysozyme
Matthews, B. W. (1975) · Biochimica et Biophysica Acta (BBA) - Protein Structure · doi:10.1016/0005-2795(75)90109-9 · Cited by 5,399 · cited 1× · see citing works
“Some details about each measure are as follow: Matthews correlation coefficient ( MCC ): this metric was introduced by Brian W. Matthews in 1975 [14], and it represents the correlation between the observed and predicted classifications, and it is calculated directly from the confusion matrix as in…”
MethodSaito, T., & Rehmsmeier, M. (2015) · PLOS ONE · doi:10.1371/journal.pone.0118432 · Cited by 5,309 · see citing works
Diagnostic tests 4: likelihood ratios
Deeks, J. J., & Altman, D. G. (2004) · BMJ · doi:10.1136/bmj.329.7458.168 · Cited by 1,600 · see citing works
Optimal classifier for imbalanced data using Matthews Correlation Coefficient metric
Boughorbel, S., Jarray, F., & El-Anbari, M. (2017) · PLOS ONE · doi:10.1371/journal.pone.0177678 · Cited by 1,222 · cited 2× · see citing works
“A coefficient of þ 1 indicates a perfect prediction, − 1 represents total disagreement between prediction and true values and zero means that no better than random prediction [16,3].”
Background
Flagged first, then by citations
Research works printed but not linked: 8
- R.M. Bolle, J.H. Connell, S. Pankanti, N.K. Ratha, A.W. Senior, Guide to biometrics, Springer Science & Business Media, 2013.
- J. Davis, M. Goadrich, The relationship between precision-recall and roc curves, in: Proceedings of the 23rd International Conference on Machine Learning, ACM, 2006, pp. 233-240.
- R.O. Duda, P.E. Hart, D.G. Stork, et al., Pattern Classification, vol. 2, Wiley, New York, 2001. second ed.
- V. Garcia, R.A. Mollineda, J.S. Sanchez, Theoretical analysis of a performance measure for imbalanced data, in: 20th International Conference on Pattern Recognition (ICPR), IEEE, 2010, pp. 617-620.
- H. He, E.A. Garcia, Learning from imbalanced data, IEEE Trans. Knowledge Data Eng. 21 (9) (2009) 1263-1284.
- C.E. Metz, Basic principles of roc analysis, in: Seminars in nuclear medicine, vol. 8, Elsevier, 1978, pp. 283-298.
- M. Sokolova, N. Japkowicz, S. Szpakowicz, Beyond accuracy, f-score and roc: a family of discriminant measures for performance evaluation, in: Australasian Joint Conference on Artificial Intelligence, Springer, 2006, pp. 1015-1021.
- A. Srinivasan, Note on the location of optimal classifiers in n-dimensional roc space. Technical Report PRG-TR-2-99, Oxford University Computing Laboratory, Oxford, England, 1999.
Cited by
Sorted by citations
Chicco, D., & Jurman, G. (2020) · BMC Genomics · doi:10.1186/s12864-019-6413-7 · Cited by 6,143 · see citing works
A Comprehensive Review on Synergy of Multi-Modal Data and AI Technologies in Medical Diagnosis
Xu, X., Li, J., Zhu, Z., Zhao, L., Wang, H., Song, C., Chen, Y., Zhao, Q., Yang, J., & Pei, Y. (2024) · Bioengineering · doi:10.3390/bioengineering11030219 · Cited by 242 · see citing works
Human performance in detecting deepfakes: A systematic review and meta-analysis of 56 papers
Diel, A., Lalgi, T., Schröter, I. C., MacDorman, K. F., Teufel, M., & Bäuerle, A. (2024) · Computers in Human Behavior Reports · doi:10.1016/j.chbr.2024.100538 · Cited by 64 · see citing works
Deep learning-based ResNeXt model in phycological studies for future
Yadav, D. P., Jalal, A. S., Garlapati, D., Hossain, K., Goyal, A., & Pant, G. (2020) · Algal Research · doi:10.1016/j.algal.2020.102018 · Cited by 61 · see citing works
Predicting employee absenteeism for cost effective interventions
Lawrance, N., Petrides, G., & Guerry, M.-A. (2021) · Decision Support Systems · doi:10.1016/j.dss.2021.113539 · Cited by 40 · see citing works
Ghi, T., Conversano, F., Ramirez Zegarra, R., Pisani, P., Dall'Asta, A., Lanzone, A., Lau, W., Vimercati, A., Iliescu, D. G., Mappa, I., et al. (2022) · Ultrasound in Obstetrics & Gynecology · doi:10.1002/uog.23739 · Cited by 32 · see citing works
Genome-based prediction of Bayesian linear and non-linear regression models for ordinal data
Pérez‐Rodríguez, P., Flores‐Galarza, S., Vaquera‐Huerta, H., del Valle‐Paniagua, D. H., Montesinos‐López, O. A., & Crossa, J. (2020) · The Plant Genome · doi:10.1002/tpg2.20021 · Cited by 24 · see citing works
Pevy, N., Christensen, H., Walker, T., & Reuber, M. (2021) · Seizure · doi:10.1016/j.seizure.2021.06.009 · Cited by 23 · see citing works
An ensembled framework for human breast cancer survivability prediction using deep learning
Mustafa, E., Jadoon, E. K., Khaliq-uz-Zaman, S., Humayun, M. A., & Maray, M. (2023) · Diagnostics · doi:10.3390/diagnostics13101688 · Cited by 24 · see citing works
Erdem Güler, S., & Patlar Akbulut, F. (2025) · IEEE Access · doi:10.1109/access.2025.3538642 · Cited by 21 · see citing works
Santamaría-García, H., Baez, S., Aponte-Canencio, D. M., Pasciarello, G. O., Donnelly-Kehoe, P. A., Maggiotti, G., Matallana, D., Hesse, E., Neely, A., Zapata, J. G., et al. (2021) · Patterns · doi:10.1016/j.patter.2020.100176 · Cited by 18 · see citing works
Automated segmentation of insect anatomy from micro‐CT images using deep learning
Toulkeridou, E., Gutierrez, C. E., Baum, D., Doya, K., & Economo, E. P. (2023) · Natural Sciences · doi:10.1002/ntls.20230010 · Cited by 13 · see citing works
t-EER: Parameter-free tandem evaluation of countermeasures and biometric comparators
Kinnunen, T. H., Lee, K. A., Tak, H., Evans, N., & Nautsch, A. (2024) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2023.3313648 · Cited by 4 · see citing works
- Authors
- Tharwat, A.
- Affiliations
- Frankfurt University of Applied Sciences (DE)
- Corresponding author
- Tharwat, Alaa
- ORCID
- Tharwat, Alaa
- Venue
- Applied Computing and Informatics
- Publisher
- Emerald
- Published
- 2020-07-30
- Volume
- 17
- Issue
- 1
- Pages
- 168-192
- ISSN
- 2210-8327 (ISSN-L)
- 2634-1964
- OpenAlex
- W2888728157
- Cited by
- 2,417
- References
- 5
- Citations by year
- 2026: 211 · 2025: 393 · 2024: 373 · 2023: 389 · 2022: 357 · 2021: 380 · 2020: 246 · 2019: 63 · 2018: 3 · 2014: 1
- FWCI
- 97.78
- Type
- journal article
- Language
- en
- Open access
- diamond
- doi.org
- Licence
- Locations
- Applied Computing and Informatics (publishedVersion, cc-by)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Imbalanced Data Classification Techniques
- Physical Sciences › Computer Science › Artificial Intelligence › Machine Learning and Data Classification
- Physical Sciences › Computer Science › Information Systems › Data Mining and Machine Learning Applications
- Keywords
- classification evaluation metrics, confusion matrix, binary classification, multi-class classification, class imbalance, precision, memory retrieval, F1 score, receiver operating characteristic, precision-recall curve
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
