Beyond Accuracy, F-Score and ROC: A Family of Discriminant Measures for Performance Evaluation
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
ML practitioners can measure performance in a multitude of ways, and there has been much discussion concerning the choice of different performance measures and approaches.
“ML practitioners can measure performance in a multitude of ways, and there has been much discussion concerning the choice of diferent performance measures and approaches [84,267].”
From Fairness in machine learning: A survey (2024) · cites this paper as [267]
Cites
Statistical Power Analysis for the Behavioral Sciences
Muller, K. E., & Cohen, J. (1989) · Technometrics · doi:10.2307/1270020 · Cited by 99,588 · see citing works
Statistical power analysis for the behavioral sciences
Cohen, J. (1990) · Computers Environment and Urban Systems · doi:10.1016/0198-9715(90)90050-4 · Cited by 65,406 · see citing works
The Nature of Statistical Learning Theory
Vapnik, V. N. (1995) · doi:10.1007/978-1-4757-2440-0 · Cited by 39,138 · see citing works
Seel, N. M. (2012) · Springer eBooks · doi:10.1007/978-1-4419-1428-6_5195 · Cited by 19,525 · see citing works
Harrison's Principles of Internal Medicine.
(1988) · Annals of Internal Medicine · doi:10.7326/0003-4819-108-3-499_2 · Cited by 12,528 · see citing works
Statistical Comparisons of Classifiers over Multiple Data Sets
Demšar, J. (2006) · doi:10.5555/1248547.1248548 · Cited by 11,966 · see citing works
Witten, I. H. (2008) · Cited by 9,115 · see citing works
Mining and summarizing customer reviews
Hu, M., & Liu, B. (2004) · Proceedings of the tenth ACM SIGKDD international conference on Knowledge discovery and data mining · doi:10.1145/1014052.1014073 · Cited by 7,793 · see citing works
Thumbs up?: sentiment classification using machine learning techniques
Pang, B., Lee, L., & Vaithyanathan, S. (2002) · Proceedings of the ACL-02 conference on Empirical methods in natural language processing - EMNLP '02 · doi:10.3115/1118693.1118704 · Cited by 7,024 · see citing works
Chiang, L. H., Russell, E. L., & Braatz, R. D. (2001) · Advanced textbooks in control and signal processing · doi:10.1007/978-1-4471-0347-9_3 · Cited by 5,053 · see citing works
Approximate Statistical Tests for Comparing Supervised Classification Learning Algorithms
Dietterich, T. G. (1998) · Neural Computation · doi:10.1162/089976698300017197 · Cited by 3,840 · see citing works
Chawla, N. V., Japkowicz, N., & Kotcz, A. (2004) · ACM SIGKDD Explorations Newsletter · doi:10.1145/1007730.1007733 · Cited by 2,074 · see citing works
Self-citationHarrison's Principles of Internal Medicine
Verghese, A. (1994) · JAMA · doi:10.1001/jama.1994.03520080084052 · Cited by 1,367 · see citing works
Selecting the right objective measure for association analysis
Tan, P., Kumar, V., & Srivastava, J. (2003) · Information Systems · doi:10.1016/s0306-4379(03)00072-3 · Cited by 573 · see citing works
Cherkassky, V., & Mulier, F. M. (1998) · Cited by 384 · see citing works
Methods in behavioral research
Cozby, P. C. (1977) · Cited by 374 · see citing works
SPECIAL ISSUE ON LEARNING FROM IMBALANCED DATA SETS
Chawla, N. V., Japkowicz, N., & Kołcz, A. (2004) · Cited by 325 · see citing works
Self-citationInterval estimation for Cohen's kappa as a measure of agreement
Blackman, N. J., & Koval, J. J. (2000) · Statistics in Medicine · doi:10.1002/(sici)1097-0258(20000315)19:5 · Cited by 219 · see citing works
Noninvasive Carotid Artery Testing: A Meta-analytic Review
Blakeley, D. D., Oddone, E. Z., Hasselblad, V., et al. (1995) · Annals of Internal Medicine · doi:10.7326/0003-4819-122-5-199503010-00007 · Cited by 218 · see citing works
Linear Hinge Loss and Average Margin
Gentile, C., & Warmuth, M. K. (1998) · Cited by 112 · see citing works
Learning with Decision Lists of Data-Dependent Features
Marchand, M., & Sokolova, M. (2005) · Cited by 47 · see citing works
Self-citationNoninvasive Carotid Artery Testing
Oddone, E. Ž., Blakeley, D., & Matchar, D. B. (1995) · Annals of Internal Medicine · doi:10.7326/0003-4819-123-8-199510150-00016 · Cited by 41 · see citing works
Improving Melody Classification By Discriminant Feature Extraction And Fusion.
Li, M., & Sleep, M. R. (2004) · UEA Digital Repository (University of East Anglia) · doi:10.5281/zenodo.1416727 · Cited by 7 · see citing works
Learning from communication data: Language in electronic business negotiations
Sokolova, M. (2006) · uO Research (University of Ottawa) · doi:10.20381/ruor-12891 · Cited by 4 · see citing works
Self-citationSupporting user task based conversations via e-mail.
Boparai, J., & Kay, J. (2002) · Australasian Document Computing Symposium · Cited by 2 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
A systematic analysis of performance measures for classification tasks
Sokolova, M., & Lapalme, G. (2009) · Information Processing & Management · doi:10.1016/j.ipm.2009.03.002 · Cited by 7,515 · see citing works
Self-citationChicco, D., & Jurman, G. (2020) · BMC Genomics · doi:10.1186/s12864-019-6413-7 · Cited by 6,143 · see citing works
“…when the dataset is unbalanced (the number of samples in one class is much larger than the number of samples in the other classes), accuracy cannot be considered a reliable measure anymore, because it provides an overoptimistic estimation of the classifier ability on the majority class [32–35].”
BackgroundLei, Y., Jia, F., Lin, J., Xing, S., & Ding, S. X. (2016) · IEEE Transactions on Industrial Electronics · doi:10.1109/tie.2016.2519325 · Cited by 1,191 · see citing works
Fairness in machine learning: A survey
Caton, S., & Haas, C. (2024) · ACM Computing Surveys · doi:10.1145/3616865 · Cited by 646 · see citing works
Deep learning based multimodal biomedical data fusion: an overview and comparative review
Duan, J., Xiong, J., Li, Y., & Ding, W. (2024) · Information Fusion · doi:10.1016/j.inffus.2024.102536 · Cited by 284 · see citing works
Machine learning algorithms for social media analysis: A survey
T.K., B., Annavarapu, C. S. R., & Bablani, A. (2021) · Computer Science Review · doi:10.1016/j.cosrev.2021.100395 · Cited by 276 · see citing works
Log-based anomaly detection with deep learning: how far are we?
Le, V.-H., & Zhang, H. (2022) · Proceedings of the 44th International Conference on Software Engineering · doi:10.1145/3510003.3510155 · Cited by 257 · see citing works
A Survey on LLM-Generated Text Detection: Necessity, Methods, and Future Directions
Wu, J., Yang, S., Zhan, R., Yuan, Y., Chao, L. S., & Wong, D. F. (2025) · Computational Linguistics · doi:10.1162/coli_a_00549 · Cited by 212 · see citing works
BreakHis based breast cancer automatic diagnosis using deep learning: Taxonomy, survey and insights
Benhammou, Y., Achchab, B., Herrera, F., & Tabik, S. (2020) · Neurocomputing · doi:10.1016/j.neucom.2019.09.044 · Cited by 170 · see citing works
Fairness in Machine Learning: A Survey
Caton, S., & Haas, C. (2020) · arXiv · doi:10.48550/arxiv.2010.04053 · Cited by 646 · see citing works
BTR: training asynchronous Boolean models using single-cell expression data
Lim, C. Y., Wang, H., Woodhouse, S., Piterman, N., Wernisch, L., Fisher, J., & Göttgens, B. (2016) · BMC Bioinformatics · doi:10.1186/s12859-016-1235-y · Cited by 81 · see citing works
Wee, C.-Y., Wang, L., Shi, F., Yap, P.-T., & Shen, D. (2014) · Human Brain Mapping · doi:10.1002/hbm.22411 · Cited by 89 · see citing works
Screening articles for systematic reviews with ChatGPT
Syriani, E., David, I., & Kumar, G. (2024) · Journal of Computer Languages · doi:10.1016/j.cola.2024.101287 · Cited by 82 · see citing works
Riganello, F., Larroque, S. K., Bahri, M. A., Heine, L., Martial, C., Carrière, M., Charland-Verville, V., Aubinet, C., Vanhaudenhuyse, A., Chatelle, C., et al. (2018) · Frontiers in Neurology · doi:10.3389/fneur.2018.00769 · Cited by 70 · see citing works
Predicting employee absenteeism for cost effective interventions
Lawrance, N., Petrides, G., & Guerry, M.-A. (2021) · Decision Support Systems · doi:10.1016/j.dss.2021.113539 · Cited by 40 · see citing works
Automatic classification of literature in systematic reviews on food safety using machine learning
van den Bulk, L. M., Bouzembrak, Y., Gavai, A., Liu, N., van den Heuvel, L. J., & Marvin, H. J. P. (2022) · Current Research in Food Science · doi:10.1016/j.crfs.2021.12.010 · Cited by 38 · see citing works
Human-in-the-loop AI in government: a case study
Benedikt, L., Joshi, C., Nolan, L., Henstra-Hill, R., Shaw, L., & Hook, S. (2020) · Proceedings of the 25th International Conference on Intelligent User Interfaces · doi:10.1145/3377325.3377489 · Cited by 24 · see citing works
Deep learning for Amazon satellite image analysis
Bragilevsky, L., & Bajic, I. V. (2017) · 2017 IEEE Pacific Rim Conference on Communications, Computers and Signal Processing (PACRIM) · doi:10.1109/pacrim.2017.8121895 · Cited by 17 · see citing works
Multimodal deep learning ensemble framework for skin cancer detection
Saeed, M. A., Afify, Y. M., Badr, N. L., & Helal, N. A. (2025) · Scientific Reports · doi:10.1038/s41598-025-30534-z · Cited by 7 · see citing works
Eilifsen, T. N., Shrestha, B., & Bours, P. (2023) · 2023 21st International Conference on Emerging eLearning Technologies and Applications (ICETA) · doi:10.1109/iceta61311.2023.10343952 · Cited by 3 · see citing works
Althammer, A., Berger, F., Spring, O., Simon, P., Girrbach, F., Dieing, M., Brunner, J. O., Shmygalev, S., Bartenschlager, C. C., & Heller, A. R. (2025) · Information · doi:10.3390/info16100888
- Authors
- Sokolova, M., Japkowicz, N., & Szpakowicz, S.
- Affiliations
- Université de Montréal (CA)
- University of Ottawa (CA)
- Venue
- Lecture Notes in Computer Science
- Publisher
- Springer Berlin Heidelberg
- Published
- 2006
- Pages
- 1015-1021
- ISSN
- 0302-9743 (ISSN-L)
- 1611-3349
- OpenAlex
- W172260869
- Cited by
- 1,974
- References
- 26
- Citations by year
- 2026: 66 · 2025: 192 · 2024: 241 · 2023: 229 · 2022: 235 · 2021: 260 · 2020: 184 · 2019: 133 · 2018: 84 · 2017: 68 · 2016: 54 · 2015: 47 · 2014: 54 · 2013: 31 · 2012: 29
- FWCI
- 5.40
- Type
- conference paper
- Language
- en
- Open access
- closed
- Locations
- Lecture notes in computer science (publishedVersion)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Imbalanced Data Classification Techniques
- Physical Sciences › Computer Science › Artificial Intelligence › Machine Learning and Data Classification
- Physical Sciences › Computer Science › Information Systems › Data Mining and Machine Learning Applications
- Keywords
- classifier evaluation, Youden index, likelihood, discriminatory power, failure prevention, diagnosis, electronic negotiation, ROC curve, F-score, accuracy
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review