A systematic comparison of supervised classifiers
Open accessAbstract
Pattern recognition has been employed in a myriad of industrial, commercial and academic applications. Many techniques have been devised to tackle such a diversity of applications. Despite the long tradition of pattern recognition research, there is no technique that yields the best classification in all scenarios. Therefore, as many techniques as possible should be considered in high accuracy applications. Typical related works either focus on the performance of a given algorithm or compare various classification methods. In many occasions, however, researchers who are not experts in the field of machine learning have to deal with practical classification tasks without an in-depth knowledge about the underlying parameters. Actually, the adequate choice of classifiers and parameters in such practical circumstances constitutes a long-standing problem and is one of the subjects of the current paper. We carried out a performance study of nine well-known classifiers implemented in the Weka framework and compared the influence of the parameter configurations on the accuracy. The default configuration of parameters in Weka was found to provide near optimal performance for most cases, not including methods such as the support vector machine (SVM). In addition, the k-nearest neighbor method frequently allowed the best accuracy. In certain conditions, it was possible to improve the quality of SVM by more than 20% with respect to their default parameter configuration.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Commonly used machine learning algorithms differ in their attributes.
“The commonly used ML algorithms have different attributes (Table 4) and therefore it is crucial to choose an appropriate algorithm for the multi-omics analysis.”
From Using machine learning approaches for multi-omics data analysis: A review (2021) · cites this paper as Amancio et al., 2014
Many reviews examine the strengths and weaknesses of different machine learning algorithms on single-omics and multi-omics datasets.
“In the literature, many reviews cover the key strengths and weaknesses of different ML algorithms using single omics (Amancio et al., 2014;López Pineda et al., 2015;Sakr et al., 2017;Uddin et al., 2019) and multi-omics (Ma et al., 2016;Francescatto et al., 2018;J. Xu et al., 2019;Sathyanarayanan et al., 2020) datasets.”
From Using machine learning approaches for multi-omics data analysis: A review (2021) · cites this paper as Amancio et al., 2014
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Horn, R. A., & Johnson, C. R. (1985) · Cambridge University Press eBooks · doi:10.1017/cbo9780511810817 · Cited by 25,760 · see citing works
Data Mining: Practical Machine Learning Tools and Techniques
Witten, I. H., Frank, E., & Hall, M. A. (2011) · Elsevier eBooks · doi:10.1016/c2009-0-19715-5 · Cited by 25,493 · see citing works
Horn, R. A., & Johnson, C. R. (1985) · Cited by 22,203 · see citing works
Seel, N. M. (2012) · Springer eBooks · doi:10.1007/978-1-4419-1428-6_5195 · Cited by 19,525 · see citing works
No free lunch theorems for optimization
Wolpert, D. H., & Macready, W. G. (1997) · IEEE Transactions on Evolutionary Computation · doi:10.1109/4235.585893 · Cited by 14,237 · see citing works
Statistical Comparisons of Classifiers over Multiple Data Sets
Demšar, J. (2006) · doi:10.5555/1248547.1248548 · Cited by 11,966 · see citing works
(2012) · Cited by 9,062 · see citing works
Statistical pattern recognition: a review
Jain, A. K., Duin, P., & Mao, J. (2000) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/34.824819 · Cited by 6,795 · see citing works
Top 10 algorithms in data mining
Wu, X., Kumar, V., Ross Quinlan, J., Ghosh, J., Yang, Q., Motoda, H., McLachlan, G. J., Ng, A., Liu, B., Yu, P. S., et al. (2008) · Knowledge and Information Systems · doi:10.1007/s10115-007-0114-2 · Cited by 5,667 · see citing works
Chiang, L. H., Russell, E. L., & Braatz, R. D. (2001) · Advanced textbooks in control and signal processing · doi:10.1007/978-1-4471-0347-9_3 · Cited by 5,053 · see citing works
Artificial neural networks: a tutorial
Jain, A. K., Mao, J., & Mohiuddin, K. M. (1996) · Computer · doi:10.1109/2.485891 · Cited by 4,989 · see citing works
Data mining and knowledge discovery: making sense out of data
Feyyad, U. M. (1996) · IEEE Expert · doi:10.1109/64.539013 · Cited by 4,644 · see citing works
Pattern Recognition and Machine Learning
Neal, R. M. (2007) · Technometrics · doi:10.1198/tech.2007.s518 · Cited by 4,629 · see citing works
Supervised Machine Learning: A Review of Classification Techniques
Kotsiantis, S. B. (2007) · Informatica · Cited by 4,192 · see citing works
Big data: a revolution that will transform how we live, work, and think
(2013) · Choice Reviews Online · doi:10.5860/choice.50-6804 · Cited by 3,657 · see citing works
Mathematical Statistics and Data Analysis
Rabinowitz, L., & Rice, J. A. (1989) · Technometrics · doi:10.2307/3556160 · Cited by 3,142 · see citing works
Using AUC and accuracy in evaluating learning algorithms
Huang, J., & Ling, C. X. (2005) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2005.50 · Cited by 2,148 · see citing works
The Lack of A Priori Distinctions Between Learning Algorithms
Wolpert, D. H. (1996) · Neural Computation · doi:10.1162/neco.1996.8.7.1341 · Cited by 1,652 · see citing works
Core Vector Machines: Fast SVM Training on Very Large Data Sets
Tsang, I. W.-H., Kwok, J. T.-Y., & Cheung, P.-M. (2005) · doi:10.5555/1046920.1058114 · Cited by 935 · see citing works
Supervised pattern recognition in food analysis
Berrueta, L. Á., Alonso‐Salces, R. M., & Héberger, K. (2007) · Journal of Chromatography A · doi:10.1016/j.chroma.2007.05.024 · Cited by 931 · see citing works
Yang, J. J., Frangi, A. F., Yang, J.-Y., et al. (2005) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2005.33 · Cited by 829 · see citing works
On Comparing Classifiers: Pitfalls to Avoid and a Recommended Approach
Salzberg, S. L. (1997) · Data Mining and Knowledge Discovery · doi:10.1023/a:1009752403260 · Cited by 822 · see citing works
The support vector machine under test
MEYER, D. A., Leisch, F., & Hornik, K. (2003) · Neurocomputing · doi:10.1016/s0925-2312(03)00431-4 · Cited by 773 · see citing works
Book Review: Pattern Recognition and Machine Learning
Nasrabadi, N. M. (2007) · Journal of Electronic Imaging · doi:10.1117/1.2819119 · Cited by 636 · see citing works
Machine learning of molecular electronic properties in chemical compound space
Montavon, G., Rupp, M., Gobre, V., Vazquez-Mayagoitia, A., Hansen, K., Tkatchenko, A., Müller, K.-R., & von Lilienfeld, O. A. (2013) · New Journal of Physics · doi:10.1088/1367-2630/15/9/095003 · Cited by 631 · see citing works
Neural Networks and Related Methods for Classification
Ripley, B. D. (1994) · Journal of the Royal Statistical Society Series B (Statistical Methodology) · doi:10.1111/j.2517-6161.1994.tb01990.x · Cited by 627 · see citing works
Machine learning of molecular electronic properties in chemical compound space
Montavon, G., Rupp, M., Gobre, V. V., Vázquez‐Mayagoitia, Á., Hansen, K., Tkatchenko, A., Müller, K., & von Lilienfeld, O. A. (2013) · MPG.PuRe (Max Planck Society) · doi:10.3929/ethz-b-000071765 · Cited by 414 · see citing works
Nearest-neighbor methods in learning and vision : theory and practice
Shakhnarovich, G., Indyk, P., & Darrell, T. J. (2005) · Cited by 400 · see citing works
Big Data: A Revolution That Will Transform How We Live, Work, and Think
Walker, S. J. (2014) · International Journal of Advertising · doi:10.2501/ija-33-1-181-183 · Cited by 376 · see citing works
Mathematical Statistics and Data Analysis.
Leonard, T., & Rice, J. A. (1989) · Journal of the American Statistical Association · doi:10.2307/2289973 · Cited by 272 · see citing works
Invariant pattern recognition: A review
Wood, J. J. (1996) · Pattern Recognition · doi:10.1016/0031-3203(95)00069-0 · Cited by 247 · see citing works
Wolpert, D. H., & Macready, W. G. (2005) · IEEE Transactions on Evolutionary Computation · doi:10.1109/tevc.2005.856205 · Cited by 233 · see citing works
Automatic Identification of Subcellular Phenotypes on Human Cell ArraysPreprint
Conrad, C., Erfle, H., Warnat, P., Daigle, N., Lörch, T., Ellenberg, J., Pepperkok, R., & Eils, R. (2004) · Genome Research · doi:10.1101/gr.2383804 · Cited by 222 · see citing works
Generalized k-nearest neighbor rules
Bezdek, J. C., Chuah, S. K., & Leep, D. (1986) · Fuzzy Sets and Systems · doi:10.1016/0165-0114(86)90004-7 · Cited by 164 · see citing works
Meta-learning for evolutionary parameter optimization of classifiers
Reif, M., Faisal Shafait, B., & Dengel, A. R. (2012) · Machine Learning · doi:10.1007/s10994-012-5286-7 · Cited by 153 · see citing works
Generating Correlation Matrices
Marsaglia, G., & Olkin, I. (1984) · SIAM Journal on Scientific and Statistical Computing · doi:10.1137/0905034 · Cited by 115 · see citing works
Prechelt, L. (1996) · Neural Networks · doi:10.1016/0893-6080(95)00123-9 · Cited by 107 · see citing works
Big Data Analytics: Disruptive Technologies for Changing the Game
Sathi, A. (2012) · Cited by 103 · see citing works
Hirschberger, M., Qi, Y., & Steuer, R. E. (2005) · European Journal of Operational Research · doi:10.1016/j.ejor.2005.10.014 · Cited by 101 · see citing works
On the parameter optimization of Support Vector Machines for binary classification
Gaspar, P., Carbonell, J., & Oliveira, J. L. (2012) · Berichte aus der medizinischen Informatik und Bioinformatik/Journal of integrative bioinformatics · doi:10.1515/jib-2012-201 · Cited by 82 · see citing works
Parameter Optimization of Kernel-based One-class Classifier on Imbalance Learning
Zhuang, L., & Dai, H. (2006) · Journal of Computers · doi:10.4304/jcp.1.7.32-40 · Cited by 69 · see citing works
A comparison of classification methods for predicting Chronic Fatigue Syndrome based on genetic data
Huang, L.-C., Hsu, S.-Y., & Lin, E. (2009) · Journal of Translational Medicine · doi:10.1186/1479-5876-7-81 · Cited by 67 · see citing works
A Study on Optimal Parameter Tuning for Rocchio Text Classifier
Moschitti, A. (2003) · Lecture notes in computer science · doi:10.1007/3-540-36618-0_30 · Cited by 46 · see citing works
Comparison Between Five Classifiers for Automatic Scoring of Human Sleep Recordings
Becq, G., Charbonnier, S., Chapotot, F., Buguet, A., Bourdon, L., & Baconnier, P. (2005) · Studies in computational intelligence · doi:10.1007/11011620_8 · Cited by 45 · see citing works
Automated, High Accuracy Classification of Parkinsonian Disorders: A Pattern Recognition Approach
Marquand, A. F., Filippone, M., Ashburner, J., Girolami, M., Mourão-Miranda, J., Barker, G. J., Williams, S., Leigh, P. N., & Blain, C. R. V. (2013) · PLoS ONE · doi:10.1371/journal.pone.0069237 · Cited by 44 · see citing works
More "normal" than normal: scaling distributions and complex systems
Willinger, W., Alderson, D., Doyle, J. C., et al. (2004) · CaltechAUTHORS (California Institute of Technology) · doi:10.5555/1161734.1161766 · Cited by 43 · see citing works
Comparing intermittency and network measurements of words and their dependence on authorship
Amancio, D., Altmann, E., Oliveira, O., et al. (2011) · MPG.PuRe (Max Planck Society) · Cited by 43 · see citing works
Self-citationGene classification using expression profiles: a feasibility study
Kuramochi, M., & Karypis, G. (2001) · Proceedings-- IEEE International Symposium on Bioinformatics and Bioengineering · doi:10.1109/bibe.2001.974429 · Cited by 41 · see citing works
Comparing intermittency and network measurements of words and their dependence on authorship
Amancio, D. R., Altmann, E. G., Oliveira, O. N., et al. (2011) · New Journal of Physics · doi:10.1088/1367-2630/13/12/123024 · Cited by 40 · see citing works
Self-citationGENE CLASSIFICATION USING EXPRESSION PROFILES: A FEASIBILITY STUDY
Kuramochi, M., & Karypis, G. (2005) · International Journal of Artificial Intelligence Tools · doi:10.1142/s0218213005002302 · Cited by 39 · see citing works
A Comparative Study of Selected Classifiers with Classification Accuracy in User Profiling
Cufoglu, A., Lohi, M., & Madani, K. (2009) · doi:10.1109/csie.2009.954 · Cited by 37 · see citing works
Associated evolution of a support vector machine-based classifier for pedestrian detection
Cao, X., Xu, Y., Chen, D., et al. (2008) · Information Sciences · doi:10.1016/j.ins.2008.10.020 · Cited by 34 · see citing works
Parameter Optimization of Kernel-Based One-Class Classifier on Imbalance Text Learning
Zhuang, L., & Dai, H. (2006) · Lecture notes in computer science · doi:10.1007/978-3-540-36668-3_47 · Cited by 33 · see citing works
Comparison of Fuzzy and Neural Classifiers for Road Accidents Analysis
Sayed, T., & Abdelwahab, W. (1998) · Journal of Computing in Civil Engineering · doi:10.1061/(asce)0887-3801(1998)12:1(42 · Cited by 30 · see citing works
More "Normal'' Than Normal: Scaling Distributions and Complex Systems
Willinger, W., Alderson, D., Doyle, J. C., et al. (2005) · Winter Simulation Conference · doi:10.1109/wsc.2004.1371310 · Cited by 30 · see citing works
Comparison of fast nearest neighbour classifiers for handwritten character recognition
Micó, L., & Oncina, J. (1998) · Pattern Recognition Letters · doi:10.1016/s0167-8655(98)00007-5 · Cited by 29 · see citing works
On the parameter optimization of Support Vector Machines for binary classification
Gaspar, P., Carbonell, J., & Oliveira, J. L. (2012) · PubMed · doi:10.2390/biecoll-jib-2012-201 · Cited by 29 · see citing works
The Validation and Assessment of Machine Learning: A Game of Prediction from High-Dimensional Data
Pers, T. H., Albrechtsen, A., Holst, C., et al. (2009) · PLoS ONE · doi:10.1371/journal.pone.0006287 · Cited by 27 · see citing works
RBF Network Methods for Face Detection and Attentional Frames
Howell, A. J., & Buxton, H. (2002) · Neural Processing Letters · doi:10.1023/a:1015743231018 · Cited by 21 · see citing works
Algorithm AS 213: Generation of Population Correlation Matrices with Specified Eigenvalues
Lin, S. P., & Bendel, R. B. (1985) · Journal of the Royal Statistical Society Series C (Applied Statistics) · doi:10.2307/2347377 · Cited by 16 · see citing works
Accuracy Measures for the Comparison of ClassifiersPreprint
Labatut, V., & Cherifi, H. (2012) · arXiv (Cornell University) · doi:10.48550/arxiv.1207.3790 · Cited by 13 · see citing works
A Comparative Study of Machine Learning Methods for Detecting Promoters in Bacterial DNA Sequences
Tavares, L. G., Lopes, H. S., & Lima, C. R. E. (2008) · Lecture notes in computer science · doi:10.1007/978-3-540-85984-0_115 · Cited by 12 · see citing works
Comparison of Classifiers for Human Activity Recognition
Perez‐Concha, O., Piccardi, M., Garcı́a, J., et al. (2007) · Lecture notes in computer science · doi:10.1007/978-3-540-73055-2_21 · Cited by 8 · see citing works
On Improving the Generalization of SVM Classifier
Seetha, H., Narasimha Murty, M., & Saravanan, R. D. (2011) · Communications in computer and information science · doi:10.1007/978-3-642-22786-8_2 · Cited by 7 · see citing works
Improving the Naïve Bayes Classifier
Fan, L., & Poh, K. L. (2009) · IGI Global eBooks · doi:10.4018/978-1-59904-849-9.ch130 · Cited by 6 · see citing works
Analyzing Parameter Sensitivity and Classifier Representations for Real-Valued XCS
Wada, A., Takadama, K., Shimohara, K., et al. (2007) · Springer eBooks · doi:10.1007/978-3-540-71231-2_1 · Cited by 5 · see citing works
Flagged first, then by citations
Research works printed but not linked: 3
- [13]Smetanin YG (1995) Neural networks as systems for pattern recognition: a review. Pattern Recognition and Image Analysis 5: 254– 293.
- [25]Wu X, Kumar V, Quinlan JR, Ghosh J (2007) Top 10 algorithms in data mining. Springer-Verlag.
- [53]Ian H, Frank E (2005) Data mining: pratical machine learning tools and techiniques. Morgan Kaufmann.
Cited by
Sorted by citations
Using machine learning approaches for multi-omics data analysis: A review
Reel, P. S., Reel, S., Pearson, E., Trucco, E., & Jefferson, E. (2021) · Biotechnology Advances · doi:10.1016/j.biotechadv.2021.107739 · Cited by 903 · see citing works
Ensemble of keyword extraction methods and classifiers in text classification
Onan, A., Korukoğlu, S., & Bulut, H. (2016) · Expert Systems with Applications · doi:10.1016/j.eswa.2016.03.045 · Cited by 630 · see citing works
On Utility and Privacy in Synthetic Genomic Data
Oprisanu, B., Ganev, G., & De Cristofaro, E. (2022) · Proceedings 2022 Network and Distributed System Security Symposium · doi:10.14722/ndss.2022.24092 · Cited by 23 · see citing works
- Authors
- Amancio, D. R., Comin, C. H., Casanova, D., Travieso, G., Bruno, O. M., Rodrigues, F. A., & da Fontoura Costa, L.
- Affiliations
- Universidade de São Paulo (BR)
- Corresponding author
- Amancio, Diego Raphael
- ORCID
- Venue
- PLoS ONE
- Publisher
- Public Library of Science (PLoS)
- Published
- 2014-04-24
- Volume
- 9
- Issue
- 4
- Pages
- e94137
- ISSN
- 1932-6203 (ISSN-L)
- PMC
- PMC3998948
- OpenAlex
- W1997113291
- Cited by
- 225
- References
- 66
- Citations by year
- 2026: 6 · 2025: 8 · 2024: 9 · 2023: 14 · 2022: 26 · 2021: 31 · 2020: 29 · 2019: 22 · 2018: 29 · 2017: 15 · 2016: 18 · 2015: 16 · 2014: 2
- FWCI
- 16.93
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Locations
- PLoS ONE (publishedVersion, cc-by)
- Europe PMC (PubMed Central) (publishedVersion, other-oa)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- Scientific Electronic Library Online (São Paulo Research Foundation, Latin American and Caribbean Center on Health Sciences Information, Conselho Nacional de Desenvolvimento Científico e Tecnológico) (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- RePEc: Research Papers in Economics (submittedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- arXiv (Cornell University)
- Greater South Information System
- Funders
- Fundação de Amparo à Pesquisa do Estado de São Paulo (2014/, 2011/22639-8, 11/50761-2, 2010/00927-9)
- Conselho Nacional de Desenvolvimento Científico e Tecnológico (2011/50761-2, 573583/2008-0, 305940/2010-4, 2010-3)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Machine Learning and Data Classification
- Physical Sciences › Computer Science › Information Systems › Data Mining and Machine Learning Applications
- Physical Sciences › Computer Science › Artificial Intelligence › Text and Document Classification Technologies
- Keywords
- supervised classification, classifier comparison, WEKA, parameter setting, support vector machine, k-nearest neighbors, classification accuracy, pattern recognition
- MeSH terms
- Data Interpretation, Statistical, ROC Curve, Support Vector Machine
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
