The DOI matches its Crossref record; 38 of the 38 references it cites were found in a registry or an index.
Effect of Dataset Size and Train/Test Split Ratios in QSAR/QSPR Multiclass Classification
Open accessAbstract
Applied datasets can vary from a few hundred to thousands of samples in typical quantitative structure-activity/property (QSAR/QSPR) relationships and classification. However, the size of the datasets and the train/test split ratios can greatly affect the outcome of the models, and thus the classification performance itself. We compared several combinations of dataset sizes and split ratios with five different machine learning algorithms to find the differences or similarities and to select the best parameter settings in nonbinary (multiclass) classification. It is also known that the models are ranked differently according to the performance merit(s) used. Here, 25 performance parameters were calculated for each model, then factorial ANOVA was applied to compare the results. The results clearly show the differences not just between the applied machine learning algorithms but also between the dataset sizes and to a lesser extent the train/test split ratios. The XGBoost algorithm could outperform the others, even in multiclass modeling. The performance parameters reacted differently to the change of the sample set size; some of them were much more sensitive to this factor than the others. Moreover, significant differences could be detected between train/test split ratios as well, exerting a great effect on the test validation of our models.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
XGBoost: A Scalable Tree Boosting System
Chen, T., & Guestrin, C. (2016) · Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining · doi:10.1145/2939672.2939785 · Cited by 54,569
SMOTE synthetic minority over-sampling technique
Chawla, N. V., Bowyer, K. W., Hall, L. O., & Kegelmeyer, W. P. (2002) · Journal of Artificial Intelligence Research · doi:10.1613/jair.953 · Cited by 32,865
A systematic analysis of performance measures for classification tasks
Sokolova, M., & Lapalme, G. (2009) · Information Processing & Management · doi:10.1016/j.ipm.2009.03.002 · Cited by 7,515
Extreme Learning Machine for Regression and Multiclass Classification
Huang, G.-B., Zhou, H., Ding, X., & Zhang, R. (2012) · IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics) · doi:10.1109/tsmcb.2011.2168604 · Cited by 5,550
Statistical Modeling: The Two Cultures (with comments and a rejoinder by the author)
Breiman, L. (2001) · Statistical Science · doi:10.1214/ss/1009213726 · Cited by 4,340
A direct adaptive method for faster backpropagation learning: the RPROP algorithm
Riedmiller, M., & Braun, H. (2002) · IEEE International Conference on Neural Networks · doi:10.1109/icnn.1993.298623 · Cited by 3,917
Specht, D. F. (1990) · Neural Networks · doi:10.1016/0893-6080(90)90049-q · Cited by 3,762
A systematic study of the class imbalance problem in convolutional neural networks
Buda, M., Maki, A., & Mazurowski, M. A. (2018) · Neural Networks · doi:10.1016/j.neunet.2018.07.011 · Cited by 3,088 · cited 1×
“While oversampling can lead to overfitting in “classical” machine learning models, convolutional neural networks (CNNs) are less prone to overfitting [7].”
BackgroundWhy is Tanimoto index an appropriate choice for fingerprint-based similarity calculations?
Bajusz, D., Rácz, A., & Héberger, K. (2015) · Journal of Cheminformatics · doi:10.1186/s13321-015-0069-3 · Cited by 1,621
Self-citationLuque, A., Carrasco, A., Martín, A., & de las Heras, A. (2019) · Pattern Recognition · doi:10.1016/j.patcog.2019.02.023 · Cited by 1,128
Support Vector Machines for classification and regression
Brereton, R. G., & Lloyd, G. R. (2010) · The Analyst · doi:10.1039/b918972f · Cited by 1,036
Clustering-based undersampling in class-imbalanced data
Lin, W.-C., Tsai, C.-F., Hu, Y.-H., & Jhang, J.-S. (2017) · Information Sciences · doi:10.1016/j.ins.2017.05.008 · Cited by 832
Specht, D. F. (1990) · IEEE Transactions on Neural Networks · doi:10.1109/72.80210 · Cited by 576
Classifier ensembles: Select real-world applications
Oza, N. C., & Tumer, K. (2008) · Information Fusion · doi:10.1016/j.inffus.2007.07.002 · Cited by 313
Sum of ranking differences compares methods or models fairly
Héberger, K. (2010) · TrAC Trends in Analytical Chemistry · doi:10.1016/j.trac.2009.09.009 · Cited by 301
Self-citationClass-imbalanced classifiers for high-dimensional data
Lin, W.-J., & Chen, J. J. (2013) · Briefings in Bioinformatics · doi:10.1093/bib/bbs006 · Cited by 284
Watanabe, R., Esaki, T., Kawashima, H., Natsume-Kitatani, Y., Nagao, C., Ohashi, R., & Mizuguchi, K. (2018) · Molecular Pharmaceutics · doi:10.1021/acs.molpharmaceut.8b00785 · Cited by 194
Multi-Level Comparison of Machine Learning Classifiers and Their Performance Metrics
Rácz, A., Bajusz, D., & Héberger, K. (2019) · Molecules · doi:10.3390/molecules24152811 · Cited by 170
Self-citationQSAR Modeling of Imbalanced High-Throughput Screening Data in PubChem
Zakharov, A. V., Peach, M. L., Sitzmann, M., & Nicklaus, M. C. (2014) · Journal of Chemical Information and Modeling · doi:10.1021/ci400737s · Cited by 140
Puga, J. L., Krzywinski, M. I., & Altman, N. (2015) · Nature Methods · doi:10.1038/nmeth.3335 · Cited by 111
Orozco-Arias, S., Piña, J. S., Tabares-Soto, R., Castillo, L., Guyot, R., & Isaza, G. A. (2020) · Processes · doi:10.3390/pr8060638 · Cited by 104
Generic performance measure for multiclass-classifiers
Kautz, T., Eskofier, B. M., & Pasluosta, C. F. (2017) · Pattern Recognition · doi:10.1016/j.patcog.2017.03.008 · Cited by 95
Decision threshold adjustment in class prediction
Chen, J. J., Tsai, C., Moon, H., Ahn, H., Young, J., & Chen, C. (2006) · SAR and QSAR in environmental research · doi:10.1080/10659360600787700 · Cited by 89
Ensemble of Classifiers Based on Multiobjective Genetic Sampling for Imbalanced Data
Fernandes, E. R. Q., de Carvalho, A. C. P. L. F., & Yao, X. (2019) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2019.2898861 · Cited by 83
Intercorrelation Limits in Molecular Descriptor Preselection for QSAR/QSPR
Rácz, A., Bajusz, D., & Héberger, K. (2019) · Molecular Informatics · doi:10.1002/minf.201800154 · Cited by 65
Self-citationClassification ensembles for unbalanced class sizes in predictive toxicology
Chen, J. J., Tsai, C., Young, J. F., & Kodell, R. L. (2005) · SAR and QSAR in environmental research · doi:10.1080/10659360500468468 · Cited by 62
Binary classification of imbalanced datasets using conformal prediction
Norinder, U., & Boyer, S. K. (2017) · Journal of Molecular Graphics and Modelling · doi:10.1016/j.jmgm.2017.01.008 · Cited by 59
Idakwo, G., Thangapandian, S., Luttrell, J., Zhou, Z., Zhang, C., & Gong, P. (2019) · Frontiers in Physiology · doi:10.3389/fphys.2019.01044 · Cited by 55
Škuta, C., Cortés‐Ciriano, I., Dehaen, W., Kříž, P., van Westen, G. J. P., Tetko, I. V., Bender, A., & Svozil, D. (2020) · Journal of Cheminformatics · doi:10.1186/s13321-020-00443-6 · Cited by 50
Enhancing techniques for learning decision trees from imbalanced data
Chaabane, I., Guermazi, R., & Hammami, M. (2019) · Advances in Data Analysis and Classification · doi:10.1007/s11634-019-00354-x · Cited by 44
Binary and multi-class classification for androgen receptor agonists, antagonists and binders
Piir, G., Sild, S., & Maran, U. (2020) · Chemosphere · doi:10.1016/j.chemosphere.2020.128313 · Cited by 38
Thresholding Strategies for Deep Learning with Highly Imbalanced Big Data
Johnson, J. M., & Khoshgoftaar, T. M. (2020) · Advances in intelligent systems and computing · doi:10.1007/978-981-15-6759-9_9 · Cited by 27
AECID: Asymmetric entropy for classifying imbalanced data
Guermazi, R., Chaabane, I., & Hammami, M. (2018) · Information Sciences · doi:10.1016/j.ins.2018.07.076 · Cited by 26
Predicting molecular activity on nuclear receptors by multitask neural networks
Valsecchi, C., Collarile, M., Grisoni, F., Todeschini, R., Ballabio, D., & Consonni, V. (2020) · Journal of Chemometrics · doi:10.1002/cem.3325 · Cited by 26
Žuvela, P., Lovrić, M., Yousefian-Jazi, A., & Liu, J. J. (2020) · Industrial & Engineering Chemistry Research · doi:10.1021/acs.iecr.9b05766 · Cited by 25
A Comparative Study of Naive Bayes and k-NN Algorithm for Multi-class Drug Molecule Classification
Mandal, L., & Jana, N. D. (2019) · IEEE India Conference · doi:10.1109/indicon47234.2019.9029095 · Cited by 14
König, C., Gini, G. C., Crăciun, M., & Benfenati, E. (2004) · International Journal of Pattern Recognition and Artificial Intelligence · doi:10.1142/s0218001404003423 · Cited by 13
Wu, Q., Lin, Y.-P., Zhu, T.-F., & Zhang, Y. (2020) · Journal of Intelligent & Fuzzy Systems · doi:10.3233/jifs-190821 · Cited by 9
Cited by
COVID-19 cough classification using machine learning and global smartphone recordings
Pahar, M., Klopper, M., Warren, R., & Niesler, T. (2021) · Computers in Biology and Medicine · doi:10.1016/j.compbiomed.2021.104572 · Cited by 290
Metadata
- Authors
- Rácz, A., Bajusz, D., & Héberger, K.
- Venue
- Molecules
- Publisher
- MDPI AG
- Published
- 2021-02-19
- Volume
- 26
- Issue
- 4
- Pages
- 1111
- ISSN
- 1420-3049 (ISSN-L)
- OpenAlex
- W3131057326
- Cited by
- 307
- References
- 38
- FWCI
- 24.37
- Type
- journal article
- Language
- en
- Open access
- gold
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.