The DOI matches its Crossref record; its reference list is not available.
Impact of Dataset Size on Classification Performance: An Empirical Evaluation in the Medical Domain
Open accessAbstract
Dataset size is considered a major concern in the medical domain, where lack of data is a common occurrence. This study aims to investigate the impact of dataset size on the overall performance of supervised classification models. We examined the performance of six widely-used models in the medical field, including support vector machine (SVM), neural networks (NN), C4.5 decision tree (DT), random forest (RF), adaboost (AB), and naïve Bayes (NB) on eighteen small medical UCI datasets. We further implemented three dataset size reduction scenarios on two large datasets and analyze the performance of the models when trained on each resulting dataset with respect to accuracy, precision, recall, f-score, specificity, and area under the ROC curve (AUC). Our results indicated that the overall performance of classifiers depend on how much a dataset represents the original distribution rather than its size. Moreover, we found that the most robust model for limited medical data is AB and NB, followed by SVM, and then RF and NN, while the least robust model is DT. Furthermore, an interesting observation is that a robust machine learning model to limited dataset does not necessary imply that it provides the best performance compared to other models.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The most prominent data quality dimension in the DL papers is amount of data, which is empirically shown to benefit performance, albeit in a saturating manner.
“The most prominent dimension is amount of data123–126,129,146–159,183–189 which is empirically shown to benefit performance, albeit in a saturating manner.”
From The METRIC-framework for assessing data quality for trustworthy AI in medicine: A systematic review (2024) · cites this paper as [124]
Section 4.2 (Large Datasets; page not specified) reports that neural-network performance degraded in 42% of dataset-size reduction scenarios and explains why more data can improve performance. The paper does not establish a saturating benefit, and its experiments concern medical classification models rather than DL papers generally. The claim that amount of data is the most prominent dimension is the citing authors’ framing, not a finding attributed to this source. (LLM, conf=0.85)
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
No citation lists for this paper
No citation lists for this paper yet — checked again daily.
References 29 · Cited by 410
Research works printed but not linked: 21
- Rahman, M.S., and Sultana, M. (2017). Performance of Firth-and logF-type penalized methods in risk prediction for small or sparse binary data. BMC Med. Res. Methodol., 17. · DOI
- Marcoulides, 2005, Discovering Knowledge in Data: An Introduction to Data Mining, Daniel T. Larose, J. Am. Stat. Assoc., 100, 1465, 10.1198/jasa.2005.s61 · DOI
- Bartling, S., and Friesike, S. (2014). Challenges of Open Data in Medical Research. Opening Science, Springer. · DOI
- Anthony, 1993, Bounding sample size with the Vapnik-Chervonenkis dimension, Discret. Appl. Math., 42, 65, 10.1016/0166-218X(93)90179-R · DOI
- Andonie, 2010, Extreme data mining: Inference from small datasets, Int. J. Comput. Commun. Control, 5, 280, 10.15837/ijccc.2010.3.2481 · DOI
- Andonie, 2006, Fuzzy artmap with input relevances, IEEE Trans. Neural Netw., 17, 929, 10.1109/TNN.2006.875988 · DOI
- Chen, 2017, A PSO based virtual sample generation method for small sample sets: Applications to regression datasets, Eng. Appl. Artif. Intell., 59, 236, 10.1016/j.engappai.2016.12.024 · DOI
- Li, 2017, The attribute-trend similarity method to improve learning performance for small datasets, Int. J. Prod. Res., 55, 1898, 10.1080/00207543.2016.1213447 · DOI
- Yang, 2011, A novel virtual sample generation method based on gaussian distribution, Knowl. Based Syst., 24, 740, 10.1016/j.knosys.2010.12.010 · DOI
- Li, 2012, Extending attribute information for small data set classification, IEEE Trans. Knowl. Data Eng., 24, 452, 10.1109/TKDE.2010.254 · DOI
- Patil, 2017, Dataset Classification by Extending Attribute Information for Improving Classification Accuracy, Int. J. Innov. Trends Eng. Res., 2, 1
- Lin, 2018, An attribute extending method to improve learning performance for small datasets, Neurocomput, 286, 75, 10.1016/j.neucom.2018.01.071 · DOI
- Coqueret, 2017, Approximate NORTA simulations for virtual sample generation, Expert Syst. Appl., 73, 69, 10.1016/j.eswa.2016.12.027 · DOI
- Choi, 2017, Data properties and the performance of sentiment classification for electronic commerce applications, Inf. Syst. Front., 19, 993, 10.1007/s10796-017-9741-7 · DOI
- Zhu, 2016, Do we need more training data?, Int. J. Comput. Vis., 119, 76, 10.1007/s11263-015-0812-2 · DOI
- Barbedo, 2018, Impact of dataset size and variety on the effectiveness of deep learning and transfer learning for plant disease classification, Comput. Electron. Agric., 153, 46, 10.1016/j.compag.2018.08.013 · DOI · Paper page
- Kusonmano, K., Netzer, M., Pfeifer, B., Baumgartner, C., Liedl, K.R., and Graber, A. (2009, January 26). Evaluation of the impact of dataset characteristics for classification problems in biological applications. Proceedings of the International Conference on Bioinformatics and Biomedicine, Venice, Italy.
- Ruparel, 2013, Learning from Small Data Set to Build Classification Model: A Survey, Proc. IJCA Int. Conf. Recent Trends Eng. Technol., 4, 23
- Zhang, 2000, Neural networks for classification: A survey, IEEE Trans. Syst. Man Cybern. Part C, 30, 451, 10.1109/5326.897072 · DOI
- Zhang, 2017, Empirical study of seven data mining algorithms on different characteristics of datasets for biomedical classification applications, BioMed. Eng. OnLine, 16, 125, 10.1186/s12938-017-0416-x · DOI
- Eibe, F., Hall, M., Witten, I., and Pal, J. (2016). The weka workbench. Online Appendix for Data Mining: Practical Machine Learning Tools and Techniques, Morgan Kaufmann Publishers Inc.
Metadata
- Authors
- Althnian, A., AlSaeed, D., Al-Baity, H., Samha, A., Dris, A. B., Alzakari, N., Abou Elwafa, A., & Kurdi, H.
- Affiliations
- King Saud University (SA)
- King Abdulaziz City for Science and Technology (SA)
- Computer Science Department, College of Computer and Information Sciences, King Saud University, Riyadh 11451, Saudi Arabia
- Corresponding author
- Kurdi, Heba
- ORCID
- Venue
- Applied Sciences
- Publisher
- MDPI AG
- Published
- 2021-01-15
- Volume
- 11
- Issue
- 2
- Pages
- 796
- ISSN
- 2076-3417 (ISSN-L)
- OpenAlex
- W3119036516
- Cited by
- 410
- References
- 29
- Citations by year
- 2026: 62 · 2025: 104 · 2024: 110 · 2023: 66 · 2022: 52 · 2021: 16
- FWCI
- 53.59
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- Locations
- Applied Sciences (publishedVersion, cc-by)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- DSpace@MIT (Massachusetts Institute of Technology) (submittedVersion, cc-by-nc)
- DSpace@MIT (Massachusetts Institute of Technology) (submittedVersion, cc-by)
- Funders
- Researchers Supporting Unit at King Saud University (RSP- 2020/204)
- Topics
- Health Sciences › Health Professions › Health Information Management › Artificial Intelligence in Healthcare
- Physical Sciences › Computer Science › Artificial Intelligence › Imbalanced Data Classification Techniques
- Physical Sciences › Computer Science › Artificial Intelligence › Machine Learning and Data Classification
- Keywords
- dataset size, small medical datasets, supervised classification, classifier robustness, data reduction, support vector machine, random forest, AdaBoost, naive Bayes, decision tree, artificial neural networks, model comparison
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
