The DOI matches its Crossref record; 35 of the 35 references it cites were found in a registry or an index.
Predicting sample size required for classification performance
Open accessAbstract
Background: Supervised learning methods need annotated data in order to generate efficient models. Annotated data, however, is a relatively scarce resource and can be expensive to obtain. For both passive and active learning methods, there is a need to estimate the size of the annotated sample required to reach a performance target. Methods: We designed and implemented a method that fits an inverse power law model to points of a given learning curve created using a small annotated training set. Fitting is carried out using nonlinear weighted least squares optimization. The fitted model is then used to predict the classifier's performance and confidence interval for larger sample sizes. For evaluation, the nonlinear weighted curve fitting method was applied to a set of learning curves generated using clinical text and waveform classification tasks with active and passive sampling methods, and predictions were validated using standard goodness of fit measures. As control we used an un-weighted fitting method. Results: A total of 568 models were fitted and the model predictions were compared with the observed performances. Depending on the data set and sampling method, it took between 80 to 560 annotated samples to achieve mean average and root mean squared error below 0.01. Results also show that our weighted fitting method outperformed the baseline un-weighted method (p < 0.05). Conclusions: This paper describes a simple and effective sample size prediction algorithm that conducts weighted fitting of learning curves. The algorithm outperformed an un-weighted algorithm described in previous literature. It can help researchers determine annotation sample size for supervised machine learning.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Text classification from labeled and unlabeled documents using EM
Nigam, K., Mccallum, A. K., Thrun, S., & Mitchell, T. (2000) · Machine Learning · doi:10.1023/a:1007692713085 · Cited by 2,736
Knowledge-Based Intelligent Information and Engineering Systems
高下, 大., タカシタ, タ., Takashita, T., Jain, L. C., イトカワ, ツ., Itokawa, T., 北須賀, 輝., キタスカ, テ., Kitasuka, T., 有次, 正., et al. (2008) · Lecture notes in computer science · doi:10.1007/978-3-540-85565-1 · Cited by 1,699
THE LEARNING CURVE: HISTORICAL REVIEW AND COMPREHENSIVE SURVEY
Yelle, L. E. (1979) · Decision Sciences · doi:10.1111/j.1540-5915.1979.tb00026.x · Cited by 1,541
Some Practical Guidelines for Effective Sample Size Determination
Lenth, R. V. (2001) · The American Statistician · doi:10.1198/000313001317098149 · Cited by 1,139
Sample Size Estimation: How Many Individuals Should Be Studied?
Eng, J. (2003) · Radiology · doi:10.1148/radiol.2272012051 · Cited by 811
Sample Size Planning for Statistical Power and Accuracy in Parameter Estimation
Maxwell, S. E., Kelley, K., & Rausch, J. R. (2008) · Annual Review of Psychology · doi:10.1146/annurev.psych.59.103006.093735 · Cited by 542
Health Technology Assessment (HTA)
Greiner, W. (2008) · Gesundheitsökonomische Evaluationen · doi:10.1007/978-3-540-49559-8_19 · Cited by 462
Incorporating diversity in active learning with support vector machines
Brinker, K. (2003) · Cited by 458
Algorithm 573: NL2SOL—An Adaptive Nonlinear Least-Squares Algorithm [E4]
Dennis, J. E., Gay, D. M., & Welsch, R. E. (1981) · ACM Transactions on Mathematical Software · doi:10.1145/355958.355966 · Cited by 405
Efficient progressive sampling
Provost, F., Jensen, D., & Oates, T. (1999) · Proceedings of the fifth ACM SIGKDD international conference on Knowledge discovery and data mining · doi:10.1145/312129.312188 · Cited by 357
Cited by
Machine learning algorithm validation with a limited sample size
Vabalas, A., Gowen, E., Poliakoff, E., & Casson, A. J. (2019) · PLOS ONE · doi:10.1371/journal.pone.0224365 · Cited by 1,901
Supervised Machine Learning: A Brief Primer
Jiang, T., Gradus, J. L., & Rosellini, A. J. (2020) · Behavior Therapy · doi:10.1016/j.beth.2020.05.002 · Cited by 847
Current Applications and Future Impact of Machine Learning in Radiology
Choy, G., Khalilzadeh, O., Michalski, M., Do, S., Samir, A. E., Pianykh, O. S., Geis, J. R., Pandharipande, P. V., Brink, J. A., & Dreyer, K. J. (2018) · Radiology · doi:10.1148/radiol.2018171820 · Cited by 800
Intraoperative Tissue Identification Using Rapid Evaporative Ionization Mass Spectrometry
Balog, J., Sasi-Szabó, L., Kinross, J., Lewis, M. R., Muirhead, L. J., Veselkov, K., Mirnezami, R., Dezső, B., Damjanovich, L., Darzi, A., et al. (2013) · Science Translational Medicine · doi:10.1126/scitranslmed.3005623 · Cited by 608
Alwosheel, A., van Cranenburgh, S., & Chorus, C. G. (2018) · Journal of Choice Modelling · doi:10.1016/j.jocm.2018.07.002 · Cited by 524
Metadata
- Authors
- Figueroa, R. L., Zeng-Treitler, Q., Kandula, S., & Ngo, L. H.
- Venue
- BMC Medical Informatics and Decision Making
- Published
- 2012-02-15
- Volume
- 12
- Issue
- 1
- Pages
- 8-8
- ISSN
- 1472-6947 (ISSN-L)
- OpenAlex
- W2132886902
- Cited by
- 520
- References
- 35
- FWCI
- 6.97
- Type
- journal article
- Language
- en
- Open access
- gold
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Predicting sample size required for classification performance. https://app.cochat.ai/papers/10.1186/1472-6947-12-8
- The paper
- Figueroa, R. L., Zeng-Treitler, Q., Kandula, S., & Ngo, L. H. (2012). Predicting sample size required for classification performance. BMC Medical Informatics and Decision Making. https://doi.org/10.1186/1472-6947-12-8
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.