The DOI matches its Crossref record; 78 of the 80 references it cites were found in a registry or an index.
Calculating the sample size required for developing a clinical prediction model
Open accessAbstract
Clinical prediction models aim to predict outcomes in individuals, to inform diagnosis or prognosis in healthcare. Hundreds of prediction models are published in the medical literature each year, yet many are developed using a dataset that is too small for the total number of participants or outcome events. This leads to inaccurate predictions and consequently incorrect healthcare decisions for some individuals. In this article, the authors provide guidance on how to calculate the sample size required to develop a clinical prediction model.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“In step three, the sample size needed to reduce model overfitting is calculated, assuming an expected shrinkage rate of 10%, with an expected number of variables P to be included in the final predictive model of 30: S is the expected shrinkage factor, and the noise ratio evaluation index R2cs is 0.3; the calculated sample size is 740 cases.”
From Machine learning-driven prediction model for successful weaning of patients from mechanical ventilation in ICU (2026) · cites this paper as Riley [13]
Experimental — accuracy not yet measuredThe check wasn’t confident either way, so it gives no verdict — never disputed.
In the citing paper
“Genomic and molecular data support integration of tumour biology and genetic risk into decision-making, particularly in precision medicine applications [42].”
From Artificial Intelligence in Clinical Decision-Making: A Comprehensive Review of Diagnostic, Prognostic, and Therapeutic Applications, Validation Gaps, Bias, and Deployment Challenges (2026) · cites this paper as [42]
Experimental — accuracy not yet measuredIn the citing paper
“The sample size calculated this way is the minimum for a standard statistical model.”
From Developing clinical prediction models: a step-by-step guide (2024) · cites this paper as [50]
In the citing paper
“Sample size calculations50 indicated a minimum sample of 2082 patients, which is larger than the available sample, raising concerns about possible overfitting issues (step 6).”
From Developing clinical prediction models: a step-by-step guide (2024) · cites this paper as [50]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Regression shrinkage and selection via the lasso
Tibshirani, R. (1996) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/j.2517-6161.1996.tb02080.x · Cited by 51,482
The Elements of Statistical Learning
Hastie, T., Tibshirani, R., & Friedman, J. (2009) · Springer Series in Statistics · doi:10.1007/978-0-387-84858-7 · Cited by 25,970
Regularization and Variable Selection Via the Elastic Net
Zou, H., & Hastie, T. (2005) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/j.1467-9868.2005.00503.x · Cited by 20,957
A simulation study of the number of events per variable in logistic regression analysis
Peduzzi, P., Concato, J., Kemper, E., Holford, T. R., & Feinstein, A. R. (1996) · Journal of Clinical Epidemiology · doi:10.1016/s0895-4356(96)00236-3 · Cited by 9,655
Harrell (2015) · Springer Series in Statistics · doi:10.1007/978-3-319-19425-7 · Cited by 6,878
A note on a general definition of the coefficient of determination
NAGELKERKE, N. J. D. (1991) · Biometrika · doi:10.1093/biomet/78.3.691 · Cited by 5,927
Moons, K. G. M., Altman, D. G., Reitsma, J. B., Ioannidis, J. P. A., Macaskill, P., Steyerberg, E. W., Vickers, A. J., Ransohoff, D. F., & Collins, G. S. (2015) · Annals of Internal Medicine · doi:10.7326/m14-0698 · Cited by 5,147
Self-citationRelaxing the Rule of Ten Events per Variable in Logistic and Cox Regression
Vittinghoff, E., & McCulloch, C. E. (2007) · American Journal of Epidemiology · doi:10.1093/aje/kwk052 · Cited by 4,359
Clinical Prediction Models: A Practical Approach to Development, Validation, and Updating
Steyerberg, E. W. (2019) · Statistics for Biology and Health · doi:10.1007/978-3-030-16399-0 · Cited by 3,281
PROBAST: A Tool to Assess the Risk of Bias and Applicability of Prediction Model Studies
Wolff, R. F., Moons, K. G. M., Riley, R. D., Whiting, P. F., Westwood, M., Collins, G. S., Reitsma, J. B., Kleijnen, J., Mallett, S., & for the PROBAST Group† (2019) · Annals of Internal Medicine · doi:10.7326/m18-1376 · Cited by 2,984
Self-citation
Flagged first, then by citations
2 of the 80 references couldn’t be matched to a known paper.
Research works printed but not linked: 2
- 10.1177/962280218784726 · DOI
- McFadden, 1974, Frontiers in Econometrics New York., 104
Cited by
Sorted by citations
Items 501–600 of 1,003
Capobianco, I., Oldhafer, K. J., Fard-Aghaie, M. H., Robles-Campos, R., Brusadín, R., Petrowsky, H., Linecker, M., Mehrabi, A., Hoffmann, K., Li, J., et al. (2022) · HepatoBiliary Surgery and Nutrition · doi:10.21037/hbsn-21-396 · Cited by 12
“14, and a 95 confidence interval confirm that the power of the model was sufficient, as the required sample size should be at least 230 patients (33).”
BackgroundMethodWen, S., Du, X., Chen, Y., Xia, J., Wang, R., Zhu, M., Peng, W., Spitaleri, G., Hofman, P. M., Bironzo, P., et al. (2022) · Translational Lung Cancer Research · doi:10.21037/tlcr-22-300 · Cited by 12
Li, J., Wu, X., Wu, Y., Fong, D., Song, Y., Xu, S., Kim, C., Lin, X., & Pandian, V. (2025) · Journal of Medical Internet Research · doi:10.2196/70822 · Cited by 11
“The sample size was determined using the rule of thumb of at least 10 events per variable as recommended for valid statistical inferences [22], requiring a minimum of 90 subjects for each linear regression analysis with up to 9 variables.”
Yang, X., Dou, F., Tang, G., Xiu, R., & Zhao, X. (2025) · BMC Cancer · doi:10.1186/s12885-025-14387-3 · Cited by 11
“To estimate the sample size, the pmsampsize package (version 1.1.3) was used [17].”
InfluentialAnglim, B., TOMLINSON, G. A., Paquette, J., & McDermott, C. D. (2022) · BJOG An International Journal of Obstetrics & Gynaecology · doi:10.1111/1471-0528.17225 · Cited by 11
Ribbons, K., Johnson, S. J., Ditton, E., Wills, A., Mason, G., Flynn, T. S., Cochrane, J. A., Pollack, M. R. P., Walker, F. R., & Nilsson, M. (2023) · JMIR Research Protocols · doi:10.2196/48801 · Cited by 11
“Recently, Riley et al [34] have developed a more refined calculation of the required sample size, which depends not only on the number of candidate predictor parameters but also on the total number of participants, the outcome proportion (incidence) in the study population, and the expected…”
InfluentialNakafero, G., Card, T. R., Grainge, M. J., Williams, H. C., Taal, M. W., Aithal, G. P., Fox, C. P., Mallen, C. D., van der Windt, D., Stevenson, M. D., et al. (2023) · EClinicalMedicine · doi:10.1016/j.eclinm.2023.102213 · Cited by 11
Luchinin, A. S. (2023) · Clinical oncohematology · doi:10.21320/2500-2139-2023-16-1-27-36 · Cited by 20
Liu, D., Song, D., Ning, W., Guo, Y., Lei, T., Qu, Y., Zhang, M., Gu, C., Wang, H., Ji, J., et al. (2023) · Cancers · doi:10.3390/cancers15225483 · Cited by 13
“The 10 EPV is a widely applied principle, suggesting that the number of positive events should be at least 10 times the number of predictor variables, currently a classical rule for sample sizes in clinical prediction models [25–27].”
BackgroundTeza, H., Pattanateepapon, A., Lertpimonchai, A., Vathesatogkit, P., McKay, G. J., Attia, J., & Thakkinstian, A. (2023) · JMIR Formative Research · doi:10.2196/48351 · Cited by 11
“To the best of our knowledge, there are no explicit guidelines for sample size estimation for machine learning models, but previous studies have recommended that this be based on disease prevalence estimates [24].”
BackgroundSwarbrick, C. J., Poulton, T., Martin, P., Partridge, J. S. L., & Moppett, I. K. (2023) · BMJ Open · doi:10.1136/bmjopen-2023-076803 · Cited by 11
Predicting outcomes following lower extremity open revascularization using machine learning
Li, B., Verma, R., Beaton, D., Tamim, H., Hussain, M. A., Hoballah, J. J., Lee, D. S., Wijeysundera, D. N., de Mestral, C., Mamdani, M., et al. (2024) · Scientific Reports · doi:10.1038/s41598-024-52944-1 · Cited by 11
“Based on a validated sample size calculator for clinical prediction models, to achieve a minimum AUROC of 0.7 with an outcome rate of ~ 10% and 37 input features, the minimum sample size required is 6960 patients with 696 events 45,46 .”
InfluentialGuo, J., Xu, X., Qian, G., Wang, T., Xu, K., He, J., Long, Y., Zhang, Q., Jing, W., Li, Z., et al. (2024) · International Journal of Surgery · doi:10.1097/js9.0000000000001143 · Cited by 12
“For this study, the sample size should be a minimum of 310, which is 10 times the number of variables (31) in the current study [33] .”
BackgroundXia, P., Jiang, Y., Cai, F., Peng, S., & Xu, Z. (2024) · Frontiers in Public Health · doi:10.3389/fpubh.2024.1380218 · Cited by 11
Breath metabolomics for diagnosis of acute respiratory distress syndrome
Zhang, S., Hagens, L. A., Heijnen, N. F. L., Smit, M. R., Brinkman, P., Fenn, D. W., van der Poll, T., Schultz, M. J., Bergmans, D. C. J. J., Schnabel, R. M., et al. (2024) · Critical Care · doi:10.1186/s13054-024-04882-7 · Cited by 11
“To ensure sufficient statistical power of this analysis, the sample size of the patients with certain diagnosis in the derivation cohort was calculated retrospectively using the pmsampsize package in Rstudio (version 4.0.3) [20, 21].”
MethodInfluentialMeng, Z. W., Ruan, Y., Fisher, S., Bishay, K., Chau, M., Howarth, M., Cartwright, S., Chen, Y., Dixon, E., Heitman, S. J., et al. (2024) · DEN Open · doi:10.1002/deo2.355 · Cited by 11
“24 According to the “rule of 10” cases per degree of freedom, 24 we estimated that the data would support an algorithm with a maximum of 15 degrees of freedom.”
BackgroundMethodMoretto, R., Germani, M. M., Borelli, B., Conca, V., Rossini, D., Boraschi, P., Donati, F., Urbani, L., Lonardi, S., Bergamo, F., et al. (2024) · ESMO Open · doi:10.1016/j.esmoop.2024.102991 · Cited by 12
“26 However, more fl exible approaches have been proposed for clinical prediction models, 27 and our sample size (in terms of number of patients and events) can be considered adequate to estimate with enough accuracy the main outcome (the probability of early relapse).”
BackgroundChen, L., Ashton-James, C. E., Shi, B., Radojčić, M. R., Anderson, D., Chen, Y., Preen, D. B., Hopper, J. L., Li, S., Bui, M., et al. (2024) · BMC Medicine · doi:10.1186/s12916-024-03388-x · Cited by 11
“To ensure precise predictions and prevent overfitting, the maximum number of candidate predictor parameters was estimated based on the criteria proposed (details in Supplementary C) by Riley et al. [31].”
MethodLeenaerts, N., Soyster, P. D., Ceccarini, J., Sunaert, S. G., Fisher, A. J., & Vrieze, E. (2024) · Psychological Medicine · doi:10.1017/s0033291724000862 · Cited by 16
Fang, Z. H., Gao, B., Wang, Z., Chen, X., & Liu, M. (2024) · Frontiers in Medicine · doi:10.3389/fmed.2024.1401443 · Cited by 11
Zeng, J., Su, X., Lin, S., Li, Z., Zhao, Y., & Zheng, Z. (2024) · JACC Advances · doi:10.1016/j.jacadv.2024.101326 · Cited by 11
Rothschild, J. A., Hofmeyr, S., McLaren, S. J., & Maunder, E. (2024) · Sports Medicine · doi:10.1007/s40279-024-02131-z · Cited by 13
“However, our sample size is below the minimum size of 240 (using the rule of 234 + number of predictors) needed for precise estimates of the residual standard deviation [41].”
Tian, Z., Wang, D., Sun, X., Cui, C., & Wang, H. (2024) · Frontiers in Physiology · doi:10.3389/fphys.2024.1473659 · Cited by 14
Zhang, J., Cao, Q., Mao, C., Xu, J., Li, Y., Mu, Y., Huang, G., Chen, D., Deng, X., Xu, T., et al. (2025) · BMC Pregnancy and Childbirth · doi:10.1186/s12884-025-07442-1 · Cited by 11
“A crucial consideration is that the effective sample size should be ten times the number of independent variables [22, 23].”
Risk Prediction Models for Perioperative Hypothermia: A Systematic Review
Liu, J., Liu, F., Xu, W., Du, L., Li, Y., Liang, A., Li, B., & Zhang, M. (2025) · Journal of Multidisciplinary Healthcare · doi:10.2147/jmdh.s538891 · Cited by 11
Bedogni, G., Tamini, S., Caroli, D., Cicolini, S., Domenicali, M., & Sartório, A. (2021) · Journal of Clinical Medicine · doi:10.3390/jcm10071470 · Cited by 11
“[25,26], we choose the lowest vale of Cox-Snell R2 that we detected in our experience at developing multivariable models for the prediction of fatty liver.”
MethodRodriguez, G. C., Bodnar, L. M., Brooks, M. M., Wahed, A. S., Kennedy, E. H., Schisterman, E. F., & Naimi, A. I. (2021) · American Journal of Epidemiology · doi:10.1093/aje/kwab220 · Cited by 11
Goto, T., Kudo, D., Uchimido, R., Hayakawa, M., Yamakawa, K., Abe, T., Shiraishi, A., & Kushimoto, S. (2022) · Critical Care · doi:10.1186/s13054-022-04020-1 · Cited by 11
“The sample size for model development (n = 3694, of which 9% had the target phenotype) was enough to ensure precise predictions and minimise overfitting [17].”
BackgroundLi, J., Zhang, Y., Chen, Q., Pan, Z., Chen, J., Sun, M., Wang, J., Li, Y., & Ye, Q. (2022) · Frontiers in Oncology · doi:10.3389/fonc.2022.975563 · Cited by 11
“Age (IQR) [Range] 61 (54, 66) [21-81] 45 (35, 58) [22-90] 63 (58, 69) [33-78] 30 (24, 43)[19-74]”
Li, P., Li, K., Wang, F., Zhang, Z., Cai, S., & Cheng, L. (2022) · Sustainability · doi:10.3390/su142012998 · Cited by 11
“6] 3 rnn unit [4,8,12,16,20,24] 4 output step [1,2,3,4] 5 interval step [0,1,2]”
BackgroundMethodWang, W., & Jin, X. (2023) · Intelligent Pharmacy · doi:10.1016/j.ipha.2023.04.010 · Cited by 11
Fan, G., Fu, S., Zheng, M., Xu, W., Ma, G., Zhang, F., Li, M., Liu, X., & Zhao, W. (2023) · Aging Clinical and Experimental Research · doi:10.1007/s40520-023-02527-3 · Cited by 11
“According to the principle of 10EPV [15], a total of 6 variables were included in the multivariate analysis, with results of 10*6/0.”
MethodSun, Y., Ren, G., Ren, J., Shan, W., Han, X., Lian, Y., Wang, T., & Wang, Q. (2021) · Frontiers in Neurology · doi:10.3389/fneur.2021.612569 · Cited by 11
“According to a number of empirical investigations (18), at least 10 outcome events per variable (EPV) were acquired.”
BackgroundNijman, R., Borensztajn, D., Zachariasse, J. M., Hajema, C., Freitas, P., Greber‐Platzer, S., Smit, F. J., Alves, C. F., van der Lei, J., Steyerberg, E. W., et al. (2021) · PLoS ONE · doi:10.1371/journal.pone.0254366 · Cited by 11
Morris, T. P., & van Smeden, M. (2021) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2021.09.026 · Cited by 12
“Though there have been attempts[11], it may in some cases be impossible to come up with sensible and usable formulas, as in the context of clinical prediction models[12].”
Heo, K., Lee, J., & Ah, Y. (2022) · Scientific Reports · doi:10.1038/s41598-022-09095-y · Cited by 11
Wang, Y., Zheng, R., Wu, Y., Liu, T., Hao, L., Liu, J., Shi, L., & Guo, Q. (2025) · International Journal of Nursing Studies · doi:10.1016/j.ijnurstu.2025.105094 · Cited by 13
van Royen, F. S.-A., Joosten, L. P. T., van Smeden, M., Slottje, P., Rutten, F. H., Geersing, G., & van Doorn, S. (2022) · PLoS ONE · doi:10.1371/journal.pone.0266750 · Cited by 10
Prognostic model for atrial fibrillation after cardiac surgery: a UK cohort study
Chung, S., O’Brien, B., Lip, G. Y. H., Fields, K. G., Muehlschlegel, J. D., Thakur, A., Clifton, D. A., Collins, P. G. S., Watkinson, P., & Providencia, R. A. (2022) · Clinical Research in Cardiology · doi:10.1007/s00392-022-02068-1 · Cited by 9
“The minimum sample size for developing the was estimated [28].”
BackgroundSakal, C., Li, J., Xiang, Y., & Li, X. (2022) · Journal of Affective Disorders · doi:10.1016/j.jad.2022.09.034 · Cited by 10
Ezzati, A., Fanning, K. M., Reed, M. L., & Lipton, R. B. (2023) · Headache The Journal of Head and Face Pain · doi:10.1111/head.14459 · Cited by 10
Zhang, R., Liu, Z., Zhang, Y., Pei, Y., He, Y., Yu, J., You, C., Ma, L., & Fang, F. (2023) · Frontiers in Neurology · doi:10.3389/fneur.2023.1078926 · Cited by 10
He, S., Huang, Y., Peng, Y., Chai, J., & Chen, K. (2023) · Lipids in Health and Disease · doi:10.1186/s12944-023-01874-4 · Cited by 9
“effective IVCS The count of variables in the regression equations was controlled to equal the number of adverse events divided by 5 − 10 owing to the constrained sample size [32].”
MethodZhao, Y., Cheng, Y., Wang, H., Du, H., Sun, J., Xu, M., Luo, Y., Liu, S., Guo, X., & Xiong, W. G. (2023) · Pulmonology · doi:10.1016/j.pulmoe.2023.10.002 · Cited by 10
Dynamic Nomogram for Subsyndromal Delirium in Adult Intensive Care Unit: A Prospective Cohort Study
Cheng, J., Lao, Y., Chen, X., Qiao, X., Sui, W., Gong, X., & Zhuang, Y. (2023) · Neuropsychiatric Disease and Treatment · doi:10.2147/ndt.s432776 · Cited by 13
Liang, J., Zhang, J., Lou, Z., & Tang, X. (2023) · Frontiers in Medicine · doi:10.3389/fmed.2023.1263930 · Cited by 10
Xiong, H., Wang, D., Song, P., Quan, X., Zhang, M., Huang, S., Liu, X., Chen, Q., He, X., Hu, X., et al. (2024) · Journal of Vascular Surgery · doi:10.1016/j.jvs.2024.02.036 · Cited by 10
Zhang, L., Jin, S., Wang, Y., Zhang, Z., Jia, H., Li, D., & Lü, Q. (2024) · Radiotherapy and Oncology · doi:10.1016/j.radonc.2024.110339 · Cited by 10
Qin, T., Fan, C., Liu, Q., Wang, J., & Zhu, X. (2024) · Journal of Advanced Nursing · doi:10.1111/jan.16253 · Cited by 10
Zheng, Z., Huang, Y., Zhao, Y., Shi, J., Zhang, S., & Zhao, Y. (2024) · The American Journal of Surgery · doi:10.1016/j.amjsurg.2024.115912 · Cited by 10
Using Machine Learning to Predict Outcomes Following Transfemoral Carotid Artery Stenting
Li, B., Eisenberg, N., Beaton, D., Lee, D. S., Al‐Omran, L., Wijeysundera, D. N., Hussain, M. A., Rotstein, O. D., de Mestral, C., Mamdani, M., et al. (2024) · Journal of the American Heart Association · doi:10.1161/jaha.124.035425 · Cited by 11
“40 Our cohort of 35 214 patients with 3257 primary events satisfied this sample size requirement.”
Li, C., Zhao, K., Ren, Q., Chen, L., Zhang, Y., Wang, G., & Xie, K. (2024) · Frontiers in Cellular and Infection Microbiology · doi:10.3389/fcimb.2024.1488505 · Cited by 11
“The sample size is calculated using the formula n = ( 1 : 96 d ) 2 f (1 − f ) where f represents the expected outcome ratio ( f = 0.46), d is the set margin of error ( d = 0.05) (Riley et al., 2020).”
InfluentialLi, Y., Xi, L., Sun, H., Yu, F., Liang, Q., Qie, T., & Dai, B. (2024) · International Journal of General Medicine · doi:10.2147/ijgm.s489482 · Cited by 10
“Monocytes are recognized as a potential source of MMP-9, exacerbating brain tissue injury, while lymphocytes are believed to exhibit neuroprotective properties.”
InfluentialPredicting lack of clinical improvement following varicose vein ablation using machine learning
Li, B., Eisenberg, N., Beaton, D., Lee, D. S., Al‐Omran, L., Wijeysundera, D. N., Hussain, M. A., Rotstein, O. D., de Mestral, C., Mamdani, M., et al. (2024) · Journal of Vascular Surgery Venous and Lymphatic Disorders · doi:10.1016/j.jvsv.2024.102162 · Cited by 13
Masuda, S., Fukasawa, T., Inokuchi, S., Otsuki, B., Murata, K., Shimizu, T., Sono, T., Honda, S., Shima, K., Sakamoto, M., et al. (2024) · Scientific Reports · doi:10.1038/s41598-024-82359-x · Cited by 10
Pan, S., Shi, T., Ji, J., Wang, K., Jiang, K., Yu, Y., & Li, C. (2025) · Frontiers in Immunology · doi:10.3389/fimmu.2024.1539465 · Cited by 11
Mastrantoni, L., Garufi, G., Giordano, G., Maliziola, N., Monte, E. D., Arcuri, G., Frescura, V., Rotondi, A., Orlandi, A., Carbognin, L., et al. (2025) · npj Breast Cancer · doi:10.1038/s41523-025-00727-w · Cited by 11
Janse, R. J., Abu–Hanna, A., Vagliano, I., Stel, V. S., Jager, K. J., Tripepi, G. L., Zoccali, C., Dekker, F. W., & van Diepen, M. (2025) · Clinical Kidney Journal · doi:10.1093/ckj/sfaf059 · Cited by 12
“The number of candidate predictors may be limited by the size of the available data, for which sample size calculations for certain techniques are available [15 , 16 , 17 ].”
de Reus, D. C., Kuijten, R. H., Saha, P., Lastoria, D. A. A., Warr-Esser, A., Taylor, C. F. C., Groot, O. Q., Lui, D. F., Verlaan, J., & Tobert, D. G. (2025) · The Spine Journal · doi:10.1016/j.spinee.2025.03.018 · Cited by 11
Zheng, T., Yuanyuan, Z., Zhou, Z., Ye, R., Jia, H., Chen, W., Zheng, M., & Chen, Y. (2025) · Surgery · doi:10.1016/j.surg.2025.109334 · Cited by 10
Gou, D., Wu, H., Min, C., Peng, X., Jiang, M., Zhang, L., Luo, X., & Tao, M. (2025) · Journal of Affective Disorders · doi:10.1016/j.jad.2025.04.033 · Cited by 10
A machine learning–based framework for predicting postpartum chronic pain: a retrospective study
Liu, F., Li, T., Zhou, D., Shi, S., & Gong, X. (2025) · BMC Medical Informatics and Decision Making · doi:10.1186/s12911-025-03004-9 · Cited by 10
“The sample size selected for the clinical prediction model establishment met the standard of 10 events per variable [31].”
Major, F. R., Pickering, T. A., Stefanescu, K., Singh, M., Clark, D. H., Inaba, K., Nahmias, J. T., Tay-Lasso, E., Alvarez, C., Chen, J. L., et al. (2025) · Anesthesia & Analgesia · doi:10.1213/ane.0000000000007569 · Cited by 10
Lavelle-Hill, R. E., Smith, G., & Murayama, K. (2025) · Advances in Methods and Practices in Psychological Science · doi:10.1177/25152459251345696 · Cited by 11
“Several studies have provided formulas to evaluate the adequacy of sample size in certain applied-machine-learning fields (e.g., in medicine; Riley et al., 2019a, 2019b, 2020; van Smeden et al., 2019), but the extent to which these methods can be extended to different classes of machine-learning…”
Rosella, L. C., O’Neill, M., Fisher, S., Hurst, M., Diemert, L. M., Kornas, K., Hong, A., & Manuel, D. G. (2020) · Diagnostic and Prognostic Research · doi:10.1186/s41512-020-00086-z · Cited by 10
“90) and the expected model performance in terms of overall model fit (R(2)) [38, 39].”
BackgroundRees, C. A., Kisenge, R. R., Ideh, R. C., Kamara, J., Samma, A., Godfrey, E. K., Manji, H. K., Sudfeld, C. R., Westbrook, A. L., Niescierenko, M. L., et al. (2022) · BMJ Paediatrics Open · doi:10.1136/bmjpo-2021-001379 · Cited by 10
Spek, M., Venekamp, R. P., GROOT, E. D., Geersing, G., Erkelens, D. C. A., van Smeden, M., Rutten, F. H., & Zwart, D. L. M. (2022) · BMJ Open · doi:10.1136/bmjopen-2021-059549 · Cited by 10
Gao, Y., Chen, Y., Hu, M., Gan, T., Sun, X., Zhang, Z., He, W., & Wu, X. (2022) · Journal of Applied Gerontology · doi:10.1177/07334648221097084 · Cited by 11
“%) of included models were developed or validated based on insufficient sample size, which would introduce model overfitting, cause inaccurate predictions, and subsequently limit the generalizability outside the original data set (Riley et al., 2020).”
BackgroundEvaluating future risk of NAFLD in adolescents: a prediction and decision curve analysis
Abeysekera, K. W. M., Orr, J. G., Gordon, F. H., Howe, L. D., Hamilton-Shield, J. P., Heron, J., & Hickman, M. (2022) · BMC Gastroenterology · doi:10.1186/s12876-022-02401-y · Cited by 12
Huang, W., Wu, Q., Zhang, Y., Tian, C., Huang, H., Wang, H., & Mao, J. (2022) · Frontiers in Surgery · doi:10.3389/fsurg.2022.986185 · Cited by 10
“(21), at least 10 outcome events are needed per variable (EPV≥ 10).”
BackgroundPei, B., Jin, C., Cao, S., Ji, N., Xia, M., & Jiang, H. (2023) · Frontiers in Medicine · doi:10.3389/fmed.2023.1203023 · Cited by 10
Zhang, H., Dai, J., Zhang, W., Sun, X., Sun, Y., Wang, L., Li, H., & Zhang, J. (2023) · Frontiers in Endocrinology · doi:10.3389/fendo.2023.1216832 · Cited by 10
DNA Logical Computing on a Transistor for Cancer Molecular Diagnosis
Kong, D., Zhang, S., Ma, X., Yang, Y., Dai, C., Geng, L., Liu, Y., & Wei, D. (2024) · Angewandte Chemie · doi:10.1002/ange.202407039 · Cited by 10
Kritmetapak, K., Charoensri, S., Thaopanya, R., & Pongchaiyakul, C. (2020) · International Journal of General Medicine · doi:10.2147/ijgm.s277957 · Cited by 10
Wang, X., Jin, Y., Xu, X., Wang, H., & Feng, S. (2022) · International Urogynecology Journal · doi:10.1007/s00192-022-05105-9 · Cited by 10
Connell, S. P., Frantzi, M., Latosińska, A., Webb, M., Mullen, W., Pejchinοvski, M., Salji, M. J., Mischak, H., Cooper, C. S., Clark, J., et al. (2022) · Cancers · doi:10.3390/cancers14081995 · Cited by 10
Jiang, X., Zhang, D., Li, B., Yan, M., Hu, X., Wang, L., & Gao, L. (2022) · Arthroscopy The Journal of Arthroscopic and Related Surgery · doi:10.1016/j.arthro.2022.11.007 · Cited by 10
Machine Learning in Clinical Journals
Singh, K., Beam, A. L., & Nallamothu, B. K. (2020) · Circulation Cardiovascular Quality and Outcomes · doi:10.1161/circoutcomes.120.007491 · Cited by 10
“For instance, while there is helpful statistical guidance on the determination of appropriate sample sizes for prediction models, 27 it is unclear how this guidance could be applied to models utilizing radiology images consisting of millions of pixels.”
BackgroundXu, W., Tu, H., Xiong, X., Peng, Y., & Cheng, T. (2022) · Clinical Interventions in Aging · doi:10.2147/cia.s369885 · Cited by 9
“According to sample size calculation requirements, a minimum of 10 events per variable were necessary for the multivariate analysis.(11,12) A total of 1348 patients were eventually included.”
BackgroundLee, H., Hyun, S., & Park, S. (2022) · Transactions of the Royal Society of Tropical Medicine and Hygiene · doi:10.1093/trstmh/trac108 · Cited by 9
Nomogram for predicting the risk of postoperative myasthenic crisis in patients with thymectomy
Ruan, Z., Su, Y., Tang, Y., Lin, J., Lü, Q., Zhou, Y., Guo, R., Liu, Y., Li, H., Sun, C., et al. (2023) · Annals of Clinical and Translational Neurology · doi:10.1002/acn3.51752 · Cited by 9
Yan, Y., Gong, H.-R., Hu, J., Wu, D., Zheng, Z., Wang, L., & Lei, C. (2023) · Frontiers in Cardiovascular Medicine · doi:10.3389/fcvm.2023.1094997 · Cited by 9
“The estimation of overall outcome prevalence in the target population, the number of candidate predictors, and the predicted model performance regarding the overall model fit (R(2)) (22) were used to determine the sample size for binary outcome prediction model development.”
MethodIshøi, L., Thorborg, K., Kallemose, T., Kemp, J. L., Reiman, M. P., Nielsen, M. F., & Hölmich, P. (2023) · British Journal of Sports Medicine · doi:10.1136/bjsports-2022-105534 · Cited by 9
Wilkinson, B. G., Bangura, A., Burke, C. E., O’Connor, K., Gupta, J., Demyanovich, H. K., Healey, K. M., O’Hara, N. N., Schneiderman, B. A., Nascone, J. W., et al. (2023) · Injury · doi:10.1016/j.injury.2023.110965 · Cited by 9
Fente, B. M., Asaye, M. M., Tesema, G. A., & Gudayu, T. W. (2023) · BMC Pregnancy and Childbirth · doi:10.1186/s12884-023-06018-1 · Cited by 9
“As a general rule, the required sample size in the development of a prediction model is at least 10 events per predictor [36].”
BackgroundWang, Z., Chao, Y., Xu, M. M., Zhao, W., & Hu, X. (2024) · Scientific Reports · doi:10.1038/s41598-024-52061-z · Cited by 9
Billichová, M., Coan, L. J., Czanner, S., Kováčová, M., Sharifian, F., & Czanner, G. (2024) · PLoS ONE · doi:10.1371/journal.pone.0297190 · Cited by 9
Park, L. J., Moloo, H., Ramsay, T., Thavorn, K., Presseau, J., Zwiep, T., Martel, G., Devereaux, P. J., Talarico, R., & McIsaac, D. I. (2024) · British Journal of Anaesthesia · doi:10.1016/j.bja.2024.03.018 · Cited by 9
Zheng, G., Cai, J., Deng, H., Yang, H., Xiong, W., Chen, E., Bai, H., & He, J. (2024) · Antimicrobial Resistance and Infection Control · doi:10.1186/s13756-024-01394-5 · Cited by 9
“The calculation of the required sample was carried out by R (version 4.1.1) software with the pmsampsize package, according to the methods from Riley et al.’s article [17].”
MethodInfluentialShields, B. M., Carlsson, A., Patel, K. A., Knupp, J., Kaur, A., Johnston, D., Colclough, K., Larsson, H. E., Forsander, G., Samuelsson, U., et al. (2024) · Scientific Reports · doi:10.1038/s41598-024-60160-0 · Cited by 9
“We performed sample size calculations using the pmsampsize package in R, in line with the approach described by Riley et al. 20 .”
MethodCalibration plots for multistate risk prediction models
Pate, A., Sperrin, M., Riley, R. D., Peek, N. B., van Staa, T. P., Sergeant, J. C., Mamas, M. A., Lip, G. Y. H., O’Flaherty, M., Barrowman, M., et al. (2024) · Statistics in Medicine · doi:10.1002/sim.10094 · Cited by 9
“Sample size criteria do not yet exist for multistate models, 68 but would be dependent on the event rates of all the possible transitions.”
BackgroundChen, Z., Zhao, Y., Xue, X., Gu, X., Li, Y., & Hua, Y. (2024) · Foot & Ankle International · doi:10.1177/10711007241252091 · Cited by 9
Rees, C. A., Kisenge, R. R., Godfrey, E. K., Ideh, R. C., Kamara, J., Coleman-Nekar, Y.-J. G., Samma, A., Manji, H. K., Sudfeld, C. R., Westbrook, A. L., et al. (2024) · The Journal of Pediatrics · doi:10.1016/j.jpeds.2024.114147 · Cited by 9
Kouw, I. W. K., Melchers, M., Mändul, M., Arabi, Y. M., Casaer, M. P., Cotoia, A., Gunst, J., Malbrain, M. L. N. G., Schaller, S. J., Starkopf, J., et al. (2024) · Clinical Nutrition ESPEN · doi:10.1016/j.clnesp.2024.07.023 · Cited by 9
He, X., Dong, M., Xiong, H., Zhu, Y., Ping, F., Wang, B., & Kang, Y. (2024) · BMC Pulmonary Medicine · doi:10.1186/s12890-024-03153-z · Cited by 11
“The sample size of this study was calculated following formulas reported [29].”
MethodZhu, Y., Hernández–Dı́az, S., Bateman, B. T., Gray, K. J., Alt, E. M., Straub, L., Taylor, L. G., Ouellet‐Hellstrom, R., Ma, Y., Qiang, Y., et al. (2024) · American Journal of Epidemiology · doi:10.1093/aje/kwae372 · Cited by 9
Jang, S., & Hwang, S.-O. (2024) · European Journal of Obstetrics & Gynecology and Reproductive Biology · doi:10.1016/j.ejogrb.2024.09.030 · Cited by 9
Shi, J., Mao, X., Cheng, J., Shao, L., Shan, X., & Zhu, Y. (2024) · Frontiers in Cellular and Infection Microbiology · doi:10.3389/fcimb.2024.1475428 · Cited by 9
Naye, F., Légaré, F., Cachinho, C., Gérard, T., Toupin‐April, K., Sasseville, M., Paquette, J., LeBlanc, A., Gaboury, I., Poitras, M.-È., et al. (2024) · BMC Family Practice · doi:10.1186/s12875-024-02667-z · Cited by 9
“…method proposed by Riley et al. [36] using an estimated prevalence of the outcome = 0.10 (i.e., prevalence of clinically significant decisional conflict in primary care) [11], an estimated Nagelkerke’s R 2 = 0.15 [36] and an estimated number of predictors in the initial model = 30 [36].”
InfluentialWu, H., Liao, C.-C., Peng, C.-F., Lee, T.-Y., & Liao, P. (2024) · Health Information Science and Systems · doi:10.1007/s13755-024-00324-4 · Cited by 9
“Thus, the model receives at least 10 instances of training for each variable [13].”
Metadata
- Authors
- Riley, R. D., Ensor, J., Snell, K. I. E., Harrell, F. E., Martin, G. P., Reitsma, J. B., Moons, K. G. M., Collins, G., & van Smeden, M.
- Affiliations
- Keele University (GB)
- Vanderbilt University (US)
- Manchester Academic Health Science Centre (GB)
- University Medical Center Utrecht (NL)
- Corresponding author
- Riley, Richard D
- ORCID
- Venue
- BMJ
- Publisher
- BMJ
- Published
- 2020-03-18
- Volume
- 368
- Pages
- m441
- ISSN
- 1756-1833 (ISSN-L)
- OpenAlex
- W3012413426
- Cited by
- 2,917
- References
- 80
- Citations by year
- 2026: 863 · 2025: 606 · 2024: 531 · 2023: 402 · 2022: 275 · 2021: 174 · 2020: 58
- FWCI
- 160.59
- Type
- journal article
- Language
- en
- Open access
- bronze
- doi.org
- Licence
- www.bmj.com/company/legal-information/terms-conditions/legal-information/tdm-licencepolicy (tdm)
- Locations
- BMJ (publishedVersion)
- PubMed (publishedVersion)
- Research Explorer (The University of Manchester) (publishedVersion, other-oa)
- Keele Research Repository (Keele University) (submittedVersion, cc-by)
- Utrecht University Repository (Utrecht University) (submittedVersion, other-oa)
- HighWire Press Open Archive (submittedVersion)
- University of Birmingham Research Portal (University of Birmingham) (submittedVersion)
- Funders
- Georgia Clinical and Translational Science Alliance (UL1TR002243)
- Cancer Research UK (27294)
- National Institute for Health and Care Research (SPCR-142)
- Department of Health and Social Care
- Nederlandse Organisatie voor Wetenschappelijk Onderzoek (91208004, 918.10.615, project 9120.8004)
- National Institutes of Health (UL1TR002243)
- NIHR School for Primary Care Research
- National Center for Advancing Translational Sciences (TR002243)
- Topics
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Epidemiology
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Clinical Trials
- Keywords
- sample size determination, clinical prediction model, model development, prediction accuracy, prognostic model, diagnostic model, small sample size
- MeSH terms
- Clinical Decision-Making, Forecasting, Humans, Models, Theoretical, Sample Size
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review