The DOI matches its Crossref record; 78 of the 80 references it cites were found in a registry or an index.
Calculating the sample size required for developing a clinical prediction model
Open accessAbstract
Clinical prediction models aim to predict outcomes in individuals, to inform diagnosis or prognosis in healthcare. Hundreds of prediction models are published in the medical literature each year, yet many are developed using a dataset that is too small for the total number of participants or outcome events. This leads to inaccurate predictions and consequently incorrect healthcare decisions for some individuals. In this article, the authors provide guidance on how to calculate the sample size required to develop a clinical prediction model.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“In step three, the sample size needed to reduce model overfitting is calculated, assuming an expected shrinkage rate of 10%, with an expected number of variables P to be included in the final predictive model of 30: S is the expected shrinkage factor, and the noise ratio evaluation index R2cs is 0.3; the calculated sample size is 740 cases.”
From Machine learning-driven prediction model for successful weaning of patients from mechanical ventilation in ICU (2026) · cites this paper as Riley [13]
Experimental — accuracy not yet measuredThe check wasn’t confident either way, so it gives no verdict — never disputed.
In the citing paper
“Genomic and molecular data support integration of tumour biology and genetic risk into decision-making, particularly in precision medicine applications [42].”
From Artificial Intelligence in Clinical Decision-Making: A Comprehensive Review of Diagnostic, Prognostic, and Therapeutic Applications, Validation Gaps, Bias, and Deployment Challenges (2026) · cites this paper as [42]
Experimental — accuracy not yet measuredIn the citing paper
“The sample size calculated this way is the minimum for a standard statistical model.”
From Developing clinical prediction models: a step-by-step guide (2024) · cites this paper as [50]
In the citing paper
“Sample size calculations50 indicated a minimum sample of 2082 patients, which is larger than the available sample, raising concerns about possible overfitting issues (step 6).”
From Developing clinical prediction models: a step-by-step guide (2024) · cites this paper as [50]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Regression shrinkage and selection via the lasso
Tibshirani, R. (1996) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/j.2517-6161.1996.tb02080.x · Cited by 51,482
The Elements of Statistical Learning
Hastie, T., Tibshirani, R., & Friedman, J. (2009) · Springer Series in Statistics · doi:10.1007/978-0-387-84858-7 · Cited by 25,970
Regularization and Variable Selection Via the Elastic Net
Zou, H., & Hastie, T. (2005) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/j.1467-9868.2005.00503.x · Cited by 20,957
A simulation study of the number of events per variable in logistic regression analysis
Peduzzi, P., Concato, J., Kemper, E., Holford, T. R., & Feinstein, A. R. (1996) · Journal of Clinical Epidemiology · doi:10.1016/s0895-4356(96)00236-3 · Cited by 9,655
Harrell (2015) · Springer Series in Statistics · doi:10.1007/978-3-319-19425-7 · Cited by 6,878
A note on a general definition of the coefficient of determination
NAGELKERKE, N. J. D. (1991) · Biometrika · doi:10.1093/biomet/78.3.691 · Cited by 5,927
Moons, K. G. M., Altman, D. G., Reitsma, J. B., Ioannidis, J. P. A., Macaskill, P., Steyerberg, E. W., Vickers, A. J., Ransohoff, D. F., & Collins, G. S. (2015) · Annals of Internal Medicine · doi:10.7326/m14-0698 · Cited by 5,147
Self-citationRelaxing the Rule of Ten Events per Variable in Logistic and Cox Regression
Vittinghoff, E., & McCulloch, C. E. (2007) · American Journal of Epidemiology · doi:10.1093/aje/kwk052 · Cited by 4,359
Clinical Prediction Models: A Practical Approach to Development, Validation, and Updating
Steyerberg, E. W. (2019) · Statistics for Biology and Health · doi:10.1007/978-3-030-16399-0 · Cited by 3,281
PROBAST: A Tool to Assess the Risk of Bias and Applicability of Prediction Model Studies
Wolff, R. F., Moons, K. G. M., Riley, R. D., Whiting, P. F., Westwood, M., Collins, G. S., Reitsma, J. B., Kleijnen, J., Mallett, S., & for the PROBAST Group† (2019) · Annals of Internal Medicine · doi:10.7326/m18-1376 · Cited by 2,984
Self-citation
Research works printed but not linked: 2
- 10.1177/962280218784726 · DOI
- McFadden, 1974, Frontiers in Econometrics New York., 104
Cited by
Sorted by citations
Items 701–800 of 1,003
Chen, M., Liu, P., Yi, J.-Z., Li, X., He, T., Wu, H., Yang, J., & Lv, Q. (2022) · Scientific Reports · doi:10.1038/s41598-022-18727-2 · Cited by 8
Development of a Root Caries Prediction Model in a Population of Dental Attenders
Fee, P. A., Cassie, H., Clarkson, J., Hall, A. F., Ricketts, D., Walsh, T., & Goulão, B. (2022) · Caries Research · doi:10.1159/000526797 · Cited by 8
Gjærde, L. K., Ostrowski, S. R., Schierbeck, F., Andersen, N. S., Friis, L. S., Kornblit, B. T., Petersen, S. L., Schjødt, I. M., & Sengeløv, H. (2022) · Transplantation and Cellular Therapy · doi:10.1016/j.jtct.2022.11.009 · Cited by 8
Chambers, P., Forster, M. D., Patel, A., Duncan, N., Kipps, E., Wong, I. C. K., Jani, Y. H., & Li, W. (2022) · ESMO Open · doi:10.1016/j.esmoop.2022.100743 · Cited by 8
Clinical prediction model for prognosis in kidney transplant recipients (KIDMO): study protocol
Schwab, S., Sidler, D., Haidar, F., Kuhn, C., Schaub, S., Koller, M., Mellac, K., Stürzinger, U., Tischhauser, B., Binet, I., et al. (2023) · Diagnostic and Prognostic Research · doi:10.1186/s41512-022-00139-5 · Cited by 7
“Sample size Guidelines regarding the minimum required sample size for the development of a new multivariable prediction model have been proposed [30].”
BackgroundUric acid/superoxide dismutase can predict progression of gestational hypertension to preeclampsia
Yun, L., Yu, X., & Xu, R. (2023) · Frontiers in Cardiovascular Medicine · doi:10.3389/fcvm.2023.1148376 · Cited by 10
“Sample size calculation: based on the principle (19) that the sample size should be at least 10–20 times the number of independent variables, 19 variables were included in this study: age, height, weight, BMI, BP, HR, UA, BUN, TBIL, TBA, FFA, DBIL, ALP, Scr, cystatin C, GLU, SOD, Na+, and K+.”
BackgroundLv, J., Wang, C., Gao, X., Yang, J., Zhang, X., Ye, Y., Dong, Q., Fu, R., Sun, H., Yan, X., et al. (2023) · BMJ Open · doi:10.1136/bmjopen-2022-069505 · Cited by 8
Zhang, H., Zuo, L., Li, J., Geng, Z., Ge, S., Song, X., Wang, Y., Zhang, X., Wang, L. Y., Zhao, T., et al. (2023) · Frontiers in Immunology · doi:10.3389/fimmu.2023.1126217 · Cited by 8
“As previously reported (13), the validation cohort sample size (n) was calculated according to the following formula:”
MethodTanaka, M. D., Geubels, B. M., Grotenhuis, B. A., Marijnen, C. A. M., Peters, F. P., van der Mierden, S., Maas, M., & Couwenberg, A. M. (2023) · Cancers · doi:10.3390/cancers15153945 · Cited by 8
“A sample size calculation can be helpful to identify the maximum number of candidate predictors or to identify a dataset that is too small for a particular research question [84].”
BackgroundWang, Y., Wang, L., Tang, X., Zhang, Y., Zhang, N., Zhi, B., & Niu, X.-K. (2023) · BMC Medical Imaging · doi:10.1186/s12880-023-01074-7 · Cited by 8
“There is no accepted method to estimate the sample size required to develop a predictive model; however, an established rule of thumb is to ensure that at least 10 events per predictive parameter are considered for inclusion in the predictive model equation [16].”
Alalawi, A., Evans, D. W., Liew, B. X. W., Peolsson, A. L. C., Heneghan, N. R., Rushton, A., Peterson, G., Barbero, M., & Falla, D. (2022) · Journal of Clinical Medicine · doi:10.3390/jcm11030555 · Cited by 9
“Sample size calculations were handled using the pmsampsize package, carried out using R software [41].”
BackgroundMethodInfluentialKe, J. X. C., McIsaac, D. I., George, R. B., Branco, P., Cook, E. F., Beattie, W. S., Urquhart, R., & MacDonald, D. B. (2022) · Canadian Journal of Anesthesia/Journal canadien d'anesthésie · doi:10.1007/s12630-022-02287-0 · Cited by 6
“Nevertheless, EPV has many limitations, and there is limited guidance for models involving[ 30 predictors.(24) In the derivation set, first we removed or combined nonvital sign features based on correlation matrix (collinearity), domain knowledge, and data quality.”
BackgroundMettananda, C. D., Egodage, T., Dantanarayana, C., Fernando, R., Ranaweera, L., Luke, N. V., Ranawaka, C. K., Kottahachchi, D., Pathmeswaran, A., de Silva, H. J., et al. (2023) · BMJ Open · doi:10.1136/bmjopen-2022-063959 · Cited by 7
“The sample size was calculated for developing a clinical prediction model 35 36 assuming an R 2 value of 0.2 for the logistic model, 10 parameters in the model, and 0.05 acceptable difference in apparent and adjusted R 2 , 0.05 margin of error in the estimation of intercept and an anticipated…”
InfluentialChen, C., Zhao, B., Pan, Y. P., Chen, L., Yang, X., Lv, M., Qiu, L., Yang, M., Ying, X., Wang, M., et al. (2023) · Acta Obstetricia Et Gynecologica Scandinavica · doi:10.1111/aogs.14758 · Cited by 6
“19 For model 1 predicting extremely PTB, the significance of each variable was assessed by univariate logistic regression.”
MethodWang, L., Qin, F., Zhen, L., Li, R., Tao, S., & Li, G. (2024) · Journal of Clinical Nursing · doi:10.1111/jocn.17031 · Cited by 7
Peng, H., Wu, X., Cui, X., Liu, S., Liang, Y., Cai, X., Shi, M., Zhong, R., Li, C., Liu, J., et al. (2024) · Translational Lung Cancer Research · doi:10.21037/tlcr-23-800 · Cited by 7
“%) via stratified sampling (35).”
Wang, F., Zhu, Y., Wang, L., Huang, C., Mei, R., Deng, L.-E., Yang, X., Xu, Y., Zhang, L., & Xu, M. (2024) · Asia-Pacific Journal of Oncology Nursing · doi:10.1016/j.apjon.2024.100546 · Cited by 8
“22,23 To ensure the predictive performance of the model and avoid over fi tting, the calculation of the sample size took into account the estimated coef fi cient of determination ( R 2 ) of the model the anticipated number of predictive parameters to be included, and the shrinkage level required for…”
A simple nomogram to predict dengue shock syndrome: A study of 4522 south east Asian children
Tran, P. N. T., Siranart, N., Sukmark, T., Limothai, U., Tachaboon, S., Tantawichien, T., Thisyakorn, C., Thisyakorn, U., & Srisawat, N. (2024) · Journal of Medical Virology · doi:10.1002/jmv.29874 · Cited by 7
“Sample size was calculated following a recommendation for developing 23 and validating clinical predictive models, 24,25 using pmsamplesize package in Rstudio program (Supporting Information S1: Section 1).”
Ma, J., Tian, M., Zhu, Y., Hu, J., Zhang, Y., & Li, X. (2024) · Scientific Reports · doi:10.1038/s41598-024-70464-w · Cited by 7
Common Critiques and Recommendations for Studies in Neurology Using Machine Learning Methods
Speiser, J. L., Kerr, W. T., & Ziegler, A. (2024) · Neurology · doi:10.1212/wnl.0000000000209861 · Cited by 7
Nejad, M. C., Matthiesen, R. V., Dukovska‐Popovska, I., Jakobsen, T. L., & Johansen, J. (2024) · International Journal of Medical Informatics · doi:10.1016/j.ijmedinf.2024.105631 · Cited by 7
Lindman, I., Augustsson, S. R., Baranto, A., Sansone, M., & Abrahamson, J. (2024) · BMJ Open Sport & Exercise Medicine · doi:10.1136/bmjsem-2024-002253 · Cited by 7
The Search for Sepsis Biomarkers: A Tale of Promises, Pitfalls, and Potential
Zimmermann, T., Brealey, D. A., & Singer, M. (2024) · Critical Care Medicine · doi:10.1097/ccm.0000000000006560 · Cited by 7
“Instead of concluding “larger studies are required”, power calculations should be performed during study planning to ensure that the sample size will support the intended analysis to enhance the likelihood of detecting a true effect (12).”
Zhang, Z., Zhu, M., & Jiang, W. (2025) · Clinical and Translational Science · doi:10.1111/cts.70118 · Cited by 7
“The minimum sample size required for the model was calculated using the pmsampsize package in R [17].”
Frydenlund, J., Cosgrave, N., Moriarty, F., Wallace, E., Kirke, C., Williams, D. J., Bennett, K., & Cahir, C. (2025) · European Geriatric Medicine · doi:10.1007/s41999-024-01152-1 · Cited by 7
“To compute the minimum sample size required for the development of a new multivariable prediction model, we used the criteria proposed by Riley et al. [28].”
Kareemi, H., Li, H., Rajaram, A., Holodinsky, J. K., Hall, J. N., Grant, L., Goel, G., Hayward, J., Mehta, S., Ben‐Yakov, M., et al. (2025) · Canadian Journal of Emergency Medicine · doi:10.1007/s43678-024-00826-w · Cited by 8
Xu, M., Zhu, W., Hou, S., Xu, H., Xia, J., Lin, L., Fu, H., You, M., Wang, J., Xie, Z., et al. (2025) · Chinese Medical Journal · doi:10.1097/cm9.0000000000003433 · Cited by 8
Velez, T., Ibrahim, Z., Duru, K., Velez, D., Triantafyllou, M., McKinley, K. W., Saif, P., Kratimenos, P., Clark, A., & Koutroulis, I. (2025) · International Journal of Medical Informatics · doi:10.1016/j.ijmedinf.2025.105905 · Cited by 7
Risk prediction models for patients with recurrent diabetic foot ulcers: A systematic review.
Zhou, Z., Jia, Y., Yan, H., Xu, J.-L., Wang, S.-Y., & Wen, J. (2025) · Public Health · doi:10.1016/j.puhe.2025.105744 · Cited by 7
Kim, K. H., Mun, S. B., Kim, B.-C., Kim, K. G., & Kim, K. G. (2025) · PLoS ONE · doi:10.1371/journal.pone.0322529 · Cited by 7
Sánchez, J., Caraballo, A., Chérrez-Ojeda, I., García, E., Larco, J. I., Ramon, G. D., Velásquez‐Lopera, M. M., & Jaimes, F. A. (2025) · The Journal of Allergy and Clinical Immunology In Practice · doi:10.1016/j.jaip.2025.05.051 · Cited by 7
Zhu, Y., Liu, Y., Zhao, Y., Lu, Q., Wang, W., Chen, T., Ji, P., & Chen, T. (2025) · Clinical Oral Implants Research · doi:10.1111/clr.70011 · Cited by 7
Yu, Y., Zhang, S., Li, H., & Xue, F. (2025) · BMC Psychiatry · doi:10.1186/s12888-025-07189-1 · Cited by 7
“We used the pmsamplesize R package [30] to estimate the minimum required sample size to satisfy three criteria: (1) a global shrinkage factor ≥ 0.9, (2) small absolute difference of ≤ 0.05 in adjust Nagelkerke’s R2, and (3) precise estimation of the overall risk in the population.”
InfluentialLi, Y., Mei, Z., Sun, G., Liu, Z., Li, J., Ong, M. E. H., Fan, H., & Cao, C. (2025) · BMC Public Health · doi:10.1186/s12889-025-24724-7 · Cited by 7
“Assuming 18 candidate predictor variables and a conservative 15% of the maximal Cox-Snell R2, we estimated that the minimum sample size for fitting the regression models was 14,638 [17, 18].”
Yong, W., Peng, D., Ye, K., Gao, J., & Cao, R.-X. (2026) · Frontiers in Endocrinology · doi:10.3389/fendo.2026.1720574 · Cited by 7
Machine Learning for Predicting Stroke Risk Stratification Using Multiomics Data: Systematic Review
Yoo, H. Y., Shin, H., Kim, E. E., & Son, Y.-J. (2026) · Journal of Medical Internet Research · doi:10.2196/85654 · Cited by 7
“Given the small sample size and high-dimensional feature space, this boundary estimate is more appropriately interpreted as an optimistic methodological artifact rather than valid or generalizable predictive performance [48,49].”
Deng, X., Yang, M. S., Wang, S., Zhou, B., Wang, K., Zhang, Z., & Niu, W. (2022) · Frontiers in Neuroscience · doi:10.3389/fnins.2021.709374 · Cited by 8
Huang, H., Wang, J., Zhu, Y., Liu, J., Zhang, L., Shi, W., Hu, W., Ding, Y., Zhou, R. J., & Jiang, H. (2023) · Journal of Clinical Medicine · doi:10.3390/jcm12031066 · Cited by 7
“With the help of machine-learning, more accurate models can be developed for clinical diagnosis, prediction, and decision-making [26,32,33].”
MethodHarrison, T. G., Hemmelgarn, B. R., James, M. T., Sawhney, S., Manns, B., Tonelli, M. A., Ruzycki, S. M., Zarnke, K. B., Wilson, T., McCaughey, D., et al. (2023) · BMC Nephrology · doi:10.1186/s12882-023-03093-6 · Cited by 7
“Sample Size We performed sample size calculations recommended for binary outcomes [18], using the ‘pmsampsize’ module in Stata software version 16 and 17 (StataCorp) [19].”
MethodSex differences in clinical outcomes amongst 1105 patients admitted with hip fractures
Fluck, D. S., Lisk, R., Yeong, K., Mahmood, R., Robin, J. G., Fry, C. H., & Han, T. S. (2023) · Internal and Emergency Medicine · doi:10.1007/s11739-023-03264-1 · Cited by 7
Chivu, C.-D., Crăciun, M.-D., Pițigoi, D., Aramă, V., Luminos, M. L., Jugulete, G., Apostolescu, C. G., & Streinu‐Cercel, A. (2023) · Vaccines · doi:10.3390/vaccines11050899 · Cited by 7
“When selecting input variables, we restricted the number to avoid overfitting the model and chose only the independent variable [21].”
MethodMao, L., Li, J., Schwedt, T. J., Berisha, V., Nikjou, D., Wu, T., Dumkrieger, G. M., Ross, K., & Chong, C. D. (2023) · Cephalalgia · doi:10.1177/03331024231172736 · Cited by 7
“In our study, the number of features (290 imaging measures and five questionnaire variables) exceeded the number of participants, which can lead to an overfitted model (19).”
BackgroundShen, Y. C., Xiong, Y., Cao, Q., Li, Y., Xiang, W., Wang, L., Nie, Q., Tang, B., Yang, Y., & Hong, D. (2023) · Journal of Thrombosis and Thrombolysis · doi:10.1007/s11239-023-02828-4 · Cited by 7
“56%, the total sample size was calculated to be at least 100 [21].”
BackgroundJebeile, H., Lister, N. B., Libesman, S., Hunter, K. E., McMaster, C. M., Johnson, B. J., Baur, L. A., Paxton, S. J., Garnett, S. P., Ahern, A. L., et al. (2023) · PLoS ONE · doi:10.1371/journal.pone.0282401 · Cited by 7
“If there is an appropriate sample size (59-61), this process is 377 repeated for each study and meta-analysis, and is used to summarise estimates and 378 heterogeneity to evaluate how reliable the model is when applied to different populations 379 (58).”
MethodHua, X., Long, Z., Wang, S., Xu, F., Wang, M.-D., Chen, J., Zhang, Y., Ni, W., & Yun-sheng, G. (2023) · Frontiers in Nutrition · doi:10.3389/fnut.2023.1162280 · Cited by 7
“However, the enrolled participants were 841, and the events of this study amounted to 86, with an excess rate of 10 events per variable in multivariate models, indicating sufficient evaluating power (24).”
BackgroundBelavý, D. L., Tagliaferri, S. D., Tegenthoff, M., Enax‐Krumova, E., Schlaffke, L., Buehring, B., Schulte, T. L., Schmidt, S. H., Wilke, H., Angelova, M. N., et al. (2023) · PLoS ONE · doi:10.1371/journal.pone.0282346 · Cited by 10
Chambers, P., Watson, M., Bridgewater, J. A., Forster, M. D., Roylance, R. R., Burgoyne, R., Masento, S., Steventon, L., King, J. H., Duncan, N., et al. (2023) · Cancer Medicine · doi:10.1002/cam4.6418 · Cited by 7
Held, U., Forzy, T., Signorell, A., Deforth, M., Burgstaller, J., & Wertli, M. (2023) · Pain · doi:10.1097/j.pain.0000000000003023 · Cited by 6
Wang, K., Wang, G., Liu, Y., Dong, L., Niu, Y., & Li, G. (2024) · World Journal of Urology · doi:10.1007/s00345-023-04698-0 · Cited by 7
“Our model still needs validation by external and multi-center, and we performed the sample size estimation with the formula developed by Riley et al. [28].”
Miao, W., Lu, Y., Xv, H., Chen, Z., Yang, W., Qian, X., Chen, J., & Geng, G. (2024) · BMJ Open · doi:10.1136/bmjopen-2023-075466 · Cited by 6
“25 It requires the prevalence of the binary outcome and the number of candidate predictor parameters.”
BackgroundSánchez, J., Jaimes, F. A., García, E., Zakzuk, J., Cardona, R., & Velásquez‐Lopera, M. M. (2024) · PLoS ONE · doi:10.1371/journal.pone.0295791 · Cited by 9
Shen, X., Li, J., Yan, H., Zhou, S., Yang, S., & Li, W. (2024) · Heliyon · doi:10.1016/j.heliyon.2024.e29934 · Cited by 7
“Furthermore, the relatively small sample size limited our ability to include a more extensive set of covariates in our analysis [29,30].”
BackgroundPredicting Reduction Mammaplasty Total Resection Weight With Machine Learning
Seu, M. Y., Rezania, N., Murray, C. E., Qiao, M. T., Arnold, S. H., Siotos, C., Ferraro, J. J., Jazayeri, H. E., Hood, K. C., Shenaq, D. S., et al. (2024) · Annals of Plastic Surgery · doi:10.1097/sap.0000000000004016 · Cited by 7
Risk Prediction Models for Sarcopenia in Dialysis Patients: A Systematic Review
Leng, Y.-J., Wang, G., Xie, R.-N., Jiang, X., Li, C.-X., Nie, Z., & Li, T. (2024) · Journal of Renal Nutrition · doi:10.1053/j.jrn.2024.05.009 · Cited by 7
Hua, X., Xu, F., Shi, W. S., Long, Z., Huang, X., Duan, F., Wang, S., Zhang, C., Wang, M.-D., Ni, W., et al. (2024) · BMC Cancer · doi:10.1186/s12885-024-12499-w · Cited by 7
“This is an excess rate of 10 events per variable in the multivariate models, which demonstrates that there was enough power for evaluation [21].”
BackgroundThe Florida Geriatric Head Trauma CT Clinical Decision Rule
Shih, R. D., Alter, S. M., Wells, M., Solano, J. J., Engström, G., Clayton, L. M., Hughes, P. G., Goldstein, L. N., Lottenberg, L. L., & Ouslander, J. G. (2024) · Journal of the American Geriatrics Society · doi:10.1111/jgs.19057 · Cited by 7
“We used the method described by Riley et al. 26 Based on a margin of error of ≤ 0.05 and an anticipated outcome proportion in the study population of 0.05, 73 patients with the outcome would be required, or a total sample size of 1460 patients.”
InfluentialPredicting inferior vena cava filter complications using machine learning
Li, B., Eisenberg, N., Beaton, D., Lee, D. S., Al‐Omran, L., Wijeysundera, D. N., Hussain, M. A., Rotstein, O. D., de Mestral, C., Mamdani, M., et al. (2024) · Journal of Vascular Surgery Venous and Lymphatic Disorders · doi:10.1016/j.jvsv.2024.101943 · Cited by 7
Dejardin, D., Kraxner, A., Schindler, E., Städler, N., & Wolbers, M. (2024) · Frontiers in Immunology · doi:10.3389/fimmu.2024.1351584 · Cited by 6
“A dataset of suf fi cient size and quality is essential in order to develop a robust prognostic model (Riley et al. (40)).”
Lee, R., Griffiths, S. L., Gkoutos, G. V., Wood, S. J., Bravo-Merodio, L., Lalousis, P. A., Everard, L., Jones, P. B., Fowler, D. G., Hodegkins, J., et al. (2024) · Schizophrenia Research · doi:10.1016/j.schres.2024.09.010 · Cited by 8
“Whilst research has acknowledged that the 10 events per variable (EPV) rule is not a strict criterion (Riley et al., 2020; van Smeden et al., 2019), low EPVs tend to result in poorer performance (Steyerberg et al., 2003).”
BackgroundYang, Y., Yan, Y., Zhou, Z., Zhang, J., Han, H., Zhang, W., Wang, X., Chen, C., Ge, W., Pan, J., et al. (2024) · International Journal of Cardiology · doi:10.1016/j.ijcard.2024.132925 · Cited by 7
Wu, J., Li, J., Huang, B., Dong, S., Wu, L., Shen, X., & Zheng, Z. (2025) · Translational Oncology · doi:10.1016/j.tranon.2025.102281 · Cited by 7
“Moreover, we estimated sample size by using pmsampsize package of R software (R2cs = 0.28, parameters = 10, prevalence = 0.15), and then at least 294 cases were required for model development [34].”
InfluentialDong, J., Jin, Z., Li, C., Yang, J., Jiang, Y., Li, Z., Chen, C., Zhang, B., Ye, Z., Hu, Y., et al. (2025) · Journal of Medical Internet Research · doi:10.2196/68509 · Cited by 10
“Additionally, we followed the 4-step procedure proposed by Riley et al [22] to calculate the required sample size.”
Aires, F., Rodrigues, E. D., Piñeiro‐Lamas, M., Herdeiro, M. T., Figueiras, A., Oliveira, M. J., Marques, M., & Pinto, A. T. (2022) · Cancers · doi:10.3390/cancers14030491 · Cited by 7
“The final predictive multivariate model was internally validated [42,43] with the repeated data-splitting technique [44].”
MethodTefre, S., Lilja‐Cyron, A., Arvidsson, L., Bartek, J. J., Corell, A., Forsse, A., Glud, A. N., Hamdeh, S. A., Hansen, F. L., Huotarinen, A., et al. (2022) · BMJ Open · doi:10.1136/bmjopen-2021-055570 · Cited by 7
Xu, F., Zhao, L.-H., Wang, X.-H., Wang, C., Yu, C., Zhang, X., Ning, L.-Y., Huang, H., Su, J., & Wang, X. (2022) · Diabetology & Metabolic Syndrome · doi:10.1186/s13098-022-00795-z · Cited by 7
“Statistical analysis According to the literature [24], the required sample size for a binary outcome is calculated as the following”
MethodFarooq, S., Hattle, M., Dazzan, P., Kingstone, T., Ajnakina, O., Shiers, D., Nettis, M. A., Lawrence, A. J., Riley, R. D., & van der Windt, D. (2022) · BMJ Open · doi:10.1136/bmjopen-2021-056420 · Cited by 7
Mori, H., Kataoka, Y., Harada‐Shirado, K., Kawano, N., Hayakawa, M., Seki, Y., Uchiyama, T., Yamakawa, K., Ishikura, H., Irie, Y., et al. (2022) · Thrombosis Journal · doi:10.1186/s12959-022-00390-2 · Cited by 9
Roberts, D. J., Mor, R., Rosen, M. N., Talarico, R., Lalu, M. M., Jerath, A., Wijeysundera, D. N., & McIsaac, D. I. (2022) · Anesthesia & Analgesia · doi:10.1213/ane.0000000000006232 · Cited by 7
Yuan, L., Shen, L., Ji, M., Wen, X., Wang, S., Huang, P., Li, Y., & Xu, J. (2022) · Digestive Diseases and Sciences · doi:10.1007/s10620-022-07768-2 · Cited by 7
Risk Prediction and Machine Learning
Balczewski, E. A., Cao, J., & Singh, K. (2023) · Clinical Journal of the American Society of Nephrology · doi:10.2215/cjn.0000000000000083 · Cited by 7
Chen, X., Wang, X.-F., Pan, Z.-C., Zhang, D., Zhu, K.-C., Jiang, T., Kong, X., Xie, R., Sun, L., Tao, B., et al. (2023) · Frontiers in Endocrinology · doi:10.3389/fendo.2023.1109322 · Cited by 7
“The sample size of our study was large enough and already met the ten events per variable principle, which is a widely advocated minimal criterion of the sample size to avoid overfitting in regression analysis (33).”
BackgroundYordanov, T. R., Lopes, R. R., Ravelli, A. C. J., Vis, M. M., Houterman, S., Marquering, H. A., & Abu–Hanna, A. (2023) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2023.02.021 · Cited by 7
Duss, S. B., Bernasconi, C., Steck, A., Brill, A., Manconi, M., Dekkers, M. P. J., Schmidt, M. H., & Bassetti, C. L. (2023) · European Journal of Neurology · doi:10.1111/ene.15784 · Cited by 7
“The final number of patients enrolled in the pre- determined recruitment period end evaluable for this analysis (437 patients, 394 contributing to the key regression analysis) is still considered adequate, also in consideration of the number of primary events [27].”
BackgroundThe Timing of Initiating Hydrocortisone and Long-term Mortality in Septic Shock
Zhang, L., Gu, W., Huang, T., Lyu, J., & Yin, H. (2023) · Anesthesia & Analgesia · doi:10.1213/ane.0000000000006516 · Cited by 8
Feng, Z., Lin, H., Liu, Z., Yan, L., Wang, Y., Li, B., Liu, E., Han, C., Shi, Z., Liu, Z., et al. (2023) · Computer Methods and Programs in Biomedicine · doi:10.1016/j.cmpb.2023.107617 · Cited by 7
Liu, J., Li, S., Cao, Q., Zhang, Y., Nickel, D. M., Zhu, J., & Cheng, J. (2023) · Annals of Surgical Oncology · doi:10.1245/s10434-023-13756-1 · Cited by 7
Verweij, L. P. E., Sierevelt, I. N., van der Woude, H.-J., Hekman, K. M. C., Veeger, H. E. J., & van den Bekerom, M. P. J. (2023) · Arthroscopy The Journal of Arthroscopic and Related Surgery · doi:10.1016/j.arthro.2023.07.060 · Cited by 7
Olsen, M., André, F., Delfin, C., Håkansson, A., & Claesdotter‐Knutsson, E. (2023) · Emerging Trends in Drugs Addictions and Health · doi:10.1016/j.etdah.2023.100058 · Cited by 7
Deconstructing depression by machine learning: the POKAL-PSY study
Eder, J., Pfeiffer, L., Wichert, S. P., Keeser, B., Simon, M. S., Popovic, D., Glocker, C., Brunoni, A. R., Schneider, A., Gensichen, J. S., et al. (2023) · European Archives of Psychiatry and Clinical Neuroscience · doi:10.1007/s00406-023-01720-9 · Cited by 7
“Also, this is a sufficient number of participants for machine learning algorithms [106].”
BackgroundWilligenburg, T., van Son, M. J., van de Pol, S. M. G., Eppinga, W. S. C., Lagendijk, J. J. W., de Boer, H. C. J., Moerland, M. A., van Zyp, J. R. N. V. D. V., & Peters, M. (2021) · Clinical and Translational Radiation Oncology · doi:10.1016/j.ctro.2021.06.005 · Cited by 7
Jin, S., Wang, Z., Tian, L., Sun, Z., Lin, Z., & Qian, D. (2021) · BMJ Open · doi:10.1136/bmjopen-2021-053452 · Cited by 8
Galli, M. G., Djurić, O., Besutti, G., Ottone, M., Amidei, L., Bitton, L., Bonilauri, C., Boracchia, L., Campanale, S., Curcio, V., et al. (2022) · BMJ Open · doi:10.1136/bmjopen-2021-052665 · Cited by 7
Benzekry, S., Grangeon, M., Karlsen, M., Alexa, M., Bicalho-Frazeto, I., Chaleat, S., Tomasini, P., Barbolosi, D., Barlési, F., & Greillier, L. (2021) · medRxiv · doi:10.1101/2021.11.30.21267064 · Cited by 7
Labbé, D., Tao, G., Maas, E. T., Best, K. L., Bundon, A., Sauvé, J., & Miller, W. C. (2023) · Psychology of sport and exercise · doi:10.1016/j.psychsport.2023.102501 · Cited by 6
Castellano, J., del Campo, R. L., & Hileno, R. (2023) · Biology of Sport · doi:10.5114/biolsport.2024.132984 · Cited by 6
“…in the present study ( R 2adj = .82; M and SD of TotDisTea = 78.3 and 6.3 km, respectively) and the four-step pro - cedure proposed by Riley et al. [33, 34], future football studies will be able to more adequately calculate the minimum sample size re - quired to build a linear regression model to…”
BackgroundInfluentialKamber, F., Roth, S., Bolliger, D., Seeberger, E. E., Nienhaus, J., Mueller, C. E., Buse, G. L., & Mauermann, E. (2024) · Canadian Journal of Anesthesia/Journal canadien d anesthésie · doi:10.1007/s12630-023-02677-y · Cited by 6
“Although literature published after our sample size estimation has suggested that more events per predictor may be required, 34,35 our analysis still allows for a multivariable analysis.”
Li, B., Shaikh, F., Zamzam, A., Abdin, R., & Qadura, M. (2024) · Medicina · doi:10.3390/medicina60060997 · Cited by 6
“The sample size of this study was 336 patients (290 with CAS and 46 without CAS), which was determined using a validated sample size calculator for clinical prediction models [35].”
MethodNik‐Ahd, F., Zhao, S., Boscardin, W. J., Wang, L., Covinsky, K. E., & Suskind, A. M. (2024) · The Journal of Urology · doi:10.1097/ju.0000000000003978 · Cited by 6
“34 As the study population was far greater, there was less concern about model overfitting and, thus, a validation cohort was not employed.”
Chantakhow, S., Tepmalai, K., Tantraworasin, A., & Khorana, J. (2024) · Journal of Pediatric Surgery · doi:10.1016/j.jpedsurg.2024.161696 · Cited by 6
Hunt, A., Law, H., Carney, R., Mulholland, R. E., Flores, A., Smith, C. T., Varese, F., Parker, S., Yung, A. R., & Bonnett, L. J. (2024) · Frontiers in Psychiatry · doi:10.3389/fpsyt.2024.1408738 · Cited by 6
Wu, J., Li, J., Zhang, H., Wu, L., Shen, X., & Lv, W. (2024) · European Journal of Radiology · doi:10.1016/j.ejrad.2024.111836 · Cited by 6
Todd, E., Orr, R., Gamage, E., West, E. C., Jabeen, T., McGuinness, A. J., George, V., Phuong-Nguyen, K., Voglsanger, L. M., Jennings, L., et al. (2024) · Computers in Biology and Medicine · doi:10.1016/j.compbiomed.2024.109521 · Cited by 5
Construction and validation of a nomogram model for predicting diabetic peripheral neuropathy
Liu, H., Liu, Q., Chen, M., Lu, C., & Feng, P. (2024) · Frontiers in Endocrinology · doi:10.3389/fendo.2024.1419115 · Cited by 6
Factors Associated with Pediatric Drowning-Associated Lung Injury
Shenoi, R., Crowe, J. E., Dorfman, S. R., Bergmann, K. R., Mistry, R. D., Hariharan, S. L., Tavarez, M. M., Wai, S. S., Jones, J. L., Langhan, M. L., et al. (2024) · The Journal of Pediatrics · doi:10.1016/j.jpeds.2024.114459 · Cited by 6
Molin, K., Barry, N., Gill, S., Hassan, G. M., Francis, R. J., Ong, J. S. L., Ebert, M. A., & Kendrick, J. (2025) · Physical and Engineering Sciences in Medicine · doi:10.1007/s13246-024-01516-8 · Cited by 6
“Although data from two centres were used, the sample size from the second centre was insufficient for validation [48].”
Tack, B., Vita, D., Mbuyamba, J., Ntangu, E., Vuvu, H., Kahindo, I., Ngina, J., Luyindula, A., Nama, N., Mputu, T., et al. (2025) · BMC Infectious Diseases · doi:10.1186/s12879-024-10319-x · Cited by 7
Zhao, Q., Liu, M., Kai-xia, G. A. O., Zhang, B.-S., Qi, F.-Y., Xing, T.-R., Liu, C.-C., & Gao, J.-P. (2025) · Scientific Reports · doi:10.1038/s41598-025-91075-z · Cited by 6
“A widely accepted guideline recommends at least 10 events per variable (EPV) 13 .”
Bonsdorff, A., Kjeseth, T., Kirkegård, J., de Ponthaud, C., Ghorbani, P., Wennerblom, J. H., Williamsson, C., Acher, A. W., Thillai, M., Tarvainen, T., et al. (2025) · British journal of surgery · doi:10.1093/bjs/znae313 · Cited by 6
“The number of predictor parameters that could be used in model development given the available sample was calculated with the pmsampsize package in R 10 .”
Zhou, Y., Wu, Z. K., Zeng, L., & Chen, R. (2025) · BMC Medicine · doi:10.1186/s12916-025-04048-4 · Cited by 9
“…required a minimum sample size of 7954 participants (125 outcomes) based on a maximum of 20 parameters, Cox-Snell R 2 of 0.02235 (15% of the maximum permitted value, 0.149), estimated outcome rate of 0.0053 per person year, and a 3 year time horizon, and a mean follow-up period of 2.96 years [36].”
InfluentialChen, J., Zhong, J., Zhuang, Y., Deng, B., Hong, J., Lin, Y., Su, Z., & Wen, X. (2025) · Thyroid · doi:10.1089/thy.2024.0586 · Cited by 6
“Based on a 23% carcinoma prevalence from the KSThR2016 study, 20 and the C indexes from existing prediction models, we planned to include approximately 720 nodules, as Riley et al. 21 indicated.”
Metadata
- Authors
- Riley, R. D., Ensor, J., Snell, K. I. E., Harrell, F. E., Martin, G. P., Reitsma, J. B., Moons, K. G. M., Collins, G., & van Smeden, M.
- Affiliations
- Keele University (GB)
- Vanderbilt University (US)
- Manchester Academic Health Science Centre (GB)
- University Medical Center Utrecht (NL)
- Corresponding author
- Riley, Richard D
- ORCID
- Venue
- BMJ
- Publisher
- BMJ
- Published
- 2020-03-18
- Volume
- 368
- Pages
- m441
- ISSN
- 1756-1833 (ISSN-L)
- OpenAlex
- W3012413426
- Cited by
- 2,917
- References
- 80
- Citations by year
- 2026: 863 · 2025: 606 · 2024: 531 · 2023: 402 · 2022: 275 · 2021: 174 · 2020: 58
- FWCI
- 160.59
- Type
- journal article
- Language
- en
- Open access
- bronze
- doi.org
- Licence
- www.bmj.com/company/legal-information/terms-conditions/legal-information/tdm-licencepolicy (tdm)
- Locations
- BMJ (publishedVersion)
- PubMed (publishedVersion)
- Research Explorer (The University of Manchester) (publishedVersion, other-oa)
- Keele Research Repository (Keele University) (submittedVersion, cc-by)
- Utrecht University Repository (Utrecht University) (submittedVersion, other-oa)
- HighWire Press Open Archive (submittedVersion)
- University of Birmingham Research Portal (University of Birmingham) (submittedVersion)
- Funders
- Georgia Clinical and Translational Science Alliance (UL1TR002243)
- Cancer Research UK (27294)
- National Institute for Health and Care Research (SPCR-142)
- Department of Health and Social Care
- Nederlandse Organisatie voor Wetenschappelijk Onderzoek (91208004, 918.10.615, project 9120.8004)
- National Institutes of Health (UL1TR002243)
- NIHR School for Primary Care Research
- National Center for Advancing Translational Sciences (TR002243)
- Topics
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Epidemiology
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Clinical Trials
- Keywords
- sample size determination, clinical prediction model, model development, prediction accuracy, prognostic model, diagnostic model, small sample size
- MeSH terms
- Clinical Decision-Making, Forecasting, Humans, Models, Theoretical, Sample Size
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review