The DOI matches its Crossref record; 78 of the 80 references it cites were found in a registry or an index.
Calculating the sample size required for developing a clinical prediction model
Open accessAbstract
Clinical prediction models aim to predict outcomes in individuals, to inform diagnosis or prognosis in healthcare. Hundreds of prediction models are published in the medical literature each year, yet many are developed using a dataset that is too small for the total number of participants or outcome events. This leads to inaccurate predictions and consequently incorrect healthcare decisions for some individuals. In this article, the authors provide guidance on how to calculate the sample size required to develop a clinical prediction model.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“In step three, the sample size needed to reduce model overfitting is calculated, assuming an expected shrinkage rate of 10%, with an expected number of variables P to be included in the final predictive model of 30: S is the expected shrinkage factor, and the noise ratio evaluation index R2cs is 0.3; the calculated sample size is 740 cases.”
From Machine learning-driven prediction model for successful weaning of patients from mechanical ventilation in ICU (2026) · cites this paper as Riley [13]
Experimental — accuracy not yet measuredThe check wasn’t confident either way, so it gives no verdict — never disputed.
In the citing paper
“Genomic and molecular data support integration of tumour biology and genetic risk into decision-making, particularly in precision medicine applications [42].”
From Artificial Intelligence in Clinical Decision-Making: A Comprehensive Review of Diagnostic, Prognostic, and Therapeutic Applications, Validation Gaps, Bias, and Deployment Challenges (2026) · cites this paper as [42]
Experimental — accuracy not yet measuredIn the citing paper
“The sample size calculated this way is the minimum for a standard statistical model.”
From Developing clinical prediction models: a step-by-step guide (2024) · cites this paper as [50]
In the citing paper
“Sample size calculations50 indicated a minimum sample of 2082 patients, which is larger than the available sample, raising concerns about possible overfitting issues (step 6).”
From Developing clinical prediction models: a step-by-step guide (2024) · cites this paper as [50]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Regression shrinkage and selection via the lasso
Tibshirani, R. (1996) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/j.2517-6161.1996.tb02080.x · Cited by 51,482
The Elements of Statistical Learning
Hastie, T., Tibshirani, R., & Friedman, J. (2009) · Springer Series in Statistics · doi:10.1007/978-0-387-84858-7 · Cited by 25,970
Regularization and Variable Selection Via the Elastic Net
Zou, H., & Hastie, T. (2005) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/j.1467-9868.2005.00503.x · Cited by 20,957
A simulation study of the number of events per variable in logistic regression analysis
Peduzzi, P., Concato, J., Kemper, E., Holford, T. R., & Feinstein, A. R. (1996) · Journal of Clinical Epidemiology · doi:10.1016/s0895-4356(96)00236-3 · Cited by 9,655
Harrell (2015) · Springer Series in Statistics · doi:10.1007/978-3-319-19425-7 · Cited by 6,878
A note on a general definition of the coefficient of determination
NAGELKERKE, N. J. D. (1991) · Biometrika · doi:10.1093/biomet/78.3.691 · Cited by 5,927
Moons, K. G. M., Altman, D. G., Reitsma, J. B., Ioannidis, J. P. A., Macaskill, P., Steyerberg, E. W., Vickers, A. J., Ransohoff, D. F., & Collins, G. S. (2015) · Annals of Internal Medicine · doi:10.7326/m14-0698 · Cited by 5,147
Self-citationRelaxing the Rule of Ten Events per Variable in Logistic and Cox Regression
Vittinghoff, E., & McCulloch, C. E. (2007) · American Journal of Epidemiology · doi:10.1093/aje/kwk052 · Cited by 4,359
Clinical Prediction Models: A Practical Approach to Development, Validation, and Updating
Steyerberg, E. W. (2019) · Statistics for Biology and Health · doi:10.1007/978-3-030-16399-0 · Cited by 3,281
PROBAST: A Tool to Assess the Risk of Bias and Applicability of Prediction Model Studies
Wolff, R. F., Moons, K. G. M., Riley, R. D., Whiting, P. F., Westwood, M., Collins, G. S., Reitsma, J. B., Kleijnen, J., Mallett, S., & for the PROBAST Group† (2019) · Annals of Internal Medicine · doi:10.7326/m18-1376 · Cited by 2,984
Self-citation
Flagged first, then by citations
2 of the 80 references couldn’t be matched to a known paper.
Research works printed but not linked: 2
- 10.1177/962280218784726 · DOI
- McFadden, 1974, Frontiers in Econometrics New York., 104
Cited by
Sorted by citations
Items 401–500 of 1,003
Zeng, Z., Wang, X., Chen, Y., Zhou, H., Zhu, W., Xiong, X., Tang, J., & Zhao, Q. (2023) · Health and Quality of Life Outcomes · doi:10.1186/s12955-023-02183-1 · Cited by 20
“The sample size was estimated according to the equation n = [m(10− 15)](1+ 20%) , where m is the number of independent variables to be included in the planned linear regression [19, 23].”
MethodHuang, H., Meng, F., Qi, Y., Yan, X., Qi, J., Wu, Y., Lin, Y., Xu, C., & He, F. (2024) · BMC Public Health · doi:10.1186/s12889-024-18548-0 · Cited by 16
“We performed power analysis to evaluate the sample size required for models with time-to-event outcomes according to the literature [18].”
MethodPawuś, D., Porażko, T., & Paszkiel, S. (2024) · IEEE Access · doi:10.1109/access.2024.3413595 · Cited by 21
“The articles [70], [71], [72], [73], [74], [75] collectively delve into the realm of clinical prediction models, emphasizing their pivotal role in contemporary healthcare.”
BackgroundSmith, L. M., Mahoney, D. W., Bamlet, W. R., Yu, F., Liu, S., Goggins, M., Darabi, S., Majumder, S., Wang, Q.-L., Coté, G. A., et al. (2024) · Pancreatology · doi:10.1016/j.pan.2024.10.012 · Cited by 17
Kavak, M. T., & Karadoǧan, S. (2012) · PubMed · Cited by 16
van de Sande, D., van Genderen, M. E., Verhoef, C., van Bommel, J., Gommers, D., van Unen, E., Huiskens, J., & Grünhagen, D. J. (2021) · Surgery · doi:10.1016/j.surg.2021.05.005 · Cited by 15
Zhao, W., Sun, P., Li, W., & Shang, L. (2025) · Infection and Drug Resistance · doi:10.2147/idr.s459830 · Cited by 16
“34 According to EPV guidelines, the minimum required number of events (the lesser of positive or negative cases) must be at least ten times the number of independent variables.”
Adriaenssens, N., Scholtes, B., Bruyndonckx, R., Verbakel, J. Y. J., Sutter, A. D., Heytens, S., den Bruel, A. V., Desombere, I., Damme, P. V., Goossens, H., et al. (2022) · BMJ Open · doi:10.1136/bmjopen-2021-054688 · Cited by 14
Yonel, Z., Kocher, T., Chapple, I., Dietrich, T., Völzke, H., Nauck, M. A., Collins, P. G. S., Gray, L. J., & Holtfreter, B. (2022) · Journal of Dental Research · doi:10.1177/00220345221129807 · Cited by 17
“A recent series of papers by Riley et al. (2018, 2020) highlights the importance of adequate sample sizes when developing models and outlines a novel method to calculate the required sample size and number of events per sample.”
BackgroundMethodInfluentialDankert, A., Neumann-Schirmbeck, B., Dohrmann, T., Greiwe, G., Plümer, L., Löser, B., Sehner, S., Zöllner, C., & Petzoldt, M. (2022) · Anesthesia & Analgesia · doi:10.1213/ane.0000000000006235 · Cited by 16
Dai, T., Wu, D., Tang, J., Liu, Z., & Zhang, M. (2023) · Journal of Gastrointestinal Oncology · doi:10.21037/jgo-22-1307 · Cited by 15
Yu, L., Zhang, X., Jiang, F., Zheng, H., Li, X., Ye, X., Gong, X., & Xing, H. (2023) · Journal of Affective Disorders · doi:10.1016/j.jad.2023.07.058 · Cited by 15
He, W., Fu, X., & Chen, S. (2023) · Journal of Translational Medicine · doi:10.1186/s12967-023-04487-8 · Cited by 20
“2 We use a sample size calculated as n = 1.96 φ is the expected outcome ratio ( φ = 0.29), δ is the set margin of error ( δ = 0.05) [22].”
MethodInfluentialZapf, M. A. C., Fabbri, D., Andrews, J. C., Li, G., Freundlich, R. E., Al-Droubi, S. S., & Wanderer, J. P. (2023) · Journal of Clinical Anesthesia · doi:10.1016/j.jclinane.2023.111272 · Cited by 15
Odeyemi, Y., Lal, A., Barreto, E. F., LeMahieu, A., Yadav, H., Gajic, O., & Schulte, P. J. (2023) · Biomolecules and Biomedicine · doi:10.17305/bb.2023.9754 · Cited by 15
“2% and the inclusion of 32 predictor variables in the model (16).”
BackgroundUsing machine learning to predict outcomes following suprainguinal bypass
Li, B., Eisenberg, N., Beaton, D., Lee, D. S., Aljabri, B. A., Wijeysundera, D. N., Rotstein, O. D., de Mestral, C., Mamdani, M., Roche‐Nagle, G., et al. (2023) · Journal of Vascular Surgery · doi:10.1016/j.jvs.2023.09.037 · Cited by 16
Nemeth, B., Smeets, M. J. R., Pedersen, A. B., Kristiansen, E. B., Nelissen, R. G. H. H., Whyte, M., Roberts, L. N., de Lusignan, S., le Cessie, S., Cannegieter, S. C., et al. (2023) · Journal of Thrombosis and Haemostasis · doi:10.1016/j.jtha.2023.09.033 · Cited by 15
Random forest algorithm for predicting postoperative delirium in older patients
Sheng, W., Tang, X., Hu, X., Liu, P., Liu, L., Miao, H., Wang, D., & Li, T. (2024) · Frontiers in Neurology · doi:10.3389/fneur.2023.1325941 · Cited by 15
Developing a model for the implementation of designing out waste in construction
Hassan, N. M., & Alashwal, A. M. (2024) · Architectural Engineering and Design Management · doi:10.1080/17452007.2024.2311670 · Cited by 15
“Given this population size, a sample of approximately 196 respondents was determined using relevant sample size equations by considering a 95% confidence level and 0.07 confidence interval (Moinester & Gottfried, 2014; Riley et al., 2020).”
InfluentialWang, L., Wu, Y., Deng, L., Tian, X., & Ma, J. (2024) · BMC Anesthesiology · doi:10.1186/s12871-024-02598-3 · Cited by 15
“By applying Riley et al.’s method, the estimated minimum sample size for developing the new model was determined to be 874, with 202 positive events of postoperative ICU admission [18].”
MethodProteome-scale prediction of molecular mechanisms underlying dominant genetic diseases
(2024) · doi:10.1371/journal.pone.0307312 · Cited by 15
“We sought to rationally select and design surrogate features that make up for the loss of protein structural and functional information.”
BackgroundLi, E., Ai, F., Tian, Q., Yang, H., Tang, P., & Guo, B. (2025) · BMC Psychiatry · doi:10.1186/s12888-025-06657-y · Cited by 16
“A total of 945 eligible participants were included, meeting the minimum sample size required for machine learning models [19, 20].”
Ajnakina, O., Agbedjro, D., McCammon, R. J., Faul, J. D., Murray, R. M., Stahl, D. R., & Steptoe, A. (2021) · BMC Medical Research Methodology · doi:10.1186/s12874-020-01204-7 · Cited by 15
Farhat, H., Laughton, J., Gangaram, P., Aifa, K. E., Khenissi, M. C., Zaghouani, O., Khadhraoui, M., Gargouri, I., & Alinier, G. (2022) · Global Security Health Science and Policy · doi:10.1080/23779497.2022.2069142 · Cited by 15
“Recent research suggest that a maximum error tolerance of α = 0.1 would also be tolerable (Perneger et al., 2015; Riley et al., 2020).”
BackgroundInfluentialYang, L., Lu, G., Diao, H., Zhang, Y., Wang, Z., Liu, X., Ma, Q., Yu, H., & Li, Y. (2025) · Microbiology Spectrum · doi:10.1128/spectrum.02460-24 · Cited by 17
“The effective sample size in prediction research (development and validation) was determined by the number of outcome events (22, 23), which was defined to have at least 10 outcome events per variable to ensure accuracy (24).”
Kooper, C. C., van der Zee, C. W., Oosterlaan, J., Plötz, F. B., & Königs, M. (2022) · Journal of Neurotrauma · doi:10.1089/neu.2022.0369 · Cited by 14
“51 Moreover, most studies did not report all candidate predictors considered in the models and only reported on the significant predictors that were included in the final models.”
Myhrvold, B. L., Axén, I., Leach, M., Sundberg, T., & Gausel, A. M. (2023) · BMC Health Services Research · doi:10.1186/s12913-023-09354-2 · Cited by 14
“to calculate for the required sample size for multivariable linear regression modeling [16].”
MethodTian, Y., Zhang, Y., He, J., Chen, L., Hao, P., Li, T., Peng, L., Chong, W., Hai, Y., You, C., et al. (2023) · European Stroke Journal · doi:10.1177/23969873231184667 · Cited by 14
Guo, H., Wang, T., Li, C., Yu, J., Zhu, R., Wang, M., Zhu, Y., & Wang, J. (2023) · Knee Surgery Sports Traumatology Arthroscopy · doi:10.1007/s00167-023-07488-8 · Cited by 14
“According to the requirements for developing a clinical prediction model [33], under the condition of target error ≤ 0.05, the required sample size was calculated to be at least 70 cases, so our sample size (547 cases) was apparently sufficient.”
InfluentialBleeding Scoring Systems in Neonates: A Systematic Review
Sokou, R., Parastatidou, S., Konstantinidi, A., Tsantes, A. G., Iacovidou, N. M., Piovani, D., Bonovas, S., & Tsantes, A. Ε. (2023) · Seminars in Thrombosis and Hemostasis · doi:10.1055/s-0043-1777070 · Cited by 14
“77 Future prediction modelling studies on bleeding risk in the neonatal fi eld should adhere to such guiding principle to Prediction Models for Bleeding Risk in Neonates Sokou et al. 633 T h i s do c u m en t w a s do w n l oaded f o r pe r s ona l u s e on l y .”
Guo, H., Song, B., Zhou, R., Yu, J., Chen, P., Yang, B., Pan, N., Li, C., Zhu, Y., & Wang, J. (2023) · Clinical Interventions in Aging · doi:10.2147/cia.s436816 · Cited by 14
Li, Q., Lv, H., Chen, Y., Shen, J., Shi, J., Zhou, C., & Yan, F. (2024) · International Journal of Medical Informatics · doi:10.1016/j.ijmedinf.2024.105343 · Cited by 15
Skelton, E., Cromb, D., Smith, A. J., Harrison, G., Rutherford, M., Malamateniou, C., & Ayers, S. (2024) · BMC Pregnancy and Childbirth · doi:10.1186/s12884-024-06469-0 · Cited by 16
“Although the pre-determined target sample size of n = 70 was achieved, it is likely that a greater number of participants would provide further power in the quantitative findings [82].”
BackgroundWiens, M. O., Nguyen, V., Bone, J. N., Kumbakumba, E., Businge, S., Tagoola, A., Sherine, S. O., Byaruhanga, E., Ssemwanga, E., Barigye, C., et al. (2024) · PLOS Global Public Health · doi:10.1371/journal.pgph.0003050 · Cited by 14
“For the present analysis, we determined the sample size required to develop a prediction model based on criteria proposed by Riley et al., 2020.25 For binary outcomes, three criteria are recommended based on: 1) reducing overfitting (caused by small sample sizes or too many candidate predictors…”
MethodInfluentialBelsti, Y., Moran, L., Goldstein, R. F., Mousa, A., Cooray, S. D., Baker, S., Gupta, Y., Patel, A. A., Tandon, N., Ajanthan, S., et al. (2024) · Clinical Nutrition · doi:10.1016/j.clnu.2024.06.006 · Cited by 16
Ruan, L., Xu, S., Qin, Y., Tang, H., Li, X., Gaoliang, Y., Wang, D., Tang, C., & Qiao, Y. (2024) · Journal of Inflammation Research · doi:10.2147/jir.s454904 · Cited by 14
Xia, F., Li, Q., Xu, L., Chen, X., Li, G., Li, L., Cheng, Z., Zhang, J., Deng, C., Li, J., et al. (2024) · Scientific Reports · doi:10.1038/s41598-024-66222-7 · Cited by 14
Rempakos, A., Alexandrou, M., Mutlu, D., Kalyanasundaram, A., Ybarra, L. F., Bagur, R., Choi, J. W., Poommipanit, P. B., Khatri, J. J., Young, L. D., et al. (2024) · JACC: Cardiovascular Interventions · doi:10.1016/j.jcin.2024.04.043 · Cited by 14
He, W., Chen, S., Fu, X., Xu, L.-C., Xie, J., & Wan, J. (2024) · Biomolecules and Biomedicine · doi:10.17305/bb.2024.10725 · Cited by 15
“The R package “pmsampsize,” version 1.1.2, was employed to calculate the required minimum sample size for training [21].”
InfluentialDai, T., Bao, M., Zhang, M., Wang, Z., Tang, J., & Liu, Z. (2024) · BMC Medical Informatics and Decision Making · doi:10.1186/s12911-024-02627-8 · Cited by 14
“According to literature on PSH incidence [4], a minimum of 334 patients must be included to adhere to the Events Per Variable (EPV) principle for predictive model studies [13].”
BackgroundZhong, F., Lin, X., Zheng, C., Tang, S., Yin, Y., Wang, K., Dai, Z., Hu, Z., & Peng, Z. (2024) · BMC Infectious Diseases · doi:10.1186/s12879-024-09898-6 · Cited by 14
“In constructing the model, model development was performed with all patient data (avoiding data segmentation), and internal validation of model discrimination and calibration was performed using bootstrap with 1000 resamples [17].”
MethodRedfors, B., Byttner, A., Bengtsson, D., Watson, P., Lannemyr, L., Lundgren, P., Gäbel, J., Rawshani, A., & Henningsson, A. (2024) · Journal of Cardiothoracic and Vascular Anesthesia · doi:10.1053/j.jvca.2024.09.009 · Cited by 14
Asadi, F., Rahimi, M., Ramezanghorbani, N., & Almasi, S. (2025) · Cancer Reports · doi:10.1002/cnr2.70138 · Cited by 16
Huang, R., Meng, T., Zha, Q., Cheng, K., Zhou, X., Zheng, J., Zhang, D., & Liu, R. (2021) · Critical Care · doi:10.1186/s13054-021-03661-y · Cited by 14
“However, most of these laboratory indicators did not meet the requirements for inclusion in multivariate analysis [31, 32].”
BackgroundRelationship between the difference in electric pulp test values and the diagnostic type of pulpitis
Sui, H., Lv, Y., Xiao, M., Zhou, L., Qiao, F., Zheng, J., Sun, C., Fu, J., Chen, Y., Liu, Y., et al. (2021) · BMC Oral Health · doi:10.1186/s12903-021-01696-9 · Cited by 14
López, P. B., Doménech‐García, V., Palsson, T. S., Christensen, S. W., Silva, P. D. B., Langella, F., Berjano, P. L., Jensen, P. S., Riis, A., Baroncini, A., et al. (2021) · BMJ Open · doi:10.1136/bmjopen-2021-052602 · Cited by 14
Lee, R. J., Wysocki, O., Zhou, C., Shotton, R., Tivey, A., Lever, L., Woodcock, J., Albiges, L. K., Angelakas, A., Arnold, D., et al. (2022) · JCO Clinical Cancer Informatics · doi:10.1200/cci.21.00177 · Cited by 13
“Median (range) 70 (5-99) 63 (5-86) 71 (34-95) 68 (19-93) 80.”
Levy, T. J., Coppa, K., Cang, J., Barnaby, D. P., Paradis, M. D., Cohen, S. L., Makhnevich, A., van Klaveren, D., Kent, D. M., Davidson, K. W., et al. (2022) · Nature Communications · doi:10.1038/s41467-022-34646-2 · Cited by 14
“…balance for this tradeoff, our approach was to determine the size of the window based on a combination of formulas that estimate the proper sample size for LR models 52 and to perform a hyperparameter optimization that estimates the proper sample size for the custom GLM model (Supplementary Fig.”
MethodWhat if we wait? Using synthetic waiting lists to estimate treatment effects in routine outcome data
Kaiser, T., Brakemeier, E., & Herzog, P. (2023) · Psychotherapy Research · doi:10.1080/10503307.2023.2182241 · Cited by 16
“To ensure that our sample size requirements for multivariable prediction models are met, we used the formulas provided by Riley et al. (2020).”
MethodHall, S., Birks, J., Anderson, I., Bacon, A., Brennan, P. M., Bennett, D., Chavredakis, E., Critchley, G. R., Dow, G. R., Downer, J., et al. (2023) · BMJ Open · doi:10.1136/bmjopen-2022-070504 · Cited by 14
Quantitative Systems Pharmacology and Machine Learning: A Match Made in Heaven or Hell?
Tindall, M. J., Cucurull-Sanchez, L., Mistry, H. B., & Yates, J. W. T. (2023) · Journal of Pharmacology and Experimental Therapeutics · doi:10.1124/jpet.122.001551 · Cited by 14
Dong, F., Ren, X., Huang, K., Wang, Y., Jiao, J., & Yang, T. (2021) · Frontiers in Medicine · doi:10.3389/fmed.2021.630870 · Cited by 14
“A large sample size can develop a robust model (11).”
BackgroundMethodLiu, Y., Xu, Y., Guo, L., Chen, Z., Xia, X., Chen, F., Tang, L., Jiang, H., & Xie, C. (2025) · BMC Medical Informatics and Decision Making · doi:10.1186/s12911-025-03082-9 · Cited by 19
“To develop a stable model, we determined the minimum sample size using the methodology outlined by Riley et al. [49].”
Schiborn, C., Kuhn, T. E., Mühlenbruch, K., Kuxhaus, O., Weikert, C., Fritsche, A., Kaaks, R., & Schulze, M. B. (2021) · Scientific Reports · doi:10.1038/s41598-021-99103-4 · Cited by 13
Myhrvold, B. L., Kongsted, A., Irgens, P. M. S., Robinson, H. S., & Vøllestad, N. K. (2022) · BMC Musculoskeletal Disorders · doi:10.1186/s12891-022-05558-5 · Cited by 13
“to calculate for the efficient sample size for multivariable linear regression modeling [44].”
MethodHeiderscheit, B. C., Blemker, S. S., Opar, D. A., Stiffler-Joachim, M. R., Bedi, A., Hart, J. M., Mortensen, B., Kliethermes, S., The HAMIR Study Group, Baer, G. S., et al. (2022) · BMC Sports Science Medicine and Rehabilitation · doi:10.1186/s13102-022-00520-3 · Cited by 13
Virdee, P. S., Patnick, J., Watkinson, P., Holt, T., & Birks, J. (2022) · Cancers · doi:10.3390/cancers14194779 · Cited by 17
“The required sample size was calculated using the pmsampsize package in Stata software [22].”
MethodDeep learning in acute vertigo diagnosis
Rastall, D. P. W., & Green, K. E. (2022) · Journal of the Neurological Sciences · doi:10.1016/j.jns.2022.120454 · Cited by 13
How to Develop and Validate Prediction Models for Orthopedic Outcomes
Zaniletti, I., Larson, D. R., Lewallen, D. G., Berry, D. J., & Kremers, H. M. (2022) · The Journal of Arthroplasty · doi:10.1016/j.arth.2022.12.032 · Cited by 13
Built environment factors predictive of early rapid lung function decline in cystic fibrosis
Gecili, E., Brokamp, C., Rasnick, E., Afonso, P. M., Andrinopoulou, E., Dexheimer, J. W., Clancy, J. P., Keogh, R. H., Ni, Y., Palipana, A., et al. (2023) · Pediatric Pulmonology · doi:10.1002/ppul.26352 · Cited by 13
Cau, R., Muscogiuri, G., Pisu, F., Gatti, M., Velthuis, B. K., Loewe, C., Cademartiri, F., Pontone, G., Montisci, R., Guglielmo, M., et al. (2023) · Journal of Thoracic Imaging · doi:10.1097/rti.0000000000000709 · Cited by 14
Tian, J., Cui, R., Song, H., Zhao, Y., & Zhou, T. (2023) · International Urology and Nephrology · doi:10.1007/s11255-023-03646-6 · Cited by 13
“It is recommended to use all data for model development to avoid data splitting (e.g., into model derivation and validation samples), and to apply resampling methods (e.g., bootstrapping) for internal validation [50].”
Rivera-Villegas, H., Martínez-Guerra, B. A., Garcia-Couturier, R., Xancal-Salvador, L. F., Esteban-Kenel, V., Jaimes-Aquino, R. A., Mendoza-Rojas, M., Sánchez, A. C., Méndez-Ramos, S., Alonso-Montoya, J. E., et al. (2023) · Antibiotics · doi:10.3390/antibiotics12071130 · Cited by 13
“Considering a primary outcome probability of 50% [2,3], a mean absolute percentage error of 10%, and the identification of up to 10 potential predictors, we calculated a sample size of at least 165 patients [14].”
MethodByrne, J. F., Mongan, D., Murphy, J., Healy, C., Föcking, M., Cannon, M. D., & Cotter, D. (2023) · Translational Psychiatry · doi:10.1038/s41398-023-02623-y · Cited by 14
“This systematic review was prospectively registered with PROS-PERO, CRD42022302047.”
MethodYin, C., jiang, J., He, M., Zhong, K., Tang, S., Deng, L., & Wang, Y. (2024) · International Journal of Surgery · doi:10.1097/js9.0000000000001406 · Cited by 13
Jin, Y., Li, W., Wu, Y., Wang, Q., Xiang, Z., Long, Z., Liang, H., Zou, J., Zhu, Z., & Dai, X. (2024) · International Journal of Surgery · doi:10.1097/js9.0000000000001764 · Cited by 13
“In this research, the sample size of the binary outcome prediction model was calculated according to the following formula [31] : ).”
Guo, H., Cao, J., He, S., Wei, M., Meng, D., Yu, I., Wang, Z., Chang, X., Yang, G., & Wang, Z. (2024) · JMIR Aging · doi:10.2196/58175 · Cited by 19
“This study’s sample size offers valuable insights, yet further validation in larger cohorts is recommended in alignment with the sample size guidelines proposed [82] for clinical prediction models prior to clinical implementation.”
Predicting preterm birth using electronic medical records from multiple prenatal visits
Huang, C., Long, X., van der Ven, M., Kaptein, M. C., Oei, S. G., & van den Heuvel, E. R. (2024) · BMC Pregnancy and Childbirth · doi:10.1186/s12884-024-07049-y · Cited by 15
Farsetti, D., Barbieri, M., Magni, E., Zamagni, G., Monasta, L., Maso, G., Vasapollo, B., Pometti, F., Ferrazzi, E. M., Lees, C. C., et al. (2025) · American Journal of Obstetrics and Gynecology · doi:10.1016/j.ajog.2025.01.001 · Cited by 13
AI ‐Based Platelet‐Independent Noninvasive Test for Liver Fibrosis in MASLD Patients
Wakabayashi, S., Kimura, T., Tamaki, N., Iwadare, T., Okumura, T., Kobayashi, H., Yamashita, Y., Tanaka, N., Kurosaki, M., & Umemura, T. (2025) · JGH Open · doi:10.1002/jgh3.70150 · Cited by 13
Lin, F., Zhang, Z., Wang, J., Liang, C., Xu, J., Zeng, X., Zeng, Q., Chen, H., Zhuang, J., Ma, Y., et al. (2025) · EClinicalMedicine · doi:10.1016/j.eclinm.2025.103166 · Cited by 13
“The sample size was calculated beforehand using the online software (https:// mvansmeden.shinyapps.io/BeyondEPV/) by Riley RD et al. 30 (Parameter: number of candidate predictors = 10; events fraction = 0 ⋅ 34; criterion value rMPSE = 0 ⋅ 05), resulting in at least 870 participants and 29 ⋅ 7…”
Ninety-Day Mortality: Redefining the Perioperative Period After Lung Resection
Taylor, M., Grant, S. W., West, D., Shackcloth, M. J., Woolley, S. M., Naidu, B. V., & Shah, R. D. (2020) · Clinical Lung Cancer · doi:10.1016/j.cllc.2020.12.011 · Cited by 13
Lo, Y. H., & Siu, A. Y. C. (2021) · Journal of Emergency Medicine · doi:10.1016/j.jemermed.2021.07.058 · Cited by 11
Chandna, A., Mahajan, R., Gautam, P., Mwandigha, L., Gunasekaran, K., Bhusan, D., Cheung, A. T. L., Day, N., Dittrich, S., Dondorp, A. M., et al. (2022) · Clinical Infectious Diseases · doi:10.1093/cid/ciac224 · Cited by 15
Chen, J., Li, L., Chen, T., Yang, X., Ru, H., Li, X., Yang, X., Xie, Q., & Xu, L. (2022) · BMC Infectious Diseases · doi:10.1186/s12879-022-07368-5 · Cited by 13
“Sample size calculation The calculation of study sample size followed the method of developing a clinical prediction model [27, 28].”
MethodShanley, E., Thigpen, C., Collins, G., Arden, N., Noonan, T., Wyland, D. J., Kissenberth, M., & Bullock, G. (2022) · Journal of Orthopaedic and Sports Physical Therapy · doi:10.2519/jospt.2022.11072 · Cited by 12
Veerbeek, J. M., Pohl, J., Luft, A. R., & Held, J. P. O. (2022) · PLoS ONE · doi:10.1371/journal.pone.0272777 · Cited by 13
Zysman, M., Asselineau, J., Saut, O., Frison, É., Oranger, M., Maurac, A., Charriot, J., Achkir, R., Regueme, S. C., Klein, É., et al. (2023) · European Radiology · doi:10.1007/s00330-023-09759-x · Cited by 13
“A sample size of the development cohort was calculated using Riley and colleagues’ approach [29].”
Gao, S., Albu, E., Tuand, K., Cossey, V., Rademakers, F. E., CALSTER, B. V., & Wynants, L. (2023) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2023.07.019 · Cited by 14
Wu, X., Liu, Y., Che, J., Cheng, N., Dong, W., Liu, H., & Dong, X. (2025) · Journal of Alzheimer s Disease · doi:10.1177/13872877241305961 · Cited by 13
“To determine the optimal number of features, we adopted the method proposed by Riley et al., 45 using the pmsampsize package in R to estimate sample size and feature quantity.”
Endebu, T., Taye, G., & Deressa, W. A. (2025) · BMC Medical Informatics and Decision Making · doi:10.1186/s12911-025-03030-7 · Cited by 12
“Key among these are estimating the overall outcome proportion with adequate precision, targeting a small mean absolute prediction error, and establishing a shrinkage factor to minimize optimism in the apparent R² Nagelkerke [35].”
InfluentialStatistical Models in Clinical Studies
Matsui, S., Le‐Rademacher, J. G., & Mandrekar, S. J. (2021) · Journal of Thoracic Oncology · doi:10.1016/j.jtho.2021.02.021 · Cited by 12
van de Kuit, A., Oosterhoff, J. H. F., Dijkstra, H., Sprague, S., Bzovsky, S., Bhandari, M., Swiontkowski, M. F., Schemitsch, E. H., IJpma, F. F. A., Poolman, R. W., et al. (2022) · Clinical Orthopaedics and Related Research · doi:10.1097/corr.0000000000002283 · Cited by 12
Shamsutdinova, D. D., Das‐Munshi, J., Ashworth, M. J., Roberts, A., & Stahl, D. R. (2023) · International Journal of Medical Informatics · doi:10.1016/j.ijmedinf.2023.105019 · Cited by 13
Mases, A., de Heredia, S. B., Gallart, L., Román, L., Bosch, L., Núñez, M., Rueda, M., Recasens, L., & Sabaté, S. (2023) · Anesthesia & Analgesia · doi:10.1213/ane.0000000000006469 · Cited by 12
Dhana, A. V., Gupta, R. K., Hamada, Y., Kengne, A. P., Kerkhoff, A. D., Yoon, C., Cattamanchi, A., Reeve, B. W. P., Theron, G., Ndlangalavu, G., et al. (2023) · European Respiratory Review · doi:10.1183/16000617.0021-2023 · Cited by 12
“The derived population was considered sufficient [26] and sample size calculations are provided in the appendix.”
BackgroundLedger, A., Ceusters, J., Valentin, L., Testa, A. C., Holsbeke, C. V., Franchi, D., Bourne, T., Froyman, W., Timmerman, D., & CALSTER, B. V. (2023) · BMC Medical Research Methodology · doi:10.1186/s12874-023-02103-3 · Cited by 12
“The recently established guidance for minimum sample size to develop a regression-based prediction model is a crucial step forward [55, 56].”
BackgroundJiang, F., Li, X., Liu, L., Xie, Z., Wu, X., & Wang, Y. (2024) · BioScience Trends · doi:10.5582/bst.2023.01327 · Cited by 12
Risk-stratified monitoring for sulfasalazine toxicity: prognostic model development and validation
Abhishek, A., Grainge, M. J., Card, T. R., Williams, H. C., Taal, M. W., Aithal, G. P., Fox, C. P., Mallen, C. D., Stevenson, M. D., Nakafero, G., et al. (2024) · RMD Open · doi:10.1136/rmdopen-2023-003980 · Cited by 12
“The data analysis was undertaken using the Stata command ‘mi estimate’ in a combined dataset that included all imputations.”
MethodKang, M., Zhu, C., Lai, M., Weng, J., Zhuang, Y., He, H., Qiu, Y., Wu, Y., Qi, Z., Zhang, W., et al. (2024) · The Journal of Clinical Endocrinology & Metabolism · doi:10.1210/clinem/dgae475 · Cited by 12
Li, J., Li, G., Liu, Z., Yang, X., & Yang, Q. (2024) · American Journal of Infection Control · doi:10.1016/j.ajic.2024.07.006 · Cited by 13
Yu, Q., & Yu, H. (2025) · Journal of Clinical Nursing · doi:10.1111/jocn.17508 · Cited by 12
Kim, J. K., Yadav, P., Bitcon, C., Keefe, D. T., Comella, A., Taghavi, K., Kattini, R., Davis-Dao, C. A., Khoury, A. E., Khondker, A., et al. (2025) · Pediatric Nephrology · doi:10.1007/s00467-025-06701-9 · Cited by 12
“…(accounting for the fact that few ‘strong’ predictors were found in the past such as nadir creatinine), a sample size of 116 would be required to satisfy the criteria outlined by Riley et al., and this would provide sufficient power to determine statistical significance of the prognostic tool [10].”
Utsumi, T., Ishitsuka, N., Noro, T., Suzuki, Y., Iijima, S., Sugizaki, Y., Somoto, T., Oka, R., Endo, T., Kamiya, N., et al. (2025) · International Journal of Urology · doi:10.1111/iju.70075 · Cited by 12
“Even with a fixed dataset, sample size calculations can determine whether the available sample size is sufficient and how many predictors can be included before overfitting becomes a concern [44].”
Pathak, P., Panday, S. B., & Ahn, J. (2021) · Clinical Nutrition · doi:10.1016/j.clnu.2021.11.027 · Cited by 12
Harrison, T. G., Hemmelgarn, B. R., James, M. T., Sawhney, S., Lam, N. N., Ruzycki, S. M., Wilson, T., & Ronksley, P. E. (2022) · CJC Open · doi:10.1016/j.cjco.2022.07.008 · Cited by 12
Gehringer, C. K., Martin, G. P., Hyrich, K. L., Verstappen, S. M. M., & Sergeant, J. C. (2022) · Seminars in Arthritis and Rheumatism · doi:10.1016/j.semarthrit.2022.152076 · Cited by 13
Chiang, T.-Y., Wang, Y.-K., Huang, W., Huang, S., & Chu, Y. (2022) · Frontiers in Medicine · doi:10.3389/fmed.2022.943596 · Cited by 12
“Additionally, to avoid overfitting the multivariable regression model, we included only a limited number of independent variables that we thought were clinically meaningful (32).”
MethodVirdee, P. S., Bankhead, C. R., Koshiaris, C., Wright, C., Oke, J. L., Withrow, D. R., Swain, S., Collins, K. K., Chammas, L., Tamm, A., et al. (2023) · Diagnostic and Prognostic Research · doi:10.1186/s41512-022-00138-6 · Cited by 12
Wu, X., Tang, F., Li, H., Chen, C., Zhang, H., Liu, X., Lai, H., Li, Q., Deng, L., & Ye, Z. (2023) · Journal of Psychosomatic Research · doi:10.1016/j.jpsychores.2023.111385 · Cited by 12
Metadata
- Authors
- Riley, R. D., Ensor, J., Snell, K. I. E., Harrell, F. E., Martin, G. P., Reitsma, J. B., Moons, K. G. M., Collins, G., & van Smeden, M.
- Affiliations
- Keele University (GB)
- Vanderbilt University (US)
- Manchester Academic Health Science Centre (GB)
- University Medical Center Utrecht (NL)
- Corresponding author
- Riley, Richard D
- ORCID
- Venue
- BMJ
- Publisher
- BMJ
- Published
- 2020-03-18
- Volume
- 368
- Pages
- m441
- ISSN
- 1756-1833 (ISSN-L)
- OpenAlex
- W3012413426
- Cited by
- 2,917
- References
- 80
- Citations by year
- 2026: 863 · 2025: 606 · 2024: 531 · 2023: 402 · 2022: 275 · 2021: 174 · 2020: 58
- FWCI
- 160.59
- Type
- journal article
- Language
- en
- Open access
- bronze
- doi.org
- Licence
- www.bmj.com/company/legal-information/terms-conditions/legal-information/tdm-licencepolicy (tdm)
- Locations
- BMJ (publishedVersion)
- PubMed (publishedVersion)
- Research Explorer (The University of Manchester) (publishedVersion, other-oa)
- Keele Research Repository (Keele University) (submittedVersion, cc-by)
- Utrecht University Repository (Utrecht University) (submittedVersion, other-oa)
- HighWire Press Open Archive (submittedVersion)
- University of Birmingham Research Portal (University of Birmingham) (submittedVersion)
- Funders
- Georgia Clinical and Translational Science Alliance (UL1TR002243)
- Cancer Research UK (27294)
- National Institute for Health and Care Research (SPCR-142)
- Department of Health and Social Care
- Nederlandse Organisatie voor Wetenschappelijk Onderzoek (91208004, 918.10.615, project 9120.8004)
- National Institutes of Health (UL1TR002243)
- NIHR School for Primary Care Research
- National Center for Advancing Translational Sciences (TR002243)
- Topics
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Epidemiology
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Clinical Trials
- Keywords
- sample size determination, clinical prediction model, model development, prediction accuracy, prognostic model, diagnostic model, small sample size
- MeSH terms
- Clinical Decision-Making, Forecasting, Humans, Models, Theoretical, Sample Size
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review