Cited by 2,917Statistical Methods in EpidemiologyMeta-analysis and systematic reviewsStatistical Methods in Clinical Trials

The DOI matches its Crossref record; 78 of the 80 references it cites were found in a registry or an index.

Calculating the sample size required for developing a clinical prediction model

Open access

Riley, Richard D, Ensor, Joie, Snell, Kym I E, Harrell, Frank E, Martin, Glen P, Reitsma, Johannes B, Moons, Karel G M, Collins, Gary, van Smeden, Maarten·2020·BMJ

DOI 10.1136/bmj.m441

Open this paper in CoChat — free account Open PDF

Abstract

Clinical prediction models aim to predict outcomes in individuals, to inform diagnosis or prognosis in healthcare. Hundreds of prediction models are published in the medical literature each year, yet many are developed using a dataset that is too small for the total number of participants or outcome events. This leads to inaccurate predictions and consequently incorrect healthcare decisions for some individuals. In this article, the authors provide guidance on how to calculate the sample size required to develop a clinical prediction model.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

Integrity signals

Registry

Crossref · 10.1136/bmj.m441

Cites

78 of 78 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Regression shrinkage and selection via the lasso
Tibshirani, R. (1996) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/j.2517-6161.1996.tb02080.x · Cited by 51,482
The Elements of Statistical Learning
Hastie, T., Tibshirani, R., & Friedman, J. (2009) · Springer Series in Statistics · doi:10.1007/978-0-387-84858-7 · Cited by 25,970
Regularization and Variable Selection Via the Elastic Net
Zou, H., & Hastie, T. (2005) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/j.1467-9868.2005.00503.x · Cited by 20,957
A simulation study of the number of events per variable in logistic regression analysis
Peduzzi, P., Concato, J., Kemper, E., Holford, T. R., & Feinstein, A. R. (1996) · Journal of Clinical Epidemiology · doi:10.1016/s0895-4356(96)00236-3 · Cited by 9,655
Regression Modeling Strategies: With Applications to Linear Models, Logistic and Ordinal Regression, and Survival Analysis
Harrell (2015) · Springer Series in Statistics · doi:10.1007/978-3-319-19425-7 · Cited by 6,878
A note on a general definition of the coefficient of determination
NAGELKERKE, N. J. D. (1991) · Biometrika · doi:10.1093/biomet/78.3.691 · Cited by 5,927
Transparent Reporting of a multivariable prediction model for Individual Prognosis Or Diagnosis (TRIPOD): Explanation and Elaboration
Moons, K. G. M., Altman, D. G., Reitsma, J. B., Ioannidis, J. P. A., Macaskill, P., Steyerberg, E. W., Vickers, A. J., Ransohoff, D. F., & Collins, G. S. (2015) · Annals of Internal Medicine · doi:10.7326/m14-0698 · Cited by 5,147
Self-citation
Relaxing the Rule of Ten Events per Variable in Logistic and Cox Regression
Vittinghoff, E., & McCulloch, C. E. (2007) · American Journal of Epidemiology · doi:10.1093/aje/kwk052 · Cited by 4,359
Clinical Prediction Models: A Practical Approach to Development, Validation, and Updating
Steyerberg, E. W. (2019) · Statistics for Biology and Health · doi:10.1007/978-3-030-16399-0 · Cited by 3,281
PROBAST: A Tool to Assess the Risk of Bias and Applicability of Prediction Model Studies
Wolff, R. F., Moons, K. G. M., Riley, R. D., Whiting, P. F., Westwood, M., Collins, G. S., Reitsma, J. B., Kleijnen, J., Mallett, S., & for the PROBAST Group† (2019) · Annals of Internal Medicine · doi:10.7326/m18-1376 · Cited by 2,984
Self-citation

Flagged first, then by citations

Show all 80 references

2 of the 80 references couldn’t be matched to a known paper.

Research works printed but not linked: 2

10.1177/962280218784726 · DOI
McFadden, 1974, Frontiers in Econometrics New York., 104

Sorted by citations

Items 901–1,000 of 1,003

Predicting outcomes following endovascular aortoiliac revascularization using machine learning
Li, B., Aljabri, B. A., Beaton, D., Al‐Omran, L., Hussain, M. A., Lee, D. S., Wijeysundera, D. N., Rotstein, O. D., de Mestral, C., Mamdani, M., et al. (2025) · npj Digital Medicine · doi:10.1038/s41746-025-01865-y · Cited by 5
A comprehensive guide to selecting the right modeling strategy for explanatory and predictive data analysis
Niazy, M., Murphy, H. M., Nadeem, K., & Ricker, N. (2025) · Canadian Journal of Microbiology · doi:10.1139/cjm-2025-0038 · Cited by 5
Development and validation of a risk identification model for frailty in stroke survivors: new evidence from CHARLS
Wang, J., Kwan, R. Y. C., Suen, L. K. P., Lam, S. C., & Liu, N. (2025) · BMC Public Health · doi:10.1186/s12889-025-24198-7 · Cited by 5
“The sample size was calculated for this study using the sample size estimation method proposed by Richard D Riley and his team [26] for developing clinical prediction models.”
Machine learning-based prediction model for intraoperative hypothermia risk in thoracoscopic lobectomy patients: A SHAP analysis
Chen, R., Ma, X., Liu, M., Deng, X., Wei, F., Gui, L., & Luo, S. (2025) · Medicine · doi:10.1097/md.0000000000044202 · Cited by 6
“[20] Accordingly, the required sample size for this study was calculated to range from 338 to 507 participants.”
Development and validation of an explainable machine learning model for predicting sepsis risk following flexible ureteroscopic lithotripsy
Li, R., Zhang, B., Zeng, L., Mo, J., Zhang, J., & Bi, S. (2025) · Urolithiasis · doi:10.1007/s00240-025-01856-4 · Cited by 5
The DNA Methylation Marker ZNF671 Has Prognostic Value for Progressing Cervical Intraepithelial Neoplasia
Dübbel, L., Riebisch, A. K., Knoll, K., Hippe, J., Marticke, C., Schild-Suhren, M., & Malik, E. (2025) · Cancers · doi:10.3390/cancers17193095 · Cited by 5
“We calculated the sample size with [31] N = ( 1.78 0.075 Therefore, we planned for 141 patients, but were able to recruit 139 patients only because 2 withdrew their consent.”
Influential
Predicting psychological distress in informal caregivers of older people with dementia: development and interpretation of a nomogram and XGBoost model
Shi, W., Tian, S., Wang, Y., Cai, C., Cheng, B., Li, C., Liu, X., & Shi, C. (2025) · BMC Nursing · doi:10.1186/s12912-025-03905-0 · Cited by 5
“We applied an established rule of thumb for the sample size required to develop a binary outcome prediction model-10 Events Per Variable (10 EPV) [32].”
Construction and validation of an oral frailty risk prediction model for community-dwelling older adults
Lv, K., Yu, P., Xue, Y., Su, J., Ren, Y., & Tang, J. T. (2025) · BMC Geriatrics · doi:10.1186/s12877-025-06393-8 · Cited by 5
“Based on the event/variable method of calculating sample size [13], the sample size for predictive modeling should be at least ten times the number of predictors.”
Development and validation of a nomogram-based prediction model for hospital-acquired carbapenem-resistant Acinetobacter baumannii in critically ill patients: a multicenter retrospective cohort study
Jiang, W., Dai, A., Cao, L., Feng, M. H., Zhou, P., Cheng, J., Tang, Y. M., Luo, X., & Tang, J. (2025) · Frontiers in Cellular and Infection Microbiology · doi:10.3389/fcimb.2025.1679272 · Cited by 5
“According to Riley et al (Riley et al., 2020), a minimum of 10 – 20 outcome events per predictor parameter (EPV) is recommended for reliable logistic regression modeling.”
Prediction models developed using artificial intelligence: similar predictive performances with highly varying predictions for individuals – an illustration in deep vein thrombosis
Yusufujiang, M., Navarro, C. L. A., Damen, J. A., Takada, T., Geersing, G., Hooft, L., Schuit, E., Moons, K. G. M., de Jong, V. M. T., & van Smeden, M. (2026) · Diagnostic and Prognostic Research · doi:10.1186/s41512-025-00216-5 · Cited by 5
“We used the R package pmsampsize [24] to calculate the sample size required to develop a new prediction model based on logistic regression.”
Ten Core Concepts for Ensuring Data Equity in Public Health.
Wang, Y.-R., Boyd, A. E., Rountree, L., Ren, Y., Nyhan, K., Nagar, R., Higginbottom, J., Ranney, M. L., Parikh, H., & Mukherjee, B. (2026) · JAMA Health Forum · doi:10.1001/jamahealthforum.2025.6031 · Cited by 6
Externally validated yet undertrained: sample size deficits in machine learning-based radiomics
Koçak, B., Banaz, T., Söyleyici, M., Köse, F., Keleş, A., & Meşe, İ. (2026) · European Radiology · doi:10.1007/s00330-026-12543-2 · Cited by 6
“Minimum required sample sizes were calculated using the Riley framework via the pmsampsize R package (version 1.1.3), based on reported training AUROC, prevalence, and ﬁ nal feature counts [30, 31].”
Influential
Bayesian sequential learning for prognostication in extremity soft tissue sarcoma (BayeSarc): a retrospective, multicentre cohort study
Callegaro, D., Tinè, G., Pasquali, S., Stacchiotti, S., Casali, P. G., Wunder, J. S., Ferguson, P. C., Griffin, A. M., Strauss, D. C., Hayes, A. J., et al. (2026) · The Lancet Oncology · doi:10.1016/s1470-2045(26)00067-7 · Cited by 5
COVID-19 Prediction Models Need Robust and Transparent Development
Collins, P. G. S. (2021) · Disaster Medicine and Public Health Preparedness · doi:10.1017/dmp.2021.119 · Cited by 5
“Using published sample size formulae for developing prediction models 3 prevalence of 31/150=0.21, along with a conservative (unreported) estimate of the anticipated model R 2 (based on 15% of the maximum possible R 2 , which is dependent on the outcome prevalence, i.e., 15% of 0.64 =0.096), the…”
MethodInfluential
Development of an Injury Burden Prediction Model in Professional Baseball Pitchers
Bullock, G. S., Thigpen, C. A., Collins, P. G. S., Arden, N., Noonan, T. J., Kissenberth, M. J., & Shanley, E. (2022) · International Journal of Sports Physical Therapy · doi:10.26603/001c.39741 · Cited by 5
The Anzhen Risk Scoring System for Acute Type A Aortic Dissection: A Prospective Observational Study Protocol
Jia, B., Luo, C., Li, C., Ge, Y., Zhong, Y., Qiao, Z., Hu, H., Chen, S., & Zhu, J. (2023) · Cardiovascular Innovations and Applications · doi:10.15212/cvia.2023.0002 · Cited by 5
Risk factors for poor sleep quality in patients with inflammatory bowel disease in China: A multicenter study
Zeng, S., Liu, C., Zhang, J., An, P., Liu, Z., Jiang, C., Shi, J., Wu, K., & Dong, W. (2023) · Frontiers in Psychiatry · doi:10.3389/fpsyt.2023.1130396 · Cited by 5
Predicting outcomes at the individual patient level: what is the best method?
Liu, Q., Ostinelli, E. G., Crescenzo, F. D., Li, Z., Tomlinson, A., Salanti, G., Cipriani, A., & Efthimiou, O. (2023) · BMJ Mental Health · doi:10.1136/bmjment-2023-300701 · Cited by 5
Mediating Role of Self-esteem and Trust in the Relationship Between Childhood Trauma and Romantic Attachment in Indian Adults
Chaturvedi, S., & Arya, B. (2023) · Journal of Research and Health · doi:10.32598/jrh.13.5.2296.1 · Cited by 5
A model to predict level of adherence to prehabilitation in older adults with frailty having cancer surgery
Shaw, J. F., Hladkowicz, E., McCartney, C. J. L., Bryson, G. L., & McIsaac, D. I. (2023) · Canadian Journal of Anesthesia/Journal canadien d anesthésie · doi:10.1007/s12630-023-02559-3 · Cited by 5
“24 A sample estimate following these recommendations suggested that a sample size of 308 would be more appropriate for our model.”
Method
Establishment and validation of a nomogram model for riskprediction of hepatic encephalopathy: a retrospective analysis
Yao, C., Huang, L., Wang, M., Mao, D., Wang, M., Zheng, J., Long, F., Huang, J., Liu, X., Zhang, R., et al. (2023) · Scientific Reports · doi:10.1038/s41598-023-47012-z · Cited by 5
Can a Psychologic Profile Predict Successful Return to Full Duty After a Musculoskeletal Injury?
Greenlee, T. A., Bullock, G. S., Teyhen, D. S., & Rhon, D. I. (2023) · Clinical Orthopaedics and Related Research · doi:10.1097/corr.0000000000002935 · Cited by 5
“An a priori sample size calculation was performed [36] that used R 2 = 0.64 from the primary study [34, 35], with a prevalence of 0.34, indicating that 27 allowable parameters could be included to reduce the risk of over ﬁ tting.”
MethodInfluential
Longitudinal observational cohort study: Speech for Intelligent cognition change tracking and DEtection of Alzheimer’s Disease (SIDE-AD)
Saunders, S., Haider, F., Ritchie, C. W., Terrera, G. M., & Luz, S. F. (2024) · BMJ Open · doi:10.1136/bmjopen-2023-082388 · Cited by 6
“…speech conducted by our group 33 as a basis, and assuming a model R 2 of 0.3, a regularisation factor of 0.8, mean arousal and valence scores of 30, standard deviation (SD) of 22 and a reduced feature set of 40 features would require sample size of 421 participants, according to the formula…”
BackgroundMethodInfluential
Development and external validation of a prediction model for the premature circuit clotting of continuous renal replacement therapy in critically ill patients
Yang, E., Wang, Q., Guo, J., Wei, J., Zhang, C., Zhao, W., He, X., Bo, E., Mao, Y., & Yang, H. (2024) · Intensive and Critical Care Nursing · doi:10.1016/j.iccn.2024.103703 · Cited by 6
Prevalence and Factors Associated With Willingness to Sustain Pandemic-Induced Digital Work in the General Population and Moderating Effects of Screen Hours: Cross-Sectional Study
Li, J., Fong, D., Ho, M. M., Choi, E. P. H., Lok, K. Y. W., Lee, J. J., Duan, W., Wong, J. Y. H., & Lin, C. (2024) · Journal of Medical Internet Research · doi:10.2196/53321 · Cited by 5
“The sample size was determined using the rule of thumb, that is, a minimum of 10 events per variable [16].”
Method
Equations to estimate human milk intake in infants aged 7 to 10 months: prediction models from a cross-sectional study
Haszard, J. J., Heath, A. M., Taylor, R. W., Bruckner, B. R., Katiforis, I., McLean, N. H., Cox, A. M., Brown, K. J., Casale, M., Jupiterwala, R. M., et al. (2024) · American Journal of Clinical Nutrition · doi:10.1016/j.ajcnut.2024.04.009 · Cited by 5
Planned but ever published? A retrospective analysis of clinical prediction model studies registered on clinicaltrials.gov since 2000
White, N. M., Parsons, R., Borg, D. N., Collins, P. G. S., & Barnett, A. G. (2024) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2024.111433 · Cited by 5
Mixing efficiency optimization of Tesla-type flow channel for total temperature simulation device
Gan, R., Yang, G., Chu, J., Wang, W., Wu, Z., & Li, B. (2024) · Aerospace Science and Technology · doi:10.1016/j.ast.2024.109435 · Cited by 5
Risk factors of preoperative deep vein thrombosis in patients with non-traumatic osteonecrosis of the femoral head
Wu, D., Wang, T., Li, C., Cheng, X., Yang, Z., Zhu, Y., & Zhang, Y. (2024) · BMC Musculoskeletal Disorders · doi:10.1186/s12891-024-07736-z · Cited by 5
“According to the requirements for developing the clinical prediction model, the sample size should be at least 10 times the number of variables [23].”
Method
Prediction of Postoperative Sore Throat in Patients After Day-case Surgery With General Anesthesia: A Retrospective Study
Zhou, Z., Gao, L., Lv, Z., Chen, L., Lu, K., Cai, J., Sun, J., & Chen, X. (2024) · Journal of PeriAnesthesia Nursing · doi:10.1016/j.jopan.2024.03.022 · Cited by 5
Development and external validation of a model to predict multidrug‐resistant bacterial infections in patients with cirrhosis
Marciano, S., Piano, S. S., Singh, V., Caraceni, P., Maiwall, R., Alessandria, C., Fernández, J., Kim, D. J., Kim, S. E., Soares, E. C., et al. (2024) · Liver International · doi:10.1111/liv.16063 · Cited by 6
“1,2 Using Riley's approach 15 to estimate a sample size with 15 potential parameters, keeping a mean absolute prediction error of .05, a shrinkage factor 10%, a low optimism of .05, and an expected Cox-Snell R2 statistic of .1, 16 a total sample size of 1274 individuals is required for model…”
Predicting long-term survival among patients with HCC
Goldberg, D. S., Reese, P. P., Kaplan, D. A., Zarnegarnia, Y., Gaddipati, N., Gaddipati, S., John, B. V., & Blandon, C. (2024) · Hepatology Communications · doi:10.1097/hc9.0000000000000581 · Cited by 4
“[62 – 65] Therefore, we were confident our analyses would be sufficiently powered.”
Method
Predicting post-COVID-19 condition in children and young people up to 24 months after a positive SARS-CoV-2 PCR-test: the CLoCk study
Nugawela, M. D., Stephenson, T. J., Shafran, R., Chalder, T., Dalrymple, E., Ford, T. J., Fox‐Smith, L., Harnden, A., Heyman, I., Ladhani, S. N., et al. (2024) · BMC Medicine · doi:10.1186/s12916-024-03708-1 · Cited by 7
Predicting Adverse Perinatal Outcomes in Dichorionic Twin Pregnancies: A Multicentre Cohort Study
Giorgione, V., Trapani, M., Lopian, M., Brutto, M., Ferrante, M. G., Bhide, A., Jani, J. C., Badr, D. A., Ghi, T., Thilaganathan, B., et al. (2025) · BJOG An International Journal of Obstetrics & Gynaecology · doi:10.1111/1471-0528.18125 · Cited by 5
Predicting Neoplastic Polyp in Patients With Gallbladder Polyps Using Interpretable Machine Learning Models: Retrospective Cohort Study
He, Z., Yang, S., Cao, J., Gao, H., & Peng, C. (2025) · Cancer Medicine · doi:10.1002/cam4.70739 · Cited by 5
TRANSCENDENT (Transforming Research by Assessing Neuroinformatics across the Spectrum of Concussion by Embedding iNterdisciplinary Data-collection to Enable Novel Treatments): protocol for a prospective observational cohort study of concussion patients with embedded comparative effectiveness research within a network of learning health system concussion clinics in Canada
Zemek, R. L., Albrecht, L. M., Johnston, S., Leddy, J. J., Ledoux, A., Reed, N. P., Silverberg, N. D., Yeates, K. O., Lamoureux, M., Anderson, C., et al. (2025) · BMJ Open · doi:10.1136/bmjopen-2024-095292 · Cited by 5
The risk prediction models for cognitive frailty in the older people in China: a systematic review and meta-analysis
Ren, M., Guo, H., Guo, Y., Guo, W., & Zhu, L. (2025) · BMC Geriatrics · doi:10.1186/s12877-025-05961-2 · Cited by 5
“…future research should establish appropriate inclusion and exclusion criteria and select suitable sample size estimation methods based on the study type to enhance the representativeness and generalizability of the sample [34]; Secondly, the handling of continuous variables was inappropriate.”
Integrating clinical predictors and glial fibrillary acidic protein in prediction models for the prehospital identification of intracerebral haemorrhage in suspected stroke
Almubayyidh, M., Jenkins, D. A., Gaude, E., & Parry‐Jones, A. (2025) · BMJ Neurology Open · doi:10.1136/bmjno-2025-001160 · Cited by 5
“30 This limitation could potentially lead to overfitting and limit the generalisability of our results.”
Preoperative prediction of major adverse outcomes after total arch replacement in acute type A aortic dissection based on machine learning ensemble
Luo, H., Liu, X., Yang, Y., Tang, B., He, P., Ding, L., Wang, Z., & Shi, J. (2025) · Scientific Reports · doi:10.1038/s41598-025-06936-4 · Cited by 5
Development and Validation of a Nomogram for Predicting Oral Frailty Risk in Elderly Patients With Ischaemic Stroke
Xiao, W., Gu, D., Zhang, M., Liao, J., Xu, T., Deng, T., & Zhao, Y. (2025) · Journal of Clinical Nursing · doi:10.1111/jocn.17855 · Cited by 5
Impact of different acute low back pain definitions on the predictors and on the risk of transition to chronic low back pain: a prospective longitudinal cohort study
Osagie, R. O., Tufa, I., Fonseca, A. A., Pagé, M. G., Lacasse, A., Stone, L. S., Rainville, P., Roy, M., Tétreault, P., Fortin, M., et al. (2025) · Pain · doi:10.1097/j.pain.0000000000003669 · Cited by 5
“3,76 Based on the criteria proposed for developing clinical prediction models 67 and anticipating using up to 4 predictors based on previous studies, 31,80,83 an outcome proportion of 0.5 (ie, transition to cLBP), and a target mean absolute error of 0.08 between observed and true outcome…”
Influential
Dietary patterns and psoriasis severity in Thai patients: a machine learning approach for small sample data
Boonkrong, P., Shakya, S., Kraunamkam, W., & Simmachan, T. (2025) · Scientific Reports · doi:10.1038/s41598-025-17657-z · Cited by 5
“This approach led to a substantial improvement in predictive performance compared to models trained without bootstrapping (Table S5), consistent with findings from prior simulation studies 23,40 .”
Development and Validation of a Predictive Model of Pain Modulation Profile to Guide Chronic Pain Treatment: A Study Protocol
Vincenot, M., Coulombe-Lévêque, A., Sean, M., Lemyre, F. C., Gendron, L., Marchand, S., & Léonard, G. (2021) · Frontiers in Pain Research · doi:10.3389/fpain.2021.606422 · Cited by 5
Development and validation of a new prognostic index for mortality risk in multimorbid adults
Gastens, V., Chioléro, A., Anker, D., Schneider, C., Feller, M., Bauer, D. C., Rodondi, N., & Giovane, C. D. (2022) · PLoS ONE · doi:10.1371/journal.pone.0271923 · Cited by 6
“[20] and implemented in the pmsampsize library for the R environment [21].”
Background
Development and validation of a prediction rule for patients suspected of acute coronary syndrome in primary care: a cross-sectional study
Wouters, L., Zwart, D. L. M., Erkelens, D. C. A., Adriaansen, E. J. M., den Ruijter, H. M., GROOT, E. D., Damoiseaux, R. A. M. J., Hoes, A. W., van Smeden, M., & Rutten, F. H. (2022) · BMJ Open · doi:10.1136/bmjopen-2022-064402 · Cited by 5
Development and validation of personalised risk prediction models for early detection and diagnosis of primary liver cancer among the English primary care population using the QResearch® database: research protocol and statistical analysis plan
Liao, W., Jepsen, P., Coupland, C. A. C., Innes, H. A., Matthews, P. C., Campbell, C., Barnes, E. J., & Hippisley‐Cox, J. (2022) · Diagnostic and Prognostic Research · doi:10.1186/s41512-022-00133-x · Cited by 5
Development of Prognostic Prediction Model to Estimate Mortality for Frail Oldest Old: Prospective Cohort Study
Gu, Y., Wu, W., Kong, C., Luo, Q.-Q., Li, R., Tan, X., & Zhang, Q. (2022) · The Journals of Gerontology Series A · doi:10.1093/gerona/glac256 · Cited by 5
“Our sample and the number of events far exceed requirements (37) and therefore are expected to provide robust estimates.”
OPtimal TIming of antenatal COrticosteroid administration in pregnancies complicated by early-onset fetal growth REstriction (OPTICORE): study protocol of a multicentre, retrospective cohort study
van de Meent, M., Kleuskens, D. G., Ganzevoort, W., Gordijn, S. J., Kooi, E. M. W., Onland, W., van Rijn, B. B., Duvekot, J. J., Kornelisse, R. F., Al‐Nasiry, S., et al. (2023) · BMJ Open · doi:10.1136/bmjopen-2022-070729 · Cited by 5
Immuno-genomic-radiomics to predict response of biliary tract cancer to camrelizumab plus GEMOX in a single-arm phase II trial
Liu, Q., Tang, J., Chen, Y., Guo, F., Ma, L., Pan, L., Tian, Y., Wu, X., Zhang, Y., & Chen, X. (2023) · JHEP Reports · doi:10.1016/j.jhepr.2023.100763 · Cited by 5
Risk Factors Associated with the Development of Hospital-Acquired Infections in Hospitalized Patients with Severe COVID-19
Solís-Huerta, F., Martínez-Guerra, B. A., Román-Montes, C. M., Tamez-Torres, K. M., Rajme‐López, S., Ortíz-Conchi, N., López-García, N. I., Villalobos-Zapata, G. I., Rangel‐Cordero, A., Santiago-Cruz, J., et al. (2023) · Antibiotics · doi:10.3390/antibiotics12071108 · Cited by 5
Development and validation of a multimodal feature fusion prognostic model for lumbar degenerative disease based on machine learning: a study protocol
Wang, Z., Xiyun, Z., Li, Y., Zhang, H., Daping, Q., Qi, X., Chen, Y., & Zhang, X. (2023) · BMJ Open · doi:10.1136/bmjopen-2023-072139 · Cited by 5
Screening Tool Risk Score Assessment in the Emergency Department for Geriatric (S-TRIAGE) in 28-day mortality
Supatanakij, P., Imok, K., & Suttapanit, K. (2023) · International Journal of Emergency Medicine · doi:10.1186/s12245-023-00538-5 · Cited by 5
Preventing secondary screw perforation following proximal humerus fracture after locking plate fixation: a new clinical prognostic risk stratification model
Xu, D., Tu, Z.-S., Ji, M., Niu, W., & Xu, W. (2023) · Archives of Orthopaedic and Trauma Surgery · doi:10.1007/s00402-023-05130-3 · Cited by 5
“The number of samples in a logistic regression analysis should be at least 10 times the number of covariates included [28].”
Prediction of Sjögren’s disease diagnosis using matched electronic dental-health record data
Mao, J. M., Gomez, G. G. F., Wang, M., Xu, H., & Thyvalikakath, T. P. (2024) · BMC Medical Informatics and Decision Making · doi:10.1186/s12911-024-02448-9 · Cited by 5
“Larger sample sizes increase the likelihood of building a robust model that can be externally validated, though machine learning models may require even larger sample sizes than traditional models [61].”
Background
Prediction model protocols indicate better adherence to recommended guidelines for study conduct and reporting
Dhiman, P., Ma, J., Kirtley, S., Mouka, E., Waldron, C., Whittle, R., & Collins, P. G. S. (2024) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2024.111287 · Cited by 5
Predictive value of the random forest model based on bioelectrical impedance analysis parameter trajectories for short-term prognosis in stroke patients
Yang, J., Peng, J., Liu, G., & Li, F. (2024) · European journal of medical research · doi:10.1186/s40001-024-01964-8 · Cited by 5
“This research adopted the sample size estimation for prediction model recommended by Richard D. Riley [37].”
Method
Scientific Tasks in Biomedical and Oncological Research: Describing, Predicting, and Explaining
Vera-Ponce, V. J., Zuzunaga-Montoya, F. E., Vásquez-Romero, L. E. M., Sanchez-Tamay, N. M., Loayza-Castro, J. A., & Carrillo, C. I. G. D. (2024) · Journal of cancer research updates · doi:10.30683/1929-2279.2024.13.08 · Cited by 5
“For diagnostic and screening models, where the objective is to predict a present but not directly observable condition, analytical cross-sectional studies are the most appropriate design [32].”
Prognostic model for esophagogastric variceal rebleeding after endoscopic treatment in liver cirrhosis: A Chinese multicenter study
Zhan, J., Chen, J., Yu, J., Xu, F., Xing, F.-F., Wang, D., Yang, M., Xing, F., Wang, J., & Mu, Y. (2024) · World Journal of Gastroenterology · doi:10.3748/wjg.v31.i2.100234 · Cited by 5
“The derivation cohort sample size was calculated using the methodology proposed by Riley et al [18].”
The Naples pediatric food allergy (NAPFA) score: A multivariable model for the prediction of food allergy in children
Carucci, L., Biancardi, L., Nocerino, R., Ciliberti, L., Caldaria, E., Bedogni, G., Palmese, F., Calabrò, F., & Canani, R. B. (2025) · Pediatric Allergy and Immunology · doi:10.1111/pai.70071 · Cited by 5
“27,28 In detail, we calculated that 627 subjects were needed to detect a Cox-Snell R 2 of .377, 29”
Development and Validation of a Novel Nomogram Risk Prediction Model for In-Hospital Death Following Extended Aortic Arch Repair for Acute Type A Aortic Dissection
Chen, Q., Wang, Y., Zhang, Y., Liu, F., Shao, K., Lai, H., Wang, C., & Ji, Q. (2025) · Reviews in Cardiovascular Medicine · doi:10.31083/rcm26943 · Cited by 5
“Based on a margin of error ≤ 0.05 and following established recommendations for developing clinical prediction models [20], the minimum required sample size was calculated to be 139 patients.”
Influential
Predicting the use of antibiotics after initial symptomatic treatment of an uncomplicated urinary tract infection: analyses performed after a randomised controlled trial
Vik, I., Mdala, I., Bollestad, M., Córdoba, G., Bjerrum, L., Neumark, T., Damsgaard, E., Bærheim, A., Grude, N., & Lindbæk, M. (2020) · BMJ Open · doi:10.1136/bmjopen-2019-035074 · Cited by 5
A protocol for the development and internal validation of a model to predict clinical response to antihistamines in urticaria patients
Sánchez, J., Velásquez‐Lopera, M. M., & Jaimes, F. A. (2020) · PLoS ONE · doi:10.1371/journal.pone.0239962 · Cited by 6
A Matrix Metalloproteinase‐2‐Based Nomogram to Assess the Risk of Encapsulating Peritoneal Sclerosis in Peritoneal Dialysis Patients
Chou, C., Tseng, C., Chen, J., Hung, C., & Huang, C. (2021) · BioMed Research International · doi:10.1155/2021/6666441 · Cited by 5
The development and validation of a prognostic model to PREDICT Relapse of depression in adult patients in primary care: protocol for the PREDICTR study
Moriarty, A. S., Paton, L. W., Snell, K. I. E., Riley, R. D., Buckman, J. E. J., Gilbody, S. M., Chew-Graham, C. A., Ali, S., Pilling, S., Meader, N., et al. (2021) · Diagnostic and Prognostic Research · doi:10.1186/s41512-021-00101-x · Cited by 5
Preoperative serum alpha-hydroxybutyrate dehydrogenase level as a predictor of postoperative mortality and morbidity after noncardiac surgery: A propensity-adjusted analysis
Zhu, Y., Bi, Y., Zhang, Y., Ma, J., & Liu, B. (2022) · Surgery · doi:10.1016/j.surg.2021.08.045 · Cited by 5
Standardization of risk prediction model reporting in cancer‐associated thrombosis: Communication from the ISTH SSC subcommittee on hemostasis and malignancy
Sanfilippo, K. M., Wang, T., Carrier, M., Falanga, A., Gage, B. F., Khorana, A. A., Maraveyas, A., Soff, G. A., Wells, P. S., & Zwicker, J. I. (2022) · Journal of Thrombosis and Haemostasis · doi:10.1111/jth.15759 · Cited by 5
A workflow for automatic, high precision livestock diagnostic screening of locomotor kinematics
Mielke, F., Ginneken, C. J. V., & Aerts, P. (2023) · Frontiers in Veterinary Science · doi:10.3389/fvets.2023.1111140 · Cited by 5
Inappropriate antibiotic prescribing for acute bronchiolitis in Colombia: a predictive model
Buendía, J. A., & Feliciano-Alfonso, J. E. (2021) · Journal of Pharmaceutical Policy and Practice · doi:10.1186/s40545-020-00284-6 · Cited by 5
Predictive factors for failure of the levonorgestrel releasing intrauterine system in women with heavy menstrual bleeding
Beelen, P., van den Brink, M. J., Herman, M. C., Geomini, P. M. A. J., Duijnhoven, R. G., & Bongers, M. Y. (2021) · BMC Women s Health · doi:10.1186/s12905-021-01210-x · Cited by 5
“Power in a prognostic model is driven by the number of prognostic factors in the model, the incidence of the outcome, and the number of observations [16].”
Background
Development and validation of a life expectancy estimator for multimorbid older adults: a cohort study protocol
Gastens, V., Giovane, C. D., Anker, D., Feller, M., Syrogiannouli, L., Schwab, N., Bauer, D. C., Rodondi, N., & Chioléro, A. (2021) · BMJ Open · doi:10.1136/bmjopen-2020-048168 · Cited by 5
A comparison of regression models for static and dynamic prediction of a prognostic outcome during admission in electronic health care recordsPreprint
Gao, S., Albu, E., Putter, H., Stijnen, P., Rademakers, F. E., Cossey, V., Debaveye, Y. A., Janssens, C., CALSTER, B. V., & Wynants, L. (2024) · Research Square · doi:10.21203/rs.3.rs-4384973/v1 · Cited by 5
Commentary: core descriptor sets using consensus methods support ‘table one’ consistency
Lee, M. J., Lamidi, S., Williams, K. M., Blackwell, S. E., Rashid, A., Coe, P. O., Fearnhead, N. S., Blencowe, N., & Hind, D. (2024) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2024.111470 · Cited by 5
Development of a clinical prediction tool for extubation failure in pediatric cardiac intensive care unit
Saengsin, K., Sittiwangkul, R., Borisuthipandit, T., Wongyikul, P., Tanasombatkul, K., Phanacharoensawad, T., Moonsawat, G., Trongtrakul, K., & Phinyo, P. (2024) · Frontiers in Pediatrics · doi:10.3389/fped.2024.1346198 · Cited by 4
“To estimate the minimum sample size required for developing a multivariable prediction model for binary outcomes, the expected incidence of extubation failure from previous study was 35% (3, 23).”
Predicting Endometrial Hyperplasia and Endometrial Cancer on Recurrent Abnormal Uterine Bleeding
Veeranaraphanit, U., Saeaib, N., & Srisintorn, W. (2024) · Obstetrics and Gynecology · doi:10.1097/aog.0000000000005641 · Cited by 4
“…model, we backwardly specified the maximum number of predictors using available retrospective data with the process recommended by Riley et al. 17 They described a four-step sample size calculation for clinical prediction models, aiming for 1) precise estimation of the overall outcome, 2)…”
Prediction models for adjacent vertebral fractures after vertebral augmentation: a systematic review and meta-analysis
Sun, D., Wen, Y., Yu, Q., Long, Y., Liu, Y., Zhou, Y. X., & Yu, Y. (2025) · European Spine Journal · doi:10.1007/s00586-025-08785-1 · Cited by 6
“The sample sizes of the included studies were too small to meet this requirement, increasing the risk of over-fitting [ 35].”
Controlling nutritional status score predicts posthepatectomy liver failure: an online interpretable machine learning prediction model
Yuan, J., Zhang, R. Q., Guo, Q., Aji, T., & Shao, Y. M. (2025) · European Journal of Gastroenterology & Hepatology · doi:10.1097/meg.0000000000002965 · Cited by 4
“[24] and via the predictive model sample size calculator ‘pmsampsize1.”
Construction and Validation of a Risk Prediction Model for Early Severe Intraventricular Hemorrhage in Very Low Birth Weight Infants
Shen, F., Xu, J., Rong, H., Zhang, J., Yang, Y., & Li, X. (2025) · The Kaohsiung Journal of Medical Sciences · doi:10.1002/kjm2.70037 · Cited by 4
“The sample size was calculated using R software (version 4.2.1) based on clinical prediction model guidelines [25].”
Influential
Development and validation of an in-hospital major adverse cardiovascular events risk model for young patients with acute coronary syndrome: a retrospective cohort study
Zheng, J., Li, J., Li, T., Hu, F., Cheng, D., & Lu, C. (2025) · PeerJ · doi:10.7717/peerj.19513 · Cited by 4
“Determining the effective sample size in a prediction study is dependent on the number of outcome events that occur ( Riley et al., 2020 ).”
Development of a risk prediction model for postpartum stress urinary incontinence: a multicenter retrospective longitudinal study in Indonesia
Andarini, E., Liang, S., Zhang, Y., Li, Y., Li, Y., & Cai, W. (2025) · PeerJ · doi:10.7717/peerj.19308 · Cited by 4
“The sample size was determined based on the recommended minimum requirement of 100 events (positive samples) for predictive modeling ( Riley et al., 2020 ).”
Development and Validation of a Lifestyle-Based 10-Year Risk Prediction Model of Colorectal Cancer for Early Stratification: Evidence from a Longitudinal Screening Cohort in China
Pu, J., Zhou, B., Yao, Y., Wu, Z., Wen, Y., Xu, R., & Xu, H. (2025) · Nutrients · doi:10.3390/nu17111898 · Cited by 4
Establishment and validation of a model for predicting depression risk in stroke patients
Lin, F., & Zhou, M. (2025) · BMC Psychiatry · doi:10.1186/s12888-025-06885-2 · Cited by 4
“This study strictly adhered to the methodological standards (events per variable ≥ 10, shrinkage factor > 0.9, TRIPOD type 2a) [11].”
Influential
Machine learning prediction of clinical pregnancy in endometriosis patients following fresh IVF/ICSI-ET
Wan, X., Yu, M., Wu, X., Huang, Z., & Tan, J. (2025) · European journal of medical research · doi:10.1186/s40001-025-03113-1 · Cited by 4
“According to the"10 EPV"rule for establishing clinical prediction models, it is generally believed that each variable needs to include at least 10 events [16].”
Predicting risk of early-onset sepsis in low-resource neonatal units using routine healthcare data: development and evaluation of multivariable statistical and machine learning models
Lowther, E., Khan, N., Cortina‐Borja, M., Chimhini, G., Neal, S. R., Mangiza, M., Fitzgerald, F., Heys, M., & Chimhuya, S. (2025) · BMJ Paediatrics Open · doi:10.1136/bmjpo-2025-003617 · Cited by 4
“21”
An interpretable machine learning model for predicting metabolic dysfunction‐associated steatotic liver disease in patients with type 2 diabetes
Zhou, Z., Gao, N., Liu, J., Ma, X., Ge, Z., & Cheng, J. (2025) · Diabetes Obesity and Metabolism · doi:10.1111/dom.70168 · Cited by 6
“The sample size for the study was calculated using the ‘ pmsampsize ’ package in R, which is based on the criteria proposed by Riley et al. 37 for determining the required minimum sample size to develop a reliable prediction model.”
Risk factors and predictive modeling of intraoperative hypothermia in laparoscopic surgery patients
Cheng, Y., & Sheng, H. (2025) · BMC Surgery · doi:10.1186/s12893-025-03186-z · Cited by 5
“This exceeds the minimum EPV threshold of 20 recommended for prediction models with modest anticipated R² values [12].”
Development and validation of diagnostic and prognostic prediction tools for dental caries in young children through prospective and cross-sectional observational studies: a protocol
Khazaei, Y., Kodikara, S., Butler, C. A., Messina, N. L., Cao, K. L., Dashper, S. G., & Silva, M. (2025) · BMJ Open · doi:10.1136/bmjopen-2025-105145 · Cited by 5
Machine learning risk stratification strategy for multiple myeloma: Insights from the EMN–HARMONY Alliance platform
Orgueira, A. M., Perez, M. S. G., D’Agostino, M., Cairns, D. A., Larocca, A., Palacios, J. J. L., Wester, R., Bertsch, U., Waage, A., Zamagni, E., et al. (2025) · HemaSphere · doi:10.1002/hem3.70228 · Cited by 6
Automated Chronic Obstructive Pulmonary Disease Phenotyping and Control Assessment in Primary Care: Retrospective Multicenter Study Using the Seleida Model
Viejo, J. D. M., & Ros, F. M. N. (2025) · JMIR Medical Informatics · doi:10.2196/74932 · Cited by 4
“This yielded an events-per-variable ratio of 23.5 for the 2 selected predictors—annual rescue inhaler canisters and respiratory antibiotic courses dispensed—well above the recommended ≥10 events-per-variable threshold for stable logistic regression estimates [29,41,42].”
Influential
Establishment and comparison of three fear of progression risk prediction models for gynecological malignancies patients based on machine learning
Xiong, A., Wang, J., Wang, Z., Qi, D., Yu, Y., Xia, L., & Gao, J. (2025) · Frontiers in Oncology · doi:10.3389/fonc.2025.1632026 · Cited by 4
Predicting Postoperative Stress Urinary Incontinence After Prolapse Surgery via Machine Learning and Regression Models: Development and Validation Study
Su, M., Wang, S., & Liu, X. (2025) · JMIR Medical Informatics · doi:10.2196/76021 · Cited by 5
“The Riley 4-step method, specifically used in clinical prediction models, was adopted for sample size calculation [15].”
Relative hyperglycemia and new-onset atrial fibrillation after coronary artery bypass grafting: a focus on stress hyperglycemia ratio
Cao, H., Gui, L., Wei, J., Zheng, M., Cheng, M., & Ning, C. (2025) · BMC Cardiovascular Disorders · doi:10.1186/s12872-025-05231-4 · Cited by 4
An individualized risk prediction tool for ectopic pregnancy within the first 10 weeks of gestation based on machine learning algorithms
Du, X., Chen, Q., Lü, M., Hu, J., Chen, C., Huang, K., Ji, C., Zhou, Z., Zou, J., & Ruan, H. (2025) · Frontiers in Medicine · doi:10.3389/fmed.2025.1726606 · Cited by 4
Understanding risk of poor outcomes in adults hospitalised with respiratory syncytial virus infection: evidence from a multicentre UK cohort
Morelli, T., Purcell, M., Panaguiton, J., Patel, R. C., Weinberg, S. E., Hulston, G., Siy-Yap, H., Davies, E. A., Keating, A., Saidy, I., et al. (2026) · Thorax · doi:10.1136/thorax-2025-224192 · Cited by 4
“Sample size adequacy was assessed using pmsampsize, 24 assuming an anticipated C-statistic of 0.80, outcome prevalence of 0.11 and a target shrinkage factor ≥0.90.”
Influential
A machine learning model for prediction of early-onset neonatal sepsis in low-income and middle-income countries: development and validation study
Kainth, D., Gupta, A., Singh, P., Prakash, S., Thukral, A., Deorari, A. K., Kapoor, M., Agarwal, R. K., Sethi, T., & Sankar, M. J. (2026) · BMJ Paediatrics Open · doi:10.1136/bmjpo-2025-003561 · Cited by 4
“The model output was compared with the true disease label, and a 2×2 table was constructed to compute sensitivity, specificity, PPV, NPV and likelihood ratios.”
Prognostic score for predicting respiratory admissions among patients with chronic obstructive pulmonary disease in primary care: development and validation in population cohorts (Birmingham Lung Improvement Studies (BLISS))
Jordan, R. E., Keene, S. J., Franssen, F. M. E., Fitzmaurice, D., Adderley, N. J., Dickens, A., Martin, J., Sitch, A. J., Enocson, A., Jowett, S. M., et al. (2026) · BMJ · doi:10.1136/bmj-2025-084521 · Cited by 4
Diagnostic performance of multislice spiral computed tomography in differentiating pancreatic acinar cell carcinoma from pancreatic ductal adenocarcinoma
Yu, J., Cao, Y., Xia, F., Lan, Q., Zhou, J. M., Wang, Z., & Ren, S. (2026) · Abdominal Radiology · doi:10.1007/s00261-026-05617-8 · Cited by 4
Development of a prediction nomogram model of recurrent febrile seizures in pediatric children
Shao, L., & Yu, Y. (2023) · European Journal of Pediatrics · doi:10.1007/s00431-023-05133-7 · Cited by 4
“Based on the sample size calculation formula [21]: 2 3) prevalence rate of recurrent FS [4]; given μ α/2 = 1.96, and δ = 0.05.”
BackgroundInfluential
Unraveling variations and enhancing prediction of successful sphincter-preserving resection for low rectal cancer: a post hoc analysis of the multicentre LASRE randomized clinical trial
Wang, X., Jiang, W., Deng, Y., Chen, Z., Zheng, Z., Sun, Y., Xie, Z., Lu, X., Huang, S., Yu, L., et al. (2024) · International Journal of Surgery · doi:10.1097/js9.0000000000001014 · Cited by 4
“[14] .”
Discriminatory ability of perioperative heart rate variability in predicting postoperative complications in major urologic surgery: a prospective cohort study
Ryan, T., Walker, A. M., & Liepert, D. J. (2024) · Scientific Reports · doi:10.1038/s41598-024-62930-2 · Cited by 4
Association between perception of harm and valence of shoulder movement images with disability levels related to chronic shoulder pain
Tozzo, M. C., dos Reis, F. J. J., Alaiti, R. K., Hotta, G. H., & de Oliveira, A. S. (2024) · Pain · doi:10.1097/j.pain.0000000000003337 · Cited by 4

Authors: Riley, R. D., Ensor, J., Snell, K. I. E., Harrell, F. E., Martin, G. P., Reitsma, J. B., Moons, K. G. M., Collins, G., & van Smeden, M.
Affiliations: Keele University (GB)
Vanderbilt University (US)
Manchester Academic Health Science Centre (GB)
University Medical Center Utrecht (NL)
Corresponding author: Riley, Richard D
ORCID: Riley, Richard D
Ensor, Joie
Snell, Kym I E
Harrell, Frank E
Martin, Glen P
Reitsma, Johannes B
Moons, Karel G M
Collins, Gary
Venue: BMJ
Publisher: BMJ
Published: 2020-03-18
Volume: 368
Pages: m441
ISSN: 1756-1833 (ISSN-L)
DOI: 10.1136/bmj.m441
OpenAlex: W3012413426
Cited by: 2,917
References: 80
Citations by year: 2026: 863 · 2025: 606 · 2024: 531 · 2023: 402 · 2022: 275 · 2021: 174 · 2020: 58
FWCI: 160.59
Type: journal article
Language: en
Open access: bronze
doi.org
Licence: www.bmj.com/company/legal-information/terms-conditions/legal-information/tdm-licencepolicy (tdm)
Locations: BMJ (publishedVersion)
PubMed (publishedVersion)
Research Explorer (The University of Manchester) (publishedVersion, other-oa)
Keele Research Repository (Keele University) (submittedVersion, cc-by)
Utrecht University Repository (Utrecht University) (submittedVersion, other-oa)
HighWire Press Open Archive (submittedVersion)
University of Birmingham Research Portal (University of Birmingham) (submittedVersion)
Funders: Georgia Clinical and Translational Science Alliance (UL1TR002243)
Cancer Research UK (27294)
National Institute for Health and Care Research (SPCR-142)
Department of Health and Social Care
Nederlandse Organisatie voor Wetenschappelijk Onderzoek (91208004, 918.10.615, project 9120.8004)
National Institutes of Health (UL1TR002243)
NIHR School for Primary Care Research
National Center for Advancing Translational Sciences (TR002243)
Topics: Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Epidemiology
Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Clinical Trials
Keywords: sample size determination, clinical prediction model, model development, prediction accuracy, prognostic model, diagnostic model, small sample size
MeSH terms: Clinical Decision-Making, Forecasting, Humans, Models, Theoretical, Sample Size

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

Calculating the sample size required for developing a clinical prediction model

Abstract

Overview

Integrity signals

Verification

Claims the paper makes

Claims other papers make about this one

Self-check

Citations

Cites

Cited by

Related

Metadata

Record

About these checks