The DOI matches its Crossref record; 36 of the 36 references it cites were found in a registry or an index.
Minimum sample size for developing a multivariable prediction model: Part I – Continuous outcomes
Open accessAbstract
In the medical literature, hundreds of prediction models are being developed to predict health outcomes in individuals. For continuous outcomes, typically a linear regression model is developed to predict an individual's outcome value conditional on values of multiple predictors (covariates). To improve model development and reduce the potential for overfitting, a suitable sample size is required in terms of the number of subjects (n) relative to the number of predictor parameters (p) for potential inclusion. We propose that the minimum value of n should meet the following four key criteria: (i) small optimism in predictor effect estimates as defined by a global shrinkage factor of≥0.9; (ii) small absolute difference of≤0.05 in the apparent and adjustedR2; (iii) precise estimation (a margin of error≤10% of the true value) of the model's residual standard deviation; and similarly, (iv) precise estimation of the mean predicted outcome value (model intercept). The criteria require prespecification of the user's chosen p and the model's anticipatedR2as informed by previous studies. The value of n that meets all four criteria provides the minimum sample size required for model development. In an applied example, a new model to predict lung function in African‐American women using 25 predictor parameters requires at least 918 subjects to meet all criteria, corresponding to at least 36.7 subjects per predictor parameter. Even larger sample sizes may be needed to additionally ensure precise estimates of key predictor effects, especially when important categorical predictors have low prevalence in certain categories.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Regression shrinkage and selection via the lasso
Tibshirani, R. (1996) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/j.2517-6161.1996.tb02080.x · Cited by 51,482
Moons, K. G. M., Altman, D. G., Reitsma, J. B., Ioannidis, J. P. A., Macaskill, P., Steyerberg, E. W., Vickers, A. J., Ransohoff, D. F., & Collins, G. S. (2015) · Annals of Internal Medicine · doi:10.7326/m14-0698 · Cited by 5,147
Self-citationInternal validation of predictive models
Steyerberg, E. W., Harrell, F. E., Borsboom, G. J. J. M., Eijkemans, M. J. C., Vergouwe, Y., & Habbema, J. D. F. (2001) · Journal of Clinical Epidemiology · doi:10.1016/s0895-4356(01)00341-9 · Cited by 2,746
Self-citationPrognosis Research Strategy (PROGRESS) 3: Prognostic Model Research
Steyerberg, E. W., Moons, K. G. M., van der Windt, D. A., Hayden, J. A., Perel, P., Schroter, S., Riley, R. D., Hemingway, H., Altman, D. G., & for the PROGRESS Group (2013) · PLoS Medicine · doi:10.1371/journal.pmed.1001381 · Cited by 1,508
Self-citationSteyerberg, E. W. (2009) · Statistics for Biology and Health · doi:10.1007/978-0-387-77244-8 · Cited by 1,491
Riley, R. D., Snell, K. I., Ensor, J., Burke, D. L., Harrell Jr, F. E., Moons, K. G., & Collins, G. S. (2019) · Statistics in Medicine · doi:10.1002/sim.7992 · Cited by 1,409
Self-citationBootstrap Methods: Another Look at the Jackknife
Efron, B. (1992) · Springer Series in Statistics · doi:10.1007/978-1-4612-4380-9_41 · Cited by 1,374
The number of subjects per variable required in linear regression analyses
Austin, P. C., & Steyerberg, E. W. (2015) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2014.12.014 · Cited by 1,059
Prognosis Research Strategy (PROGRESS) 2: Prognostic Factor Research
Riley, R. D., Hayden, J. A., Steyerberg, E. W., Moons, K. G. M., Abrams, K., Kyzas, P. A., Malats, N., Briggs, A., Schroter, S., Altman, D. G., et al. (2013) · PLoS Medicine · doi:10.1371/journal.pmed.1001380 · Cited by 823
Self-citationToward using confidence intervals to compare correlations.
Zou, G. Y. (2007) · Psychological Methods · doi:10.1037/1082-989x.12.4.399 · Cited by 740
How to develop a more accurate risk prediction model when there are few events
Pavlou, M., Ambler, G., Seaman, S. R., Guttmann, O., Elliott, P., King, M., & Omar, R. Z. (2015) · BMJ · doi:10.1136/bmj.h3868 · Cited by 664
A guide to systematic review and meta-analysis of prediction model performance
Debray, T. P. A., Damen, J. A. A. G., Snell, K. I. E., Ensor, J., Hooft, L., Reitsma, J. B., Riley, R. D., & Moons, K. G. M. (2017) · BMJ · doi:10.1136/bmj.i6460 · Cited by 657
Self-citationPredictive value of statistical models.
Van Houwelingen, J. C., & Le Cessie, S. (1990) · Statistics in Medicine · doi:10.1002/sim.4780091109 · Cited by 621
Regression, Prediction and Shrinkage
Copas, J. B. (1983) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/j.2517-6161.1983.tb01258.x · Cited by 612
Adequate sample size for developing prediction models is not simply related to events per variable
Ogundimu, E. O., Altman, D. G., & Collins, G. S. (2016) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2016.02.031 · Cited by 546
Self-citationSample Size Planning for Statistical Power and Accuracy in Parameter Estimation
Maxwell, S. E., Kelley, K., & Rausch, J. R. (2008) · Annual Review of Psychology · doi:10.1146/annurev.psych.59.103006.093735 · Cited by 542
R 2 Measures Based on Wald and Likelihood Ratio Joint Significance Tests
Magee, L. (1990) · The American Statistician · doi:10.1080/00031305.1990.10475731 · Cited by 368
Kelley, K., & Maxwell, S. E. (2003) · Psychological Methods · doi:10.1037/1082-989x.8.3.305 · Cited by 310
Sample Sizes When Using Multiple Linear Regression for Prediction
Knofczynski, G. T., & Mundfrom, D. (2008) · Educational and Psychological Measurement · doi:10.1177/0013164407310131 · Cited by 306
R 2 Measures Based on Wald and Likelihood Ratio Joint Significance Tests
Magee, L. (1990) · The American Statistician · doi:10.2307/2685352 · Cited by 274
Genetic Ancestry in Lung-Function Predictions
Kumar, R., Seibold, M. A., Aldrich, M. C., Williams, L. K., Reiner, A. P., Colangelo, L., Galanter, J., Gignoux, C., Hu, D., Sen, S., et al. (2010) · New England Journal of Medicine · doi:10.1056/nejmoa0907897 · Cited by 256
Methods for the Behavioral, Educational, and Social Sciences: An R package
Kelley, K. (2007) · Behavior Research Methods · doi:10.3758/bf03192993 · Cited by 255
Harrell, F. E. (2015) · Springer Series in Statistics · doi:10.1007/978-3-319-19425-7_13 · Cited by 255
Self-citationConfidence Intervals for Standardized Effect Sizes: Theory, Application, and Implementation
Kelley, K. (2007) · Journal of Statistical Software · doi:10.18637/jss.v020.i08 · Cited by 236
Kelley, K. (2007) · Behavior Research Methods · doi:10.3758/bf03192966 · Cited by 157
Using regression models for prediction: shrinkage and regression to the mean
Copas, J. B. (1997) · Statistical Methods in Medical Research · doi:10.1177/096228029700600206 · Cited by 142
Riley, R. D., Kauser, I., Bland, M., Thijs, L., Staessen, J. A., Wang, J., Gueyffier, F., & Deeks, J. J. (2013) · Statistics in Medicine · doi:10.1002/sim.5726 · Cited by 98
Self-citationShrinkage and Penalized Likelihood as Methods to Improve Predictive Accuracy
van Houwelingen, H. C. (2001) · Statistica Neerlandica · doi:10.1111/1467-9574.00154 · Cited by 88
Increasing statistical power without increasing sample size.
McClelland, G. H. (2000) · American Psychologist · doi:10.1037/0003-066x.55.8.963 · Cited by 81
Tufts PACE Clinical Predictive Model Registry: update 1990 through 2015
Wessler, B. S., Paulus, J. K., Lundquist, C. M., Ajlan, M., Natto, Z. S., Janes, W., Jethmalani, N., Raman, G., Lutz, J. S., & Kent, D. M. (2017) · Diagnostic and Prognostic Research · doi:10.1186/s41512-017-0021-2 · Cited by 57
Determining Sample Size for Accurate Estimation of the Squared Multiple Correlation Coefficient
Algina, J. J., & Olejnik, S. F. (2000) · Multivariate Behavioral Research · doi:10.1207/s15327906mbr3501_5 · Cited by 43
Tables of Upper Percentage Point of the Multiple Correlation Coefficient
Lee, Y.-S. (1972) · Biometrika · doi:10.1093/biomet/59.1.175 · Cited by 35
Hendry, D. F., & Nielsen, B. J. (2012) · Princeton University Press eBooks · doi:10.2307/j.ctv30pnvtj · Cited by 5
Tan, L. (2012) · Scholarship@Western (Western University) · Cited by 4
Harrell, F., Jr. (1996) · doi:10.1002/(sici)1097-0258(19960229)15:4<361::aid-sim168>3.0.co;2-4
A simple method of sample size calculation for linear and logistic regression
(1998) · doi:10.1002/(sici)1097-0258(19980730)17:14<1623::aid-sim871>3.0.co;2-s
Cited by
Collins, G. S., Moons, K. G. M., Dhiman, P., Riley, R. D., Beam, A. L., Van Calster, B., Ghassemi, M., Liu, X., Reitsma, J. B., van Smeden, M., et al. (2024) · BMJ · doi:10.1136/bmj-2023-078378 · Cited by 4,718
Self-citationCalculating the sample size required for developing a clinical prediction model
Riley, R. D., Ensor, J., Snell, K. I. E., Harrell, F. E., Martin, G. P., Reitsma, J. B., Moons, K. G. M., Collins, G., & van Smeden, M. (2020) · BMJ · doi:10.1136/bmj.m441 · Cited by 2,917
Moons, K. G. M., Wolff, R. F., Riley, R. D., Whiting, P. F., Westwood, M., Collins, G. S., Reitsma, J. B., Kleijnen, J., & Mallett, S. (2019) · Annals of Internal Medicine · doi:10.7326/m18-1377 · Cited by 1,669
Self-citationRiley, R. D., Snell, K. I., Ensor, J., Burke, D. L., Harrell Jr, F. E., Moons, K. G., & Collins, G. S. (2019) · Statistics in Medicine · doi:10.1002/sim.7992 · Cited by 1,409
Self-citationMoons, K. G. M., Damen, J. A. A., Kaul, T., Hooft, L., Andaur Navarro, C., Dhiman, P., Beam, A. L., Van Calster, B., Celi, L. A., Denaxas, S., et al. (2025) · BMJ · doi:10.1136/bmj-2024-082505 · Cited by 1,069
Self-citation
Metadata
- Authors
- Riley, R. D., Snell, K. I. E., Ensor, J., Burke, D. L., Harrell, F. E., Moons, K. G. M., & Collins, G. S.
- Venue
- Statistics in Medicine
- Publisher
- Wiley
- Published
- 2018-10-22
- Volume
- 38
- Issue
- 7
- Pages
- 1262-1275
- ISSN
- 0277-6715 (ISSN-L)
- 1097-0258
- OpenAlex
- W2896476100
- Cited by
- 351
- References
- 36
- FWCI
- 14.31
- Type
- journal article
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.
