Variable selection strategies and its importance in clinical prediction modelling.
Open accessAbstract
Clinical prediction models are used frequently in clinical practice to identify patients who are at risk of developing an adverse outcome so that preventive measures can be initiated. A prediction model can be developed in a number of ways; however, an appropriate variable selection strategy needs to be followed in all cases. Our purpose is to introduce readers to the concept of variable selection in prediction modelling, including the importance of variable selection and variable reduction strategies. We will discuss the various variable selection techniques that can be applied during prediction model building (backward elimination, forward selection, stepwise selection and all possible subset selection), and the stopping rule/selection criteria in variable selection (p values, Akaike information criterion, Bayesian information criterion and Mallows’ C p statistic). This paper focuses on the importance of including appropriate variables, following the proper steps, and adopting the proper methods when selecting variables for prediction models.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Paired reviewers independently assessed the quality of included studies using the Cochrane Collaboration's Risk of Bias Tool for RCTs.
“Paired reviewers independently assessed the quality of included studies using the Cochrane Collaboration’s Risk of Bias Tool for RCTs.33”
From Acceptance and Commitment Therapy Among Informal Caregivers of People With Chronic Health Conditions: A Systematic Review and Meta-Analysis (2023) · cites this paper as [33]
The paper’s Variable reduction strategies section discusses systematic reviews as a way to identify candidate variables for prediction models, not how reviewers assessed study quality. Searches for paired reviewers, the Cochrane Risk of Bias Tool, and RCTs found no passage in our copy addressing the claimed assessment procedure. (LLM, conf=0.76)
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
(1990) · Choice Reviews Online · doi:10.5860/choice.27-3338 · Cited by 35,305 · see citing works
Hosmer, D. W., Lemeshow, S., & Sturdivant, R. X. (2013) · Wiley Series in Probability and Statistics · doi:10.1002/9781118548387 · Cited by 13,861 · see citing works
Multimodel Inference: Understanding AIC and BIC in Model Selection
Burnham, K. P., & Anderson, D. R. (2004) · Sociological Methods & Research · doi:10.1177/0049124104268644 · Cited by 11,648 · see citing works
A simulation study of the number of events per variable in logistic regression analysis
Peduzzi, P., Concato, J., Kemper, E., Holford, T. R., & Feinstein, A. R. (1996) · Journal of Clinical Epidemiology · doi:10.1016/s0895-4356(96)00236-3 · Cited by 9,655 · see citing works
Regression Modeling Strategies
Harrell, F. E. (2001) · Springer series in statistics · doi:10.1007/978-1-4757-3462-1 · Cited by 9,357 · see citing works
Kuhn, M., & Johnson, K. (2013) · doi:10.1007/978-1-4614-6849-3 · Cited by 6,788 · see citing works
Relaxing the Rule of Ten Events per Variable in Logistic and Cox Regression
Vittinghoff, E., & McCulloch, C. E. (2007) · American Journal of Epidemiology · doi:10.1093/aje/kwk052 · Cited by 4,359 · see citing works
(2000) · Applied Physics Letters · doi:10.1162/153244303322753616 · Cited by 3,581 · see citing works
Clinical Prediction Models: A Practical Approach to Development, Validation, and Updating
Steyerberg, E. W. (2019) · Statistics for Biology and Health · doi:10.1007/978-3-030-16399-0 · Cited by 3,281 · see citing works
Peduzzi, P., Concato, J., Feinstein, A. R., & Holford, T. R. (1995) · Journal of Clinical Epidemiology · doi:10.1016/0895-4356(95)00048-8 · Cited by 2,154 · see citing works
Babyak, M. A. (2004) · Psychosomatic Medicine · doi:10.1097/01.psy.0000127692.23278.a9 · Cited by 2,061 · see citing works
Towards better clinical prediction models: seven steps for development and an ABCD for validation
Steyerberg, E. W., & Vergouwe, Y. (2014) · European Heart Journal · doi:10.1093/eurheartj/ehu207 · Cited by 1,928 · see citing works
Regression modelling strategies for improved prognostic prediction
Harrell, F. E., Lee, K. L., Califf, R. M., Pryor, D. B., & Rosati, R. A. (1984) · Statistics in Medicine · doi:10.1002/sim.4780030207 · Cited by 1,804 · see citing works
Prognosis and prognostic research: Developing a prognostic model
Royston, P., Moons, K. G. M., Altman, D. G., & Vergouwe, Y. (2009) · BMJ · doi:10.1136/bmj.b604 · Cited by 1,172 · see citing works
Model selection for ecologists: the worldviews of AIC and BIC
Aho, K., Derryberry, D., & Peterson, T. (2014) · Ecology · doi:10.1890/13-1452.1 · Cited by 1,165 · see citing works
The Bayesian information criterion: background, derivation, and applications
Neath, A. A., & Cavanaugh, J. E. (2011) · Wiley Interdisciplinary Reviews Computational Statistics · doi:10.1002/wics.199 · Cited by 1,070 · see citing works
Adequate sample size for developing prediction models is not simply related to events per variable
Ogundimu, E. O., Altman, D. G., & Collins, G. S. (2016) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2016.02.031 · Cited by 546 · see citing works
Model selection and Akaike Information Criteria: An example from wine ratings and prices
Snipes, M., & Taylor, D. C. (2014) · Wine Economics and Policy · doi:10.1016/j.wep.2014.03.001 · Cited by 225 · see citing works
How to Establish Clinical Prediction Models
Lee, Y.-H., Bang, H., & Kim, D. J. (2016) · Endocrinology and Metabolism · doi:10.3803/enm.2016.31.1.38 · Cited by 180 · see citing works
Prediction Error and Its Estimation for Subset-Selected Models
Roecker, E. B. (1991) · Technometrics · doi:10.2307/1269417 · Cited by 123 · see citing works
The Interpretation of Mallows's C p -Statistic
Gilmour, S. G. (1996) · Journal of the Royal Statistical Society Series D (The Statistician) · doi:10.2307/2348411 · Cited by 113 · see citing works
Prediction Error and Its Estimation for Subset-Selected Models
Roecker, E. B. (1991) · Technometrics · doi:10.1080/00401706.1991.10484873 · Cited by 110 · see citing works
Strategies for comparing treatments on a binary response with multi-centre data
Agresti, A., & Hartzel, J. S. (2000) · Statistics in Medicine · doi:10.1002/(sici)1097-0258(20000430)19:8 · Cited by 102 · see citing works
Variable selection methods in regression: Ignorable problem, outing notable solution
Ratner, B. (2010) · Journal of Targeting Measurement and Analysis for Marketing · doi:10.1057/jt.2009.26 · Cited by 99 · see citing works
A Hypertension Risk Score for Middle‐Aged and Older Adults
Kshirsagar, A. V., Chiu, Y., Bomback, A. S., August, P., Viera, A. J., Colindres, R. E., & Bang, H. (2010) · Journal of Clinical Hypertension · doi:10.1111/j.1751-7176.2010.00343.x · Cited by 75 · see citing works
Holden, R. J., Brown, R., Scanlon, M. C., & Karsh, B. (2012) · Journal of the American Medical Informatics Association · doi:10.1136/amiajnl-2011-000754 · Cited by 73 · see citing works
Bleeker, S., Derksen‐Lubsen, G., Grobbee, D. E., et al. (2006) · Acta Paediatrica · doi:10.1111/j.1651-2227.2006.00033.x · Cited by 69 · see citing works
Predicting serious bacterial infection in young children with fever without apparent source
Moons, K. G. M. (2001) · Acta Paediatrica · doi:10.1080/080352501317130236 · Cited by 67 · see citing works
Prediction models for the risk of new-onset hypertension in ethnic Chinese in Taiwan
Chien, K., Hsu, H.-C., Su, T., Chang, W.-T., Sung, F.-C., Chen, M.-F., & Lee, Y.-T. (2010) · Journal of Human Hypertension · doi:10.1038/jhh.2010.63 · Cited by 61 · see citing works
Endolumenal therapies for gastroesophageal reflux disease: are they dead?
Pearl, J. P., & Marks, J. M. (2006) · Surgical Endoscopy · doi:10.1007/s00464-006-0890-x · Cited by 21 · see citing works
Dang, J. T., Tran, C., Switzer, N. J., et al. (2019) · Surgical Endoscopy · doi:10.1007/s00464-019-06932-6 · Cited by 20 · see citing works
Flagged first, then by citations
Research works printed but not linked: 2
- [4]Guyon I, Elisseeff A An introduction to variable and feature selection. Journal of machine learning research 2003; 3: 1157– 82.
- [28]Gilmour SG The interpretation of Mallows's Cp-statistic. The Statistician 1996; 1: 49– 56.
Cited by
Sorted by citations
Ye, F., Lee, J. J., Xue, D., & Yu, D. S.-F. (2023) · JAMA Network Open · doi:10.1001/jamanetworkopen.2023.46216 · Cited by 23 · see citing works
Bartoli, F., Cavaleri, D., Moretti, F., Bachi, B., Calabrese, A., Callovini, T., Cioni, R. M., Riboldi, I., Nacinovich, R., Crocamo, C., et al. (2020) · Medicina · doi:10.3390/medicina56110613 · Cited by 20 · see citing works
Pain Characteristics and Quality of Life in Older People at High Risk of Future Hospitalization
Johansson, M. M., Barbero, M., Peolsson, A., Falla, D., Cescon, C., Folli, A., & Dong, H.-J. (2021) · International Journal of Environmental Research and Public Health · doi:10.3390/ijerph18030958 · Cited by 18 · see citing works
Nielsen, S. S., Skou, S. T., Larsen, A. E., Søndergaard, J., & Christensen, J. R. (2022) · Scandinavian Journal of Pain · doi:10.1515/sjpain-2021-0062 · Cited by 17 · see citing works
- Authors
- Chowdhury, M. Z. I., & Turin, T. C.
- Affiliations
- University of Calgary (CA)
- Corresponding author
- Turin, Tanvir C
- ORCID
- Turin, Tanvir C
- Venue
- Family Medicine and Community Health
- Publisher
- BMJ
- Published
- 2020-02
- Volume
- 8
- Issue
- 1
- Pages
- e000262
- ISSN
- 2009-8774 (ISSN-L)
- 2305-6983
- PMC
- PMC7032893
- OpenAlex
- W3008867523
- Cited by
- 1,011
- References
- 31
- Citations by year
- 2026: 128 · 2025: 245 · 2024: 228 · 2023: 193 · 2022: 127 · 2021: 82 · 2020: 6 · 2017: 1
- FWCI
- 71.89
- Type
- journal article
- Language
- en
- Open access
- diamond
- doi.org
- Licence
- Locations
- Family Medicine and Community Health (publishedVersion, cc-by-nc)
- PubMed Central (publishedVersion, cc-by-nc)
- PubMed (publishedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Topics
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods and Inference
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Epidemiology
- Health Sciences › Health Professions › Health Information Management › Artificial Intelligence in Healthcare
- Keywords
- feature selection, clinical prediction model, backward elimination, forward selection, stepwise selection, stopping rules, Akaike information criterion, Bayesian information criterion, Mallows' Cp, variable reduction, risk prediction
- MeSH terms
- Adverse Outcome Pathways, Bayes Theorem, Humans, Models, Statistical, Biomedical Research
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
