The DOI matches its Crossref record; 69 of the 69 references it cites were found in a registry or an index.
Variable selection – A review and recommendations for the practicing statistician
Open accessAbstract
Statistical models support medical research by facilitating individualized outcome prognostication conditional on independent variables or by estimating effects of risk factors adjusted for covariates. Theory of statistical models is well‐established if the set of independent variables to consider is fixed and small. Hence, we can assume that effect estimates are unbiased and the usual methods for confidence interval estimation are valid. In routine work, however, it is not known a priori which covariates should be included in a model, and often we are confronted with the number of candidate variables in the range 10–30. This number is often too large to be considered in a statistical model. We provide an overview of various available variable selection methods that are based on significance or information criteria, penalized likelihood, the change‐in‐estimate criterion, background knowledge, or combinations thereof. These methods were usually developed in the context of a linear regression model and then transferred to more generalized linear models or models for censored survival data. Variable selection, in particular if used in explanatory modeling where effect estimates are of central interest, can compromise stability of a final model, unbiasedness of regression coefficients, and validity of p‐values or confidence intervals. Therefore, we give pragmatic recommendations for the practicing statistician on application of variable selection methods in general (low‐dimensional) modeling problems and on performing stability investigations and inference. We also propose some quantities based on resampling the entire variable selection process to be routinely reported by software packages offering automated variable selection algorithms.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Breiman, L. (2001) · Machine Learning · doi:10.1023/a:1010933404324 · Cited by 133,538
Regression shrinkage and selection via the lasso
Tibshirani, R. (1996) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/j.2517-6161.1996.tb02080.x · Cited by 51,482
Estimating the Dimension of a Model
Schwarz, G. (1978) · The Annals of Statistics · doi:10.1214/aos/1176344136 · Cited by 47,033
Model selection and multimodel inference: a practical information-theoretic approach.
Guthery, F. S., Burnham, K. P., & Anderson, D. R. (2003) · The Journal of Wildlife Management · doi:10.2307/3802723 · Cited by 41,495
Regression Models and Life-Tables
Cox, D. R. (1972) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/j.2517-6161.1972.tb00899.x · Cited by 39,793
Regression Models and Life-Tables
Cox, D. R. (1992) · Springer Series in Statistics · doi:10.1007/978-1-4612-4380-9_37 · Cited by 26,297
The Elements of Statistical Learning
Hastie, T., Tibshirani, R., & Friedman, J. (2009) · Springer Series in Statistics · doi:10.1007/978-0-387-84858-7 · Cited by 25,970
Regularization Paths for Generalized Linear Models via Coordinate Descent
Friedman, J., Hastie, T., & Tibshirani, R. (2010) · Journal of Statistical Software · doi:10.18637/jss.v033.i01 · Cited by 19,927
Hosmer, D. W., Lemeshow, S., & Sturdivant, R. X. (2013) · Wiley Series in Probability and Statistics · doi:10.1002/9781118548387 · Cited by 13,861
Variable Selection via Nonconcave Penalized Likelihood and its Oracle Properties
Fan, J., & Li, R. (2001) · Journal of the American Statistical Association · doi:10.1198/016214501753382273 · Cited by 8,900
The adaptive lasso and its oracle properties
Zou, H. (2006) · Journal of the American Statistical Association · doi:10.1198/016214506000000735 · Cited by 7,470
Harrell (2015) · Springer Series in Statistics · doi:10.1007/978-3-319-19425-7 · Cited by 6,878
Statistical Modeling: The Two Cultures (with comments and a rejoinder by the author)
Breiman, L. (2001) · Statistical Science · doi:10.1214/ss/1009213726 · Cited by 4,340
Purposeful selection of variables in logistic regression
Bursac, Z., Gauss, C. H., Williams, D. K., & Hosmer, D. W. (2008) · Source Code for Biology and Medicine · doi:10.1186/1751-0473-3-17 · Cited by 4,035
Causal Diagrams for Epidemiologic Research
Greenland, S., Pearl, J., & Robins, J. M. (1999) · Epidemiology · doi:10.1097/00001648-199901000-00008 · Cited by 3,728
Air Pollution and Hospital Admissions for Heart Disease in Eight U.S. Counties
Schwartz, J. (1999) · Epidemiology · doi:10.1097/00001648-199901000-00005 · Cited by 3,038
Shmueli, G. (2010) · Statistical Science · doi:10.1214/10-sts330 · Cited by 2,517
Simulation Study of Confounder-Selection Strategies
Maldonado, G., & Greenland, S. (1993) · American Journal of Epidemiology · doi:10.1093/oxfordjournals.aje.a116813 · Cited by 2,491
THE IMPACT OF CONFOUNDER SELECTION CRITERIA ON EFFECT ESTIMATION
MICKEY, R. M., & GREENLAND, S. (1989) · American Journal of Epidemiology · doi:10.1093/oxfordjournals.aje.a115101 · Cited by 2,362
Meinshausen, N., & Bühlmann, P. (2010) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/j.1467-9868.2010.00740.x · Cited by 2,215
RE:“;THE IMPACT OF CONFOUNDER SELECTION CRITERIA ON EFFECT ESTIMATION”
Greenland, S., & Mickey, R. M. (1989) · American Journal of Epidemiology · doi:10.1093/oxfordjournals.aje.a115409 · Cited by 2,108
Regression modelling strategies for improved prognostic prediction
Harrell, F. E., Lee, K. L., Califf, R. M., Pryor, D. B., & Rosati, R. A. (1984) · Statistics in Medicine · doi:10.1002/sim.4780030207 · Cited by 1,798
Maternal Pesticide Exposure from Multiple Sources and Selected Congenital Anomalies
Shaw, G. M., Wasserman, C. R., OʼMalley, C. D., Nelson, V., & Jackson, R. J. (1999) · Epidemiology · doi:10.1097/00001648-199901000-00011 · Cited by 1,784
The Gross Composition of the Body
SIRI, W. E. (1956) · Advances in Biological and Medical Physics · doi:10.1016/b978-1-4832-3110-5.50011-x · Cited by 1,765
Model Selection: An Integral Part of Inference
Buckland, S. T., Burnham, K. P., & Augustin, N. H. (1997) · Biometrics · doi:10.2307/2533961 · Cited by 1,717
Steyerberg, E. W. (2009) · Statistics for Biology and Health · doi:10.1007/978-0-387-77244-8 · Cited by 1,491
An asymptotic equivalence of choice of model by cross validation and Akaike's criterion
Stone, M. (1977) · Journal of the Royal Statistical Society Series B (Statistical Methodology) · doi:10.1111/j.2517-6161.1977.tb01603.x · Cited by 1,286
Risk prediction models: II. External validation, model updating, and impact assessment
Moons, K. G. M., Kengne, A. P., Grobbee, D. E., Royston, P., Vergouwe, Y., Altman, D. G., & Woodward, M. (2012) · Heart · doi:10.1136/heartjnl-2011-301247 · Cited by 1,195
Reporting Recommendations for Tumor Marker Prognostic Studies (REMARK): Explanation and Elaboration
Altman, D. G., McShane, L. M., Sauerbrei, W., & Taube, S. E. (2012) · PLoS Medicine · doi:10.1371/journal.pmed.1001216 · Cited by 1,013
Inappropriate use of bivariable analysis to screen risk factors for use in multivariable analysis
Sun, G.-W., Shook, T. L., & Kay, G. L. (1996) · Journal of Clinical Epidemiology · doi:10.1016/0895-4356(96)00025-x · Cited by 933
Boosting With the L2Loss: Regression and Classification
Bühlmann, P., & Yu, B. (2003) · Journal of the American Statistical Association · doi:10.1198/016214503000125 · Cited by 845
Empirical Model-Building and Response Surfaces.
Pike, D. J., Box, G. E. P., & Draper, N. R. (1988) · Journal of the Royal Statistical Society. Series A (Statistics in Society) · doi:10.2307/2982196 · Cited by 716
Royston, P., & Sauerbrei, W. (2008) · Wiley Series in Probability and Statistics · doi:10.1002/9780470770771 · Cited by 712
MODEL SELECTION AND INFERENCE: FACTS AND FICTION
Leeb, H., & Pötscher, B. M. (2005) · Econometric Theory · doi:10.1017/s0266466605050036 · Cited by 701
A bootstrap resampling procedure for model building: Application to the cox regression model
Sauerbrei, W., & Schumacher, M. (1992) · Statistics in Medicine · doi:10.1002/sim.4780111607 · Cited by 600
Five myths about variable selection
Heinze, G., & Dunkler, D. (2017) · Transplant International · doi:10.1111/tri.12895 · Cited by 528
Self-citationPrinciples of multilevel modelling
Greenland, S. (2000) · International Journal of Epidemiology · doi:10.1093/ije/29.1.158 · Cited by 481
Cox, D. R., & Hinkley, D. V. (1979) · Chapman & Hall/CRC eBooks · doi:10.1201/b14832 · Cited by 473
A New Criterion for Confounder Selection
VanderWeele, T. J., & Shpitser, I. (2011) · Biometrics · doi:10.1111/j.1541-0420.2011.01619.x · Cited by 434
Some Surprising Results about Covariate Adjustment in Logistic Regression Models
Robinson, L. D., & Jewell, N. P. (1991) · International Statistical Review / Revue Internationale de Statistique · doi:10.2307/1403444 · Cited by 410
Statistical learning and selective inference
Taylor, J., & Tibshirani, R. J. (2015) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1507583112 · Cited by 398
Commentary: Prognostic models: clinically useful or quickly forgotten?
Wyatt, J. C., & Altman, D. G. (1995) · BMJ · doi:10.1136/bmj.311.7019.1539 · Cited by 370
The Mathematical Principles of Natural Philosophy
MacMahon, J. J. (1965) · Philosophical Studies · doi:10.5840/philstudies19651431 · Cited by 369
Courvoisier, D. S., Combescure, C., Agoritsas, T., Gayet-Ageron, A., & Perneger, T. V. (2011) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2010.11.012 · Cited by 320
The Use of Resampling Methods to Simplify Regression Models in Medical Statistics
Sauerbrei, W. (1999) · Journal of the Royal Statistical Society Series C: Applied Statistics · doi:10.1111/1467-9876.00155 · Cited by 310
Invited Commentary: Variable Selection versus Shrinkage in the Control of Multiple Confounders
Greenland, S. (2007) · American Journal of Epidemiology · doi:10.1093/aje/kwm355 · Cited by 283
Why Stepdown Procedures in Variable Selection
Mantel, N. (1970) · Technometrics · doi:10.1080/00401706.1970.10488701 · Cited by 280
Statistical Foundations for Model-Based Adjustments
Greenland, S., & Pearce, N. (2015) · Annual Review of Public Health · doi:10.1146/annurev-publhealth-031914-122559 · Cited by 274
Augmented backward elimination: a pragmatic and purposeful way to develop statistical models
Dunkler, D., Plischke, M., Leffondré, K., & Heinze, G. (2014) · PLoS ONE · doi:10.1371/journal.pone.0113677 · Cited by 194
Self-citationIs a Cutoff of 10% Appropriate for the Change-in-Estimate Criterion of Confounder Identification?
Lee, P. H. (2014) · Journal of Epidemiology · doi:10.2188/jea.je20130062 · Cited by 189
On model selection and model misspecification in causal inference.
Vansteelandt, S., Bekaert, M., & Claeskens, G. (2012) · Statistical Methods in Medical Research · doi:10.1177/0962280210387717 · Cited by 172
STRengthening Analytical Thinking for Observational Studies: the STRATOS initiative
Sauerbrei, W., Abrahamowicz, M., Altman, D. G., le Cessie, S., Carpenter, J. R., & initiative, O. B. O. T. S. (2014) · Statistics in Medicine · doi:10.1002/sim.6265 · Cited by 147
A review of statistical updating methods for clinical prediction models
Su, T.-L., Jaki, T., Hickey, G. L., Buchan, I., & Sperrin, M. (2018) · Statistical Methods in Medical Research · doi:10.1177/0962280215626466 · Cited by 147
Covariate selection for the nonparametric estimation of an average treatment effect
de Luna, X., Waernbaum, I., & Richardson, T. S. (2011) · Biometrika · doi:10.1093/biomet/asr041 · Cited by 142
Applied Survival Analysis: Regression Modeling of Time to Event Data.
Muche, R. (2001) · International Journal of Epidemiology · doi:10.1093/ije/30.2.408 · Cited by 136
Fitting Percentage of Body Fat to Simple Body Measurements
Johnson, R. W. (1996) · Journal of Statistics Education · doi:10.1080/10691898.1996.11910505 · Cited by 135
Royston, P. J., & Sauerbrei, W. (2003) · Statistics in Medicine · doi:10.1002/sim.1310 · Cited by 124
Pathway-Based Genomics Prediction using Generalized Elastic Net
Sokolov, A., Carlin, D. E., Paull, E., Baertsch, R., & Stuart, J. M. (2016) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1004790 · Cited by 113
Subsampling Versus Bootstrapping in Resampling-Based Model Selection for Multivariable Regression
Bin, R. D., Janitza, S., Sauerbrei, W., & Boulesteix, A. (2015) · Biometrics · doi:10.1111/biom.12381 · Cited by 106
Regression with Linear Predictors
Andersen, P. K., & Skovgaard, L. T. (2010) · Statistics in the health sciences · doi:10.1007/978-1-4419-7170-8 · Cited by 90
Evans, D., Chaix, B., Lobbedez, T., Verger, C., & Flahault, A. (2012) · BMC Medical Research Methodology · doi:10.1186/1471-2288-12-156 · Cited by 90
Shrinkage and Penalized Likelihood as Methods to Improve Predictive Accuracy
van Houwelingen, H. C. (2001) · Statistica Neerlandica · doi:10.1111/1467-9574.00154 · Cited by 86
Binder, H., & Schumacher, M. C. (2009) · BMC Bioinformatics · doi:10.1186/1471-2105-10-18 · Cited by 82
On stability issues in deriving multivariable regression models
Sauerbrei, W., Buchholz, A., Boulesteix, A., & Binder, H. (2014) · Biometrical Journal · doi:10.1002/bimj.201300222 · Cited by 64
Schumacher, M. C., Holländer, N., Schwarzer, G., Binder, H., & Sauerbrei, W. (2012) · Chapman & Hall/CRC eBooks · doi:10.1201/b11800-30 · Cited by 57
Cross-Validation, Shrinkage and Variable Selection in Linear Regression Revisited
van Houwelingen, H. C., & Sauerbrei, W. (2013) · Open Journal of Statistics · doi:10.4236/ojs.2013.32011 · Cited by 50
Global, Parameterwise and Joint Shrinkage Factor Estimation
Dunkler, D., Sauerbrei, W., & Heinze, G. (2016) · Journal of Statistical Software · doi:10.18637/jss.v069.i08 · Cited by 38
Self-citationAugustin, N. H., Sauerbrei, W., & Schumacher, M. C. (2005) · Statistical Modelling · doi:10.1191/1471082x05st089oa · Cited by 36
Buchholz, A., Holländer, N., & Sauerbrei, W. (2007) · Computational Statistics & Data Analysis · doi:10.1016/j.csda.2007.10.007 · Cited by 31
Cited by
Calculating the sample size required for developing a clinical prediction model
Riley, R. D., Ensor, J., Snell, K. I. E., Harrell, F. E., Martin, G. P., Reitsma, J. B., Moons, K. G. M., Collins, G., & van Smeden, M. (2020) · BMJ · doi:10.1136/bmj.m441 · Cited by 2,917
Tennant, P. W. G., Murray, E. J., Arnold, K. F., Berrie, L., Fox, M. P., Gadd, S. C., Harrison, W. J., Keeble, C., Ranker, L. R., Textor, J., et al. (2021) · International Journal of Epidemiology · doi:10.1093/ije/dyaa213 · Cited by 1,052
Tyralis, H., Papacharalampous, G., & Langousis, A. (2019) · Water · doi:10.3390/w11050910 · Cited by 771
Stability of clinical prediction models developed using statistical or machine learning methods
Riley, R. D., & Collins, G. S. (2023) · Biometrical Journal · doi:10.1002/bimj.202200302 · Cited by 211
Smith, M., Manduchi, B., Burke, É., Carroll, R., McCallion, P., & McCarron, M. (2020) · Research in Developmental Disabilities · doi:10.1016/j.ridd.2019.103557 · Cited by 141
Metadata
- Authors
- Heinze, G., Wallisch, C., & Dunkler, D.
- Venue
- Biometrical Journal
- Publisher
- Wiley
- Published
- 2018-01-02
- Volume
- 60
- Issue
- 3
- Pages
- 431-449
- ISSN
- 0323-3847 (ISSN-L)
- 1521-4036
- OpenAlex
- W2782176193
- Cited by
- 1,639
- References
- 69
- FWCI
- 97.35
- Type
- journal article
- Language
- en
- Open access
- hybrid
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.