The DOI matches its Crossref record.
Backward, forward and stepwise automated subset selection algorithms: Frequency of obtaining authentic and noise variables
Abstract
The use of automated subset search algorithms is reviewed and issues concerning model selection and selection criteria are discussed. In addition, a Monte Carlo study is reported which presents data regarding the frequency with which authentic and noise variables are selected by automated subset algorithms. In particular, the effects of the correlation between predictor variables, the number of candidate predictor variables, the size of the sample, and the level of significance for entry and deletion of variables were studied for three automated subset algorithms: BACKWARD ELIMINATION, FORWARD SELECTION, and STEPWISE. Results indicated that: (1) the degree of correlation between the predictor variables affected the frequency with which authentic predictor variables found their way into the final model; (2) the number of candidate predictor variables affected the number of noise variables that gained entry to the model; (3) the size of the sample was of little practical importance in determining the number of authentic variables contained in the final model; and (4) the population multiple coefficient of determination could be faithfully estimated by adopting a statistic that is adjusted by the total number of candidate predictor variables rather than the number of variables in the final model.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cited by
Strobl, C., Malley, J., & Tutz, G. (2009) · Psychological Methods · doi:10.1037/a0016973 · Cited by 2,583
FORWARD SELECTION OF EXPLANATORY VARIABLES
Blanchet, F. G., Legendre, P., & Borcard, D. (2008) · Ecology · doi:10.1890/07-0986.1 · Cited by 2,116
Towards better clinical prediction models: seven steps for development and an ABCD for validation
Steyerberg, E. W., & Vergouwe, Y. (2014) · European Heart Journal · doi:10.1093/eurheartj/ehu207 · Cited by 1,928
Tu, J. V. (1996) · Journal of Clinical Epidemiology · doi:10.1016/s0895-4356(96)00002-9 · Cited by 1,818
Babyak, M. A. (2004) · Psychosomatic Medicine · doi:10.1097/01.psy.0000127692.23278.a9 · Cited by 2,061
Why do we still use stepwise modelling in ecology and behaviour?
WHITTINGHAM, M. J., STEPHENS, P. A., BRADBURY, R. B., & FRECKLETON, R. P. (2006) · Journal of Animal Ecology · doi:10.1111/j.1365-2656.2006.01141.x · Cited by 1,520
A Baseline for the Multivariate Comparison of Resting-State Networks
Allen, E. A., Erhardt, E. B., Damaraju, E., Gruner, W., Segall, J. M., Silva, R. F., Havlicek, M., Rachakonda, S., Fries, J., Kalyanam, R., et al. (2011) · Frontiers in Systems Neuroscience · doi:10.3389/fnsys.2011.00002 · Cited by 1,293
Lindbom, L., Pihlgren, P., & Jonsson, N. (2005) · Computer Methods and Programs in Biomedicine · doi:10.1016/j.cmpb.2005.04.005 · Cited by 1,135
Forstmeier, W., & Schielzeth, H. (2011) · Behavioral Ecology and Sociobiology · doi:10.1007/s00265-010-1038-5 · Cited by 1,120
Inappropriate use of bivariable analysis to screen risk factors for use in multivariable analysis
Sun, G.-W., Shook, T. L., & Kay, G. L. (1996) · Journal of Clinical Epidemiology · doi:10.1016/0895-4356(96)00025-x · Cited by 933
Moons, K. G. M., Kengne, A. P., Woodward, M., Royston, P., Vergouwe, Y., Altman, D. G., & Grobbee, D. E. (2012) · Heart · doi:10.1136/heartjnl-2011-301246 · Cited by 889
Steyerberg, E. W., Eijkemans, M. J. C., Harrell, F. E., & Habbema, J. D. F. (2000) · Statistics in Medicine · doi:10.1002/(sici)1097-0258(20000430)19:8<1059::aid-sim412>3.0.co;2-0 · Cited by 785
Bootstrap Methods for Developing Predictive Models
Austin, P. C., & Tu, J. V. (2004) · The American Statistician · doi:10.1198/0003130043277 · Cited by 622
Modeling causes of death: an integrated approach using CODEm
Foreman, K. J., Lozano, R., Lopez, A. D., & Murray, C. J. (2012) · Population Health Metrics · doi:10.1186/1478-7954-10-1 · Cited by 564
Steyerberg, E. W., Eijkemans, M. J. C., Harrell, F. E., & Habbema, J. D. F. (2001) · Medical Decision Making · doi:10.1177/0272989x0102100106 · Cited by 550
Smith, G. (2018) · Journal of Big Data · doi:10.1186/s40537-018-0143-6 · Cited by 532
Stepwise Selection in Small Data Sets A Simulation Study of Bias in Logistic Regression Analysis
Steyerberg, E. (1999) · Journal of Clinical Epidemiology · doi:10.1016/s0895-4356(99)00103-1 · Cited by 495
How should variable selection be performed with multiply imputed data?
Wood, A. M., White, I. R., & Royston, P. (2008) · Statistics in Medicine · doi:10.1002/sim.3177 · Cited by 485
Stepwise Model Fitting and Statistical Inference: Turning Noise into Signal Pollution
Mundry, R., & Nunn, C. L. (2009) · The American Naturalist · doi:10.1086/593303 · Cited by 440
McNeish, D. M. (2015) · Multivariate Behavioral Research · doi:10.1080/00273171.2015.1036965 · Cited by 432
The confounding effect of class size on the validity of object-oriented metrics
El Emam, K., Benlarbi, S., Goel, N., & Rai, S. N. (2001) · IEEE Transactions on Software Engineering · doi:10.1109/32.935855 · Cited by 422
Johnson, C. J., Seip, D. R., & Boyce, M. S. (2004) · Journal of Applied Ecology · doi:10.1111/j.0021-8901.2004.00899.x · Cited by 375
Information theory and hypothesis testing: a call for pluralism
STEPHENS, P. A., BUSKIRK, S. W., HAYWARD, G. D., & MARTÍNEZ DEL RIO, C. (2005) · Journal of Applied Ecology · doi:10.1111/j.1365-2664.2005.01002.x · Cited by 360
The prediction of faulty classes using object-oriented design metrics
El Emam, K., Melo, W., & Machado, J. C. (2001) · Journal of Systems and Software · doi:10.1016/s0164-1212(00)00086-8 · Cited by 359
Advanced Statistics: Linear Regression, Part II: Multiple Linear Regression
Marill, K. A. (2004) · Academic Emergency Medicine · doi:10.1111/j.1553-2712.2004.tb01379.x · Cited by 341
Austin, P. C., & Tu, J. V. (2004) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2004.04.003 · Cited by 341
Consequences of spatial autocorrelation for niche‐based models
SEGURADO, P., ARAÚJO, M. B., & KUNIN, W. E. (2006) · Journal of Applied Ecology · doi:10.1111/j.1365-2664.2006.01162.x · Cited by 335
WINTLE, B. A., ELITH, J., & POTTS, J. M. (2005) · Austral Ecology · doi:10.1111/j.1442-9993.2005.01514.x · Cited by 305
Presenting logistic regression-based landslide susceptibility results
Lombardo, L., & Mai, P. M. (2018) · Engineering Geology · doi:10.1016/j.enggeo.2018.07.019 · Cited by 290
The Use of Bayesian Model Averaging to Better Represent Uncertainty in Ecological Models
WINTLE, B. A., McCARTHY, M. A., VOLINSKY, C. T., & KAVANAGH, R. P. (2003) · Conservation Biology · doi:10.1111/j.1523-1739.2003.00614.x · Cited by 286
Bayesian Model Averaging in Proportional Hazard Models: Assessing the Risk of a Stroke
Volinsky, C. T., Madigan, D., Raftery, A. E., & Kronmal, R. A. (1997) · Journal of the Royal Statistical Society Series C: Applied Statistics · doi:10.1111/1467-9876.00082 · Cited by 226
“Derksen and Keselman (1992) provided a literature review of studies which critique stepwise methods for linear regression.”
BackgroundMultivariable Modeling Strategies
Harrell, F. E. (2015) · Springer Series in Statistics · doi:10.1007/978-3-319-19425-7_4 · Cited by 221
Does cognitive functioning predict chronic pain? Results from a prospective surgical cohort
Attal, N., Masselin-Dubois, A., Martinez, V., Jayr, C., Albi, A., Fermanian, J., Bouhassira, D., & Baudic, S. (2014) · Brain · doi:10.1093/brain/awt354 · Cited by 204
What makes articles highly cited?
Antonakis, J., Bastardoz, N., Liu, Y., & Schriesheim, C. A. (2014) · The Leadership Quarterly · doi:10.1016/j.leaqua.2013.10.014 · Cited by 148
Italian consumers’ income and food waste behavior
Setti, M., Falasconi, L., Segrè, A., Cusano, I., & Vittuari, M. (2016) · British Food Journal · doi:10.1108/bfj-11-2015-0427 · Cited by 145
Factors affecting citation rates in environmental science
Vanclay, J. K. (2013) · Journal of Informetrics · doi:10.1016/j.joi.2012.11.009 · Cited by 134
Khan, P. M., & Roy, K. (2018) · Expert Opinion on Drug Discovery · doi:10.1080/17460441.2018.1542428 · Cited by 113
Decision‐making in clinical nursing: investigating contributing factors
Hoffman, K., Donoghue, J., & Duffield, C. (2004) · Journal of Advanced Nursing · doi:10.1046/j.1365-2648.2003.02860.x · Cited by 108
The misuse and abuse of statistics in biomedical research
Thiese, M. S., Arnold, Z. C., & Walker, S. D. (2015) · Biochemia Medica · doi:10.11613/bm.2015.001 · Cited by 87
“Purely statistical approaches to model building, including forward and backward stepwise building may result in different “final” main effects models, both in relation to variables included and relationships identified (35,36).”
MethodOubre, B., Daneault, J.-F., Jung, H.-T., Whritenour, K., Miranda, J. G. V., Park, J., Ryu, T., Kim, Y., & Lee, S. I. (2020) · IEEE Transactions on Neural Systems and Rehabilitation Engineering · doi:10.1109/tnsre.2020.2966950 · Cited by 78
Machine Learning Approach to Simulate Soil CO2 Fluxes under Cropping Systems
Adjuik, T. A., & Davis, S. C. (2022) · Agronomy · doi:10.3390/agronomy12010197 · Cited by 56
Multivariate regression: the pitfalls of automated variable selection
Sainani, K. L. (2013) · PM&R · doi:10.1016/j.pmrj.2013.07.007 · Cited by 43
How Time Pressure in Different Phases of Decision-Making Influences Human-AI Collaboration
Cao, S., Gomez, C., & Huang, C.-M. (2023) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3610068 · Cited by 48
Hahlbeck, N., Scales, K. L., Dewar, H., Maxwell, S. M., Bograd, S. J., & Hazen, E. L. (2017) · Fisheries Research · doi:10.1016/j.fishres.2017.03.011 · Cited by 34
Finding Associations in Dense Genetic Maps: A Genetic Algorithm Approach
Clark, T. G., De Iorio, M., Griffiths, R. C., & Farrall, M. (2005) · Human Heredity · doi:10.1159/000088845 · Cited by 18
Trinh, M., Khou, V., Kalloniatis, M., & Nivison-Smith, L. (2021) · Investigative Opthalmology & Visual Science · doi:10.1167/iovs.62.13.13 · Cited by 14
Roalkvam, I. (2020) · Journal of Computer Applications in Archaeology · doi:10.5334/jcaa.60 · Cited by 7
Montero-Zamora, P., Brown, E. C., Ringwalt, C. L., Schwartz, S. J., Prado, G., & Ortiz-García, J. (2022) · Prevention Science · doi:10.1007/s11121-021-01301-z · Cited by 6
Harrell, F., Jr. (1996) · doi:10.1002/(sici)1097-0258(19960229)15:4<361::aid-sim168>3.0.co;2-4
Metadata
- Authors
- Derksen, S., & Keselman, H. J.
- Venue
- British Journal of Mathematical and Statistical Psychology
- Publisher
- Wiley
- Published
- 1992-11
- Volume
- 45
- Issue
- 2
- Pages
- 265-282
- ISSN
- 0007-1102 (ISSN-L)
- 2044-8317
- OpenAlex
- W2052779929
- Cited by
- 771
- References
- 0
- FWCI
- 0.46
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.