The DOI matches its Crossref record; 44 of the 44 references it cites were found in a registry or an index.
Comparison of variable selection methods for clinical predictive modeling
Open accessAbstract
Objective Modern machine learning-based modeling methods are increasingly applied to clinical problems. One such application is in variable selection methods for predictive modeling. However, there is limited research comparing the performance of classic and modern for variable selection in clinical datasets. Materials and Methods We analyzed the performance of eight different variable selection methods: four regression-based methods (stepwise backward selection using p-value and AIC, Least Absolute Shrinkage and Selection Operator, and Elastic Net) and four tree-based methods (Variable Selection Using Random Forest, Regularized Random Forests, Boruta, and Gradient Boosted Feature Selection). We used two clinical datasets of different sizes, a multicenter adult clinical deterioration cohort and a single center pediatric acute kidney injury cohort. Method evaluation included measures of parsimony, variable importance, and discrimination. Results In the large, multicenter dataset, the modern tree-based Variable Selection Using Random Forest and the Gradient Boosted Feature Selection methods achieved the best parsimony. In the smaller, single-center dataset, the classic regression-based stepwise backward selection using p-value and AIC methods achieved the best parsimony. In both datasets, variable selection tended to decrease the accuracy of the random forest models and increase the accuracy of logistic regression models. Conclusions The performance of classic regression-based and modern tree-based variable selection methods is associated with the size of the clinical dataset used. Classic regression-based variable selection methods seem to achieve better parsimony in clinical prediction problems in smaller datasets while modern tree-based methods perform better in larger datasets.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Regression shrinkage and selection via the lasso
Tibshirani, R. (1996) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/j.2517-6161.1996.tb02080.x · Cited by 51,482
The Elements of Statistical Learning
Hastie, T., Tibshirani, R., & Friedman, J. (2009) · Springer Series in Statistics · doi:10.1007/978-0-387-84858-7 · Cited by 25,970
Regularization and Variable Selection Via the Elastic Net
Zou, H., & Hastie, T. (2005) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/j.1467-9868.2005.00503.x · Cited by 20,957
mice: Multivariate Imputation by Chained Equations in R
Buuren, S. V., & Groothuis-Oudshoorn, K. (2011) · Journal of Statistical Software · doi:10.18637/jss.v045.i03 · Cited by 13,870
APACHE II: A severity of disease classification system
KNAUS, W. A., DRAPER, E. A., WAGNER, D. P., & ZIMMERMAN, J. E. (1985) · Critical Care Medicine · doi:10.1097/00003246-198510000-00009 · Cited by 13,664
APACHE II-A Severity of Disease Classification System: Reply
Knaus, W. A., Draper, E. A., Wagner, D. P., & Zimmerman, J. E. (1986) · Critical Care Medicine · doi:10.1097/00003246-198608000-00028 · Cited by 13,376
Vincent, J. L., Moreno, R., Takala, J., Willatts, S., De Mendonça, A., Bruining, H., Reinhart, C. K., Suter, P. M., & Thijs, L. G. (1996) · Intensive Care Medicine · doi:10.1007/bf01709751 · Cited by 11,674
Harrell (2015) · Springer Series in Statistics · doi:10.1007/978-3-319-19425-7 · Cited by 6,878
(2010) · University of Twente Research Information · Cited by 6,037
APACHE II: a severity of disease classification system.
Knaus, W. A., Draper, E. S., Wagner, D. P., & Zimmerman, J. E. (1985) · PubMed · Cited by 6,034
Feature Selection with the Boruta Package
Kursa, M. B., & Rudnicki, W. R. (2010) · Journal of Statistical Software · doi:10.18637/jss.v036.i11 · Cited by 5,659
A review of feature selection techniques in bioinformatics
Saeys, Y., Inza, I., & Larrañaga, P. (2007) · Bioinformatics · doi:10.1093/bioinformatics/btm344 · Cited by 5,244
Purposeful selection of variables in logistic regression
Bursac, Z., Gauss, C. H., Williams, D. K., & Hosmer, D. W. (2008) · Source Code for Biology and Medicine · doi:10.1186/1751-0473-3-17 · Cited by 4,035
Bias in random forest variable importance measures: Illustrations, sources and a solution
Strobl, C., Boulesteix, A.-L., Zeileis, A., & Hothorn, T. (2007) · BMC Bioinformatics · doi:10.1186/1471-2105-8-25 · Cited by 3,688
Clinical Prediction Models: A Practical Approach to Development, Validation, and Updating
Steyerberg, E. W. (2019) · Statistics for Biology and Health · doi:10.1007/978-3-030-16399-0 · Cited by 3,281
Variable selection using random forests
Genuer, R., Poggi, J.-M., & Tuleau-Malot, C. (2010) · Pattern Recognition Letters · doi:10.1016/j.patrec.2010.03.014 · Cited by 2,619
PRISM III: An updated Pediatric Risk of Mortality score
Pollack, M. M., Patel, K. M., & Ruttimann, U. E. (1996) · Critical Care Medicine · doi:10.1097/00003246-199605000-00004 · Cited by 1,759
The Lack of A Priori Distinctions Between Learning Algorithms
Wolpert, D. H. (1996) · Neural Computation · doi:10.1162/neco.1996.8.7.1341 · Cited by 1,647
The elements of statistical learning: Data mining, inference, and prediction
Franklin, J. (2005) · The Mathematical Intelligencer · doi:10.1007/bf02985802 · Cited by 1,400 · cited 3×
“The goal of variable selection methods is to increase the signal-to-noise ratio in the data in order to develop appropriately fitted models that will make accurate predictions when new, unseen data is used as input [7, 8].”
BackgroundMethodBest care at lower cost: the path to continuously learning health care in America
(2014) · Choice Reviews Online · doi:10.5860/choice.51-3277 · Cited by 1,363
Big Data In Health Care: Using Analytics To Identify And Manage High-Risk And High-Cost Patients
Bates, D. W., Saria, S., Ohno-Machado, L., Shah, A., & Escobar, G. (2014) · Health Affairs · doi:10.1377/hlthaff.2014.0041 · Cited by 1,235
Sauerbrei, W., Royston, P., & Binder, H. (2007) · Statistics in Medicine · doi:10.1002/sim.3148 · Cited by 1,171
Predictive data mining in clinical medicine: Current issues and guidelines
BELLAZZI, R., & ZUPAN, B. (2008) · International Journal of Medical Informatics · doi:10.1016/j.ijmedinf.2006.11.006 · Cited by 823
Modern modelling techniques are data hungry: a simulation study for predicting dichotomous endpoints
van der Ploeg, T., Austin, P. C., & Steyerberg, E. W. (2014) · BMC Medical Research Methodology · doi:10.1186/1471-2288-14-137 · Cited by 809 · cited 5×
“However, there has been limited research comparing the performance of classic and modern modeling methods in clinical datasets [5].”
BackgroundMethodResultInfluentialClinical Prediction Models: A Practical Approach to Development, Validation and Updating
Mann, C. J. H. (2009) · Kybernetes · doi:10.1108/k.2009.06738fae.002 · Cited by 783
PELOD-2: An Update of the PEdiatric Logistic Organ Dysfunction Score
Leteurtre, S., Duhamel, A., Salleron, J., Grandbastien, B., Lacroix, J., & Leclerc, F. (2013) · Critical Care Medicine · doi:10.1097/ccm.0b013e31828a2bbd · Cited by 677
Churpek, M. M., Yuen, T. C., Winslow, C., Meltzer, D. O., Kattan, M. W., & Edelson, D. P. (2016) · Critical Care Medicine · doi:10.1097/ccm.0000000000001571 · Cited by 638
Self-citationMICE: Multivariate Imputation by Chained Equations in R
van Buuren, S., & Groothuis-Oudshoorn, K. (2010) · Utrecht University Repository (Utrecht University) · Cited by 496
Stepwise Selection in Small Data Sets A Simulation Study of Bias in Logistic Regression Analysis
Steyerberg, E. (1999) · Journal of Clinical Epidemiology · doi:10.1016/s0895-4356(99)00103-1 · Cited by 495
Haury, A.-C., Gestraud, P., & Vert, J.-P. (2011) · PLoS ONE · doi:10.1371/journal.pone.0028210 · Cited by 392
Franklin, J. S. (2005) · The Mathematical Intelligencer · Cited by 392
Feature selection methods for big data bioinformatics: A survey from the search perspective
Wang, L., Wang, Y., & Chang, Q. (2016) · Methods · doi:10.1016/j.ymeth.2016.08.014 · Cited by 331
Multicenter Development and Validation of a Risk Stratification Tool for Ward Patients
Churpek, M. M., Yuen, T. C., Winslow, C., Robicsek, A. A., Meltzer, D. O., Gibbons, R. D., & Edelson, D. P. (2014) · American Journal of Respiratory and Critical Care Medicine · doi:10.1164/rccm.201406-1022oc · Cited by 259
Self-citationVariable selection: current practice in epidemiological studies
Walter, S., & Tiemeier, H. (2009) · European Journal of Epidemiology · doi:10.1007/s10654-009-9411-2 · Cited by 228
Gradient boosted feature selection
Xu, Z., Huang, G., Weinberger, K. Q., & Zheng, A. X. (2014) · Proceedings of the 20th ACM SIGKDD international conference on Knowledge discovery and data mining · doi:10.1145/2623330.2623635 · Cited by 194
O'Quigley, J., & Chevret, S. (1991) · Statistics in Medicine · doi:10.1002/sim.4780101104 · Cited by 179
Bagherzadeh-Khiabani, F., Ramezankhani, A., Azizi, F., Hadaegh, F., Steyerberg, E. W., & Khalili, D. (2016) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2015.10.002 · Cited by 163
Cho, B. H., Yu, H., Kim, K., Kim, T. H., Kim, I. Y., & Kim, S. I. (2007) · Artificial Intelligence in Medicine · doi:10.1016/j.artmed.2007.09.005 · Cited by 159
Cost-Sensitive Tree of ClassifiersPreprint
Xu, Z., Kusner, M. J., Weinberger, K. Q., & Chen, M. (2012) · arXiv (Cornell University) · doi:10.48550/arxiv.1210.2771 · Cited by 113
Data Mining on Imbalanced Data Sets
Gu, Q., Cai, Z., Zhu, L., & Huang, B. (2008) · International Conference on Advanced Computer Theory and Engineering · doi:10.1109/icacte.2008.26 · Cited by 112
Performance of using multiple stepwise algorithms for variable selection
Wiegand, R. E. (2010) · Statistics in Medicine · doi:10.1002/sim.3943 · Cited by 93
Sanchez-Pinto, L. N., & Khemani, R. G. (2016) · Pediatric Critical Care Medicine · doi:10.1097/pcc.0000000000000750 · Cited by 52 · cited 3×
“…the performance of eight different variable selection methods, both regression-based and tree-based methods, in two clinical datasets of different sizes used for predictive modeling: a multicenter adult clinical deterioration cohort and a single center pediatric acute kidney injury cohort [14, 15].”
BackgroundMethodSelf-citationEvaluating variable selection methods for diagnosis of myocardial infarction.
Dreiseitl, S., Ohno‐Machado, L., & Vinterbo, S. A. (1999) · PubMed · Cited by 21
van der Ploeg, T., & Steyerberg, E. W. (2016) · BMC Research Notes · doi:10.1186/s13104-016-1945-2 · Cited by 20
Cited by
A comparison of random forest variable selection methods for classification prediction modeling
Speiser, J. L., Miller, M. E., Tooze, J., & Ip, E. (2019) · Expert Systems with Applications · doi:10.1016/j.eswa.2019.05.028 · Cited by 1,561
The potential of artificial intelligence to improve patient safety: a scoping review
Bates, D. W., Levine, D., Syrowatka, A., Kuznetsova, M., Craig, K. J. T., Rui, A., Jackson, G. P., & Rhee, K. (2021) · npj Digital Medicine · doi:10.1038/s41746-021-00423-6 · Cited by 323
Arnoriaga-Rodríguez, M., Mayneris-Perxachs, J., Burokas, A., Contreras-Rodríguez, O., Blasco, G., Coll, C., Biarnés, C., Miranda-Olivos, R., Latorre, J., Moreno-Navarrete, J.-M., et al. (2020) · Cell Metabolism · doi:10.1016/j.cmet.2020.09.002 · Cited by 157
Lima-Filho, L. M. A., Lopes, L. W., & Filho, T. D. M. E. S. (2024) · Journal of Voice · doi:10.1016/j.jvoice.2024.11.002 · Cited by 3
Metadata
- Authors
- Sanchez-Pinto, L. N., Venable, L. R., Fahrenbach, J., & Churpek, M. M.
- Venue
- International Journal of Medical Informatics
- Publisher
- Elsevier BV
- Published
- 2018-08
- Volume
- 116
- Pages
- 10-17
- ISSN
- 1386-5056 (ISSN-L)
- OpenAlex
- W2804586924
- Cited by
- 276
- References
- 44
- FWCI
- 16.12
- Type
- journal article
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.