The DOI matches its Crossref record; 43 of the 43 references it cites were found in a registry or an index.
Correlation and variable importance in random forests
Open accessAbstract
This paper is about variable selection with the random forests algorithm in presence of correlated predictors. In high-dimensional regression or classification frameworks, variable selection is a difficult task, that becomes even more challenging in the presence of highly correlated predictors. Firstly we provide a theoretical study of the permutation importance measure for an additive regression model. This allows us to describe how the correlation between predictors impacts the permutation importance. Our results motivate the use of the recursive feature elimination (RFE) algorithm for variable selection in this context. This algorithm recursively eliminates the variables using permutation importance measure as a ranking criterion. Next various simulation experiments illustrate the efficiency of the RFE algorithm for selecting a small number of variables together with a good prediction error. Finally, this selection algorithm is tested on the Landsat Satellite data from the UCI Machine Learning Repository.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“To avoid the importance dilution problem between correlated parameters (Gregorutti et al., 2017), we first built the Random Forest model to select the most important parameter in each subcategory, and this process resulted in 59 parameters, which were then used as the input for the final Random Forest models.”
From How Have Global River Widths Changed Over Time? (2022) · cites this paper as Gregorutti et al., 2017
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Breiman, L. (2001) · Machine Learning · doi:10.1023/a:1010933404324 · Cited by 133,538
Regression shrinkage and selection via the lasso
Tibshirani, R. (1996) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/j.2517-6161.1996.tb02080.x · Cited by 51,482
Classification and Regression Trees.
Gordon, A. D., Breiman, L., Friedman, J. H., Olshen, R. A., & Stone, C. J. (1984) · Biometrics · doi:10.2307/2530946 · Cited by 23,522
Classification and Regression Trees.
Ryzin, J. V., Breiman, L., Friedman, J. H., Olshen, R. A., & Stone, C. J. (1986) · Journal of the American Statistical Association · doi:10.2307/2288003 · Cited by 20,825
Breiman, L. (1996) · Machine Learning · doi:10.1007/bf00058655 · Cited by 16,325
Linear Statistical Inference and its Applications: Second Editon
Rao, C. R. (1973) · Wiley Series in Probability and Statistics · doi:10.1002/9780470316436 · Cited by 10,489
Gene selection for cancer classification using support vector machines
Guyon, I., Weston, J., Barnhill, S., & Vapnik, V. (2002) · Machine Learning · doi:10.1023/a:1012487302797 · Cited by 9,954 · cited 4×
“The second algorithm was first proposed by Guyon et al. [14] in the context of support vector machines (SVM) and is referred to as Recursive Feature Elimination (RFE).”
MethodInfluentialWrappers for feature subset selection
Kohavi, R., & John, G. H. (1997) · Artificial Intelligence · doi:10.1016/s0004-3702(97)00043-x · Cited by 8,947
An introduction to variable and feature selection
GuyonIsabelle, & ElisseeffAndré (2003) · Journal of Machine Learning Research · doi:10.5555/944919.944968 · Cited by 7,715
Linear Statistical Inference and Its Applications.
Robertson, C. A., & Rao, C. R. (1975) · Biometrics · doi:10.2307/2529568 · Cited by 3,598
Selection of relevant features and examples in machine learning
Blum, A. L., & Langley, P. (1997) · Artificial Intelligence · doi:10.1016/s0004-3702(97)00063-5 · Cited by 3,298
Conditional variable importance for random forests
Strobl, C., Boulesteix, A.-L., Kneib, T., Augustin, T., & Zeileis, A. (2008) · BMC Bioinformatics · doi:10.1186/1471-2105-9-307 · Cited by 3,254
Gene selection and classification of microarray data using random forest
Díaz-Uriarte, R., & Alvarez de Andrés, S. (2006) · BMC Bioinformatics · doi:10.1186/1471-2105-7-3 · Cited by 2,929
Variable selection using random forests
Genuer, R., Poggi, J.-M., & Tuleau-Malot, C. (2010) · Pattern Recognition Letters · doi:10.1016/j.patrec.2010.03.014 · Cited by 2,619
Meinshausen, N., & Bühlmann, P. (2010) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/j.1467-9868.2010.00740.x · Cited by 2,215
Linear Statistical Inference and its Applications.
Sprent, P., & Rao, C. R. (1974) · Journal of the Royal Statistical Society. Series A (General) · doi:10.2307/2345151 · Cited by 1,747
Selection bias in gene extraction on the basis of microarray gene-expression data
Ambroise, C., & McLachlan, G. J. (2002) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.102102699 · Cited by 1,480
Empirical characterization of random forest variable importance measures
Archer, K. J., & Kimes, R. V. (2008) · Computational Statistics & Data Analysis · doi:10.1016/j.csda.2007.08.015 · Cited by 1,123
Variable Importance Assessment in Regression: Linear Regression versus Random Forest
Grömping, U. (2009) · The American Statistician · doi:10.1198/tast.2009.08199 · Cited by 1,103
Variable selection using svm based criteria
Rakotomamonjy, A. (2003) · Journal of Machine Learning Research · doi:10.5555/944919.944977 · Cited by 712
Consistency of Random Forests and Other Averaging Classifiers
Gérard Biau, A., Devroye, L., & Lugosi, G. (2008) · HAL · Cited by 678
A survey on filter techniques for feature selection in gene expression microarray analysis
Lazar, C., Taminau, J., Meganck, S., Steenhoff, D., Coletta, A., Molter, C., de Schaetzen, V., Duque, R., Bersini, H., & Nowe, A. (2012) · IEEE/ACM Transactions on Computational Biology and Bioinformatics · doi:10.1109/tcbb.2012.33 · Cited by 619
Stability of feature selection algorithms: a study on high-dimensional spaces
Kalousis, A., Prados, J., & Hilario, M. (2007) · Knowledge and Information Systems · doi:10.1007/s10115-006-0040-8 · Cited by 614
Scornet, E., Biau, G., & Vert, J.-P. (2015) · The Annals of Statistics · doi:10.1214/15-aos1321 · Cited by 612
Classification with correlated features: unreliability of feature ranking and solutions
Toloşi, L., & Lengauer, T. (2011) · Bioinformatics · doi:10.1093/bioinformatics/btr300 · Cited by 445
Dimensionality Reduction via Sparse Support Vector Machines
Bi, J., Bennett, K. P., Embrechts, M. J., Breneman, C. M., Song, M., Guyon, I., & Elisseeff, A. (2003) · Cited by 407
Haury, A.-C., Gestraud, P., & Vert, J.-P. (2011) · PLoS ONE · doi:10.1371/journal.pone.0028210 · Cited by 392
Variable importance in binary regression trees and forests
Ishwaran, H. (2012) · Cited by 379
Nicodemus, K. K., Malley, J. D., Strobl, C., & Ziegler, A. (2010) · BMC Bioinformatics · doi:10.1186/1471-2105-11-110 · Cited by 376
Variable importance in binary regression trees and forests
Ishwaran, H. (2007) · Electronic Journal of Statistics · doi:10.1214/07-ejs039 · Cited by 353
Jiang, H., Deng, Y., Chen, H.-S., Tao, L., Sha, Q., Chen, J., Tsai, C.-J., & Zhang, S. (2004) · BMC Bioinformatics · doi:10.1186/1471-2105-5-81 · Cited by 290
Svetnik, V., Liaw, A., Tong, C., & Wang, T. (2004) · Lecture Notes in Computer Science · doi:10.1007/978-3-540-25966-4_33 · Cited by 288
A new variable selection approach using Random Forests
Hapfelmeier, A., & Ulm, K. (2013) · Computational Statistics & Data Analysis · doi:10.1016/j.csda.2012.09.020 · Cited by 259
Correlated variables in regression: Clustering and sparse estimation
Bühlmann, P., Rütimann, P., van de Geer, S., & Zhang, C. (2013) · Journal of Statistical Planning and Inference · doi:10.1016/j.jspi.2013.05.019 · Cited by 209
Overfitting in making comparisons between variable selection methods
ReunanenJuha (2003) · Journal of Machine Learning Research · doi:10.5555/944919.944978 · Cited by 205
Nicodemus, K. K. (2011) · Briefings in Bioinformatics · doi:10.1093/bib/bbr016 · Cited by 204
Predictor correlation impacts machine learning algorithms: implications for genomic studies
Nicodemus, K. K., & Malley, J. D. (2009) · Bioinformatics · doi:10.1093/bioinformatics/btp331 · Cited by 170
Zhu, R., Zeng, D., & Kosorok, M. R. (2015) · Journal of the American Statistical Association · doi:10.1080/01621459.2015.1036994 · Cited by 168
Statistical Inference for Variable Importance
van der Laan, M. J. (2006) · The International Journal of Biostatistics · doi:10.2202/1557-4679.1008 · Cited by 165
Empirical comparison of tree ensemble variable importance measures
Auret, L., & Aldrich, C. (2010) · Chemometrics and Intelligent Laboratory Systems · doi:10.1016/j.chemolab.2010.12.004 · Cited by 125
Chen, D. S., Barry, A. E., Leliwa‐Sytek, A., Smith, T.-A., Peterson, I., Brown, S. M., Migot‐Nabias, F., Deloron, P., Kortok, M., Marsh, K., et al. (2011) · PLoS ONE · doi:10.1371/journal.pone.0016629 · Cited by 116
Statistical Classification of Abnormal Blood Profiles in Athletes
Sottas, P., Robinson, N., Giraud, S., et al. (2006) · The International Journal of Biostatistics · doi:10.2202/1557-4679.1011 · Cited by 99
Zeng, D., Zhu, R., & Kosorok, M. R. (2020) · Carolina Digital Repository (University of North Carolina at Chapel Hill) · doi:10.17615/v9dj-6b88 · Cited by 8
Cited by
Biau, G., & Scornet, E. (2016) · Test · doi:10.1007/s11749-016-0481-7 · Cited by 4,149
Fisher, A., Rudin, C., & Dominici, F. (2018) · arXiv · doi:10.48550/arxiv.1801.01489 · Cited by 1,165
Evaluation of variable selection methods for random forests and omics data sets
Degenhardt, F., Seifert, S., & Szymczak, S. (2019) · Briefings in Bioinformatics · doi:10.1093/bib/bbx124 · Cited by 724
Machine learning pipeline for battery state-of-health estimation
Roman, D., Saxena, S., Robu, V., Pecht, M., & Flynn, D. (2021) · Nature Machine Intelligence · doi:10.1038/s42256-021-00312-3 · Cited by 660
VSURF: An R Package for Variable Selection Using Random Forests
Genuer, R., Poggi, J.-M., & Tuleau-Malot, C. (2015) · The R Journal · doi:10.32614/rj-2015-018 · Cited by 619
Metadata
- Authors
- Gregorutti, B., Michel, B., & Saint-Pierre, P.
- Venue
- Statistics and Computing
- Published
- 2016-03-23
- Volume
- 27
- Issue
- 3
- Pages
- 659-678
- ISSN
- 0960-3174 (ISSN-L)
- 1573-1375
- OpenAlex
- W2169281690
- Cited by
- 1,073
- References
- 43
- FWCI
- 37.33
- Type
- journal article
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.