The DOI matches its Crossref record; 76 of the 76 references it cites were found in a registry or an index.
Benchmark for filter methods for feature selection in high-dimensional classification data
Open accessAbstract
Feature selection is one of the most fundamental problems in machine learning and has drawn increasing attention due to high-dimensional data sets emerging from different fields like bioinformatics. For feature selection, filter methods play an important role, since they can be combined with any machine learning model and can heavily reduce run time of machine learning algorithms. The aim of the analyses is to review how different filter methods work, to compare their performance with respect to both run time and predictive accuracy, and to provide guidance for applications. Based on 16 high-dimensional classification data sets, 22 filter methods are analyzed with respect to run time and accuracy when combined with a classification method. It is concluded that there is no group of filter methods that always outperforms all other methods, but recommendations on filter methods that perform well on many of the data sets are made. Also, groups of filters that are similar with respect to the order in which they rank the features are found. For the analyses, the R machine learning package mlr is used. It provides a uniform programming API and therefore is a convenient tool to conduct feature selection using filter methods.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
R: A Language and Environment for Statistical Computing
R Core Team (2025) · doi:10.32614/r.manuals · Cited by 352,181
Breiman, L. (2001) · Machine Learning · doi:10.1023/a:1010933404324 · Cited by 133,538
Regression shrinkage and selection via the lasso
Tibshirani, R. (1996) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/j.2517-6161.1996.tb02080.x · Cited by 51,482
limma powers differential expression analyses for RNA-sequencing and microarray studies
Ritchie, M. E., Phipson, B., Wu, D., Hu, Y., Law, C. W., Shi, W., & Smyth, G. K. (2015) · Nucleic Acids Research · doi:10.1093/nar/gkv007 · Cited by 40,458
Classification and Regression Trees.
Gordon, A. D., Breiman, L., Friedman, J. H., Olshen, R. A., & Stone, C. J. (1984) · Biometrics · doi:10.2307/2530946 · Cited by 23,522
The meaning and use of the area under a receiver operating characteristic (ROC) curve.
Hanley, J. A., & McNeil, B. J. (1982) · Radiology · doi:10.1148/radiology.143.1.7063747 · Cited by 21,856
Classification and Regression Trees.
Ryzin, J. V., Breiman, L., Friedman, J. H., Olshen, R. A., & Stone, C. J. (1986) · Journal of the American Statistical Association · doi:10.2307/2288003 · Cited by 20,825
Use of Ranks in One-Criterion Variance Analysis
Kruskal, W. H., & Wallis, W. A. (1952) · Journal of the American Statistical Association · doi:10.1080/01621459.1952.10483441 · Cited by 13,912
Smyth, G. K. (2004) · Statistical Applications in Genetics and Molecular Biology · doi:10.2202/1544-6115.1027 · Cited by 11,829
In silico prediction of protein-protein interactions in human macrophages
Souiai, O., Guerfali, F., Ben Miled, S., Brun, C., & Benkahla, A. (2014) · BMC Research Notes · doi:10.1186/1756-0500-7-157 · Cited by 10,803
Significance analysis of microarrays applied to the ionizing radiation response
Tusher, V. G., Tibshirani, R., & Chu, G. (2001) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.091062498 · Cited by 10,568
Peng, H., Long, F., & Ding, C. (2005) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2005.159 · Cited by 10,544
Wrappers for feature subset selection
Kohavi, R., & John, G. H. (1997) · Artificial Intelligence · doi:10.1016/s0004-3702(97)00043-x · Cited by 8,947
An introduction to variable and feature selection
GuyonIsabelle, & ElisseeffAndré (2003) · Journal of Machine Learning Research · doi:10.5555/944919.944968 · Cited by 7,715
An Introduction to Variable and Feature Selection.
Guyon, I., & Elisseeff, A. (2003) · Cited by 6,576
A review of feature selection techniques in bioinformatics
Saeys, Y., Inza, I., & Larrañaga, P. (2007) · Bioinformatics · doi:10.1093/bioinformatics/btm344 · Cited by 5,244
A survey on feature selection methods
Chandrashekar, G., & Sahin, F. (2014) · Computers & Electrical Engineering · doi:10.1016/j.compeleceng.2013.11.024 · Cited by 5,189
Feature selection for classification
DASH, M., & LIU, H. (1997) · Intelligent Data Analysis · doi:10.1016/s1088-467x(97)00008-5 · Cited by 3,574
(1998) · Cited by 3,476
Encyclopedia of Machine Learning
(2010) · Springer eBooks · doi:10.1007/978-0-387-30164-8 · Cited by 3,465
ranger: A Fast Implementation of Random Forests for High Dimensional Data in C++ and R
Wright, M. N., & Ziegler, A. (2017) · Journal of Statistical Software · doi:10.18637/jss.v077.i01 · Cited by 3,340
Conditional variable importance for random forests
Strobl, C., Boulesteix, A.-L., Kneib, T., Augustin, T., & Zeileis, A. (2008) · BMC Bioinformatics · doi:10.1186/1471-2105-9-307 · Cited by 3,254
Multi-Interval Discretization of Continuous-Valued Attributes for Classification Learning
Fayyad, U. M., & Irani, K. B. (1993) · International Joint Conference on Artificial Intelligence · Cited by 2,870
Toward integrating feature selection algorithms for classification and clustering
Liu, H., & Yu, L. (2005) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2005.66 · Cited by 2,706
Regularization Paths for Cox's Proportional Hazards Model via Coordinate Descent
Simon, N., Friedman, J., Hastie, T., & Tibshirani, R. (2011) · Journal of Statistical Software · doi:10.18637/jss.v039.i05 · Cited by 2,499
Feature Selection: A Data Perspective
Li, J., Cheng, K., Wang, S., Morstatter, F., Trevino, R. P., Tang, J., & Liu, H. (2018) · ACM Computing Surveys · doi:10.1145/3136625 · Cited by 2,442
An extensive empirical study of feature selection metrics for text classification
Forman, G. (2003) · Journal of Machine Learning Research · doi:10.5555/944919.944974 · Cited by 2,394
Feature selection in machine learning: A new perspective
Cai, J., Luo, J., Wang, S., & Yang, S. (2018) · Neurocomputing · doi:10.1016/j.neucom.2017.11.077 · Cited by 2,061
Efficient Feature Selection via Analysis of Relevance and Redundancy
Yu, L., & Liu, H. (2004) · doi:10.5555/1005332.1044700 · Cited by 1,983
A Survey on Evolutionary Computation Approaches to Feature Selection
Xue, B., Zhang, M., Browne, W. N., & Yao, X. (2016) · IEEE Transactions on Evolutionary Computation · doi:10.1109/tevc.2015.2504420 · Cited by 1,841
kernlab- An S4 Package for Kernel Methods in R
Karatzoglou, A., Smola, A., Hornik, K., & Zeileis, A. (2004) · Journal of Statistical Software · doi:10.18637/jss.v011.i09 · Cited by 1,786
Graph-based approach for airborne light detection and ranging segmentation
Vilariño, D. L., Cabaleiro, J. C., Martínez, J., Rivera, F. F., & Pena, T. F. (2017) · Journal of Applied Remote Sensing · doi:10.1117/1.jrs.11.015020 · Cited by 1,721
Feature subset selection using a genetic algorithm
Yang, J., & Honavar, V. (1998) · IEEE Intelligent Systems · doi:10.1109/5254.671091 · Cited by 1,362
A review of feature selection methods with applications
Jovic, A., Brkic, K., & Bogunovic, N. (2015) · 2015 38th International Convention on Information and Communication Technology, Electronics and Microelectronics (MIPRO) · doi:10.1109/mipro.2015.7160458 · Cited by 1,209
OpenML: networked science in machine learning
Vanschoren, J., van Rijn, J. N., Bischl, B., & Torgo, L. (2014) · ACM SIGKDD Explorations Newsletter · doi:10.1145/2641190.2641198 · Cited by 1,141
Self-citationConditional Likelihood Maximisation: A Unifying Framework for Mutual Information Feature Selection
Gavin Brown, B., Pocock, A., Zhao, M., & Luján, M. (2012) · Research Explorer (The University of Manchester) · Cited by 1,031
A Review of Feature Selection and Feature Extraction Methods Applied on Microarray Data
Hira, Z. M., & Gillies, D. F. (2015) · Advances in Bioinformatics · doi:10.1155/2015/198363 · Cited by 936
Data Classification: Algorithms and Applications
Aggarwal, C. C. (2014) · doi:10.1201/b17320?locatt=mode:legacy · Cited by 879
A review of feature selection methods on synthetic data
Bolón-Canedo, V., Sánchez-Maroño, N., & Alonso-Betanzos, A. (2013) · Knowledge and Information Systems · doi:10.1007/s10115-012-0487-8 · Cited by 830
Feature selection for classification: A review
Tang, J., Alelyani, S., & Liu, H. (2014) · doi:10.1201/b17320-7 · Cited by 757
rpart: Recursive Partitioning and Regression Trees
Therneau, T., & Atkinson, B. (1999) · CRAN: Contributed Packages · doi:10.32614/cran.package.rpart · Cited by 717
Feature Extraction Construction and Selection: A Data Mining Perspective
MTW, Liu, H., & Motoda, H. (1999) · Journal of the American Statistical Association · doi:10.2307/2669967 · Cited by 713
A review of microarray datasets and applied feature selection methods
Bolón-Canedo, V., Sánchez-Maroño, N., Alonso-Betanzos, A., Benítez, J. M., & Herrera, F. (2014) · Information Sciences · doi:10.1016/j.ins.2014.05.042 · Cited by 666
A Review of Feature Selection and Its Methods
Venkatesh, B., & Anuradha, J. (2019) · Cybernetics and Information Technologies · doi:10.2478/cait-2019-0001 · Cited by 631
A survey on filter techniques for feature selection in gene expression microarray analysis
Lazar, C., Taminau, J., Meganck, S., Steenhoff, D., Coletta, A., Molter, C., de Schaetzen, V., Duque, R., Bersini, H., & Nowe, A. (2012) · IEEE/ACM Transactions on Computational Biology and Bioinformatics · doi:10.1109/tcbb.2012.33 · Cited by 619
Stability of feature selection algorithms: a study on high-dimensional spaces
Kalousis, A., Prados, J., & Hilario, M. (2007) · Knowledge and Information Systems · doi:10.1007/s10115-006-0040-8 · Cited by 614
Discovering Knowledge in Data: An Introduction to Data Mining
Larose, D. T., & Larose, C. D. (2014) · Wiley eBooks · doi:10.1002/9781118874059 · Cited by 602
Fast Binary Feature Selection with Conditional Mutual Information
FleuretFrançois (2004) · Journal of Machine Learning Research · doi:10.5555/1005332.1044711 · Cited by 593
Recursive Partitioning and Regression Trees
Therneau, T. M., Atkinson, B., & Ripley, B. D. (2015) · Cited by 471
Wrapper–Filter Feature Selection Algorithm Using a Memetic Framework
Zhu, Z., Ong, Y.-S., & Dash, M. (2007) · IEEE Transactions on Systems, Man and Cybernetics, Part B (Cybernetics) · doi:10.1109/tsmcb.2006.883267 · Cited by 443
Filter Methods for Feature Selection – A Comparative Study
Sánchez-Maroño, N., Alonso-Betanzos, A., & Tombilla-Sanromán, M. (2007) · Lecture Notes in Computer Science · doi:10.1007/978-3-540-77226-2_19 · Cited by 425
Filter versus wrapper gene selection approaches in DNA microarray domains
Inza, I., Larrañaga, P., Blanco, R., & Cerrolaza, A. J. (2004) · Artificial Intelligence in Medicine · doi:10.1016/j.artmed.2004.01.007 · Cited by 425
Liu, H., Li, J., & Wong, L. (2002) · PubMed · Cited by 410
Swarm Intelligence Algorithms for Feature Selection: A Review
Brezočnik, L., Fister, I., & Podgorelec, V. (2018) · Applied Sciences · doi:10.3390/app8091521 · Cited by 374
Pattern Recognition and Signal Processing
(1978) · Springer eBooks · doi:10.1007/978-94-009-9941-1 · Cited by 354
Information-Theoretic Feature Selection in Microarray Data Using Variable Complementarity
Meyer, P. E., Schretter, C., & Bontempi, G. (2008) · IEEE Journal of Selected Topics in Signal Processing · doi:10.1109/jstsp.2008.923858 · Cited by 280
Bischl, B., Lang, M., Kotthoff, L., Schratz, P., Schiffner, J., Richter, J., Jones, Z., Casalicchio, G., Gallo, M., & Binder, M. (2013) · CRAN: Contributed Packages · doi:10.32614/cran.package.mlr · Cited by 277
Self-citationModern Multivariate Statistical Techniques: Regression, Classification and Manifold Learning
Maindonald, J. (2009) · Journal of Statistical Software · doi:10.18637/jss.v029.b11 · Cited by 256
Gérard Biau, A., Cadre, B., & Rouvìère, L. (2019) · Machine Learning · doi:10.1007/s10994-019-05787-1 · Cited by 190
Resampling Methods for Meta-Model Validation with Recommendations for Evolutionary Computation
Bischl, B., Mersmann, O., Trautmann, H., & Weihs, C. (2012) · Evolutionary Computation · doi:10.1162/evco_a_00069 · Cited by 186
Self-citationKerschke, P., & Trautmann, H. (2019) · Evolutionary Computation · doi:10.1162/evco_a_00236 · Cited by 173
Feature clustering based support vector machine recursive feature elimination for gene selection
Huang, X., Zhang, L., Wang, B., Li, F., & Zhang, Z. (2017) · Applied Intelligence · doi:10.1007/s10489-017-0992-2 · Cited by 146
A Comparative Study on Feature Selection Methods for Drug Discovery
Liu, Y. (2004) · Journal of Chemical Information and Computer Sciences · doi:10.1021/ci049875d · Cited by 134
EFS-MI: an ensemble feature selection method for classification
Hoque, N., Singh, M., & Bhattacharyya, D. K. (2017) · Complex & Intelligent Systems · doi:10.1007/s40747-017-0060-x · Cited by 132
Ghosh, M., Adhikary, S., Ghosh, K. K., Sardar, A., Begum, S., & Sarkar, R. (2018) · Medical & Biological Engineering & Computing · doi:10.1007/s11517-018-1874-4 · Cited by 130
Wah, Y. B., Ibrahim, N., Hamid, H. A., et al. (2018) · Pertanika journal of science & technology · Cited by 123
Measuring the Stability of Feature Selection
Nogueira, S., & Gavin Brown, B. (2016) · Lecture notes in computer science · doi:10.1007/978-3-319-46227-1_28 · Cited by 94
Aphinyanaphongs, Y., Fu, L. D., Li, Z., Peskin, E. R., Efstathiadis, E., Aliferis, C., & Statnikov, A. (2014) · Journal of the Association for Information Science and Technology · doi:10.1002/asi.23110 · Cited by 85
A Comprehensive Comparison on Evolutionary Feature Selection Approaches to Classification
Xue, B., Zhang, M., & Browne, W. N. (2015) · International Journal of Computational Intelligence and Applications · doi:10.1142/s146902681550008x · Cited by 79
A New Filter Feature Selection Based on Criteria Fusion for Gene Microarray Data
Ke, W., Wu, C., Wu, Y., & Xiong, N. N. (2018) · IEEE Access · doi:10.1109/access.2018.2873634 · Cited by 52
OpenML: An R package to connect to the machine learning platform OpenML
Casalicchio, G., Bossek, J., Lang, M., Kirchhoff, D., Kerschke, P., Hofner, B., Seibold, H., Vanschoren, J., & Bischl, B. (2017) · Computational Statistics · doi:10.1007/s00180-017-0742-2 · Cited by 43
Self-citationDarshan, S. L. S., & Jaidhar, C. D. (2018) · Procedia Computer Science · doi:10.1016/j.procs.2017.12.046 · Cited by 32
FSelector: Selecting Attributes
Piotr Romanski, L. K. (2009) · doi:10.32614/cran.package.fselector · Cited by 27
samr: SAM: Significance Analysis of Microarrays
Tibshirani, R., Seo, M. J., Chu, G., et al. (2005) · doi:10.32614/cran.package.samr · Cited by 20
Bommert, A., Rahnenführer, J., & Lang, M. (2017) · Computational and Mathematical Methods in Medicine · doi:10.1155/2017/7907163 · Cited by 15
Self-citation(2014) · doi:10.5555/2627435.2697065
Cited by
Hyperparameter optimization: Foundations, algorithms, best practices, and open challenges
Bischl, B., Binder, M., Lang, M., Pielok, T., Richter, J., Coors, S., Thomas, J., Ullmann, T., Becker, M., Boulesteix, A., et al. (2023) · WIREs Data Mining and Knowledge Discovery · doi:10.1002/widm.1484 · Cited by 930
Self-citationA Review of Feature Selection Methods for Machine Learning-Based Disease Risk Prediction
Pudjihartono, N., Fadason, T., Kempa-Liehr, A. W., & O'Sullivan, J. M. (2022) · Frontiers in Bioinformatics · doi:10.3389/fbinf.2022.927312 · Cited by 880
Adversarial Machine Learning for Network Intrusion Detection Systems: A Comprehensive Survey
He, K., Kim, D. D., & Asghar, M. R. (2023) · IEEE Communications Surveys & Tutorials · doi:10.1109/comst.2022.3233793 · Cited by 450
Xu, Y., Zhang, X., Li, H., Zheng, H., Zhang, J., Olsen, M. S., Varshney, R. K., Prasanna, B. M., & Qian, Q. (2022) · Molecular Plant · doi:10.1016/j.molp.2022.09.001 · Cited by 363
Shaik, T., Tao, X., Li, Y., Dann, C., McDonald, J., Redmond, P., & Galligan, L. (2022) · IEEE Access · doi:10.1109/access.2022.3177752 · Cited by 286
Metadata
- Authors
- Bommert, A., Sun, X., Bischl, B., Rahnenführer, J., & Lang, M.
- Venue
- Computational Statistics & Data Analysis
- Publisher
- Elsevier BV
- Published
- 2020-03
- Volume
- 143
- Pages
- 106839
- ISSN
- 0167-9473 (ISSN-L)
- OpenAlex
- W2973941913
- Cited by
- 725
- References
- 76
- FWCI
- 57.91
- Type
- journal article
- Language
- en
- Open access
- hybrid
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.