The DOI matches its Crossref record; 85 of the 85 references it cites were found in a registry or an index.
Feature Selection: A literature Review
Abstract
Relevant feature identification has become an essential task to apply data mining algorithms effectively in real-world scenarios. Therefore, many feature selection methods have been proposed to obtain the relevant feature or feature subsets in the literature to achieve their objectives of classification and clustering. This paper introduces the concepts of feature relevance, general procedures, evaluation criteria, and the characteristics of feature selection. A comprehensive overview, categorization, and comparison of existing feature selection methods are also done, and the guidelines are also provided for user to select a feature selection algorithm without knowing the information of each algorithm. We conclude this work with real world applications, challenges, and future research directions of feature selection.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
A Mathematical Theory of Communication
Shannon, C. E. (1948) · Bell System Technical Journal · doi:10.1002/j.1538-7305.1948.tb00917.x · Cited by 48,949
The Elements of Statistical Learning
Hastie, T., Friedman, J., & Tibshirani, R. (2001) · Springer Series in Statistics · doi:10.1007/978-0-387-21606-5 · Cited by 24,162
On Information and Sufficiency
Kullback, S., & Leibler, R. A. (1951) · The Annals of Mathematical Statistics · doi:10.1214/aoms/1177729694 · Cited by 20,193
(2012) · Encyclopedia of the Sciences of Learning · doi:10.1007/978-1-4419-1428-6_5195 · Cited by 19,370
The Mathematical Theory of Communication
Shannon, C. E., Weaver, W., & Wiener, N. (1950) · Physics Today · doi:10.1063/1.3067010 · Cited by 9,801
Wrappers for feature subset selection
Kohavi, R., & John, G. H. (1997) · Artificial Intelligence · doi:10.1016/s0004-3702(97)00043-x · Cited by 8,947
Feature selection for classification
DASH, M., & LIU, H. (1997) · Intelligent Data Analysis · doi:10.1016/s1088-467x(97)00008-5 · Cited by 3,574
(1998) · Cited by 3,476
Selection of relevant features and examples in machine learning
Blum, A. L., & Langley, P. (1997) · Artificial Intelligence · doi:10.1016/s0004-3702(97)00063-5 · Cited by 3,298
Estimating attributes: Analysis and extensions of RELIEF
Kononenko, I. (1994) · Lecture Notes in Computer Science · doi:10.1007/3-540-57868-4_57 · Cited by 2,958
Devijver, P. A., & Kittler, J. (1982) · Prentice-Hall International eBooks · Cited by 2,910
Toward integrating feature selection algorithms for classification and clustering
Liu, H., & Yu, L. (2005) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2005.66 · Cited by 2,706
An extensive empirical study of feature selection metrics for text classification
Forman, G. (2003) · Journal of Machine Learning Research · doi:10.5555/944919.944974 · Cited by 2,394
Irrelevant Features and the Subset Selection Problem
John, G. H., Kohavi, R., & Pfleger, K. (1994) · Machine Learning Proceedings 1994 · doi:10.1016/b978-1-55860-335-6.50023-4 · Cited by 2,368
Feature selection for high-dimensional data: a fast correlation-based filter solution
Yu, L., & Liu, H. (2003) · Cited by 2,305
Feature Selection for Knowledge Discovery and Data Mining
Liu, H., & Motoda, H. (1998) · Springer eBooks · doi:10.1007/978-1-4615-5689-3 · Cited by 2,198
Feature selection: evaluation, application, and small sample performance
Jain, A., & Zongker, D. (1997) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/34.574797 · Cited by 2,171
Bhattacharyya, A. (1943) · Medical Entomology and Zoology · Cited by 2,002
The feature selection problem: traditional methods and a new algorithm
Kira, K., & Rendell, L. A. (1992) · National Conference on Artificial Intelligence · Cited by 1,897
Correlation-based Feature Selection for Discrete and Numeric Class Machine Learning
Hall, M. A. (2000) · Cited by 1,715
Toward optimal feature selection
Koller, D., & Sahami, M. (1996) · Cited by 1,482
Subset Selection in Regression
(2003) · Technometrics · doi:10.1198/tech.2003.s144 · Cited by 1,463
Unsupervised feature selection using feature similarity
Mitra, P., Murthy, C. A., & Pal, S. K. (2002) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/34.990133 · Cited by 1,460
Subspace clustering for high dimensional data: a review
Parsons, L., Haque, E., & Liu, H. (2004) · ACM SIGKDD Explorations Newsletter · doi:10.1145/1007730.1007731 · Cited by 1,346
A Branch and Bound Algorithm for Feature Subset Selection
Narendra, & Fukunaga (1977) · IEEE Transactions on Computers · doi:10.1109/tc.1977.1674939 · Cited by 1,245
A review of feature selection methods on synthetic data
Bolón-Canedo, V., Sánchez-Maroño, N., & Alonso-Betanzos, A. (2013) · Knowledge and Information Systems · doi:10.1007/s10115-012-0487-8 · Cited by 830
Feature selection for classification: A review
Tang, J., Alelyani, S., & Liu, H. (2014) · doi:10.1201/b17320-7 · Cited by 757
A probabilistic approach to feature selection - a filter solution
Liu, H., & Setiono, R. (1996) · Cited by 665
Robust Feature Selection Using Ensemble Feature Selection Techniques
Saeys, Y., Abeel, T., & Van de Peer, Y. (2008) · Lecture Notes in Computer Science · doi:10.1007/978-3-540-87481-2_21 · Cited by 663
Feature selection algorithms: a survey and experimental evaluation
Molina, L. C., Belanche, L., & Àngela Nebot, B. (2003) · Proceedings · doi:10.1109/icdm.2002.1183917 · Cited by 653
Feature selection for high-dimensional genomic microarray data
Xing, E. P., Jordan, M. I., & Karp, R. M. (2001) · Cited by 635
Learning with many irrelevant features
Almuallim, H., & Dietterich, T. G. (1991) · Cited by 623
Stability of feature selection algorithms: a study on high-dimensional spaces
Kalousis, A., Prados, J., & Hilario, M. (2007) · Knowledge and Information Systems · doi:10.1007/s10115-006-0040-8 · Cited by 614
Caruana, R., & Freitag, D. (1994) · Machine Learning Proceedings 1994 · doi:10.1016/b978-1-55860-335-6.50012-x · Cited by 545
Training a 3-node neural network is NP-complete
Blum, A. L., & Rivest, R. L. (1992) · Neural Networks · doi:10.1016/s0893-6080(05)80010-3 · Cited by 543
Advances in Instance Selection for Instance-Based Learning Algorithms
Brighton, H., & Mellish, C. (2002) · Data Mining and Knowledge Discovery · doi:10.1023/a:1014043630878 · Cited by 532
Learning Boolean concepts in the presence of many irrelevant features
Almuallim, H., & Dietterich, T. G. (1994) · Artificial Intelligence · doi:10.1016/0004-3702(94)90084-1 · Cited by 520
Learning quickly when irrelevant attributes abound: A new linear-threshold algorithm
Littlestone, N. (1988) · Machine Learning · doi:10.1007/bf00116827 · Cited by 497
Prototype and Feature Selection by Sampling and Random Mutation Hill Climbing Algorithms
Skalak, D. B. (1994) · Machine Learning Proceedings 1994 · doi:10.1016/b978-1-55860-335-6.50043-x · Cited by 486
Aliferis, C., Statnikov, A., Tsamardinos, I., Mani, S., & Koutsoukos, X. (2010) · doi:10.5555/1756006.1756013 · Cited by 469
Nonparametric weighted feature extraction for classification
Kuo, B.-C., & Landgrebe, D. A. (2004) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2004.825578 · Cited by 425
Performance of feature-selection methods in the classification of high-dimension data
Hua, J., Tembe, W. D., & Dougherty, E. R. (2009) · Pattern Recognition · doi:10.1016/j.patcog.2008.08.001 · Cited by 395
Feature selection for clustering - a filter solution
Dash, M., Choi, K., Scheuermann, P., & Liu, H. (2003) · 2002 IEEE International Conference on Data Mining, 2002. Proceedings. · doi:10.1109/icdm.2002.1183893 · Cited by 383
Feature Selection for Clustering: A Review
Alelyani, S., Tang, J., & Liu, H. (2018) · Data Clustering · doi:10.1201/9781315373515-2 · Cited by 371
Adaptive Intrusion Detection: A Data Mining Approach
Lee, W., Stolfo, S. J., & Mok, K. W. (2000) · Artificial Intelligence Review · doi:10.1023/a:1006624031083 · Cited by 347
Li, S., Wu, H., Wan, D., & Zhu, J. (2011) · Knowledge-Based Systems · doi:10.1016/j.knosys.2010.07.003 · Cited by 308
Feature selection in unsupervised learning via evolutionary search
Kim, Y., Street, W. N., & Menczer, F. (2000) · Proceedings of the sixth ACM SIGKDD international conference on Knowledge discovery and data mining · doi:10.1145/347090.347169 · Cited by 288
Feature Selection with Ensembles, Artificial Variables, and Redundancy Elimination
Tuv, E., Borisov, A., Runger, G. C., & Torkkola, K. (2009) · Journal of Machine Learning Research · doi:10.5555/1577069.1755828 · Cited by 280
Unsupervised feature selection applied to content-based retrieval of lung images
Dy, J. G., Brodley, C. E., Kak, A., Broderick, L. S., & Aisen, A. M. (2003) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2003.1182100 · Cited by 255
Using Decision Trees to Improve Case-Based Learning
Cardie, C. (1993) · Machine Learning Proceedings 1993 · doi:10.1016/b978-1-55860-307-3.50010-1 · Cited by 255
Adaptive Forward-Backward Greedy Algorithm for Learning Sparse Representations
Zhang, T. (2011) · IEEE Transactions on Information Theory · doi:10.1109/tit.2011.2146690 · Cited by 248
Redundancy based feature selection for microarray data
Yu, L., & Liu, H. (2004) · Proceedings of the tenth ACM SIGKDD international conference on Knowledge discovery and data mining · doi:10.1145/1014052.1014149 · Cited by 244
Feature Subset Selection and Order Identification for Unsupervised Learning
Dy, J., & Brodley, C. E. (2000) · Cited by 234
Efficient Algorithms for Minimizing Cross Validation Error
Moore, A. W., & Lee, M. S. (1994) · Machine Learning Proceedings 1994 · doi:10.1016/b978-1-55860-335-6.50031-3 · Cited by 232
Huang, C.-Y. (., Lai, C.-Y., & Cheng, K.-T. (. (2009) · Electronic Design Automation · doi:10.1016/b978-0-12-374364-0.50011-4 · Cited by 211
Bolón-Canedo, V., Sánchez-Maroño, N., & Alonso-Betanzos, A. (2011) · Expert Systems with Applications · doi:10.1016/j.eswa.2010.11.028 · Cited by 208
A Comparison of Seven Techniques for Choosing Subsets of Pattern Recognition Properties
Mucciardi, A. N., & Gose, E. E. (1971) · IEEE Transactions on Computers · doi:10.1109/t-c.1971.223398 · Cited by 202
Gene selection algorithm by combining reliefF and mRMR
Zhang, Y., Ding, C., & Li, T. (2008) · BMC Genomics · doi:10.1186/1471-2164-9-s2-s27 · Cited by 199
An ensemble of filters and classifiers for microarray data classification
Bolón-Canedo, V., Sánchez-Maroño, N., & Alonso-Betanzos, A. (2012) · Pattern Recognition · doi:10.1016/j.patcog.2011.06.006 · Cited by 183
Dimensionality reduction of unsupervised data
Dash, M., Liu, H., & Yao, J. (2002) · IEEE International Conference on Tools with Artificial Intelligence (ICTAI) · doi:10.1109/tai.1997.632300 · Cited by 178
A two-stage gene selection scheme utilizing MRMR filter and GA wrapper
El Akadi, A., Amine, A., El Ouardighi, A., & Aboutajdine, D. (2011) · Knowledge and Information Systems · doi:10.1007/s10115-010-0288-x · Cited by 168
Self-adaptive differential evolution for feature selection in hyperspectral image data
Ghosh, A., Datta, A., & Ghosh, S. (2013) · Applied Soft Computing · doi:10.1016/j.asoc.2012.11.042 · Cited by 167
Feature Selection and Classification – A Probabilistic Wrapper Approach
Liu, H., & Setiono, R. (2022) · Industrial and Engineering Applications of Artificial Intelligence and Expert Systems · doi:10.1201/9780429332111-72 · Cited by 146
Feature Selection Methods: Genetic Algorithms vs. Greedy-like Search
Vafaie, H. (2009) · Cited by 139
Iterative RELIEF for feature weighting
Sun, Y., & Li, J. (2006) · International Conference on Machine Learning · doi:10.1145/1143844.1143959 · Cited by 134
Customer Retention via Data Mining
Ng, K., & Liu, H. (2000) · Artificial Intelligence Review · doi:10.1023/a:1006676015154 · Cited by 105
A Unifying View on Instance Selection
Reinartz, T. P. (2002) · Data Mining and Knowledge Discovery · doi:10.1023/a:1014047731786 · Cited by 94
Abusamra, H. (2013) · Procedia Computer Science · doi:10.1016/j.procs.2013.10.003 · Cited by 90
Why does bagging work? a Bayesian account and its implications
Domingos, P. (1997) · Cited by 89
Oblivious Decision Trees and Abstract Cases
Langley, P., & Sage, S. (1994) · Cited by 87
Efficient learning of selective Bayesian network classifiers
Singh, M., & Provan, G. M. (1996) · Cited by 81
Concept-based feature generation and selection for information retrieval
Egozi, O., Gabrilovich, E., & Markovitch, S. (2008) · Cited by 77
Saari, P., Eerola, T., & Lartillot, O. (2011) · IEEE Transactions on Audio Speech and Language Processing · doi:10.1109/tasl.2010.2101596 · Cited by 73
Highly discriminative statistical features for email classification
Gómez, J. C., Boiy, E., & Moens, M. (2011) · Knowledge and Information Systems · doi:10.1007/s10115-011-0403-7 · Cited by 65
Feature Selection Using Mutual Information: An Experimental Study
Liu, H., Liu, L., & Zhang, H. (2008) · Lecture notes in computer science · doi:10.1007/978-3-540-89197-0_24 · Cited by 42
Causal filter selection in microarray data
Bontempi, G., & Meyer, P. E. (2010) · Cited by 42
A COMPARATIVE EVALUATION OF MEDIUM-AND LARGE-SCALE FEATURE SELECTORS FOR PATTERN CLASSIFIERS
Kudo, M., & Sklansky, J. (1998) · Czech digital mathematics library · Cited by 33
Feature Selection and Dimensionality Reduction in Genomics and Proteomics
Hauskrecht, M., Pelikan, R. C., Valko, M., & Lyons‐Weiler, J. (2007) · Springer eBooks · doi:10.1007/978-0-387-47509-7_7 · Cited by 29
Randomized Variable Elimination
Stracuzzi, D. J., & Utgoff, P. E. (2002) · doi:10.5555/1005332.1044704 · Cited by 27
A Feature Selection Algorithm Capable of Handling Extremely Large Data Dimensionality
Sun, Y., Todorović, S., & Goodison, S. (2008) · SIAM International Conference on Data Mining · doi:10.1137/1.9781611972788.48 · Cited by 26
Implementing ReliefF filters to extract meaningful features from genetic lifetime datasets
Beretta, L., & Santaniello, A. (2010) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2010.12.003 · Cited by 26
Multi-view Ensemble Learning for Poem Data Classification Using SentiWordNet
Kumar, V., & Minz, S. (2014) · Smart innovation, systems and technologies · doi:10.1007/978-3-319-07353-8_8 · Cited by 21
Self-citationDiscriminative Feature Selection via Multiclass Variable Memory Markov Model
Slonim, N., Bejerano, G., Fine, S., & Tishby, N. (2003) · EURASIP Journal on Advances in Signal Processing · doi:10.1155/s111086570321115x · Cited by 21
Scalable Feature Selection for Multi-class Problems
Chidlovskii, B., & Lecerf, L. (2008) · Lecture notes in computer science · doi:10.1007/978-3-540-87479-9_33 · Cited by 20
Obtaining scalable and accurate classification in large-scale spatio-temporal domains
Vainer, I., Kraus, S., Kaminka, G. A., & Slovin, H. (2010) · Knowledge and Information Systems · doi:10.1007/s10115-010-0348-2 · Cited by 18
Cited by
Stability of feature selection algorithm: A review
Khaire, U. M., & Dhanalakshmi, R. (2022) · Journal of King Saud University - Computer and Information Sciences · doi:10.1016/j.jksuci.2019.06.012 · Cited by 630
Feature selection and classification systems for chronic disease prediction: A review
Jain, D., & Singh, V. (2018) · Egyptian Informatics Journal · doi:10.1016/j.eij.2018.03.002 · Cited by 368
A comprehensive survey on recent metaheuristics for feature selection
Dokeroglu, T., Deniz, A., & Kiziloz, H. E. (2022) · Neurocomputing · doi:10.1016/j.neucom.2022.04.083 · Cited by 356
Condition Monitoring using Machine Learning: A Review of Theory, Applications, and Recent Advances
Surucu, O., Gadsden, S. A., & Yawney, J. (2023) · Expert Systems with Applications · doi:10.1016/j.eswa.2023.119738 · Cited by 336
Hou, S., Liu, Y., & Yang, Q. (2022) · Journal of Rock Mechanics and Geotechnical Engineering · doi:10.1016/j.jrmge.2021.05.004 · Cited by 267
Metadata
- Authors
- Kumar, V.
- Venue
- The Smart Computing Review
- Published
- 2014-06-30
- Volume
- 4
- Issue
- 3
- ISSN
- 2234-4624 (ISSN-L)
- OpenAlex
- W1995806857
- Cited by
- 659
- References
- 85
- FWCI
- 23.11
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.