The DOI matches its Crossref record; 78 of the 78 references it cites were found in a registry or an index.
Toward integrating feature selection algorithms for classification and clustering
Abstract
This paper introduces concepts and algorithms of feature selection, surveys existing feature selection algorithms for classification and clustering, groups and compares different algorithms with a categorizing framework based on search strategies, evaluation criteria, and data mining tasks, reveals unattempted combinations, and provides guidelines in selecting feature selection algorithms. With the categorizing framework, we continue our efforts toward-building an integrated system for intelligent feature selection. A unifying platform is proposed as an intermediate step. An illustrative example is presented to show how existing feature selection algorithms can be integrated into a meta algorithm that can take advantage of individual algorithms. An added advantage of doing so is to help a user employ a suitable algorithm without knowing details of each algorithm. Some real-world applications are included to demonstrate the use of feature selection in data mining. We conclude this work by identifying trends and challenges of feature selection research and development.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Data mining: concepts and techniques
(2012) · Choice Reviews Online · doi:10.5860/choice.49-3305 · Cited by 28,531
Data Mining: Practical Machine Learning Tools and Techniques
(2011) · Elsevier eBooks · doi:10.1016/c2009-0-19715-5 · Cited by 25,437
The Elements of Statistical Learning
Hastie, T., Friedman, J., & Tibshirani, R. (2001) · Springer Series in Statistics · doi:10.1007/978-0-387-21606-5 · Cited by 24,162
Witten, I. H. (2008) · Cited by 9,143
Wrappers for feature subset selection
Kohavi, R., & John, G. H. (1997) · Artificial Intelligence · doi:10.1016/s0004-3702(97)00043-x · Cited by 8,947
Data mining: practical machine learning tools and techniques with Java implementations
Witten, I. H., & Frank, E. (2002) · ACM SIGMOD Record · doi:10.1145/507338.507355 · Cited by 5,137
Data Mining - Concepts and Techniques.
Perner, P. (2002) · Künstliche Intell. · Cited by 4,886
A Comparative Study on Feature Selection in Text Categorization
Yang, Y., & Pedersen, J. (1997) · Cited by 4,790
Feature selection for classification
DASH, M., & LIU, H. (1997) · Intelligent Data Analysis · doi:10.1016/s1088-467x(97)00008-5 · Cited by 3,574
Self-citationSelection of relevant features and examples in machine learning
Blum, A. L., & Langley, P. (1997) · Artificial Intelligence · doi:10.1016/s0004-3702(97)00063-5 · Cited by 3,298
Estimating attributes: Analysis and extensions of RELIEF
Kononenko, I. (1994) · Lecture Notes in Computer Science · doi:10.1007/3-540-57868-4_57 · Cited by 2,958
Devijver, P. A., & Kittler, J. (1982) · Prentice-Hall International eBooks · Cited by 2,910
Multi-Interval Discretization of Continuous-Valued Attributes for Classification Learning
Fayyad, U. M., & Irani, K. B. (1993) · International Joint Conference on Artificial Intelligence · Cited by 2,870
Text classification from labeled and unlabeled documents using EM
Nigam, K., Mccallum, A. K., Thrun, S., & Mitchell, T. (2000) · Machine Learning · doi:10.1023/a:1007692713085 · Cited by 2,736
From data mining to knowledge discovery: an overview
Fayyad, U. M., Piatetsky-Shapiro, G., & Smyth, P. (1996) · Knowledge Discovery and Data Mining · Cited by 2,395
Irrelevant Features and the Subset Selection Problem
John, G. H., Kohavi, R., & Pfleger, K. (1994) · Machine Learning Proceedings 1994 · doi:10.1016/b978-1-55860-335-6.50023-4 · Cited by 2,368
Feature selection for high-dimensional data: a fast correlation-based filter solution
Yu, L., & Liu, H. (2003) · Cited by 2,305
Self-citationFeature Selection for Knowledge Discovery and Data Mining
Liu, H., & Motoda, H. (1998) · Springer eBooks · doi:10.1007/978-1-4615-5689-3 · Cited by 2,198
Self-citationFeature selection: evaluation, application, and small sample performance
Jain, A., & Zongker, D. (1997) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/34.574797 · Cited by 2,171
The feature selection problem: traditional methods and a new algorithm
Kira, K., & Rendell, L. A. (1992) · National Conference on Artificial Intelligence · Cited by 1,897
Correlation-based Feature Selection for Discrete and Numeric Class Machine Learning
Hall, M. A. (2000) · Cited by 1,715
Image Retrieval: Current Techniques, Promising Directions, and Open Issues
Rui, Y., Huang, T. S., & Chang, S.-F. (1999) · Journal of Visual Communication and Image Representation · doi:10.1006/jvci.1999.0413 · Cited by 1,601
Database mining: a performance perspective
Agrawal, R., Imielinski, T., & Swami, A. (1993) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/69.250074 · Cited by 1,505
Toward optimal feature selection
Koller, D., & Sahami, M. (1996) · Cited by 1,482
Subset Selection in Regression
(2003) · Technometrics · doi:10.1198/tech.2003.s144 · Cited by 1,463
Unsupervised feature selection using feature similarity
Mitra, P., Murthy, C. A., & Pal, S. K. (2002) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/34.990133 · Cited by 1,460
Data preparation for data mining
Pyle, D. (1999) · Cited by 1,368
Feature subset selection using a genetic algorithm
Yang, J., & Honavar, V. (1998) · IEEE Intelligent Systems · doi:10.1109/5254.671091 · Cited by 1,362
Subspace clustering for high dimensional data: a review
Parsons, L., Haque, E., & Liu, H. (2004) · ACM SIGKDD Explorations Newsletter · doi:10.1145/1007730.1007731 · Cited by 1,346
Self-citationA Branch and Bound Algorithm for Feature Subset Selection
Narendra, & Fukunaga (1977) · IEEE Transactions on Computers · doi:10.1109/tc.1977.1674939 · Cited by 1,245
Discretization: An Enabling Technique
Liu, H., Hussain, F., Tan, C. L., & Dash, M. (2002) · Data Mining and Knowledge Discovery · doi:10.1023/a:1016304305535 · Cited by 981
Self-citationSubset Selection in Regression
Miller, A. (2002) · Chapman & Hall/CRC eBooks · doi:10.1201/9781420035933 · Cited by 929
Feature Extraction Construction and Selection: A Data Mining Perspective
MTW, Liu, H., & Motoda, H. (1999) · Journal of the American Statistical Association · doi:10.2307/2669967 · Cited by 713
Self-citationSelection of Relevant Features in Machine Learning.
Langley, P. (1994) · doi:10.21236/ada292575 · Cited by 709
A probabilistic approach to feature selection - a filter solution
Liu, H., & Setiono, R. (1996) · Cited by 665
Self-citationFeature selection for high-dimensional genomic microarray data
Xing, E. P., Jordan, M. I., & Karp, R. M. (2001) · Cited by 635
Learning with many irrelevant features
Almuallim, H., & Dietterich, T. G. (1991) · Cited by 623
Feature Extraction, Construction and Selection
(1998) · Springer eBooks · doi:10.1007/978-1-4615-5725-8 · Cited by 573
Caruana, R., & Freitag, D. (1994) · Machine Learning Proceedings 1994 · doi:10.1016/b978-1-55860-335-6.50012-x · Cited by 545
Training a 3-node neural network is NP-complete
Blum, A. L., & Rivest, R. L. (1992) · Neural Networks · doi:10.1016/s0893-6080(05)80010-3 · Cited by 543
Advances in Instance Selection for Instance-Based Learning Algorithms
Brighton, H., & Mellish, C. (2002) · Data Mining and Knowledge Discovery · doi:10.1023/a:1014043630878 · Cited by 532
Learning Boolean concepts in the presence of many irrelevant features
Almuallim, H., & Dietterich, T. G. (1994) · Artificial Intelligence · doi:10.1016/0004-3702(94)90084-1 · Cited by 520
Prototype and Feature Selection by Sampling and Random Mutation Hill Climbing Algorithms
Skalak, D. B. (1994) · Machine Learning Proceedings 1994 · doi:10.1016/b978-1-55860-335-6.50043-x · Cited by 486
Clustering Objects on Subsets of Attributes (with Discussion)
Friedman, J. H., & Meulman, J. J. (2004) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/j.1467-9868.2004.02059.x · Cited by 418
Text Categorization with Support Vector Machines. How to Represent Texts in Input Space?
Leopold, E., & Kindermann, J. (2002) · Machine Learning · doi:10.1023/a:1012491419635 · Cited by 403
ON AUTOMATIC FEATURE SELECTION
SIEDLECKI, W., & SKLANSKY, J. (1988) · International Journal of Pattern Recognition and Artificial Intelligence · doi:10.1142/s0218001488000145 · Cited by 392
Feature selection for clustering - a filter solution
Dash, M., Choi, K., Scheuermann, P., & Liu, H. (2003) · 2002 IEEE International Conference on Data Mining, 2002. Proceedings. · doi:10.1109/icdm.2002.1183893 · Cited by 383
Self-citationAdaptive Intrusion Detection: A Data Mining Approach
Lee, W., Stolfo, S. J., & Mok, K. W. (2000) · Artificial Intelligence Review · doi:10.1023/a:1006624031083 · Cited by 347
Feature Subset Selection Using a Genetic Algorithm
Yang, J., & Honavar, V. (1998) · Feature Extraction, Construction and Selection · doi:10.1007/978-1-4615-5725-8_8 · Cited by 326
Emerging trends in business analytics
Kohavi, R., Rothleder, N. J., & Simoudis, E. (2002) · Communications of the ACM · doi:10.1145/545151.545177 · Cited by 319
Feature selection in unsupervised learning via evolutionary search
Kim, Y., Street, W. N., & Menczer, F. (2000) · Proceedings of the sixth ACM SIGKDD international conference on Knowledge discovery and data mining · doi:10.1145/347090.347169 · Cited by 288
Using Decision Trees to Improve Case-Based Learning
Cardie, C. (1993) · Machine Learning Proceedings 1993 · doi:10.1016/b978-1-55860-307-3.50010-1 · Cited by 255
Redundancy based feature selection for microarray data
Yu, L., & Liu, H. (2004) · Proceedings of the tenth ACM SIGKDD international conference on Knowledge discovery and data mining · doi:10.1145/1014052.1014149 · Cited by 244
Self-citationFeature Subset Selection and Order Identification for Unsupervised Learning
Dy, J., & Brodley, C. E. (2000) · Cited by 234
Efficient Algorithms for Minimizing Cross Validation Error
Moore, A. W., & Lee, M. S. (1994) · Machine Learning Proceedings 1994 · doi:10.1016/b978-1-55860-335-6.50031-3 · Cited by 232
Instance Selection and Construction for Data Mining
(2001) · Springer eBooks · doi:10.1007/978-1-4757-3359-4 · Cited by 216
Huang, C.-Y. (., Lai, C.-Y., & Cheng, K.-T. (. (2009) · Electronic Design Automation · doi:10.1016/b978-0-12-374364-0.50011-4 · Cited by 211
Feature selection using rough sets theory
Modrzejewski, M. (1993) · Lecture notes in computer science · doi:10.1007/3-540-56602-3_138 · Cited by 206
A Comparison of Seven Techniques for Choosing Subsets of Pattern Recognition Properties
Mucciardi, A. N., & Gose, E. E. (1971) · IEEE Transactions on Computers · doi:10.1109/t-c.1971.223398 · Cited by 202
A selective sampling approach to active feature selection
Liu, H., Motoda, H., & Yu, L. (2004) · Artificial Intelligence · doi:10.1016/j.artint.2004.05.009 · Cited by 178
Self-citationDimensionality reduction of unsupervised data
Dash, M., Liu, H., & Yao, J. (2002) · IEEE International Conference on Tools with Artificial Intelligence (ICTAI) · doi:10.1109/tai.1997.632300 · Cited by 178
Self-citationBusiness applications of data mining
Apte, C., Liu, B., Pednault, E. P. D., & Smyth, P. (2002) · Communications of the ACM · doi:10.1145/545151.545178 · Cited by 173
Evolving data into mining solutions for insights
Fayyad, U. M., & Uthurusamy, R. (2002) · Communications of the ACM · doi:10.1145/545151.545174 · Cited by 150
Feature Selection and Classification – A Probabilistic Wrapper Approach
Liu, H., & Setiono, R. (2022) · Industrial and Engineering Applications of Artificial Intelligence and Expert Systems · doi:10.1201/9780429332111-72 · Cited by 146
Self-citation35 Use of distance measures, information measures and error bounds in feature evaluation
Ben-Bassat, M. (1982) · Handbook of statistics · doi:10.1016/s0169-7161(82)02038-0 · Cited by 144
Feature Selection Methods: Genetic Algorithms vs. Greedy-like Search
Vafaie, H. (2009) · Cited by 139
EVOLVING DATA MINING INTO SOLUTIONS FOR INSIGHTS
Uthurusamy, R., & Fayyad, U. M. (2002) · Communications of the ACM · Cited by 131
On Feature Selection: Learning with Exponentially Many Irrelevant Features as Training Examples
Ng, A. Y. (1998) · Cited by 124
Customer Retention via Data Mining
Ng, K., & Liu, H. (2000) · Artificial Intelligence Review · doi:10.1023/a:1006676015154 · Cited by 105
Self-citationContext-Sensitive Feature Selection for Lazy Learners
Domingos, P. (1997) · Springer eBooks · doi:10.1007/978-94-017-2053-3_9 · Cited by 98
A Unifying View on Instance Selection
Reinartz, T. P. (2002) · Data Mining and Knowledge Discovery · doi:10.1023/a:1014047731786 · Cited by 94
Why does bagging work? a Bayesian account and its implications
Domingos, P. (1997) · Cited by 89
Novel Methods for Feature Subset Selection with Respect to Problem Knowledge
Pudil, P., & Novovičová, J. (1998) · Springer eBooks · doi:10.1007/978-1-4615-5725-8_7 · Cited by 68
Control-Sensitive Feature Selection for Lazy Learners
Domingos, F. (1997) · Artificial Intelligence Review · doi:10.1023/a:1006508722917 · Cited by 63
Likelihood-Based Data Squashing: A Modeling Approach to Instance Construction
Madigan, D., Raghavan, N., DuMouchel, W., et al. (2002) · Data Mining and Knowledge Discovery · doi:10.1023/a:1014095614948 · Cited by 62
Handling Large Unsupervised Data via Dimensionality Reduction.
Dash, M., & Liu, H. (1999) · Cited by 38
Self-citationRandomized Variable Elimination
Stracuzzi, D. J., & Utgoff, P. E. (2002) · doi:10.5555/1005332.1044704 · Cited by 27
Discriminative Feature Selection via Multiclass Variable Memory Markov Model
Slonim, N., Bejerano, G., Fine, S., & Tishby, N. (2003) · EURASIP Journal on Advances in Signal Processing · doi:10.1155/s111086570321115x · Cited by 21
Cited by
Neighborhood rough set based heterogeneous feature subset selection
Hu, Q., Yu, D., Liu, J., & Wu, C. (2008) · Information Sciences · doi:10.1016/j.ins.2008.05.024 · Cited by 1,085
A review of feature selection methods on synthetic data
Bolón-Canedo, V., Sánchez-Maroño, N., & Alonso-Betanzos, A. (2013) · Knowledge and Information Systems · doi:10.1007/s10115-012-0487-8 · Cited by 830
A Survey of Evolutionary Algorithms for Clustering
Hruschka, E. R., Campello, R. J. G. B., Freitas, A. A., & de Carvalho, A. C. P. L. F. (2009) · IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews) · doi:10.1109/tsmcc.2008.2007252 · Cited by 759
Feature selection using Joint Mutual Information Maximisation
Bennasar, M., Hicks, Y., & Setchi, R. (2015) · Expert Systems with Applications · doi:10.1016/j.eswa.2015.07.007 · Cited by 659
Social Media Mining: An Introduction
Zafarani, R., Abbasi, M. A., & Liu, H. (2014) · Cambridge University Press eBooks · doi:10.1017/cbo9781139088510 · Cited by 592
Self-citation
Metadata
- Authors
- Liu, H., & Yu, L.
- Venue
- IEEE Transactions on Knowledge and Data Engineering
- Published
- 2005-04
- Volume
- 17
- Issue
- 4
- Pages
- 491-502
- ISSN
- 1041-4347 (ISSN-L)
- OpenAlex
- W2102831150
- Cited by
- 2,706
- References
- 78
- FWCI
- 82.97
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.
