Particle swarm optimization for feature selection in classification: a multi-objective approach
Abstract
Classification problems often have a large number of features in the data sets, but not all of them are useful for classification. Irrelevant and redundant features may even reduce the performance. Feature selection aims to choose a small number of relevant features to achieve similar or even better classification performance than using all features. It has two main conflicting objectives of maximizing the classification performance and minimizing the number of features. However, most existing feature selection algorithms treat the task as a single objective problem. This paper presents the first study on multi-objective particle swarm optimization (PSO) for feature selection. The task is to generate a Pareto front of nondominated solutions (feature subsets). We investigate two PSO-based multi-objective feature selection algorithms. The first algorithm introduces the idea of nondominated sorting into PSO to address feature selection problems. The second algorithm applies the ideas of crowding, mutation, and dominance to PSO to search for the Pareto front solutions. The two multi-objective algorithms are compared with two conventional feature selection methods, a single objective feature selection method, a two-stage feature selection algorithm, and three well-known evolutionary multi-objective algorithms on 12 benchmark data sets. The experimental results show that the two PSO-based multi-objective algorithms can automatically evolve a set of nondominated solutions. The first algorithm outperforms the two conventional methods, the single objective method, and the two-stage algorithm. It achieves comparable results with the existing three well-known multi-objective algorithms in most cases. The second algorithm achieves better results than the first algorithm and all other methods mentioned previously.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Automatic feature selection techniques include particle swarm optimization, recursive feature elimination, principal component analysis, and auto encoders.
“Automatic feature selection techniques include, e.g., particle swarm optimization [54], recursive feature elimination, principal component analysis [17], and auto encoders [35].”
From The pipeline for the continuous development of artificial intelligence models—Current state of research and practice (2023) · cites this paper as [54]
Cites
Kennedy, J., & Eberhart, R. (2002) · Proceedings of ICNN'95 - International Conference on Neural Networks · doi:10.1109/icnn.1995.488968 · Cited by 58,192 · see citing works
A fast and elitist multiobjective genetic algorithm: NSGA-II
Deb, K., Pratap, A., Agarwal, S., & Meyarivan, T. (2002) · IEEE Transactions on Evolutionary Computation · doi:10.1109/4235.996017 · Cited by 50,053 · see citing works
The Elements of Statistical Learning
Hastie, T., Tibshirani, R., & Friedman, J. (2009) · Springer Series in Statistics · doi:10.1007/978-0-387-84858-7 · Cited by 25,970 · see citing works
UCI Machine Learning Repository
Asuncion, A. (2007) · Medical Entomology and Zoology · Cited by 24,190 · see citing works
A modified particle swarm optimizer
Shi, Y., & Eberhart, R. (1998) · 1998 IEEE International Conference on Evolutionary Computation Proceedings. IEEE World Congress on Computational Intelligence (Cat. No.98TH8360) · doi:10.1109/icec.1998.699146 · Cited by 11,497 · see citing works
Wrappers for feature subset selection
Kohavi, R., & John, G. H. (1997) · Artificial Intelligence · doi:10.1016/s0004-3702(97)00043-x · Cited by 8,940 · see citing works
(2019) · WORLD SCIENTIFIC eBooks · doi:10.1142/9789811205675_0004 · Cited by 6,316 · see citing works
SPEA2: Improving the strength pareto evolutionary algorithm
Zitzler, E., Laumanns, M., & Thiele, L. (2001) · doi:10.3929/ethz-a-004284029 · Cited by 5,069 · see citing works
A discrete binary version of the particle swarm algorithm
Kennedy, J., & Eberhart, R. C. (2002) · 1997 IEEE International Conference on Systems, Man, and Cybernetics. Computational Cybernetics and Simulation · doi:10.1109/icsmc.1997.637339 · Cited by 4,767 · see citing works
Feature Selection for Classification
Dash, M., & Liu, H. (1997) · Intelligent Data Analysis · doi:10.1016/s1088-467x(97)00008-5 · Cited by 3,594 · see citing works
(2000) · Applied Physics Letters · doi:10.1162/153244303322753616 · Cited by 3,581 · see citing works
Floating search methods in feature selection
Pudil, P., Novovičová, J., & Kittler, J. (1994) · Pattern Recognition Letters · doi:10.1016/0167-8655(94)90127-9 · Cited by 3,129 · see citing works
A Practical Approach to Feature Selection
Kira, K., & Rendell, L. A. (1992) · Elsevier eBooks · doi:10.1016/b978-1-55860-247-2.50037-1 · Cited by 3,090 · see citing works
Computational Intelligence: An Introduction
(2006) · Springer eBooks · doi:10.1007/1-84628-184-9_1 · Cited by 2,027 · see citing works
Engelbrecht, A. P. (2007) · Wiley eBooks · doi:10.1002/9780470512517 · Cited by 1,459 · see citing works
An analysis of particle swarm optimizers
van den Bergh, F., & Engelbrecht, A. P. (2002) · UpSpace Institutional Repository (University of Pretoria) · Cited by 1,421 · see citing works
Lin, S.-W., Ying, K., Chen, S., et al. (2007) · Expert Systems with Applications · doi:10.1016/j.eswa.2007.08.088 · Cited by 906 · see citing works
A Direct Method of Nonparametric Measurement Selection
Whitney, A. W. (1971) · IEEE Transactions on Computers · doi:10.1109/t-c.1971.223410 · Cited by 855 · see citing works
Feature selection based on rough sets and particle swarm optimization
Wang, X., Yang, J. M., Teng, X., et al. (2006) · Pattern Recognition Letters · doi:10.1016/j.patrec.2006.09.003 · Cited by 845 · see citing works
Improving PSO-Based Multi-objective Optimization Using Crowding, Mutation and ∈-Dominance
Sierra, M. R., & Coello, C. A. C. (2005) · Lecture notes in computer science · doi:10.1007/978-3-540-31880-4_35 · Cited by 784 · see citing works
On the effectiveness of receptors in recognition systems
Marill, T., & Green, D. (1963) · IEEE Transactions on Information Theory · doi:10.1109/tit.1963.1057810 · Cited by 665 · see citing works
Feature subset selection in large dimensionality domains
Gheyas, I. A., & Smith, L. S. (2009) · Pattern Recognition · doi:10.1016/j.patcog.2009.06.009 · Cited by 597 · see citing works
Improved binary PSO for feature selection using gene expression data
Chuang, L., Chang, H., Tu, C.-J., & Yang, C. (2007) · Computational Biology and Chemistry · doi:10.1016/j.compbiolchem.2007.09.005 · Cited by 596 · see citing works
A distributed PSO–SVM hybrid system with feature selection and parameter optimization
Huang, C.-L., & Dun, J.-F. (2007) · Applied Soft Computing · doi:10.1016/j.asoc.2007.10.007 · Cited by 576 · see citing works
Caruana, R., & Freitag, D. (1994) · Elsevier eBooks · doi:10.1016/b978-1-55860-335-6.50012-x · Cited by 557 · see citing works
A Non-dominated Sorting Particle Swarm Optimizer for Multiobjective Optimization
Li, X. (2003) · Lecture notes in computer science · doi:10.1007/3-540-45105-6_4 · Cited by 536 · see citing works
Learning Boolean concepts in the presence of many irrelevant features
Almuallim, H., & Dietterich, T. G. (1994) · Artificial Intelligence · doi:10.1016/0004-3702(94)90084-1 · Cited by 519 · see citing works
Wrapper–Filter Feature Selection Algorithm Using a Memetic Framework
Zhu, Z., Ong, Y.-S., & Dash, M. (2007) · IEEE Transactions on Systems Man and Cybernetics Part B (Cybernetics) · doi:10.1109/tsmcb.2006.883267 · Cited by 444 · see citing works
Ünler, A., & Murat, A. (2010) · European Journal of Operational Research · doi:10.1016/j.ejor.2010.02.032 · Cited by 437 · see citing works
Theory of the hypervolume indicator
Auger, A., Bader, J., Brockhoff, D., et al. (2009) · doi:10.1145/1527125.1527138 · Cited by 378 · see citing works
Genetic programming for simultaneous feature selection and classifier design
Muni, D. P., Pal, N. R., & Das, J. (2006) · IEEE Transactions on Systems Man and Cybernetics Part B (Cybernetics) · doi:10.1109/tsmcb.2005.854499 · Cited by 358 · see citing works
An improved particle swarm optimization for feature selection
Liu, Y., Wang, G., Chen, H., Dong, H., Zhu, X., & Wang, S. (2011) · Journal of Bionic Engineering · doi:10.1016/s1672-6529(11)60020-6 · Cited by 324 · see citing works
Large-scale attribute selection using wrappers
Gütlein, M., Frank, E., Hall, M., & Karwath, A. (2009) · IEEE Symposium on Computational Intelligence and Data Mining (CIDM) · doi:10.1109/cidm.2009.4938668 · Cited by 269 · see citing works
Different metaheuristic strategies to solve the feature selection problem
Yusta, S. C. (2008) · Pattern Recognition Letters · doi:10.1016/j.patrec.2008.11.012 · Cited by 261 · see citing works
Using Decision Trees to Improve Case-Based Learning
Cardie, C. (1993) · Elsevier eBooks · doi:10.1016/b978-1-55860-307-3.50010-1 · Cited by 258 · see citing works
Improved binary particle swarm optimization using catfish effect for feature selection
Chuang, L., Tsai, S.-W., & Yang, C. (2011) · Expert Systems with Applications · doi:10.1016/j.eswa.2011.04.057 · Cited by 240 · see citing works
Statistics on the table: The history of statistical concepts and methods
Schneider, I. (2005) · The Mathematical Intelligencer · doi:10.1007/bf02984818 · Cited by 170 · see citing works
Binary particle swarm optimisation for feature selection: A filter based approach
Cervante, L., Xue, B., Zhang, M., & Shang, L. (2012) · IEEE Congress on Evolutionary Computation (CEC) · doi:10.1109/cec.2012.6256452 · Cited by 167 · see citing works
Self-citationMulti-objective Feature Selection with NSGA II
Hamdani, T. M., Won, J.-M., Alimi, A. M., et al. (2007) · Lecture notes in computer science · doi:10.1007/978-3-540-71618-1_27 · Cited by 163 · see citing works
Evolutionary Rough Feature Selection in Gene Expression Data
Banerjee, M., Mitra, S., & Banka, H. (2007) · IEEE Transactions on Systems Man and Cybernetics Part C (Applications and Reviews) · doi:10.1109/tsmcc.2007.897498 · Cited by 144 · see citing works
Java-ML: A Machine Learning Library
Abeel, T., de Peer, Y. V., & Saeys, Y. (2009) · Ghent University Academic Bibliography (Ghent University) · doi:10.5555/1577069.1577103 · Cited by 113 · see citing works
Ant colony optimization based network intrusion feature selection and detection
Gao, H., Yang, H.-H., & Wang, X. (2005) · International Conference on Machine Learning and Cybernetics · doi:10.1109/icmlc.2005.1527615 · Cited by 87 · see citing works
New fitness functions in binary particle swarm optimisation for feature selection
Xue, B., Zhang, M., & Browne, W. N. (2012) · IEEE Congress on Evolutionary Computation (CEC) · doi:10.1109/cec.2012.6256617 · Cited by 85 · see citing works
Self-citationFeature subset selection by particle swarm optimization with fuzzy fitness function
Chakraborty, B. (2008) · doi:10.1109/iske.2008.4731082 · Cited by 75 · see citing works
Genetic algorithm with fuzzy fitness function for feature selection
Chakraborty, B. (2002) · doi:10.1109/isie.2002.1026085 · Cited by 73 · see citing works
An approach to feature selection for keystroke dynamics systems based on PSO and feature weighting
Azevedo, G. L. F. B. G., Cavalcanti, G. D. C., & Filho, E. C. B. C. (2007) · IEEE Congress on Evolutionary Computation (CEC) · doi:10.1109/cec.2007.4424936 · Cited by 72 · see citing works
Particle Swarm Optimization based Adaboost for face detection
Mohemmed, A. W., Zhang, M., & Johnston, M. (2009) · IEEE Congress on Evolutionary Computation (CEC) · doi:10.1109/cec.2009.4983254 · Cited by 60 · see citing works
Self-citationChaotic maps in binary particle swarm optimization for feature selection
Yang, C., Chuang, L., Li, J.-C., et al. (2008) · doi:10.1109/smcia.2008.5045944 · Cited by 54 · see citing works
The Mathematical Intelligencer
Pickover, C. A. (1992) · Leonardo · doi:10.2307/1575881 · Cited by 52 · see citing works
Pareto front feature selection
Neshatian, K., & Zhang, M. (2009) · Genetic and Evolutionary Computation Conference (GECCO) · doi:10.1145/1569901.1570040 · Cited by 49 · see citing works
Self-citationGenetic Programming for Feature Subset Ranking in Binary Classification Problems
Neshatian, K., & Zhang, M. (2009) · Lecture notes in computer science · doi:10.1007/978-3-642-01181-8_11 · Cited by 48 · see citing works
Self-citationCervante, L., Xue, B., Shang, L., et al. (2012) · Lecture notes in computer science · doi:10.1007/978-3-642-35101-3_27 · Cited by 43 · see citing works
Self-citationA Rough Set Based Hybrid Method to Feature Selection
He, M. (2008) · International Symposium on Knowledge Acquisition and Modeling (KAM) · doi:10.1109/kam.2008.12 · Cited by 35 · see citing works
Using genetic programming for context-sensitive feature scoring in classification problems
Neshatian, K., & Zhang, M. (2011) · Connection Science · doi:10.1080/09540091.2011.630065 · Cited by 34 · see citing works
Self-citationXue, B., Zhang, M., & Browne, W. N. (2012) · QUT ePrints (Queensland University of Technology) · Cited by 28 · see citing works
Self-citationAdaptive Particle Swarm Optimizer for Feature Selection
Esseghir, M. A., Gonçalves, G., & Slimani, Y. (2010) · Lecture notes in computer science · doi:10.1007/978-3-642-15381-5_28 · Cited by 26 · see citing works
Genetic Programming for Feature Ranking in Classification Problems
Neshatian, K., Zhang, M., & Andreae, P. M. (2008) · Lecture notes in computer science · doi:10.1007/978-3-540-89694-4_55 · Cited by 26 · see citing works
Self-citationImproving Relevance Measures Using Genetic Programming
Neshatian, K., & Zhang, M. (2012) · Lecture notes in computer science · doi:10.1007/978-3-642-29139-5_9 · Cited by 25 · see citing works
Self-citationUnsupervised Elimination of Redundant Features Using Genetic Programming
Neshatian, K., & Zhang, M. (2009) · Lecture notes in computer science · doi:10.1007/978-3-642-10439-8_44 · Cited by 25 · see citing works
Self-citationHamed, H. N. A., Kasabov, N. K., & Mariyam, S. (2011) · InTech eBooks · doi:10.5772/10545 · Cited by 18 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
Particle Swarm Optimization: A Comprehensive Survey
Shami, T. M., El-Saleh, A. A., Alswaitti, M., Al-Tashi, Q., Summakieh, M. A., & Mirjalili, S. (2022) · IEEE Access · doi:10.1109/access.2022.3142859 · Cited by 1,433 · see citing works
Rolling element bearing fault diagnosis using convolutional neural network and vibration image
Hoang, D.-T., & Kang, H.-J. (2019) · Cognitive Systems Research · doi:10.1016/j.cogsys.2018.03.002 · Cited by 450 · see citing works
Steidl, M., Felderer, M., & Ramler, R. (2023) · Journal of Systems and Software · doi:10.1016/j.jss.2023.111615 · Cited by 126 · see citing works
- Authors
- Xue, B., Zhang, M., & Browne, W. N.
- Affiliations
- Victoria University of Wellington (NZ)
- Venue
- IEEE Transactions on Cybernetics
- Published
- 2013-12
- Volume
- 43
- Issue
- 6
- Pages
- 1656-1671
- ISSN
- 2168-2267 (ISSN-L)
- 2168-2275
- OpenAlex
- W2013885787
- Cited by
- 1,282
- References
- 61
- Citations by year
- 2026: 52 · 2025: 87 · 2024: 96 · 2023: 130 · 2022: 126 · 2021: 156 · 2020: 139 · 2019: 144 · 2018: 123 · 2017: 84 · 2016: 61 · 2015: 47 · 2014: 26 · 2013: 10
- FWCI
- 25.94
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- IEEE Transactions on Cybernetics (publishedVersion)
- PubMed (publishedVersion)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Metaheuristic Optimization Algorithms Research
- Physical Sciences › Computer Science › Computational Theory and Mathematics › Advanced Multi-Objective Optimization Algorithms
- Physical Sciences › Computer Science › Artificial Intelligence › Evolutionary Algorithms and Applications
- Keywords
- feature selection, particle swarm optimization, multi-objective optimization, classification, Pareto front, non-dominated sorting, crowding, genetic mutations, feature subsets, feature redundancy, evolutionary algorithms
- MeSH terms
- Algorithms, Artificial Intelligence, Decision Support Techniques, Pattern Recognition, Automated, Information Storage and Retrieval
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
