The DOI matches its Crossref record; 45 of the 45 references it cites were found in a registry or an index.
Feature selection using Joint Mutual Information Maximisation
Open accessAbstract
Feature selection is used in many application areas relevant to expert and intelligent systems, such as data mining and machine learning, image processing, anomaly detection, bioinformatics and natural language processing. Feature selection based on information theory is a popular approach due its computational efficiency, scalability in terms of the dataset dimensionality, and independence from the classifier. Common drawbacks of this approach are the lack of information about the interaction between the features and the classifier, and the selection of redundant and irrelevant features. The latter is due to the limitations of the employed goal functions leading to overestimation of the feature significance. To address this problem, this article introduces two new nonlinear feature selection methods, namely Joint Mutual Information Maximisation (JMIM) and Normalised Joint Mutual Information Maximisation (NJMIM); both these methods use mutual information and the ‘maximum of the minimum’ criterion, which alleviates the problem of overestimation of the feature significance as demonstrated both theoretically and experimentally. The proposed methods are compared using eleven publically available datasets with five competing methods. The results demonstrate that the JMIM method outperforms the other methods on most tested public datasets, reducing the relative average classification error by almost 6% in comparison to the next best performing method. The statistical significance of the results is confirmed by the ANOVA test. Moreover, this method produces the best trade-off between accuracy and stability.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Elements of Information Theory
Cover, T. M., & Thomas, J. A. (2001) · Wiley eBooks · doi:10.1002/0471200611 · Cited by 36,808
UCI Machine Learning Repository
Asuncion, A. (2007) · Medical Entomology and Zoology · Cited by 25,410
(2012) · Encyclopedia of the Sciences of Learning · doi:10.1007/978-1-4419-1428-6_5195 · Cited by 19,370
Elements of Information Theory
Cover, T. M., & Thomas, J. A. (2005) · doi:10.1002/047174882x · Cited by 13,791
Turk, M., & Pentland, A. (1991) · Journal of Cognitive Neuroscience · doi:10.1162/jocn.1991.3.1.71 · Cited by 13,733
The Hungarian method for the assignment problem
Kuhn, H. W. (1955) · Naval Research Logistics Quarterly · doi:10.1002/nav.3800020109 · Cited by 13,120
Peng, H., Long, F., & Ding, C. (2005) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2005.159 · Cited by 10,544
An introduction to variable and feature selection
GuyonIsabelle, & ElisseeffAndré (2003) · Journal of Machine Learning Research · doi:10.5555/944919.944968 · Cited by 7,715
Statistical pattern recognition: a review
Jain, A. K., Duin, P. W., & Mao, J. (2000) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/34.824819 · Cited by 6,767
An Introduction to Variable and Feature Selection.
Guyon, I., & Elisseeff, A. (2003) · Cited by 6,576
A review of feature selection techniques in bioinformatics
Saeys, Y., Inza, I., & Larrañaga, P. (2007) · Bioinformatics · doi:10.1093/bioinformatics/btm344 · Cited by 5,244
A survey on feature selection methods
Chandrashekar, G., & Sahin, F. (2014) · Computers & Electrical Engineering · doi:10.1016/j.compeleceng.2013.11.024 · Cited by 5,189
Thirteen Ways to Look at the Correlation Coefficient
Lee Rodgers, J., & Nicewander, W. A. (1988) · The American Statistician · doi:10.1080/00031305.1988.10475524 · Cited by 4,052
Abe, S. (2001) · Springer eBooks · doi:10.1007/978-1-4471-0285-4 · Cited by 3,721
A Practical Approach to Feature Selection
Kira, K., & Rendell, L. A. (1992) · Machine Learning Proceedings 1992 · doi:10.1016/b978-1-55860-247-2.50037-1 · Cited by 2,978
Toward integrating feature selection algorithms for classification and clustering
Liu, H., & Yu, L. (2005) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2005.66 · Cited by 2,706
Using mutual information for selecting features in supervised neural net learning
Battiti, R. (1994) · IEEE Transactions on Neural Networks · doi:10.1109/72.298224 · Cited by 2,148
Efficient Feature Selection via Analysis of Relevance and Redundancy
Yu, L., & Liu, H. (2004) · doi:10.5555/1005332.1044700 · Cited by 1,983
Supervised and Unsupervised Discretization of Continuous Features
Dougherty, J., Kohavi, R., & Sahami, M. (1995) · Machine Learning Proceedings 1995 · doi:10.1016/b978-1-55860-377-6.50032-3 · Cited by 1,936
Feature Extraction: Foundations and Applications (Studies in Fuzziness and Soft Computing)
Guyon, I., Gunn, S., Nikravesh, M., & Zadeh, L. A. (2006) · Springer eBooks · Cited by 1,845
Feature extraction : foundations and applications
Guyon, I., Gunn, S. R., Nikravesh, M., & zadeh, L. N. (2006) · Cited by 1,658
A direct LDA algorithm for high-dimensional data — with application to face recognition
Yu, H., & Yang, J. (2001) · Pattern Recognition · doi:10.1016/s0031-3203(00)00162-x · Cited by 1,578
Conditional Likelihood Maximisation: A Unifying Framework for Mutual Information Feature Selection
Brown, G., Pocock, A., Zhao, M., & Luján, M. (2012) · Research Explorer (The University of Manchester) · Cited by 1,359
Normalized Mutual Information Feature Selection
Estevez, P. A., Tesmer, M., Perez, C. A., & Zurada, J. M. (2009) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.2008.2005601 · Cited by 1,288
Computational Methods of Feature Selection
(2007) · Chapman & Hall/CRC eBooks · doi:10.1201/9781584888796 · Cited by 1,271
A review of feature selection methods based on mutual information
Vergara, J. R., & Estévez, P. A. (2014) · Neural Computing and Applications · doi:10.1007/s00521-013-1368-0 · Cited by 1,255
Input feature selection for classification problems
Kwak, N., & Choi, C.-H. (2002) · IEEE Transactions on Neural Networks · doi:10.1109/72.977291 · Cited by 938
A review of feature selection methods on synthetic data
Bolón-Canedo, V., Sánchez-Maroño, N., & Alonso-Betanzos, A. (2013) · Knowledge and Information Systems · doi:10.1007/s10115-012-0487-8 · Cited by 830 · cited 1×
“For a more detailed review of the feature selection methods recent review articles in this area are recommended (Bolón-Canedo et al., 2013; Brown et al., 2012; Chandrashekar & Sahin, 2014; Vergara & Estévez, 2014).”
Fast Binary Feature Selection with Conditional Mutual Information
FleuretFrançois (2004) · Journal of Machine Learning Research · doi:10.5555/1005332.1044711 · Cited by 593
Minimum redundancy feature selection from microarray gene expression data
Ding, C., & Peng, H. (2004) · Computational Systems Bioinformatics. CSB2003. Proceedings of the 2003 IEEE Bioinformatics Conference. CSB2003 · doi:10.1109/csb.2003.1227396 · Cited by 506
MIFS-ND: A mutual information-based feature selection method
Hoque, N., Bhattacharyya, D. K., & Kalita, J. K. (2014) · Expert Systems with Applications · doi:10.1016/j.eswa.2014.04.019 · Cited by 442 · cited 2×
“Feature selection is used in many application areas relevant to expert and intelligent systems, such as data mining and machine learning, image processing, anomaly detection, bioinformatics and natural language processing (Hoque, Bhattacharyya, & Kalita, 2014).”
On the relationship between feature selection and classification accuracy
Janecek, A., Gansterer, W. N., Demel, M., & Ecker, G. F. (2008) · Phaidra (Universität Wien) · Cited by 373
A stability index for feature selection
Kuncheva, L. I. (2007) · Conference on Artificial Intelligence for Applications · Cited by 354
Information-Theoretic Feature Selection in Microarray Data Using Variable Complementarity
Meyer, P. E., Schretter, C., & Bontempi, G. (2008) · IEEE Journal of Selected Topics in Signal Processing · doi:10.1109/jstsp.2008.923858 · Cited by 280
Object recognition with informative features and linear classification
Vidal-Naquet, & Ullman (2003) · Proceedings Ninth IEEE International Conference on Computer Vision · doi:10.1109/iccv.2003.1238356 · Cited by 266
Machine Learning Based on Attribute Interactions
Jakulin, A. (2005) · Repository of the University of Ljubljana (University of Ljubljana) · Cited by 248
Stable feature selection via dense feature groups
Yu, L., Ding, C., & Loscalzo, S. (2008) · Proceedings of the 14th ACM SIGKDD international conference on Knowledge discovery and data mining · doi:10.1145/1401890.1401986 · Cited by 178
Fast multi-label feature selection based on information-theoretic feature ranking
Lee, J., & Kim, D.-W. (2015) · Pattern Recognition · doi:10.1016/j.patcog.2015.04.009 · Cited by 165
On the Use of Variable Complementarity for Feature Selection in Cancer Classification
Meyer, P. E., & Bontempi, G. (2006) · Lecture Notes in Computer Science · doi:10.1007/11732242_9 · Cited by 159
An evaluation of classifier-specific filter measure performance for feature selection
Freeman, C., Kulić, D., & Basir, O. (2014) · Pattern Recognition · doi:10.1016/j.patcog.2014.11.010 · Cited by 97
A Powerful Feature Selection approach based on Mutual Information
Akadi, A. E., Ouardighi, A. E., & Aboutajdine, D. (2008) · Cited by 71
Lin, T., Li, H.-T., & Tsai, K. (2003) · Journal of Chemical Information and Computer Sciences · doi:10.1021/ci030295a · Cited by 57
Feature selection using data envelopment analysis
Zhang, Y., Yang, A., Xiong, C., Wang, T., & Zhang, Z. (2014) · Knowledge-Based Systems · doi:10.1016/j.knosys.2014.03.022 · Cited by 46
Attribute Interactions in Machine Learning
Jakulin, A. (2003) · Repository of the University of Ljubljana (University of Ljubljana) · Cited by 30
Cited by
Stability of feature selection algorithm: A review
Khaire, U. M., & Dhanalakshmi, R. (2022) · Journal of King Saud University - Computer and Information Sciences · doi:10.1016/j.jksuci.2019.06.012 · Cited by 630
Greylag Goose Optimization: Nature-inspired optimization algorithm
El-kenawy, E.-S. M., Khodadadi, N., Mirjalili, S., Abdelhamid, A. A., Eid, M. M., & Ibrahim, A. (2024) · Expert Systems with Applications · doi:10.1016/j.eswa.2023.122147 · Cited by 586
A survey on wearable sensor modality centred human activity recognition in health care
Wang, Y., Cang, S., & Yu, H. (2019) · Expert Systems with Applications · doi:10.1016/j.eswa.2019.04.057 · Cited by 409
Thakkar, A., & Lohiya, R. (2022) · Artificial Intelligence Review · doi:10.1007/s10462-021-10037-9 · Cited by 350
Feature selection methods for big data bioinformatics: A survey from the search perspective
Wang, L., Wang, Y., & Chang, Q. (2016) · Methods · doi:10.1016/j.ymeth.2016.08.014 · Cited by 331
Metadata
- Authors
- Bennasar, M., Hicks, Y., & Setchi, R.
- Venue
- Expert Systems with Applications
- Publisher
- Elsevier BV
- Published
- 2015-12
- Volume
- 42
- Issue
- 22
- Pages
- 8520-8532
- ISSN
- 0957-4174 (ISSN-L)
- OpenAlex
- W2143043751
- Cited by
- 659
- References
- 45
- FWCI
- 33.16
- Type
- journal article
- Language
- en
- Open access
- hybrid
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.