The DOI matches its Crossref record.
Using mutual information for selecting features in supervised neural net learning
Abstract
This paper investigates the application of the mutual information criterion to evaluate a set of candidate features and to select an informative subset to be used as input data for a neural network classifier. Because the mutual information measures arbitrary dependencies between random variables, it is suitable for assessing the "information content" of features in complex classification tasks, where methods bases on linear relations (like the correlation) are prone to mistakes. The fact that the mutual information is independent of the coordinates chosen permits a robust estimation. Nonetheless, the use of the mutual information for tasks characterized by high input dimensionality requires suitable approximations because of the prohibitive demands on computation and samples. An algorithm is proposed that is based on a "greedy" selection of the features and that takes both the mutual information with respect to the output class and with respect to the already-selected features into account. Finally the results of a series of experiments are discussed.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“This is in sharp contrast with Shannon information, which is well known to be difficult to estimate for high dimensional or continuous random variables (Battiti, 1994).”
From A Theory of Usable Information Under Computational Constraints (2020) · cites this paper as Battiti, 1994
Citations
Cites
No references could be resolved yet.
Research works printed but not linked: 23
- 10.1109/72.80236 · DOI
- 10.1109/5.58325 · DOI · Paper page
- li, 1990, Mutual information functions versus correlation functions, J Stat Phys, 60, 823, 10.1007/BF01025996 · DOI
- 10.1162/neco.1989.1.3.402 · DOI
- 10.1109/72.80306 · DOI
- etti-spaccamela, 1990, Learning Boolean formulae using decision trees, Proc 3rd Workshop on Parallel Architectures and Neural Networks, 797
- mozer, 1989, Advances in neural information processing systems, 1, 107
- 10.1023/A:1022611825350 · DOI
- pernot, 1991, Determining the relevant parameters for the classification on a multi-layer perceptron: Application to radar data, Proc 1991 Int Conf Artificial Neural Networks ICANN-91, 797
- rumelhart, 1987, Parallel Distributed Processing Explorations in the Microstructure of Cognition, 1, 318
- 10.1111/j.1469-1809.1936.tb02137.x · DOI · Paper page
- duda, 1973, Pattern Classification and Scene Analysis
- 10.1103/PhysRevA.33.1134 · DOI
- 10.1016/0167-2789(89)90263-7 · DOI
- 10.1016/0893-6080(88)90023-8 · DOI
- 10.1016/0893-6080(89)90003-8 · DOI · Paper page
- bridle, 1990, Advances in neural information processing systems, 2, 211
- 10.1016/0893-6080(89)90030-0 · DOI
- 10.1109/18.87006 · DOI
- shannon, 1949, The Mathematical Theory of Communication
- teukolsky, 1988, Numerical Recipes in C
- weigend, 1991, Advances in neural information processing systems, 3, 875
- zemel, 1991, Advances in neural information processing systems, 3, 299
Cited by
Sorted by citations
A survey on feature selection methods
Chandrashekar, G., & Sahin, F. (2014) · Computers & Electrical Engineering · doi:10.1016/j.compeleceng.2013.11.024 · Cited by 5,189
Feature Selection: A Data Perspective
Li, J., Cheng, K., Wang, S., Morstatter, F., Trevino, R. P., Tang, J., & Liu, H. (2018) · ACM Computing Surveys · doi:10.1145/3136625 · Cited by 2,442
Ghamisi, P., Yokoya, N., Li, J., Liao, W., Liu, S., Plaza, J., Rasti, B., & Plaza, A. (2017) · IEEE Geoscience and Remote Sensing Magazine · doi:10.1109/mgrs.2017.2762087 · Cited by 909
MIFS-ND: A mutual information-based feature selection method
Hoque, N., Bhattacharyya, D. K., & Kalita, J. K. (2014) · Expert Systems with Applications · doi:10.1016/j.eswa.2014.04.019 · Cited by 442
Tutorial on practical tips of the most influential data preprocessing algorithms in data mining
García, S., Luengo, J., & Herrera, F. (2016) · Knowledge-Based Systems · doi:10.1016/j.knosys.2015.12.006 · Cited by 310
Metadata
- Authors
- Battiti, R.
- Venue
- IEEE Transactions on Neural Networks
- Published
- 1994-07
- Volume
- 5
- Issue
- 4
- Pages
- 537-550
- ISSN
- 1045-9227
- Cited by
- 2,148
- References
- 0
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Using mutual information for selecting features in supervised neural net learning. https://app.cochat.ai/papers/10.1109/72.298224
- The paper
- Battiti, R. (1994). Using mutual information for selecting features in supervised neural net learning. IEEE Transactions on Neural Networks. https://doi.org/10.1109/72.298224
