Random KNN feature selection - a fast and stable alternative to Random Forests
Open accessThe DOI matches its Crossref record; 5 of the 5 references it cites were found in a registry or an index.
Abstract
BACKGROUND: Successfully modeling high-dimensional data involving thousands of variables is challenging. This is especially true for gene expression profiling experiments, given the large number of genes involved and the small number of samples available. Random Forests (RF) is a popular and widely used approach to feature selection for such "small n, large p problems." However, Random Forests suffers from instability, especially in the presence of noisy and/or unbalanced inputs. RESULTS: We present RKNN-FS, an innovative feature selection procedure for "small n, large p problems." RKNN-FS is based on Random KNN (RKNN), a novel generalization of traditional nearest-neighbor modeling. RKNN consists of an ensemble of base k-nearest neighbor models, each constructed from a random subset of the input variables. To rank the importance of the variables, we define a criterion on the RKNN framework, using the notion of support. A two-stage backward model selection method is then developed based on this criterion. Empirical results on microarray data sets with thousands of variables and relatively few samples show that RKNN-FS is an effective feature selection approach for high-dimensional data. RKNN is similar to Random Forests in terms of classification accuracy without feature selection. However, RKNN provides much better classification accuracy than RF when each method incorporates a feature-selection step. Our results show that RKNN is significantly more stable and more robust than Random Forests for feature selection when the input data are noisy and/or unbalanced. Further, RKNN-FS is much faster than the Random Forests feature selection method (RF-FS), especially for large scale problems, involving thousands of variables and multiple classes. CONCLUSIONS: Given the superiority of Random KNN in classification performance when compared with Random Forests, RKNN-FS's simplicity and ease of implementation, and its superiority in speed and stability, we propose RKNN-FS as a faster and more stable alternative to Random Forests in classification problems involving feature selection for high-dimensional datasets.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Breiman, L. (2001) · Machine Learning · doi:10.1023/a:1010933404324 · Cited by 133,538
Nearest neighbor pattern classification
Cover, T., & Hart, P. (1967) · IEEE Transactions on Information Theory · doi:10.1109/tit.1967.1053964 · Cited by 16,936
Note on the Sampling Error of the Difference Between Correlated Proportions or Percentages
McNemar, Q. (1947) · Psychometrika · doi:10.1007/bf02295996 · Cited by 4,754
Missing Value Estimation Methods for DNA Microarrays
Troyanskaya, O., Cantor, M., Sherlock, G., Brown, P., Hastie, T., Tibshirani, R., Botstein, D., & Altman, R. B. (2001) · Bioinformatics · doi:10.1093/bioinformatics/17.6.520 · Cited by 4,230
Random Forests and Adaptive Nearest Neighbors
Lin, Y., & Jeon, Y. (2006) · Journal of the American Statistical Association · doi:10.1198/016214505000001230 · Cited by 507
Cited by
A Survey on Malware Detection Using Data Mining Techniques
Ye, Y., Li, T., Adjeroh, D., & Iyengar, S. S. (2018) · ACM Computing Surveys · doi:10.1145/3073559 · Cited by 623
Self-citationMachine Learning-Enabled Internet of Things (IoT): Data, Applications, and Industry Perspective
Bzai, J., Alam, F., Dhafer, A., Bojović, M., Altowaijri, S. M., Niazi, I. K., & Mehmood, R. (2022) · Electronics · doi:10.3390/electronics11172676 · Cited by 110
Narath, S. H., Mautner, S. I., Svehlikova, E., Schultes, B., Pieber, T. R., Sinner, F. M., Gander, E., Libiseller, G., Schimek, M. G., Sourij, H., et al. (2016) · PLOS ONE · doi:10.1371/journal.pone.0161425 · Cited by 65
Salient experiences are represented by unique transcriptional signatures in the mouse brain
Mukherjee, D., Ignatowska-Jankowska, B. M., Itskovits, E., Gonzales, B. J., Turm, H., Izakson, L., Haritan, D., Bleistein, N., Cohen, C., Amit, I., et al. (2018) · eLife · doi:10.7554/elife.31220 · Cited by 46
Metadata
- Authors
- Li, S., Harner, E. J., & Adjeroh, D. A.
- Venue
- BMC Bioinformatics
- Published
- 2011-11-18
- Volume
- 12
- Issue
- 1
- Pages
- 450-450
- ISSN
- 1471-2105 (ISSN-L)
- OpenAlex
- W2079177468
- Cited by
- 128
- References
- 5
- FWCI
- 1.53
- Type
- journal article
- Language
- en
- Open access
- gold
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Random KNN feature selection - a fast and stable alternative to Random Forests. https://app.cochat.ai/papers/10.1186/1471-2105-12-450
- The paper
- Li, S., Harner, E. J., & Adjeroh, D. A. (2011). Random KNN feature selection - a fast and stable alternative to Random Forests. BMC Bioinformatics. https://doi.org/10.1186/1471-2105-12-450
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.