The DOI matches its Crossref record; 49 of the 50 references it cites were found in a registry or an index.
Does feature selection improve classification accuracy? Impact of sample size and feature selection on classification using anatomical magnetic resonance images
Abstract
There are growing numbers of studies using machine learning approaches to characterize patterns of anatomical difference discernible from neuroimaging data. The high-dimensionality of image data often raises a concern that feature selection is needed to obtain optimal accuracy. Among previous studies, mostly using fixed sample sizes, some show greater predictive accuracies with feature selection, whereas others do not. In this study, we compared four common feature selection methods. 1) Pre-selected region of interests (ROIs) that are based on prior knowledge. 2) Univariate t-test filtering. 3) Recursive feature elimination (RFE), and 4) t-test filtering constrained by ROIs. The predictive accuracies achieved from different sample sizes, with and without feature selection, were compared statistically. To demonstrate the effect, we used grey matter segmented from the T1-weighted anatomical scans collected by the Alzheimer's disease Neuroimaging Initiative (ADNI) as the input features to a linear support vector machine classifier. The objective was to characterize the patterns of difference between Alzheimer's disease (AD) patients and cognitively normal subjects, and also to characterize the difference between mild cognitive impairment (MCI) patients and normal subjects. In addition, we also compared the classification accuracies between MCI patients who converted to AD and MCI patients who did not convert within the period of 12 months. Predictive accuracies from two data-driven feature selection methods (t-test filtering and RFE) were no better than those achieved using whole brain data. We showed that we could achieve the most accurate characterizations by using prior knowledge of where to expect neurodegeneration (hippocampus and parahippocampal gyrus). Therefore, feature selection does improve the classification accuracies, but it depends on the method adopted. In general, larger sample sizes yielded higher accuracies with less advantage obtained by using knowledge from the existing literature.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Hence, the neural network method critically depends on relatively larger training and test datasets compared to image processing methods [52-54].”
From Automated Defect Detection and Decision-Support in Gas Turbine Blade Inspection (2021) · cites this paper as [52]
In the citing paper
“However, they require significant amounts of data to accurately perform a classification, especially with increasing number of classes [52-54].”
From Automated Defect Detection and Decision-Support in Gas Turbine Blade Inspection (2021) · cites this paper as [52]
Citations
Cites
Regression shrinkage and selection via the lasso
Tibshirani, R. (1996) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/j.2517-6161.1996.tb02080.x · Cited by 51,482
LIBSVM: A library for support vector machines
Chang, C.-C., & Lin, C.-J. (2011) · ACM Transactions on Intelligent Systems and Technology · doi:10.1145/1961189.1961199 · Cited by 40,918
Self-citationThe Nature of Statistical Learning Theory
Vapnik, V. N. (1995) · doi:10.1007/978-1-4757-2440-0 · Cited by 39,138
Book Review: Pattern Recognition and Machine Learning
Nasrabadi, N. M. (2007) · Journal of Electronic Imaging · doi:10.1117/1.2819119 · Cited by 22,066
Regularization and Variable Selection Via the Elastic Net
Zou, H., & Hastie, T. (2005) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/j.1467-9868.2005.00503.x · Cited by 20,957
An Introduction to Support Vector Machines and Other Kernel-based Learning Methods
Cristianini, N., & Shawe-Taylor, J. (2000) · doi:10.1017/cbo9780511801389 · Cited by 13,855
Libsvm : A library for support vector machines
Ferrari, V. (2008) · Medical Entomology and Zoology · Cited by 10,123
Gene selection for cancer classification using support vector machines
Guyon, I., Weston, J., Barnhill, S., & Vapnik, V. (2002) · Machine Learning · doi:10.1023/a:1012487302797 · Cited by 9,954
Pattern Recognition and Machine Learning
(2007) · Kybernetes · doi:10.1108/03684920710743466 · Cited by 8,545
An introduction to variable and feature selection
GuyonIsabelle, & ElisseeffAndré (2003) · Journal of Machine Learning Research · doi:10.5555/944919.944968 · Cited by 8,354
A fast diffeomorphic image registration algorithm
Ashburner, J. (2007) · NeuroImage · doi:10.1016/j.neuroimage.2007.07.007 · Cited by 8,122
Ashburner, J., & Friston, K. J. (2005) · NeuroImage · doi:10.1016/j.neuroimage.2005.02.018 · Cited by 7,507
An Introduction to Variable and Feature Selection.
Guyon, I., & Elisseeff, A. (2003) · Cited by 6,576
The Nature of Statistical Learning Theory
Sain, S. R., & Vapnik, V. N. (1996) · Technometrics · doi:10.2307/1271324 · Cited by 5,528
A review of feature selection techniques in bioinformatics
Saeys, Y., Inza, I., & Larrañaga, P. (2007) · Bioinformatics · doi:10.1093/bioinformatics/btm344 · Cited by 5,389
An Introduction to Support Vector Machines and Other Kernel‐based Learning Methods
(2001) · Kybernetes · doi:10.1108/k.2001.30.1.103.6 · Cited by 3,359
Multimodal classification of Alzheimer's disease and mild cognitive impairment
Zhang, D., Wang, Y., Zhou, L., Yuan, H., & Shen, D. G. (2011) · NeuroImage · doi:10.1016/j.neuroimage.2011.01.008 · Cited by 1,340
Construction of a 3D probabilistic atlas of human cortical structures
Shattuck, D. W., Mirza, M., Adisetiyo, V., Hojatkashani, C., Salamon, G., Narr, K. L., Poldrack, R. A., Bilder, R. M., & Toga, A. W. (2007) · NeuroImage · doi:10.1016/j.neuroimage.2007.09.031 · Cited by 1,208
Automatic classification of MR scans in Alzheimer’s disease.
Kloppel, S., Stonnington, C. M., Chu, C., Draganski, B., Scahill, R. I., Rohrer, J. D., Fox, N. C., Jack, C. R., Ashburner, J., & Frackowiak, R. S. J. (2008) · Brain · doi:10.1093/brain/awm319 · Cited by 1,170
Cuingnet, R., Gérardin, E., Tessieras, J., Auzias, G., Lehéricy, S., Habert, M., Marie Chupin, A., Benali, H., & Olivier Colliot, A. (2010) · NeuroImage · doi:10.1016/j.neuroimage.2010.06.013 · Cited by 995
Mackay, D. J. C. (1995) · Network Computation in Neural Systems · doi:10.1088/0954-898x/6/3/011 · Cited by 915
Mourão-Miranda, J., Bokde, A. L. W., Born, C., Hampel, H. J., & Stetter, M. (2005) · NeuroImage · doi:10.1016/j.neuroimage.2005.06.070 · Cited by 700
Martino, F. D., Valente, G., Staeren, N., Ashburner, J., Goebel, R., & Formisano, E. (2008) · NeuroImage · doi:10.1016/j.neuroimage.2008.06.037 · Cited by 514
Gérardin, E., Chételat, G., Marie Chupin, A., Cuingnet, R., Desgranges, B., Kim, H., Niethammer, M., Dubois, B., Lehéricy, S., Garnero, L., et al. (2009) · NeuroImage · doi:10.1016/j.neuroimage.2009.05.036 · Cited by 433
Disease state prediction from resting state functional connectivity
Craddock, R. C., Holtzheimer, P. E., Hu, X. P., & Mayberg, H. S. (2009) · Magnetic Resonance in Medicine · doi:10.1002/mrm.22159 · Cited by 411
Mapping gray matter loss with voxel-based morphometry in mild cognitive impairment
Chételat, G., Desgranges, B., de la Sayette, V., Viader, F., Eustache, F., & Baron, J. (2002) · NeuroReport · doi:10.1097/00001756-200210280-00022 · Cited by 405
Detection of grey matter loss in mild Alzheimer's disease with voxel based morphometry
Frisoni, G. B. (2002) · Journal of Neurology Neurosurgery & Psychiatry · doi:10.1136/jnnp.73.6.657 · Cited by 393
Sparse estimation automatically selects voxels relevant for the decoding of fMRI activity patterns
Yamashita, O., Sato, M.-A., Yoshioka, T., Tong, F., & Kamitani, Y. (2008) · NeuroImage · doi:10.1016/j.neuroimage.2008.05.050 · Cited by 319
Sparse logistic regression for whole-brain classification of fMRI data
Ryali, S., Supekar, K., Abrams, D. A., & Menon, V. (2010) · NeuroImage · doi:10.1016/j.neuroimage.2010.02.040 · Cited by 271
Predicting clinical scores from magnetic resonance scans in Alzheimer's disease
Stonnington, C. M., Chu, C., Klöppel, S., Jack, C. R., Ashburner, J., & Frackowiak, R. S. J. (2010) · NeuroImage · doi:10.1016/j.neuroimage.2010.03.051 · Cited by 261
Computing average shaped tissue probability templates
Ashburner, J., & Friston, K. J. (2008) · NeuroImage · doi:10.1016/j.neuroimage.2008.12.008 · Cited by 232
Prognostic and diagnostic potential of the structural neuroanatomy of depression
Costafreda, S. G., Chu, C., Ashburner, J., & Fu, C. H. Y. (2009) · PLoS ONE · doi:10.1371/journal.pone.0006353 · Cited by 232
Self-citationA voxel based morphometry study on mild cognitive impairment
Pennanen, C. (2004) · Journal of Neurology Neurosurgery & Psychiatry · doi:10.1136/jnnp.2004.035600 · Cited by 218
fMRI Activity Patterns in Human LOC Carry Information about Object Exemplars within Category
Eger, E., Ashburner, J., Haynes, J., Dolan, R. J., & Rees, G. E. (2007) · Journal of Cognitive Neuroscience · doi:10.1162/jocn.2008.20019 · Cited by 191
Prediction and interpretation of distributed neural activity with sparse models
CARROLL, M., Cecchi, G., Rish, I., Garg, R. D., & RAO, A. (2008) · NeuroImage · doi:10.1016/j.neuroimage.2008.08.020 · Cited by 164
Yong Fan, A., Shen, D. G., & Davatzikos, C. A. (2005) · Lecture notes in computer science · doi:10.1007/11566465_1 · Cited by 151
Multivariate examination of brain abnormality using both structural and functional MRI
Yong Fan, A., Rao, H., Hurt, H., Giannetta, J. M., Korczykowski, M., Shera, D. M., Avants, B., Gee, J. C., Wang, J., & Shen, D. G. (2007) · NeuroImage · doi:10.1016/j.neuroimage.2007.04.009 · Cited by 135
Filipovych, R., & Davatzikos, C. A. (2011) · NeuroImage · doi:10.1016/j.neuroimage.2010.12.066 · Cited by 130
Interpretable Classifiers for fMRI Improve Prediction of Purchases
Grosenick, L., Greer, S. M., & Knutson, B. D. (2008) · IEEE Transactions on Neural Systems and Rehabilitation Engineering · doi:10.1109/tnsre.2008.926701 · Cited by 109
Marquand, A. F., Simoni, S. D., O’Daly, O., Williams, S., Mourão-Miranda, J., & Mehta, M. A. (2011) · Neuropsychopharmacology · doi:10.1038/npp.2011.9 · Cited by 96
Automatic detection of preclinical neurodegeneration
Klöppel, S., Chu, C., Tan, G. C.-Y., Draganski, B., Johnson, H. J., Paulsen, J. S., Kienzle, W., Tabrizi, S. J., Ashburner, J., Frackowiak, R. S. J., et al. (2009) · Neurology · doi:10.1212/01.wnl.0000341768.28646.b6 · Cited by 92
Independent Component Analysis-Based Classification of Alzheimer's Disease MRI Data
Yang, W., Lui, R. L.-M., Gao, J., Chan, T. F.-C., Yau, S., Sperling, R. A., & Huang, X. (2011) · Journal of Alzheimer s Disease · doi:10.3233/jad-2011-101371 · Cited by 85
Precuneus amyloid burden is associated with reduced cholinergic activity in Alzheimer disease
Ikonomović, M. D., Klunk, W. E., Abrahamson, E. E., Wuu, J., Mathis, C. A., Scheff, S. W., Mufson, E. J., & DeKosky, S. T. (2011) · Neurology · doi:10.1212/wnl.0b013e3182231419 · Cited by 82
Kernel regression for fMRI pattern prediction
Chu, C., Ni, Y., Tan, G. C.-Y., Saunders, C. J., & Ashburner, J. (2010) · NeuroImage · doi:10.1016/j.neuroimage.2010.03.058 · Cited by 81
Pathological Aging of the Brain
de Bastos-Leite, A. J., Scheltens, P., & Barkhof, F. (2004) · Topics in Magnetic Resonance Imaging · doi:10.1097/01.rmr.0000168070.90113.dc · Cited by 67
Predictive Accuracy as an Achievable Goal of Science
Forster, M. R. (2002) · Philosophy of Science · doi:10.1086/341840 · Cited by 57
Multivariate models of inter-subject anatomical variability
Ashburner, J., & Klöppel, S. (2010) · NeuroImage · doi:10.1016/j.neuroimage.2010.03.059 · Cited by 46
Discrimination of AD and normal subjects from MRI: Anatomical versus statistical regions
Peláez‐Coca, M. D., Bossa, M. N., & Olmos, S. C. (2010) · Neuroscience Letters · doi:10.1016/j.neulet.2010.10.007 · Cited by 13
Chu, C., Bandettini, P. A., Ashburner, J., Marquand, A. F., & Klöppel, S. (2010) · doi:10.1109/wbd.2010.11 · Cited by 6
Flagged first, then by citations
1 of the 50 references couldn’t be matched to a known paper.
Research works printed but not linked: 2
- Pelaez-Coca, 2011, Discrimination of AD and normal subjects from MRI: anatomical versus statistical regions, Neurosci. Lett., 487, 113, 10.1016/j.neulet.2010.10.007 · DOI
- Tipping, 2001, Sparse Bayesian learning and the relevance vector machine, J. Mach. Learn. Res., 1, 211
Cited by
Sorted by citations
Orrù, G., Pettersson-Yeo, W., Marquand, A. F., Sartori, G., & Mechelli, A. (2012) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2012.01.004 · Cited by 1,012
Single subject prediction of brain disorders in neuroimaging: Promises and pitfalls
Arbabshirani, M. R., Plis, S., Sui, J., & Calhoun, V. D. (2017) · NeuroImage · doi:10.1016/j.neuroimage.2016.02.079 · Cited by 912
Rathore, S., Habes, M., Iftikhar, M. A., Shacklett, A., & Davatzikos, C. (2017) · NeuroImage · doi:10.1016/j.neuroimage.2017.03.057 · Cited by 622
A Review of Feature Reduction Techniques in Neuroimaging
Mwangi, B., Tian, T. S., & Soares, J. C. (2014) · Neuroinformatics · doi:10.1007/s12021-013-9204-3 · Cited by 559
PRoNTo: Pattern Recognition for Neuroimaging Toolbox
Schrouff, J., Rosa, M. J., Rondina, J. M., Marquand, A. F., Chu, C., Ashburner, J., Phillips, C. L. M., Richiardi, J., & Mourão-Miranda, J. (2013) · Neuroinformatics · doi:10.1007/s12021-013-9178-1 · Cited by 424
Metadata
- Authors
- Chu, C., Hsu, A.-L., Chou, K.-H., Bandettini, P., & Lin, C.
- Affiliations
- National Institutes of Health (US)
- National Yang Ming University (TW)
- Venue
- NeuroImage
- Publisher
- Elsevier BV
- Published
- 2012-03
- Volume
- 60
- Issue
- 1
- Pages
- 59-70
- ISSN
- 1053-8119 (ISSN-L)
- OpenAlex
- W2019583087
- Cited by
- 319
- References
- 50
- Citations by year
- 2026: 3 · 2025: 14 · 2024: 14 · 2023: 24 · 2022: 22 · 2021: 38 · 2020: 33 · 2019: 25 · 2018: 19 · 2017: 32 · 2016: 25 · 2015: 20 · 2014: 25 · 2013: 16 · 2012: 9
- FWCI
- 11.01
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- NeuroImage (publishedVersion)
- PubMed (publishedVersion)
- Funders
- Wellcome Trust
- National Science Council (100-2627-B-010-004-, NSC 100-2628-E-010-002-MY3)
- National Institutes of Health
- National Health Research Institutes (NHRI-EX100-9813EC)
- National Institute of Mental Health
- Topics
- Health Sciences › Medicine › Psychiatry and Mental health › Dementia and Cognitive Impairment Research
- Life Sciences › Neuroscience › Cognitive Neuroscience › Functional Brain Connectivity Studies
- Health Sciences › Health Professions › Health Information Management › Artificial Intelligence in Healthcare
- Keywords
- feature selection, sample size, anatomical MRI, Alzheimer's disease, mild cognitive impairment, grey matter, region of interest selection, recursive feature elimination, hippocampus, parahippocampal gyrus, neurodegeneration, linear support vector machine, classification accuracy, ADNI
- MeSH terms
- Aged, Alzheimer Disease, Female, Humans, Magnetic Resonance Imaging, Male, Reproducibility of Results, Sample Size, Neuroimaging, Cognitive Dysfunction
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
