Multimodal neuroimaging feature learning with multimodal stacked deep polynomial networks for diagnosis of Alzheimer’s disease
Abstract
The accurate diagnosis of Alzheimer's disease (AD) and its early stage, i.e., mild cognitive impairment, is essential for timely treatment and possible delay of AD. Fusion of multimodal neuroimaging data, such as magnetic resonance imaging (MRI) and positron emission tomography (PET), has shown its effectiveness for AD diagnosis. The deep polynomial networks (DPN) is a recently proposed deep learning algorithm, which performs well on both large-scale and small-size datasets. In this study, a multimodal stacked DPN (MM-SDPN) algorithm, which MM-SDPN consists of two-stage SDPNs, is proposed to fuse and learn feature representation from multimodal neuroimaging data for AD diagnosis. Specifically speaking, two SDPNs are first used to learn high-level features of MRI and PET, respectively, which are then fed to another SDPN to fuse multimodal neuroimaging information. The proposed MM-SDPN algorithm is applied to the ADNI dataset to conduct both binary classification and multiclass classification tasks. Experimental results indicate that MM-SDPN is superior over the state-of-the-art multimodal feature-learning-based algorithms for AD diagnosis.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Shi et al. achieved 0.9713 ± 0.0444 accuracy for AD vs. NC on ADNI using high-level features of MRI and PET.
“| Shi et al. [79] | high-level features of MRI and PET | MRI, PET | ADNI | 0.9713 ± 0.0444 (ACC: AD vs. NC) |”
From A Comprehensive Review on Synergy of Multi-Modal Data and AI Technologies in Medical Diagnosis (2024) · cites this paper as [79]
Cites
LIBSVM: A library for support vector machines
Chang, C.-C., & Lin, C.-J. (2011) · ACM Transactions on Intelligent Systems and Technology · doi:10.1145/1961189.1961199 · Cited by 40,918 · see citing works
McKhann, G., Drachman, D., Folstein, M., Katzman, R., Price, D., & Stadlan, E. M. (1984) · Neurology · doi:10.1212/wnl.34.7.939 · Cited by 27,915 · see citing works
Wu, Y., & Vapnik, V. N. (1999) · Technometrics · doi:10.2307/1271368 · Cited by 27,236 · see citing works
Reducing the dimensionality of data with neural networks
Hinton, G. E., & Salakhutdinov, R. R. (2006) · Science · doi:10.1126/science.1127647 · Cited by 20,928 · see citing works
Deep learning in neural networks: an overview
Schmidhuber, J. (2015) · Neural Networks · doi:10.1016/j.neunet.2014.09.003 · Cited by 18,570 · see citing works
A fast learning algorithm for deep belief nets
Hinton, G. E., Osindero, S., & Teh, Y.-W. (2006) · Neural Computation · doi:10.1162/neco.2006.18.7.1527 · Cited by 16,336 · see citing works
Representation Learning: A Review and New Perspectives
Bengio, Y., Courville, A., & Vincent, P. (2013) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2013.50 · Cited by 13,001 · see citing works
Libsvm : A library for support vector machines
Ferrari, V. (2008) · Medical Entomology and Zoology · Cited by 10,124 · see citing works
Zhang, Y., Brady, M., & Smith, S. M. (2001) · IEEE Transactions on Medical Imaging · doi:10.1109/42.906424 · Cited by 7,628 · see citing works
Greedy layer-wise training of deep networks
Bengio, Y., Lamblin, P., Popovici, D., & Larochelle, H. (2007) · Advances in Neural Information Processing Systems 19 · doi:10.7551/mitpress/7503.003.0024 · Cited by 4,841 · see citing works
Hypothetical model of dynamic biomarkers of the Alzheimer's pathological cascade
Jack, C. R., Knopman, D. S., Jagust, W. J., Shaw, L. M., Aisen, P. S., Weiner, M. W., Petersen, R. C., & Trojanowski, J. Q. (2010) · The Lancet Neurology · doi:10.1016/s1474-4422(09)70299-6 · Cited by 4,806 · see citing works
A nonparametric method for automatic correction of intensity nonuniformity in MRI data
Sled, J. G., Zijdenbos, A. P., & Evans, A. C. (1998) · IEEE Transactions on Medical Imaging · doi:10.1109/42.668698 · Cited by 4,731 · see citing works
The Alzheimer’s disease neuroimaging initiative (ADNI): MRI methods
Jack, C. R., Bernstein, M. A., Fox, N. C., Thompson, P., Alexander, G., Harvey, D., Borowski, B., Britson, P. J., L. Whitwell, J., Ward, C., et al. (2008) · Journal of Magnetic Resonance Imaging · doi:10.1002/jmri.21049 · Cited by 4,347 · see citing works
Forecasting the global burden of Alzheimer's disease
Brookmeyer, R., Johnson, E., Ziegler‐Graham, K., & Arrighi, H. M. (2007) · Alzheimer's & Dementia · doi:10.1016/j.jalz.2007.04.381 · Cited by 3,370 · see citing works
Jeurgen, K., Assheuer, J., & Paxinos, G. (1997) · Medical Entomology and Zoology · Cited by 1,416 · see citing works
Multimodal classification of Alzheimer's disease and mild cognitive impairment
Zhang, D., Wang, Y., Zhou, L., Yuan, H., & Shen, D. G. (2011) · NeuroImage · doi:10.1016/j.neuroimage.2011.01.008 · Cited by 1,312 · see citing works
(2012) · neural information processing systems · Cited by 1,291 · see citing works
HAMMER: hierarchical attribute matching mechanism for elastic registration
Shen, D. G., & Davatzikos, C. A. (2002) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2002.803111 · Cited by 1,089 · see citing works
Hierarchical Feature Representation and Multimodal Fusion with Deep Learning for AD/MCI Diagnosis
Suk, H.-I., Lee, S.-W., & Shen, D. (2014) · NeuroImage · doi:10.1016/j.neuroimage.2014.06.077 · Cited by 968 · see citing works
Cuingnet, R., Gérardin, E., Tessieras, J., Auzias, G., Lehéricy, S., Habert, M., Marie Chupin, A., Benali, H., & Colliot, O. (2010) · NeuroImage · doi:10.1016/j.neuroimage.2010.06.013 · Cited by 960 · see citing works
Deep convolutional neural networks for multi-modality isointense infant brain image segmentation
Zhang, W., Li, R., Deng, H., Wang, L., Lin, W., Ji, S., & Shen, D. (2015) · NeuroImage · doi:10.1016/j.neuroimage.2014.12.061 · Cited by 817 · see citing works
Multimodal Neuroimaging Feature Learning for Multiclass Diagnosis of Alzheimer's Disease
Liu, S., Liu, S., Cai, W., Che, H., Pujol, S., Kikinis, R., Feng, D., Fulham, M. J., & ADNI (2015) · IEEE Transactions on Biomedical Engineering · doi:10.1109/tbme.2014.2372011 · Cited by 637 · see citing works
Deep learning for neuroimaging: a validation study
Plis, S., Hjelm, D., Salakhutdinov, R., Allen, E. A., Bockholt, H. J., Long, J. D., Johnson, H. J., Paulsen, J. S., Turner, J. A., & Calhoun, V. D. (2014) · Frontiers in Neuroscience · doi:10.3389/fnins.2014.00229 · Cited by 617 · see citing works
A Review of Feature Reduction Techniques in Neuroimaging
Mwangi, B., Tian, T. S., & Soares, J. C. (2014) · Neuroinformatics · doi:10.1007/s12021-013-9204-3 · Cited by 559 · see citing works
Predicting Alzheimer's disease: a neuroimaging study with 3D convolutional neural networksPreprint
Payan, A., & Montana, G. (2015) · International Conference on Pattern Recognition Applications and Methods · doi:10.48550/arxiv.1502.02506 · Cited by 546 · see citing works
Shin, H.-C., Orton, M. R., Collins, D. J., Doran, S. J., & Leach, M. O. (2013) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2012.277 · Cited by 488 · see citing works
Deep Learning-Based Feature Representation for AD/MCI Classification
Suk, H., & Shen, D. G. (2013) · Lecture notes in computer science · doi:10.1007/978-3-642-40763-5_72 · Cited by 463 · see citing works
Deep Learning and Its Applications to Signal and Information Processing [Exploratory DSP
Yu, D., & Deng, L. (2010) · IEEE Signal Processing Magazine · doi:10.1109/msp.2010.939038 · Cited by 459 · see citing works
Random forest-based similarity measures for multi-modal classification of Alzheimer's disease
Gray, K. R., Aljabar, P., Heckemann, R. A., Hammers, A., & Rueckert, D. (2012) · NeuroImage · doi:10.1016/j.neuroimage.2012.09.065 · Cited by 455 · see citing works
Chris Hinrichs, A., Singh, V., Xu, G., & Johnson, S. C. (2010) · NeuroImage · doi:10.1016/j.neuroimage.2010.10.081 · Cited by 405 · see citing works
Ensemble sparse classification of Alzheimer's disease
Liu, M., Zhang, D., & Shen, D. G. (2012) · NeuroImage · doi:10.1016/j.neuroimage.2012.01.055 · Cited by 313 · see citing works
A Robust Deep Model for Improved Classification of AD/MCI Patients
Li, F., Tran, L., Thung, K., Ji, S., Shen, D. G., & Jiang, L. (2015) · IEEE Journal of Biomedical and Health Informatics · doi:10.1109/jbhi.2015.2429556 · Cited by 300 · see citing works
Multi-modal classification of Alzheimer's disease using nonlinear graph fusion
Tong, T., Gray, K. R., Gao, Q., Chen, L., & Rueckert, D. (2016) · Pattern Recognition · doi:10.1016/j.patcog.2016.10.009 · Cited by 258 · see citing works
Manifold Learning of Brain MRIs by Deep Learning
Initiative, A. D. N., Brosch, T., & Tam, R. C. (2013) · Lecture notes in computer science · doi:10.1007/978-3-642-40763-5_78 · Cited by 236 · see citing works
Falahati, F., Westman, E., & Simmons, A. (2014) · Journal of Alzheimer s Disease · doi:10.3233/jad-131928 · Cited by 229 · see citing works
Carneiro, G., Nascimento, J. C., & Freitas, A. (2011) · IEEE Transactions on Image Processing · doi:10.1109/tip.2011.2169273 · Cited by 202 · see citing works
General overview on the merits of multimodal neuroimaging data fusion
Uludaǧ, K., & Roebroeck, A. F. (2014) · NeuroImage · doi:10.1016/j.neuroimage.2014.05.018 · Cited by 192 · see citing works
Dyrba, M., Grothe, M. J., Kirste, T., & Teipel, S. J. (2015) · Human Brain Mapping · doi:10.1002/hbm.22759 · Cited by 185 · see citing works
Pernet, C. R. (2014) · Frontiers in Neuroscience · doi:10.3389/fnins.2014.00001 · Cited by 175 · see citing works
Natural Image Bases to Represent Neuroimaging Data
Gupta, A., Ayhan, M. S., & Maida, A. S. (2013) · Cited by 170 · see citing works
3D Anatomical Atlas of the Human Brain
Kabani, N. J., MacDonald, D. J., Holmes, C. J., & Evans, A. C. (1998) · NeuroImage · doi:10.1016/s1053-8119(18)31550-7 · Cited by 167 · see citing works
Deep sparse multi-task learning for feature selection in Alzheimer’s disease diagnosis
Initiative, T. A. D. N., Suk, H., Lee, S., & Shen, D. G. (2015) · Brain Structure and Function · doi:10.1007/s00429-015-1059-y · Cited by 164 · see citing works
Deep Learning and Its Applications to Signal and Information Processing
Yu, D., & Deng, L. (2011) · IEEE Signal Processing Magazine · Cited by 159 · see citing works
Biomarkers in dementia: clinical utility and new directions
Ahmed, R. M., Paterson, R. W., Warren, J. D., Zetterberg, H., O'Brien, J. T., Fox, N. C., Halliday, G. M., & Schott, J. M. (2014) · Journal of Neurology Neurosurgery & Psychiatry · doi:10.1136/jnnp-2014-307662 · Cited by 155 · see citing works
Neuroimaging and Other Biomarkers for Alzheimer's Disease: The Changing Landscape of Early Detection
Risacher, S. L., & Saykin, A. J. (2013) · Annual Review of Clinical Psychology · doi:10.1146/annurev-clinpsy-050212-185535 · Cited by 155 · see citing works
Manifold regularized multitask feature learning for multimodality disease classification
Jie, B., Zhang, D., Cheng, B., et al. (2014) · Human Brain Mapping · doi:10.1002/hbm.22642 · Cited by 154 · see citing works
Multimodal neuroimaging computing: a review of the applications in neuropsychiatric disorders
Liu, S., Cai, W., Liu, S., Zhang, F., FULHAM, M., Feng, D., Pujol, S., & Kikinis, R. (2015) · Brain Informatics · doi:10.1007/s40708-015-0019-x · Cited by 150 · see citing works
Multiple Kernel Learning in the Primal for Multimodal Alzheimer’s Disease Classification
Liu, F., Zhou, L., Shen, C., & Yin, J. (2013) · IEEE Journal of Biomedical and Health Informatics · doi:10.1109/jbhi.2013.2285378 · Cited by 130 · see citing works
Carneiro, G., & Nascimento, J. C. (2013) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2013.96 · Cited by 122 · see citing works
Brain atlas of the Mongolian gerbil (Meriones unguiculatus) in CT/MRI-aided stereotaxic coordinates
Radtke‐Schuller, S., Schuller, G., Angenstein, F., Großer, O. S., Goldschmidt, J., & Budinger, E. (2016) · Brain Structure and Function · doi:10.1007/s00429-016-1259-0 · Cited by 117 · see citing works
Robust Deformable-Surface-Based Skull-Stripping for Large-Scale Studies
Wang, Y., Nie, J., Yap, P., Shi, F., Guo, L., & Shen, D. G. (2011) · Lecture notes in computer science · doi:10.1007/978-3-642-23626-6_78 · Cited by 116 · see citing works
Scoring by nonlocal image patch estimator for early detection of Alzheimer's disease
Pierrick Coupé, A., Eskildsen, S. F., Manjón, J. V., Fonov, V., Pruessner, J. C., Michèle Allard, A., & Collins, D. L. (2012) · NeuroImage Clinical · doi:10.1016/j.nicl.2012.10.002 · Cited by 109 · see citing works
Wang, Y., Nie, J., Yap, P., Li, G., Shi, F., Geng, X., Guo, L., Shen, D. G., & Initiative, A. D. N. (2014) · PLoS ONE · doi:10.1371/journal.pone.0077810 · Cited by 93 · see citing works
Nonlinear feature transformation and deep fusion for Alzheimer's Disease staging analysis
Shi, B., Chen, Y., Zhang, P., Smith, C. D., & Liu, J. (2016) · Pattern Recognition · doi:10.1016/j.patcog.2016.09.032 · Cited by 92 · see citing works
Alzheimer’s Disease: A Clinical Practice-Oriented Review
Alves, L., Correia, A. S., Miguel, R., Alegria, P., & Bugalho, P. (2012) · Frontiers in Neurology · doi:10.3389/fneur.2012.00063 · Cited by 69 · see citing works
An Algorithm for Training Polynomial NetworksPreprint
Livni, R., Shalev‐Shwartz, S., & Shamir, O. (2013) · arXiv (Cornell University) · doi:10.48550/arxiv.1304.7045 · Cited by 34 · see citing works
A review of heterogeneous data mining for brain disorder identification
Cao, B., Kong, X., & Yu, P. S. (2015) · Brain Informatics · doi:10.1007/s40708-015-0021-3 · Cited by 21 · see citing works
Neuroimaging for the Diagnosis and Study of Psychiatric Disorders [Life Sciences]
Turner, J. A., Potkin, S. G., Brown, G. G., Keator, D. B., McCarthy, G., & Glover, G. H. (2007) · IEEE Signal Processing Magazine · doi:10.1109/msp.2007.4286570 · Cited by 8 · see citing works
Randomized Denoising Autoencoders for Smaller and Efficient Imaging Based AD Clinical Trials
Ithapu, V. K., Singh, V., Okonkwo, O. C., & Johnson, S. C. (2014) · Lecture notes in computer science · doi:10.1007/978-3-319-10470-6_59 · Cited by 6 · see citing works
Neuroimaging for the Diagnosis and Study of Psychiatric Disorders
Turner, J. A., Potkin, S. G., Brown, G. G., et al. (2007) · IEEE Signal Processing Magazine · Cited by 5 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
A survey on deep learning in medical image analysis
Litjens, G., Kooi, T., Bejnordi, B. E., Setio, A. A. A., Ciompi, F., Ghafoorian, M., van der Laak, J. A. W. M., van Ginneken, B., & Sánchez, C. I. (2017) · Medical Image Analysis · doi:10.1016/j.media.2017.07.005 · Cited by 15,332 · see citing works
Wen, J., Thibeau-Sutre, E., Diaz-Melo, M., Samper-González, J., Routier, A., Bottani, S., Dormont, D., Durrleman, S., Burgos, N., & Colliot, O. (2020) · Medical Image Analysis · doi:10.1016/j.media.2020.101694 · Cited by 809 · see citing works
Lian, C., Liu, M., Zhang, J., & Shen, D. (2020) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2018.2889096 · Cited by 544 · see citing works
Li, Y., El Habib Daho, M., Conze, P.-H., Zeghlache, R., Le Boité, H., Tadayoni, R., Cochener, B., Lamard, M., & Quellec, G. (2024) · Computers in Biology and Medicine · doi:10.1016/j.compbiomed.2024.108635 · Cited by 274 · see citing works
Self-citationA Comprehensive Review on Synergy of Multi-Modal Data and AI Technologies in Medical Diagnosis
Xu, X., Li, J., Zhu, Z., Zhao, L., Wang, H., Song, C., Chen, Y., Zhao, Q., Yang, J., & Pei, Y. (2024) · Bioengineering · doi:10.3390/bioengineering11030219 · Cited by 242 · see citing works
Deep learning for Alzheimer's disease diagnosis: A survey
Khojaste-Sarakhsi, M., Haghighi, S. S., Ghomi, S. M. T. F., & Marchiori, E. (2022) · Artificial Intelligence in Medicine · doi:10.1016/j.artmed.2022.102332 · Cited by 194 · see citing works
An effective multimodal image fusion method using MRI and PET for Alzheimer’s disease diagnosis
Song, J., Zheng, J., Li, P., Lu, X., Zhu, G., & Shen, P. (2021) · Frontiers in Digital Health · doi:10.3389/fdgth.2021.637386 · Cited by 178 · see citing works
Multi-modality approaches for medical support systems: A systematic review of the last decade
Salvi, M., Loh, H. W., Seoni, S., Barua, P. D., García, S., Molinari, F., & Acharya, U. R. (2024) · Information Fusion · doi:10.1016/j.inffus.2023.102134 · Cited by 144 · see citing works
Qiu, S., Chang, G. H., Panagia, M., Gopal, D. M., Au, R., & Kolachalama, V. B. (2018) · Alzheimer's & Dementia: Diagnosis, Assessment & Disease Monitoring · doi:10.1016/j.dadm.2018.08.013 · Cited by 110 · see citing works
Relation-induced multi-modal shared representation learning for Alzheimer’s disease diagnosis
Ning, Z., Xiao, Q., Feng, Q., Chen, W., & Zhang, Y. (2021) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2021.3063150 · Cited by 106 · see citing works
Multi-scale enhanced graph convolutional network for mild cognitive impairment detection
Lei, B., Zhu, Y., Yu, S., Hu, H., Xu, Y., Yue, G., Wang, T., Zhao, C., Chen, S., Yang, P., et al. (2023) · Pattern Recognition · doi:10.1016/j.patcog.2022.109106 · Cited by 96 · see citing works
Multi-Modal Neuroimaging Feature Fusion for Diagnosis of Alzheimer’s Disease
Zhang, T., & Shi, M. (2020) · Journal of Neuroscience Methods · doi:10.1016/j.jneumeth.2020.108795 · Cited by 91 · see citing works
Amini, M., Pedram, M. M., Moradi, A., & Ouchani, M. (2021) · Computational and Mathematical Methods in Medicine · doi:10.1155/2021/5514839 · Cited by 84 · see citing works
Deep Learning in Multimodal Medical Image Analysis
Xu, Y. (2019) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-32962-4_18 · Cited by 32 · see citing works
- Authors
- Shi, J., Zheng, X., Li, Y., Zhang, Q., & Ying, S.
- Affiliations
- Shanghai University (CN)
- Shenzhen University (CN)
- Venue
- IEEE Journal of Biomedical and Health Informatics
- Published
- 2018-01
- Volume
- 22
- Issue
- 1
- Pages
- 173-183
- ISSN
- 2168-2194 (ISSN-L)
- 2168-2208
- OpenAlex
- W2574038793
- Cited by
- 444
- References
- 60
- Citations by year
- 2026: 18 · 2025: 35 · 2024: 46 · 2023: 59 · 2022: 60 · 2021: 64 · 2020: 54 · 2019: 55 · 2018: 44 · 2017: 9
- FWCI
- 36.36
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- IEEE Journal of Biomedical and Health Informatics (publishedVersion)
- PubMed (publishedVersion)
- Funders
- National Natural Science Foundation of China (61471231, 81627804, 61401267, U1201256, 61471245, 11471208)
- Projects of Guangdong R/D Foundation and the Fundamental Science Projects of Shenzhen City (JCYJ20140418091413514, ZDSYS20140508141148477, JCYJ20160308095019383)
- Topics
- Health Sciences › Medicine › Psychiatry and Mental health › Dementia and Cognitive Impairment Research
- Health Sciences › Medicine › Physiology › Alzheimer's disease research and treatments
- Health Sciences › Health Professions › Health Information Management › Artificial Intelligence in Healthcare
- Keywords
- Alzheimer's disease diagnosis, mild cognitive impairment, multimodal neuroimaging, magnetic resonance imaging, PET, multimodal fusion, representation learning, ADNI dataset, binary classification, multi-class classification
- MeSH terms
- Machine Learning, Algorithms, Alzheimer Disease, Humans, ROC Curve, Neural Networks, Computer, Neuroimaging, Multimodal Imaging
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review