Dual attention multi-instance deep learning for Alzheimer’s disease diagnosis with structural MRI
Abstract
Structural magnetic resonance imaging (sMRI) is widely used for the brain neurological disease diagnosis, which could reflect the variations of brain. However, due to the local brain atrophy, only a few regions in sMRI scans have obvious structural changes, which are highly correlative with pathological features. Hence, the key challenge of sMRI-based brain disease diagnosis is to enhance the identification of discriminative features. To address this issue, we propose a dual attention multi-instance deep learning network (DA-MIDL) for the early diagnosis of Alzheimer's disease (AD) and its prodromal stage mild cognitive impairment (MCI). Specifically, DA-MIDL consists of three primary components: 1) the Patch-Nets with spatial attention blocks for extracting discriminative features within each sMRI patch whilst enhancing the features of abnormally changed micro-structures in the cerebrum, 2) an attention multi-instance learning (MIL) pooling operation for balancing the relative contribution of each patch and yield a global different weighted representation for the whole brain structure, and 3) an attention-aware global classifier for further learning the integral features and making the AD-related classification decisions. Our proposed DA-MIDL model is evaluated on the baseline sMRI scans of 1689 subjects from two independent datasets (i.e., ADNI and AIBL). The experimental results show that our DA-MIDL model can identify discriminative pathological locations and achieve better classification performance in terms of accuracy and generalizability, compared with several state-of-the-art methods.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Zhu et al. achieved 0.9193 accuracy for AD vs. NC vs. MCI and 0.9287 AUC on ADNI and AIBL using patch proposals selected from MRI scans.
“| Zhu et al. [59] | Patch proposals selected from the MRI scans | MRI | ADNI, AIBL | 0.9193 (ACC: AD vs. NC vs. MCI) 0.9287 (AUC) |”
From A Comprehensive Review on Synergy of Multi-Modal Data and AI Technologies in Medical Diagnosis (2024) · cites this paper as [59]
Transformer is valuable for processing and modeling diverse modalities of medical data, encompassing clinical texts, medical images, and time series data.
“In the realm of medical diagnosis, A transformer proves valuable for processing and modeling diverse modalities of medical data, encompassing clinical texts, medical images, and time series data [59,60,61].”
From A Comprehensive Review on Synergy of Multi-Modal Data and AI Technologies in Medical Diagnosis (2024) · cites this paper as [59]
Cites
Girshick, R. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.169 · Cited by 28,105 · see citing works
CBAM: Convolutional Block Attention Module
Woo, S., Park, J., Lee, J.-Y., & Kweon, I. S. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-01234-2_1 · Cited by 25,853 · see citing works
Jenkinson, M., Beckmann, C. F., Behrens, T. E. J., Woolrich, M. W., & Smith, S. M. (2012) · NeuroImage · doi:10.1016/j.neuroimage.2011.09.015 · Cited by 11,314 · see citing works
Voxel-Based Morphometry—The Methods
Ashburner, J., & Friston, K. J. (2000) · NeuroImage · doi:10.1006/nimg.2000.0582 · Cited by 8,676 · see citing works
Show, Attend and Tell: Neural Image Caption Generation with Visual Attention
Xu, K., Ba, J., Kiros, R., Cho, K., Courville, A., Salakhutdinov, R., Zemel, R., & Bengio, Y. (2015) · arXiv · doi:10.48550/arxiv.1502.03044 · Cited by 8,248 · see citing works
BrainNet Viewer: A Network Visualization Tool for Human Brain Connectomics
Xia, M., Wang, J., & He, Y. (2013) · PLoS ONE · doi:10.1371/journal.pone.0068910 · Cited by 4,106 · see citing works
2013 Alzheimer's disease facts and figures
Thies, W. H., & Bleiler, L. (2013) · Alzheimer s & Dementia · doi:10.1016/j.jalz.2013.02.003 · Cited by 3,331 · see citing works
The clinical use of structural MRI in Alzheimer disease
Frisoni, G. B., Fox, N. C., Jack, C. R., Scheltens, P., & Thompson, P. M. (2010) · Nature Reviews Neurology · doi:10.1038/nrneurol.2009.215 · Cited by 2,113 · see citing works
Memory and Executive Function in Aging and AD
Buckner, R. L. (2004) · Neuron · doi:10.1016/j.neuron.2004.09.006 · Cited by 1,508 · see citing works
Enhancement of MR Images Using Registration for Signal Averaging
Holmes, C. J., Hoge, R. D., Collins, D. L., Woods, R. P., Toga, A. W., & Evans, A. C. (1998) · Journal of Computer Assisted Tomography · doi:10.1097/00004728-199803000-00032 · Cited by 1,422 · see citing works
Multimodal classification of Alzheimer's disease and mild cognitive impairment
Zhang, D., Wang, Y., Zhou, L., Yuan, H., & Shen, D. G. (2011) · NeuroImage · doi:10.1016/j.neuroimage.2011.01.008 · Cited by 1,312 · see citing works
Self-citationThe amygdala: Neurobiological aspects of emotion, memory and mental dysfunction
Scheel-Krüger, J. (1993) · Journal of Chemical Neuroanatomy · doi:10.1016/0891-0618(93)90037-5 · Cited by 1,277 · see citing works
Jovicich, J., Czanner, S., Greve, D. N., Haley, E., van der Kouwe, A. J. W., Gollub, R. L., Kennedy, D. N., Schmitt, F., Brown, G. G., MacFall, J. R., et al. (2005) · NeuroImage · doi:10.1016/j.neuroimage.2005.09.046 · Cited by 1,269 · see citing works
Attention-based Deep Multiple Instance LearningPreprint
Ilse, M., Tomczak, J. M., & Welling, M. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1802.04712 · Cited by 1,235 · see citing works
HAMMER: hierarchical attribute matching mechanism for elastic registration
Shen, D. G., & Davatzikos, C. A. (2002) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2002.803111 · Cited by 1,089 · see citing works
Hierarchical Feature Representation and Multimodal Fusion with Deep Learning for AD/MCI Diagnosis
Suk, H.-I., Lee, S.-W., & Shen, D. (2014) · NeuroImage · doi:10.1016/j.neuroimage.2014.06.077 · Cited by 968 · see citing works
Cuingnet, R., Gérardin, E., Tessieras, J., Auzias, G., Lehéricy, S., Habert, M., Marie Chupin, A., Benali, H., & Colliot, O. (2010) · NeuroImage · doi:10.1016/j.neuroimage.2010.06.013 · Cited by 960 · see citing works
Single subject prediction of brain disorders in neuroimaging: Promises and pitfalls
Arbabshirani, M. R., Plis, S., Sui, J., & Calhoun, V. D. (2017) · NeuroImage · doi:10.1016/j.neuroimage.2016.02.079 · Cited by 912 · see citing works
Wen, J., Thibeau-Sutre, E., Diaz-Melo, M., Samper-González, J., Routier, A., Bottani, S., Dormont, D., Durrleman, S., Burgos, N., & Colliot, O. (2020) · Medical Image Analysis · doi:10.1016/j.media.2020.101694 · Cited by 809 · see citing works
Alzheimer's disease drug development pipeline: 2019
Cummings, J., Lee, G., Ritter, A., Sabbagh, M., & Zhong, K. (2019) · Alzheimer's & Dementia: Translational Research & Clinical Interventions · doi:10.1016/j.trci.2019.05.008 · Cited by 762 · see citing works
Scahill, R. I., Schott, J. M., Stevens, J., Rossor, M. N., & Fox, N. C. (2002) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.052587399 · Cited by 748 · see citing works
In Vivo Mapping of Gray Matter Loss with Voxel-Based Morphometry in Mild Alzheimer's Disease
Baron, J., Chételat, G., Desgranges, B., Perchey, G., Landeau, B., Sayette, V. D. L., & Eustache, F. (2001) · NeuroImage · doi:10.1006/nimg.2001.0848 · Cited by 683 · see citing works
Zhang, D., & Shen, D. G. (2011) · NeuroImage · doi:10.1016/j.neuroimage.2011.09.069 · Cited by 679 · see citing works
Self-citationHippocampus and entorhinal cortex in mild cognitive impairment and early AD
Pennanen, C., Kivipelto, M., Tuomainen, S., Hartikainen, P. H., Hänninen, T., Laakso, M. P., Hallikainen, M., Vanhanen, M., Nissinen, A. M., Helkala, E., et al. (2003) · Neurobiology of Aging · doi:10.1016/s0197-4580(03)00084-8 · Cited by 646 · see citing works
Rathore, S., Habes, M., Iftikhar, M. A., Shacklett, A., & Davatzikos, C. (2017) · NeuroImage · doi:10.1016/j.neuroimage.2017.03.057 · Cited by 622 · see citing works
Liu, M., Li, F., Yan, H., Wang, K., Ma, Y., Shen, L., & Xu, M. (2020) · NeuroImage · doi:10.1016/j.neuroimage.2019.116459 · Cited by 568 · see citing works
Multiple instance classification: Review, taxonomy and comparative study
Amores, J. (2013) · Artificial Intelligence · doi:10.1016/j.artint.2013.06.003 · Cited by 562 · see citing works
Lian, C., Liu, M., Zhang, J., & Shen, D. (2020) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2018.2889096 · Cited by 544 · see citing works
Revisiting multiple instance neural networks
Wang, X., Yan, Y., Tang, P., Bai, X., & Liu, W. (2017) · Pattern Recognition · doi:10.1016/j.patcog.2017.08.026 · Cited by 502 · see citing works
Qiu, S., Joshi, P. S., Miller, M. I., Xue, C., Zhou, X., Karjadi, C., Chang, G. H., Joshi, A. S., Dwyer, B., Zhu, S., et al. (2020) · Brain · doi:10.1093/brain/awaa137 · Cited by 472 · see citing works
Imaging cerebral atrophy: normal ageing to Alzheimer's disease
Fox, N. C., & Schott, J. M. (2004) · The Lancet · doi:10.1016/s0140-6736(04)15441-x · Cited by 431 · see citing works
Landmark-based deep multi-instance learning for brain disease diagnosis
Liu, M., Zhang, J., Adeli, E., & Shen, D. G. (2017) · Medical Image Analysis · doi:10.1016/j.media.2017.10.005 · Cited by 424 · see citing works
Lin, W.-M., Tong, T., Gao, Q., Guo, D., Du, X., Yang, Y., Guo, G., Xiao, M., Du, M., Qu, X., et al. (2018) · Frontiers in Neuroscience · doi:10.3389/fnins.2018.00777 · Cited by 391 · see citing works
Vulnerable Neural Systems and the Borderland of Brain Aging and Neurodegeneration
Jagust, W. J. (2013) · Neuron · doi:10.1016/j.neuron.2013.01.002 · Cited by 321 · see citing works
Ensemble sparse classification of Alzheimer's disease
Liu, M., Zhang, D., & Shen, D. G. (2012) · NeuroImage · doi:10.1016/j.neuroimage.2012.01.055 · Cited by 313 · see citing works
Self-citationCurrent and future treatments in Alzheimer’s disease
Atri, A. (2019) · Seminars in Neurology · doi:10.1055/s-0039-1678581 · Cited by 309 · see citing works
Diagnosis of Alzheimer’s Disease via Multi-Modality 3D Convolutional Neural Network
Huang, Y., Xu, J., Zhou, Y., Tong, T., Zhuang, X., & (ADNI), T. A. D. N. I. (2019) · Frontiers in Neuroscience · doi:10.3389/fnins.2019.00509 · Cited by 296 · see citing works
Eskildsen, S. F., Pierrick Coupé, A., García-Lorenzo, D., Fonov, V., Pruessner, J. C., & Collins, D. L. (2012) · NeuroImage · doi:10.1016/j.neuroimage.2012.09.058 · Cited by 269 · see citing works
Salvatore, C., Cerasa, A., Battista, P., Gilardi, M. C., Quattrone, A., & Castiglioni, I. (2015) · Frontiers in Neuroscience · doi:10.3389/fnins.2015.00307 · Cited by 259 · see citing works
Wang, H., Shen, Y., Wang, S., Xiao, T., Deng, L., Wang, X., & Zhao, X. (2018) · Neurocomputing · doi:10.1016/j.neucom.2018.12.018 · Cited by 258 · see citing works
Falahati, F., Westman, E., & Simmons, A. (2014) · Journal of Alzheimer s Disease · doi:10.3233/jad-131928 · Cited by 229 · see citing works
Detecting Anatomical Landmarks for Fast Alzheimer’s Disease Diagnosis
Zhang, J., Gao, Y., Gao, Y.-Z., Munsell, B., & Shen, D. (2016) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2016.2582386 · Cited by 214 · see citing works
Multiple instance learning for classification of dementia in brain MRI
Tong, T., Wolz, R., Gao, Q., Guerrero, R., Hajnal, J. V., & Rueckert, D. (2014) · Medical Image Analysis · doi:10.1016/j.media.2014.04.006 · Cited by 213 · see citing works
Liu, M., Zhang, D., & Shen, D. G. (2016) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2016.2515021 · Cited by 200 · see citing works
Self-citationCho, Y., Seong, J., Jeong, Y., & Shin, S. Y. (2011) · NeuroImage · doi:10.1016/j.neuroimage.2011.09.085 · Cited by 191 · see citing works
Alzheimer's disease diagnosis based on multiple cluster dense convolutional networks
Li, F., & Liu, M. (2018) · Computerized Medical Imaging and Graphics · doi:10.1016/j.compmedimag.2018.09.009 · Cited by 185 · see citing works
3D Anatomical Atlas of the Human Brain
Kabani, N. J., MacDonald, D. J., Holmes, C. J., & Evans, A. C. (1998) · NeuroImage · doi:10.1016/s1053-8119(18)31550-7 · Cited by 167 · see citing works
Hypergraph based multi-task feature selection for multimodal classification of Alzheimer's disease
Shao, W., Peng, Y., Zu, C., Wang, M., & Zhang, D. (2019) · Computerized Medical Imaging and Graphics · doi:10.1016/j.compmedimag.2019.101663 · Cited by 167 · see citing works
Self-citationVounou, M., Janoušová, E., Wolz, R., Stein, J. L., Thompson, P. M., Rueckert, D., & Montana, G. (2011) · NeuroImage · doi:10.1016/j.neuroimage.2011.12.029 · Cited by 147 · see citing works
Aderghal, K., Khvostikov, A. V., Крылов, А., Benois‐Pineau, J., Afdel, K., & Catheline, G. (2018) · Proceedings - IEEE Symposium on Computer-Based Medical Systems · doi:10.1109/cbms.2018.00067 · Cited by 144 · see citing works
Leandrou, S., Petroudi, S., Kyriacou, P. A., Reyes‐Aldasoro, C. C., & Pattichis, C. S. (2018) · IEEE Reviews in Biomedical Engineering · doi:10.1109/rbme.2018.2796598 · Cited by 135 · see citing works
Compensation for surface coil sensitivity variation in magnetic resonance imaging
Narayana, P. A., Brey, W. W., Kulkarni, M. V., & Sievenpiper, C. (1988) · Magnetic Resonance Imaging · doi:10.1016/0730-725x(88)90401-8 · Cited by 131 · see citing works
Multiple Instance Learning for Heterogeneous Images: Training a CNN for Histopathology
Couture, H. D., Marron, J. S., Perou, C. M., Troester, M. A., & Niethammer, M. (2018) · Lecture notes in computer science · doi:10.1007/978-3-030-00934-2_29 · Cited by 74 · see citing works
Multi-label Thoracic Disease Image Classification with Cross-Attention Networks
Ma, C., Wang, H., & Hoi, S. C. H. (2019) · Lecture notes in computer science · doi:10.1007/978-3-030-32226-7_81 · Cited by 68 · see citing works
Cao, P., Liu, X.-L., Yang, J.-Z., Zhao, D.-Z., Huang, M.-X., Zhang, J., & Zaiane, O. R. (2017) · Comput. Biol. Medicine · doi:10.1016/j.compbiomed.2017.10.002 · Cited by 57 · see citing works
A Biomarker for Alzheimer’s Disease Based on Patterns of Regional Brain Atrophy
Frenzel, S., Wittfeld, K., Habes, M., Klinger-König, J., Bülow, R., Völzke, H., & Grabe, H. J. (2020) · Frontiers in Psychiatry · doi:10.3389/fpsyt.2019.00953 · Cited by 51 · see citing works
Accurate prediction of AD patients using cortical thickness networks
Dai, D., He, H., Vogelstein, J. T., & Hou, Z. (2012) · Machine Vision and Applications · doi:10.1007/s00138-012-0462-0 · Cited by 46 · see citing works
Fang, Z., Chen, Y., Nie, D., Lin, W., & Shen, D. G. (2019) · Lecture notes in computer science · doi:10.1007/978-3-030-32248-9_12 · Cited by 40 · see citing works
Context-Constrained Multiple Instance Learning for Histopathology Image Segmentation
Xu, Y. L., Zhang, J., Chang, E., Lai, M., & Tu, Z. (2012) · Lecture notes in computer science · doi:10.1007/978-3-642-33454-2_77 · Cited by 39 · see citing works
Automatic brain labeling via multi-atlas guided fully convolutional networks
Fang, L., Zhang, L., Nie, D., Cao, X., Rekik, I., Lee, S., He, H., & Shen, D. G. (2018) · Medical Image Analysis · doi:10.1016/j.media.2018.10.012 · Cited by 35 · see citing works
Lian, C., Liu, M., Wang, L., & Shen, D. G. (2019) · Lecture notes in computer science · doi:10.1007/978-3-030-32251-9_18 · Cited by 29 · see citing works
Bui, T. D., Wang, L., Chen, J., Lin, W., Li, G., & Shen, D. G. (2019) · Lecture notes in computer science · doi:10.1007/978-3-030-33391-1_28 · Cited by 23 · see citing works
Tree-Guided Sparse Coding for Brain Disease Classification
Liu, M., Zhang, D., Yap, P., & Shen, D. G. (2012) · Lecture notes in computer science · doi:10.1007/978-3-642-33454-2_30 · Cited by 22 · see citing works
Self-citationIdentification of Cerebral Small Vessel Disease Using Multiple Instance Learning
Chen, L., Tong, T., Ho, C. P., et al. (2015) · Lecture notes in computer science · doi:10.1007/978-3-319-24553-9_64 · Cited by 19 · see citing works
Hierarchical Ensemble of Multi-level Classifiers for Diagnosis of Alzheimer’s Disease
Liu, M., Zhang, D., Yap, P., et al. (2012) · Lecture notes in computer science · doi:10.1007/978-3-642-35428-1_4 · Cited by 18 · see citing works
Self-citationMonte-Carlo Sampling Applied to Multiple Instance Learning for Histological Image Classification
Combalia, M., & Vilaplana, V. (2018) · Lecture notes in computer science · doi:10.1007/978-3-030-00889-5_31 · Cited by 15 · see citing works
Chen, G., Zhao, J., Zhang, R., Wang, T., Zhang, G., & Lei, B. (2019) · Lecture notes in computer science · doi:10.1007/978-3-030-32956-3_21 · Cited by 12 · see citing works
The Amygdala: Neurobiological Aspects of Emotion, Memory, and Mental Dysfunction
Goldstein, L. H. (1992) · PubMed Central · Cited by 7 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
A Comprehensive Review on Synergy of Multi-Modal Data and AI Technologies in Medical Diagnosis
Xu, X., Li, J., Zhu, Z., Zhao, L., Wang, H., Song, C., Chen, Y., Zhao, Q., Yang, J., & Pei, Y. (2024) · Bioengineering · doi:10.3390/bioengineering11030219 · Cited by 242 · see citing works
Deep learning for Alzheimer's disease diagnosis: A survey
Khojaste-Sarakhsi, M., Haghighi, S. S., Ghomi, S. M. T. F., & Marchiori, E. (2022) · Artificial Intelligence in Medicine · doi:10.1016/j.artmed.2022.102332 · Cited by 194 · see citing works
Kaur, A., Mittal, M., Bhatti, J. S., Thareja, S., & Singh, S. (2024) · Artificial Intelligence in Medicine · doi:10.1016/j.artmed.2024.102928 · Cited by 54 · see citing works
Effects of MRI scanner manufacturers in classification tasks with deep learning models
Kushol, R., Parnianpour, P., Wilman, A. H., Kalra, S., & Yang, Y.-H. (2023) · Scientific Reports · doi:10.1038/s41598-023-43715-5 · Cited by 54 · see citing works
- Authors
- Zhu, W., Sun, L., Huang, J., Han, L., & Zhang, D.
- Affiliations
- Nanjing University of Aeronautics and Astronautics (CN)
- Manchester Metropolitan University (GB)
- Venue
- IEEE Transactions on Medical Imaging
- Published
- 2021-09
- Volume
- 40
- Issue
- 9
- Pages
- 2354-2366
- ISSN
- 0278-0062 (ISSN-L)
- 1558-254X
- OpenAlex
- W3158631207
- Cited by
- 278
- References
- 68
- Citations by year
- 2026: 45 · 2025: 72 · 2024: 68 · 2023: 61 · 2022: 28 · 2021: 4
- FWCI
- 28.85
- Type
- journal article
- Language
- en
- Open access
- closed
- Licence
- Locations
- IEEE Transactions on Medical Imaging (publishedVersion)
- PubMed (publishedVersion)
- e-space (Manchester Metropolitan University) (acceptedVersion)
- Funders
- National Natural Science Foundation of China (61861130366, 61876082, 61732006, 62006115)
- National Key Research and Development Program of China (2018YFC2001600, 2018YFC2001602, 2018ZX10201002)
- Royal Society-Academy of Medical Sciences Newton Advanced Fellowship (NAF\R1\180371)
- Topics
- Health Sciences › Medicine › Psychiatry and Mental health › Dementia and Cognitive Impairment Research
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Medical Image Segmentation Techniques
- Life Sciences › Neuroscience › Cognitive Neuroscience › Functional Brain Connectivity Studies
- Keywords
- Alzheimer's disease diagnosis, mild cognitive impairment, structural MRI, multiple instance learning, attention mechanism, patch-based feature extraction, brain atrophy, early diagnosis
- MeSH terms
- Deep Learning, Alzheimer Disease, Attention, Brain, Humans, Magnetic Resonance Imaging, Cognitive Dysfunction
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
