Multimodal attention-based deep learning for Alzheimer’s disease diagnosis
Open accessThe DOI matches its Crossref record; 34 of the 34 references it cites were found in a registry or an index.
Abstract
Objective: Alzheimer’s disease (AD) is the most common neurodegenerative disorder with one of the most complex pathogeneses, making effective and clinically actionable decision support difficult. The objective of this study was to develop a novel multimodal deep learning framework to aid medical professionals in AD diagnosis. Materials and Methods: We present a Multimodal Alzheimer’s Disease Diagnosis framework (MADDi) to accurately detect the presence of AD and mild cognitive impairment (MCI) from imaging, genetic, and clinical data. MADDi is novel in that we use cross-modal attention, which captures interactions between modalities—a method not previously explored in this domain. We perform multi-class classification, a challenging task considering the strong similarities between MCI and AD. We compare with previous state-of-the-art models, evaluate the importance of attention, and examine the contribution of each modality to the model’s performance. Results: MADDi classifies MCI, AD, and controls with 96.88% accuracy on a held-out test set. When examining the contribution of different attention schemes, we found that the combination of cross-modal attention with self-attention performed the best, and no attention layers in the model performed the worst, with a 7.9% difference in F1-scores. Discussion: Our experiments underlined the importance of structured clinical data to help machine learning models contextualize and interpret the remaining modalities. Extensive ablation studies showed that any multimodal mixture of input features without access to structured clinical information suffered marked performance losses. Conclusion: This study demonstrates the merit of combining multiple input modalities via cross-modal attention to deliver highly accurate AD diagnostic decision support.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source doesn’t support the claim as written.
In the citing paper
“such as Golovanevsky et al. (2022), who reported a 27.2 percentage point (from 0.583 to 0.855) accuracy increase over unimodal”
From Navigating the landscape of multimodal AI in medicine: A scoping review on technical challenges and clinical applications (2025) · cites this paper as Golovanevsky et al. (2022)
Checked by a reasoning model — may be wrong.
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“| Golovanevsky et al. [89] | imaging, genetic, and clinical data | Gene-based | GEO | 0.9688 (ACC) |”
From A Comprehensive Review on Synergy of Multi-Modal Data and AI Technologies in Medical Diagnosis (2024) · cites this paper as [89]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“The MADDi cross-modal attention network exemplifies this paradigm by linking structural MRI, genome-wide SNP profiles, and longitudinal clinical data. Its attention maps demonstrate that hippocampal thinning amplifies the effect of inflammation-related SNP clusters on memory decline, suggesting an immune-mediated feedback mechanism266.”
From Artificial intelligence-driven multi-omics approaches in Alzheimer's disease: Progress, challenges, and future directions (2025) · cites this paper as [266]
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., & Polosukhin, I. (2017) · arXiv · doi:10.48550/arxiv.1706.03762 · Cited by 120,404 · cited 2×
“We compute the dot products of the query with all keys, divide each by the square root of d k , and apply a Softmax function, which converts a vector of numbers into a vector of probabilities, to obtain the weights on the values: Following the success of the Transformer [26], we use the multi-head attention module, wh…”
MethodExploiting Generative AI to Scale up Intelligent Tutoring Systems
Jakubův, J., Chvalovský, K., Goertzel, Z., Kaliszyk, C., Olšák, M., Piotrowski, B., Schulz, S., Suda, M., & Urban, J. (2023) · DROPS (Schloss Dagstuhl – Leibniz Center for Informatics) · doi:10.4230/lipics.itp.2023.19 · Cited by 76,658
Dermatologist-level classification of skin cancer with deep neural networks
Esteva, A., Kuprel, B., Novoa, R. A., Ko, J., Swetter, S. M., Blau, H. M., & Thrun, S. (2017) · Nature · doi:10.1038/nature21056 · Cited by 14,362
2013 Alzheimer's disease facts and figures
Alzheimer's Association, Thies, W., & Bleiler, L. (2013) · Alzheimer's & Dementia · doi:10.1016/j.jalz.2013.02.003 · Cited by 3,328
(2011) · Cited by 2,752
LXMERT: Learning Cross-Modality Encoder Representations from Transformers
Tan, H., & Bansal, M. (2019) · Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP) · doi:10.18653/v1/d19-1514 · Cited by 2,319 · cited 4×
“Cross-modal attention: In each bi-directional cross-modal attention layer [22], there are two unidirectional cross-modal attention sub-layers: one from modality 1 to modality 2 and one from modality 2 to modality 1.”
BackgroundMethodInfluentialAPI design for machine learning software: experiences from the scikit-learn project
Buitinck, L., Louppe, G., Blondel, M., Pedregosa, F., Mueller, A., Grisel, O., Niculae, V., Prettenhofer, P., Gramfort, A., Grobler, J., et al. (2013) · arXiv (Cornell University) · doi:10.48550/arxiv.1309.0238 · Cited by 1,871
Mueller, S. G., Weiner, M. W., Thal, L. J., Petersen, R. C., Jack, C. R., Jagust, W., Trojanowski, J. Q., Toga, A. W., & Beckett, L. (2005) · Alzheimer's & Dementia · doi:10.1016/j.jalz.2005.06.003 · Cited by 1,233 · cited 2×
“Many successful studies were conducted using the ADNI dataset [5].”
BackgroundMethodPetersen, R. C. (2016) · Continuum · doi:10.1212/con.0000000000000313 · Cited by 909
Multimodal deep learning for biomedical data fusion: a review
Stahlschmidt, S. R., Ulfenborg, B., & Synnergren, J. (2022) · Briefings in Bioinformatics · doi:10.1093/bib/bbab569 · Cited by 766 · cited 1×
“Even with DL’s advantage, single-modality input is often insufficient to support clinical decision-making [16].”
Background
Cited by
A Comprehensive Review on Synergy of Multi-Modal Data and AI Technologies in Medical Diagnosis
Xu, X., Li, J., Zhu, Z., Zhao, L., Wang, H., Song, C., Chen, Y., Zhao, Q., Yang, J., & Pei, Y. (2024) · Bioengineering · doi:10.3390/bioengineering11030219 · Cited by 242
Multi-modality approaches for medical support systems: A systematic review of the last decade
Salvi, M., Loh, H. W., Seoni, S., Barua, P. D., García, S., Molinari, F., & Acharya, U. R. (2024) · Information Fusion · doi:10.1016/j.inffus.2023.102134 · Cited by 144
“In recent years, there has been a prevalence of DL use and more sophisticated data fusion techniques like graph neural network [29], cross-modal attention [36], and dynamic fusion strategy [61].”
BackgroundSchouten, D., Nicoletti, G., Dille, B., Chia, C., Vendittelli, P., Schuurmans, M., Litjens, G., & Khalili, N. (2025) · Medical Image Analysis · doi:10.1016/j.media.2025.103621 · Cited by 162
Teoh, J. R., Dong, J., Zuo, X., Lai, K. W., Hasikin, K. B., & Wu, X. (2024) · PeerJ Computer Science · doi:10.7717/peerj-cs.2298 · Cited by 139
Ren, F., Wei, J., Chen, Q., Hu, M., Yu, L., Mi, J., Zhou, X., Qin, D., Wu, J., & Wu, A. (2025) · Acta Pharmaceutica Sinica B · doi:10.1016/j.apsb.2025.07.030 · Cited by 97
Metadata
- Authors
- Golovanevsky, M., Eickhoff, C., & Singh, R.
- Affiliations
- Brown University (US)
- Corresponding author
- Eickhoff, Carsten
- Venue
- Journal of the American Medical Informatics Association
- Publisher
- Oxford University Press (OUP)
- Published
- 2022-09-23
- Volume
- 29
- Issue
- 12
- Pages
- 2014-2022
- ISSN
- 1067-5027 (ISSN-L)
- 1527-974X
- PMC
- PMC9667156
- OpenAlex
- W4283205155
- Cited by
- 165
- References
- 32
- Citations by year
- 2026: 50 · 2025: 67 · 2024: 34 · 2023: 13 · 2022: 1
- FWCI
- 18.65
- Type
- journal article
- Language
- en
- Open access
- green
- www.ncbi.nlm.nih.gov
- Licence
- academic.oup.com/journals/pages/open_access/funder_policies/chorus/standard_publication_model (vor)
- Locations
- Journal of the American Medical Informatics Association (publishedVersion)
- PubMed Central (publishedVersion)
- PubMed (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Funders
- National Institute of General Medical Sciences (T32GM128596)
- Topics
- Health Sciences › Medicine › Psychiatry and Mental health › Dementia and Cognitive Impairment Research
- Physical Sciences › Computer Science › Artificial Intelligence › Machine Learning in Healthcare
- Health Sciences › Health Professions › Health Information Management › Artificial Intelligence in Healthcare
- Keywords
- Alzheimer's disease diagnosis, mild cognitive impairment, cross-modal attention, multimodal deep learning, self-attention, structured clinical data, genetic data, neuroimaging, decision support, multi-class classification
- MeSH terms
- Machine Learning, Deep Learning, Alzheimer Disease, Humans, Magnetic Resonance Imaging, Cognitive Dysfunction
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Multimodal attention-based deep learning for Alzheimer’s disease diagnosis. https://app.cochat.ai/papers/10.1093/jamia/ocac168
- The paper
- Golovanevsky, M., Eickhoff, C., & Singh, R. (2022). Multimodal attention-based deep learning for Alzheimer’s disease diagnosis. Journal of the American Medical Informatics Association. https://doi.org/10.1093/jamia/ocac168
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.