Identifying progression-specific Alzheimer’s subtypes using multimodal transformer
Open accessAbstract
Alzheimer's disease (AD) is the most prevalent neurodegenerative disease, yet its current treatments are limited to stopping disease progression. Moreover, the effectiveness of these treatments remains uncertain due to the heterogeneity of the disease. Therefore, it is essential to identify disease subtypes at a very early stage. Current data-driven approaches can be used to classify subtypes during later stages of AD or related disorders, but making predictions in the asymptomatic or prodromal stage is challenging. Furthermore, the classifications of most existing models lack explainability, and these models rely solely on a single modality for assessment, limiting the scope of their analysis. Thus, we propose a multimodal framework that utilizes early-stage indicators, including imaging, genetics, and clinical assessments, to classify AD patients into progression-specific subtypes at an early stage. In our framework, we introduce a tri-modal co-attention mechanism (Tri-COAT) to explicitly capture cross-modal feature associations. Data from the Alzheimer's Disease Neuroimaging Initiative (ADNI) (slow progressing = 177, intermediate = 302, and fast = 15) were used to train and evaluate Tri-COAT using a 10-fold stratified cross-testing approach. Our proposed model outperforms baseline models and sheds light on essential associations across multimodal features supported by known biological mechanisms. The multimodal design behind Tri-COAT allows it to achieve the highest classification area under the receiver operating characteristic curve while simultaneously providing interpretability to the model predictions through the co-attention mechanism.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Tri-COAT demonstrates that hippocampal atrophy modulates the predictive contribution of lipidomic and genomic features, bridging molecular changes with regional neurodegeneration.
“Additionally, multimodal attention networks such as Tri-COAT demonstrate that hippocampal atrophy modulates the predictive contribution of lipidomic and genomic features, bridging molecular changes with regional neurodegeneration264.”
From Artificial intelligence-driven multi-omics approaches in Alzheimer's disease: Progress, challenges, and future directions (2025) · cites this paper as [264]
Section 4.3 reports that Tri-COAT learned putative associations among genetic features, brain regions, and clinical symptoms. It does not report lipidomic features or demonstrate that hippocampal atrophy modulates the predictive contribution of lipidomic and genomic features; Section 3.1 describes the model inputs as imaging, genotype, and clinical data. No page numbers are available in our copy. (LLM, conf=0.90)
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Scikit-learn: Machine Learning in Python
Pedregosa, F., Varoquaux, G., Gramfort, A., Michel, V., Thirion, B., Grisel, O., Blondel, M., Müller, A., Nothman, J., Louppe, G., et al. (2012) · arXiv · doi:10.48550/arxiv.1201.0490 · Cited by 71,016 · see citing works
PLINK: a tool set for whole-genome association and population-based linkage analyses.
Purcell, S., Neale, B., Todd-Brown, K., Thomas, L., Ferreira, M. A. R., Bender, D., Maller, J., Sklar, P., de Bakker, P. I. W., Daly, M. J., et al. (2007) · The American Journal of Human Genetics · doi:10.1086/519795 · Cited by 34,089 · see citing works
McKhann, G. M., Knopman, D. S., Chertkow, H., Hyman, B. T., Jack, C. R., Kawas, C. H., Klunk, W. E., Koroshetz, W. J., Manly, J. J., Mayeux, R., et al. (2011) · Alzheimer's & Dementia · doi:10.1016/j.jalz.2011.03.005 · Cited by 18,657 · see citing works
Fischl, B. (2012) · NeuroImage · doi:10.1016/j.neuroimage.2012.01.021 · Cited by 9,526 · see citing works
Measuring the thickness of the human cerebral cortex from magnetic resonance images
Fischl, B. R., & Dale, A. M. (2000) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.200033797 · Cited by 5,789 · see citing works
The Alzheimer’s disease neuroimaging initiative (ADNI): MRI methods
Jack, C. R., Bernstein, M. A., Fox, N. C., Thompson, P., Alexander, G., Harvey, D., Borowski, B., Britson, P. J., L. Whitwell, J., Ward, C., et al. (2008) · Journal of Magnetic Resonance Imaging · doi:10.1002/jmri.21049 · Cited by 4,347 · see citing works
circlize implements and enhances circular visualization in R
Gu, Z., Gu, L., Eils, R., Schlesner, M., & Brors, B. (2014) · Bioinformatics · doi:10.1093/bioinformatics/btu393 · Cited by 4,290 · see citing works
Advancing research diagnostic criteria for Alzheimer's disease: the IWG-2 criteria
Dubois, B., Feldman, H. H., Jacova, C., Hampel, H., Molinuevo, J. L., Blennow, K., DeKosky, S. T., Gauthier, S., Selkoe, D., Bateman, R., et al. (2014) · The Lancet Neurology · doi:10.1016/s1474-4422(14)70090-0 · Cited by 3,321 · see citing works
New insights into the genetic etiology of Alzheimer’s disease and related dementias
Bellenguez, C., Küçükali, F., Jansen, I. E., Kleineidam, L., Moreno-Grau, S., Amin, N., Naj, A. C., Campos-Martin, R., Grenier-Boley, B., Andrade, V., et al. (2022) · Nature Genetics · doi:10.1038/s41588-022-01024-z · Cited by 2,498 · see citing works
2022 Alzheimer's disease facts and figures
Anonymous (2022) · Alzheimer s & Dementia · doi:10.1002/alz.12638 · Cited by 2,384 · see citing works
A Simple Generalisation of the Area Under the ROC Curve for Multiple Class Classification Problems
Hand, D. J., & Till, R. J. (2001) · Machine Learning · doi:10.1023/a:1010920819831 · Cited by 2,276 · see citing works
The NHGRI-EBI GWAS Catalog: knowledgebase and deposition resource
Sollis, E., Mosaku, A., Abid, A., Buniello, A., Cerezo, M., Gil, L., Groza, T., Güneş, O., Hall, P., Hayhurst, J., et al. (2023) · Nucleic Acids Research · doi:10.1093/nar/gkac1010 · Cited by 1,641 · see citing works
Flamingo: a Visual Language Model for Few-Shot Learning
Alayrac, J.-B., Donahue, J., Luc, P., Miech, A., Barr, I., Hasson, Y., Lenc, K., Mensch, A., Millican, K., Reynolds, M., et al. (2022) · arXiv · doi:10.48550/arxiv.2204.14198 · Cited by 1,555 · see citing works
Huang, S.-C., Pareek, A., Seyyedi, S., Banerjee, I., & Lungren, M. P. (2020) · npj Digital Medicine · doi:10.1038/s41746-020-00341-z · Cited by 934 · see citing works
Swanson, C. J., Zhang, Y., Dhadda, S., Wang, J., Kaplow, J., Lai, R. Y. K., Lannfelt, L., Bradley, H., Rabe, M., Koyama, A., et al. (2021) · Alzheimer's Research & Therapy · doi:10.1186/s13195-021-00813-8 · Cited by 847 · see citing works
Tau pathology and neurodegeneration contribute to cognitive impairment in Alzheimer’s disease
Bejanin, A., Schonhaut, D. R., La Joie, R., Kramer, J. H., Baker, S. L., Sosa, N., Ayakta, N., Cantwell, A., Janabi, M., Lauriola, M., et al. (2017) · Brain · doi:10.1093/brain/awx243 · Cited by 778 · see citing works
Four distinct trajectories of tau deposition identified in Alzheimer’s disease
Initiative, T. A. D. N., Vogel, J. W., Young, A. L., Oxtoby, N. P., Smith, R., Ossenkoppele, R., Strandberg, O. T., La Joie, R., Aksman, L. M., Grothe, M. J., et al. (2021) · Nature Medicine · doi:10.1038/s41591-021-01309-6 · Cited by 741 · see citing works
VATT: Transformers for Multimodal Self-Supervised Learning from Raw Video, Audio and TextPreprint
Akbari, H., Yuan, L., Qian, R., Chuang, W.-H., Chang, S., Cui, Y., & Gong, B. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2104.11178 · Cited by 433 · see citing works
Multimodal Co-Attention Transformer for Survival Prediction in Gigapixel Whole Slide Images
Chen, R. J., Lu, M. Y., Weng, W.-H., Chen, T. Y., Williamson, D. F., Manz, T., Shady, M., & Mahmood, F. (2021) · 2021 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00398 · Cited by 398 · see citing works
Dhillon, S. (2021) · Drugs · doi:10.1007/s40265-021-01569-z · Cited by 293 · see citing works
Dubois, B., von Arnim, C. A. F., Burnie, N., Bozeat, S., & Cummings, J. (2023) · Alzheimer's Research & Therapy · doi:10.1186/s13195-023-01314-6 · Cited by 288 · see citing works
Zhou, T., Thung, K., Zhu, X., & Shen, D. G. (2018) · Human Brain Mapping · doi:10.1002/hbm.24428 · Cited by 270 · see citing works
Shcherbinin, S., Evans, C. D., Lu, M., Andersen, S. W., Pontecorvo, M. J., Willis, B. A., Gueorguieva, I., Hauck, P. M., Brooks, D. A., Mintun, M. A., et al. (2022) · JAMA Neurology · doi:10.1001/jamaneurol.2022.2793 · Cited by 210 · see citing works
A Method for Estimating Progression Rates in Alzheimer Disease
Doody, R. S., Massman, P. J., & Dunn, J. K. (2001) · Archives of Neurology · doi:10.1001/archneur.58.3.449 · Cited by 194 · see citing works
A multiomics approach to heterogeneity in Alzheimer’s disease: focused review and roadmap
Badhwar, A., McFall, G. P., Sapkota, S., Black, S. E., Chertkow, H. M., Duchesne, S., Masellis, M., Li, L., Dixon, R. A., & Bellec, P. (2019) · Brain · doi:10.1093/brain/awz384 · Cited by 168 · see citing works
Predicting progression of Alzheimer's disease
Doody, R. S., Pavlik, V., Massman, P. J., Rountree, S. D., Darby, E. J., & Chan, W. (2010) · Alzheimer s Research & Therapy · doi:10.1186/alzrt25 · Cited by 128 · see citing works
Dadu, A., Satone, V. K., Kaur, R., Hashemi, S. H., Leonard, H. L., Iwaki, H., Makarious, M. B., Billingsley, K. J., Bandrés‐Ciga, S., Sargent, L. J., et al. (2022) · npj Parkinson s Disease · doi:10.1038/s41531-022-00439-z · Cited by 121 · see citing works
Multiview learning for understanding functional multiomics
Nguyen, N. D., & Wang, D. (2020) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1007677 · Cited by 107 · see citing works
Spatial-Temporal Patterns of β-Amyloid Accumulation
Collij, L. E., Salvadó, G., Wottschel, V., Mastenbroek, S. E., Schoenmakers, P., Heeman, F., Aksman, L., Wink, A. M., van Berckel, B. N. M., van der Flier, W. M., et al. (2022) · Neurology · doi:10.1212/wnl.0000000000200148 · Cited by 97 · see citing works
El–Sappagh, S., Ali, F., Abuhmed, T., Singh, J., & Alonso, J. M. (2022) · Neurocomputing · doi:10.1016/j.neucom.2022.09.009 · Cited by 87 · see citing works
Multi-cohort and longitudinal Bayesian clustering study of stage and subtype in Alzheimer’s disease
Poulakis, K., Pereira, J. B., Muehlboeck, J., Wahlund, L., Smedby, Ö., Volpe, G., Masters, C. L., Ames, D. J., Niimi, Y., Iwatsubo, T., et al. (2022) · Nature Communications · doi:10.1038/s41467-022-32202-6 · Cited by 84 · see citing works
Multi-scale semi-supervised clustering of brain images: Deriving disease subtypes
Wen, J., Varol, E., Sotiras, A., Yang, Z., Chand, G. B., Erus, G., Shou, H., Abdulkadir, A., Hwang, G., Dwyer, D., et al. (2021) · Medical Image Analysis · doi:10.1016/j.media.2021.102304 · Cited by 65 · see citing works
Self-citationTerada, S., Sato, S., Nagao, S., Ikeda, C., Shindo, A., Hayashi, S., Oshima, E., Yokota, O., & Uchitomi, Y. (2013) · Psychiatry Research Neuroimaging · doi:10.1016/j.pscychresns.2013.03.006 · Cited by 54 · see citing works
Quality improvement in neurology
Foster, N. L., Bondi, M. W., Das, R. R., Foss, M., Hershey, L. A., Koh, S., Logan, R., Poole, C., Shega, J. W., Sood, A., et al. (2019) · Neurology · doi:10.1212/wnl.0000000000008259 · Cited by 47 · see citing works
Odusami, M., Maskeliūnas, R., & Damaševičius, R. (2023) · Journal of Personalized Medicine · doi:10.3390/jpm13101496 · Cited by 42 · cited 1× · see citing works
“Deep learning models have effectively identified diagnostic groups [23] and sub-types [18] using multimodal imaging data and correlation-based approaches that allow a greater explainability of of the relationships between the features learned by the model.”
BackgroundNovel Alzheimer’s disease subtypes identified using a data and knowledge driven strategy
Mitelpunkt, A., Galili, T., Kozlovski, T., Bregman, N., Shachar, N., Marcus-Kalish, M., & Benjamini, Y. (2020) · Scientific Reports · doi:10.1038/s41598-020-57785-2 · Cited by 40 · see citing works
Zhang, S., Zhao, H., Wang, W., Wang, Z., Luo, X., Hramov, A. E., & Kurths, J. (2023) · Neurocomputing · doi:10.1016/j.neucom.2023.126512 · Cited by 35 · see citing works
Matías‐Guiu, J. A., Cabrera‐Martín, M. N., Valles‐Salgado, M., Pérez-Pérez, A., Rognoni, T., Moreno‐Ramos, T., Carreras, J. L., & Matías‐Guiu, J. (2017) · American Journal of Geriatric Psychiatry · doi:10.1016/j.jagp.2017.02.002 · Cited by 30 · see citing works
Hassen, S. B., Neji, M., Hussain, Z. U., Hussain, A., Alimi, A. M., & Frikha, M. (2024) · Neurocomputing · doi:10.1016/j.neucom.2024.127325 · Cited by 30 · see citing works
Association of CD2AP neuronal deposits with Braak neurofibrillary stage in Alzheimer’s disease
Camacho, J., Rábano, A., Marazuela, P., Bonaterra‐Pastra, A., Serna, G., Moliné, T., y Cajal, S. R., Martínez-Sáez, E. A., & Hernández‐Guillamón, M. (2021) · Brain Pathology · doi:10.1111/bpa.13016 · Cited by 23 · see citing works
Emon, M. A., Heinson, A., Wu, P., Domingo‐Fernándéz, D., Sood, M., Vrooman, H. A., Corvol, J., Scordis, P., Hofmann‐Apitius, M., & Fröhlich, H. (2020) · Scientific Reports · doi:10.1038/s41598-020-76200-4 · Cited by 21 · see citing works
Early Diagnosis of Alzheimer’s Disease: Moving Toward a Blood-Based Biomarkers Era
Tao, Q., Lin, R.-R., & Wu, Z. (2023) · Clinical Interventions in Aging · doi:10.2147/cia.s394821 · Cited by 18 · see citing works
Deep multiview learning to identify imaging-driven subtypes in mild cognitive impairment
Feng, Y., Kim, M., Yao, X., Liu, K., Long, Q., Shen, L., & Initiative, A. D. N. (2022) · BMC Bioinformatics · doi:10.1186/s12859-022-04946-x · Cited by 16 · see citing works
Self-citationMartí‐Juan, G., Sanromà, G., & Piella, G. (2019) · PLoS ONE · doi:10.1371/journal.pone.0211121 · Cited by 15 · see citing works
Genomics transformer for diagnosing Parkinson's disease
Reyes, D. M., Kim, M., Chao, H., et al. (2022) · IEEE-EMBS International Conference on Biomedical and Health Informatics (BHI ...) · doi:10.1109/bhi56158.2022.9926815 · Cited by 11 · see citing works
Self-citationConnectome transformer with anatomically inspired attention for Parkinson's diagnosis
Reyes, D. M., Kim, M., Chao, H., Shen, L., & Yan, P. (2022) · ACM International Conference on Bioinformatics, Computational Biology, and Health Informatics (ACM-BCB) · doi:10.1145/3535508.3545544 · Cited by 6 · see citing works
Self-citationProsser, A. M. J., Evenden, D., Holmes, R. B., et al. (2020) · Alzheimer s & Dementia · doi:10.1002/alz.045900 · Cited by 2 · see citing works
Flagged first, then by citations
Research works printed but not linked: 3
- [25]Su C. Hou Y. Xu J. Xu J. Brendel M. Maasch J.R.M.A. Bai Z. Zhang H. Zhu Y. Henchcliffe C. Parkinson’s Disease Progression, 2022. Pages: 2021.07.18.21260731 medRxiv 2022 2021.07.18.21260731 10.1101/2021.07.18.21260731 · DOI
- [34]Ba J.L. Kiros J.R. Hinton G.E. Layer Normalization arXiv 2016 1607.06450
- [35]Vaswani A. Shazeer N. Parmar N. Uszkoreit J. Jones L. Gomez A.N. Kaiser L. Polosukhin I. Attention is All you Need Proceedings of the Advances in Neural Information Processing Systems Curran Associates, Inc. Red Hook, NY, USA 2017 Volume 30
Cited by
Sorted by citations
Schouten, D., Nicoletti, G., Dille, B., Chia, C., Vendittelli, P., Schuurmans, M., Litjens, G., & Khalili, N. (2025) · Medical Image Analysis · doi:10.1016/j.media.2025.103621 · Cited by 108 · see citing works
Ren, F., Wei, J., Chen, Q., Hu, M., Yu, L., Mi, J., Zhou, X., Qin, D., Wu, J., & Wu, A. (2025) · Acta Pharmaceutica Sinica B · doi:10.1016/j.apsb.2025.07.030 · Cited by 97 · see citing works
- Authors
- Machado Reyes, D., Chao, H., Hahn, J., Shen, L., Yan, P., & Initiative, F. T. A. D. N.
- Affiliations
- Department of Biomedical Engineering, Center for Biotechnology and Interdisciplinary Studies, Rensselaer Polytechnic Institute, Troy, NY 12180, USA
- Rensselaer Polytechnic Institute (US)
- University of Pennsylvania (US)
- Corresponding author
- Yan, Pingkun
- Venue
- Journal of Personalized Medicine
- Publisher
- MDPI AG
- Published
- 2024-04-15
- Volume
- 14
- Issue
- 4
- Pages
- 421
- ISSN
- 2075-4426 (ISSN-L)
- PMC
- PMC11051083
- OpenAlex
- W4394807540
- Cited by
- 17
- References
- 47
- Citations by year
- 2026: 10 · 2025: 5 · 2024: 2
- FWCI
- 5.00
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- Locations
- Journal of Personalized Medicine (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- Funders
- training grant (T32AG078123, 2046708, 1837964, U01 AG066833, U01 AG068057, RF1 AG063481)
- NSF CAREER (T32AG078123, 2046708, 1837964, U01 AG066833, U01 AG068057, RF1 AG063481)
- NSF IIS (T32AG078123, 2046708, 1837964, U01 AG066833, U01 AG068057, RF1 AG063481)
- NIH (T32AG078123, 2046708, 1837964, U01 AG066833, U01 AG068057, RF1 AG063481)
- Topics
- Health Sciences › Medicine › Psychiatry and Mental health › Dementia and Cognitive Impairment Research
- Health Sciences › Medicine › Physiology › Alzheimer's disease research and treatments
- Health Sciences › Health Professions › Health Information Management › Artificial Intelligence in Healthcare
- Keywords
- Alzheimer's disease, disease subtypes, disease progression, multimodal classification, co-attention, prodromal Alzheimer's disease, neuroimaging, genetic data, clinical evaluation, explainable AI, ADNI
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
