Long-term cognitive decline prediction based on multi-modal data using Multimodal3DSiameseNet: transfer learning from Alzheimer’s disease to Parkinson’s disease
Open accessAbstract
Purpose Monitoring and predicting the cognitive state of subjects with neurodegenerative disorders is crucial to provide appropriate treatment as soon as possible. In this work, we present a machine learning approach using multimodal data (brain MRI and clinical) from two early medical visits, to predict the longer-term cognitive decline of patients. Using transfer learning, our model can be successfully transferred from one neurodegenerative disease (Alzheimer's) to another (Parkinson's).Methods Our model is a Deep Neural Network with siamese sub-modules dedicated to extracting features from each modality. We pre-train it with data from ADNI (Alzheimer's disease), then transfer it on the smaller PPMI dataset (Parkinson's disease). We show that, even when we do not fine-tune the filters learnt from the ADNI MRIs, the transferred model's results are satisfying on PPMI.Results The first main result is that our model provides satisfying long-term predictions of cognitive decline from any pair of early visits, with no fixed time delay between these visits (provided the potential decline has started at the second visit). The second main result is that the prediction performance on Parkinson's dataset (PPMI) reaches an AUC of 0.81 on PPMI after transfer learning from Alzheimer's dataset (ADNI), without even having to re-train the image filters, versus an AUC of 0.72 for the model trained from scratch on PPMI.Conclusions First, our model is effective for predicting long-term cognitive decline from only two visits, even with irregular intervals of time. When dealing with neurodegenerative diseases, where patients often miss some control visits, this is an important finding. Second, our model is able to transfer the knowledge learnt from one neurodegenerative disease (Alzheimer's) to another (Parkinson's), when using the same imaging modalities (brain MRI) and different clinical variables. This makes it usable even for diseases that are rare or under-studied.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Santos, A. G., da Rocha, G. O., & de Andrade, J. B. (2019) · Scientific Reports · doi:10.1038/s41598-018-37186-2 · Cited by 8,495 · see citing works
The Alzheimer’s disease neuroimaging initiative (ADNI): MRI methods
Jack, C. R., Bernstein, M. A., Fox, N. C., Thompson, P., Alexander, G., Harvey, D., Borowski, B., Britson, P. J., L. Whitwell, J., Ward, C., et al. (2008) · Journal of Magnetic Resonance Imaging · doi:10.1002/jmri.21049 · Cited by 4,347 · see citing works
Striving for Simplicity: The All Convolutional Net
Springenberg, J. T., Dosovitskiy, A., Brox, T., & Riedmiller, M. (2014) · arXiv · doi:10.48550/arxiv.1412.6806 · Cited by 3,447 · see citing works
SIGNATURE VERIFICATION USING A “SIAMESE” TIME DELAY NEURAL NETWORK
BROMLEY, J., BENTZ, J. W., BOTTOU, L., GUYON, I., LECUN, Y., MOORE, C., SÄCKINGER, E., & SHAH, R. (1994) · Series in Machine Perception and Artificial Intelligence · doi:10.1142/9789812797926_0003 · Cited by 2,350 · see citing works
The Parkinson Progression Marker Initiative (PPMI)
MAREK, K. L., Jennings, D. L., Lasch, S., Siderowf, A. D., Tanner, C. M., Simuni, T., Coffey, C. S., Kieburtz, K. D., Flagg, E., Chowdhury, S., et al. (2011) · Progress in Neurobiology · doi:10.1016/j.pneurobio.2011.09.005 · Cited by 1,888 · see citing works
Deep multimodal learning: a survey on recent advances and trends
Ramachandram, D., & Taylor, G. W. (2017) · IEEE Signal Processing Magazine · doi:10.1109/msp.2017.2738401 · Cited by 1,139 · cited 1× · see citing works
“Among the possible data fusion strategies [2], Xu et al.”
Mapping Scores Onto Stages: Mini-Mental State Examination and Clinical Dementia Rating
Perneczky, R., Wagenpfeil, S., Komossa, K., Grimmer, T., Diehl, J., & Kurz, A. (2006) · The American Journal of Geriatric Psychiatry · doi:10.1097/01.jgp.0000192478.82189.a8 · Cited by 768 · see citing works
Residual and plain convolutional neural networks for 3D brain MRI classification
Korolev, S., Safiullin, A., Belyaev, M., & Dodonova, Y. (2017) · 2017 IEEE 14th International Symposium on Biomedical Imaging (ISBI 2017) · doi:10.1109/isbi.2017.7950647 · Cited by 435 · see citing works
Predicting alzheimer’s disease progression using multi-modal deep learning approach
Lee, G., Nho, K., Kang, B., Sohn, K.-A., Kim, D., Initiative, F. A. D. N., Weiner, M. W., Aisen, P., Petersen, R., Jack, C. R., et al. (2019) · Scientific Reports · doi:10.1038/s41598-018-37769-z · Cited by 406 · see citing works
Bhagwat, N., Viviano, J. D., Voineskos, A. N., Chakravarty, M. M., & Initiative, A. D. N. (2018) · PLOS Computational Biology · doi:10.1371/journal.pcbi.1006376 · Cited by 157 · see citing works
Distant Domain Transfer Learning for Medical Imaging
Niu, S., Liu, M., Liu, Y., Wang, J., & Song, H. H. (2021) · IEEE Journal of Biomedical and Health Informatics · doi:10.1109/jbhi.2021.3051470 · Cited by 106 · see citing works
Suganthe, R. C., Muthusamy, G., Sevalaiappan, L. R., & Senthilkumaran, N. (2022) · Acta Polytechnica Hungarica · doi:10.12700/aph.19.5.2022.5.7 · Cited by 48 · see citing works
Differential diagnosis of parkinsonism based on deep metabolic imaging indices
Wu, P., Zhao, Y. S., Wu, J., Brendel, M., Lu, J., Ge, J., Bernhardt, A. M., Li, L., Alberts, I., Katzdobler, S., et al. (2022) · Journal of Nuclear Medicine · doi:10.2967/jnumed.121.263029 · Cited by 46 · see citing works
Shibata, H., Uchida, Y., Inui, S., Kan, H., Sakurai, K., Oishi, N., Ueki, Y., Oishi, K., & Matsukawa, N. (2021) · Parkinsonism & Related Disorders · doi:10.1016/j.parkreldis.2021.12.004 · Cited by 42 · see citing works
Deep Learning Approach to Classify Parkinson’s Disease from MRI Samples
Basnin, N., Nahar, N., Anika, F. A., et al. (2021) · Lecture notes in computer science · doi:10.1007/978-3-030-86993-9_48 · Cited by 36 · see citing works
Ya, Y., Ji, L., Jia, Y., et al. (2022) · Frontiers in Aging Neuroscience · doi:10.3389/fnagi.2022.808520 · Cited by 30 · see citing works
Wu, Y., Jia, M., Xiang, C., Lin, S., Jiang, Z., & Fang, Y. (2022) · Psychiatry Research · doi:10.1016/j.psychres.2022.114434 · Cited by 19 · see citing works
Evans, S., McRae‐McKee, K., Hadjichrysanthou, C., Wong, M. M., Ames, D. J., López, O. L., de Wolf, F., Anderson, R. M., Initiative, A. D. N., ageing, A. I. B. A. L. F. S. O., et al. (2019) · Alzheimer s & Dementia Translational Research & Clinical Interventions · doi:10.1016/j.trci.2019.04.005 · Cited by 9 · see citing works
Predicting Brain Degeneration with a Multimodal Siamese Neural Network
Ostertag, C., Beurton‐Aimar, M., Visani, M., Urruty, T., & Bertet, K. (2020) · International Conference on Image Processing Theory, Tools and Applications (IPTA) · doi:10.1109/ipta50016.2020.9286657 · Cited by 8 · see citing works
Self-citationHomemade TS-Net for Automatic Face Recognition
Lin, S., Zhao, Z., & Su, F. (2016) · ACM International Conference on Multimedia Retrieval (ICMR) · doi:10.1145/2911996.2911999 · Cited by 7 · see citing works
Flagged first, then by citations
Research works printed but not linked: 1
- [7]Donahue J, Jia Y, Vinyals O, Hoffman J, Zhang N, Tzeng E, Darrell T (2014) Decaf: a deep convolutional activation feature for generic visual recognition. In: International conference on machine learning, pp 647–655
Cited by
Sorted by citations
Schouten, D., Nicoletti, G., Dille, B., Chia, C., Vendittelli, P., Schuurmans, M., Litjens, G., & Khalili, N. (2025) · Medical Image Analysis · doi:10.1016/j.media.2025.103621 · Cited by 108 · see citing works
A systematic review of intermediate fusion in multimodal deep learning for biomedical applications
Guarrasi, V., Aksu, F., Caruso, C. M., Di Feola, F., Rofena, A., Ruffini, F., & Soda, P. (2025) · Image and Vision Computing · doi:10.1016/j.imavis.2025.105509 · Cited by 95 · see citing works
Screen, Sample, Stratify: Biomarkers and Machine Learning Compress Dementia Pathways
Battaglia, S., & Tanaka, M. (2026) · Biomedicines · doi:10.3390/biomedicines14010159 · Cited by 2 · see citing works
- Authors
- Ostertag, C., Visani, M., Urruty, T., & Beurton-Aimar, M.
- Affiliations
- Centre National de la Recherche Scientifique (FR)
- Trường Đại học Khoa học và Công nghệ Hà Nội (VN)
- Corresponding author
- Visani, Muriel
- Venue
- International Journal of Computer Assisted Radiology and Surgery
- Published
- 2023-03-25
- Volume
- 18
- Issue
- 5
- Pages
- 809-818
- ISSN
- 1861-6410 (ISSN-L)
- 1861-6429
- PMC
- PMC10038771
- OpenAlex
- W4360820229
- Cited by
- 26
- References
- 21
- Citations by year
- 2026: 6 · 2025: 8 · 2024: 9 · 2023: 3
- FWCI
- 5.29
- Type
- journal article
- Language
- en
- Open access
- bronze
- doi.org
- Licence
- Locations
- International Journal of Computer Assisted Radiology and Surgery (publishedVersion)
- PubMed Central (publishedVersion)
- PubMed (publishedVersion)
- HAL (submittedVersion)
- Funders
- Conseil Régional Aquitaine (RCB 2018-A02699-46)
- Topics
- Health Sciences › Medicine › Psychiatry and Mental health › Dementia and Cognitive Impairment Research
- Health Sciences › Health Professions › Health Information Management › Artificial Intelligence in Healthcare
- Physical Sciences › Computer Science › Artificial Intelligence › Machine Learning in Healthcare
- Keywords
- cognitive decline prediction, multimodal fusion, brain MRI, Siamese network, transfer learning, Alzheimer's disease, Parkinson's disease, neurodegenerative diseases, clinical data, ADNI, PPMI
- MeSH terms
- Machine Learning, Alzheimer Disease, Humans, Magnetic Resonance Imaging, Parkinson Disease, Disease Progression, Neurodegenerative Diseases, Cognitive Dysfunction
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
