Cited by 968Dementia and Cognitive Impairment ResearchAlzheimer's disease research and treatmentsFunctional Brain Connectivity Studies

Hierarchical Feature Representation and Multimodal Fusion with Deep Learning for AD/MCI Diagnosis

Open access
Suk, Heung-Il, Lee, Seong-Whan, Shen, Dinggang·2014·NeuroImage

Abstract

For the last decade, it has been shown that neuroimaging can be a potential tool for the diagnosis of Alzheimer's Disease (AD) and its prodromal stage, Mild Cognitive Impairment (MCI), and also fusion of different modalities can further provide the complementary information to enhance diagnostic accuracy. Here, we focus on the problems of both feature representation and fusion of multimodal information from Magnetic Resonance Imaging (MRI) and Positron Emission Tomography (PET). To our best knowledge, the previous methods in the literature mostly used hand-crafted features such as cortical thickness, gray matter densities from MRI, or voxel intensities from PET, and then combined these multimodal features by simply concatenating into a long vector or transforming into a higher-dimensional kernel space. In this paper, we propose a novel method for a high-level latent and shared feature representation from neuroimaging modalities via deep learning. Specifically, we use Deep Boltzmann Machine (DBM)(2), a deep network with a restricted Boltzmann machine as a building block, to find a latent hierarchical feature representation from a 3D patch, and then devise a systematic method for a joint feature representation from the paired patches of MRI and PET with a multimodal DBM. To validate the effectiveness of the proposed method, we performed experiments on ADNI dataset and compared with the state-of-the-art methods. In three binary classification problems of AD vs. healthy Normal Control (NC), MCI vs. NC, and MCI converter vs. MCI non-converter, we obtained the maximal accuracies of 95.35%, 85.67%, and 74.58%, respectively, outperforming the competing methods. By visual inspection of the trained model, we observed that the proposed method could hierarchically discover the complex latent patterns inherent in both MRI and PET.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1016/j.neuroimage.2014.06.077

Cites

90 of 90 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Flagged first, then by citations

2 of the 92 references couldn’t be matched to a known paper.

Cited by

Sorted by citations

Authors
Suk, H.-I., Lee, S.-W., & Shen, D.
Affiliations
  • University of North Carolina at Chapel Hill (US)
  • Korea University (KR)
Corresponding author
Shen, Dinggang
Venue
NeuroImage
Publisher
Elsevier BV
Published
2014-11
Volume
101
Pages
569-582
ISSN
1053-8119 (ISSN-L)
OpenAlex
W2126598020
Cited by
968
References
92
Citations by year
2026: 62 · 2025: 74 · 2024: 81 · 2023: 93 · 2022: 104 · 2021: 128 · 2020: 117 · 2019: 107 · 2018: 85 · 2017: 57 · 2016: 41 · 2015: 17 · 2014: 1
FWCI
29.50
Type
journal article
Language
en
Open access
Locations
  • NeuroImage (publishedVersion)
  • PubMed (publishedVersion)
  • Europe PMC (PubMed Central) (acceptedVersion)
  • PubMed Central (acceptedVersion)
  • Carolina Digital Repository (University of North Carolina at Chapel Hill) (submittedVersion)
  • UNC Libraries
Funders
  • NIH (EB006733, EB008374, EB009634, AG041721, MH100217, AG042599)
  • National Research Foundation (2012-005741)
  • Korean Government
Topics
  • Health Sciences › Medicine › Psychiatry and Mental health › Dementia and Cognitive Impairment Research
  • Health Sciences › Medicine › Physiology › Alzheimer's disease research and treatments
  • Life Sciences › Neuroscience › Cognitive Neuroscience › Functional Brain Connectivity Studies
Keywords
Alzheimer's disease, mild cognitive impairment, multimodal fusion, MRI-PET fusion, hierarchical feature representation, deep Boltzmann machine, neuroimaging diagnosis, MCI conversion, ADNI
MeSH terms
Aged, Aged, 80 and over, Alzheimer Disease, Artificial Intelligence, Female, Humans, Image Processing, Computer-Assisted, Magnetic Resonance Imaging, Male, Middle Aged, Neural Networks, Computer, Positron-Emission Tomography, Cognitive Dysfunction, Multimodal Imaging

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review