Cited by 70Dementia and Cognitive Impairment ResearchArtificial Intelligence in HealthcareExplainable Artificial Intelligence (XAI)

Data analysis with Shapley values for automatic subject selection in Alzheimer’s disease data sets using interpretable machine learning

Open access
Bloch, Louise, Friedrich, Christoph M., for the Alzheimer’s Disease Neuroimaging Initiative·2021·Alzheimer's Research & Therapy

Published version of a preprint

Abstract

Background: For the recruitment and monitoring of subjects for therapy studies, it is important to predict whether mild cognitive impaired (MCI) subjects will prospectively develop Alzheimer’s disease (AD). Machine learning (ML) is suitable to improve early AD prediction. The etiology of AD is heterogeneous, which leads to high variability in disease patterns. Further variability originates from multicentric study designs, varying acquisition protocols, and errors in the preprocessing of magnetic resonance imaging (MRI) scans. The high variability makes the differentiation between signal and noise difficult and may lead to overfitting. This article examines whether an automatic and fair data valuation method based on Shapley values can identify the most informative subjects to improve ML classification. Methods: An ML workflow was developed and trained for a subset of the Alzheimer’s Disease Neuroimaging Initiative (ADNI) cohort. The validation was executed for an independent ADNI test set and for the Australian Imaging, Biomarker and Lifestyle Flagship Study of Ageing (AIBL) cohort. The workflow included volumetric MRI feature extraction, feature selection, sample selection using Data Shapley, random forest (RF), and eXtreme Gradient Boosting (XGBoost) for model training as well as Kernel SHapley Additive exPlanations (SHAP) values for model interpretation. Results: The RF models, which excluded 134 of the 467 training subjects based on their RF Data Shapley values, outperformed the base models that reached a mean accuracy of 62.64% by 5.76% (3.61 percentage points) for the independent ADNI test set. The XGBoost base models reached a mean accuracy of 60.00% for the AIBL data set. The exclusion of those 133 subjects with the smallest RF Data Shapley values could improve the classification accuracy by 2.98% (1.79 percentage points). The cutoff values were calculated using an independent validation set. Conclusion: The Data Shapley method was able to improve the mean accuracies for the test sets. The most informative subjects were associated with the number of ApolipoproteinE ε 4 (ApoE ε 4) alleles, cognitive test results, and volumetric MRI measurements.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1186/s13195-021-00879-4

Cites

94 of 94 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • SHAP can equitably compare and rank the influence of diverse inputs-such as MRI-derived volumetric features, cognitive test scores, sociodemographic variables, and genetic markers like ApoE alleles-on a model's classification outcome.

    “Its strength is particularly evident in multimodal diagnostic contexts, where it can equitably compare and rank the influence of diverse inputs—such as MRI-derived volumetric features, cognitive test scores, sociodemographic variables, and genetic markers like ApoE alleles—on a model’s classification outcome”

    From Multimodal neuroimaging and AI integration in cognitive disorders: advances, challenges, and future directions for precision medicine (2026) · cites this paper as Bloch et al., 2021

    The Results (Training of XGBoost classifiers; page unavailable) describe SHAP values that rank features by mean absolute value, including cognitive scores and MRI volumes. The Discussion also identifies ApoE ε4 alleles among important features. The Data sets section reports demographic variables, but the retrieved passages do not establish that SHAP compared their influence on classification outcomes; nor does the paper explicitly establish that the comparison is equitable. (LLM, conf=0.82)

    Experimental — accuracy not yet measured

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Flagged first, then by citations

2 of the 96 references couldn’t be matched to a known paper.

Research works printed but not linked: 2

  1. [17]D’Urso C EXPERIENCE: glitches in databases, how to ensure data quality by outlier detection techniques ACM J Data Inf Qual 2016 7 3 14
  2. [21]Goodfellow I, Pouget-Abadie J, Mirza M, Xu B, Warde-Farley D, Ozair S, Courville A, Bengio Y. Generative adversarial nets. In: Advances in Neural Information Processing Systems (NIPS 2014): 8-13 December 2014; Montreal, Canada: 2014. p. 2672–80. https://papers.nips.cc/paper/2014/file/5ca3e9b122f61f8f06494c97b1afccf3-Paper.pdf. Accessed 05 Feb 2021.

Cited by

Sorted by citations

Authors
Bloch, L., Friedrich, C. M., & Initiative, F. T. A. D. N.
Affiliations
Dortmund University of Applied Sciences and Arts (DE)
Corresponding author
Friedrich, Christoph M.
Venue
Alzheimer's Research & Therapy
Published
2021-09-15
Volume
13
Issue
1
Pages
155-155
ISSN
1758-9193 (ISSN-L)
OpenAlex
W3133501805
Cited by
68
References
96
Citations by year
2026: 9 · 2025: 15 · 2024: 19 · 2023: 11 · 2022: 14
FWCI
8.07
Type
journal article
Language
en
Open access
Locations
  • Alzheimer s Research & Therapy (publishedVersion, cc-by)
  • Research Square (acceptedVersion, cc-by)
  • Europe PMC (PubMed Central) (publishedVersion, other-oa)
  • PubMed Central (publishedVersion, cc-by)
  • PubMed (publishedVersion)
  • DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
  • Informationszentrum Lebenswissenschaften (ZB MED) (submittedVersion, cc-by)
Funders
  • U.S. Department of Defense (U01 AG024904, W81XWH, W81XWH-12-2-, W81XWH-12, W81XWH1220012, AG024904)
  • Foundation for the National Institutes of Health (W81XWH-12-2-0012, U01AG024904, Grant U01 AG024904, AG024904)
  • Alzheimer's Association (U01 AG024904, W81XWH-12-2-0012)
  • Bristol-Myers Squibb (U01 AG024904, AG024904, W81XWH-12-2-0012)
  • Eli Lilly and Company (U01 AG024904, AG024904, W81XWH-12-2-0012)
  • Pfizer (AG024904, W81XWH-12-2-0012, U01 AG024904)
  • Biogen (W81XWH-12-2-0012, U01 AG024904, AG024904)
  • University of Southern California (W81XWH-12-2-0012, U01 AG024904)
  • F. Hoffmann-La Roche (W81XWH-12-2-0012, AG024904, U01 AG024904)
  • Meso Scale Diagnostics (U01 AG024904, AG024904, W81XWH-12-2-0012)
  • Alzheimer's Disease Neuroimaging Initiative (W81XWH-12-2–0012, U01AG024904, AG024904, W81XWH-12-2-, Department of Defense award number W81XWH-12-2-0012)
  • BioClinica (U01 AG024904, W81XWH-12-2-0012, AG024904)
  • Novartis Pharmaceuticals Corporation (W81XWH-12-2-0012, U01 AG024904, AG024904)
  • Northern California Institute for Research and Education (U01 AG024904, W81XWH-12-2-0012)
  • Eisai (W81XWH-12-2-0012, U01 AG024904, AG024904)
  • Servier (AG024904, W81XWH-12-2-0012, U01 AG024904)
  • H. Lundbeck A/S (U01 AG024904, AG024904, W81XWH-12-2-0012)
  • IXICO (AG024904, W81XWH-12-2-0012, U01 AG024904)
  • National Institutes of Health (3u01ag024904-10s2, W81XWH, W81XWH-12-2-0012, U01-AG024904, Grant U01 AG024904, AG024904)
  • Genentech (W81XWH-12-2-0012, AG024904, U01 AG024904)
  • Canadian Institutes of Health Research (U01 AG024904, W81XWH-12-2-0012)
  • National Institute on Aging (U01-AG-024904)
  • National Institute of Biomedical Imaging and Bioengineering (W81XWH-12-2-0012, U01 AG024904, AG024904)
Topics
  • Health Sciences › Medicine › Psychiatry and Mental health › Dementia and Cognitive Impairment Research
  • Health Sciences › Health Professions › Health Information Management › Artificial Intelligence in Healthcare
  • Physical Sciences › Computer Science › Artificial Intelligence › Explainable Artificial Intelligence (XAI)
Keywords
Shapley value, sample selection, Alzheimer's disease, mild cognitive impairment, early disease prediction, model interpretability, APOE ε4, random forest, XGBoost, ADNI, AIBL
MeSH terms
Machine Learning, Data Analysis, Alzheimer Disease, Australia, Humans, Magnetic Resonance Imaging, Cognitive Dysfunction

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review