Cited by 72Dementia and Cognitive Impairment ResearchMedical Image Segmentation TechniquesFunctional Brain Connectivity Studies

Assessing atrophy measurement techniques in dementia: results from the MIRIAD atrophy challenge

Open access
Cash, David M., Frost, Chris, Iheme, Leonardo O., Ünay, Devrim, Kandemir, Melek, Fripp, Jurgen, Salvado, Olivier, Bourgeat, Pierrick +22 more·2015·NeuroImage

Abstract

Structural MRI is widely used for investigating brain atrophy in many neurodegenerative disorders, with several research groups developing and publishing techniques to provide quantitative assessments of this longitudinal change. Often techniques are compared through computation of required sample size estimates for future clinical trials. However interpretation of such comparisons is rendered complex because, despite using the same publicly available cohorts, the various techniques have been assessed with different data exclusions and different statistical analysis models. We created the MIRIAD atrophy challenge in order to test various capabilities of atrophy measurement techniques. The data consisted of 69 subjects (46 Alzheimer's disease, 23 control) who were scanned multiple (up to twelve) times at nine visits over a follow-up period of one to two years, resulting in 708 total image sets. Nine participating groups from 6 countries completed the challenge by providing volumetric measurements of key structures (whole brain, lateral ventricle, left and right hippocampi) for each dataset and atrophy measurements of these structures for each time point pair (both forward and backward) of a given subject. From these results, we formally compared techniques using exactly the same dataset. First, we assessed the repeatability of each technique using rates obtained from short intervals where no measurable atrophy is expected. For those measures that provided direct measures of atrophy between pairs of images, we also assessed symmetry and transitivity. Then, we performed a statistical analysis in a consistent manner using linear mixed effect models. The models, one for repeated measures of volume made at multiple time-points and a second for repeated "direct" measures of change in brain volume, appropriately allowed for the correlation between measures made on the same subject and were shown to fit the data well. From these models, we obtained estimates of the distribution of atrophy rates in the Alzheimer's disease (AD) and control groups and of required sample sizes to detect a 25% treatment effect, in relation to healthy ageing, with 95% significance and 80% power over follow-up periods of 6, 12, and 24months. Uncertainty in these estimates, and head-to-head comparisons between techniques, were carried out using the bootstrap. The lateral ventricles provided the most stable measurements, followed by the brain. The hippocampi had much more variability across participants, likely because of differences in segmentation protocol and less distinct boundaries. Most methods showed no indication of bias based on the short-term interval results, and direct measures provided good consistency in terms of symmetry and transitivity. The resulting annualized rates of change derived from the model ranged from, for whole brain: -1.4% to -2.2% (AD) and -0.35% to -0.67% (control), for ventricles: 4.6% to 10.2% (AD) and 1.2% to 3.4% (control), and for hippocampi: -1.5% to -7.0% (AD) and -0.4% to -1.4% (control). There were large and statistically significant differences in the sample size requirements between many of the techniques. The lowest sample sizes for each of these structures, for a trial with a 12month follow-up period, were 242 (95% CI: 154 to 422) for whole brain, 168 (95% CI: 112 to 282) for ventricles, 190 (95% CI: 146 to 268) for left hippocampi, and 158 (95% CI: 116 to 228) for right hippocampi. This analysis represents one of the most extensive statistical comparisons of a large number of different atrophy measurement techniques from around the globe. The challenge data will remain online and publicly available so that other groups can assess their methods.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1016/j.neuroimage.2015.07.087

Cites

66 of 66 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Flagged first, then by citations

2 of the 68 references couldn’t be matched to a known paper.

Research works printed but not linked: 2

  1. Fischl B. FreeSurfer Neuroimage 62 2012 774 781 22248573 PMC3685476
  2. Wang L. Yushkevich P. Ourselin S. MICCAI 2012 Workshop on Novel Imaging Biomarkers for Alzheimer's Disease and Related Disorders (NIBAD12) 2012

Cited by

Sorted by citations

Authors
Cash, D. M., Frost, C., Iheme, L. O., Ünay, D., Kandemir, M., Fripp, J., Salvado, O., Bourgeat, P., Reuter, M., Fischl, B., Lorenzi, M., Frisoni, G. B., Pennec, X., Pierson, R. K., Gunter, J. L., Senjem, M. L., Jack, C. R., Guizard, N., Fonov, V. S., Collins, D. L., Modat, M., Cardoso, M. J., Leung, K. K., Wang, H., Das, S. R., Yushkevich, P. A., Malone, I. B., Fox, N. C., Schott, J. M., & Ourselin, S.
Affiliations
  • National Hospital for Neurology and Neurosurgery (GB)
  • London School of Hygiene & Tropical Medicine (GB)
  • Bahçeşehir University (TR)
  • Commonwealth Scientific and Industrial Research Organisation (AU)
  • Harvard University (US)
  • Institut national de recherche en sciences et technologies du numérique (FR)
  • University of Geneva (CH)
  • University of Iowa (US)
  • Mayo Clinic (US)
  • Montreal Neurological Institute and Hospital (CA)
  • University College London (GB)
  • IBM Research - Almaden (US)
  • University of Pennsylvania (US)
Corresponding author
Cash, David M.
Venue
NeuroImage
Publisher
Elsevier BV
Published
2015-12
Volume
123
Pages
149-164
ISSN
1053-8119 (ISSN-L)
OpenAlex
W2127117802
Cited by
72
References
68
Citations by year
2026: 2 · 2025: 2 · 2024: 4 · 2023: 6 · 2022: 5 · 2021: 11 · 2020: 6 · 2019: 15 · 2018: 7 · 2017: 8 · 2016: 5
FWCI
5.42
Type
journal article
Language
en
Open access
Locations
  • NeuroImage (publishedVersion, cc-by)
  • PubMed Central (publishedVersion, cc-by)
  • PubMed (publishedVersion)
  • Archive ouverte UNIGE (University of Geneva) (publishedVersion)
  • LSHTM Research Online (London School of Hygiene and Tropical Medicine) (acceptedVersion)
  • HAL (submittedVersion, other-oa)
  • UCL Discovery (University College London) (submittedVersion)
  • Archive ouverte UNIGE (University of Geneva) (submittedVersion, cc-by)
Funders
  • NIHR UCL/UCLH Biomedical Research Centre
  • NIHR Queen Square Dementia Biomedical Research Unit
Topics
  • Health Sciences › Medicine › Psychiatry and Mental health › Dementia and Cognitive Impairment Research
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Medical Image Segmentation Techniques
  • Life Sciences › Neuroscience › Cognitive Neuroscience › Functional Brain Connectivity Studies
Keywords
brain atrophy, Alzheimer's disease, structural MRI, hippocampal atrophy, lateral ventricle, brain volume, sample size determination, linear mixed models, repeatability, healthy aging, healthy controls, segmentation variability, head-to-head comparison
MeSH terms
Aged, Alzheimer Disease, Atrophy, Brain, Data Interpretation, Statistical, Female, Hippocampus, Humans, Image Interpretation, Computer-Assisted, Magnetic Resonance Imaging, Male, Middle Aged, Reproducibility of Results

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review