Cited by 43Advanced MRI Techniques and ApplicationsAdvanced Neuroimaging Techniques and ApplicationsReliability and Agreement in Measurement

Test-retest reliability and sample size estimates after MRI scanner relocation

Open access
Melzer, Tracy R., Keenan, Ross J., Leeper, Gareth J., Kingston-Smith, Stephen, Felton, Simon A., Green, Sarah K., Henderson, Kerry J., Palmer, Nickolas J. +3 more·2020·NeuroImage

Abstract

OBJECTIVE: Many factors can contribute to the reliability and robustness of MRI-derived metrics. In this study, we assessed the reliability and reproducibility of three MRI modalities after an MRI scanner was relocated to a new hospital facility. METHODS: Twenty healthy volunteers (12 females, mean age (standard deviation) ​= ​41 (11) years, age range [25-66]) completed three MRI sessions. The first session (S1) was one week prior to the 3T GE HDxt scanner relocation. The second (S2) occurred nine weeks after S1 and at the new location; a third session (S3) was acquired 4 weeks after S2. At each session, we acquired structural T1-weighted, pseudo-continuous arterial spin labelled, and diffusion tensor imaging sequences. We used longitudinal processing streams to create 12 summary MRI metrics, including total gray matter (GM), cortical GM, subcortical GM, white matter (WM), and lateral ventricle volume; mean cortical thickness; total surface area; average gray matter perfusion, and average diffusion tensor metrics along principal white matter pathways. We compared mean MRI values and variance at the old scanner location to multiple sessions at the new location using Bayesian multi-level regression models. K-fold cross validation allowed identification of important predictors. Whole-brain analyses were used to investigate any regional differences. Furthermore, we calculated within-subject coefficient of variation (wsCV), intraclass correlation coefficient (ICC), and dice similarity index (SI) of cortical segmentations across scanner relocation and within-site. Additionally, we estimated sample sizes required to robustly detect a 4% difference between two groups across MRI metrics. RESULTS: All global MRI metrics exhibited little mean difference and small variability (bar cortical gray matter perfusion) both across scanner relocation and within-site repeat. T1- and DTI-derived tissue metrics showed ​< ​|0.3|% mean difference and <1.2% variance across scanner location and <|0.4|% mean difference and <0.8% variance within the new location, with between-site intraclass correlation coefficient (ICC) ​> ​0.80 and within-subject coefficient of variation (wsCV) ​< ​1.4%. Mean cortical gray matter perfusion had the highest between-session variability (6.7% [0.3, 16.7], estimate [95% uncertainty interval]), and hence the smallest ICC (0.71 [0.44,0.92]) and largest wsCV (13.4% [5.4, 18.1]). No global metric exhibited evidence of a meaningful mean difference between scanner locations. However, surface area showed evidence of a mean difference within-site repeat (between S2 and S3). Whole-brain analyses revealed no significant areas of difference between scanner relocation or within-site. For all metrics, we found no support for a systematic difference in variance across relocation sites compared to within-site test-retest reliability. Necessary sample sizes to detect a 4% difference between two independent groups varied from a maximum of n ​= ​362 per group (cortical gray matter perfusion), to total gray matter volume (n ​= ​114), average fractional anisotropy (n ​= ​23), total gray matter volume normalized by intracranial volume (n ​= ​19), and axial diffusivity (n ​= ​3 per group). CONCLUSION: Cortical gray matter perfusion was the most variable metric investigated (necessitating large sample sizes to identify group differences), with other metrics showing substantially less variability. Scanner relocation appeared to have a negligible effect on variability of the global MRI metrics tested. This manuscript reports within-site test-retest variability to act as a tool for calculating sample size in future investigations. Our results suggest that when all other parameters are held constant (e.g., sequence parameters and MRI processing), the effect of scanner relocation is indistinguishable from within-site variability, but may need to be considered depending on the question being investigated.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1016/j.neuroimage.2020.116608

Cites

60 of 60 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Previous studies with this dataset have pointed out that scanner variability is negligible.

    “Although two different scanners were used (Siemens model Trio; scanner 1 = 285 subjects, 95.6 %; and scanner 2 = 13 subjects, 4.4 %), previous studies with this dataset have pointed out that this variability is negligible (Dickerson et al., 2008; Fox et al., 2012; Melzer et al., 2020; Noble et al., 2017a; Noble et al., 2017b; Zhou et al., 2018).”

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Flagged first, then by citations

Cited by

Sorted by citations

Authors
Melzer, T. R., Keenan, R. J., Leeper, G. J., Kingston-Smith, S., Felton, S. A., Green, S. K., Henderson, K. J., Palmer, N. J., Shoorangiz, R., Almuqbel, M. M., & Myall, D. J.
Affiliations
  • New Zealand Brain Research Institute (NZ)
  • Christchurch Hospital (NZ)
  • Pacific Radiology (New Zealand) (NZ)
  • University of Canterbury (NZ)
Corresponding author
Melzer, Tracy R.
Venue
NeuroImage
Publisher
Elsevier BV
Published
2020-05
Volume
211
Pages
116608
ISSN
1053-8119 (ISSN-L)
OpenAlex
W3004839452
Cited by
43
References
59
Citations by year
2026: 7 · 2025: 3 · 2024: 5 · 2023: 7 · 2022: 10 · 2021: 11
FWCI
3.42
Type
journal article
Language
en
Open access
Locations
  • NeuroImage (publishedVersion, cc-by)
  • PubMed (publishedVersion)
  • DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
Funders
Health Research Council of New Zealand
Topics
  • Health Sciences › Medicine › Radiology, Nuclear Medicine and Imaging › Advanced MRI Techniques and Applications
  • Health Sciences › Medicine › Radiology, Nuclear Medicine and Imaging › Advanced Neuroimaging Techniques and Applications
  • Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Reliability and Agreement in Measurement
Keywords
test-retest reliability, reproducibility, sample size determination, arterial spin labeling, diffusion tensor imaging, T1-weighted imaging, cortical thickness, lateral ventricle volume, white matter, cortical segmentation, healthy volunteers, k-fold cross-validation, Dice similarity index, intraclass correlation coefficient
MeSH terms
Adult, Aged, Cerebral Cortex, Female, Humans, Magnetic Resonance Imaging, Male, Middle Aged, Reproducibility of Results, Sample Size, Magnetic Resonance Angiography, Diffusion Tensor Imaging, Neuroimaging, White Matter, Gray Matter

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review