The DOI matches its Crossref record; 52 of the 52 references it cites were found in a registry or an index.
Hippocampal volume change measurement: Quantitative assessment of the reproducibility of expert manual outlining and the automated methods FreeSurfer and FIRST
Abstract
Background To measure hippocampal volume change in Alzheimer's disease (AD) or mild cognitive impairment (MCI), expert manual delineation is often used because of its supposed accuracy. It has been suggested that expert outlining yields poorer reproducibility as compared to automated methods, but this has not been investigated. Aim To determine the reproducibilities of expert manual outlining and two common automated methods for measuring hippocampal atrophy rates in healthy aging, MCI and AD. Methods From the Alzheimer's Disease Neuroimaging Initiative (ADNI), 80 subjects were selected: 20 patients with AD, 40 patients with mild cognitive impairment (MCI) and 20 healthy controls (HCs). Left and right hippocampal volume change between baseline and month-12 visit was assessed by using expert manual delineation, and by the automated software packages FreeSurfer (longitudinal processing stream) and FIRST. To assess reproducibility of the measured hippocampal volume change, both back-to-back (BTB) MPRAGE scans available for each visit were analyzed. Hippocampal volume change was expressed in μL, and as a percentage of baseline volume. Reproducibility of the 1-year hippocampal volume change was estimated from the BTB measurements by using linear mixed model to calculate the limits of agreement (LoA) of each method, reflecting its measurement uncertainty. Using the delta method, approximate p-values were calculated for the pairwise comparisons between methods. Statistical analyses…
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“However, various evaluations have also revealed limitations of that approach (Mccarthy 2015; Keller et al., 2012; Mulder et al., 2014), which might be critical in aging and multi-site studies.”
From A comparison of Freesurfer and multi-atlas MUSE for brain anatomy segmentation: Findings about size and age bias, and inter-scanner stability in multi-site aging studies (2020) · cites this paper as Mulder et al., 2014
Citations
Cites
STATISTICAL METHODS FOR ASSESSING AGREEMENT BETWEEN TWO METHODS OF CLINICAL MEASUREMENT
Martin Bland, J., & Altman, D. (1986) · The Lancet · doi:10.1016/s0140-6736(86)90837-8 · Cited by 47,965
Beckett, A. H., Rowland, M., & TURNER, P. (1965) · The Lancet · doi:10.1016/s0140-6736(65)91037-8 · Cited by 16,917
Measuring agreement in method comparison studies
Bland, J. M., & Altman, D. G. (1999) · Statistical Methods in Medical Research · doi:10.1177/096228029900800204 · Cited by 9,619
Fischl, B., Salat, D. H., Busa, E., Albert, M., Dieterich, M., Haselgrove, C., van der Kouwe, A., Killiany, R., Kennedy, D., Klaveness, S., et al. (2002) · Neuron · doi:10.1016/s0896-6273(02)00569-x · Cited by 8,610
Zhang, Y., Brady, M., & Smith, S. M. (2001) · IEEE Transactions on Medical Imaging · doi:10.1109/42.906424 · Cited by 7,628
The Alzheimer’s disease neuroimaging initiative (ADNI): MRI methods
Jack, C. R., Bernstein, M. A., Fox, N. C., Thompson, P., Alexander, G., Harvey, D., Borowski, B., Britson, P. J., L. Whitwell, J., Ward, C., et al. (2008) · Journal of Magnetic Resonance Imaging · doi:10.1002/jmri.21049 · Cited by 4,347
Automatically Parcellating the Human Cerebral Cortex
Fischl, B. R. (2003) · Cerebral Cortex · doi:10.1093/cercor/bhg087 · Cited by 4,208
Within-subject template estimation for unbiased longitudinal image analysis
Reuter, M., Schmansky, N. J., Rosas, H. D., & Fischl, B. (2012) · NeuroImage · doi:10.1016/j.neuroimage.2012.02.084 · Cited by 2,356
A Bayesian model of shape and appearance for subcortical brain segmentation
Patenaude, B., Smith, S. M., Kennedy, D. N., & Jenkinson, M. (2011) · NeuroImage · doi:10.1016/j.neuroimage.2011.02.046 · Cited by 2,287
Warfield, S. K., Zou, K. H., & Wells, W. M. (2004) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2004.828354 · Cited by 2,072
Cerebrospinal fluid biomarker signature in Alzheimer's disease neuroimaging initiative subjects
Shaw, L. M., Vanderstichele, H., Knapik‐Czajka, M., Clark, C. M., Aisen, P. S., Petersen, R. C., Blennow, K., Soares, H., Simon, A., Lewczuk, P., et al. (2009) · Annals of Neurology · doi:10.1002/ana.21610 · Cited by 2,065
Agreement Between Methods of Measurement with Multiple Observations Per Individual
Bland, J. M., & Altman, D. G. (2007) · Journal of Biopharmaceutical Statistics · doi:10.1080/10543400701329422 · Cited by 1,864 · cited 1×
“…patient having two hippocampi, modeling of both fi xed and random effects variables, and handling of missing data, by which it provided superior estimates of each method's limits of agreement than those that could have been obtained from simpler analysis types (Bland and Altman, 1986, 1999, 2007).”
The Alzheimer's Disease Neuroimaging Initiative
Mueller, S. G., Weiner, M. W., Thal, L. J., Petersen, R. C., Jack, C., Jagust, W., Trojanowski, J. Q., Toga, A. W., & Beckett, L. (2005) · Neuroimaging Clinics of North America · doi:10.1016/j.nic.2005.09.008 · Cited by 1,525
Highly accurate inverse consistent registration: A robust approach
Reuter, M., Rosas, H. D., & Fischl, B. R. (2010) · NeuroImage · doi:10.1016/j.neuroimage.2010.07.020 · Cited by 1,443
The Alzheimer's Disease Neuroimaging Initiative: A review of papers published since its inception
Weiner, M. W., Veitch, D. P., Aisen, P. S. S., Beckett, L. A., Cairns, N. J., Green, R. C., Harvey, D. J., Jack, C. R., Jagust, W. J., Liu, E., et al. (2013) · Alzheimer s & Dementia · doi:10.1016/j.jalz.2013.05.1769 · Cited by 1,033
Rate of medial temporal lobe atrophy in typical aging and Alzheimer's disease
Jack, C. R., Petersen, R. C., Xu, Y., O'Brien, P. C., Smith, G. E., Ivnik, R. J., Tangalos, E. G., & Kokmen, E. (1998) · Neurology · doi:10.1212/wnl.51.4.993 · Cited by 666
Morey, R. A., Petty, C. M., Xu, Y., Pannu Hayes, J., Wagner, H. R., Lewis, D. V., LaBar, K. S., Styner, M., & McCarthy, G. (2009) · NeuroImage · doi:10.1016/j.neuroimage.2008.12.033 · Cited by 540
The Alzheimer's Disease Neuroimaging Initiative: A review of papers published since its inception
Weiner, M. W., Veitch, D. P., Aisen, P. S. S., Beckett, L. A., Cairns, N. J., Green, R. C., Harvey, D. J., Jack, C. R., Jagust, W. J., Liu, E., et al. (2011) · Alzheimer s & Dementia · doi:10.1016/j.jalz.2011.09.172 · Cited by 497
Gronenschild, E. H. B. M., Habets, P., Jacobs, H. I. L., Mengelers, R., Rozendaal, N., van Os, J., & Marcelis, M. C. (2012) · PLoS ONE · doi:10.1371/journal.pone.0038234 · Cited by 370
Who Invented the Delta Method?
Hoef, J. M. V. (2012) · The American Statistician · doi:10.1080/00031305.2012.687494 · Cited by 267
Wang, L., Swank, J. S., Glick, I. E., Gado, M. H., Miller, M. I., Morris, J. C., & Csernansky, J. G. (2003) · NeuroImage · doi:10.1016/s1053-8119(03)00361-6 · Cited by 263
Tae, W., Kim, S. S., Lee, K. U., Nam, E., & Kim, K. W. (2008) · Neuroradiology · doi:10.1007/s00234-008-0383-9 · Cited by 250
Leung, K. K., Barnes, J., Ridgway, G. R., Bartlett, J. W., Clarkson, M. J., Macdonald, K. E., Schuff, N. W., Fox, N. C., & Sebastien Ourselin, A. (2010) · NeuroImage · doi:10.1016/j.neuroimage.2010.03.018 · Cited by 244
Routine quantitative analysis of brain and cerebrospinal fluid spaces with MR imaging
Kikinis, R., Shenton, M. E., Gerig, G., Martin, J., Anderson, M., Metcalf, D., Guttmann, C. R. G., McCarley, R. W., Jolesz, F. A., Lorensen, W. E., et al. (1992) · Journal of Magnetic Resonance Imaging · doi:10.1002/jmri.1880020603 · Cited by 237
Comparison of automated and manual MRI volumetry of hippocampus in normal aging and dementia
Hsu, Y., Schuff, N. W., Du, A., Mark, K. E., Zhu, X., Hardin, D., & Weiner, M. W. (2002) · Journal of Magnetic Resonance Imaging · doi:10.1002/jmri.10163 · Cited by 217
Kennedy, K. M., Erickson, K. I., Rodrigue, K. M., Voss, M. W., Colcombe, S. J., Kramer, A. F., Acker, J. D., & Raz, N. (2008) · Neurobiology of Aging · doi:10.1016/j.neurobiolaging.2007.12.020 · Cited by 214
Hippocampus segmentation in MR images using atlas registration, voxel classification, and graph cuts
van der Lijn, F., den Heijer, T., Breteler, M. M. B., & Niessen, W. J. (2008) · NeuroImage · doi:10.1016/j.neuroimage.2008.07.058 · Cited by 209
A practical approach to Bland-Altman plots and variation coefficients for log transformed variables
Euser, A. M., Dekker, F. W., & le Cessie, S. (2008) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2007.11.003 · Cited by 171
Hippocampal volume assessment in temporal lobe epilepsy: How good is automated segmentation?
Pardoe, H., Pell, G. S., Abbott, D. F., & Jackson, G. D. (2009) · Epilepsia · doi:10.1111/j.1528-1167.2009.02243.x · Cited by 164
Boccardi, M., Ganzola, R., Bocchetta, M., Pievani, M., Redolfi, A., Bartzokis, G., Camicioli, R. M., Csernansky, J. G., de Leon, M. J., deToledo‐Morrell, L., et al. (2011) · Journal of Alzheimer s Disease · doi:10.3233/jad-2011-0004 · Cited by 144
Disease Tracking Markers for Alzheimer's Disease at the Prodromal (MCI) Stage
Drago, V., Babiloni, C., Bartrés‐Faz, D., Caroli, A., Bosch, B., Hensch, T., Didic, M., Klafki, H., Pievani, M., Jovicich, J., et al. (2011) · Journal of Alzheimer s Disease · doi:10.3233/jad-2011-0043 · Cited by 144
Westman, E., Simmons, A., Muehlboeck, J., Mecocci, P., Vellas, B. J., Tsolaki, M., Kłoszewska, I., Soininen, H. S., Weiner, M. W., Lovestone, S., et al. (2011) · NeuroImage · doi:10.1016/j.neuroimage.2011.06.065 · Cited by 143
Crum, W. R., Scahill, R. I., & Fox, N. C. (2001) · NeuroImage · doi:10.1006/nimg.2001.0744 · Cited by 137
Dewey, J. J., Hana, G., Russell, T., Price, J., McCaffrey, D., Harezlak, J., Sem, E., Anyanwu, J. C., Guttmann, C. R. G., NAVIA, B. A., et al. (2010) · NeuroImage · doi:10.1016/j.neuroimage.2010.03.033 · Cited by 126
Barnes, J., Foster, J., Boyes, R. G., Pepple, T., Moore, E. K., Schott, J. M., Frost, C. D., Scahill, R. I., & Fox, N. C. (2008) · NeuroImage · doi:10.1016/j.neuroimage.2008.01.012 · Cited by 115
Sánchez‐Benavides, G., Gómez‐Ansón, B., Sainz, A., Vives‐Gilabert, Y., Delfino, M., & Peña‐Casanova, J. (2010) · Psychiatry Research Neuroimaging · doi:10.1016/j.pscychresns.2009.10.011 · Cited by 109
Lehmann, M., Douiri, A., Kim, L. G., Marc Modat, A., Chan, D., Sebastien Ourselin, A., Barnes, J., & Fox, N. C. (2009) · NeuroImage · doi:10.1016/j.neuroimage.2009.10.056 · Cited by 106
Shen, D. G., & Davatzikos, C. A. (2004) · NeuroImage · doi:10.1016/j.neuroimage.2003.12.015 · Cited by 106
Differentiating AD from aging using semiautomated measurement of hippocampal atrophy rates
Barnes, J., Scahill, R. I., Boyes, R. G., Frost, C. D., Lewis, E. B., Rossor, C. L., Rossor, M. N., & Fox, N. C. (2004) · NeuroImage · doi:10.1016/j.neuroimage.2004.06.028 · Cited by 103
Reduced sample sizes for atrophy outcomes in Alzheimer's disease trials: baseline adjustment
Schott, J. M., Bartlett, J. W., Barnes, J., Leung, K. K., Sebastien Ourselin, A., & Fox, N. C. (2010) · Neurobiology of Aging · doi:10.1016/j.neurobiolaging.2010.04.011 · Cited by 82
Comparison of Manual and Automated Determination of Hippocampal Volumes in MCI and Early AD
Shen, L., Saykin, A. J., Kim, S., Firpi, H. A., West, J. D., Risacher, S. L., McDonald, B. C., McHugh, T., Wishart, H. A., & Flashman, L. A. (2010) · Brain Imaging and Behavior · doi:10.1007/s11682-010-9088-x · Cited by 80
Power Calculations for Clinical Trials in Alzheimer's Disease
Ard, M. C., & Edland, S. D. (2011) · Journal of Alzheimer s Disease · doi:10.3233/jad-2011-0062 · Cited by 74
Caramanos, Z., Fonov, V., Francis, S. J., Narayanan, S., Pike, G. B., Collins, D. L., & Arnold, D. L. (2009) · NeuroImage · doi:10.1016/j.neuroimage.2009.08.008 · Cited by 74
Doring, T. M., Kubo, T. T. A., da Cruz, L. C. H., Juruena, M. F., Fainberg, J., Domingues, R. C., & Gasparetto, E. L. (2011) · Journal of Magnetic Resonance Imaging · doi:10.1002/jmri.22473 · Cited by 72
Appearance-Based Segmentation of Medial Temporal Lobe Structures
Duchesne, S., Pruessner, J. C., & Collins, D. L. (2002) · NeuroImage · doi:10.1006/nimg.2002.1188 · Cited by 67
Automatic hippocampal segmentation in temporal lobe epilepsy: Impact of developmental abnormalities
Kim, H., Marie Chupin, A., Olivier Colliot, A., Bernhardt, B. C., Bernasconi, N., & Bernasconi, A. (2011) · NeuroImage · doi:10.1016/j.neuroimage.2011.11.040 · Cited by 59
Cover, K. S., van Schijndel, R. A., van Dijk, B. W., Redolfi, A., Knol, D. L., Frisoni, G. B., Barkhof, F., & Vrenken, H. (2011) · Psychiatry Research Neuroimaging · doi:10.1016/j.pscychresns.2011.02.012 · Cited by 50
Self-citationLucarelli, R. T., Peshock, R. M., McColl, R. W., Hulsey, K., Ayers, C. R., Whittemore, A. R., & King, K. S. (2012) · American Journal of Neuroradiology · doi:10.3174/ajnr.a3308 · Cited by 49
Asymmetric bias in user guided segmentations of brain structures
Maltbie, E. A., Bhatt, K., Paniagua, B., Smith, R. G., Graves, M. M., Mosconi, M. W., Peterson, S. M., White, S., Blocher, J., El-Sayed, M., et al. (2011) · NeuroImage · doi:10.1016/j.neuroimage.2011.08.025 · Cited by 48
Barnes, J., Boyes, R. G., Lewis, E. B., Schott, J. M., Frost, C. D., Scahill, R. I., & Fox, N. C. (2006) · Neurobiology of Aging · doi:10.1016/j.neurobiolaging.2006.07.008 · Cited by 47
van de Pol, L. A., Barnes, J., Scahill, R. I., Frost, C. D., Lewis, E. B., Boyes, R. G., van Schijndel, R. A., Scheltens, P., Fox, N. C., & Barkhof, F. (2006) · NeuroImage · doi:10.1016/j.neuroimage.2006.10.033 · Cited by 41
Self-citationTakao, H., Abe, O., Hayashi, N., Kabasawa, H., & Ohtomo, K. (2010) · Journal of Magnetic Resonance Imaging · doi:10.1002/jmri.22237 · Cited by 20
Cited by
Sorted by citations
Brain atrophy in Alzheimer’s Disease and aging
Pini, L., Pievani, M., Bocchetta, M., Altomare, D., Bosco, P., Cavedo, E., Galluzzi, S., Marizzoni, M., & Frisoni, G. B. (2016) · Ageing Research Reviews · doi:10.1016/j.arr.2016.01.002 · Cited by 836
Weiner, M. W., Veitch, D. P., Aisen, P. S. S., Beckett, L. A., Cairns, N. J., Green, R. C., Harvey, D. J., Jack, C. R., Jagust, W. J., Morris, J. C., et al. (2017) · Alzheimer s & Dementia · doi:10.1016/j.jalz.2016.11.007 · Cited by 325
Hippocampal volume across age: Nomograms derived from over 19,700 people in UK Biobank
Nobis, L., Manohar, S. G., Smith, S. M., Alfaro-Almagro, F., Jenkinson, M., Mackay, C. E., & Husain, M. (2019) · NeuroImage: Clinical · doi:10.1016/j.nicl.2019.101904 · Cited by 236
Yamanakkanavar, N., Choi, J. Y., & Lee, B. (2020) · Sensors · doi:10.3390/s20113243 · Cited by 235
A systematic review and meta-analysis of longitudinal hippocampal atrophy in healthy human ageing
Fraser, M. A., Shaw, M. E., & Cherbuin, N. (2015) · NeuroImage · doi:10.1016/j.neuroimage.2015.03.035 · Cited by 175
Metadata
- Authors
- Mulder, E. R., de Jong, R. A., Knol, D. L., van Schijndel, R. A., Cover, K. S., Visser, P. J., Barkhof, F., & Vrenken, H.
- Affiliations
- Amsterdam UMC Location Vrije Universiteit Amsterdam (NL)
- Gemeente Amsterdam (NL)
- Corresponding author
- Vrenken, Hugo
- ORCID
- Venue
- NeuroImage
- Publisher
- Elsevier BV
- Published
- 2014-05
- Volume
- 92
- Pages
- 169-181
- ISSN
- 1053-8119 (ISSN-L)
- OpenAlex
- W1981789870
- Cited by
- 122
- References
- 52
- Citations by year
- 2026: 4 · 2025: 2 · 2024: 4 · 2023: 8 · 2022: 9 · 2021: 8 · 2020: 14 · 2019: 12 · 2018: 11 · 2017: 18 · 2016: 16 · 2015: 14 · 2014: 2
- FWCI
- 12.73
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- NeuroImage (publishedVersion)
- PubMed (publishedVersion)
- Research Publications (Maastricht University) (publishedVersion)
- Amsterdam UMC (VU Amsterdam) - Institutional Repository (publishedVersion)
- UCL Discovery (University College London) (submittedVersion)
- Pure Amsterdam UMC (submittedVersion)
- VU Research Portal (submittedVersion)
- Research Publications (Maastricht University) (submittedVersion)
- Data Archiving and Networked Services (DANS) (submittedVersion)
- Funders
- Bristol-Myers Squibb
- Seventh Framework Programme (283562)
- Topics
- Health Sciences › Medicine › Psychiatry and Mental health › Dementia and Cognitive Impairment Research
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Medical Image Segmentation Techniques
- Health Sciences › Medicine › Radiology, Nuclear Medicine and Imaging › Retinal Imaging and Analysis
- Keywords
- hippocampal atrophy, reproducibility, FreeSurfer, FIRST, Alzheimer's disease, mild cognitive impairment, healthy aging, limits of agreement, longitudinal MRI, MPRAGE, segmentation errors, measurement uncertainty, ADNI cohort
- MeSH terms
- Aged, Algorithms, Alzheimer Disease, Artificial Intelligence, Atrophy, Female, Hippocampus, Humans, Image Enhancement, Image Interpretation, Computer-Assisted, Magnetic Resonance Imaging, Male, Organ Size, Pattern Recognition, Automated, Sensitivity and Specificity, Software, Software Validation, Reproducibility of Results, Observer Variation, Imaging, Three-Dimensional, Cognitive Dysfunction
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
