Large-scale evaluation of ANTs and FreeSurfer cortical thickness measurements
Abstract
Many studies of the human brain have explored the relationship between cortical thickness and cognition, phenotype, or disease. Due to the subjectivity and time requirements in manual measurement of cortical thickness, scientists have relied on robust software tools for automation which facilitate the testing and refinement of neuroscientific hypotheses. The most widely used tool for cortical thickness studies is the publicly available, surface-based FreeSurfer package. Critical to the adoption of such tools is a demonstration of their reproducibility, validity, and the documentation of specific implementations that are robust across large, diverse imaging datasets. To this end, we have developed the automated, volume-based Advanced Normalization Tools (ANTs) cortical thickness pipeline comprising well-vetted components such as SyGN (multivariate template construction), SyN (image registration), N4 (bias correction), Atropos (n-tissue segmentation), and DiReCT (cortical thickness estimation). In this work, we have conducted the largest evaluation of automated cortical thickness measures in publicly available data, comparing FreeSurfer and ANTs measures computed on 1205 images from four open data sets (IXI, MMRR, NKI, and OASIS), with parcellation based on the recently proposed Desikan-Killiany-Tourville (DKT) cortical labeling protocol. We found good scan-rescan repeatability with both FreeSurfer and ANTs measures. Given that such assessments of precision do not necessarily reflect accuracy or an ability to make statistical inferences, we further tested the neurobiological validity of these approaches by evaluating thickness-based prediction of age and gender. ANTs is shown to have a higher predictive performance than FreeSurfer for both of these measures. In promotion of open science, we make all of our scripts, data, and results publicly available which complements the use of open image data sets and the open source availability of the proposed ANTs cortical thickness pipeline.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
There is variability between cortical values obtained from Freesurfer with other diffeomorphic volume based algorithms such as ANTs.
“We also recognize that there is variability between cortical values obtained from Freesurfer with other diffeomorphic volume based algorithms such as ANTs (Tustison et al., 2014)”
From Quantifying cortical development in typically developing toddlers and young children, 1–6 years of age (2017) · cites this paper as Tustison et al., 2014
Cites
Breiman, L. (2001) · Machine Learning · doi:10.1023/a:1010933404324 · Cited by 133,538 · see citing works
Intraclass correlations: Uses in assessing rater reliability.
Shrout, P. E., & Fleiss, J. L. (1979) · Psychological Bulletin · doi:10.1037/0033-2909.86.2.420 · Cited by 23,433 · see citing works
Advances in functional and structural MR image analysis and implementation as FSL
Smith, S. M., Jenkinson, M., Woolrich, M. W., Beckmann, C. F., Behrens, T. E. J., Johansen-Berg, H., Bannister, P. R., De Luca, M., Drobnjak, I., Flitney, D. E., et al. (2004) · NeuroImage · doi:10.1016/j.neuroimage.2004.07.051 · Cited by 13,630 · see citing works
Desikan, R. S., Ségonne, F., Fischl, B., Quinn, B. T., Dickerson, B. C., Blacker, D., Buckner, R. L., Dale, A. M., Maguire, R. P., Hyman, B. T., et al. (2006) · NeuroImage · doi:10.1016/j.neuroimage.2006.01.021 · Cited by 13,324 · see citing works
Fischl, B., Salat, D. H., Busa, E., Albert, M., Dieterich, M., Haselgrove, C., van der Kouwe, A., Killiany, R., Kennedy, D., Klaveness, S., et al. (2002) · Neuron · doi:10.1016/s0896-6273(02)00569-x · Cited by 8,610 · see citing works
Automatic 3D intersubject registration of MR volumetric data in standardized Talairach space
Collins, D. L., Neelin, P., Peters, T. M., & Evans, A. C. (1994) · Journal of Computer Assisted Tomography · doi:10.1097/00004728-199403000-00005 · Cited by 3,431 · cited 1× · see citing works
“…a single individual (e.g., the Talairach atlas; Talairach and Tournoux, 1988) or a publicly available average of multiple individuals (e.g., the MNI template by Collins et al., 1994 or the ICBM template by Mazziotta et al., 1995), or an average template constructed from the individuals under study.”
BackgroundHan, X., Jovicich, J., Salat, D., van der Kouwe, A., Quinn, B., Czanner, S., Busa, E., Pacheco, J., Albert, M., Killiany, R., et al. (2006) · NeuroImage · doi:10.1016/j.neuroimage.2006.02.051 · Cited by 1,508 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
Harmonization of cortical thickness measurements across scanners and sites
Fortin, J.-P., Cullen, N., Sheline, Y. I., Taylor, W. D., Aselcioglu, I., Cook, P. A., Adams, P., Cooper, C., Fava, M., McGrath, P. J., et al. (2018) · NeuroImage · doi:10.1016/j.neuroimage.2017.11.024 · Cited by 1,580 · see citing works
Self-citationCiric, R., Wolf, D. H., Power, J. D., Roalf, D. R., Baum, G. L., Ruparel, K., Shinohara, R. T., Elliott, M. A., Eickhoff, S. B., Davatzikos, C., et al. (2017) · NeuroImage · doi:10.1016/j.neuroimage.2017.03.020 · Cited by 1,042 · see citing works
Deep learning and alignment of spatially resolved single-cell transcriptomes with Tangram
Biancalani, T., Scalia, G., Buffoni, L., Avasthi, R., Lu, Z., Sanger, A., Tokcan, N., Vanderburg, C. R., Segerstolpe, Å., Zhang, M., et al. (2021) · Nature Methods · doi:10.1038/s41592-021-01264-7 · Cited by 931 · see citing works
Best practices in data analysis and sharing in neuroimaging using MRI
Nichols, T. E., Das, S., Eickhoff, S. B., Evans, A. C., Glatard, T., Hanke, M., Kriegeskorte, N., Milham, M. P., Poldrack, R. A., Poline, J.-B., et al. (2017) · Nature Neuroscience · doi:10.1038/nn.4500 · Cited by 714 · see citing works
Self-citationZhao, B., Luo, T., Li, T., Li, Y., Zhang, J., Shan, Y., Wang, X., Yang, L., Zhou, F., Zhu, Z., et al. (2019) · Nature Genetics · doi:10.1038/s41588-019-0516-6 · Cited by 295 · see citing works
Gennatas, E. D., Avants, B. B., Wolf, D. H., Satterthwaite, T. D., Ruparel, K., Ciric, R., Hakonarson, H., Gur, R. E., & Gur, R. C. (2017) · The Journal of Neuroscience · doi:10.1523/jneurosci.3550-16.2017 · Cited by 302 · see citing works
Self-citationAcute social isolation evokes midbrain craving responses similar to hunger
Tomova, L., Wang, K. L., Thompson, T., Matthews, G. A., Takahashi, A., Tye, K. M., & Saxe, R. (2020) · Nature Neuroscience · doi:10.1038/s41593-020-00742-z · Cited by 283 · see citing works
Converging evidence for the neuroanatomic basis of combinatorial semantics in the angular gyrus
Price, A. R., Bonner, M. F., Peelle, J. E., & Grossman, M. (2015) · The Journal of Neuroscience · doi:10.1523/jneurosci.3446-14.2015 · Cited by 262 · see citing works
Remer, J., Croteau-Chonka, E., Dean, D. C., D’Arpino, S., Dirks, H., Whiley, D., & Deoni, S. C. L. (2017) · NeuroImage · doi:10.1016/j.neuroimage.2017.04.010 · Cited by 152 · see citing works
Shou, H., Yang, Z., Satterthwaite, T. D., Cook, P. A., Bruce, S. E., Shinohara, R. T., Rosenberg, B., & Sheline, Y. I. (2017) · NeuroImage: Clinical · doi:10.1016/j.nicl.2017.01.030 · Cited by 108 · see citing works
Self-citationEverything Matters: The ReproNim Perspective on Reproducible Neuroimaging
Kennedy, D. N., Abraham, S. A., Bates, J. F., Crowley, A., Ghosh, S., Gillespie, T., Goncalves, M., Grethe, J. S., Halchenko, Y. O., Hanke, M., et al. (2019) · Frontiers in Neuroinformatics · doi:10.3389/fninf.2019.00001 · Cited by 113 · see citing works
Suspected non-AD pathology in mild cognitive impairment
Wisse, L. E. M., Butala, N., Das, S. R., Davatzikos, C., Dickerson, B. C., Vaishnavi, S. N., Yushkevich, P. A., & Wolk, D. A. (2015) · Neurobiology of Aging · doi:10.1016/j.neurobiolaging.2015.08.029 · Cited by 78 · see citing works
Self-citationDiminished Cortical Thickness Is Associated with Impulsive Choice in Adolescence
Pehlivanova, M., Wolf, D. H., Sotiras, A., Kaczkurkin, A. N., Moore, T. M., Ciric, R., Cook, P. A., Garcia de La Garza, A., Rosen, A. F. G., Ruparel, K., et al. (2018) · The Journal of Neuroscience · doi:10.1523/jneurosci.2200-17.2018 · Cited by 79 · see citing works
Self-citationNeocortical origin and progression of gray matter atrophy in nonamnestic Alzheimer's disease
Phillips, J. S., Da Re, F., Dratch, L., Xie, S. X., Irwin, D. J., McMillan, C. T., Vaishnavi, S. N., Ferrarese, C., Lee, E. B., Shaw, L. M., et al. (2018) · Neurobiology of Aging · doi:10.1016/j.neurobiolaging.2017.11.008 · Cited by 74 · see citing works
Medaglia, J. D., Satterthwaite, T. D., Kelkar, A., Ciric, R., Moore, T. M., Ruparel, K., Gur, R. C., Gur, R. E., & Bassett, D. S. (2018) · NeuroImage · doi:10.1016/j.neuroimage.2017.10.048 · Cited by 64 · see citing works
Apathy in Frontotemporal Degeneration: Neuroanatomical Evidence of Impaired Goal-directed Behavior
Massimo, L., Powers, J. P., Evans, L. K., McMillan, C. T., Rascovsky, K., Eslinger, P., Ersek, M., Irwin, D. J., & Grossman, M. (2015) · Frontiers in Human Neuroscience · doi:10.3389/fnhum.2015.00611 · Cited by 69 · see citing works
Automatic measurement of prosody in behavioral variant FTD
Nevler, N., Ash, S., Jester, C., Irwin, D. J., Liberman, M., & Grossman, M. (2017) · Neurology · doi:10.1212/wnl.0000000000004236 · Cited by 61 · see citing works
Ash, S., Nevler, N., Phillips, J., Irwin, D. J., McMillan, C. T., Rascovsky, K., & Grossman, M. (2019) · Brain and Language · doi:10.1016/j.bandl.2019.04.006 · Cited by 47 · see citing works
Deep learning in spatial transcriptomics: Learning from the next next-generation sequencing
Heydari, A. A., & Sindi, S. S. (2023) · Biophysics Reviews · doi:10.1063/5.0091135 · Cited by 44 · see citing works
Automated analysis of lexical features in frontotemporal degeneration
Cho, S., Nevler, N., Ash, S., Shellikeri, S., Irwin, D. J., Massimo, L., Rascovsky, K., Olm, C., Grossman, M., & Liberman, M. (2021) · Cortex · doi:10.1016/j.cortex.2021.01.012 · Cited by 41 · see citing works
Callow, D. D., Won, J., Pena, G. S., Jordan, L. S., Arnold-Nedimala, N. A., Kommula, Y., Nielson, K. A., & Smith, J. C. (2021) · Frontiers in Aging Neuroscience · doi:10.3389/fnagi.2021.645258 · Cited by 27 · see citing works
Deep Learning Based Multimodal Progression Modeling for Alzheimer’s Disease
Yang, L., Wang, X., Guo, Q., Gladstein, S., Wooten, D., Li, T., Robieson, W. Z., Sun, Y., Huang, X., & Initiative, F. T. A. D. N. (2021) · Statistics in Biopharmaceutical Research · doi:10.1080/19466315.2021.1884129 · Cited by 28 · see citing works
MacDonald, M. E., Williams, R. J., Rajashekar, D., Stafford, R. B., Hanganu, A., Sun, H., Berman, A. J. L., McCreary, C. R., Frayne, R., Forkert, N. D., et al. (2020) · Neurobiology of Aging · doi:10.1016/j.neurobiolaging.2020.06.019 · Cited by 21 · see citing works
Automatic classification of AD pathology in FTD phenotypes using natural speech
Cho, S., Olm, C. A., Ash, S., Shellikeri, S., Agmon, G., Cousins, K. A. Q., Irwin, D. J., Grossman, M., Liberman, M., & Nevler, N. (2024) · Alzheimer's & Dementia · doi:10.1002/alz.13748 · Cited by 13 · see citing works
Hussein, R., Shin, D., Zhao, M. Y., Guo, J., Davidzon, G., Steinberg, G., Moseley, M., & Zaharchuk, G. (2024) · Medical Image Analysis · doi:10.1016/j.media.2023.103072 · Cited by 9 · see citing works
Massimo, L., Rennert, L., Xie, S. X., Olm, C., Bove, J., Van Deerlin, V., Irwin, D. J., Grossman, M., & McMillan, C. T. (2021) · Neurobiology of Aging · doi:10.1016/j.neurobiolaging.2021.07.018 · Cited by 6 · see citing works
- Authors
- Tustison, N. J., Cook, P. A., Klein, A., Song, G., Das, S. R., Duda, J. T., Kandel, B. M., van Strien, N., Stone, J. R., Gee, J. C., & Avants, B. B.
- Affiliations
- University of Virginia (US)
- University of Pennsylvania (US)
- Sage Bionetworks (US)
- Corresponding author
- Tustison, Nicholas J.
- ORCID
- Venue
- NeuroImage
- Publisher
- Elsevier BV
- Published
- 2014-10
- Volume
- 99
- Pages
- 166-179
- ISSN
- 1053-8119 (ISSN-L)
- OpenAlex
- W2066470779
- Cited by
- 670
- References
- 7
- Citations by year
- 2026: 36 · 2025: 55 · 2024: 78 · 2023: 75 · 2022: 71 · 2021: 79 · 2020: 74 · 2019: 56 · 2018: 46 · 2017: 48 · 2016: 29 · 2015: 19 · 2014: 4
- FWCI
- 29.81
- Type
- journal article
- Language
- en
- Open access
- closed
- Licence
- Locations
- NeuroImage (publishedVersion)
- PubMed (publishedVersion)
- Funders
- U.S. Army Medical Research Acquisition Activity (W81XWH-09-2-0055)
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Medical Image Segmentation Techniques
- Health Sciences › Medicine › Radiology, Nuclear Medicine and Imaging › Advanced Neuroimaging Techniques and Applications
- Life Sciences › Neuroscience › Cognitive Neuroscience › Functional Brain Connectivity Studies
- Keywords
- cortical thickness, FreeSurfer, age prediction, sex prediction, template fabrication, image registration, bias correction
- MeSH terms
- Adolescent, Adult, Aged, Aged, 80 and over, Aging, Algorithms, Cerebral Cortex, Child, Child, Preschool, Female, Humans, Image Processing, Computer-Assisted, Magnetic Resonance Imaging, Male, Middle Aged, Sex Characteristics, Software, Reproducibility of Results, Databases, Factual, Young Adult
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
