Cited by 4,796Medical Image Segmentation TechniquesAdvanced MRI Techniques and ApplicationsBrain Tumor Detection and Classification

A reproducible evaluation of ANTs similarity metric performance in brain image registration

Open access
Avants, Brian B., Tustison, Nicholas J., Song, Gang, Cook, Philip A., Klein, Arno, Gee, James C.·2011·NeuroImage

Abstract

The United States National Institutes of Health (NIH) commit significant support to open-source data and software resources in order to foment reproducibility in the biomedical imaging sciences. Here, we report and evaluate a recent product of this commitment: Advanced Neuroimaging Tools (ANTs), which is approaching its 2.0 release. The ANTs open source software library consists of a suite of state-of-the-art image registration, segmentation and template building tools for quantitative morphometric analysis. In this work, we use ANTs to quantify, for the first time, the impact of similarity metrics on the affine and deformable components of a template-based normalization study. We detail the ANTs implementation of three similarity metrics: squared intensity difference, a new and faster cross-correlation, and voxel-wise mutual information. We then use two-fold cross-validation to compare their performance on openly available, manually labeled, T1-weighted MRI brain image data of 40 subjects (UCLA's LPBA40 dataset). We report evaluation results on cortical and whole brain labels for both the affine and deformable components of the registration. Results indicate that the best ANTs methods are competitive with existing brain extraction results (Jaccard=0.958) and cortical labeling approaches. Mutual information affine mapping combined with cross-correlation diffeomorphic mapping gave the best cortical labeling results (Jaccard=0.669±0.022). Furthermore, our two-fold cross-validation allows us to quantify the similarity of templates derived from different subgroups. Our open code, data and evaluation scripts set performance benchmark parameters for this state-of-the-art toolkit. This is the first study to use a consistent transformation framework to provide a reproducible evaluation of the isolated effect of the similarity metric on optimal template construction and brain labeling.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1016/j.neuroimage.2010.09.025

Cites

6 of 6 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Local NCC performs competitively but has high variance, demonstrating the instability of the loss.

    “Local NCC [6] performs competitively but has high variance, demonstrating the instability of the loss.”

    From Intraoperative 2D/3D Image Registration via Differentiable X-Ray Rendering (2024) · cites this paper as [6]

    Results §4.2 reports competitive brain-extraction performance for registrations using CC and a relatively large standard deviation for the (CC, CC) pairing (0.937 ± 0.0210). But the (CC, MI) pairing has a much smaller standard deviation (0.958 ± 0.0054), and the paper attributes improved performance to MI affine initialization. It does not conclude that the local NCC loss itself is unstable; page numbers are unavailable in our copy. (LLM, conf=0.88)

    Experimental — accuracy not yet measured
  • DiffPose develops a sparse multiscale variant of local normalized cross correlation that is robust to local minima and faster to evaluate.

    “we develop a sparse multiscale variant of local normalized cross correlation [6, 7] that is robust to local minima and faster to evaluate.”

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Flagged first, then by citations

Research works printed but not linked: 69

  1. Acosta-Cabronero J Williams GB Pereira JMS Pengas G Nestor PJ The impact of skull-stripping and radio-frequency bias correction on grey-matter segmentation for voxel-based morphometry Neuroimage 2008 Feb 39 4 1654 1665 18065243
  2. Ardekani BA Guckemus S Bachman A Hoptman MJ Wojtaszek M Nierenberg J Quantitative comparison of algorithms for inter-subject registration of 3D volumetric brain MRI scans J Neurosci Methods 2005 Mar 142 1 67 76 15652618
  3. Arnold VI Ordinary Differential Equations 1991 Springer-Verlag
  4. Ashburner J A fast diffeomorphic image registration algorithm Neuroimage 2007 Oct 38 1 95 113 17761438
  5. Ashburner J Friston K Voxel-based morphometry—The methods Neuroimage 2000 11 805 821 10860804
  6. Avants B Anderson C Grossman M Gee JC Spatiotemporal normalization for longitudinal analysis of gray matter atrophy in frontotemporal dementia Med Image Comput Comput Assist Interv Int Conf Med Image Comput Comput Assist Interv 2007 10 Pt 2 303 310 18044582
  7. Avants BB Cook PA Ungar L Gee JC Grossman M Dementia induces correlated reductions in white matter integrity and cortical thickness: A multivariate neuroimaging study with sparse canonical correlation analysis Neuroimage 2010a Apr 50 3 1004 1016 20083207 PMC2953719
  8. Avants BB Epstein CL Grossman M Gee JC Symmetric diffeomorphic image registration with cross-correlation: Evaluating automated labeling of elderly and neurodegenerative brain Med Image Anal 2008 Feb 12 1 26 41 17659998 PMC2276735
  9. Avants BB Yushkevich P Pluta J Minkoff D Korczykowski M Detre J Gee JC The optimal template effect in hippocampus studies of diseased populations Neuroimage 2010b Feb 49 3 2457 2466 19818860 PMC2818274
  10. Brown LG A survey of image registration techniques ACM Computing Surveys 1992 December 24 4 325 376
  11. Christensen GE Rabbitt RD Miller MI Deformable templates using large deformation kinematics IEEE Transactions on Image Processing 1996 5 10 1435 1447 18290061
  12. Clausi DA Jernigan ME A fast method to determine cooccurrence texture features IEEE Trans. Geo. Rem. Sens 1998 36 1 298 300
  13. Crum WR Hill DLG Hawkes DJ Information theoretic similarity measures in non-rigid registration Inf Process Med Imaging 2003 Jul 18 378 387 15344473
  14. D’Agostino E Maes F Vandermeulen D Suetens P A viscous fluid model for multimodal non-rigid image registration using mutual information Med Image Anal 2003 Dec 7 4 565 575 14561559
  15. de Boer R Vrooman HA Ikram MA Vernooij MW Breteler MMB van der Lugt A Niessen WJ Accuracy and reproducibility study of automatic MRI brain tissue segmentation methods Neuroimage 2010 Jul 51 3 1047 1056 20226258
  16. Ebin DG Marsden J Groups of diffeomorphisms and the motion of an incompressible fluid Annals of Mathematics 1970 92 102 163
  17. Fischl B Dale AM Measuring the thickness of the human cerebral cortex from magnetic resonance images Proc Natl Acad Sci U S A 2000 Sep 97 20 11050 11055 10984517 PMC27146
  18. Gan R Chung ACS Multi-dimensional mutual information based robust image registration using maximum distance-gradient-magnitude Inf Process Med Imaging 2005 19 210 221 17354697
  19. Hammers A Allom R Koepp MJ Free SL Myers R Lemieux L Mitchell TN Brooks DJ Duncan JS Three-dimensional maximum probability atlas of the human brain, with particular reference to the temporal lobe Hum Brain Mapp 2003 Aug 19 4 224 247 12874777 PMC6871794
  20. Heckemann RA Hajnal JV Aljabar P Rueckert D Hammers A Automatic anatomical brain MRI segmentation combining label propagation and decision fusion Neuroimage 2006 Oct 33 1 115 126 16860573
  21. Heckemann RA Keihaninejad S Aljabar P Rueckert D Hajnal JV Hammers A Initiative ADN Improving intersubject image registration using tissue-class information benefits robustness and accuracy of multi-atlas based anatomical segmentation Neuroimage 2010 May 51 1 221 227 20114079
  22. Hellier P Barillot C Coupling dense and landmark-based approaches for nonrigid registration IEEE Trans Med Imaging 2003 Feb 22 2 217 227 12715998
  23. Hellier P Barillot C Corouge I Gibaud B Goualher GL Collins D Evans A Malandain G Ayache N Christensen G Johnson H Retrospective evaluation of intersubject brain registration IEEE Trans Med Imaging 2003 Sep 22 9 1120 1130 12956267
  24. Huang T Yang G Tang G A fast two-dimensional median filtering algorithm IEEE ASSP 1979 27 1 13 18
  25. Ibanez L Avila RS Aylward SR Open source and open science: how it is changing the medical imaging community Proc. of the International Symposium on Biomedical Imaging 2006
  26. Ibanez L Ng L Gee JC Aylward S Registration patterns: The generic framework for image registration of the Insight Toolkit IEEE International Symposium on Biomedical Imaging 2002 July 345 348
  27. Jia H Wu G Wang Q Shen D Absorb: Atlas building by self-organized registration and bundling Neuroimage 2010 Jul 51 3 1057 1070 20226255 PMC2862873
  28. Joshi S Davis B Jomier M Gerig G Unbiased diffeomorphic atlas construction for computational anatomy Neuroimage 2004 23 Suppl. 1 S151 S160 15501084
  29. Klein A Andersson J Ardekani BA Ashburner J Avants B Chiang M-C Christensen GE Collins LD Gee J Hellier P Song JH Jenkinson M Lepage C Rueckert D Thompson P Vercauteren T Woods RP Mann JJ Parsey RV Evaluation of 14 non-linear deformation algorithms applied to human brain MRI registration Neuroimage 2009 Jan PMC2747506 19195496
  30. Klein A Ghosh SS Avants B Yeo BTT Fischl B Ardekani B Gee JC Mann JJ Parsey RV Evaluation of volume-based and surface-based brain image registration methods Neuroimage 2010a May 51 1 214 220 20123029 PMC2862732
  31. Klein S Staring M Murphy K Viergever MA Pluim JPW Elastix: A toolbox for intensity-based medical image registration IEEE Trans Med Imaging 2010b Jan 29 1 196 205 19923044
  32. Kovacevic J From the editor-in-chief IEEE Transactions on Image Processing 2006 15 12
  33. Kriegeskorte N Simmons WK Bellgowan PSF Baker CI Circular analysis in systems neuroscience: the dangers of double dipping Nat Neurosci 2009 May 12 5 535 540 19396166 PMC2841687
  34. Learned-Miller EG Data driven image models through continuous joint alignment IEEE Trans Pattern Anal Mach Intell 2006 Feb 28 2 236 250 16468620
  35. Lim KO Pfefferbaum A Segmentation of MR brain images into cerebrospinal fluid spaces, white and gray matter J Comput Assist Tomogr 1989 13 4 588 593 2745775
  36. Loeckx D Slagmolen P Maes F Vandermeulen D Suetens P Nonrigid image registration using conditional mutual information IEEE Trans Med Imaging 2010 Jan 29 1 19 29 19447700
  37. Mattes D Haynor DR Vesselle H Lewellen TK Eubank W PET-CT image registration in the chest using free-form deformations IEEE Trans Med Imaging 2003 Jan 22 1 120 128 12703765
  38. Miller MI Trouve A Younes L On the metrics and euler-lagrange equations of computational anatomy Annu Rev Biomed Eng 2002 4 375 405 12117763
  39. Miller MI Trouvè A Younes L Geodesic shooting for computational anatomy J. Mathematical Imaging and VisionSubmitted 2005 PMC2897162 20613972
  40. Neu SC Valentino DJ Toga AW The loni debabeler: a mediator for neuroimaging software Neuroimage 2005 Feb 24 4 1170 1179 15670695
  41. Noblet V Heinrich C Heitz F Armspach J-P Retrospective evaluation of a topology preserving non-rigid registration method Med Image Anal 2006 Jun 10 3 366 384 16497537
  42. Park JG Lee C Skull stripping based on region growing for magnetic resonance brain images Neuroimage 2009 Oct 47 4 1394 1407 19389477
  43. Patel V Dinov ID Horn JDV Thompson PM Toga AW Loni mind: metadata in nifti for dwi Neuroimage 2010 Jun 51 2 665 676 20206274 PMC2952839
  44. Pluta J Avants BB Glynn S Awate S Gee JC Detre JA Appearance and incomplete label matching for diffeomorphic template based hippocampus segmentation Hippocampus 2009 Jun 19 6 565 571 19437413
  45. Rogelj P Kovacic S Symmetric image registration Med Image Anal 2006 Jun 10 3 484 493 15896998
  46. Rohlfing T Brandt R Menzel R Maurer CR Evaluation of atlas selection strategies for atlas-based image segmentation with application to confocal microscopy images of bee brains Neuroimage 2004 Apr 21 4 1428 1442 15050568
  47. Rueckert D Sonoda L Hayes C Hill D Leach M Hawkes D Nonrigid registration using free-form deformations: Application to breast MR images IEEE Trans Med Imaging 1999 Aug 18 8 712 721 10534053
  48. Sadananthan SA Zheng W Chee MWL Zagorodnov V Skull stripping using graph cuts Neuroimage 2010 Jan 49 1 225 239 19732839
  49. Schleicher A Morosan P Amunts K Zilles K Quantitative architectural analysis: A new approach to cortical mapping J Autism Dev Disord 2009 Jul 19582566
  50. Ségonne F Dale AM Busa E Glessner M Salat D Hahn HK Fischl B A hybrid approach to the skull stripping problem in MRI Neuroimage 2004 Jul 22 3 1060 1075 15219578
  51. Shattuck DW Mirza M Adisetiyo V Hojatkashani C Salamon G Narr KL Poldrack RA Bilder RM Toga AW Construction of a 3D probabilistic atlas of human cortical structures Neuroimage 2008 Feb 39 3 1064 1080 18037310 PMC2757616
  52. Shen D Davatzikos C Hammer: Hierarchical attribute matching mechanism for elastic registration IEEE Transactions on Medical Imaging 2002 21 11 1421 1439 12575879
  53. Song G Avants B Gee J Multi-start method with prior learning for image registration Proceedings of the Workshop on Mathematical Methods in Biomedical Image Analysis 2007
  54. Studholme C Hill DL Hawkes DJ Automated three-dimensional registration of magnetic resonance and positron emission tomography brain images by multiresolution optimization of voxel similarity measures Med Phys 1997 Jan 24 1 25 35 9029539
  55. Tao G He R Datta S Narayana PA Symmetric inverse consistent nonlinear registration driven by mutual information Comput Methods Programs Biomed 2009 Aug 95 2 105 115 19268386 PMC2744864
  56. Thirion JP Image matching as a diffusion process: an analogy with Maxwell’s demons Medical Image Analysis 1998 2 3 243 260 9873902
  57. Tustison NJ Avants BB Gee JC Directly manipulated free-form deformation image registration IEEE Trans Image Process 2009a Mar 18 3 624 635 19171516
  58. Tustison NJ Awate SP Song G Cook TS Gee JC A new information-theoretic measure to control the robustness-sensitivity trade-off for DMFFD point-set registration Inf Process Med Imaging 2009b 21 215 226 19694265
  59. Tustison NJ Gee JC N-D Ck B-spline scattered data approximation Insight Journal 2005 published online
  60. Vercauteren T Pennec X Perchant A Ayache N Diffeomorphic demons using itk’s finite difference solver hierarchy Insight Journal—2007 MICCAI Open Science Workshop 2007
  61. Vercauteren T Pennec X Perchant A Ayache N Diffeomorphic demons: Efficient non-parametric image registration Neuroimage 2009 45 1 Suppl S61 S72 19041946
  62. Viola P Wells WM Alignment by maximization of mutual information International Journal of Computer Vision 1997 24 2 137 154
  63. Wells WM Efficient synthesis of gaussian filters by cascaded uniform filters IEEE Trans. Pattern Anal. Machine Intell 1986 8 2 234 239 21869341
  64. Woods RP Grafton ST Watson JD Sicotte NL Mazziotta JC Automated image registration: Ii. intersubject validation of linear and nonlinear models J Comput Assist Tomogr 1998 22 1 153 165 9448780
  65. Yassa MA Stark SM Bakker A Albert MS Gallagher M Stark CEL High-resolution structural and functional MRI of hippocampal CA3 and dentate gyrus in patients with amnestic mild cognitive impairment Neuroimage 2010 Jul 51 3 1242 1252 20338246 PMC2909476
  66. Yoo TS Metaxas DN Open science-combining open data and open source software: Medical image analysis with the Insight Toolkit Med Image Anal 2005 Dec 9 6 503 506 16169766
  67. Yushkevich PA Avants BB Das SR Pluta J Altinay M Craige C Initiative ADN Bias in estimation of hippocampal atrophy using deformation-based morphometry arises from asymmetric global normalization: An illustration in ADNI 3T MRI data Neuroimage 2010 Apr 50 2 434 445 20005963 PMC2823935
  68. Yushkevich PA Avants BB Pluta J Das S Minkoff D Mechanic-Hamilton D Glynn S Pickup S Liu W Gee JC Grossman M Detre JA A high-resolution computational atlas of the human hippocampus from postmortem magnetic resonance imaging at 9.4 t Neuroimage 2009 Jan 44 2 385 398 18840532 PMC2650508
  69. Zhang J Liang L Anderson JR Gatewood L Rottenberg DA Strother SC Evaluation and comparison of glm- and cva-based fMRI processing pipelines with java-based fMRI processing pipeline evaluation system Neuroimage 2008 Jul 41 4 1242 1252 18482849 PMC4277234

Cited by

Sorted by citations

Authors
Avants, B. B., Tustison, N. J., Song, G., Cook, P. A., Klein, A., & Gee, J. C.
Affiliations
  • University of Pennsylvania (US)
  • New York State Psychiatric Institute (US)
Corresponding author
Avants, Brian B.
Venue
NeuroImage
Publisher
Elsevier BV
Published
2011-02
Volume
54
Issue
3
Pages
2033-2044
ISSN
1053-8119 (ISSN-L)
OpenAlex
W2136145485
Cited by
4,796
References
6
Citations by year
2026: 483 · 2025: 563 · 2024: 545 · 2023: 473 · 2022: 489 · 2021: 459 · 2020: 439 · 2019: 362 · 2018: 286 · 2017: 225 · 2016: 155 · 2015: 119 · 2014: 97 · 2013: 61 · 2012: 24
FWCI
36.02
Type
journal article
Language
en
Open access
Locations
  • NeuroImage (publishedVersion)
  • PubMed (publishedVersion)
  • PubMed Central (acceptedVersion)
Funders
National Institute of Biomedical Imaging and Bioengineering (R01EB006266-01)
Topics
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Medical Image Segmentation Techniques
  • Health Sciences › Medicine › Radiology, Nuclear Medicine and Imaging › Advanced MRI Techniques and Applications
  • Life Sciences › Neuroscience › Neurology › Brain Tumor Detection and Classification
Keywords
ants, brain image registration, similarity measures, mutual information, cross correlation function, diffeomorphic registration, affine registration, template fabrication, brain extraction, T1-weighted imaging, reproducibility, benchmarking
MeSH terms
Algorithms, Brain, Diagnostic Imaging, Head, Humans, Image Processing, Computer-Assisted, Models, Anatomic, Models, Neurological, Population, Software, Reproducibility of Results, Linear Models, Databases, Factual

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review