CoChat
Sign Up
Cited by 654Medical Image Segmentation TechniquesDomain Adaptation and Few-Shot LearningRetinal Imaging and Analysis

Quicksilver: Fast predictive image registration – A deep learning approach

Open access
Yang, Xiao, Kwitt, Roland, Styner, Martin, Niethammer, Marc·2017·NeuroImage
DOI 10.1016/j.neuroimage.2017.07.008PMC PMC6036629
Open this paper in CoChat — free account

Abstract

This paper introduces Quicksilver, a fast deformable image registration method. Quicksilver registration for image-pairs works by patch-wise prediction of a deformation model based directly on image appearance. A deep encoder-decoder network is used as the prediction model. While the prediction strategy is general, we focus on predictions for the Large Deformation Diffeomorphic Metric Mapping (LDDMM) model. Specifically, we predict the momentum-parameterization of LDDMM, which facilitates a patch-wise prediction strategy while maintaining the theoretical properties of LDDMM, such as guaranteed diffeomorphic mappings for sufficiently strong regularization. We also provide a probabilistic version of our prediction network which can be sampled during the testing time to calculate uncertainties in the predicted deformations. Finally, we introduce a new correction network which greatly increases the prediction accuracy of an already existing prediction network. We show experimental results for uni-modal atlas-to-image as well as uni-/multi-modal image-to-image registrations. These experiments demonstrate that our method accurately predicts registrations obtained by numerical optimization, is very fast, achieves state-of-the-art registration results on four standard validation datasets, and can jointly learn an image similarity measure. Quicksilver is freely available as an open-source software.

AI snapshot

Written from the paper’s full text.

TL;DR

Quicksilver is a deep-learning registration method that predicts the initial momentum of LDDMM shooting patch-by-patch from moving/target image patches, with optional Bayesian uncertainty and a correction network. It achieves registration accuracy close to numerical LDDMM optimization on several brain MRI datasets while running much faster.

Key findings

  • Predicting the initial momentum gave better atlas-to-image accuracy than predicting initial velocity or displacement; momentum and velocity networks produced positive Jacobian determinants in all tested cases, whereas direct displacement prediction did not guarantee diffeomorphic maps.
  • On the four Klein et al. validation datasets, the prediction+correction model (LPC) approached numerical LDDMM optimization (LO): paired TOST indicated LPC was statistically equivalent to LO within a 2% relative threshold on all four datasets, and label overlap was top-tier with few outliers.
  • Runtime for a 229×193×193 volume: LDDMM optimization took about 10.8 minutes, while Quicksilver prediction took 7.63 s plus 8.9 s for shooting, totaling 18.43 s, a roughly 35× speedup; adding the correction network doubled time but remained an order of magnitude faster than optimization. Patch pruning and stride 14 reduced the number of predicted patches by 99.995% for such volumes.
  • Multimodal T1w–T2w prediction reduced deformation error relative to affine registration and was only slightly worse than the T1w–T1w upper bound; training with only 10 images (90 registration pairs) still worked with a small performance drop. Bayesian sampling produced uncertainty maps with high values near ventricles and large deformations.

Methods

  • The method uses the LDDMM shooting formulation and trains a deep encoder-decoder network to predict vector-valued initial momentum from paired 15×15×15 patches of moving and target images. Whole-image prediction uses a sliding window with stride 14 and patch pruning, followed by LDDMM shooting to generate deformations.
  • Data: atlas-to-image used 150 OASIS images (100 training, 50 testing); image-to-image used 373 OASIS images for training and tested on LPBA40, IBSR18, MGH10, and CUMC12, totaling 2,168 test registrations; multimodal used the IBIS 3D dataset with 375 T1w/T2w images (359 training, 16 testing, 250 test cases). A correction network was trained sequentially on the residual between ground-truth and predicted momentum; a probabilistic version used dropout (probability 0.2) and Monte Carlo sampling.
  • Evaluation included voxel-wise deformation error percentiles, Jacobian determinant positivity, LDDMM energy, target overlap of labels, paired t-tests, paired TOST equivalence tests, and runtime/multi-GPU scaling.

Limitations

  • The paper states that the 15×15×15 patch receptive field limits prediction of very large deformations; OASIS training data had only 0.2% of voxels with deformations above 10 mm, so large deformations were underrepresented. Outliers with inconsistent anatomy can cause extreme optimization deformations that prediction may not match, though the conservative prediction can also avoid implausible warps.
  • The correction network doubles runtime, and applying correction multiple times can produce more drastic deformations and numerical instability; 12 LPP cases in LPBA40 failed LDDMM shooting due to integration problems. Patch pruning may not transfer to parametric models if parameter locality is not guaranteed, and multi-GPU scaling is limited by communication overhead and PyTorch’s beta-stage performance.
  • Evident from the text: validation is restricted to brain MRI with a specific LDDMM regularizer and SSD similarity, so generalization to other organs, modalities, or registration models is not demonstrated.

Model-written summary. Check it against the paper before relying on it.

Integrity signals

Registry

Crossref · 10.1016/j.neuroimage.2017.07.008

Cites

77 of 77 references verified0 citation errors · 0 not checkable

Statistics

Nothing to recompute.

Claims the paper makes

0 of 43 cited sources checked · 0 supported · 0 flagged

  • Findingnone given

    The correction network substantially improves the prediction accuracy of an existing prediction network.

    “Finally, we introduce a new correction network which greatly increases the prediction accuracy of an already existing prediction network.”
    Quote in the paperSubstantial claim
    • The authors state this as an assumption: A set of LDDMM parameters obtained by numerical optimization already exists and yields good registration results.
  • Findingown data

    The method predicts accurate deformations for multi-modal registration.

    “Our experiments show that our approach also predicts accurate deformations for multi-modal registration.”
    Quote in the paperSubstantial claim
    • The authors state this as an assumption: A set of LDDMM parameters obtained by numerical optimization already exists and yields good registration results.
  • Findingown data

    Quicksilver achieves state-of-the-art registration accuracy on four standard validation datasets.

    “achieves state-of-the-art registration results on four standard validation datasets”
    • Also stated in: § 1. Introduction · boosted
      “demonstrate registration accuracies on these datasets on par with the state-of-the-art”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [33] Kingma, 2014

      Not checked in the free run.

    • The authors state this as an assumption: A set of LDDMM parameters obtained by numerical optimization already exists and yields good registration results.
  • Findingown data

    Quicksilver predicts registrations that closely match those obtained by numerical optimization.

    “These experiments demonstrate that our method accurately predicts registrations obtained by numerical optimization”
    Quote in the paperSubstantial claim
    • The authors state this as an assumption: A set of LDDMM parameters obtained by numerical optimization already exists and yields good registration results.
  • Findingown data

    With the correction network, predicted registrations reach a level of accuracy similar to optimization-based registrations in validation experiments.

    “Experiments show that the correction network improves registration results to the point where optimization-based and predicted registrations achieve a similar level of registration accuracy on registration validation experiments.”
    Quote in the paperSubstantial claim
    • The authors state this as an assumption: A set of LDDMM parameters obtained by numerical optimization already exists and yields good registration results.
  • May combine two claims. 46
  • Classified as probably not a substantial claim. 22

  • May combine two claims.

    Findingown data

    Using patch pruning and a large sliding-window stride yields large speedups over optimization while keeping good registration accuracy.

    “We experimentally show that by using patch pruning and a large sliding window stride, our method achieves dramatic speedups compared to the optimization approach, while maintaining good registration accuracy.”
    Quote in the paperSubstantial claim
    • The authors state this as an assumption: A set of LDDMM parameters obtained by numerical optimization already exists and yields good registration results.

  • May combine two claims.

    Findingown data

    The image-to-image training set contains 1,002,404 15×15×15 patches, about 3.4 billion voxels, which is much larger than the network parameter count.

    “the total number of 15 × 15 × 15 3D training patches to train the prediction network is 1,002,404. This amounts to approximately 3.4 billion voxels and is much larger than the total number of parameters in the network.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Findingnone given

    The proposed network contains 975,360 three-dimensional filters and 21,826,344 parameters in total.

    “The number of 3D filters used in the network is 975,360. The overall number of parameters 21,826,344.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingreasoning

    Compared with the accuracy boost from LP to LPC, LPC2 and LPC3 did not greatly improve registration accuracy and may even generate worse results, such as LPC3 on LPBA40.

    “Compared to the label overlapping accuracy boost from LP to LPC, LPC2 and LPC3 do not greatly improve the registration accuracy, and may even generate worse results (e.g., LPC3 on LPBA40).”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    Compared with the initial LDDMM energy based on affine registration, both LP and LPC have drastically lower LDDMM energy values, and these values are only slightly higher than those for LO.

    “Compared with the initial LDDMM energy based on affine registration to the atlas space in the initial column, both LP and LPC have drastically lower LDDMM energy values; further, these values are only slightly higher than those for LO.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    LPP, LPC2, and LPC3 generated more drastic deformations than LO, with more spread-out histograms indicating stronger expansions and contractions.

    “LPP, LPC2 and LPC3 generate more drastic deformations (i.e., more spread out histograms indicating areas of stronger expansions and contractions).”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data

    LPP, LPC2, and LPC3 produce more drastic deformations that can lead to worse label overlap and numerical stability problems.

    “In fact, they produce more drastic deformations that can lead to worse label overlap and even numerical stability problems.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingreasoning

    In that failure case, the prediction result still generated reasonable deformations while matching brain structures as much as possible.

    “Our prediction result, however, still generates reasonable deformations (where plausibility is based on the deformations that were observed during training) while matching the brain structures as much as possible.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    Using the correction network together with the prediction network doubled computation time, but overall runtime remained an order of magnitude faster than direct LDDMM optimization.

    “Using the correction network together with the prediction network doubles the computation time, but the overall runtime is still an order of magnitude faster than direct LDDMM optimization.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    At stride 1, computational cost increased about 3000-fold in 3D, resulting in runtimes of about 5.5 hours for 3D image registration, or 11 hours when the correction network was also used.

    “Note that, at a stride of 1, computational cost increases about 3000-fold in 3D, resulting in runtimes of about 5½ hours for 3D image registration (eleven hours when the correction network is also used).”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    Using the prediction network with stride 14, initial momentum prediction took on average 7.63 seconds, and subsequent geodesic shooting took 8.9 seconds, for a total runtime of 18.43 seconds.

    “Using our prediction network with a sliding window stride of 14, the initial momentum prediction time is, on average, 7.63 seconds. Subsequent geodesic shooting to generate the deformation field takes 8.9 seconds, resulting in a total runtime of 18.43 seconds.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    At a 2% relative threshold difference, LPC was statistically equivalent to LO on all four datasets and to many other top methods, such as LPC versus SyN on MGH10 and IBSR18.

    “For a relative threshold difference of 2% LPC can be considered statistically equivalent to LO for all four datasets and to many of the other top methods (e.g., LPC vs. SyN on MGH10 and IBSR18).”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [71] Zhang, 2015

      Not checked in the free run.

  • May combine two claims.

    Findingown data

    Among the prediction models, LPC improved prediction accuracy compared with LP and generally achieved the highest deformation prediction accuracy for up to 80% of voxels.

    “Among the prediction models, LPC improves the prediction accuracy compared with LP, and generally achieves the highest deformation prediction accuracy for up to 80% of the voxels.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    The multi-modal networks (T1w-T2w, LP/LPC) significantly reduced deformation error compared with affine registration and suffered only a slight loss in accuracy compared with their T1w-T1w counterparts.

    “The multi-modal networks (T1w-T2w, LP/LPC) significantly reduce deformation error compared to affine registration, and only suffer a slight loss in accuracy compared to their T1w-T1w counterparts.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    In the OASIS training images, deformations had a median of 2.609 mm, similar to the median in the four testing datasets, but only 0.2% of voxels had deformations larger than 10 mm.

    “Second, the deformations in the OASIS training images have a median of 2.609 mm, which is similar to the median observed in the four testing datasets. However, only 0.2% of the voxels in the OASIS training dataset have deformations larger than 10 mm.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    LPC showed a slight performance drop relative to LO, and LP, which uses only the prediction model without correction, showed a slightly larger drop.

    “A slight performance drop with respect to LO can be observed for LPC and a slightly bigger performance drop for LP, which only uses the prediction model, but no correction model.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingreasoning

    Compared with LP, LPC generates LDDMM energy values closer to LO, indicating that prediction+correction yields momenta closer to the optimal solution than prediction alone.

    “Furthermore, compared with LP, LPC generates LDDMM energy values that are closer to LO, which indicates that using the prediction+correction approach results in momenta which are closer to the optimal solution than the ones obtained by using the prediction network only.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    LPC was on par with other prediction models for up to 99% of voxels, where the largest deformations were between 7.317 mm and 9.026 mm across the four datasets.

    “It is also on-par with other prediction models for up to 99% of the voxels, where the largest deformations are in the range between 7.317 mm-9.026 mm for the four datasets.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data

    Compared with other methods, the prediction+correction network achieves top-tier label matching accuracy at a small fraction of the computational cost.

    “Compared with other methods, our prediction+correction network achieves top-tier performance for label matching accuracy at a small fraction of the computational cost.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    Using only the prediction network results in a slight performance drop compared to numerical optimization, but the result is still competitive with top-performing registration methods.

    “Second, using only the prediction network results in a slight performance drop compared to the numerical optimization results (LO), but the result is still competitive with the top-performing registration methods.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    The largest deformations generated by LDDMM optimization were 23.393 mm for LPBA40, 36.263 mm for IBSR18, 18.753 mm for CUMC12, and 18.727 mm for MGH10.

    “As shown in the table, the largest deformations that LDDMM optimization generates are 23.393 mm for LPBA40, 36.263 mm for IBSR18, 18.753 mm for CUMC12 and 18.727 mm for MGH10.”
    Quote in the paperNot a substantial claim

  • Classified as probably not a substantial claim. May combine two claims.

    Premisecitation

    The LDDMM shooting formulation derives from Riemannian geometry and defines a distance measure between images.

    “It is based on Riemannian geometry, and hence induces a distance metric on the space of images.”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [17] Gal, 2016

      Not checked in the free run.

  • May combine two claims.

    Premisecitation

    LDDMM shooting uses the initial momentum as its registration parameter; this parameter need not be spatially smooth, can be predicted patch by patch, and determines the full geodesic path.

    “It uses the initial momentum as the registration parameter, which does not need to be spatially smooth, and hence can be predicted patch-by-patch, and from which the whole geodesic path can be computed.”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [17] Gal, 2016

      Not checked in the free run.

  • May combine two claims.

    Premisenone given

    The initial momentum is generally not smooth and is compactly supported at image edges.

    “the initial momentum is generally not smooth, but is compactly supported at image edges”
    • Also stated in: § 1. Introduction · hedged
      “the momentum is typically compactly supported around image edges”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Interpretationreasoning

    The initial velocity is obtained by smoothing the initial momentum with kernel K, so deformation smoothness is imposed after prediction rather than during it.

    “the initial velocity is generated by applying a smoothing kernel K to the initial momentum. Therefore, the smoothness of the deformation does not need to be specifically considered during the parameter prediction step, but is imposed after the prediction.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Interpretationreasoning

    Predicting the momentum preserves LDDMM's mathematical properties and allows fast prediction of diffeomorphic transformations.

    “Hence, by predicting the momentum, we retain all the convenient mathematical properties of LDDMM and, at the same time, are able to predict diffeomorphic transformations fast.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Interpretationcitation

    The momentum parameterization of LDDMM shooting is a suitable representation for regression because the momentum is usually concentrated near image edges and does not itself need to be smooth.

    “We demonstrate that the momentum-parameterization for LDDMM shooting [17] is a convenient representation for regression approaches as (i) the momentum is typically compactly supported around image edges and (ii) there are no smoothness requirements on the momentum itself.”
    • Also stated in: § Abstract · plain
      “Specifically, we predict the momentum-parameterization of LDDMM, which facilitates a patch-wise prediction strategy while maintaining the theoretical properties of LDDMM, such as guaranteed diffeomorphic mappings for sufficiently strong regularization.”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [17] Gal, 2016

      Not checked in the free run.

  • Classified as probably not a substantial claim.

    Premisecitation

    Common approaches to multi-modal registration use image synthesis or complex hand-crafted or learned similarity measures.

    “General approaches address multimodal image registration by either performing image synthesis [22, 23] to change the problem to a uni-modal image registration task, or by proposing complex, hand-crafted [24, 25, 26, 27] or learned [28, 29, 30, 31, 32] multi-modal image similarity measures.”
    Quote in the paperSubstantial claim0 of 10 sources checked
    • cites [22] Gutierrez-Becker, 2017, Guiding multimodal registration with learned optimization updates, MedIA

      Not checked in the free run.

    • cites [23] Gutiérrez-Becker, 2016

      Not checked in the free run.

    • cites [24] Haber, 2007, Image registration with guaranteed displacement regularity, Int. J. Comput. Vis. source

      Not checked in the free run.

    • cites [25] Hart, 2009, An optimal control approach for deformable registration

      Not checked in the free run.

    • cites [26] Hazlett, 2017, Early brain development in infants at high risk for autism spectrum disorder, Nature source

      Not checked in the free run.

    • cites [27] He, 2015

      Not checked in the free run.

    • cites [28] Hermosillo, 2002, Variational methods for multimodal image matching, IJCV source

      Not checked in the free run.

    • cites [29] Hill, 2001, Medical image registration, Phys. Med. Biol. source

      Not checked in the free run.

    • cites [30] Hong, 2012

      Not checked in the free run.

    • cites [31] Horn, 1981, Determining optical flow, Artif. Intell. source

      Not checked in the free run.

  • May combine two claims.

    Interpretationreasoning

    Because the moving image and the warped-back target image are in the same coordinate system, their differences are small as long as the predicted deformation is reasonable, so more accurate predictions can be expected.

    “Because M and T ○ Φ are in the same coordinate system, the differences between these two images are small as long as the predicted deformation is reasonable, and more accurate predictions can be expected.”
    Quote in the paperSubstantial claim

  • Classified as probably not a substantial claim.

    Premisecitation

    Gal et al. showed that ELBO maximization is achieved by training with dropout.

    “Gal et al. [58] show that ELBO maximization is achieved by training with dropout [57].”
    Quote in the paperSubstantial claim0 of 2 sources checked
    • cites [58] Vaillant, 2004, Statistics on diffeomorphisms via tangent space representations, NeuroImage source

      Not checked in the free run.

    • cites [57] Srivastava, 2014, Dropout: a simple way to prevent neural networks from overfitting, JMLR

      Not checked in the free run.

  • May combine two claims.

    Interpretationreasoning

    Using a large voxel stride is reasonable because the initial momentum has compact support at edges and the pooling/unpooling operations give spatial shift invariance.

    “This is reasonable for our initial momentum parameterization because of the compact support (at edges) of the initial momentum and the spatial shift invariance we obtain via the pooling/unpooling operations.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Premisecitation

    For image-based LDDMM, the momentum equals a scalar field times the image gradient, so it is zero in homogeneous regions.

    “for image-based LDDMM [17, 19, 45] the momentum can be written as m(x,t) = λ(x,t)∇I(x,t), where λ is a scalar field and ∇I is the spatial gradient of the image. Hence, for homogeneous areas, ∇I = 0 and consequentially m = 0.”
    Quote in the paperSubstantial claim0 of 3 sources checked
    • cites [17] Gal, 2016

      Not checked in the free run.

    • cites [19] Grabner, 2006, Symmetric Atlasing and Model Based Segmentation: an Application to the hippocampus in Older Adults

      Not checked in the free run.

    • cites [45] Niethammer, 2011

      Not checked in the free run.

  • Classified as probably not a substantial claim.

    Premisenone given

    Multi-modal image registration is generally much harder than uni-modal registration because image appearance can change greatly across modalities.

    “Multi-modal image registration is, in general, significantly more difficult than uni-modal image registration since image appearance can change drastically between different modalities.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Study design and conductown data

    Increasing the training data size beyond the amount used yields only marginal improvements, which are clearly outperformed by a combined prediction+correction strategy.

    “These experiments indicated that increasing the training data size further only results in marginal improvements, which are clearly outperformed by a combined prediction + correction strategy.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    Recent research suggests that a deep network's degrees of freedom can be significantly smaller than its number of parameters.

    “recent research [56] suggests that the degrees of freedom for a deep network can be significantly smaller than the number of its parameters.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [56] Springenberg, 2014

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    Dropout can be viewed as Bernoulli approximate inference in Bayesian neural networks.

    “using dropout [57], which can be viewed as (Bernoulli) approximate inference in Bayesian neural networks [58, 59].”
    Quote in the paperNot a substantial claim0 of 3 sources checked
    • cites [57] Srivastava, 2014, Dropout: a simple way to prevent neural networks from overfitting, JMLR

      Not checked in the free run.

    • cites [58] Vaillant, 2004, Statistics on diffeomorphisms via tangent space representations, NeuroImage source

      Not checked in the free run.

    • cites [59] Van Essen, 2013, WU-Minn HCP Consortium, the WU-Minn human connectome project: an overview, NeuroImage source

      Not checked in the free run.

  • Classified as probably not a substantial claim. May combine two claims.

    Interpretationnone given

    The 1-norm loss was selected to tolerate outliers and produce sharper momentum predictions.

    “We chose the 1-norm instead of the 2-norm as our loss function to be able to tolerate outliers and to generate sharper momentum predictions.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S4 (based).

    Premisenone given

    Given the initial momentum, the whole geodesic path can be generated using the geodesic shooting equations.

    “given the initial momentum, we are able to generate the whole geodesic path using the geodesic shooting equations.”
    Quote in the paperNot a substantial claim

  • Classified as probably not a substantial claim. May combine two claims.

    Premisenone given

    A diffeomorphic deformation map should have positive Jacobian determinants everywhere, otherwise foldings occur.

    “Assuming no flips of the entire coordinate system, a diffeomorphic deformation map should have positive Jacobian determinants everywhere, otherwise foldings occur in the deformation maps.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    Gutierrez et al. use a regression forest and gradient boosted trees with hand-crafted features to learn update steps for rigid and B-spline registration.

    “Gutierrez et al. [9] use a regression forest and gradient boosted trees [8] based on hand-crafted features to learn update steps for a rigid and a B-spline registration model.”
    Quote in the paperNot a substantial claim0 of 2 sources checked
    • cites [9] Cao, 2014, Multi-modal registration for correlative microscopy using image analogies, MedIA

      Not checked in the free run.

    • cites [8] Bruveris, 2012, Mixture of kernels and iterated semidirect product of diffeomorphisms groups, Multiscale Model. Simul. source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    Wang et al.'s deformation prediction performance depends heavily on the accuracy of key-point selection.

    “Wang et al. [11] predict deformations by key-point matching using sparse learning followed by dense deformation field generation with radial basis function interpolation. The performance of the method heavily depends on the accuracy of the key point selection.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [11] Cheng, 2015, Deep similarity learning for multimodal medical images, Comput. Methods Biomech. Biomed. Eng. Imaging Vis.

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Interpretationreasoning

    Performing the correction for the initial momentum in the original coordinate space of the moving image allows obtaining an overall corrected initial momentum, which is useful for statistics in a fixed coordinate system such as an atlas.

    “the correction for the initial momentum is then performed in the original coordinate space (of the moving image) which allows us to obtain an overall corrected initial momentum, m0. This is for example a useful property when the goal is to do statistics with respect to a fixed coordinate system, for example, an atlas coordinate system.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Interpretationreasoning

    Using larger patches would require a network with substantially larger capacity and likely much larger training datasets.

    “While using larger patches would be an option, this would require a network with substantially larger capacity (to store the information for larger image patches and all meaningful deformations) and would also likely require much larger training datasets8.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Premisecitation

    PReLU avoids zero gradients for negative inputs and thereby improves network performance.

    “In contrast to ReLU, PReLU avoids a zero gradient for negative inputs, effectively improving the network performance.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [52] Simpson, 2011

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S4 (based).

    Study design and conductnone given

    Keeping dropout enabled at test time maintains the probabilistic property of the network.

    “For testing, we keep the dropout layers enabled to maintain the probabilistic property of the network”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Interpretationreasoning

    The absence of this pattern for momentum suggests that it is easier for the network to learn to predict momentum, as it has become more specialized to training data obtained with stride 14.

    “That this is not the case for the momentum parameterization suggests that it is easier for the network to learn to predict the momentum, as it indeed has become more specialized to the training data which was obtained with a stride of 14.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S4 (based).

    Interpretationreasoning

    The proposed two-step prediction approach improves overall prediction accuracy.

    “To address these shortcomings, we propose a two-step prediction approach to improve overall prediction accuracy.”
    Quote in the paperNot a substantial claim

  • Classified as probably not a substantial claim.

    Premisecitation

    The vector-valued momentum has been found to behave numerically better for LDDMM optimizations.

    “the vector-valued momentum has been found to numerically better behave for LDDMM optimizations [50].”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [50] Shams, 2010, A survey of medical image registration on multicore and the GPU, IEEE Signal Process. Mag. source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

    Premisenone given

    LDDMM shooting has the ability to generate diffeomorphic deformations.

    “One of the important properties of LDDMM shooting is its ability to generate diffeomorphic deformations.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Interpretationown data

    The initial momentum parameterization is the preferred choice among the three tested because it achieves the best prediction accuracy and guarantees diffeomorphic deformations.

    “Hence, the initial momentum parameterization is the preferred choice among our three tested parameterizations as it achieves the best prediction accuracy and guarantees diffeomorphic deformations.”
    Quote in the paperSubstantial claim

  • Classified as probably not a substantial claim.

    Interpretationreasoning

    This parity is reasonable because these methods are all non-parametric diffeomorphic or homeomorphic registration methods that allow modeling large deformations.

    “This is reasonable as these methods are all non-parametric diffeomorphic or homeomorphic registration methods, allowing the modeling of large deformations between image pairs.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Interpretationreasoning

    The proposed correction network is a step towards highly accurate deformation prediction while only decreasing computation speed by a factor of 2.

    “The proposed correction network is a step towards highly accurate deformation prediction, while only decreasing the computation speed by a factor of 2.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Study design and conductreasoning

    Among the prediction models, only LPC was specifically trained to match LDDMM optimization results and to predict corrections to the initial momentum from LP in the tangent space of the moving image; LPP, LPC2, and LPC3 lacked this theoretical motivation.

    “Note that among these methods, only LPC is specifically trained to match the LDDMM optimization results and in particular to predict corrections to the initial momentum obtained by the prediction model (LP) in the tangent space of the moving image. In contrast, LPP, LPC2 and LPC3 lack this theoretical motivation.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Interpretationreasoning

    Using a smaller step size would regain numerical stability in this case.

    “Using a smaller step-size would regain numerical stability in this case.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S4 (based).

    Premisenone given

    A low LDDMM energy only implies a good registration result if the registration model is fully appropriate for the task.

    “However, note that, in general, a low energy will only imply a good registration result if the registration model is fully appropriate for the registration task.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Interpretationreasoning

    As the number of GPUs increases, communication overhead between GPUs becomes larger, explaining why computation time does not equal 11.23 divided by the number of GPUs.

    “However, as the number of GPUs increases, the communication overhead between GPUs becomes larger which explains why computation time does not equal to 11.23/number of GPUs seconds.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Interpretationown data

    The initial momentum parameterization, which can tolerate large sliding window strides, is essential for fast deformation prediction with high accuracy while guaranteeing diffeomorphic deformations.

    “Hence the initial momentum parameterization, which can tolerate large sliding window strides, is essential for fast deformation prediction with high accuracy while guaranteeing diffeomorphic deformations.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Interpretationreasoning

    Large deformations created by numerical LDDMM optimization are not always desirable, so registration errors of prediction models relative to numerical optimization are sometimes preferred.

    “These large deformations created by numerical LDDMM optimization are not always desirable; and consequentially registration errors of the prediction models with respect to the numerical optimization result are in fact sometimes preferred.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Interpretationown data

    The patch-wise prediction approach enables a multi-GPU implementation, further increasing prediction speed.

    “Moreover, the patch-wise prediction approach of our methods enables a multi-GPU implementation, further increasing the prediction speed.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Interpretationown data

    The conservative nature of patch-wise momentum prediction is more likely to generate reasonable deformations, which can be regarded as an advantage of the network.

    “This can be regarded as an advantage of our network, where the conservative nature of patch-wise momentum prediction is more likely to generate reasonable deformations.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Interpretationown data

    The proposed approach allows taking large strides for patch-wise prediction without a substantial decrease in registration accuracy, resulting in fast and accurate deformation prediction.

    “The proposed approach allows taking large strides for patch-wise prediction, without a substantial decrease in registration accuracy, resulting in fast and accurate deformation prediction.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Interpretationreasoning

    The framework is very general and can be directly applied to many other registration techniques.

    “Our framework is very general and can be directly applied to many other registration techniques.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Interpretationown data

    The method retains all theoretical properties of LDDMM and results in diffeomorphic transformations if appropriately regularized, but computes these transformations an order of magnitude faster than GPU-based optimization for the LDDMM model.

    “Our method retains all theoretical properties of LDDMM and results in diffeomorphic transformations if appropriately regularized, but computes these transformations an order of magnitude faster than a GPU-based optimization for the LDDMM model.”
    • Also stated in: § 4. Discussion · plain
      “In effect, our Quicksilver registration approach converts a notoriously slow and memory-hungry registration approach to a fast method, while retaining all of its appealing mathematical properties.”
    Quote in the paperSubstantial claim
63 of 63

  • The authors state this as an assumption.

    A set of LDDMM parameters obtained by numerical optimization already exists and yields good registration results.

    “We assume that we already have a set of LDDMM parameters which result in good registration results.”

  • The paper relies on this without support.

    The local variance of deformation fields generated from multiple dropout samples provides an uncertainty estimate of the predicted deformation field.

    “The local variance of these deformation fields can then be used as an uncertainty estimate of the predicted deformation field.”
2 of 2

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Some uncertainty estimation approaches report interval estimates by reformulating the registration objective as a probability distribution and drawing samples, using test-time NN drop-out, or sampling using test-time deformation covariance matrices in a variational autoencoder.

    “Some of these approaches report interval estimates, either by reformulating the registration objective function as a probability distribution and drawing samples (Risholm et al., 2013; Le Folgoc et al., 2016; Schultz et al., 2018), approximating the sampling process using test-time NN drop-out (Yang et al., 2017) or sampling using the test-time deformation covariance matrices embedded within a”

    From The Impact of Machine Learning on 2D/3D Registration for Image-Guided Interventions: A Systematic Review and Perspective (2021) · cites this paper as Yang et al., 2017

    The Methods section (2.2.2, page unavailable) says dropout remains enabled at test time to sample multiple predictions, and that the resulting deformation-field variance estimates uncertainty. The Results section (3.1, page unavailable) reports a variance-based uncertainty map, not interval estimates. The other sampling approaches in the citing sentence are attributed to other sources. (LLM, conf=0.85)

    Experimental — accuracy not yet measured

  • Yang et al. implemented stacked convolutional layers as an encoder-decoder approach to predict the morphing of the input pixel into its last formation using MRI brain scans from the OASIS dataset, employing LDDMM and attaining remarkable enhancements in computation time.

    “Yang et al. [323] implemented stacked convolutional layers as an encoder-decoder approach to predict the morphing of the input pixel into its last formation using MRI brain scans from the OASIS dataset. They employed a registration model known as Large Deformation Diffeomorphic Metric Mapping (LDDMM) and attained remarkable enhancements in computation time.”

    From Review of deep learning: concepts, CNN architectures, challenges, applications, future directions (2021) · cites this paper as [323]

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

Each contradiction between two passages listed here was confirmed by an independent check.

  • Image-to-image test total does not match the dataset counts

    Minor

    The four listed dataset counts sum to 2,088 registrations, not 2,168.

    Reported 2168 · recomputed 2088

  • Multimodal test-pair count is inconsistent with 16 images

    Minor

    Pairwise directed registrations among 16 images yield 16 × 15 = 240 distinct-image cases, not 250.

    Reported 250 · recomputed 240

  • Multimodal training-set size changes from 359 to 365

    Minor

    The dataset split explicitly assigns 359 images to training, but the later description calls the training set 365 images.

  • Reported prediction runtime exceeds its stated components

    Minor

    The stated prediction and shooting times add to 16.53 seconds, rather than the reported total of 18.43 seconds.

    Reported 18.43 · recomputed 16.53

4 of 4

Cites

Flagged first, then by citations

17 of the 94 references couldn’t be matched to a known paper.

Cited by

Sorted by citations

Related

Authors
Yang, X., Kwitt, R., Styner, M., & Niethammer, M.
Affiliations
  • University of North Carolina at Chapel Hill (US)
  • University of Salzburg (AT)
Corresponding author
Yang, Xiao
ORCID
Venue
NeuroImage
Publisher
Elsevier BV
Published
2017-09
Volume
158
Pages
378-396
ISSN
1053-8119 (ISSN-L)
DOI
10.1016/j.neuroimage.2017.07.008
PMC
PMC6036629
OpenAlex
W2604920239
Cited by
630
References
94
Citations by year
2026: 17 · 2025: 53 · 2024: 74 · 2023: 65 · 2022: 101 · 2021: 85 · 2020: 95 · 2019: 93 · 2018: 40 · 2017: 7
FWCI
26.68
Type
journal article
Language
en
Open access
Licence
Locations
  • NeuroImage (acceptedVersion)
  • PubMed (publishedVersion)
  • Europe PMC (PubMed Central) (acceptedVersion)
  • PubMed Central (acceptedVersion)
Funders
  • NIH (1 R41 NS091792-01, HDO55741)
  • NSF (ECCS-1148870, EECS-1711776)
  • National Science Foundation Major Research Instrumentation program (ACI-1429830)
Topics
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Medical Image Segmentation Techniques
  • Physical Sciences › Computer Science › Artificial Intelligence › Domain Adaptation and Few-Shot Learning
  • Health Sciences › Medicine › Radiology, Nuclear Medicine and Imaging › Retinal Imaging and Analysis
Keywords
deformable image registration, LDDMM, encoder-decoder architecture, multimodal image registration, image similarity measures, diffeomorphic registration
MeSH terms
Machine Learning, Algorithms, Brain, Brain Mapping, Humans, Image Processing, Computer-Assisted, Pattern Recognition, Automated, Imaging, Three-Dimensional

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Claims read by deepseek-v4.1-flash under definition version 3 (extract-v3) on 2026-10-05.

Classified by jev-1.13-20260917 (gold-guideline-v0.2) on 2026-10-05.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review