Cited by 654Medical Image Segmentation TechniquesDomain Adaptation and Few-Shot LearningRetinal Imaging and Analysis

The DOI matches its Crossref record; 77 of the 94 references it cites were found in a registry or an index; none of the 43 sources its claims cite is checked for support yet.

Quicksilver: Fast predictive image registration – A deep learning approach

Open access

Yang, Xiao, Kwitt, Roland, Styner, Martin, Niethammer, Marc·2017·NeuroImage

DOI 10.1016/j.neuroimage.2017.07.008PMC PMC6036629

Open this paper in CoChat — free account

Abstract

This paper introduces Quicksilver, a fast deformable image registration method. Quicksilver registration for image-pairs works by patch-wise prediction of a deformation model based directly on image appearance. A deep encoder-decoder network is used as the prediction model. While the prediction strategy is general, we focus on predictions for the Large Deformation Diffeomorphic Metric Mapping (LDDMM) model. Specifically, we predict the momentum-parameterization of LDDMM, which facilitates a patch-wise prediction strategy while maintaining the theoretical properties of LDDMM, such as guaranteed diffeomorphic mappings for sufficiently strong regularization. We also provide a probabilistic version of our prediction network which can be sampled during the testing time to calculate uncertainties in the predicted deformations. Finally, we introduce a new correction network which greatly increases the prediction accuracy of an already existing prediction network. We show experimental results for uni-modal atlas-to-image as well as uni-/multi-modal image-to-image registrations. These experiments demonstrate that our method accurately predicts registrations obtained by numerical optimization, is very fast, achieves state-of-the-art registration results on four standard validation datasets, and can jointly learn an image similarity measure. Quicksilver is freely available as an open-source software.

AI snapshot

Written from the paper’s full text.

TL;DR

Quicksilver is a deep-learning registration method that predicts the initial momentum of LDDMM shooting patch-by-patch from moving/target image patches, with optional Bayesian uncertainty and a correction network. It achieves registration accuracy close to numerical LDDMM optimization on several brain MRI datasets while running much faster.

Key findings

Predicting the initial momentum gave better atlas-to-image accuracy than predicting initial velocity or displacement; momentum and velocity networks produced positive Jacobian determinants in all tested cases, whereas direct displacement prediction did not guarantee diffeomorphic maps.
On the four Klein et al. validation datasets, the prediction+correction model (LPC) approached numerical LDDMM optimization (LO): paired TOST indicated LPC was statistically equivalent to LO within a 2% relative threshold on all four datasets, and label overlap was top-tier with few outliers.
Runtime for a 229×193×193 volume: LDDMM optimization took about 10.8 minutes, while Quicksilver prediction took 7.63 s plus 8.9 s for shooting, totaling 18.43 s, a roughly 35× speedup; adding the correction network doubled time but remained an order of magnitude faster than optimization. Patch pruning and stride 14 reduced the number of predicted patches by 99.995% for such volumes.
Multimodal T1w–T2w prediction reduced deformation error relative to affine registration and was only slightly worse than the T1w–T1w upper bound; training with only 10 images (90 registration pairs) still worked with a small performance drop. Bayesian sampling produced uncertainty maps with high values near ventricles and large deformations.

Methods

The method uses the LDDMM shooting formulation and trains a deep encoder-decoder network to predict vector-valued initial momentum from paired 15×15×15 patches of moving and target images. Whole-image prediction uses a sliding window with stride 14 and patch pruning, followed by LDDMM shooting to generate deformations.
Data: atlas-to-image used 150 OASIS images (100 training, 50 testing); image-to-image used 373 OASIS images for training and tested on LPBA40, IBSR18, MGH10, and CUMC12, totaling 2,168 test registrations; multimodal used the IBIS 3D dataset with 375 T1w/T2w images (359 training, 16 testing, 250 test cases). A correction network was trained sequentially on the residual between ground-truth and predicted momentum; a probabilistic version used dropout (probability 0.2) and Monte Carlo sampling.
Evaluation included voxel-wise deformation error percentiles, Jacobian determinant positivity, LDDMM energy, target overlap of labels, paired t-tests, paired TOST equivalence tests, and runtime/multi-GPU scaling.

Limitations

The paper states that the 15×15×15 patch receptive field limits prediction of very large deformations; OASIS training data had only 0.2% of voxels with deformations above 10 mm, so large deformations were underrepresented. Outliers with inconsistent anatomy can cause extreme optimization deformations that prediction may not match, though the conservative prediction can also avoid implausible warps.
The correction network doubles runtime, and applying correction multiple times can produce more drastic deformations and numerical instability; 12 LPP cases in LPBA40 failed LDDMM shooting due to integration problems. Patch pruning may not transfer to parametric models if parameter locality is not guaranteed, and multi-GPU scaling is limited by communication overhead and PyTorch’s beta-stage performance.
Evident from the text: validation is restricted to brain MRI with a specific LDDMM regularizer and SSD similarity, so generalization to other organs, modalities, or registration models is not demonstrated.

Model-written summary. Check it against the paper before relying on it.

Integrity signals

Registry

Crossref · 10.1016/j.neuroimage.2017.07.008

Cites

77 of 77 references verified0 citation errors · 0 not checkable

Statistics

Nothing to recompute.

Claims the paper makes

0 of 43 cited sources checked · 0 supported · 0 flagged

The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“Finally, we introduce a new correction network which greatly increases the prediction accuracy of an already existing prediction network.”
§ Quicksilver: Fast Predictive Image Registration – a Deep Learning Approach > Abstract
Load-bearingStrength:plain· not yet calibratedCausal:direct· not yet calibrated
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passes
- The authors state this as an assumption: A set of LDDMM parameters obtained by numerical optimization already exists and yields good registration results.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Our experiments show that our approach also predicts accurate deformations for multi-modal registration.”
§ Quicksilver: Fast Predictive Image Registration – a Deep Learning Approach > 1. Introduction
Load-bearingStrength:plain· not yet calibratedCausal:direct· not yet calibrated
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passes
- The authors state this as an assumption: A set of LDDMM parameters obtained by numerical optimization already exists and yields good registration results.
In the paper
“achieves state-of-the-art registration results on four standard validation datasets”
- Also stated in: § 1. Introduction · boosted
  “demonstrate registration accuracies on these datasets on par with the state-of-the-art”
§ Quicksilver: Fast Predictive Image Registration – a Deep Learning Approach > Abstract
What the sources say
- Not checked yetcites [33] Kingma, 2014
Load-bearingStrength:boosted· not yet calibratedCausal:direct· not yet calibrated
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passes
- The authors state this as an assumption: A set of LDDMM parameters obtained by numerical optimization already exists and yields good registration results.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“These experiments demonstrate that our method accurately predicts registrations obtained by numerical optimization”
§ Quicksilver: Fast Predictive Image Registration – a Deep Learning Approach > Abstract
Load-bearingStrength:boosted· not yet calibratedCausal:correlational· not yet calibrated
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passes
- The authors state this as an assumption: A set of LDDMM parameters obtained by numerical optimization already exists and yields good registration results.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Experiments show that the correction network improves registration results to the point where optimization-based and predicted registrations achieve a similar level of registration accuracy on registration validation experiments.”
§ Quicksilver: Fast Predictive Image Registration – a Deep Learning Approach > 1. Introduction
Load-bearingStrength:plain· not yet calibrated
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passes
- The authors state this as an assumption: A set of LDDMM parameters obtained by numerical optimization already exists and yields good registration results.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“is very fast”
§ Quicksilver: Fast Predictive Image Registration – a Deep Learning Approach > Abstract
Load-bearingStrength:plain· not yet calibrated
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passes
- The authors state this as an assumption: A set of LDDMM parameters obtained by numerical optimization already exists and yields good registration results.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“and can jointly learn an image similarity measure”
- Also stated in: § 1. Introduction · plain
  “our framework can simultaneously predict registrations and learn a multi-modal image similarity measure”
§ Quicksilver: Fast Predictive Image Registration – a Deep Learning Approach > Abstract
Load-bearingStrength:plain· not yet calibratedCausal:direct· not yet calibrated
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passes
- The authors state this as an assumption: A set of LDDMM parameters obtained by numerical optimization already exists and yields good registration results.

Show all 84 claims

It may state two separate claims as one. 46
Probably not a substantial claim. 22

A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“We experimentally show that by using patch pruning and a large sliding window stride, our method achieves dramatic speedups compared to the optimization approach, while maintaining good registration accuracy.”
§ Quicksilver: Fast Predictive Image Registration – a Deep Learning Approach > 1. Introduction
Load-bearingStrength:boosted· not yet calibratedCausal:direct· not yet calibrated
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passes
It may state two separate claims as one.
- The authors state this as an assumption: A set of LDDMM parameters obtained by numerical optimization already exists and yields good registration results.
In the paper
“For a relative threshold difference of 2% LPC can be considered statistically equivalent to LO for all four datasets and to many of the other top methods (e.g., LPC vs. SyN on MGH10 and IBSR18).”
§ Quicksilver: Fast Predictive Image Registration – a Deep Learning Approach > 3. Results > 3.2. Image-to-Image registration
What the sources say
- Not checked yetcites [71] Zhang, 2015
Load-bearingStrength:plain· not yet calibrated
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passes
It may state two separate claims as one.
In the paper
“It is based on Riemannian geometry, and hence induces a distance metric on the space of images.”
§ Quicksilver: Fast Predictive Image Registration – a Deep Learning Approach > 1. Introduction
What the sources say
- Not checked yetcites [17] Gal, 2016
Load-bearingStrength:plain· not yet calibratedCausal:direct· not yet calibrated
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passes
Probably not a substantial claim. It may state two separate claims as one.
In the paper
“It uses the initial momentum as the registration parameter, which does not need to be spatially smooth, and hence can be predicted patch-by-patch, and from which the whole geodesic path can be computed.”
§ Quicksilver: Fast Predictive Image Registration – a Deep Learning Approach > 1. Introduction
What the sources say
- Not checked yetcites [17] Gal, 2016
Load-bearingStrength:plain· not yet calibratedCausal:direct· not yet calibrated
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passes
It may state two separate claims as one.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“the initial momentum is generally not smooth, but is compactly supported at image edges”
- Also stated in: § 1. Introduction · hedged
  “the momentum is typically compactly supported around image edges”
§ Quicksilver: Fast Predictive Image Registration – a Deep Learning Approach > 2. Materials and Methods > 2.1. LDDMM Shooting
Load-bearingStrength:plain· not yet calibrated
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes? Based: unclear
It may state two separate claims as one.
In the paper
“We demonstrate that the momentum-parameterization for LDDMM shooting [17] is a convenient representation for regression approaches as (i) the momentum is typically compactly supported around image edges and (ii) there are no smoothness requirements on the momentum itself.”
- Also stated in: § Abstract · plain
  “Specifically, we predict the momentum-parameterization of LDDMM, which facilitates a patch-wise prediction strategy while maintaining the theoretical properties of LDDMM, such as guaranteed diffeomorphic mappings for sufficiently strong regularization.”
§ Quicksilver: Fast Predictive Image Registration – a Deep Learning Approach > 1. Introduction
What the sources say
- Not checked yetcites [17] Gal, 2016
Load-bearingStrength:boosted· not yet calibratedCausal:direct· not yet calibrated
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passes
It may state two separate claims as one.
None of the sources cited for this claim have been checked yet.
In the paper
“General approaches address multimodal image registration by either performing image synthesis [22, 23] to change the problem to a uni-modal image registration task, or by proposing complex, hand-crafted [24, 25, 26, 27] or learned [28, 29, 30, 31, 32] multi-modal image similarity measures.”
§ Quicksilver: Fast Predictive Image Registration – a Deep Learning Approach > 1. Introduction
What the sources say
- Not checked yetcites [22] Gutierrez-Becker, 2017, Guiding multimodal registration with learned optimization updates, MedIA
- Not checked yetcites [23] Gutiérrez-Becker, 2016
- Not checked yetcites [24] Haber, 2007, Image registration with guaranteed displacement regularity, Int. J. Comput. Vis. source
- Not checked yetcites [25] Hart, 2009, An optimal control approach for deformable registration
- Not checked yetcites [26] Hazlett, 2017, Early brain development in infants at high risk for autism spectrum disorder, Nature source
- Not checked yetcites [27] He, 2015
- Not checked yetcites [28] Hermosillo, 2002, Variational methods for multimodal image matching, IJCV source
- Not checked yetcites [29] Hill, 2001, Medical image registration, Phys. Med. Biol. source
- Not checked yetcites [30] Hong, 2012
- Not checked yetcites [31] Horn, 1981, Determining optical flow, Artif. Intell. source
Load-bearingStrength:plain· not yet calibrated
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passes
Probably not a substantial claim.
None of the sources cited for this claim have been checked yet.
In the paper
“Gal et al. [58] show that ELBO maximization is achieved by training with dropout [57].”
§ Quicksilver: Fast Predictive Image Registration – a Deep Learning Approach > 2. Materials and Methods > 2.2. Deep network for LDDMM prediction
What the sources say
- Not checked yetcites [58] Vaillant, 2004, Statistics on diffeomorphisms via tangent space representations, NeuroImage source
- Not checked yetcites [57] Srivastava, 2014, Dropout: a simple way to prevent neural networks from overfitting, JMLR
Load-bearingStrength:plain· not yet calibratedCausal:direct· not yet calibrated
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passes
Probably not a substantial claim.
None of the sources cited for this claim have been checked yet.
In the paper
“for image-based LDDMM [17, 19, 45] the momentum can be written as m(x,t) = λ(x,t)∇I(x,t), where λ is a scalar field and ∇I is the spatial gradient of the image. Hence, for homogeneous areas, ∇I = 0 and consequentially m = 0.”
§ Quicksilver: Fast Predictive Image Registration – a Deep Learning Approach > 2. Materials and Methods > 2.1. LDDMM Shooting
What the sources say
- Not checked yetcites [17] Gal, 2016
- Not checked yetcites [19] Grabner, 2006, Symmetric Atlasing and Model Based Segmentation: an Application to the hippocampus in Older Adults
- Not checked yetcites [45] Niethammer, 2011
Load-bearingStrength:plain· not yet calibratedCausal:direct· not yet calibrated
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passes
It may state two separate claims as one.
In the paper
“recent research [56] suggests that the degrees of freedom for a deep network can be significantly smaller than the number of its parameters.”
§ Quicksilver: Fast Predictive Image Registration – a Deep Learning Approach > 2. Materials and Methods > 2.2. Deep network for LDDMM prediction
What the sources say
- Not checked yetcites [56] Springenberg, 2014
Strength:hedged· not yet calibrated
Tests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passes
Probably not a substantial claim. The paper’s conclusions may not depend on it.
None of the sources cited for this claim have been checked yet.
In the paper
“using dropout [57], which can be viewed as (Bernoulli) approximate inference in Bayesian neural networks [58, 59].”
§ Quicksilver: Fast Predictive Image Registration – a Deep Learning Approach > 2. Materials and Methods > 2.2. Deep network for LDDMM prediction
What the sources say
- Not checked yetcites [57] Srivastava, 2014, Dropout: a simple way to prevent neural networks from overfitting, JMLR
- Not checked yetcites [58] Vaillant, 2004, Statistics on diffeomorphisms via tangent space representations, NeuroImage source
- Not checked yetcites [59] Van Essen, 2013, WU-Minn HCP Consortium, the WU-Minn human connectome project: an overview, NeuroImage source
Strength:plain· not yet calibrated
Tests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passes
Probably not a substantial claim. The paper’s conclusions may not depend on it.
None of the sources cited for this claim have been checked yet.
In the paper
“Gutierrez et al. [9] use a regression forest and gradient boosted trees [8] based on hand-crafted features to learn update steps for a rigid and a B-spline registration model.”
§ Quicksilver: Fast Predictive Image Registration – a Deep Learning Approach > 1. Introduction
What the sources say
- Not checked yetcites [9] Cao, 2014, Multi-modal registration for correlative microscopy using image analogies, MedIA
- Not checked yetcites [8] Bruveris, 2012, Mixture of kernels and iterated semidirect product of diffeomorphisms groups, Multiscale Model. Simul. source
Strength:plain· not yet calibratedCausal:direct· not yet calibrated
Tests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passes
Probably not a substantial claim. The paper’s conclusions may not depend on it.
In the paper
“Wang et al. [11] predict deformations by key-point matching using sparse learning followed by dense deformation field generation with radial basis function interpolation. The performance of the method heavily depends on the accuracy of the key point selection.”
§ Quicksilver: Fast Predictive Image Registration – a Deep Learning Approach > 1. Introduction
What the sources say
- Not checked yetcites [11] Cheng, 2015, Deep similarity learning for multimodal medical images, Comput. Methods Biomech. Biomed. Eng. Imaging Vis.
Strength:plain· not yet calibratedCausal:direct· not yet calibrated
Tests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passes
Probably not a substantial claim. The paper’s conclusions may not depend on it.
In the paper
“In contrast to ReLU, PReLU avoids a zero gradient for negative inputs, effectively improving the network performance.”
§ Quicksilver: Fast Predictive Image Registration – a Deep Learning Approach > 2. Materials and Methods > 2.2. Deep network for LDDMM prediction
What the sources say
- Not checked yetcites [52] Simpson, 2011
Strength:plain· not yet calibratedCausal:direct· not yet calibrated
Tests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passes
Probably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.
In the paper
“the vector-valued momentum has been found to numerically better behave for LDDMM optimizations [50].”
§ Quicksilver: Fast Predictive Image Registration – a Deep Learning Approach > 2. Materials and Methods > 2.2. Deep network for LDDMM prediction
What the sources say
- Not checked yetcites [50] Shams, 2010, A survey of medical image registration on multicore and the GPU, IEEE Signal Process. Mag. source
Load-bearingStrength:plain· not yet calibratedCausal:direct· not yet calibrated
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passes
Probably not a substantial claim.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Our method retains all theoretical properties of LDDMM and results in diffeomorphic transformations if appropriately regularized, but computes these transformations an order of magnitude faster than a GPU-based optimization for the LDDMM model.”
- Also stated in: § 4. Discussion · plain
  “In effect, our Quicksilver registration approach converts a notoriously slow and memory-hungry registration approach to a fast method, while retaining all of its appealing mathematical properties.”
§ Quicksilver: Fast Predictive Image Registration – a Deep Learning Approach > 4. Discussion
Load-bearingStrength:plain· not yet calibratedCausal:direct· not yet calibrated
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passes
It may state two separate claims as one.

63 of 63

The authors state this as an assumption.
A set of LDDMM parameters obtained by numerical optimization already exists and yields good registration results.
“We assume that we already have a set of LDDMM parameters which result in good registration results.”
The paper relies on this without support.
The local variance of deformation fields generated from multiple dropout samples provides an uncertainty estimate of the predicted deformation field.
“The local variance of these deformation fields can then be used as an uncertainty estimate of the predicted deformation field.”

2 of 2

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

Each contradiction between two passages listed here was confirmed by an independent check.

Image-to-image test total does not match the dataset counts
Minor
The four listed dataset counts sum to 2,088 registrations, not 2,168.
Reported 2168 · recomputed 2088
Multimodal test-pair count is inconsistent with 16 images
Minor
Pairwise directed registrations among 16 images yield 16 × 15 = 240 distinct-image cases, not 250.
Reported 250 · recomputed 240
Multimodal training-set size changes from 359 to 365
Minor
The dataset split explicitly assigns 359 images to training, but the later description calls the training set 365 images.
Reported prediction runtime exceeds its stated components
Minor
The stated prediction and shooting times add to 16.53 seconds, rather than the reported total of 18.43 seconds.
Reported 18.43 · recomputed 16.53

4 of 4

Adam: A Method for Stochastic Optimization
Kingma, D. P., & Ba, J. (2014) · arXiv · doi:10.48550/arxiv.1412.6980 · Cited by 98,946 · cited 2×
“The network is implemented in PyTorch, and optimized using Adam [36], where we set the learning rate to 0.”
Method
Fully convolutional networks for semantic segmentation
Long, J., Shelhamer, E., & Darrell, T. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298965 · Cited by 38,834 · cited 1×
“layer [24] to perform “unpooling” (Dec1(F ) in Fig.”
Background
Untitled work
(2014) · Cited by 33,419
Numerical optimization
Braun, W. J., & Murdoch, D. J. (2021) · Cambridge University Press eBooks · doi:10.1017/9781108993456.011 · Cited by 14,099
Untitled work
(2010) · International Conference on Machine Learning · Cited by 13,995
Fully convolutional networks for semantic segmentation
Shelhamer, E., Long, J., & Darrell, T. (2017) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2016.2572683 · Cited by 11,898
Determining optical flow
Horn, B. K. P., & Schunck, B. G. (1981) · Artificial Intelligence · doi:10.1016/0004-3702(81)90024-2 · Cited by 10,312
Numerical Optimization
Nocedal, J. M., & Wright, S. J. (2006) · Springer series in operations research/Springer series in operations research and financial engineering · doi:10.1007/978-0-387-40065-5 · Cited by 10,206
A fast diffeomorphic image registration algorithm
Ashburner, J. (2007) · NeuroImage · doi:10.1016/j.neuroimage.2007.07.007 · Cited by 8,122 · cited 3×
“Several points should be noted: ﬁrst, the LDDMM optimization performance is on par with SyN [68], ART [69] and the SPM5 DARTEL Toolbox ( SPM5D ) [70].”
Method
Determining Optical Flow
Horn, B. K. P., & Schunck, B. G. (1981) · Proceedings of SPIE, the International Society for Optical Engineering/Proceedings of SPIE · doi:10.1117/12.965761 · Cited by 7,443 · cited 1×
“Optical flow approaches, such as the classical Horn and Schunck optical flow [41], the more recent total variation approaches [42], or methods based on linear elasticity theory [1] are examples for displacementbased registration formulations.”
Method

Flagged first, then by citations

Show all 94 references

17 of the 94 references couldn’t be matched to a known paper.

Sorted by citations

Review of deep learning: concepts, CNN architectures, challenges, applications, future directions
Alzubaidi, L., Zhang, J., Humaidi, A. J., Al-Dujaili, A., Duan, Y., Al-Shamma, O., Santamaría, J., Fadhel, M. A., Al-Amidie, M., & Farhan, L. (2021) · Journal of Big Data · doi:10.1186/s40537-021-00444-8 · Cited by 7,906
Artificial intelligence in radiology
Hosny, A., Parmar, C., Quackenbush, J., Schwartz, L. H., & Aerts, H. J. W. L. (2018) · Nature Reviews Cancer · doi:10.1038/s41568-018-0016-5 · Cited by 3,803
“Studies have shown that deep learning leads to generally more consistent registrations and is an order of magnitude faster than more conventional methods 79 .”
VoxelMorph: A Learning Framework for Deformable Medical Image Registration
Balakrishnan, G., Zhao, A., Sabuncu, M. R., Guttag, J. V., & Dalca, A. V. (2019) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2019.2897538 · Cited by 2,334
“Most of these rely on ground truth warp fields [35]–[39], which are either obtained by simulating deformations and deformed images, or running classical registration methods on pairs of scans.”
Background
An overview of deep learning in medical imaging focusing on MRI
Lundervold, A. S., & Lundervold, A. (2019) · Zeitschrift für Medizinische Physik · doi:10.1016/j.zemedi.2018.11.002 · Cited by 2,058
“For example: deformable image registration [97,239]; model-to-image registration [240,241]; MRI-based attenuation correction for PET [242,243]; PET/MRI dose calculation [244]; unsupervised end-to-end learning for deformable registration of 2D CT/MR images [245]; an unsupervised learning model for deformable, pairwise…”
BackgroundMethod
Deep Learning Applications in Medical Image Analysis
Ker, J., Wang, L., Rao, J. P., & Lim, C. C. T. (2017) · IEEE Access · doi:10.1109/access.2017.2788044 · Cited by 1,574
“[83] stacked convolution layers in an encoder-decoder fashion, to predict how an input pixel would morph into its final configuration.”
Background
A Review of Deep Learning in Medical Imaging: Imaging Traits, Technology Trends, Case Studies With Progress Highlights, and Future Promises
Zhou, S. K., Greenspan, H. K., Davatzikos, C. A., Duncan, J. S., van Ginneken, B., Madabhushi, A., Prince, J. L., Rueckert, D., & Summers, R. M. (2021) · Proceedings of the IEEE · doi:10.1109/jproc.2021.3054390 · Cited by 1,373
“Still, some unique and varied approaches have achieved state-of-the-art results with relatively fast run times [132]–[135].”
Background
Recent advances and clinical applications of deep learning in medical image analysis
Chen, X., Wang, X., Zhang, K., Fung, K.-M. A., Thai, T., Moore, K. N., Mannel, R. S., Liu, H., Zheng, B., & Qiu, Y. (2022) · Medical Image Analysis · doi:10.1016/j.media.2022.102444 · Cited by 1,180
“Aside from learning a similarity, deep learning can also learn parameters of classical registration frameworks, such as the Large Diffeomorphic Distance Metric Mapping (LDDMM) model (Yang et al., 2017).”
An overview of deep learning in medical imaging focusing on MRI
Lundervold, A. S., & Lundervold, A. (2018) · NORA - Norwegian Open Research Archives · Cited by 1,012
Image Matching from Handcrafted to Deep Features: A Survey
Ma, J.-Y., Jiang, X., Fan, A., Jiang, J., & Yan, J. (2020) · International Journal of Computer Vision · doi:10.1007/s11263-020-01359-2 · Cited by 977
“Some efforts (Paul and Pati 2016; Ma et al. 2017b; Yang et al. 2017a) also attempted to modify feature detectors and descriptors to improve the registration process.”
BackgroundMethod
A deep learning framework for unsupervised affine and deformable image registration
de Vos, B. D., Berendsen, F. F., Viergever, M. A., Sokooti, H., Staring, M., & Išgum, I. (2019) · Medical Image Analysis · doi:10.1016/j.media.2018.11.010 · Cited by 818
“ge registration (Cao et al., 2017; de Vos et al., 2017; Eppenhof et al., 2018; Jaderberg et al., 2015; Krebs et al., 2017; Liao et al., 2017; Miao et al., 2016; Sokooti et al., 2017; Wu et al., 2016; Yang et al., 2017). Deep learning techniques are well suited for image registration, because they automatically learn t…”
BackgroundMethodInfluential
TransMorph: Transformer for unsupervised medical image registration
Chen, J., Frey, E. C., He, Y., Segars, W. P., Li, Y., & Du, Y. (2022) · Medical Image Analysis · doi:10.1016/j.media.2022.102615 · Cited by 883
“Cui et al. (2021) and Yang et al. (2017b) incorporated MC dropout layers in their registration network designs, allowing for the estimation of transformation uncertainty by sampling multiple deformation field predictions from the network.”
Influential
Deep learning in medical imaging and radiation therapy
Sahiner, B., Pezeshk, A. X., Hadjiiski, L. M., Wang, X., Drukker, K., Cha, K. H., Summers, R. M., & Giger, M. L. (2018) · Medical Physics · doi:10.1002/mp.13264 · Cited by 763
“322 Yang et al. 240 created a synthetic CT dataset through the use of the Radon transform for a known object and modeled different exposure conditions through adding noise to the data, for training a CNN to estimate high-dose projections from low-dose ones.”
MethodInfluential
VoxelMorph: A Learning Framework for Deformable Medical Image RegistrationPreprint
Balakrishnan, G., Zhao, A. T., Sabuncu, M. R., Guttag, J. V., & Dalca, A. V. (2019) · DSpace@MIT (Massachusetts Institute of Technology) · Cited by 728
Solving inverse problems using data-driven models
Arridge, S., Maaß, P., Öktem, O., & Schönlieb, C. (2019) · Acta Numerica · doi:10.1017/s0962492919000059 · Cited by 767
An Unsupervised Learning Model for Deformable Medical Image RegistrationPreprint
Balakrishnan, G., Zhao, A. T., Sabuncu, M. R., Guttag, J. V., & Dalca, A. V. (2018) · DSpace@MIT (Massachusetts Institute of Technology) · Cited by 673
Deep learning in medical image registration: a review
Fu, Y., Lei, Y., Wang, T., Curran, W. J., Liu, T., & Yang, X. (2020) · Physics in Medicine & Biology · doi:10.1088/1361-6560/ab843e · Cited by 661
“[175] Brain 3D-3D Yes MR Deformable [10] Brain 3D-3D Yes MR Deformable”
Self-citation
Image Synthesis in Multi-Contrast MRI With Conditional Generative Adversarial Networks
Dar, S. U. H., Yurt, M., Karacan, L., Erdem, A., Erdem, E., & Çukur, T. (2019) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2019.2901750 · Cited by 635
“first be estimated via multi-modal registration [75].”
Secure and robust machine learning for healthcare: A survey
Qayyum, A., Qadir, J., Bilal, M., & Al-Fuqaha, A. (2021) · IEEE Reviews in Biomedical Engineering · doi:10.1109/rbme.2020.3013489 · Cited by 59
“In [58], a framework for deformable image registration named”
Background
An Unsupervised Learning Model for Deformable Medical Image Registration
Balakrishnan, G., Zhao, A., Sabuncu, M. R., Dalca, A. V., & Guttag, J. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00964 · Cited by 599
“Most of these rely on ground truth warp fields or segmentations [26, 35, 39, 45], a significant drawback compared to our method, which does not require either.”
Method
Deep learning for lung cancer prognostication: A retrospective multi-cohort radiomics study
Hosny, A., Parmar, C., Coroller, T. P., Grossmann, P., Zeleznik, R., Kumar, A., Bussink, J., Gillies, R. J., Mak, R. H., & Aerts, H. J. W. L. (2018) · PLOS Medicine · doi:10.1371/journal.pmed.1002711 · Cited by 579
“Deep learning methods have outperformed their engineered feature counterparts in many tasks including mammographic lesion detection [21], mortality prediction [22], and multimodal image registration [23].”
Method
A gentle introduction to deep learning in medical image processing
Maier, A., Syben, C., Lasser, T., & Rieß, C. (2019) · Zeitschrift für Medizinische Physik · doi:10.1016/j.zemedi.2018.12.003 · Cited by 536
“The quicksilver algorithm is able to model a deformable registration and uses a patch-wise prediction directly from the image appearance [83].”
Method
Weakly-supervised convolutional neural networks for multimodal image registration
Hu, Y., Marc Modat, A., Gibson, E., Li, W., Ghavami, N., Bonmati, E., Wang, G., Bandula, S., Moore, C. M., Emberton, M., et al. (2018) · Medical Image Analysis · doi:10.1016/j.media.2018.07.002 · Cited by 476
Unsupervised Learning of Probabilistic Diffeomorphic Registration for Images and Surfaces
Dalca, A. V., Balakrishnan, G., Guttag, J., & Sabuncu, M. R. (2019) · Medical Image Analysis · doi:10.1016/j.media.2019.07.006 · Cited by 467
“Most earlier approaches demonstrated the feasibility of deep learning based registration, and required ground truth registration fields [13, 42, 59, 64, 68].”
Background
Data Augmentation Using Learned Transformations for One-Shot Medical Image Segmentation
Zhao, A. T., Balakrishnan, G., Durand, F., Guttag, J. V., & Dalca, A. V. (2019) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2019.00874 · Cited by 493
“This mature field spans optimization-based methods [4, 7, 67, 70], and recent learning-based methods [8, 9, 20, 42, 62, 72, 80].”
Method
Unsupervised Learning for Fast Probabilistic Diffeomorphic Registration
Dalca, A. V., Balakrishnan, G., Guttag, J. V., & Sabuncu, M. R. (2018) · Lecture notes in computer science · doi:10.1007/978-3-030-00928-1_82 · Cited by 418
“which can introduce a bias and necessitate significant preprocessing [23,26,28].”
Background
CycleMorph: Cycle consistent unsupervised deformable image registration
Kim, B., Kim, D. H., Park, S. H., Kim, J., Lee, J., & Ye, J. C. (2021) · Medical Image Analysis · doi:10.1016/j.media.2021.102036 · Cited by 346
“1) Supervised Learning Methods: In supervised learning approaches, the ground-truth of deformation vector ﬁelds are required, which are typically generated by the classical registration methods [12]–[14], [25], [26].”
BackgroundMethod
Different scaling of linear models and deep learning in UKBiobank brain images versus machine-learning datasets
Schulz, M.-A., Yeo, B. T. T., Vogelstein, J. T., Mourao-Miranada, J., Kather, J. N., Kording, K., Richards, B., & Bzdok, D. (2020) · Nature Communications · doi:10.1038/s41467-020-18037-z · Cited by 315
“However, a few successful exceptions have shown advantages of deep learning applied to neuroimaging data over conventional approaches, such as for the speci ﬁ c goals of image segmentation 18,34 – 37 and image registration 38,39 .”
Background
Recursive Cascaded Networks for Unsupervised Medical Image Registration
Zhao, S., Dong, Y., Chang, E., & Xu, Y. L. (2019) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.01070 · Cited by 338
“Some initial works rely on the dense ground-truth flows obtained by either traditional algorithms [14, 56] or simulating intrasubject deformations [36, 53], but their performance is restricted due to the limited quality of training data.”
BackgroundMethod
Birnet: Brain image registration using dual-supervised fully convolutional networks
Fan, J., Cao, X., Yap, P.-T., & Shen, D. (2019) · Medical Image Analysis · doi:10.1016/j.media.2019.03.006 · Cited by 297
“Moreover, a significant amount of effort is often needed to redesign and port algorithms for GPUs (Fluck et al., 2011).”
BackgroundInfluential
Going Deep in Medical Image Analysis: Concepts, Methods, Challenges, and Future Directions
Altaf, F., Islam, S. M. S., Akhtar, N., & Janjua, N. K. (2019) · IEEE Access · doi:10.1109/access.2019.2929365 · Cited by 301
“Advent of deep learning has also caused neural networks to penetrate in medical image registration [187], [188], [189], [190].”
Background
Deep neural networks and kernel regression achieve comparable accuracies for functional connectivity prediction of behavior and demographics
He, T., Kong, R., Holmes, A. J., Nguyen, M., Sabuncu, M. R., Eickhoff, S. B., Bzdok, D., Feng, J., & Yeo, B. T. T. (2020) · NeuroImage · doi:10.1016/j.neuroimage.2019.116276 · Cited by 271
“…2018; X. Zhao et al., 2018), image modality/quality transfer (Bahrami et al., 2016; Nie et al., 2017; Blumberg et al., 2018), image registration (Yang et al., 2017; Dalca et al., 2018), as well as behavioral and disease prediction (Plis et al., 2014; van der Burgh et al., 2017; Vieira et al.,…”
Background
Fast Symmetric Diffeomorphic Image Registration with Convolutional Neural Networks
Mok, T. C. W., & Chung, A. C. S. (2020) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr42600.2020.00470 · Cited by 278
“Existing CNN-based methods often ignore desirable diffeomorphic properties, including topology preservation, invertibility and inverse consistency of the transformation [7, 24, 8, 32, 11, 5].”
Learning a Probabilistic Model for Diffeomorphic Registration
Krebs, J., Delingette, H., Mailhé, B., Ayache, N., & Mansi, T. (2019) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2019.2897112 · Cited by 230
“In contrast to other probabilistic approaches such as [15], [27], we learn a low-dimensional global latent space in an encoder-decoder neural network where the deformation of a new image pair is mapped to and where similar deformations are close to each other.”
BackgroundMethod
Radiological images and machine learning: Trends, perspectives, and prospects
Zhang, Z., & Sejdić, E. (2019) · Computers in Biology and Medicine · doi:10.1016/j.compbiomed.2019.02.017 · Cited by 249
“The primary challenge remains to ﬁnd the appropriate way to match the correspondences among diﬀerent images modalities [218].”
Background
A review of multimodal medical image fusion techniques
Huang, B., Yang, F., Yin, M., Mo, X., & Zhong, C. (2020) · Computational and Mathematical Methods in Medicine · doi:10.1155/2020/8279342 · Cited by 220
SynthMorph: Learning Contrast-Invariant Registration Without Acquired Images
Hoffmann, M., Billot, B., Greve, D. N., Iglesias, J. E., Fischl, B. R., & Dalca, A. V. (2021) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2021.3116879 · Cited by 223
“ulated warps or deformation fields estimated by classical methods [21], [22], [24], [42]–[44].”
Method
TransMatch: A Transformer-Based Multilevel Dual-Stream Feature Matching Network for Unsupervised Deformable Image Registration
Chen, Z., Zheng, Y., & Gee, J. C. (2023) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2023.3288136 · Cited by 195
“The training of the related deep networks is driven either by minimizing the differences between the predicted displacement and the ground truth [22] or by penalizing image dissimilarity via a dissimilarity measurement exploited in traditional registration Fig.”
Method
Dual-stream pyramid registration network
Kang, M., Hu, X., Huang, W., Scott, M. R., & Reyes, M. (2022) · Medical Image Analysis · doi:10.1016/j.media.2022.102379 · Cited by 197
Non-rigid image registration using self-supervised fully convolutional networks without training data
Li, H., & Yong Fan, A. (2018) · IEEE International Symposium on Biomedical Imaging (ISBI) · doi:10.1109/isbi.2018.8363757 · Cited by 194
“A variety of strategies have been proposed to build training data, specifically the spatial transformations [1-3].”
Method
Medical image registration using deep neural networks: A comprehensive review
Boveiri, H. R., Khayami, R., Javidan, R., & Mehdizadeh, A. (2020) · Computers & Electrical Engineering · doi:10.1016/j.compeleceng.2020.106767 · Cited by 179
“CNN (FlowNet) (Dosovitskiy et al. 2015) LONI LBPA40 (Brain MRI) Jaccard and ASCD Directly regressing the 2D unimodal deformable registration parameters using a weakly-supervised CNN Private - cine Cardiac MRI Yang et al. (2017) X. Yang, R. Kwitt, M. Styner, and M. Niethammer, “Quicksilver: Fast predictive image regist…”
Method
Pulmonary CT Registration Through Supervised Learning With Convolutional Neural Networks
Eppenhof, K. A. J., & Pluim, J. P. W. (2018) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2018.2878316 · Cited by 173
“[6] proposed a supervised method that estimates the momentum parameterization for LDDMM shooting from image patches, which was demonstrated for registration of brain MR images.”
BackgroundMethod
mustGAN: multi-stream Generative Adversarial Networks for MR Image Synthesis
Yurt, M., Dar, S. U. H., Erdem, A., Erdem, E., Oguz, K. K., & Çukur, T. (2021) · Medical Image Analysis · doi:10.1016/j.media.2020.101944 · Cited by 187
“When an end-to-end network alternative is desired, deep-network-based registration models ( Yang et al., 2017 ) can instead be cascaded to the input of mustGAN to spatially align source-target images.”
Adversarial learning for mono- or multi-modal registration
Fan, J., Cao, X., Wang, Q., Yap, P., & Shen, D. G. (2019) · Medical Image Analysis · doi:10.1016/j.media.2019.101545 · Cited by 174
“Registration Network—A number of registration networks (Yang et al., 2017; Fan et al., 2019; Cao et al., 2018b; Balakrishnan et al., 2018) can be used in the proposed adversarial framework.”
BackgroundMethodInfluential
Applications and limitations of machine learning in radiation oncology
Jarrett, D. P., Stride, E., Vallis, K. A., & Gooding, M. J. (2019) · British Journal of Radiology · doi:10.1259/bjr.20190001 · Cited by 167
“39 In fact, learning the transformation function directly obviates the need for iterative optimization— the procedure simply becomes a much more efficient matter of applying a function.”
Background
Applications of Deep Learning to Neuro-Imaging Techniques
Zhu, G., Jiang, B., Tong, L., Xie, Y., Zaharchuk, G., & Wintermark, M. (2019) · Frontiers in Neurology · doi:10.3389/fneur.2019.00869 · Cited by 162
“Other studies also include fast predictive image registration with deep encoderdecoder network based on a Large Deformation Diffeomorphic Metric Mapping model (100).”
Method
A survey on deep learning in medical image registration: New technologies, uncertainty, evaluation metrics, and beyond
Chen, J., Liu, Y., Wei, S., Bian, Z., Subramanian, S., Carass, A., Prince, J. L., & Du, Y. (2024) · Medical Image Analysis · doi:10.1016/j.media.2024.103385 · Cited by 209
DeepAtlas: Joint Semi-supervised Learning of Image Registration and Segmentation
Xu, Z., & Niethammer, M. (2019) · Lecture notes in computer science · doi:10.1007/978-3-030-32245-8_47 · Cited by 160
“training either uses estimates from optimization-based methods [10] or is unsupervised [2].”
MethodSelf-citation
Artificial intelligence in diagnostic and interventional radiology: Where are we now?
Boeken, T., Feydy, J., Lecler, A., Soyer, P. A., Feydy, A., Barat, M., & Duron, L. (2022) · Diagnostic and Interventional Imaging · doi:10.1016/j.diii.2022.11.004 · Cited by 152
Registration of Multimodal Remote Sensing Image Based on Deep Fully Convolutional Neural Network
Zhang, H., Ni, W., Yan, W., Xiang, D., Wu, J., Yang, X., & Hui, B. (2019) · IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing · doi:10.1109/jstars.2019.2916560 · Cited by 145
“for patch-wise prediction of a deformation model is constructed in [42], where a sliding window is used to predict the LDDMM momentum locally from multimodal medical image patches.”
MethodSelf-citation
Recent Advancements and Future Prospects in Active Deep Learning for Medical Image Segmentation and Classification
Mahmood, T., Rehman, A. U., Saba, T., Nadeem, L., & Bahaj, S. A. (2023) · IEEE Access · doi:10.1109/access.2023.3313977 · Cited by 160
“Recently, Yang et al. [107] utilized the U-Net network algorithm as the baseline structure and the differential isochronous embryo algorithm to obtain the deformation field, which was then used as a label for end-to-end registration of 2D and 3D brain MR images.”
Method
Adversarial Similarity Network for Evaluating Image Alignment in Deep Learning Based Registration
Fan, J., Cao, X., Xue, Z., Yap, P., & Shen, D. G. (2018) · Lecture notes in computer science · doi:10.1007/978-3-030-00928-1_83 · Cited by 132
“Yang et al. [2] predicted the momenta in LDDMM. Rohe et al. [3] built reference deformations for training by registering manually delineated regions of interests (ROIs).”
Background
Deformable Image Registration Using a Cue-Aware Deep Regression Network
Cao, X., Yang, J., Zhang, J., Wang, Q., Yap, P., & Shen, D. G. (2018) · IEEE Transactions on Biomedical Engineering · doi:10.1109/tbme.2018.2822826 · Cited by 123
“Speciﬁ-cally, fully convolutional network (FCN) has been employed in [10] and [51] to learn the deformation momenta and stationary velocity ﬁeld (SVF), respectively.”
BackgroundMethod
HyperMorph: Amortized Hyperparameter Learning for Image Registration
Hoopes, A., Hoffmann, M., Fischl, B. R., Guttag, J. V., & Dalca, A. V. (2021) · Lecture notes in computer science · doi:10.1007/978-3-030-78191-0_1 · Cited by 155
“Learningbased registration methods employ image datasets to learn a function that rapidly computes the deformation field between image pairs [7,20,48,52,56,58,59].”
Label-driven weakly-supervised learning for multimodal deformarle image registration
Hu, Y., Marc Modat, A., Gibson, E., Ghavami, N., Bonmati, E., Moore, C. M., Emberton, M., Noble, J. A., Barratt, D. C., & Vercauteren, T. K. (2018) · IEEE International Symposium on Biomedical Imaging (ISBI) · doi:10.1109/isbi.2018.8363756 · Cited by 148
“[8, 9], or to provide better constrained transformation models [10].”
Background
Networks for Joint Affine and Non-Parametric Image Registration
Shen, Z., Xu, H., Xu, Z., & Niethammer, M. (2019) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2019.00435 · Cited by 130
“For example, In Quicksilver [33], a network is learned to predict the initial momentum of LDDMM, which can then be used to ﬁnd a diffeomorphic spatial transformation via LDDMM’s shooting equations.”
MethodSelf-citation
TeTrIS: Template Transformer Networks for Image Segmentation With Shape Priors
Lee, M. C. H., Petersen, K., Pawlowski, N., Glocker, B., & Schaap, M. (2019) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2019.2905990 · Cited by 118
“For example, [28] combine momentum-parametrization for LDDMM shooting [29] and neural networks to learn an end-to-end model for registration.”
Method
Real-time Deep Pose Estimation with Geodesic Loss for Image-to-Template Rigid Registration
Salehi, S. S. M., Khan, S., Erdogmus, D., & Gholipour, A. (2018) · arXiv · doi:10.48550/arxiv.1803.05982 · Cited by 118
Real-Time Deep Pose Estimation With Geodesic Loss for Image-to-Template Rigid Registration
Salehi, S. S. M., Khan, S., Deniz Erdogmus, B., & Gholipour, A. (2018) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2018.2866442 · Cited by 133
“[13] developed a deep autoencoder-decoder convolutional neural network (CNN) that learned to predict the Large Deformation Diffeomorphic Metric Mapping (LDDMM) model, and achieved state-of-the-art performance with an order of magnitude faster optimization in inter-subject and subject-to-atlas deformable registration.”
Method
One-Shot Learning for Deformable Medical Image Registration and Periodic Motion Tracking
Fechter, T., & Baltas, D. (2020) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2020.2972616 · Cited by 116
“Other supervised methods were presented by [9] who showed a network for predicting the momentum of large deformation diffeomorphic metric mapping model, [10] who presented a network for the registration of magnetic resonance images ( MRI ) of the brain and [11], [12] showed a method for pulmonary…”
BackgroundMethod
SDFN: Segmentation-based deep fusion network for thoracic disease classification in chest X-ray images
Liu, H., Wang, L., Nan, Y., Jin, F., Wang, Q., & Pu, J. (2019) · Computerized Medical Imaging and Graphics · doi:10.1016/j.compmedimag.2019.05.005 · Cited by 135
“cn) demonstrated remarkable strength in a variety tasks of medical image analyses, such as disease classification [2-5], lesion segmentation [6-8], and registration [9-10].”
Background
A Review on Machine Learning Strategies for Real-World Engineering Applications
Jhaveri, R. H., Revathi, A. R., Ramana, K., Raut, R., & Dhanaraj, R. K. (2022) · Mobile Information Systems · doi:10.1155/2022/1833507 · Cited by 134
Inverse-Consistent Deep Networks for Unsupervised Deformable Image RegistrationPreprint
Zhang, J. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1809.03443 · Cited by 122
“Yang et al. [55] developed a fully convolutional network (FCN) to predict 3D deformable registration, followed by a correction network to further reﬁne the predicted transformation.”
Background
VoteNet: A Deep Learning Label Fusion Method for Multi-atlas Segmentation
Ding, Z., Han, X., & Niethammer, M. (2019) · Lecture notes in computer science · doi:10.1007/978-3-030-32248-9_23 · Cited by 114
“and a warped atlas image. Output is a binary mask (after thresholding the sigmoid output at 0.5) to select trustworthy atlas voxels. speeds-up pairwise registrations by at least an order of magnitude [16] compared to numerical optimization. Since registrations are the computational bottleneck of MAS, similar speed-ups…”
MethodInfluentialSelf-citation
A deep learning framework for pancreas segmentation with multi-atlas registration and 3D level-set
Zhang, Y., Wu, J., Liu, Y., Chen, Y., Chen, W., Wu, E. X., Li, C., & Tang, X. (2020) · Medical Image Analysis · doi:10.1016/j.media.2020.101884 · Cited by 110
Demystifying Brain Tumor Segmentation Networks: Interpretability and Uncertainty Analysis
Natekar, P., Kori, A., & Krishnamurthi, G. (2020) · Frontiers in Computational Neuroscience · doi:10.3389/fncom.2020.00006 · Cited by 103
The Impact of Machine Learning on 2D/3D Registration for Image-Guided Interventions: A Systematic Review and Perspective
Unberath, M., Gao, C., Hu, Y., Judish, M., Taylor, R. H., Armand, M., & Grupp, R. (2021) · Frontiers in Robotics and AI · doi:10.3389/frobt.2021.716007 · Cited by 102
“Some of these approaches report interval estimates, either by reformulating the registration objective function as a probability distribution and drawing samples [113, 114, 115], approximating the sampling process using test-time NN drop-out [116] or sampling using the test-time deformation covariance matrices embedde…”
Method
A holistic overview of deep learning approach in medical imaging
Yousef, R., Gupta, G., Yousef, N., & Khari, M. (2022) · Multimedia Systems · doi:10.1007/s00530-021-00884-5 · Cited by 112
“Yang et al. [177] have used an autoencoder architecture for predicting of deformation diffeomorphic metrics mapping (LDDMM) to get fast deformable image registration and the results shows improvements in computational time.”
Method
Artificial intelligence performance in detecting tumor metastasis from medical radiology imaging: A systematic review and meta-analysis
Zheng, Q., Yang, L., Zeng, B., Li, J., Guo, K., Liang, Y., & Liao, G. (2020) · EClinicalMedicine · doi:10.1016/j.eclinm.2020.100669 · Cited by 101
Deep Learning in Medical Image Registration
Chen, X., Diaz-Pinto, A., Ravikumar, N., & Frangi, A. (2020) · Progress in Biomedical Engineering · doi:10.1088/2516-1091/abd37c · Cited by 106
MI-UNet: Multi-Inputs UNet Incorporating Brain Parcellation for Stroke Lesion Segmentation From T1-Weighted Magnetic Resonance Images
Zhang, Y., Wu, J., Liu, Y., Chen, Y.-F., Wu, E., & Tang, X.-Y. (2020) · IEEE journal of biomedical and health informatics · doi:10.1109/jbhi.2020.2996783 · Cited by 65
“Deep learning has been showing promising performance in registration and is very fast in the testing stage [48].”
Background
ProsRegNet: A deep learning framework for registration of MRI and histopathology images of the prostate
Shao, W., Banh, L., Kunder, C. A., Fan, R. E., Soerensen, S. J. C., Wang, J. B., Teslovich, N. C., Madhuripan, N., Jawahar, A., Ghanouni, P., et al. (2020) · Medical Image Analysis · doi:10.1016/j.media.2020.101919 · Cited by 94
A multi-scale framework with unsupervised joint training of convolutional neural networks for pulmonary deformable image registration
Jiang, Z., Yin, F.-F., Ge, Y., & Ren, L. (2019) · Physics in Medicine and Biology · doi:10.1088/1361-6560/ab5da0 · Cited by 94
“Those methods can be divided into two categories: (1) integrated deep learning (Wu et al 2016, Yang et al 2017, Kearney et al 2018), and (2) end-to-end deep learning (Cao et al 2017, Eppenhof and Pluim 2017, Sokooti et al 2017, Shan et al 2017, Balakrishnan et al 2018, de Vos et al 2019, Eppenhof…”
Method
Learning deformable registration of medical images with anatomical constraints
Mansilla, L., Milone, D. H., & Ferrante, E. (2020) · Neural Networks · doi:10.1016/j.neunet.2020.01.023 · Cited by 94
“During the last years, we have witnessed the advent of deep learning-based image registration meth-ods [25, 49, 35, 39, 46, 45, 1, 6, 11], which achieve state-of-the-art performance, and drastically reduce the required computational time.”
BackgroundMethod
A review of deep learning-based deformable medical image registration
Zou, J., Gao, B., Song, Y., & Qin, J. (2022) · Frontiers in Oncology · doi:10.3389/fonc.2022.1047215 · Cited by 91
“(82) Real DVF Brain MRI 3D CNN Deformation error, [J] NeuroImage 2017”
Method
Machine learning based liver disease diagnosis: A systematic review
Khan, R. A., Luo, Y., & Wu, F. (2021) · Neurocomputing · doi:10.1016/j.neucom.2021.08.138 · Cited by 92
CNN-based lung CT registration with multiple anatomical constraints
Hering, A., Häger, S., Moltz, J. H., Leßmann, N., Heldmann, S., & van Ginneken, B. (2021) · Medical Image Analysis · doi:10.1016/j.media.2021.102139 · Cited by 89
“Alterna-tively, conventional registration methods can be used to produce deformations by registering images (Sentker et al., 2018; Yang et al., 2017a) or other image features like landmarks or segmentations (Roh´e et al., 2017).”
Conditional Deformable Image Registration with Convolutional Neural Network
Mok, T. C. W., & Chung, A. C. S. (2021) · Lecture notes in computer science · doi:10.1007/978-3-030-87202-1_4 · Cited by 103
“Recent deep learning-based image registration (DLIR) methods [31,17,30,3,5] have achieved remarkable re-sults and showed immense potential for time-sensitive medical imaging studies such as image-guided surgery and motion tracking.”
Background
DeepFLASH: An Efficient Network for Learning-Based Medical Image Registration
Wang, J., & Zhang, M. (2020) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr42600.2020.00450 · Cited by 89
“For 3D testing, we compare our method with three baseline algorithms, including FLASH [31] (a fast optimization-based image registration method), Voxelmorph [6] (an unsupervised registration in image space) and Quicksilver [28] (a supervised method that predicts transformations in a momentum space).”
BackgroundMethodInfluential
Metric Learning for Image Registration
Niethammer, M., Kwitt, R., & Vialard, F. (2019) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2019.00866 · Cited by 81
“While we will not explore regressing the momentum parameterization here, such a formulation is expected to be straightforward, as our proposed model has a momentum-8456 parameterization similar to what has already been used successfully for regression with a deep network [48].”
BackgroundMethodInfluentialSelf-citation
Closing the Gap Between Deep and Conventional Image Registration Using Probabilistic Dense Displacement Networks
Heinrich‬, M. P. (2019) · Lecture notes in computer science · doi:10.1007/978-3-030-32226-7_6 · Cited by 94
“ponding anatomies onto one another, the problem is much less constrained than image segmentation and therefore remains an open challenge. A number of approaches has been applied to brain registration [1,17], which usually deals with localised deformations of few millimetres and for which huge labelled datasets (˛100 s…”
Background
A fully convolutional neural network for new T2-w lesion detection in multiple sclerosis
Salem, M., Valverde, S., Cabezas, M., Pareto, D., Oliver, A., Salví, J., Rovira, À., & Lladó, X. (2019) · NeuroImage Clinical · doi:10.1016/j.nicl.2019.102149 · Cited by 80
DiffuseMorph: Unsupervised Deformable Image Registration Using Diffusion Model
Kim, B., Han, I., & Ye, J. C. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-19821-2_20 · Cited by 102
“However, supervised learning methods train the networks using the ground-truth registration fields [7,8,34,47], which needs high-quality labels for training.”
Method
SymReg-GAN: Symmetric Image Registration with Generative Adversarial Networks
Zheng, Y., Sui, X., Jiang, Y., Che, T., Zhang, S., Yang, J., & Li, H. (2021) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2021.3083543 · Cited by 80
Learning Conditional Deformable Templates with Convolutional NetworksPreprint
Dalca, A. V., Rakic, M., Guttag, J. V., et al. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1908.02738 · Cited by 79
“s require an optimization for every pair of images, leading to long runtimes in practice. Recently, learning based registration methods have been proposed that offer a signiﬁcant speedup at test time [8, 9, 12, 17, 18, 23, 47, 46, 61, 64, 70]. These methods learn a network that computes the deformation ﬁeld, either in…”
Method
An Attention-Based Convolutional Neural Network for Acute Lymphoblastic Leukemia Classification
Ullah, M. Z., Zheng, Y., Song, J., et al. (2021) · Applied Sciences · doi:10.3390/app112210662 · Cited by 77
“Deep neural networks are widely utilized in the ﬁeld of computer vision, especially for medical image analysis, such as disease classiﬁcation [23], localization [24], detection [25], registration [26], and segmentation [27,28].”
Method
Prospects of deep learning for medical imaging
Kim, J., Hong, J., & Park, H. (2018) · Precision and Future Medicine · doi:10.23838/pfm.2018.00030 · Cited by 82
A review of deep learning-based three-dimensional medical image registration methods
Xiao, H., Teng, X., Liu, C., Li, T., Ren, G., Yang, R., Shen, D., & Cai, J. (2021) · Quantitative Imaging in Medicine and Surgery · doi:10.21037/qims-21-175 · Cited by 77
“used a U-Net-like CNN model to learn DVFs of the same resolution as that of input brain MR images, and achieved high registration accuracy across multiple datasets (89).”
BackgroundMethod
Non-rigid image registration using fully convolutional networks with deep self-supervisionPreprint
Li, H., & Yong Fan, A. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1709.00799 · Cited by 70
“Spatial transformations that register pairs of images can also be estimated using conventional image registration algorithms [8, 9].”
Background
An overview of deep learning methods for image registration with focus on feature-based approaches
Kuppala, K., Sandhya, B., & Barige, T. R. (2020) · International Journal of Image and Data Fusion · doi:10.1080/19479832.2019.1707720 · Cited by 74
“To improve the run time performance of the CNN approaches, auto-encoders have been used (Yang et al .”
Method
Progressively Trained Convolutional Neural Networks for Deformable Image Registration
Eppenhof, K. A. J., Lafarge, M. W., Veta, M., & Pluim, J. P. W. (2019) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2019.2953788 · Cited by 74
“Recent papers have applied deep learning to a broad spectrum of deformable registration applications, including brain MRIs [4], [6], [13], [17], [19], cardiac MRIs [5], [14], prostate MRIs and ultrasound images [10], and chest CTs [3], [5], [7], [8], [15], [18], [11].”
BackgroundMethod
Image-and-Spatial Transformer Networks for Structure-Guided Image Registration
Lee, M. C. H., Oktay, O., Schuh, A., Schaap, M., & Glocker, B. (2019) · Lecture notes in computer science · doi:10.1007/978-3-030-32245-8_38 · Cited by 87
Unsupervised Deformable Image Registration Using Cycle-Consistent CNN
Kim, B., Kim, J., Lee, J., Kim, D. H., Park, S. H., & Ye, J. C. (2019) · Lecture notes in computer science · doi:10.1007/978-3-030-32226-7_19 · Cited by 82
Robust Feature Matching for Remote Sensing Image Registration via Linear Adaptive Filtering
Jiang, X., Ma, J., Fan, A., Xu, H., Lin, G., Lü, T., & Tian, X. S. (2020) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2020.3001089 · Cited by 68
“…two images to drive an iterative optimization strategy [78]; 2) to directly predict transformation parameters using deep regression networks [79], [80]; and 3) learning to substitute one or more processes of traditional feature-based methods such as SIFT, e.g., key points detection and…”
Method
Model-Based and Data-Driven Strategies in Medical Image Computing
Rueckert, D., & Schnabel, J. A. (2019) · Proceedings of the IEEE · doi:10.1109/jproc.2019.2943836 · Cited by 76
“Broadly following these optical flow approaches, the Quicksilver method [119] is based on LDDMM [14], involving a deep regression model for predicting transformation parameters φ using image appearances, starting from LDDMM estimations for image patches as an initialization.”
Method
SDCT-AuxNet : DCT augmented stain deconvolutional CNN with auxiliary classifier for cancer diagnosis
Gehlot, S., Gupta, A., & Gupta, R. (2020) · Medical Image Analysis · doi:10.1016/j.media.2020.101661 · Cited by 65
“In medical applications, CNNs are frequently used for clas- sification (Rajpurkar et al., 2017), localization (Yan et al., 2015), detection (Sermanet et al., 2013), segmentation (Ronneberger et al., 2015), and registration (Yang et al., 2017).”
Method
Deformable MR-CT image registration using an unsupervised, dual-channel network for neurosurgical guidance
Han, R., Jones, C. K., Lee, J., Wu, P., Vagdargi, P., Uneri, A., Helm, P. A., Luciano, M., Anderson, W. S., & Siewerdsen, J. H. (2021) · Medical Image Analysis · doi:10.1016/j.media.2021.102292 · Cited by 66
A method of using deep learning to predict three‐dimensional dose distributions for intensity‐modulated radiotherapy of rectal cancer
Zhou, J., Peng, Z. M., Song, Y., Chang, Y., Pei, X., Sheng, L., & Xu, X. G. (2020) · Journal of Applied Clinical Medical Physics · doi:10.1002/acm2.12849 · Cited by 63
Artificial Intelligence in Radiation Therapy
Fu, Y., Zhang, H., Morris, E. D., Glide‐Hurst, C. K., Pai, S., Traverso, A., Wee, L., Hadžić, I., Lønne, P.-I., Shen, C., et al. (2021) · IEEE Transactions on Radiation and Plasma Medical Sciences · doi:10.1109/trpms.2021.3107454 · Cited by 75
Self-citation
Role of deep learning in infant brain MRI analysis
Mostapha, M., & Styner, M. A. (2019) · Magnetic Resonance Imaging · doi:10.1016/j.mri.2019.06.009 · Cited by 63
Self-citation
Convolutional Neural Networks with Template-Based Data Augmentation for Functional Lung Image Quantification
Tustison, N. J., Avants, B., Lin, Z., Feng, X., Cullen, N. C., Mata, J. F., Flors, L., Gee, J. C., Altes, T. A., Mugler, J. P., et al. (2018) · Academic Radiology · doi:10.1016/j.acra.2018.08.003 · Cited by 62

Authors: Yang, X., Kwitt, R., Styner, M., & Niethammer, M.
Affiliations: University of North Carolina at Chapel Hill (US)
University of Salzburg (AT)
Corresponding author: Yang, Xiao
ORCID: Yang, Xiao
Kwitt, Roland
Styner, Martin
Niethammer, Marc
Venue: NeuroImage
Publisher: Elsevier BV
Published: 2017-09
Volume: 158
Pages: 378-396
ISSN: 1053-8119 (ISSN-L)
DOI: 10.1016/j.neuroimage.2017.07.008
PMC: PMC6036629
OpenAlex: W2604920239
Cited by: 630
References: 94
Citations by year: 2026: 17 · 2025: 53 · 2024: 74 · 2023: 65 · 2022: 101 · 2021: 85 · 2020: 95 · 2019: 93 · 2018: 40 · 2017: 7
FWCI: 26.68
Type: journal article
Language: en
Open access: bronze
doi.org
Licence: www.elsevier.com/tdm/userlicense/1.0/ (tdm)
www.elsevier.com/legal/tdmrep-license (tdm)
www.elsevier.com/open-access/userlicense/1.0/ (am, 320 d)
doi.org/10.15223/policy-017 (stm-asf)
doi.org/10.15223/policy-037 (stm-asf)
doi.org/10.15223/policy-012 (stm-asf)
doi.org/10.15223/policy-029 (stm-asf)
doi.org/10.15223/policy-004 (stm-asf)
Locations: NeuroImage (acceptedVersion)
PubMed (publishedVersion)
Europe PMC (PubMed Central) (acceptedVersion)
PubMed Central (acceptedVersion)
Funders: NIH (1 R41 NS091792-01, HDO55741)
NSF (ECCS-1148870, EECS-1711776)
National Science Foundation Major Research Instrumentation program (ACI-1429830)
Topics: Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Medical Image Segmentation Techniques
Physical Sciences › Computer Science › Artificial Intelligence › Domain Adaptation and Few-Shot Learning
Health Sciences › Medicine › Radiology, Nuclear Medicine and Imaging › Retinal Imaging and Analysis
Keywords: deformable image registration, LDDMM, encoder-decoder architecture, multimodal image registration, image similarity measures, diffeomorphic registration
MeSH terms: Machine Learning, Algorithms, Brain, Brain Mapping, Humans, Image Processing, Computer-Assisted, Pattern Recognition, Automated, Imaging, Three-Dimensional

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

A language model drew the claims from the paper on 2026-10-05, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).

A second model labelled each claim on 2026-10-05 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

Quicksilver: Fast predictive image registration – A deep learning approach

Abstract

Overview

AI snapshot

TL;DR

Key findings

Methods

Limitations

Integrity signals

Verification

Claims the paper makes

Claims other papers make about this one

Self-check

Citations

Cites

Cited by

Related

Metadata

Record

About these checks