The DOI matches its Crossref record; 64 of the 64 references it cites were found in a registry or an index.
VoxelMorph: A Learning Framework for Deformable Medical Image Registration
Open accessAbstract
We present VoxelMorph, a fast learning-based framework for deformable, pairwise medical image registration. Traditional registration methods optimize an objective function for each pair of images, which can be time-consuming for large datasets or rich deformation models. In contrast to this approach, and building on recent learning-based methods, we formulate registration as a function that maps an input image pair to a deformation field that aligns these images. We parameterize the function via a convolutional neural network (CNN), and optimize the parameters of the neural network on a set of images. Given a new pair of scans, VoxelMorph rapidly computes a deformation field by directly evaluating the function. In this work, we explore two different training strategies. In the first (unsupervised) setting, we train the model to maximize standard image matching objective functions that are based on the image intensities. In the second setting, we leverage auxiliary segmentations available in the training data. We demonstrate that the unsupervised model's accuracy is comparable to state-of-the-art methods, while operating orders of magnitude faster. We also show that VoxelMorph trained with auxiliary data improves registration accuracy at test time, and evaluate the effect of training set size on registration. Our method promises to speed up medical image analysis and processing pipelines, while facilitating novel directions in learning-based registration and its applications. Our code is freely available at https://github.com/voxelmorph/voxelmorph.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Image-to-Image Translation with Conditional Adversarial Networks
Isola, P., Zhu, J.-Y., Zhou, T., & Efros, A. A. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.632 · Cited by 22,930
Measures of the Amount of Ecologic Association Between Species
Dice, L. R. (1945) · Ecology · doi:10.2307/1932409 · Cited by 11,874
Horn, B. K. P., & Schunck, B. G. (1981) · Artificial Intelligence · doi:10.1016/0004-3702(81)90024-2 · Cited by 10,312
Fischl, B. (2012) · NeuroImage · doi:10.1016/j.neuroimage.2012.01.021 · Cited by 9,526
Voxel-Based Morphometry—The Methods
Ashburner, J., & Friston, K. J. (2000) · NeuroImage · doi:10.1006/nimg.2000.0582 · Cited by 8,676
A fast diffeomorphic image registration algorithm
Ashburner, J. (2007) · NeuroImage · doi:10.1016/j.neuroimage.2007.07.007 · Cited by 8,122
AVANTS, B., EPSTEIN, C., GROSSMAN, M., & GEE, J. (2008) · Medical Image Analysis · doi:10.1016/j.media.2007.06.004 · Cited by 5,669
Nonrigid registration using free-form deformations: application to breast mr images
Rueckert, D., Sonoda, L. I., Hayes, C., Hill, D. L. G., Leach, M. O., & Hawkes, D. J. (1999) · IEEE Transactions on Medical Imaging · doi:10.1109/42.796284 · Cited by 5,265
A reproducible evaluation of ANTs similarity metric performance in brain image registration
Avants, B. B., Tustison, N. J., Song, G., Cook, P. A., Klein, A., & Gee, J. C. (2011) · NeuroImage · doi:10.1016/j.neuroimage.2010.09.025 · Cited by 4,796
Flownet: Learning optical flow with convolutional networks
Dosovitskiy, A., Fischer, P., Ilg, E., Hausser, P., Hazirbas, C., Golkov, V., Smagt, P. V. D., Cremers, D., & Brox, T. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.316 · Cited by 4,313
FlowNet 2.0: Evolution of Optical Flow Estimation with Deep Networks
Ilg, E., Mayer, N., Saikia, T., Keuper, M., Dosovitskiy, A., & Brox, T. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.179 · Cited by 3,152
Di Martino, A., Yan, C.-G., Li, Q., Denio, E., Castellanos, F. X., Alaerts, K., Anderson, J. S., Assaf, M., Bookheimer, S. Y., Dapretto, M., et al. (2014) · Molecular Psychiatry · doi:10.1038/mp.2013.78 · Cited by 2,780
Alignment by Maximization of Mutual Information
Viola, P., & Wells III, W. M. (1997) · International Journal of Computer Vision · doi:10.1023/a:1007958904918 · Cited by 2,610
High Accuracy Optical Flow Estimation Based on a Theory for Warping
Brox, T., Bruhn, A., Papenberg, N., & Weickert, J. (2004) · Lecture Notes in Computer Science · doi:10.1007/978-3-540-24673-2_3 · Cited by 2,570
Image matching as a diffusion process: an analogy with maxwell’s demons
Thirion, J.-P. (1998) · Medical Image Analysis · doi:10.1016/s1361-8415(98)80022-4 · Cited by 2,445
Evaluation of 14 nonlinear deformation algorithms applied to human brain MRI registration
Klein, A., Andersson, J., Ardekani, B. A., Ashburner, J., Avants, B., Chiang, M.-C., Christensen, G. E., Collins, D. L., Gee, J., Hellier, P., et al. (2009) · NeuroImage · doi:10.1016/j.neuroimage.2008.12.037 · Cited by 2,227
Marcus, D. S., Wang, T. H., Parker, J., Csernansky, J. G., Morris, J. C., & Buckner, R. L. (2007) · Journal of Cognitive Neuroscience · doi:10.1162/jocn.2007.19.9.1498 · Cited by 2,163
The Parkinson Progression Marker Initiative (PPMI)
Marek, K., Jennings, D., Lasch, S., Siderowf, A., Tanner, C., Simuni, T., Coffey, C., Kieburtz, K., Flagg, E., Chowdhury, S., et al. (2011) · Progress in Neurobiology · doi:10.1016/j.pneurobio.2011.09.005 · Cited by 1,892
Computing Large Deformation Metric Mappings via Geodesic Flows of Diffeomorphisms
Beg, M. F., Miller, M. I., Trouvé, A., & Younes, L. (2005) · International Journal of Computer Vision · doi:10.1023/b:visi.0000043755.93987.aa · Cited by 1,719
Diffeomorphic demons: Efficient non-parametric image registration
Vercauteren, T., Pennec, X., Perchant, A., & Ayache, N. (2009) · NeuroImage · doi:10.1016/j.neuroimage.2008.10.040 · Cited by 1,592
Deformable medical image registration: a survey
Sotiras, A., Davatzikos, C., & Paragios, N. (2013) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2013.2265603 · Cited by 1,587 · cited 1×
“Most existing deformable registration algorithms iteratively optimize a transformation based on an energy function [7].”
SIFT Flow: Dense Correspondence across Scenes and Its Applications
Liu, C., Yuen, J., & Torralba, A. (2011) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2010.147 · Cited by 1,582
Secrets of optical flow estimation and their principles
Sun, D., Roth, S., & Black, M. J. (2010) · 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2010.5539939 · Cited by 1,389
Optical Flow Estimation Using a Spatial Pyramid Network
Ranjan, A., & Black, M. J. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.291 · Cited by 1,344
Large Displacement Optical Flow: Descriptor Matching in Variational Motion Estimation
Brox, T., & Malik, J. (2011) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2010.143 · Cited by 1,295
Multiresolution elastic matching
Bajcsy, R., & Kovačič, S. (1989) · Computer Vision, Graphics, and Image Processing · doi:10.1016/s0734-189x(89)80014-3 · Cited by 1,106
Fast free-form deformation using graphics processing units
Modat, M., Ridgway, G. R., Taylor, Z. A., Lehmann, M., Barnes, J., Hawkes, D. J., Fox, N. C., & Ourselin, S. (2010) · Computer Methods and Programs in Biomedicine · doi:10.1016/j.cmpb.2009.09.002 · Cited by 1,085
HAMMER: hierarchical attribute matching mechanism for elastic registration
Shen, D., & Davatzikos, C. (2002) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2002.803111 · Cited by 1,083
DeepFlow: Large Displacement Optical Flow with Deep Matching
Weinzaepfel, P., Revaud, J., Harchaoui, Z., & Schmid, C. (2013) · 2013 IEEE International Conference on Computer Vision · doi:10.1109/iccv.2013.175 · Cited by 997
Quicksilver: Fast predictive image registration – A deep learning approach
Yang, X., Kwitt, R., Styner, M., & Niethammer, M. (2017) · NeuroImage · doi:10.1016/j.neuroimage.2017.07.008 · Cited by 630 · cited 1×
“Most of these rely on ground truth warp fields [35]–[39], which are either obtained by simulating deformations and deformed images, or running classical registration methods on pairs of scans.”
BackgroundAn Unsupervised Learning Model for Deformable Medical Image Registration
Balakrishnan, G., Zhao, A., Sabuncu, M. R., Dalca, A. V., & Guttag, J. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00964 · Cited by 599
Self-citationView Synthesis by Appearance Flow
Zhou, T., Tulsiani, S., Sun, W., Malik, J., & Efros, A. A. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-46493-0_18 · Cited by 579
Oishi, K., Faria, A., Jiang, H., Li, X., Akhter, K., Zhang, J., Hsu, J. T., Miller, M. I., van Zijl, P. C. M., Albert, M., et al. (2009) · NeuroImage · doi:10.1016/j.neuroimage.2009.01.002 · Cited by 575
Landmark matching via large deformation diffeomorphisms
Joshi, S. C., & Miller, M. I. (2000) · IEEE Transactions on Image Processing · doi:10.1109/83.855431 · Cited by 501
Weakly-supervised convolutional neural networks for multimodal image registration
Hu, Y., Modat, M., Gibson, E., Li, W., Ghavami, N., Bonmati, E., Wang, G., Bandula, S., Moore, C. M., Emberton, M., et al. (2018) · Medical Image Analysis · doi:10.1016/j.media.2018.07.002 · Cited by 458
End-to-End Unsupervised Deformable Image Registration with a Convolutional Neural Network
de Vos, B. D., Berendsen, F. F., Viergever, M. A., Staring, M., & Išgum, I. (2017) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-67558-9_24 · Cited by 450
Dense image registration through MRFs and efficient linear programming
Glocker, B., Komodakis, N., Tziritas, G., Navab, N., & Paragios, N. (2008) · Medical Image Analysis · doi:10.1016/j.media.2008.03.006 · Cited by 427
Spatial Transformation and Registration of Brain Images Using Elastically Deformable Models
Davatzikos, C. (1997) · Computer Vision and Image Understanding · doi:10.1006/cviu.1997.0605 · Cited by 375
Global image registration using a symmetric block-matching approach
Modat, M., Cash, D. M., Daga, P., Winston, G. P., Duncan, J. S., & Ourselin, S. (2014) · Journal of Medical Imaging · doi:10.1117/1.jmi.1.2.024003 · Cited by 374
Back to Basics: Unsupervised Learning of Optical Flow via Brightness Constancy and Motion Smoothness
Yu, J. J., Harley, A. W., & Derpanis, K. G. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-49409-8_1 · Cited by 368
Unsupervised Learning for Fast Probabilistic Diffeomorphic Registration
Dalca, A. V., Balakrishnan, G., Guttag, J., & Sabuncu, M. R. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-00928-1_82 · Cited by 354
Self-citationNonrigid Image Registration Using Multi-scale 3D Convolutional Neural Networks
Sokooti, H., de Vos, B., Berendsen, F., Lelieveldt, B. P. F., Išgum, I., & Staring, M. (2017) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-66182-7_27 · Cited by 340
Svf-net: learning deformable image registration using shape matching
Rohé, M.-M., Datar, M., Heimann, T., Sermesant, M., & Pennec, X. (2017) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-66182-7_31 · Cited by 330
Transformation-Grounded Image Generation Network for Novel 3D View Synthesis
Park, E., Yang, J., Yumer, E., Ceylan, D., & Berg, A. C. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.82 · Cited by 279
Robust non-rigid registration through agent-based action learning
Krebs, J., Mansi, T., Delingette, H., Zhang, L., Ghesu, F. C., Miao, S., Maier, A. K., Ayache, N., Liao, R., & Kamen, A. (2017) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-66182-7_40 · Cited by 225
Deformable Image Registration Based on Similarity-Steered CNN Regression
Cao, X., Yang, J., Zhang, J., Nie, D., Kim, M., Wang, Q., & Shen, D. (2017) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-66182-7_35 · Cited by 225
Harvard Aging Brain Study: Dataset and accessibility
Dagley, A., LaPoint, M., Huijbers, W., Hedden, T., McLaren, D. G., Chatwal, J. P., Papp, K. V., Amariglio, R. E., Blacker, D., Rentz, D. M., et al. (2017) · NeuroImage · doi:10.1016/j.neuroimage.2015.03.069 · Cited by 224
Gollub, R. L., Shoemaker, J. M., King, M. D., White, T., Ehrlich, S., Sponheim, S. R., Clark, V. P., Turner, J. A., Mueller, B. A., Magnotta, V., et al. (2013) · Neuroinformatics · doi:10.1007/s12021-013-9184-3 · Cited by 211
Multi-contrast large deformation diffeomorphic metric mapping for diffusion tensor imaging
Ceritoglu, C., Oishi, K., Li, X., Chou, M.-C., Younes, L., Albert, M., Lyketsos, C., van Zijl, P. C. M., Miller, M. I., & Mori, S. (2009) · NeuroImage · doi:10.1016/j.neuroimage.2009.04.057 · Cited by 196
Label-driven weakly-supervised learning for multimodal deformarle image registration
Hu, Y., Marc Modat, A., Gibson, E., Ghavami, N., Bonmati, E., Moore, C. M., Emberton, M., Noble, J. A., Barratt, D. C., & Vercauteren, T. K. (2018) · IEEE International Symposium on Biomedical Imaging (ISBI) · doi:10.1109/isbi.2018.8363756 · Cited by 182
Large deformation diffeomorphic metric mapping of vector fields
Cao, Y., Miller, M. I., Winslow, R. L., & Younes, L. (2005) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2005.853923 · Cited by 180
Large Displacement Optical Flow from Nearest Neighbor Fields
Chen, Z., Jin, H., Lin, Z., Cohen, S. D., & Wu, Y. N. (2013) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2013.316 · Cited by 176
Miller, M. I., Beg, M. F., Ceritoglu, C., & Stark, C. (2005) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.0503892102 · Cited by 175
CNN-Based Patch Matching for Optical Flow with Thresholded Hinge Embedding Loss
Bailer, C., Varanasi, K., & Stricker, D. (2017) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2017.290 · Cited by 127
Deep End2End Voxel2Voxel Prediction
Tran, D., Bourdev, L. D., Fergus, R., Torresani, L., & Paluri, M. (2016) · IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw.2016.57 · Cited by 121
Unsupervised convolutional neural networks for motion estimation
Ahmadi, A., & Patras, I. (2016) · Proceedings - International Conference on Image Processing · doi:10.1109/icip.2016.7532634 · Cited by 120
PatchBatch: A Batch Augmented Loss for Optical Flow
Gadot, D., & Wolf, L. (2016) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2016.459 · Cited by 102
Hernández, M., Bossa, M. N., & Olmos, S. C. (2009) · International Journal of Computer Vision · doi:10.1007/s11263-009-0219-z · Cited by 97
Yeo, B. T. T., Sabuncu, M. R., Vercauteren, T. K., Holt, D. J., Amunts, K., Zilles, K., Golland, P., & Fischl, B. R. (2010) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2010.2049497 · Cited by 65
Self-citationFrequency Diffeomorphisms for Efficient Image Registration
Zhang, M., Liao, R., Dalca, A. V., Türk, E. A., Luo, J., Grant, P. E., & Golland, P. (2017) · Lecture notes in computer science · doi:10.1007/978-3-319-59050-9_44 · Cited by 64
Self-citationThewlis, J., Zheng, S., Torr, P. H. S., & Vedaldi, A. (2016) · British Machine Vision Conference (BMVC) · doi:10.5244/c.30.145 · Cited by 46
Patch-Based Discrete Registration of Clinical Brain Images
Dalca, A. V., Bobu, A., Rost, N. S., & Golland, P. (2016) · Lecture notes in computer science · doi:10.1007/978-3-319-47118-1_8 · Cited by 42
Self-citationMulti-subject Registration for Unbiased Statistical Atlas Construction
Craene, M. D., d'Aische, A. D. B., Macq, B. M. M., & Warfield, S. K. (2004) · Lecture notes in computer science · doi:10.1007/978-3-540-30135-6_80 · Cited by 38
Quantification and Analysis of Large Multimodal Clinical Image Studies: Application to Stroke
Sridharan, R., Dalca, A. V., Fitzpatrick, K. M., Cloonan, L. K., Kanakis, A. S., Wu, O., Furie, K. L., Rosand, J. M., Rost, N. S., & Golland, P. (2013) · Lecture notes in computer science · doi:10.1007/978-3-319-02126-3_3 · Cited by 29
Self-citation
Research works printed but not linked: 14
- milham, 2012, The ADHD-200 consortium: A model to advance the translational potential of neuroimaging in clinical neuroscience, Frontiers Syst Neurosci, 6, 62
- chollet, 2015, Keras
- abadi, 2016, Tensorflow Large-scale machine learning on heterogeneous distributed systems
- holmes, 2015, Brain genomics superstruct project initial data release with structural, functional, and behavioral measures, Data Science Journal, 2
- kingma, 2014, Adam A method for stochastic optimization
- ronneberger, 2015, U-net: Convolutional networks for biomedical image segmentation, Proc Int Conf Med Image Comput Comput -Assist Intervent (MICCAI), 234
- marino, 2018, Iterative amortized inference
- pennec, 1999, Understanding the ‘Demon’s algorithm’: 3D non-rigid registration by gradient descent, Proc Int Conf Med Image Comput Comput -Assist Intervent, 597
- kim, 2018, Semi-amortized variational autoencoders
- sønderby, 2016, Amortised map inference for image super-resolution
- zhang, 2017, Advances in variational inference
- cremer, 2018, Inference suboptimality in variational autoencoders
- jaderberg, 2015, Spatial transformer networks, Proc Adv Neural Inf Process Syst, 2017
- li, 2017, Non-rigid image registration using fully convolutional networks with deep self-supervision
Cited by
Zhou, S. K., Greenspan, H., Davatzikos, C., Duncan, J. S., Van Ginneken, B., Madabhushi, A., Prince, J. L., Rueckert, D., & Summers, R. M. (2021) · Proceedings of the IEEE · doi:10.1109/jproc.2021.3054390 · Cited by 1,118
Recent advances and clinical applications of deep learning in medical image analysis
Chen, X., Wang, X., Zhang, K., Fung, K.-M., Thai, T. C., Moore, K., Mannel, R. S., Liu, H., Zheng, B., & Qiu, Y. (2022) · Medical Image Analysis · doi:10.1016/j.media.2022.102444 · Cited by 1,080
Deep learning and alignment of spatially resolved single-cell transcriptomes with Tangram
Biancalani, T., Scalia, G., Buffoni, L., Avasthi, R., Lu, Z., Sanger, A., Tokcan, N., Vanderburg, C. R., Segerstolpe, Å., Zhang, M., et al. (2021) · Nature Methods · doi:10.1038/s41592-021-01264-7 · Cited by 931
TransMorph: Transformer for unsupervised medical image registration
Chen, J., Frey, E. C., He, Y., Segars, W. P., Li, Y., & Du, Y. (2022) · Medical Image Analysis · doi:10.1016/j.media.2022.102615 · Cited by 804
Deep learning in medical image registration: a review
Fu, Y., Lei, Y., Wang, T., Curran, W. J., Liu, T., & Yang, X. (2020) · Physics in Medicine & Biology · doi:10.1088/1361-6560/ab843e · Cited by 661
Metadata
- Authors
- Balakrishnan, G., Zhao, A., Sabuncu, M. R., Guttag, J., & Dalca, A. V.
- Venue
- IEEE Transactions on Medical Imaging
- Published
- 2019-08
- Volume
- 38
- Issue
- 8
- Pages
- 1788-1800
- ISSN
- 0278-0062 (ISSN-L)
- 1558-254X
- OpenAlex
- W2891631795
- Cited by
- 2,174
- References
- 64
- FWCI
- 72.17
- Type
- journal article
- Language
- en
- Open access
- green
- Licence
- Funders
- NIH R01 grants (R01LM012719, R01AG053949, 1R21AG050122)
- the NSF NeuroNex grant (1707312)
- NSF CAREER grant (1748377)
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.