CoChat
Sign Up
Cited by 467Medical Image Segmentation Techniques3D Shape Modeling and AnalysisDomain Adaptation and Few-Shot Learning

Unsupervised Learning of Probabilistic Diffeomorphic Registration for Images and Surfaces

Open access
Dalca, Adrian V., Balakrishnan, Guha, Guttag, John, Sabuncu, Mert R.·2019·Medical Image Analysis
DOI 10.1016/j.media.2019.07.006
Open this paper in CoChat — free accountOpen PDF

Abstract

Classical deformable registration techniques achieve impressive results and offer a rigorous theoretical treatment, but are computationally intensive since they solve an optimization problem for each image pair. Recently, learning-based methods have facilitated fast registration by learning spatial deformation functions. However, these approaches use restricted deformation models, require supervised labels, or do not guarantee a diffeomorphic (topology-preserving) registration. Furthermore, learning-based registration tools have not been derived from a probabilistic framework that can offer uncertainty estimates. In this paper, we build a connection between classical and learning-based methods. We present a probabilistic generative model and derive an unsupervised learning-based inference algorithm that uses insights from classical registration methods and makes use of recent developments in convolutional neural networks (CNNs). We demonstrate our method on a 3D brain registration task for both images and anatomical surfaces, and provide extensive empirical analyses of the algorithm. Our principled approach results in state of the art accuracy and very fast runtimes, while providing diffeomorphic guarantees. Our implementation is available online at http://voxelmorph.csail.mit.edu.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1016/j.media.2019.07.006

Cites

8 of 8 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Some uncertainty estimation approaches report interval estimates by reformulating the registration objective as a probability distribution and drawing samples, using test-time NN drop-out, or sampling using test-time deformation covariance matrices in a variational autoencoder.

    “Some of these approaches report interval estimates, either by reformulating the registration objective function as a probability distribution and drawing samples (Risholm et al., 2013; Le Folgoc et al., 2016; Schultz et al., 2018), approximating the sampling process using test-time NN drop-out (Yang et al., 2017) or sampling using the test-time deformation covariance matrices embedded within a”

    From The Impact of Machine Learning on 2D/3D Registration for Image-Guided Interventions: A Systematic Review and Perspective (2021) · cites this paper as Dalca et al., 2019

    Section 2.2 (p. 3) describes a network that outputs deformation-related mean and covariance parameters, and Section 4.4.4 (p. 11) shows sampled velocity fields and their empirical variance. But Section 2.4 (p. 5) says the covariance is not used at test time and uncertainty analysis is beyond the paper’s scope. The source therefore supports covariance-based sampling as a capability, not the attributed reporting of interval estimates from test-time sampling. (LLM, conf=0.88)

    Experimental — accuracy not yet measured

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Flagged first, then by citations

Research works printed but not linked: 52

  1. [3][3] A Amir-Khalili, G Hamarneh, R Zakariaee, I Spadinger, and R Abugharbieh. Propagation of registration uncertainty during multi-fraction cervical cancer brachytherapy. Physics in Medicine & Biology , 62(20):8116, 2017. -
  2. [4][4] Vincent Arsigny, Olivier Commowick, Xavier Pennec, and Nicholas Ayache. A log-euclidean framework for statistics on diffeomorphisms. In International Conference on Medical Image Computing and Computer-Assisted Intervention , pages 924-931. Springer, 2006. -
  3. [5][5] J. Ashburner. A fast diffeomorphic image registration algorithm. Neuroimage , 38(1):95-113, 2007. -
  4. [7][7] Brian B Avants, Charles L Epstein, Murray Grossman, and James C Gee. Symmetric diffeomorphic image registration with cross-correlation: evaluating automated labeling of elderly and neurodegenerative brain. Medical image analysis , 12(1):26-41, 2008. -
  5. [9][9] Guha Balakrishnan, Amy Zhao, Mert R Sabuncu, John Guttag, and Adrian V Dalca. An unsupervised learning model for deformable medical image registration. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition , pages 9252-9260, 2018. -
  6. [10][10] Guha Balakrishnan, Amy Zhao, Mert R Sabuncu, John Guttag, and Adrian V Dalca. Voxelmorph: A learning framework for deformable medical image registration. arXiv preprint arXiv:1809.05231 , 2019. - · arXiv
  7. [12][12] Paul J Besl and Neil D McKay. Method for registration of 3-d shapes. In Sensor Fusion IV: Control Paradigms and Data Structures , volume 1611, pages 586-607. International Society for Optics and Photonics, 1992. -
  8. [13][13] Xiaohuan Cao, Jianhua Yang, Jun Zhang, Dong Nie, Minjeong Kim, Qian Wang, and Dinggang Shen. Deformable image registration based on similaritysteered cnn regression. In International Conference on Medical Image Computing and ComputerAssisted Intervention , pages 300-308. Springer, 2017. -
  9. [14][14] Yan Cao, Michael I Miller, Raimond L Winslow, and Laurent Younes. Large deformation diffeomorphic metric mapping of vector fields. IEEE transactions on medical imaging , 24(9):1216-1230, 2005. -
  10. [15][15] Can Ceritoglu, Kenichi Oishi, Xin Li, Ming-Chung Chou, Laurent Younes, Marilyn Albert, Constantine Lyketsos, Peter CM van Zijl, Michael I Miller, and Susumu Mori. Multi-contrast large deformation diffeomorphic metric mapping for diffusion tensor imaging. Neuroimage , 47(2):618-627, 2009. -
  11. [18][18] Adrian V. Dalca, Guha Balakrishnan, John Guttag, and Mert R. Sabuncu. Unsupervised learning for fast probabilistic diffeomorphic registration. In Medical Image Computing and Computer Assisted Intervention (MICCAI) , pages 729-738, Cham, 2018. Springer. -
  12. [19][19] Adrian V Dalca, Andreea Bobu, Natalia S Rost, and Polina Golland. Patch-based discrete registration of clinical brain images. In MICCAI-PATCHMI Patchbased Techniques in Medical Imaging . Springer, 2016. -
  13. [20][20] Adrian V Dalca, John Guttag, and Mert R Sabuncu. Anatomical priors in convolutional networks for unsupervised biomedical segmentation. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition , pages 9290-9299, 2018. -
  14. [21][21] Christos Davatzikos. Spatial transformation and registration of brain images using elastically deformable models. Computer Vision and Image Understanding , 66(2):207-222, 1997. -
  15. [23][23] Bob D de Vos, Floris F Berendsen, Max A Viergever, Marius Staring, and Ivana Išgum. End-to-end unsupervised deformable image registration with a convolutional neural network. In DLMIA , pages 204-212. Springer, 2017. -
  16. [24][24] A. Di Martino et al. The autism brain imaging data exchange: towards a large-scale evaluation of the 13. intrinsic brain architecture in autism. Molecular psychiatry , 19(6):659-667, 2014. -
  17. [25][25] Lee R. Dice. Measures of the amount of ecologic association between species. Ecology , 26(3):297302, 1945. -
  18. [26][26] Stanley Durrleman. Statistical models of currents for measuring the variability of anatomical curves, surfaces and their evolution . PhD thesis, Université Nice Sophia Antipolis, 2010. -
  19. [27][27] B. Fischl. Freesurfer. Neuroimage , 62(2):774-781, 2012. -
  20. [28][28] B. Glocker, N. Komodakis, G. Tziritas, N. Navab, and N. Paragios. Dense image registration through mrfs and efficient linear programming. Medical image analysis , 12(6):731-741, 2008. -
  21. [29][29] Randy L Gollub, Jody M Shoemaker, Margaret D King, Tonya White, Stefan Ehrlich, Scott R Sponheim, Vincent P Clark, Jessica A Turner, Bryon A Mueller, Vince Magnotta, et al. The MCIC collection: a shared repository of multi-modal, multi-site brain image data from a clinical investigation of schizophrenia. Neuroinformatics , 11(3):367-388, 2013. -
  22. [30][30] Mattias P Heinrich, Mark Jenkinson, Michael Brady, and Julia A Schnabel. Mrf-based deformable registration and ventilation estimation of lung ct. IEEE transactions on medical imaging , 32(7):1239-1248, 2013. -
  23. [32][32] Monica Hernandez, Matias N Bossa, and Salvador Olmos. Registration of anatomical images using paths of diffeomorphisms parameterized with stationary vector field flows. International Journal of Computer Vision , 85(3):291-306, 2009. -
  24. [34][34] Yipeng Hu, Marc Modat, Eli Gibson, Nooshin Ghavami, Ester Bonmati, Caroline M Moore, Mark Emberton, J Alison Noble, Dean C Barratt, and Tom Vercauteren. Label-driven weakly-supervised learning for multimodal deformable image registration. In Biomedical Imaging (ISBI 2018), 2018 IEEE 15th International Symposium on , pages 1070-1074. IEEE, 2018. -
  25. [36][36] Max Jaderberg, Karen Simonyan, and Andrew Zisserman. Spatial transformer networks. In Advances in neural information processing systems , pages 20172025, 2015. -
  26. [37][37] Sarang C Joshi and Michael I Miller. Landmark matching via large deformation diffeomorphisms. IEEE transactions on image processing , 9(8):13571370, 2000. -
  27. [39][39] D.P. Kingma and M. Welling. Auto-encoding variational bayes. arXiv:1312.6114 , 2013. - · arXiv
  28. [40][40] Arno Klein, Jesper Andersson, Babak A Ardekani, John Ashburner, Brian Avants, Ming-Chang Chiang, Gary E Christensen, D Louis Collins, James Gee, Pierre Hellier, et al. Evaluation of 14 nonlinear deformation algorithms applied to human brain mri registration. Neuroimage , 46(3):786-802, 2009. -
  29. [41][41] Julian Krebs, Hervé e Delingette, Boris Mailhé, Nicholas Ayache, and Tommaso Mansi. Learning a probabilistic model for diffeomorphic registration. IEEE transactions on medical imaging , 2019. -
  30. [42][42] Julian Krebs et al. Robust non-rigid registration through agent-based action learning. In Medical Image Computing and Computer-Assisted Intervention (MICCAI) , pages 344-352, Cham, 2017. Springer International Publishing. -
  31. [43][43] Julian Krebs, Tommaso Mansi, Boris Mailhé, Nicholas Ayache, and Hervé Delingette. Unsupervised probabilistic deformation modeling for robust diffeomorphic registration. In Deep Learning in Medical Image Analysis and Multimodal Learning for Clinical Decision Support , pages 101-109. Springer, 2018. -
  32. [44][44] H. Li and Y. Fan. Non-rigid image registration using fully convolutional networks with deep selfsupervision. arXiv preprint arXiv:1709.00799 , 2017. - · arXiv
  33. [45][45] Daniel S Marcus, Tracy H Wang, Jamie Parker, John G Csernansky, John C Morris, and Randy L Buckner. Open access series of imaging studies (oasis): cross-sectional mri data in young, middle aged, nondemented, and demented older adults. Journal of cognitive neuroscience , 19(9):1498-1507, 2007. -
  34. [46][46] Kenneth Marek, Danna Jennings, Shirley Lasch, Andrew Siderowf, Caroline Tanner, Tanya Simuni, Chris Coffey, Karl Kieburtz, Emily Flagg, Sohini Chowdhury, et al. The parkinson progression marker initiative (ppmi). Progress in neurobiology , 95(4):629635, 2011. -
  35. [47][47] Michael I Miga, Tuhin K Sinha, David M Cash, Robert L Galloway, and Robert J Weil. Cortical - surface registration for image-guided neurosurgery using laser-range scanning. IEEE Transactions on medical Imaging , 22(8):973-985, 2003. -
  36. [49][49] Michael I Miller, M Faisal Beg, Can Ceritoglu, and Craig Stark. Increasing the power of functional maps of the medial temporal lobe by using large deformation diffeomorphic metric mapping. Proceedings of the National Academy of Sciences , 102(27):96859690, 2005. -
  37. [50][50] Marc Modat, David M Cash, Pankaj Daga, Gavin P Winston, John S Duncan, and Sébastien Ourselin. Global image registration using a symmetric blockmatching approach. Journal of Medical Imaging , 1(2):024003, 2014. -
  38. [51][51] Marc Modat, Gerard R Ridgway, Zeike A Taylor, Manja Lehmann, Josephine Barnes, David J Hawkes, Nick C Fox, and Sébastien Ourselin. Fast freeform deformation using graphics processing units. Computer methods and programs in biomedicine , 98(3):278-284, 2010. -
  39. [53][53] Cleve Moler and Charles Van Loan. Nineteen dubious ways to compute the exponential of a matrix, twenty-five years later. SIAM review , 45(1):3-49, 2003. -
  40. [54][54] Marc Niethammer, Roland Kwitt, and FrancoisXavier Vialard. Metric learning for image registration. arXiv preprint arXiv:1904.09524 , 2019. - · arXiv
  41. [55][55] Kenichi Oishi, Andreia Faria, Hangyi Jiang, Xin Li, Kazi Akhter, Jiangyang Zhang, John T Hsu, Michael I Miller, Peter CM van Zijl, Marilyn Albert, et al. Atlas-based whole brain white matter analysis using large deformation diffeomorphic metric mapping: application to normal elderly and alzheimer's disease participants. Neuroimage , 46(2):486-499, 2009. -
  42. [56][56] Xavier Pennec, Pascal Cachier, and Nicholas Ayache. Understanding the "demon's algorithm": 3d non-rigid registration by gradient descent. pages 597605, 1999. -
  43. [57][57] Gheorghe Postelnicu, Lilla Zollei, and Bruce Fischl. Combined volumetric and surface registration. IEEE transactions on medical imaging , 28(4):508522, 2008. -
  44. [61][61] Daniel Rueckert, Luke I Sonoda, Carmel Hayes, Derek LG Hill, Martin O Leach, and David J Hawkes. Nonrigid registration using free-form deformation: Application to breast mr images. IEEE Transactions on Medical Imaging , 18(8):712-721, 1999. -
  45. [62][62] D. Shen and C. Davatzikos. Hammer: Hierarchical attribute matching mechanism for elastic registration. IEEE Trans. Med. Imag. , 21(11):1421-1439, 2002. -
  46. [63][63] Ivor JA Simpson, Mark W Woolrich, Jesper LR Andersson, Adrian R Groves, and Julia A Schnabel. A probabilistic non-rigid registration framework using local noise estimates. In 2012 9th IEEE International Symposium on Biomedical Imaging (ISBI) , pages 688691. IEEE, 2012. -
  47. [64][64] Hessam Sokooti, Bob de Vos, Floris Berendsen, Boudewijn PF Lelieveldt, Ivana Išgum, and Marius Staring. Nonrigid image registration using multiscale 3d convolutional neural networks. In MICCAI , pages 232-239, Cham, 2017. Springer. -
  48. [65][65] R. Sridharan, A.V. Dalca, K.M. Fitzpatrick, L. Cloonan, A. Kanakis, O. Wu, K.L. Furie, J. Rosand, N.S. Rost, and P. Golland. Quantification and analysis of large multimodal clinical image studies: Application to stroke. In International Workshop on Multimodal Brain Image Analysis , pages 18-30. Springer International Publishing, 2013. -
  49. [66][66] J.P. Thirion. Image matching as a diffusion process: an analogy with maxwell's demons. Medical Image Analysis , 2(3):243-260, 1998. -
  50. [67][67] Tom Vercauteren et al. Diffeomorphic demons: Efficient non-parametric image registration. NeuroImage , 45(1):S61-S72, 2009. -
  51. [69][69] BT Thomas Yeo, Mert R Sabuncu, Tom Vercauteren, Daphne J Holt, Katrin Amunts, Karl Zilles, Polina Golland, and Bruce Fischl. Learning task-optimal registration cost functions for localizing cytoarchitecture and function in the cerebral cortex. IEEE transactions on medical imaging , 29(7):1424-1441, 2010. -
  52. [70][70] Miaomiao. Zhang, Ruizhi. Liao, Adrian V Dalca, Esra A Turk, Jie Luo, P Ellen Grant, and Polina Golland. Frequency diffeomorphisms for efficient image registration. In International Conference on Information Processing in Medical Imaging , pages 559-570. Springer, 2017.

Cited by

Sorted by citations

Authors
Dalca, A. V., Balakrishnan, G., Guttag, J., & Sabuncu, M. R.
Affiliations
  • Cornell University (US)
  • MIT Computer Science and Artificial Intelligence Laboratory (US)
Corresponding author
Dalca, Adrian V.
ORCID
Venue
Medical Image Analysis
Publisher
Elsevier BV
Published
2019-10
Volume
57
Pages
226-236
ISSN
1361-8415 (ISSN-L)
DOI
10.1016/j.media.2019.07.006
OpenAlex
W2922479016
Cited by
467
References
8
Citations by year
2026: 58 · 2025: 93 · 2024: 86 · 2023: 64 · 2022: 66 · 2021: 53 · 2020: 36 · 2019: 11
FWCI
18.55
Type
journal article
Language
en
Open access
Licence
Locations
  • Medical Image Analysis (publishedVersion)
  • PubMed (publishedVersion)
  • arXiv (Cornell University) (submittedVersion)
  • DSpace@MIT (Massachusetts Institute of Technology) (submittedVersion, cc-by-nc-nd)
  • arXiv (Cornell University)
Funders
Cornell University
Topics
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Medical Image Segmentation Techniques
  • Physical Sciences › Engineering › Computational Mechanics › 3D Shape Modeling and Analysis
  • Physical Sciences › Computer Science › Artificial Intelligence › Domain Adaptation and Few-Shot Learning
Keywords
diffeomorphic registration, deformable image registration, unsupervised learning, probabilistic generative models, convolutional neural networks, uncertainty estimation, VoxelMorph
MeSH terms
Unsupervised Machine Learning, Algorithms, Brain, Humans, Image Processing, Computer-Assisted, Magnetic Resonance Imaging, Pattern Recognition, Automated, Models, Statistical, Neural Networks, Computer, Imaging, Three-Dimensional

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review