CoChat
Sign Up
Cited by 297Medical Image Segmentation TechniquesDomain Adaptation and Few-Shot LearningAdvanced Neural Network Applications

Birnet: Brain image registration using dual-supervised fully convolutional networks

Open access
Fan, Jingfan, Cao, Xiaohuan, Yap, Pew-Thian, Shen, Dinggang·2019·Medical Image Analysis
DOI 10.1016/j.media.2019.03.006PMC PMC6764428
Open this paper in CoChat — free account

Abstract

In this paper, we propose a deep learning approach for image registration by predicting deformation from image appearance. Since obtaining ground-truth deformation fields for training can be challenging, we design a fully convolutional network that is subject to dual-guidance: (1) Ground-truth guidance using deformation fields obtained by an existing registration method; and (2) Image dissimilarity guidance using the difference between the images after registration. The latter guidance helps avoid overly relying on the supervision from the training deformation fields, which could be inaccurate. For effective training, we further improve the deep convolutional network with gap filling, hierarchical loss, and multi-source strategies. Experiments on a variety of datasets show promising registration accuracy and efficiency compared with state-of-the-art methods.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1016/j.media.2019.03.006

Cites

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Fan et al. presented a Birnet for brain MRI image registration by utilizing the deformation field estimated from the traditional registration method.

    “Fan et al. (74) presented a Birnet for brain MRI image registration by utilizing the deformation field estimated from the traditional registration method.”

    From A review of deep learning-based deformable medical image registration (2022) · cites this paper as [74]

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

The citation lists haven’t been fetched yet

References 71 · Cited by 297

Research works printed but not linked: 56

  1. Andersson JL, Jenkinson M, Smith S, 2007 Non-linear registration, aka Spatial normalisation FMRIB technical report TR07JA2. FMRIB Analysis Group of the University of Oxford 2.
  2. Auzias G, Colliot O, Glaunes JA, Perrot M, Mangin J-F, Trouve A, Baillet S, 2011 Diffeomorphic brain registration under exhaustive sulcal constraints. IEEE Trans. Med. Imaging 30, 1214– 1227. 21278014
  3. Avants BB, Epstein CL, Grossman M, Gee JC, 2008 Symmetric diffeomorphic image registration with cross-correlation: evaluating automated labeling of elderly and neurodegenerative brain. Med. Image Anal. 12, 26– 41. 17659998 PMC2276735
  4. Balakrishnan G, Zhao A, Sabuncu MR, Guttag J, Dalca AV, 2018 An unsupervised learning model for deformable medical image registration. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 9252– 9260.
  5. Balakrishnan G, Zhao A, Sabuncu MR, Guttag J, Dalca AV, 2019 VoxelMorph: a learning framework for deformable medical image registration. IEEE Trans. Med. Imaging 30716034
  6. Cao T, Singh N, Jojic V, Niethammer M, 2015 Semi-coupled dictionary learning for deformation prediction, Biomedical Imaging (ISBI) In: 2015 IEEE 12th International Symposium on. IEEE, pp. 691– 694.
  7. Cao X, Yang J, Zhang J, Nie D, Kim M, Wang Q, Shen D, 2017 Deformable image registration based on similarity-steered cnn regression In: International Conference on Medical Image Computing and Computer-Assisted Intervention. Springer, pp. 300– 308. PMC5731783 29250613
  8. Fan J, Yang J, Zhao Y, Ai D, Liu Y, Wang G, Wang Y, 2017 Convex hull aided registration method (CHARM). IEEE Trans. Vis. Comput. Graph 23, 2042– 2055. 28113589
  9. Fluck O, Vetter C, Wein W, Kamen A, Preim B, Westermann R, 2011 A survey of medical image registration on graphics hardware. Comput. Methods Programs Biomed. 104, e45– e57. 21112118
  10. Gutiérrez-Becker B, Mateus D, Peter L, Navab N, 2016 Learning optimization updates for multimodal registration In: International Conference on Medical Image Computing and Computer-Assisted Intervention. Springer, pp. 19– 27.
  11. Gutierrez-Becker B, Mateus D, Peter L, Navab N, 2017 Guiding multimodal registration with learned optimization updates. Med. Image Anal 41, 2– 17. 28506641
  12. Hu Y, Modat M, Gibson E, Ghavami N, Bonmati E, Moore CM, Emberton M, Noble JA, Barratt DC, Vercauteren T, 2018 Label-driven weakly-supervised learning for multimodal deformarle image registration, Biomedical Imaging (ISBI 2018) In: 2018 IEEE 15th International Symposium on. IEEE, pp. 1070– 1074.
  13. Ilg E, Mayer N, Saikia T, Keuper M, Dosovitskiy A, Brox T, 2017 Flownet 2.0: evolution of optical flow estimation with deep networks. In: IEEE conference on computer vision and pattern recognition (CVPR), p. 6.
  14. Jenkinson M, Smith S, 2001 A global optimisation method for robust affine registration of brain images. Med. Image Anal. 5, 143– 156. 11516708
  15. Jia Y, Shelhamer E, Donahue J, Karayev S, Long J, Girshick R, Guadarrama S, Darrell T, 2014 Caffe: convolutional architecture for fast feature embedding In: Proceedings of the 22nd ACM international conference on Multimedia. ACM, pp. 675– 678.
  16. Johnson HJ, Christensen GE, 2002 Consistent landmark and intensity-based image registration. IEEE Trans. Med. Imaging 21, 450– 461. 12071616
  17. Kim M, Wu G, Wang Q, Lee S-W, Shen D, 2015 Improved image registration by sparse patch-based deformation estimation. NeuroImage 105, 257– 268. 25451481 PMC4262658
  18. Kim M, Wu G, Yap P-T, Shen D, 2012 A general fast registration framework by learning deformation–appearance correlation. IEEE Trans. Image Process 21, 1823– 1833. 21984505 PMC3355525
  19. Kingma D, Ba J, 2014 Adam: a method for stochastic optimization. arXiv:1412.6980. · arXiv
  20. Klein A, Andersson J, Ardekani BA, Ashburner J, Avants B, Chiang M-C, Christensen GE, Collins DL, Gee J, Hellier P, 2009 Evaluation of 14 nonlinear deformation algorithms applied to human brain MRI registration. NeuroImage 46, 786– 802. 19195496 PMC2747506
  21. Klein S, Staring M, Murphy K, Viergever MA, Pluim JP, 2010 Elastix: a tool-box for intensity-based medical image registration. IEEE Trans. Med. Imaging 29, 196– 205. 19923044
  22. Krebs J, e Delingette H, Mailhé B, Ayache N, Mansi T, 2019 Learning a probabilistic model for diffeomorphic registration. IEEE Trans. Med. Imaging 30716033
  23. Krebs J, Mansi T, Delingette H, Zhang L, Ghesu FC, Miao S, Maier AK, Ayache N, Liao R, Kamen A, 2017 Robust non-rigid registration through agent-based action learning In: International Conference on Medical Image Computing and Computer-Assisted Intervention. Springer, pp. 344– 352.
  24. Loeckx D, Maes F, Vandermeulen D, Suetens P, 2003 Non-rigid image registration using a statistical spline deformation model In: Biennial International Conference on Information Processing in Medical Imaging. Springer, pp. 463– 474. 15344480
  25. Lorenzi M, Ayache N, Frisoni GB, Pennec X, Initiative, A.s.D.N., 2013 LC-C-Demons: a robust and accurate symmetric diffeomorphic registration algorithm. NeuroImage 81, 470– 483. 23685032
  26. Luo Y. g., Liu P, Shi L, Luo Y, Yi L, Li A, Qin J, Heng P-A, Wang D, 2015 Accelerating neuroimage registration through parallel computation of similarity metric. PloS One 10, e0136718. 26352412 PMC4564209
  27. Muyan-Ozcelik P, Owens JD, Xia J, Samant SS, 2008 Fast deformable registration on the GPU: a CUDA implementation of demons, Computational Sciences and Its Applications In: 2008. ICCSA’08. International Conference on. IEEE, pp. 223– 233.
  28. Myronenko A, Song X, 2010 Intensity-based image registration by minimizing residual complexity. IEEE Trans. Med. Imaging 29, 1882– 1891. 20562036
  29. Oliveira FP, Tavares JMR, 2014 Medical image registration: a review. Comput. Methods Biomech. Biomed. Eng 17, 73– 93. 22435355
  30. Ou Y, Sotiras A, Paragios N, Davatzikos C, 2011 DRAMMS: deformable registration via attribute matching and mutual-saliency weighting. Med. Image Anal 15, 622– 639. 20688559 PMC3012150
  31. Parajuli N, Lu A, Stendahl JC, Zontak M, Boutagy N, Alkhalil I, Eberle M, Lin BA, O’Donnell M, Sinusas AJ, 2017 Flow network based cardiac motion tracking leveraging learned feature matching In: International Conference on Medical Image Computing and Computer-Assisted Intervention. Springer, pp. 279– 286.
  32. ur Rehman T, Haber E, Pryor G, Melonakos J, Tannenbaum A, 2009 3D non-rigid registration via optimal mass transport on the GPU. Med. Image Anal 13, 931– 940. 19135403 PMC2811327
  33. Rueckert D, Frangi AF, Schnabel JA, 2001 Automatic construction of 3D statistical deformation models using non-rigid registration In: International Conference on Medical Image Computing and Computer-Assisted Intervention. Springer, pp. 77– 84.
  34. Rueckert D, Frangi AF, Schnabel JA, 2003 Automatic construction of 3-D statistical deformation models of the brain using nonrigid registration. IEEE Trans. Med. imaging 22, 1014– 1025. 12906255
  35. Samant S, Lee S, Samant S, 2015 GPU-based unimodal deformable image registration in radiation therapy In: Xun Jia, Steve Jiang(Eds.), Graphics Processing Unit-Based High Performance Computing in Radiation Therapy. Series: Series in Medical Physics and Biomedical Engineering, 129– 148. CRC Press, pp. 129–148. ISBN: 978-1-4822-4478-6.
  36. Schmidhuber J, 2015 Deep learning in neural networks: an overview. Neural Netw. 61, 85– 117. 25462637
  37. Serag A, Aljabar P, Ball G, Counsell SJ, Boardman JP, Rutherford MA, Ed-wards AD, Hajnal JV, Rueckert D, 2012 Construction of a consistent high--definition spatio-temporal atlas of the developing brain using adaptive kernel regression. NeuroImage 59, 2255– 2265. 21985910
  38. Shamonin DP, Bron EE, Lelieveldt BP, Smits M, Klein S, Staring M, Initiative, A.s.D.N., 2013 Fast parallel image registration on CPU and GPU for diagnostic classification of Alzheimer’s disease. Front. Neuroinform 7, 50. 24474917 PMC3893567
  39. Shattuck DW, Mirza M, Adisetiyo V, Hojatkashani C, Salamon G, Narr KL, Pol-drack RA, Bilder RM, Toga AW, 2008 Construction of a 3D probabilistic atlas of human cortical structures. NeuroImage 39, 1064– 1080. 18037310 PMC2757616
  40. Shen D, Davatzikos C, 2002 HAMMER: hierarchical attribute matching mechanism for elastic registration. IEEE Trans. Med. Imaging 21, 1421– 1439. 12575879
  41. Sokooti H, de Vos B, Berendsen F, Lelieveldt BP, Išgum I, Staring M, 2017 Non-rigid image registration using multi-scale 3D convolutional neural networks In: International Conference on Medical Image Computing and Computer-Assisted Intervention. Springer, pp. 232– 239.
  42. Tang Z, Ahmad S, Yap P-T, Shen D, 2018 Multi-atlas segmentation of MR tumor brain images using low-rank based image recovery. IEEE Trans. Med. Imaging 37, 2224– 2235. 29993928 PMC6176916
  43. Tang Z, Yap P-T, Shen D, 2019 A new multi-atlas registration framework for multimodal pathological images using conventional monomodal normal atlases. IEEE Trans. Image Process. 28, 2293– 2304. PMC6579720 30571622
  44. Vercauteren T, Pennec X, Perchant A, Ayache N, 2009 Diffeomorphic demons: efficient non-parametric image registration. NeuroImage 45, S61– S72. 19041946
  45. de Vos BD, Berendsen FF, Viergever MA, Staring M, Išgum I, 2017 End-to-end unsupervised deformable image registration with a convolutional neural network In: Deep Learning in Medical Image Analysis and Multimodal Learning For Clinical Decision Support. Springer, pp. 204– 212.
  46. Wang Q, Kim M, Shi Y, Wu G, Shen D, Initiative, A.s.D.N., 2015 Predict brain MR image registration via sparse learning of appearance and transformation. Med. Image Anal. 20, 61– 75. 25476412 PMC4294959
  47. Wu G, Kim M, Wang Q, Shen D, 2014 S HAMMER: hierarchical attribute guided, symmetric diffeomorphic registration for MR brain images. Hum. Brain Mapp. 35, 1044– 1060. 23283836 PMC3835769
  48. Wu G, Yap P-T, Kim M, Shen D, 2010 TPS-HAMMER: improving HAMMER registration algorithm by soft correspondence matching and thin-plate splines based deformation interpolation. NeuroImage 49, 2225– 2233. 19878724 PMC2818392
  49. Xue Z, Shen D, Davatzikos C, 2004 Determining correspondence in 3-D MR brain images using attribute vectors as morphological signatures of voxels. IEEE Trans. Med. Imaging 23, 1276– 1291. 15493695
  50. Xue Z, Shen D, Davatzikos C, 2006a Statistical representation of high-dimensional deformation fields with application to statistically constrained 3D warping. Med. Image Anal. 10, 740– 751. 16887376
  51. Xue Z, Shen D, Karacali B, Stern J, Rottenberg D, Davatzikos C, 2006b Simulating deformations of MR brain images for validation of atlas-based segmentation and registration algorithms. NeuroImage 33, 855– 866. 16997578 PMC1752202
  52. Yang J, Shen D, Davatzikos C, Verma R, 2008 Diffusion tensor image registration using tensor geometry and orientation features In: International Conference on Medical Image Computing and Computer-Assisted Intervention. Springer, pp. 905– 913. 18982691
  53. Zacharaki E, Hogea C, Shen D, Biros G, Davatzikos C, 2009 Non-diffeomorphic registration of brain tumor images by simulating tissue loss and tumor growth. NeuroImage 46, 762– 774. 19408350 PMC2929986
  54. Zhou T, Bhaskar H, Liu F, Yang J, 2017 Graph regularized and locality-constrained coding for robust visual tracking. IEEE Trans. Circuits Syst. Video Technol 27, 2153– 2164.
  55. Zhou T, Thung K-H, Liu M, Shen D, 2019a Brain-wide genome-wide association study for alzheimer’s disease via joint projection learning and sparse regression model. IEEE Trans. Biomed. Eng 66, 165– 175. 29993426 PMC6342004
  56. Zhou T, Thung KH, Zhu X, Shen D, 2019b Effective feature learning and fusion of multimodality data using stage-wise deep neural network for dementia diagnosis. Hum. Brain Mapp 40, 1001– 1016. 30381863 PMC6865441

Authors
Fan, J., Cao, X., Yap, P.-T., & Shen, D.
Affiliations
  • University of North Carolina at Chapel Hill (US)
  • United Imaging Intelligence (China) (CN)
Corresponding author
Shen, Dinggang
ORCID
Venue
Medical Image Analysis
Publisher
Elsevier BV
Published
2019-05
Volume
54
Pages
193-206
ISSN
1361-8415 (ISSN-L)
DOI
10.1016/j.media.2019.03.006
PMC
PMC6764428
OpenAlex
W2963196212
Cited by
297
References
71
Citations by year
2026: 9 · 2025: 33 · 2024: 30 · 2023: 56 · 2022: 46 · 2021: 51 · 2020: 44 · 2019: 26 · 2018: 2
FWCI
18.66
Type
journal article
Language
en
Open access
Licence
Locations
  • Medical Image Analysis (publishedVersion)
  • PubMed (publishedVersion)
  • Europe PMC (PubMed Central) (acceptedVersion)
  • PubMed Central (acceptedVersion)
Funders
NIH (EB006733, EB008374, MH100217, MH108914, AG041721, AG049371, AG042599, AG053867, EB022880, MH110274)
Topics
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Medical Image Segmentation Techniques
  • Physical Sciences › Computer Science › Artificial Intelligence › Domain Adaptation and Few-Shot Learning
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
Keywords
brain image registration, fully convolutional network, gap filling
MeSH terms
Deep Learning, Brain Mapping, Humans, Image Processing, Computer-Assisted, Magnetic Resonance Imaging, Neural Networks, Computer

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review