The DOI matches its DataCite record; 44 of the 44 references it cites were found in a registry or an index.
Real-time Deep Pose Estimation with Geodesic Loss for Image-to-Template Rigid Registration
Open accessAbstract
With an aim to increase the capture range and accelerate the performance of state-of-the-art inter-subject and subject-to-template 3D registration, we propose deep learning-based methods that are trained to find the 3D position of arbitrarily oriented subjects or anatomy based on slices or volumes of medical images. For this, we propose regression CNNs that learn to predict the angle-axis representation of 3D rotations and translations using image features. We use and compare mean square error and geodesic loss to train regression CNNs for 3D pose estimation used in two different scenarios: slice-to-volume registration and volume-to-volume registration. Our results show that in such registration applications that are amendable to learning, the proposed deep learning methods with geodesic loss minimization can achieve accurate results with a wide capture range in real-time (<100ms). We also tested the generalization capability of the trained CNNs on an expanded age range and on images of newborn subjects with similar and different MR image contrasts. We trained our models on T2-weighted fetal brain MRI scans and used them to predict the 3D pose of newborn brains based on T1-weighted MRI scans. We showed that the trained models generalized well for the new domain when we performed image contrast transfer through a conditional generative adversarial network. This indicates that the domain of application of the trained deep regression CNNs can be further expanded to image modalities and contrasts other than those used in training. A combination of our proposed methods with accelerated optimization-based registration algorithms can dramatically enhance the performance of automatic imaging devices and image processing methods of the future.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Geodesic loss, L geo , which is the square of the geodesic distance in SE(3), has been studied for registration problems due to its convexity with respect to pose transformations [32] [33].”
From A Fully Differentiable Framework for 2D/3D Registration and the Projective Spatial Transformers (2024) · cites this paper as [32]
Citations
Cites
Fully convolutional networks for semantic segmentation
Long, J., Shelhamer, E., & Darrell, T. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298965 · Cited by 38,834
Unpaired image-to-image translation using cycle-consistent adversarial networks
Zhu, J.-Y., Park, T., Isola, P., & Efros, A. A. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.244 · Cited by 23,602
Image-to-Image Translation with Conditional Adversarial Networks
Isola, P., Zhu, J.-Y., Zhou, T., & Efros, A. A. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.632 · Cited by 22,930
A survey on deep learning in medical image analysis
Litjens, G., Kooi, T., Bejnordi, B. E., Setio, A. A. A., Ciompi, F., Ghafoorian, M., van der Laak, J. A. W. M., van Ginneken, B., & Sánchez, C. I. (2017) · Medical Image Analysis · doi:10.1016/j.media.2017.07.005 · Cited by 15,332
Fully convolutional networks for semantic segmentation
Shelhamer, E., Long, J., & Darrell, T. (2017) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2016.2572683 · Cited by 11,898
Identity mappings in deep residual networks
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-46493-0_38 · Cited by 10,741
Yushkevich, P. A., Piven, J., Hazlett, H. C., Smith, R. G., Ho, S., Gee, J. C., & Gerig, G. (2006) · NeuroImage · doi:10.1016/j.neuroimage.2006.01.015 · Cited by 8,868
Mutual-information-based registration of medical images: a survey
Pluim, J. P. W., Maintz, J. B. A., & Viergever, M. A. (2003) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2003.815867 · Cited by 2,882
PoseNet: A Convolutional Network for Real-Time 6-DOF Camera Relocalization
Kendall, A., Grimes, M., & Cipolla, R. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.336 · Cited by 2,489
Hill, D. L. G., Batchelor, P. G., Holden, M., & Hawkes, D. J. (2001) · Physics in Medicine and Biology · doi:10.1088/0031-9155/46/3/201 · Cited by 2,093
Greenspan, H. K., van Ginneken, B., & Summers, R. M. (2016) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2016.2553401 · Cited by 1,725
Deformable medical image registration: a survey
Sotiras, A., Davatzikos, C., & Paragios, N. (2013) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2013.2265603 · Cited by 1,587 · cited 1×
“The most widely used transformations in biomedical image registration range from rigid and affine to high-dimensional small or large deformations based on biophysical/biomechanical, elastic, or viscous fluid models [6].”
BackgroundMetrics for 3D Rotations: Comparison and Analysis
Huynh, D. (2009) · Journal of Mathematical Imaging and Vision · doi:10.1007/s10851-009-0161-2 · Cited by 781
A Review of 3d/2d Registration Methods for Image-Guided Interventions
Markelj, P., Tomaževič, D., Likar, B., & Pernuš, F. (2012) · Medical Image Analysis · doi:10.1016/j.media.2010.03.005 · Cited by 746
Render for CNN: Viewpoint Estimation in Images Using CNNs Trained with Rendered 3D Model Views
Su, H., Qi, C. R., Li, Y., & Guibas, L. J. (2015) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.308 · Cited by 682
Quicksilver: Fast predictive image registration – A deep learning approach
Yang, X., Kwitt, R., Styner, M., & Niethammer, M. (2017) · NeuroImage · doi:10.1016/j.neuroimage.2017.07.008 · Cited by 630
A CNN Regression Approach for Real-Time 2D/3D Registration
Miao, S., Wang, Z. J., & Liao, R. (2016) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2016.2521800 · Cited by 514
PROMO: Real‐time prospective motion correction in MRI using image‐based tracking
White, N. S., Roddey, J. C., Shankaranarayanan, A., Han, E. T., Rettmann, D. W., Santos, J. M., Kuperman, J., & Dale, A. M. (2009) · Magnetic Resonance in Medicine · doi:10.1002/mrm.22176 · Cited by 377
6-DoF object pose from semantic keypoints
Pavlakos, G., Zhou, X., Chan, A., Derpanis, K. G., & Daniilidis, K. (2017) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra.2017.7989233 · Cited by 367
Gholipour, A., Rollins, C. K., Velasco‐Annis, C., Ouaalam, A., Akhondi‐Asl, A., Afacan, O., Ortinau, C. M., Clancy, S., Limperopoulos, C., Yang, E., et al. (2017) · Scientific Reports · doi:10.1038/s41598-017-00525-w · Cited by 366
Self-citationTulsiani, S., & Malik, J. (2015) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2015.7298758 · Cited by 353
Gholipour, A., Estroff, J. A., & Warfield, S. K. (2010) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2010.2051680 · Cited by 351
Self-citationGuorong Wu, A., Kim, M., Wang, Q., Munsell, B. C., & Shen, D. G. (2015) · IEEE Transactions on Biomedical Engineering · doi:10.1109/tbme.2015.2496253 · Cited by 283
Brain Functional Localization: A Survey of Image Registration Techniques
Gholipour, A., Kehtarnavaz, N., Briggs, R. W., Devous, M. D., & Gopinath, K. (2007) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2007.892508 · Cited by 270
Self-citationSingle Image 3D Interpreter Network
Wu, J., Xue, T., Lim, J. J., Tian, Y., Tenenbaum, J. B., Torralba, A. B., & Freeman, W. T. (2016) · Lecture notes in computer science · doi:10.1007/978-3-319-46466-4_22 · Cited by 248
Single Image 3D Interpreter Network
Tian, Y., Wu, J., Xue, T., Lim, J. J., Tenenbaum, J. B., Torralba, A. B., & Freeman, W. T. (2016) · DSpace@MIT (Massachusetts Institute of Technology) · Cited by 241
Salehi, S. S. M., Deniz Erdogmus, B., & Gholipour, A. (2017) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2017.2721362 · Cited by 228
Self-citationA Deep Metric for Multimodal Registration
Simonovsky, M., Gutiérrez-Becker, B., Mateus, D., Navab, N., & Nikos Komodakis, A. (2016) · Lecture notes in computer science · doi:10.1007/978-3-319-46726-9_2 · Cited by 224
3D Pose Regression Using Convolutional Neural Networks
Mahendran, S., Ali, H., & Vidal, R. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw.2017.73 · Cited by 210
Fast Volume Reconstruction From Motion Corrupted Stacks of 2D Slices
Kainz, B., Steinberger, M., Wein, W., Kuklisova-Murgasova, M., Malamateniou, C., Keraudren, K., Torsney-Weir, T., Rutherford, M., Aljabar, P., Hajnal, J. V., et al. (2015) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2015.2415453 · Cited by 200
An artificial agent for robust image registration
Liao, R., Miao, S., De Tournemire, P., Grbic, S., Kamen, A., Mansi, T., & Comaniciu, D. (2017) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v31i1.11230 · Cited by 199
A Survey of Medical Image Registration on Multicore and the GPU
Shams, R., Sadeghi, P., Kennedy, R., & Hartley, R. (2010) · IEEE Signal Processing Magazine · doi:10.1109/msp.2009.935387 · Cited by 198
Slice-to-volume medical image registration: A survey
Ferrante, E., & Nikos Paragios, A. (2017) · Medical Image Analysis · doi:10.1016/j.media.2017.04.010 · Cited by 190
Fetal MRI: A technical update with educational aspirations
Gholipour, A., Estroff, J. A., Barnewolt, C. E., Robertson, R. L., Grant, P. E., Gagoski, B. A., Warfield, S. K., Afacan, O., Connolly, S. A., Neil, J. J., et al. (2014) · Concepts in Magnetic Resonance Part A · doi:10.1002/cmr.a.21321 · Cited by 121
Self-citationReal-time automatic fetal brain extraction in fetal MRI by deep learning
Salehi, S. S. M., Hashemi, S. R., Velasco‐Annis, C., Ouaalam, A., Estroff, J. A., Deniz Erdogmus, B., Warfield, S. K., & Gholipour, A. (2018) · IEEE International Symposium on Biomedical Imaging (ISBI) · doi:10.1109/isbi.2018.8363675 · Cited by 102
Self-citationReal-time 2D/3D registration via CNN regression
Miao, S., Wang, Z. J., Zheng, Y., & Liao, R. (2016) · 2016 IEEE 13th International Symposium on Biomedical Imaging (ISBI) · doi:10.1109/isbi.2016.7493536 · Cited by 101
Namburete, A. I. L., Xie, W., Yaqub, M., Zisserman, A., & Noble, J. A. (2018) · Medical Image Analysis · doi:10.1016/j.media.2018.02.006 · Cited by 95
3-D Reconstruction in Canonical Co-Ordinate Space From Arbitrarily Oriented 2-D Images
Hou, B., Khanal, B., Alansary, A., McDonagh, S., Davidson, A., Rutherford, M., Hajnal, J. V., Rueckert, D., Glocker, B., & Kainz, B. (2018) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2018.2798801 · Cited by 90
Marami, B., Salehi, S. S. M., Afacan, O., Scherrer, B., Rollins, C. K., Yang, E., Estroff, J. A., Warfield, S. K., & Gholipour, A. (2017) · NeuroImage · doi:10.1016/j.neuroimage.2017.04.033 · Cited by 81
Self-citationPredicting Slice-to-Volume Transformation in Presence of Arbitrary Subject Motion
Hou, B., Alansary, A., McDonagh, S., Davidson, A., Rutherford, M., Hajnal, J. V., Rueckert, D., Glocker, B., & Kainz, B. (2017) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-66185-8_34 · Cited by 68
Marami, B., Scherrer, B., Afacan, O., Erem, B., Warfield, S. K., & Gholipour, A. (2016) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2016.2555244 · Cited by 40
Self-citationGholipour, A., Polak, M., van der Kouwe, A. J. W., Nevo, E., & Warfield, S. K. (2011) · Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC) · doi:10.1109/iembs.2011.6091385 · Cited by 23
Self-citationKurugol, S., Marami, B., Afacan, O., Warfield, S. K., & Gholipour, A. (2017) · Lecture notes in computer science · doi:10.1007/978-3-319-67564-0_8 · Cited by 6
Self-citationSlow vasomotor fluctuation in fMRI of anesthetized child brain
Kiviniemi, V., Jauhiainen, J., Tervonen, O., P��kk�, E., Oikarinen, J., Vainionp��, V., Rantala, H., & Biswal, B. (2000) · Magnetic Resonance in Medicine · doi:10.1002/1522-2594(200009)44:3
Cited by
Sorted by citations
An overview of deep learning in medical imaging focusing on MRI
Lundervold, A. S., & Lundervold, A. (2019) · Zeitschrift für Medizinische Physik · doi:10.1016/j.zemedi.2018.11.002 · Cited by 2,058
An overview of deep learning in medical imaging focusing on MRI
Lundervold, A. S., & Lundervold, A. (2018) · NORA - Norwegian Open Research Archives · Cited by 1,012
Deep learning in medical image registration: a review
Fu, Y., Lei, Y., Wang, T., Curran, W. J., Liu, T., & Yang, X. (2020) · Physics in Medicine & Biology · doi:10.1088/1361-6560/ab843e · Cited by 661
Deep learning in medical image registration: a survey
Haskins, G., Kruger, U., & Yan, P. (2020) · Machine Vision and Applications · doi:10.1007/s00138-020-01060-x · Cited by 704
MSU-Net: Multi-Scale U-Net for 2D Medical Image Segmentation
Su, R. Z., Zhang, D., Liu, J., & Cheng, C. (2021) · Frontiers in Genetics · doi:10.3389/fgene.2021.639930 · Cited by 177
Metadata
- Authors
- Salehi, S. S. M., Khan, S., Erdogmus, D., & Gholipour, A.
- Affiliations
- Boston Children's Hospital (US)
- Venue
- arXiv
- Publisher
- arXiv
- Published
- 2018
- Volume
- 38
- Issue
- 2
- Pages
- 470-481
- ISSN
- 0278-0062 (ISSN-L)
- arXiv
- 1803.05982
- PMC
- PMC6438698
- OpenAlex
- W2805701966
- Cited by
- 118
- References
- 44
- Citations by year
- 2026: 9 · 2025: 17 · 2024: 19 · 2023: 18 · 2022: 21 · 2021: 15 · 2020: 15 · 2019: 2 · 2018: 2
- FWCI
- 4.25
- Type
- journal article
- Language
- en
- Open access
- green
- www.ncbi.nlm.nih.gov
- Locations
- IEEE Transactions on Medical Imaging (publishedVersion)
- PubMed (publishedVersion)
- PubMed Central (acceptedVersion)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Funders
- McKnight Foundation
- National Institute of Neurological Disorders and Stroke (R01NS106030)
- National Institute of Biomedical Imaging and Bioengineering (R01EB018988)
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Medical Image Segmentation Techniques
- Physical Sciences › Engineering › Biomedical Engineering › Medical Imaging and Analysis
- Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
- Keywords
- fetal brain MRI, slice-to-volume registration, domain generalization, conditional generative adversarial networks, real-time registration
- MeSH terms
- Deep Learning, Algorithms, Brain, Female, Fetus, Humans, Image Processing, Computer-Assisted, Infant, Newborn, Magnetic Resonance Imaging, Pregnancy, Databases, Factual
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
