Deep learning in medical image registration: a survey
Open accessThe DOI matches its Crossref record; 114 of the 114 references it cites were found in a registry or an index.
Abstract
The establishment of image correspondence through robust image registration is critical to many clinical tasks such as image fusion, organ atlas creation, and tumor growth monitoring and is a very challenging problem. Since the beginning of the recent deep learning renaissance, the medical imaging research community has developed deep learning-based approaches and achieved the state-of-the-art in many applications, including image registration. The rapid adoption of deep learning for image registration applications over the past few years necessitates a comprehensive summary and outlook, which is the main scope of this survey. This requires placing a focus on the different research areas as well as highlighting challenges that practitioners face. This survey, therefore, outlines the evolution of deep learning-based medical image registration in the context of both research challenges and relevant innovations in the past few years. Further, this survey highlights future research directions to show how this field may be possibly moved forward to the next level.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Inspiration can be drawn from the applications of 3D/3D and 2D/2D deformable image registration (Chen et al., 2020; Fu et al., 2020; Haskins et al., 2020), where machine learning techniques have dominated most existing research into registration uncertainty.”
From The Impact of Machine Learning on 2D/3D Registration for Image-Guided Interventions: A Systematic Review and Perspective (2021) · cites this paper as Haskins et al., 2020
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“DL for medical image registration has numerous applications, which were listed by some review papers [320–322].”
From Review of deep learning: concepts, CNN architectures, challenges, applications, future directions (2021) · cites this paper as [321]
The cited source says what the claim attributes to it.
In the citing paper
“Though previous reviews about deep learning-based medical image registration literature have been published (19–22), there are still some deficiencies.”
From A review of deep learning-based deformable medical image registration (2022) · cites this paper as [21]
The check wasn’t confident either way, so it gives no verdict — never disputed.
In the citing paper
“On one hand, these reviews lack a summary of the public dataset for benchmarking registration algorithms.”
From A review of deep learning-based deformable medical image registration (2022) · cites this paper as [21]
Experimental — accuracy not yet measuredThe check wasn’t confident either way, so it gives no verdict — never disputed.
In the citing paper
“On the other hand, some details of the selected literature were missing.”
From A review of deep learning-based deformable medical image registration (2022) · cites this paper as [21]
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474
Hochreiter, S., & Schmidhuber, J. (1997) · Neural Computation · doi:10.1162/neco.1997.9.8.1735 · Cited by 102,318
U-Net: Convolutional Networks for Biomedical Image Segmentation
Ronneberger, O., Fischer, P., & Brox, T. (2015) · Lecture notes in computer science · doi:10.1007/978-3-319-24574-4_28 · Cited by 99,404
LeCun, Y., Bengio, Y., & Hinton, G. (2015) · Nature · doi:10.1038/nature14539 · Cited by 84,126
Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks
Ren, S., He, K., Girshick, R., & Sun, J. (2015) · arXiv · doi:10.48550/arxiv.1506.01497 · Cited by 70,812
Fully convolutional networks for semantic segmentation
Long, J., Shelhamer, E., & Darrell, T. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298965 · Cited by 38,834
Human-level control through deep reinforcement learning
Mnih, V., Kavukcuoglu, K., Silver, D., Rusu, A. A., Veness, J., Bellemare, M. G., Graves, A., Riedmiller, M., Fidjeland, A. K., Ostrovski, G., et al. (2015) · Nature · doi:10.1038/nature14236 · Cited by 30,357
Unpaired image-to-image translation using cycle-consistent adversarial networks
Zhu, J.-Y., Park, T., Isola, P., & Efros, A. A. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.244 · Cited by 23,602 · cited 1×
“, CT) [19,46,66,116], regardless of whether or not paired training data are available [121].”
BackgroundImage-to-Image Translation with Conditional Adversarial Networks
Isola, P., Zhu, J.-Y., Zhou, T., & Efros, A. A. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.632 · Cited by 22,930 · cited 1×
“g CT) [20, 55, 77, 139], regardless of whether or not paired training data is available [145].”
BackgroundDeep learning in neural networks: an overview
Schmidhuber, J. (2015) · Neural Networks · doi:10.1016/j.neunet.2014.09.003 · Cited by 18,570
Mastering the game of Go with deep neural networks and tree search
Silver, D., Huang, A., Maddison, C. J., Guez, A., Sifre, L., van den Driessche, G., Schrittwieser, J., Antonoglou, I., Panneershelvam, V., Lanctot, M., et al. (2016) · Nature · doi:10.1038/nature16961 · Cited by 15,612
A survey on deep learning in medical image analysis
Litjens, G., Kooi, T., Bejnordi, B. E., Setio, A. A. A., Ciompi, F., Ghafoorian, M., van der Laak, J. A. W. M., van Ginneken, B., & Sánchez, C. I. (2017) · Medical Image Analysis · doi:10.1016/j.media.2017.07.005 · Cited by 15,332
TensorFlow: A system for large-scale machine learning
Abadi, M., Barham, P., Chen, J., Chen, Z., Davis, A., Dean, J., Devin, M., Ghemawat, S., Irving, G., Isard, M., et al. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1605.08695 · Cited by 11,822
Caffe: Convolutional Architecture for Fast Feature Embedding
Jia, Y., Shelhamer, E., Donahue, J., Karayev, S., Long, J., Girshick, R., Guadarrama, S., & Darrell, T. (2014) · Proceedings of the 22nd ACM international conference on Multimedia · doi:10.1145/2647868.2654889 · Cited by 11,077
Mastering the game of Go without human knowledge
Silver, D., Schrittwieser, J., Simonyan, K., Antonoglou, I., Huang, A., Guez, A., Hubert, T., Baker, L., Lai, M., Bolton, A., et al. (2017) · Nature · doi:10.1038/nature24270 · Cited by 9,084
Reinforcement Learning: A Survey
Kaelbling, L. P., Littman, M. L., & Moore, A. W. (1996) · Journal of Artificial Intelligence Research · doi:10.1613/jair.301 · Cited by 8,972
Image registration methods: a survey
Zitová, B., & Flusser, J. (2003) · Image and Vision Computing · doi:10.1016/s0262-8856(03)00137-9 · Cited by 6,423
AVANTS, B., EPSTEIN, C., GROSSMAN, M., & GEE, J. (2008) · Medical Image Analysis · doi:10.1016/j.media.2007.06.004 · Cited by 5,669
A reproducible evaluation of ANTs similarity metric performance in brain image registration
Avants, B. B., Tustison, N. J., Song, G., Cook, P. A., Klein, A., & Gee, J. C. (2011) · NeuroImage · doi:10.1016/j.neuroimage.2010.09.025 · Cited by 4,796
Multimodality image registration by maximization of mutual information
Maes, F., Collignon, A., Vandermeulen, D., Marchal, G., & Suetens, P. (1997) · IEEE Transactions on Medical Imaging · doi:10.1109/42.563664 · Cited by 4,547
Flownet: Learning optical flow with convolutional networks
Dosovitskiy, A., Fischer, P., Ilg, E., Hausser, P., Hazirbas, C., Golkov, V., Smagt, P. V. D., Cremers, D., & Brox, T. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.316 · Cited by 4,313
Elastix: a toolbox for intensity-based medical image registration
Klein, S., Staring, M., Murphy, K., Viergever, M. A., & Pluim, J. (2010) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2009.2035616 · Cited by 4,311
StarGAN: Unified Generative Adversarial Networks for Multi-domain Image-to-Image Translation
Choi, Y., Choi, M., Kim, M., Ha, J.-W., Kim, S., & Choo, J. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00916 · Cited by 4,163
Dueling Network Architectures for Deep Reinforcement Learning
Wang, Z., Schaul, T., Hessel, M., van Hasselt, H., Lanctot, M., & de Freitas, N. (2015) · arXiv · doi:10.48550/arxiv.1511.06581 · Cited by 3,300
Alignment by Maximization of Mutual Information
Viola, P., & Wells III, W. M. (1997) · International Journal of Computer Vision · doi:10.1023/a:1007958904918 · Cited by 2,610
DualGAN: Unsupervised Dual Learning for Image-to-Image Translation
Yi, Z., Zhang, H., Tan, P., & Gong, M. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.310 · Cited by 2,134
Hill, D. L. G., Batchelor, P. G., Holden, M., & Hawkes, D. J. (2001) · Physics in Medicine and Biology · doi:10.1088/0031-9155/46/3/201 · Cited by 2,093
MXNet: A Flexible and Efficient Machine Learning Library for Heterogeneous Distributed Systems
Chen, T., Li, M., Li, Y., Lin, M., Wang, N., Wang, M., Xiao, T., Xu, B., Zhang, C., & Zhang, Z. (2015) · arXiv · doi:10.48550/arxiv.1512.01274 · Cited by 1,819
Image reconstruction by domain-transform manifold learning
Zhu, B., Liu, J. Z., Cauley, S. F., Rosen, B. R., & Rosen, M. S. (2018) · Nature · doi:10.1038/nature25988 · Cited by 1,694
Yang, Q., Yan, P., Zhang, Y., Yu, H., Shi, Y., Mou, X., Kalra, M. K., Zhang, Y., Sun, L., & Wang, G. (2018) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2018.2827462 · Cited by 1,630
Self-citationDiffeomorphic demons: Efficient non-parametric image registration
Vercauteren, T., Pennec, X., Perchant, A., & Ayache, N. (2009) · NeuroImage · doi:10.1016/j.neuroimage.2008.10.040 · Cited by 1,592
SIFT Flow: Dense Correspondence across Scenes and Its Applications
Liu, C., Yuen, J., & Torralba, A. (2011) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2010.147 · Cited by 1,582
Tutorial on Variational Autoencoders
Doersch, C. (2016) · arXiv · doi:10.48550/arxiv.1606.05908 · Cited by 1,470
PCANet: A Simple Deep Learning Baseline for Image Classification?
Chan, T.-H., Jia, K., Gao, S., Lu, J., Zeng, Z., & Ma, Y. (2015) · IEEE Transactions on Image Processing · doi:10.1109/tip.2015.2475625 · Cited by 1,362
Deep Learning in Medical Imaging: General Overview
Lee, J.-G., Jun, S., Cho, Y.-W., Lee, H., Kim, G. B., Seo, J. B., & Kim, N. (2017) · Korean Journal of Radiology · doi:10.3348/kjr.2017.18.4.570 · Cited by 1,276
Phase recovery and holographic image reconstruction using deep learning in neural networks
Rivenson, Y., Zhang, Y., Günaydın, H., Teng, D., & Ozcan, A. (2017) · Light: Science & Applications · doi:10.1038/lsa.2017.141 · Cited by 1,059
A deep learning framework for unsupervised affine and deformable image registration
de Vos, B. D., Berendsen, F. F., Viergever, M. A., Sokooti, H., Staring, M., & Išgum, I. (2019) · Medical Image Analysis · doi:10.1016/j.media.2018.11.010 · Cited by 818
MIND: Modality independent neighbourhood descriptor for multi-modal deformable registration
Heinrich, M. P., Jenkinson, M., Bhushan, M., Matin, T., Gleeson, F. V., Brady, S. M., & Schnabel, J. A. (2012) · Medical Image Analysis · doi:10.1016/j.media.2012.05.008 · Cited by 707
An Unsupervised Learning Model for Deformable Medical Image Registration
Balakrishnan, G., Zhao, A., Sabuncu, M. R., Dalca, A. V., & Guttag, J. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00964 · Cited by 599
Viewpoint: when will AI exceed human performance? evidence from AI experts
Grace, K., Salvatier, J., Dafoe, A., Zhang, B., & Evans, O. (2018) · Journal of Artificial Intelligence Research · doi:10.1613/jair.1.11222 · Cited by 579
A CNN Regression Approach for Real-Time 2D/3D Registration
Miao, S., Wang, Z. J., & Liao, R. (2016) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2016.2521800 · Cited by 514
Weakly-supervised convolutional neural networks for multimodal image registration
Hu, Y., Modat, M., Gibson, E., Li, W., Ghavami, N., Bonmati, E., Wang, G., Bandula, S., Moore, C. M., Emberton, M., et al. (2018) · Medical Image Analysis · doi:10.1016/j.media.2018.07.002 · Cited by 458
End-to-End Unsupervised Deformable Image Registration with a Convolutional Neural Network
de Vos, B. D., Berendsen, F. F., Viergever, M. A., Staring, M., & Išgum, I. (2017) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-67558-9_24 · Cited by 450
The History Began from AlexNet: A Comprehensive Survey on Deep Learning Approaches
Alom, M. Z., Taha, T. M., Yakopcic, C., Westberg, S. K. E., Sidike, P., Nasrin, M. S., Essen, B. C. V., Awwal, A. A. S., & Asari, V. K. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1803.01164 · Cited by 426
Nonrigid Image Registration Using Multi-scale 3D Convolutional Neural Networks
Sokooti, H., de Vos, B., Berendsen, F., Lelieveldt, B. P. F., Išgum, I., & Staring, M. (2017) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-66182-7_27 · Cited by 340
Svf-net: learning deformable image registration using shape matching
Rohé, M.-M., Datar, M., Heimann, T., Sermesant, M., & Pennec, X. (2017) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-66182-7_31 · Cited by 330 · cited 1×
“The following year, Rohe et al. [102] also used a U-net [103] inspired network to estimate the deformation field used to register 3D cardiac MR volumes.”
MethodElastic volume reconstruction from series of ultra-thin microscopy sections
Saalfeld, S., Fetter, R., Cardona, A., & Tomancak, P. (2012) · Nature Methods · doi:10.1038/nmeth.2072 · Cited by 329
Consistent and Elastic Registration of Histological Sections Using Vector-Spline Regularization
Arganda-Carreras, I., Sorzano, C. O. S., Marabini, R., Carazo, J. M., Ortiz-de-Solorzano, C., & Kybic, J. (2006) · Lecture Notes in Computer Science · doi:10.1007/11889762_8 · Cited by 325
Intensity-Based Image Registration by Minimizing Residual Complexity
Myronenko, A., & Song, X. (2010) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2010.2053043 · Cited by 325
Applications of deep learning to MRI images: A survey
Liu, J., Pan, Y., Li, M., Chen, Z., Tang, L., Lu, C., & Wang, J. (2018) · Big Data Mining and Analytics · doi:10.26599/bdma.2018.9020001 · Cited by 291
When Will AI Exceed Human Performance? Evidence from AI Experts
Grace, K., Salvatier, J., Dafoe, A., Zhang, B., & Evans, O. (2017) · arXiv · doi:10.48550/arxiv.1705.08807 · Cited by 285 · cited 1×
“These approaches have been applied to various video/board games and have often been able to demonstrate superhuman performance [15, 40, 41, 112, 113, 124].”
BackgroundWu, G., Kim, M., Wang, Q., Munsell, B. C., & Shen, D. (2016) · IEEE Transactions on Biomedical Engineering · doi:10.1109/tbme.2015.2496253 · Cited by 278
Deformable Image Registration Based on Similarity-Steered CNN Regression
Cao, X., Yang, J., Zhang, J., Nie, D., Kim, M., Wang, Q., & Shen, D. (2017) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-66182-7_35 · Cited by 225
Robust non-rigid registration through agent-based action learning
Krebs, J., Mansi, T., Delingette, H., Zhang, L., Ghesu, F. C., Miao, S., Maier, A. K., Ayache, N., Liao, R., & Kamen, A. (2017) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-66182-7_40 · Cited by 225
Characterization and computation of local Nash equilibria in continuous games
Ratliff, L. J., Burden, S. A., & Sastry, S. S. (2013) · 2013 51st Annual Allerton Conference on Communication, Control, and Computing (Allerton) · doi:10.1109/allerton.2013.6736623 · Cited by 223
An artificial agent for robust image registration
Liao, R., Miao, S., De Tournemire, P., Grbic, S., Kamen, A., Mansi, T., & Comaniciu, D. (2017) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v31i1.11230 · Cited by 199
Diffeomorphic 3D Image Registration via Geodesic Shooting Using an Efficient Adjoint Calculation
Vialard, F.-X., Risser, L., Rueckert, D., & Cotter, C. J. (2012) · International Journal of Computer Vision · doi:10.1007/s11263-011-0481-8 · Cited by 198
Li, H., & Fan, Y. (2018) · 2018 IEEE 15th International Symposium on Biomedical Imaging (ISBI 2018) · doi:10.1109/isbi.2018.8363757 · Cited by 188
A Deep Metric for Multimodal Registration
Simonovsky, M., Gutiérrez-Becker, B., Mateus, D., Navab, N., & Komodakis, N. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-46726-9_2 · Cited by 185
Towards realtime multimodal fusion for image-guided interventions using self-similarities
Heinrich, M. P., Jenkinson, M., Papież, B. W., Brady, S. M., & Schnabel, J. A. (2013) · Lecture Notes in Computer Science · doi:10.1007/978-3-642-40811-3_24 · Cited by 176
Pulmonary CT Registration Through Supervised Learning With Convolutional Neural Networks
Eppenhof, K. A. J., & Pluim, J. P. W. (2019) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2018.2878316 · Cited by 173
Fast fit-free analysis of fluorescence lifetime imaging via deep learning
Smith, J. T., Yao, R., Sinsuebphon, N., Rudkouskaya, A., Un, N., Mazurkiewicz, J., Barroso, M., Yan, P., & Intes, X. (2019) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1912707116 · Cited by 171
Self-citationLCC-Demons: A robust and accurate symmetric diffeomorphic registration algorithm
Lorenzi, M., Ayache, N., Frisoni, G. B., & Pennec, X. (2013) · NeuroImage · doi:10.1016/j.neuroimage.2013.04.114 · Cited by 162
Deep similarity learning for multimodal medical images
Cheng, X., Zhang, L., & Zheng, Y. (2018) · Computer Methods in Biomechanics and Biomedical Engineering: Imaging & Visualization · doi:10.1080/21681163.2015.1135299 · Cited by 157
Image registration by local histogram matching
Shen, D. G. (2006) · Pattern Recognition · doi:10.1016/j.patcog.2006.08.012 · Cited by 154
Unsupervised Deep Feature Learning for Deformable Registration of MR Brain Images
Wu, G., Kim, M., Wang, Q., Gao, Y., Liao, S., & Shen, D. (2013) · Lecture Notes in Computer Science · doi:10.1007/978-3-642-40763-5_80 · Cited by 149 · cited 2×
“[109,110] were the first to use deep learning to obtain an application-specific similarity metric for registration.”
MethodLearning deep similarity metric for 3D MR–TRUS image registration
Haskins, G., Kruecker, J., Kruger, U., Xu, S., Pinto, P. A., Wood, B. J., & Yan, P. (2019) · International Journal of Computer Assisted Radiology and Surgery · doi:10.1007/s11548-018-1875-7 · Cited by 140
Self-citationInverse-Consistent Deep Networks for Unsupervised Deformable Image RegistrationPreprint
Zhang, J. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1809.03443 · Cited by 138
Fast Predictive Image Registration
Yang, X., Kwitt, R., & Niethammer, M. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-46976-8_6 · Cited by 127
Adversarial Similarity Network for Evaluating Image Alignment in Deep Learning Based Registration
Fan, J., Cao, X., Xue, Z., Yap, P.-T., & Shen, D. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-00928-1_83 · Cited by 124
Deep Learning Based Inter-modality Image Registration Supervised by Intra-modality Similarity
Cao, X., Yang, J., Wang, L., Xue, Z., Wang, Q., & Shen, D. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-00919-9_7 · Cited by 123
Label-driven weakly-supervised learning for multimodal deformarle image registration
Hu, Y., Modat, M., Gibson, E., Ghavami, N., Bonmati, E., Moore, C. M., Emberton, M., Noble, J. A., Barratt, D. C., & Vercauteren, T. (2018) · 2018 IEEE 15th International Symposium on Biomedical Imaging (ISBI 2018) · doi:10.1109/isbi.2018.8363756 · Cited by 122
Real-time Deep Pose Estimation with Geodesic Loss for Image-to-Template Rigid Registration
Salehi, S. S. M., Khan, S., Erdogmus, D., & Gholipour, A. (2018) · arXiv · doi:10.48550/arxiv.1803.05982 · Cited by 118 · cited 2×
“[90] Synthetic transforms Rigid MR Brain 11-layer CNN ResNet-18”
MethodDe Silva, T., Uneri, A., Ketcha, M. D., Reaungamornrat, S., Kleinszig, G., Vogt, S., Aygun, N., Lo, S.-F., Wolinsky, J.-P., & Siewerdsen, J. H. (2016) · Physics in Medicine & Biology · doi:10.1088/0031-9155/61/8/3009 · Cited by 116 · cited 1×
“They were able to significantly outperform registration approaches that used a state-of-the-art similarity metric given by [24].”
Training cnns for image registration from few samples with model-based data augmentation
Uzunova, H., Wilms, M., Handels, H., & Ehrhardt, J. (2017) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-66182-7_26 · Cited by 110 · cited 1×
“Unlike the above works, Uzunova et al. [125] generated ground truth data using statistical appearance models (SAMs).”
ResultNon-rigid image registration using fully convolutional networks with deep self-supervisionPreprint
Li, H., & Fan, Y. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1709.00799 · Cited by 103
Real-time 2D/3D registration via CNN regression
Miao, S., Wang, Z. J., Zheng, Y., & Liao, R. (2016) · 2016 IEEE 13th International Symposium on Biomedical Imaging (ISBI) · doi:10.1109/isbi.2016.7493536 · Cited by 101
AIRNet: Self-Supervised Affine Registration for 3D Medical Images using Neural NetworksPreprint
Chee, E. Y. L., & Wu, Z. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1810.02583 · Cited by 88
Multimodal image registration with deep context reinforcement learning
Ma, K., Wang, J., Singh, V., Tamersoy, B., Chang, Y.-J., Wimmer, A., & Chen, T. (2017) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-66182-7_28 · Cited by 87 · cited 4×
“[71] used a reinforcement learning approach to perform the rigid registration of MR/CT chest volumes.”
BackgroundMethodInfluentialHighly accurate fast lung CT registration
Rühaak, J., Heldmann, S., Kipshagen, T., & Fischer, B. (2013) · Medical Imaging · doi:10.1117/12.2006035 · Cited by 82
Lv, J., Yang, M., Zhang, J., & Wang, X. (2018) · The British Journal of Radiology · doi:10.1259/bjr.20170788 · Cited by 76
Joint Registration And Segmentation Of Xray Images Using Generative Adversarial Networks
Mahapatra, D., Ge, Z., Sedai, S., & Chakravorty, R. (2018) · Lecture notes in computer science · doi:10.1007/978-3-030-00919-9_9 · Cited by 73
Unsupervised Probabilistic Deformation Modeling for Robust Diffeomorphic Registration
Krebs, J., Mansi, T., Mailhé, B., Ayache, N., & Delingette, H. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-00889-5_12 · Cited by 72
Yoo, I., Hildebrand, D. G. C., Tobin, W. F., Lee, W.-C. A., & Jeong, W.-K. (2017) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-67558-9_29 · Cited by 67
Sloan, J. M., Goatman, K. A., & Siebert, J. P. (2018) · Proceedings of the 11th International Joint Conference on Biomedical Engineering Systems and Technologies · doi:10.5220/0006543700890099 · Cited by 59
Punithakumar, K., Boulanger, P., & Noga, M. (2017) · IEEE Access · doi:10.1109/access.2017.2755863 · Cited by 55
Pairwise domain adaptation module for CNN-based 2-D/3-D registration
Zheng, J., Miao, S., Jane Wang, Z., & Liao, R. (2018) · Journal of Medical Imaging · doi:10.1117/1.jmi.5.2.021204 · Cited by 54
A History of the Shift Toward Full Computerization of Medicine
Ambinder, E. P. (2005) · Journal of Oncology Practice · doi:10.1200/jop.2005.1.2.54 · Cited by 52
On the Adaptability of Unsupervised CNN-Based Deformable Image Registration to Unseen Image Domains
Ferrante, E., Oktay, O., Glocker, B., & Milone, D. H. (2018) · Lecture notes in computer science · doi:10.1007/978-3-030-00919-9_34 · Cited by 52
Deep deformable registration: Enhancing accuracy by fully convolutional neural net
Ghosal, S., & Ray, N. (2017) · Pattern Recognition Letters · doi:10.1016/j.patrec.2017.05.022 · Cited by 38
Linear and Deformable Image Registration with 3D Convolutional Neural Networks
Stergios, C., Mihir, S., Maria, V., Guillaume, C., Marie-Pierre, R., Stavroula, M., & Nikos, P. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-00946-5_2 · Cited by 38
GAN Based Medical Image RegistrationPreprint
Mahapatra, D. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1805.02369 · Cited by 35
Deformable MRI-Ultrasound Registration Using 3D Convolutional Neural Network
Sun, L., & Zhang, S. (2018) · Lecture notes in computer science · doi:10.1007/978-3-030-01045-4_18 · Cited by 33
Efficient Acceleration of Mutual Information Computation for Nonrigid Registration Using CUDA
Ikeda, K., Ino, F., & Hagihara, K. (2014) · IEEE Journal of Biomedical and Health Informatics · doi:10.1109/jbhi.2014.2310745 · Cited by 32
LSTM Spatial Co-transformer Networks for Registration of 3D Fetal US and MR Brain Images
Wright, R., Khanal, B., Gómez, A., Skelton, E., Matthew, J., Hajnal, J. V., Rueckert, D., & Schnabel, J. A. (2018) · Lecture notes in computer science · doi:10.1007/978-3-030-00807-9_15 · Cited by 30
A neural network approach for fast, automated quantification of DIR performance
Neylon, J., Min, Y., Low, D. A., & Santhanam, A. P. (2017) · Medical Physics · doi:10.1002/mp.12321 · Cited by 28
Semi-Supervised Deep Metrics for Image RegistrationPreprint
Sedghi, A., Luo, J., Mehrtash, A., Pieper, S., Tempany, C. M., Kapur, T., Mousavi, P., & Wells, W. M. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1804.01565 · Cited by 24
Towards Robust CT-Ultrasound Registration Using Deep Learning Methods
Sun, Y., Moelker, A. D., Niessen, W. J., & van Walsum, T. (2018) · Lecture notes in computer science · doi:10.1007/978-3-030-02628-8_5 · Cited by 21
An unsupervised network for fast microscopic image registration
Shu, C., Chen, X., Xie, Q., & Han, H. (2018) · SPIE Medical Imaging: Digital and Computational Pathology · doi:10.1117/12.2293264 · Cited by 20
Nazib, A., Fookes, C., & Perrin, D. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1810.08315 · Cited by 20
BIRNet: Brain Image Registration Using Dual-Supervised Fully Convolutional NetworksPreprint
Fan, J., Cao, X., Yap, P., & Shen, D. G. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1802.04692 · Cited by 17
CNN Driven Sparse Multi-level B-Spline Image Registration
Jiang, P., & Shackleford, J. (2018) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2018.00967 · Cited by 16
Dilated FCN for Multi-Agent 2D/3D Medical Image RegistrationPreprint
Miao, S., Piat, S., Fischer, P., et al. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1712.01651 · Cited by 15
Semi-coupled dictionary learning for deformation prediction
Cao, T., Singh, N., Jojic, V., & Niethammer, M. (2015) · 2015 IEEE 12th International Symposium on Biomedical Imaging (ISBI) · doi:10.1109/isbi.2015.7163967 · Cited by 13
FAIM -- A ConvNet Method for Unsupervised 3D Medical Image RegistrationPreprint
Kuang, D., & Schmah, T. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1811.09243 · Cited by 13
Conv2Warp: An Unsupervised Deformable Image Registration with Continuous Convolution and Warping
Ali, S., & Rittscher, J. (2019) · Lecture notes in computer science · doi:10.1007/978-3-030-32692-0_56 · Cited by 11
Ehrhardt, J., Schmidt-Richberg, A., Werner, R., & Heinz Handels, A. (2015) · Informatik aktuell · doi:10.1007/978-3-662-46224-9_37 · Cited by 11
An Automated Method for Generating Training Sets for Deep Learning based Image Registration
Ito, M., & Ino, F. (2018) · International Joint Conference on Biomedical Engineering Systems and Technologies (BIOSTEC) · doi:10.5220/0006634501400147 · Cited by 9
Deep compressive macroscopic fluorescence lifetime imaging
Yao, R., Ochoa, M., Intes, X., & Yan, P. (2018) · IEEE International Symposium on Biomedical Imaging (ISBI) · doi:10.1109/isbi.2018.8363718 · Cited by 9
Self-citationAdversarial Image Registration with Application for MR and TRUS Image FusionPreprint
Yan, P., Xu, S., Rastinehad, A. R., et al. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1804.11024 · Cited by 9
Self-citationTensor-based descriptor for image registration via unsupervised network
Liu, Q., & Leung, H. (2017) · International Conference on Information Fusion (FUSION) · doi:10.23919/icif.2017.8009689 · Cited by 7
A Perspective on Deep ImagingPreprint
Wang, G. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1609.04375 · Cited by 6
Hering, A., Kuckertz, S., Heldmann, S., et al. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1812.01859 · Cited by 2
Deep Learning based Inter-Modality Image Registration Supervised by Intra-Modality SimilarityPreprint
Cao, X., Yang, J., Wang, L., et al. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1804.10735 · Cited by 1
Research works printed but not linked: 14
- 9. Balakrishnan, G., Zhao, A., Sabuncu, M. R., Guttag, J., and Dalca, A. V. (2018b). Voxelmorph: A learning framework for deformable medical image registration. arXiv preprint arXiv:1809.05231 . · arXiv
- 10. Blendowski, M. and Heinrich, M. P. (2018). Combining mrf-based deformable registration and deep binary 3d-cnn descriptors for large lung motion estimation in copd patients. International journal of computer assisted radiology and surgery , pages 1-10.
- 17. Cheng, X., Zhang, L., and Zheng, Y. (2016). Deep similarity learning for multimodal medical images. In International conference on medical image computing and computer-assisted intervention .
- 21. Dalca, A. V., Balakrishnan, G., Guttag, J., and Sabuncu, M. R. (2018). Unsupervised learning for fast probabilistic diffeomorphic registration. arXiv preprint arXiv:1805.04605 . · arXiv
- 29. Eppenhof, K. A. J. and Pluim, J. P. (2018b). Error estimation of deformable image registration of pulmonary ct scans using convolutional neural networks. Journal of Medical Imaging , 5(2):024003.
- 35. Goodfellow, I., Pouget-Abadie, J., Mirza, M., Xu, B., Warde-Farley, D., Ozair, S., Courville, A., and Bengio, Y. (2014). Generative adversarial nets. In Advances in neural information processing systems , pages 2672-2680.
- 42. Hu, Y., Gibson, E., Ghavami, N., Bonmati, E., Moore, C. M., Emberton, M., Vercauteren, T., Noble, J. A., and Barratt, D. C. (2018a). Adversarial deformation regularization for training image registration neural networks. arXiv preprint arXiv:1805.10665 . · arXiv
- 48. Jaderberg, M., Simonyan, K., Zisserman, A., et al. (2015). Spatial transformer networks. In Advances in neural information processing systems , pages 2017-2025.
- 52. Kazeminia, S., Baur, C., Kuijper, A., van Ginneken, B., Navab, N., Albarqouni, S., and Mukhopadhyay, A. (2018). Gans for medical image analysis. arXiv preprint arXiv:1809.06222 . · arXiv
- 56. Krebs, J., Mansi, T., Mailh´ e, B., Ayache, N., and Delingette, H. (2018a). Learning structured deformations using diffeomorphic registration. arXiv preprint arXiv:1804.07172 . · arXiv
- 66. Liu, M.-Y., Breuel, T., and Kautz, J. (2017). Unsupervised image-to-image translation networks. In Advances in Neural Information Processing Systems , pages 700-708.
- 82. Paszke, A., Gross, S., Chintala, S., Chanan, G., Yang, E., DeVito, Z., Lin, Z., Desmaison, A., Antiga, L., and Lerer, A. (2017). Automatic differentiation in pytorch. In NIPS-W .
- 93. Sheikhjafari, A., Noga, M., Punithakumar, K., and Ray, N. (2018). Unsupervised deformable image registration with fully connected generative neural network. In International conference on Medical Imaging with Deep Learning .
- 113. Yang, X. (2017). Uncertainty Quantification, Image Synthesis and Deformation Prediction for Image Registration . PhD thesis, The University of North Carolina at Chapel Hill.
Cited by
Review of deep learning: concepts, CNN architectures, challenges, applications, future directions
Alzubaidi, L., Zhang, J., Humaidi, A. J., Al-Dujaili, A., Duan, Y., Al-Shamma, O., Santamaría, J., Fadhel, M. A., Al-Amidie, M., & Farhan, L. (2021) · Journal of Big Data · doi:10.1186/s40537-021-00444-8 · Cited by 7,906
Transformers in medical imaging: A survey
Shamshad, F., Khan, S., Zamir, S. W., Khan, M. H., Hayat, M., Khan, F. S., & Fu, H. (2023) · Medical Image Analysis · doi:10.1016/j.media.2023.102802 · Cited by 1,296
Zhou, S. K., Greenspan, H., Davatzikos, C., Duncan, J. S., Van Ginneken, B., Madabhushi, A., Prince, J. L., Rueckert, D., & Summers, R. M. (2021) · Proceedings of the IEEE · doi:10.1109/jproc.2021.3054390 · Cited by 1,118
“There are many survey papers on DL-based key technologies for medical image analysis [22]–[31].”
BackgroundRecent advances and clinical applications of deep learning in medical image analysis
Chen, X., Wang, X., Zhang, K., Fung, K.-M., Thai, T. C., Moore, K., Mannel, R. S., Liu, H., Zheng, B., & Qiu, Y. (2022) · Medical Image Analysis · doi:10.1016/j.media.2022.102444 · Cited by 1,080
“…, deep learning models learn a metric that quantifies the similarity between a target/moving image and a reference/fixed image; then the learned similarity metric is used in conjunction with classical (i.e., non-learning-based) frameworks to perform image registration (Haskins et al., 2020).”
BackgroundMethodInfluentialImage Matching from Handcrafted to Deep Features: A Survey
Ma, J., Jiang, X., Fan, A., Jiang, J., & Yan, J. (2021) · International Journal of Computer Vision · doi:10.1007/s11263-020-01359-2 · Cited by 990
“See more details in the registration-specific reviews (Ferrante and Paragios 2017; Haskins et al. 2020).”
Background
Metadata
- Authors
- Haskins, G., Kruger, U., & Yan, P.
- Affiliations
- Rensselaer Polytechnic Institute (US)
- Corresponding author
- Yan, Pingkun
- ORCID
- Yan, Pingkun
- Venue
- Machine Vision and Applications
- Published
- 2020-01-29
- Volume
- 31
- Issue
- 1-2
- ISSN
- 0932-8092 (ISSN-L)
- 1432-1769
- OpenAlex
- W2918257301
- Cited by
- 704
- References
- 98
- Citations by year
- 2026: 48 · 2025: 90 · 2024: 97 · 2023: 113 · 2022: 120 · 2021: 141 · 2020: 82
- FWCI
- 42.88
- Type
- journal article
- Language
- en
- Update notices
- correction (2020-02-27)
- Open access
- green
- arxiv.org
- Licence
- www.springer.com/tdm (tdm)
- www.springer.com/tdm (vor)
- Locations
- Machine Vision and Applications (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Funders
- National Institute of Biomedical Imaging and Bioengineering (R21EB028001, R01EB027898)
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Medical Image Segmentation Techniques
- Physical Sciences › Computer Science › Artificial Intelligence › Domain Adaptation and Few-Shot Learning
- Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
- Keywords
- medical image registration, deep learning, image correspondence, image fusion, tumor growth monitoring, future research directions
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.