Dilated FCN for Multi-Agent 2D/3D Medical Image Registration
Open accessAbstract
2D/3D image registration to align a 3D volume and 2D X-ray images is a challenging problem due to its ill-posed nature and various artifacts presented in 2D X-ray images. In this paper, we propose a multi-agent system with an auto attention mechanism for robust and efficient 2D/3D image registration. Specifically, an individual agent is trained with dilated Fully Convolutional Network (FCN) to perform registration in a Markov Decision Process (MDP) by observing a local region, and the final action is then taken based on the proposals from multiple agents and weighted by their corresponding confidence levels. The contributions of this paper are threefold. First, we formulate 2D/3D registration as a MDP with observations, actions, and rewards properly defined with respect to X-ray imaging systems. Second, to handle various artifacts in 2D X-ray images, multiple local agents are employed efficiently via FCN-based structures, and an auto attention mechanism is proposed to favor the proposals from regions with more reliable visual cues. Third, a dilated FCN-based training mechanism is proposed to significantly reduce the Degree of Freedom in the simulation of registration environment, and drastically improve training efficiency by an order of magnitude compared to standard CNN-based training method. We demonstrate that the proposed method achieves high robustness on both spine cone beam Computed Tomography data with a low signal-to-noise ratio and data from minimally invasive spine surgery where severe image artifacts and occlusions are presented due to metal screws and guide wires, outperforming other state-of-the-art methods (single agent-based and optimization-based) by a large margin.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Among seven studies that trained deep CNNs on synthetic data and evaluated comparably on synthetic and real data, substantial performance drops were found.
“Indeed, among the seven studies that trained deep CNNs on synthetic data and evaluated in a way that allowed for comparisons between synthetic and real data performance, we found quite substantial performance drops (Miao et al., 2018; Bier et al., 2019; Gao et al., 2020c; Doerr et al., 2020; Gu et al., 2020; Guan et al., 2020; Li et al., 2020).”
From The Impact of Machine Learning on 2D/3D Registration for Image-Guided Interventions: A Systematic Review and Perspective (2021) · cites this paper as Miao et al., 2018
The Training section says the model used both CBCT-derived X-ray/DRR pairs and additional synthetic pairs. The Testing section and Table 2 report worse results on clinical data than on CBCT data, including a substantial rise in gross failure rate for the single-agent method. But the CBCT evaluation used real X-ray images, not synthetic test data, so this paper does not establish a synthetic-versus-real performance drop as phrased. (LLM, conf=0.86)
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
A fast and elitist multiobjective genetic algorithm: NSGA-II
Deb, K., Pratap, A., Agarwal, S., & Meyarivan, T. (2002) · IEEE Transactions on Evolutionary Computation · doi:10.1109/4235.996017 · Cited by 50,053 · see citing works
Fully convolutional networks for semantic segmentation
Long, J., Shelhamer, E., & Darrell, T. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298965 · Cited by 38,834 · see citing works
Multimodality image registration by maximization of mutual information
Maes, F., Collignon, A., Vandermeulen, D., Marchal, G., & Suetens, P. (1997) · IEEE Transactions on Medical Imaging · doi:10.1109/42.563664 · Cited by 4,547 · see citing works
Flownet: Learning optical flow with convolutional networks
Dosovitskiy, A., Fischer, P., Ilg, E., Hausser, P., Hazirbas, C., Golkov, V., Smagt, P. V. D., Cremers, D., & Brox, T. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.316 · Cited by 4,313 · see citing works
Self-citationFlowNet 2.0: Evolution of Optical Flow Estimation with Deep Networks
Ilg, E., Mayer, N., Saikia, T., Keuper, M., Dosovitskiy, A., & Brox, T. (2017) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2017.179 · Cited by 3,177 · see citing works
Fully Convolutional Networks for Semantic SegmentationPreprint
Long, J., Shelhamer, E., & Darrell, T. J. (2014) · arXiv (Cornell University) · doi:10.48550/arxiv.1411.4038 · Cited by 2,490 · see citing works
The Essential Physics of Medical Imaging
Bushberg, J. T., Seibert, J. A., Leidholdt, E. M., Boone, J. M., & Goldschmidt, E. J. (2003) · Medical Physics · doi:10.1118/1.1585033 · Cited by 2,088 · see citing works
A Fast and Elitist Multiobjective Genetic Algorithm: NSGA-II
Deb, K. (2002) · Zenodo (CERN European Organization for Nuclear Research) · doi:10.5281/zenodo.6487416 · Cited by 1,552 · see citing works
DeepFlow: Large Displacement Optical Flow with Deep Matching
Weinzaepfel, P., Revaud, J., Harchaoui, Z., & Schmid, C. (2013) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2013.175 · Cited by 1,019 · see citing works
Multi-modality image registration by maximization of mutual information
Maes, F., Collignon, A. M. F., Vandermeulen, D., Marchal, G., & Suetens, P. (1996) · IEEE Workshop on Mathematical Methods in Biomedical Image Analysis (MMBIA) · doi:10.1109/mmbia.1996.534053 · Cited by 809 · see citing works
A Review of 3d/2d Registration Methods for Image-Guided Interventions
Markelj, P., Tomaževič, D., Likar, B., & Pernuš, F. (2012) · Medical Image Analysis · doi:10.1016/j.media.2010.03.005 · Cited by 746 · see citing works
A CNN Regression Approach for Real-Time 2D/3D Registration
Miao, S., Wang, Z. J., & Liao, R. (2016) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2016.2521800 · Cited by 514 · see citing works
Self-citationLearning descriptors for object recognition and 3D pose estimation
Wohlhart, P., & Lepetit, V. (2015) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2015.7298930 · Cited by 452 · see citing works
An artificial agent for robust image registration
Liao, R., Miao, S., De Tournemire, P., Grbic, S., Kamen, A., Mansi, T., & Comaniciu, D. (2017) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v31i1.11230 · Cited by 199 · see citing works
Self-citationL1 rotation averaging using the Weiszfeld algorithm
Hartley, R. I., Aftab, K., & Trumpf, J. (2011) · doi:10.1109/cvpr.2011.5995745 · Cited by 179 · see citing works
De Silva, T., Uneri, A., Ketcha, M. D., Reaungamornrat, S., Kleinszig, G., Vogt, S., Aygun, N., Lo, S.-F., Wolinsky, J.-P., & Siewerdsen, J. H. (2016) · Physics in Medicine & Biology · doi:10.1088/0031-9155/61/8/3009 · Cited by 107 · see citing works
Learning-based deformable registration of MR brain images
Wu, G., Qi, F., & Shen, D. G. (2006) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2006.879320 · Cited by 106 · see citing works
Monitoring tumor motion by real time 2D/3D registration during radiotherapy
Gendrin, C., Furtado, H., Weber, C., Bloch, C., Figl, M., Pawiro, S. A., Bergmann, H., Stock, M., Gabor Fichtinger, A., Georg, D., et al. (2011) · Radiotherapy and Oncology · doi:10.1016/j.radonc.2011.07.031 · Cited by 87 · see citing works
2D/3D image registration using regression learning
Chou, C.-R., Frederick, C. B., Mageras, G. S., Chang, S., & Pizer, S. M. (2013) · Computer Vision and Image Understanding · doi:10.1016/j.cviu.2013.02.009 · Cited by 62 · see citing works
Effective Intensity-Based 2D/3D Rigid Registration between Fluoroscopic X-Ray and CT
Knaan, D., & Joskowicz, L. (2003) · Lecture Notes in Computer Science · doi:10.1007/978-3-540-39899-8_44 · Cited by 43 · see citing works
Uneri, A., Silva, T. D., Goerres, J., Jacobson, M. W., Ketcha, M. D., Reaungamornrat, S., Kleinszig, G., Vogt, S. K., Khanna, A. J., Osgood, G. M., et al. (2017) · Physics in Medicine and Biology · doi:10.1088/1361-6560/aa62c5 · Cited by 42 · see citing works
Registration-by-regression of coronary CTA and X-ray angiography
Gouveia, A., Metz, C., Freire, L., Almeida, P., & Klein, S. (2017) · Comput. methods Biomech. Biomed. Eng. Imaging Vis. · doi:10.1080/21681163.2015.1054520 · Cited by 23 · see citing works
Miao, S., Liao, R., & Zheng, Y. (2011) · IEEE International Symposium on Biomedical Imaging: From Nano to Macro (ISBI) · doi:10.1109/isbi.2011.5872620 · Cited by 22 · see citing works
Self-citationSegmentation of X-ray Images by 3D-2D Registration Based on Multibody Physics
Schmid, J., & Chênes, C. (2015) · Lecture notes in computer science · doi:10.1007/978-3-319-16808-1_45 · Cited by 22 · see citing works
Zheng, J., Miao, S., & Liao, R. (2017) · Lecture notes in computer science · doi:10.1007/978-3-319-66185-8_73 · Cited by 21 · see citing works
Self-citationA global optimization strategy for 3D-2D registration of vascular images
Lau, K. K., & Chung, A. C. S. (2006) · British Machine Vision Conference (BMVC) · doi:10.5244/c.20.51 · Cited by 12 · see citing works
Flagged first, then by citations
4 of the 30 references couldn’t be matched to a known paper.
Research works printed but not linked: 3
- Jaderberg, M.; Simonyan, K.; Zisserman, A.; et al. 2015. Spatial transformer networks. In Advances in Neural Infor- mation Processing Systems, 2017-2025.
- Kruger, J., and Westermann, R. 2003. Acceleration tech- niques for gpu-based volume rendering. In Proceedings of the 14th IEEE Visualization 2003 (VIS'03), 38. IEEE Com- puter Society.
- Van Laarhoven, P. J., and Aarts, E. H. 1987. Simulated an- nealing. In Simulated annealing: Theory and applications. Springer. 7-15.
Cited by
Sorted by citations
Image Matching from Handcrafted to Deep Features: A Survey
Ma, J.-Y., Jiang, X., Fan, A., Jiang, J., & Yan, J. (2020) · International Journal of Computer Vision · doi:10.1007/s11263-020-01359-2 · Cited by 977 · see citing works
Deep learning in medical image registration: a review
Fu, Y., Lei, Y., Wang, T., Curran, W. J., Liu, T., & Yang, X. (2020) · Physics in Medicine & Biology · doi:10.1088/1361-6560/ab843e · Cited by 661 · see citing works
A review of multimodal image matching: Methods and applications
Jiang, X., Ma, J.-Y., Xiao, G.-B., Shao, Z.-F., & Guo, X.-J. (2021) · Information Fusion · doi:10.1016/j.inffus.2021.02.012 · Cited by 476 · see citing works
Artificial intelligence in radiation oncology
Huynh, E., Hosny, A., Guthier, C. V., Bitterman, D. S., Petit, S., Haas-Kogan, D. A., Kann, B. H., Aerts, H. J. W. L., & Mak, R. H. (2020) · Nature Reviews Clinical Oncology · doi:10.1038/s41571-020-0417-8 · Cited by 440 · see citing works
Survey on deep learning for radiotherapy
Meyer, P., Noblet, V., Mazzara, C., & Lallement, A. (2018) · Computers in Biology and Medicine · doi:10.1016/j.compbiomed.2018.05.018 · Cited by 302 · see citing works
Application of artificial intelligence in surgery
Zhou, X.-Y., Guo, Y., Shen, M., & Yang, G.-Z. (2020) · Frontiers of Medicine · doi:10.1007/s11684-020-0770-0 · Cited by 181 · see citing works
Medical image registration using deep neural networks: A comprehensive review
Boveiri, H. R., Khayami, R., Javidan, R., & Mehdizadeh, A. (2020) · Computers & Electrical Engineering · doi:10.1016/j.compeleceng.2020.106767 · Cited by 179 · see citing works
2D/3D Multimode Medical Image Registration Based on Normalized Cross-Correlation
Liu, S., Yang, B., Wang, Y., Tian, J., Yin, L., & Zheng, W. (2022) · Applied Sciences · doi:10.3390/app12062828 · Cited by 130 · see citing works
Gao, C., Killeen, B. D., Hu, Y., Grupp, R. B., Taylor, R. H., Armand, M., & Unberath, M. (2023) · Nature Machine Intelligence · doi:10.1038/s42256-023-00629-1 · Cited by 123 · see citing works
Hu, M., Zhang, J., Matkovic, L. A., Liu, T., & Yang, X. (2023) · Journal of Applied Clinical Medical Physics · doi:10.1002/acm2.13898 · Cited by 108 · see citing works
Deep Learning in Medical Image Registration
Chen, X., Diaz-Pinto, A., Ravikumar, N., & Frangi, A. (2020) · Progress in Biomedical Engineering · doi:10.1088/2516-1091/abd37c · Cited by 106 · see citing works
Low-Dose CT Denoising via Sinogram Inner-Structure Transformer
Yang, L., Li, Z., Ge, R., Zhao, J., Si, H., & Zhang, D. (2022) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2022.3219856 · Cited by 108 · see citing works
Unberath, M., Gao, C., Hu, Y., Judish, M., Taylor, R. H., Armand, M., & Grupp, R. (2021) · Frontiers in Robotics and AI · doi:10.3389/frobt.2021.716007 · Cited by 102 · see citing works
“[70, 75, 76, 78] use the image residual between yv and ŷv in a region of interest around the projected tool model.”
BackgroundMethodInfluentialHuang, W., Yang, H., Liu, X., Li, C., Zhang, I., Wang, R., Zheng, H., & Wang, S. (2021) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2021.3059282 · Cited by 81 · see citing works
Learning Dual Transformer Network for Diffeomorphic Registration
Zhang, Y., Pei, Y., & Zha, H. (2021) · Lecture notes in computer science · doi:10.1007/978-3-030-87202-1_13 · Cited by 67 · see citing works
A survey on mutual information based medical image registration algorithms
Sengupta, D., Gupta, P., & Biswas, A. (2021) · Neurocomputing · doi:10.1016/j.neucom.2021.11.023 · Cited by 67 · see citing works
Zhu, Y., & Shi, L. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-16446-0_8 · Cited by 62 · see citing works
Artificial intelligence with deep learning in nuclear medicine and radiology
Decuyper, M., Maebe, J., Holen, R. V., & Vandenberghe, S. (2021) · EJNMMI Physics · doi:10.1186/s40658-021-00426-y · Cited by 60 · see citing works
Toward automatic C-arm positioning for standard projections in orthopedic surgery
Kausch, L., Thomas, S., Kunze, H., Privalov, M., Vetter, S. Y., Franke, J., Mahnken, A. H., Maier‐Hein, L., & Maier-Hein, K. H. (2020) · International Journal of Computer Assisted Radiology and Surgery · doi:10.1007/s11548-020-02204-0 · Cited by 53 · see citing works
Multiview 2D/3D Rigid Registration via a Point-Of-Interest Network for Tracking and Triangulation
Liao, H., Lin, W.-A., Zhang, J., Zhang, J., Luo, J., & Zhou, S. K. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.01292 · Cited by 66 · see citing works
3D/2D model-to-image registration by imitation learning for cardiac procedures
Toth, D. F., Miao, S., Kurzendorfer, T., Rinaldi, C. A., Liao, R., Mansi, T., Rhode, K., & Mountney, P. (2018) · International Journal of Computer Assisted Radiology and Surgery · doi:10.1007/s11548-018-1774-y · Cited by 51 · see citing works
Self-citationA Review of Keypoints’ Detection and Feature Description in Image Registration
Liu, C., Xu, J., & Wang, F. (2021) · Scientific Programming · doi:10.1155/2021/8509164 · Cited by 47 · see citing works
Generalizing Spatial Transformers to Projective Geometry with Applications to 2D/3D Registration
Gao, C., Liu, X., Gu, W., Killeen, B., Armand, M., Taylor, R., & Unberath, M. (2020) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-59716-0_32 · Cited by 46 · see citing works
An application independent review of multimodal 3D registration methods
Saiti, E., & Theoharis, T. (2020) · Computers & Graphics · doi:10.1016/j.cag.2020.07.012 · Cited by 40 · see citing works
Artificial intelligence in image-guided radiotherapy: a review of treatment target localization
Zhao, W., Shen, L., Islam, M. T., Qin, W., Zhang, Z., Liang, X., Zhang, G., Xu, S., & Li, X. (2021) · Quantitative Imaging in Medicine and Surgery · doi:10.21037/qims-21-199 · Cited by 39 · see citing works
ψ-Net: Stacking Densely Convolutional LSTMs for Sub-Cortical Brain Structure Segmentation
Liu, L., Hu, X., Zhu, L., Fu, C., Qin, J., & Heng, P. (2020) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2020.2975642 · Cited by 38 · see citing works
Image-to-Graph Convolutional Network for 2D/3D Deformable Model Registration of Low-Contrast Organs
Nakao, M., Nakamura, M., & Matsuda, T. (2022) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2022.3194517 · Cited by 37 · see citing works
A Fully Differentiable Framework for 2D/3D Registration and the Projective Spatial Transformers
Gao, C., Feng, A., Liu, X., Taylor, R. H., Armand, M., & Unberath, M. (2024) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2023.3299588 · Cited by 36 · see citing works
“Several researchers formulated such learning-based 2D/3D registration with reinforcement learning paradigms [22], [23], [24], but they still perform pose regression at each iteration.”
BackgroundFiducial-Free 2D/3D Registration for Robot-Assisted Femoroplasty
Gao, C., Farvardin, A., Grupp, R. B., Bakhtiarinejad, M., Ma, L., Thies, M., Unberath, M., Taylor, R. H., & Armand, M. (2020) · IEEE Transactions on Medical Robotics and Bionics · doi:10.1109/tmrb.2020.3012460 · Cited by 36 · see citing works
Artificial Intelligence for Emerging Technology in Surgery: Systematic Review and Validation
Nwoye, E. O., Woo, W. L., Gao, B., & Anyanwu, T. (2022) · IEEE Reviews in Biomedical Engineering · doi:10.1109/rbme.2022.3183852 · Cited by 38 · see citing works
Pose-Dependent Weights and Domain Randomization for Fully Automatic X-Ray to CT Registration
Grimm, M., Esteban, J., Unberath, M., & Navab, N. (2021) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2021.3073815 · Cited by 34 · see citing works
Modeling the Probabilistic Distribution of Unlabeled Data for One-shot Medical Image Segmentation
Ding, Y., Yu, X., & Yang, Y. (2021) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v35i2.16212 · Cited by 33 · see citing works
Volumetric tumor tracking from a single cone-beam X-ray projection image enabled by deep learning
Dai, J., Dong, G., Zhang, C., He, W., Liu, L., Wang, T., Jiang, Y., Zhao, W., Zhao, X. J., Xie, Y., et al. (2023) · Medical Image Analysis · doi:10.1016/j.media.2023.102998 · Cited by 31 · see citing works
NeurReg: Neural Registration and Its Application to Image Segmentation
Zhu, W., Myronenko, A., Xu, Z., Li, W., Roth, H. R., Huang, Y., Milletarì, F., & Xu, D. (2020) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv45572.2020.9093506 · Cited by 31 · see citing works
Learning an Attention Model for Robust 2-D/3-D Registration Using Point-To-Plane Correspondences
Schaffert, R., Wang, J., Fischer, P., Borsdorf, A., & Maier, A. (2020) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2020.2988410 · Cited by 30 · see citing works
Self-citationSelf-Supervised 2D/3D Registration for X-Ray to CT Image Fusion
Jaganathan, S., Kukla, M., Wang, J., Shetty, K., & Maier, A. (2023) · 2023 IEEE/CVF Winter Conference on Applications of Computer Vision (WACV) · doi:10.1109/wacv56688.2023.00281 · Cited by 30 · see citing works
A Patient-Specific Self-supervised Model for Automatic X-Ray/CT Registration
Zhang, B., Faghihroohi, S., Azampour, M. F., Liu, S., Ghotbi, R., Schunkert, H., & Navab, N. (2023) · Lecture Notes in Computer Science · doi:10.1007/978-3-031-43996-4_49 · Cited by 27 · see citing works
Boundary-Sensitive Loss Function With Location Constraint for Hard Region Segmentation
Du, J., Guan, K., Liu, P., Li, Y., & Wang, T. (2022) · IEEE Journal of Biomedical and Health Informatics · doi:10.1109/jbhi.2022.3222390 · Cited by 26 · see citing works
End-to-end Ultrasound Frame to Volume Registration
Guo, H., Xu, X., Xu, S., Wood, B. J., & Yan, P. (2021) · Lecture notes in computer science · doi:10.1007/978-3-030-87202-1_6 · Cited by 25 · see citing works
Uncertainty Learning towards Unsupervised Deformable Medical Image Registration
Gong, X., Khaidem, L., Zhu, W., Zhang, B., & Doermann, D. (2022) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv51458.2022.00162 · Cited by 24 · see citing works
CAR-Net: A Deep Learning-Based Deformation Model for 3D/2D Coronary Artery Registration
Wu, W., Zhang, J., Peng, W., Xie, H., Zhang, S., & Gu, L. (2022) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2022.3168786 · Cited by 22 · see citing works
Patel, R. K., & Kashyap, M. (2023) · Computer Methods in Biomechanics and Biomedical Engineering Imaging & Visualization · doi:10.1080/21681163.2023.2243351 · Cited by 22 · see citing works
Wang, X., Huang, J., Chatzakou, M., Nõmm, S., Valla, E., Medijainen, K., Taba, P., Toomela, A., & Ruzhansky, M. V. (2023) · Biomedical Signal Processing and Control · doi:10.1016/j.bspc.2023.105436 · Cited by 22 · see citing works
Iterative Closest Graph Matching for Non-rigid 3d/2d Coronary Arteries Registration
Zhu, J., Li, H., Ai, D., Yang, Q., Fan, J., Huang, Y., Song, H., Han, Y., & Yang, J. (2021) · Computer Methods and Programs in Biomedicine · doi:10.1016/j.cmpb.2020.105901 · Cited by 22 · see citing works
Deep Iterative 2D/3D Registration
Jaganathan, S., Wang, J., Borsdorf, A., Shetty, K., & Maier, A. (2021) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-87202-1_37 · Cited by 21 · see citing works
Test-Time Training for Deformable Multi-Scale Image Registration
Zhu, W., Huang, Y., Xu, D., Qian, Z. Y., Fan, W., & Xie, X. S. (2021) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra48506.2021.9561808 · Cited by 21 · see citing works
Real-Time 2D/3D Registration via CNN Regression and Centroid Alignment
Huang, D.-X., Zhou, X.-H., Xie, X., Liu, S.-Q., Feng, Z.-Q., Hou, Z., Ma, N., & Yan, L. (2024) · IEEE Transactions on Automation Science and Engineering · doi:10.1109/tase.2023.3345927 · Cited by 20 · see citing works
Fully convolutional network with attention modules for semantic segmentation
Huang, Y., & Xu, H. (2021) · Signal Image and Video Processing · doi:10.1007/s11760-020-01828-8 · Cited by 20 · see citing works
Metric-Driven Learning of Correspondence Weighting for 2-D/3-D Image Registration
Schaffert, R., Wang, J., Fischer, P., Borsdorf, A., & Maier, A. (2019) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-12939-2_11 · Cited by 19 · see citing works
Self-citationWang, C., Xie, S., Li, K., Wang, C., Liu, X., Zhao, L., & Tsai, T.-Y. (2021) · Engineering · doi:10.1016/j.eng.2020.03.016 · Cited by 19 · see citing works
Unsupervised Cross-Modality Domain Adaptation Network for X-Ray to CT Registration
Zheng, S., Yang, X. N., Wang, Y., Ding, M., & Hou, W. W. (2021) · IEEE Journal of Biomedical and Health Informatics · doi:10.1109/jbhi.2021.3135890 · Cited by 18 · see citing works
Geng, H., Xiao, D., Yang, S., Fan, J., Fu, T., Lin, Y., Bai, Y., Ai, D., Song, H., Wang, Y., et al. (2023) · Physics in Medicine and Biology · doi:10.1088/1361-6560/acede5 · Cited by 17 · see citing works
Ultrasound Frame-to-Volume Registration via Deep Learning for Interventional Guidance
Guo, H., Xu, X., Song, X., Xu, S. Y., Chao, H., Myers, J., Türkbey, B., Pinto, P. A., Wood, B. J., & Yan, P. (2022) · IEEE Transactions on Ultrasonics Ferroelectrics and Frequency Control · doi:10.1109/tuffc.2022.3229903 · Cited by 16 · see citing works
Fast fetal head compounding from multi-view 3D ultrasound
Wright, R., Gómez, A., Zimmer, V. A., Toussaint, N., Khanal, B., Matthew, J., Skelton, E., Kainz, B., Rueckert, D., Hajnal, J. V., et al. (2023) · Medical Image Analysis · doi:10.1016/j.media.2023.102793 · Cited by 15 · see citing works
Wang, H., Ou, Y., Fang, W., Ambalathankandy, P., Goto, N., Ota, G., Okino, T., Fukae, J., Sutherland, K. L., Ikebe, M., et al. (2023) · Computerized Medical Imaging and Graphics · doi:10.1016/j.compmedimag.2023.102273 · Cited by 15 · see citing works
Non-Rigid 2D-3D Registration Using Convolutional Autoencoders
Li, P., Pei, Y., Guo, Y., Ma, G., Xu, T., & Zha, H. (2020) · 2020 IEEE 17th International Symposium on Biomedical Imaging (ISBI) · doi:10.1109/isbi45749.2020.9098602 · Cited by 13 · see citing works
Multi-View Global 2D-3D Registration Based on Branch and Bound Algorithm
Pan, J., Min, Z., Zhang, A., Ma, H., & Meng, M. Q. ‐. (2019) · IEEE International Conference on Robotics and Biomimetics (ROBIO) · doi:10.1109/robio49542.2019.8961379 · Cited by 13 · see citing works
Thermal and Visible Image Registration Using Deep Homography
Debaque, B., Perreault, H., Mercier, J. P., Drouin, M.-A., David, R., Chatelais, B., Duclos-Hindie, N., & Roy, S. (2022) · International Conference on Information Fusion (FUSION) · doi:10.23919/fusion49751.2022.9841256 · Cited by 13 · see citing works
Deep Learning Applications in Medical Imaging
Sasikala, S., Subhashini, S. J., Alli, P., & Angelina, J. J. R. (2020) · Advances in medical technologies and clinical practice book series · doi:10.4018/978-1-7998-5071-7.ch008 · Cited by 12 · see citing works
Pooling Attention-based Encoder–Decoder Network for semantic segmentation
Xu, H., Huang, Y., Hancock, E. R., Wang, S., Xuan, Q., & Zhou, W. (2021) · Computers & Electrical Engineering · doi:10.1016/j.compeleceng.2021.107260 · Cited by 12 · see citing works
Zhou, C., Cha, T., Peng, Y. F., & Li, G. (2021) · Computers in Biology and Medicine · doi:10.1016/j.compbiomed.2021.104923 · Cited by 11 · see citing works
Zhao, H., Niu, Y., Zhu, W. P., Deng, X., Zhang, G., & Zou, W. (2024) · Biomedical Signal Processing and Control · doi:10.1016/j.bspc.2024.106384 · Cited by 10 · see citing works
Iterative PnP and its application in 3D-2D vascular image registration for robot navigation
Song, J., Yang, K., Zhang, Z., Li, M., Cao, T., & Ghaffari, M. (2024) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra57147.2024.10610392 · Cited by 9 · see citing works
Cai, M., Li, Y., Xu, Y., Li, N., & Xia, K. (2022) · IEEE Transactions on Medical Robotics and Bionics · doi:10.1109/tmrb.2022.3171310 · Cited by 10 · see citing works
Burton, W. S., Myers, C. A., Stefanovic, M., Shelburne, K. B., & Rullkoetter, P. J. (2024) · Journal of Biomechanics · doi:10.1016/j.jbiomech.2024.112066 · Cited by 9 · see citing works
Agent-Based Methods for Medical Image Registration
Miao, S., & Liao, R. (2019) · Advances in computer vision and pattern recognition · doi:10.1007/978-3-030-13969-8_16 · Cited by 9 · see citing works
Self-citationDomain adaptation strategies for 3D reconstruction of the lumbar spine using real fluoroscopy data
Jecklin, S., Shen, Y., Gout, A., Suter, D., Calvet, L., Zingg, L., Straub, J., Cavalcanti, N., Farshad, M., Fürnstahl, P., et al. (2024) · Medical Image Analysis · doi:10.1016/j.media.2024.103322 · Cited by 16 · see citing works
Gui, P., He, F., Ling, B. W., Zhang, D., & Ge, Z. (2023) · Neural Computing and Applications · doi:10.1007/s00521-023-08649-z · Cited by 8 · see citing works
Global Registration of 3D Cerebral Vessels to Its 2D Projections by a New Branch-and-Bound Algorithm
Fu, K., Liu, Y., & Wang, M. (2021) · IEEE Transactions on Medical Robotics and Bionics · doi:10.1109/tmrb.2021.3055020 · Cited by 8 · see citing works
Robust Orthogonal-View 2-D/3-D Rigid Registration for Minimally Invasive Surgery
An, Z., Ma, H., Liu, L., Wang, Y., Lu, H., Zhou, C., Xiong, R., & Hu, J. (2021) · Micromachines · doi:10.3390/mi12070844 · Cited by 7 · see citing works
Learning-Based Correspondence Estimation for 2-D/3-D Registration
Schaffert, R., Weiß, M., Wang, J., Borsdorf, A., & Maier, A. (2020) · Informatik aktuell · doi:10.1007/978-3-658-29267-6_50 · Cited by 7 · see citing works
Ye, W., Wu, J., Zhang, W., Sun, L., Dong, X. Y., & Xu, S. (2025) · Bioengineering · doi:10.3390/bioengineering12020114 · Cited by 7 · see citing works
Lee, B. C., Sinha, A., Varble, N., Pritchard, W. F., Karanian, J. W., Wood, B. J., & Bydlon, T. M. (2022) · IEEE International Symposium on Biomedical Imaging (ISBI) · doi:10.1109/isbi52829.2022.9761705 · Cited by 6 · see citing works
Deformable registration of lateral cephalogram and cone‐beam computed tomography image
Zhang, Y., Qin, H., Li, P., Pei, Y., Guo, Y., Xu, T., & Zha, H. (2021) · Medical Physics · doi:10.1002/mp.15214 · Cited by 6 · see citing works
SEN-FCB: an unsupervised twinning neural network for image registration
Ma, M., Liu, G., Song, L., & Xu, Y. (2022) · Applied Intelligence · doi:10.1007/s10489-022-04109-8 · Cited by 5 · see citing works
Deep Learning in Image Processing: Part 2—Image Enhancement, Reconstruction and Registration
Pauwels, R., & Iosifidis, A. (2023) · Springer eBooks · doi:10.1007/978-3-031-43827-1_18 · Cited by 5 · see citing works
Learning the Update Operator for 2D/3D Image Registration
Jaganathan, S., Wang, J., Borsdorf, A., & Maier, A. (2021) · Informatik aktuell · doi:10.1007/978-3-658-33198-6_27 · Cited by 5 · see citing works
Yang, K., Luo, Y., Zhao, Y., Su, S., Qu, D., Zhao, X., & Song, G. (2021) · Physics in Medicine and Biology · doi:10.1088/1361-6560/abe9f5 · Cited by 5 · see citing works
Mizoe, S., Otake, Y., Miyamoto, T., Soufi, M., Nakao, S., Tanaka, Y., & Sato, Y. (2021) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-87202-1_18 · Cited by 5 · see citing works
Multiview 2D/3D image registration in minimally invasive pelvic surgery navigation
Ju, F., Wang, Y., Zhao, J., & Dong, M. (2025) · Scientific Reports · doi:10.1038/s41598-025-10884-4 · Cited by 5 · see citing works
Modality-shared MRI Image Translation Based on Conditional GAN
Deng, C., Chen, Z., Wang, R., Su, W., & Qu, Y. (2021) · IEEE International Conference on Bioinformatics and Biomedicine (BIBM) · doi:10.1109/bibm52615.2021.9669277 · Cited by 4 · see citing works
Multimodal Deep Homography Estimation Using a Domain Adaptation Generative Adversarial Network
Pouplin, T., Perreault, H., Debaque, B., Drouin, M.-A., Duclos-Hindie, N., & Roy, S. (2022) · IEEE International Conference on Big Data (BigData) · doi:10.1109/bigdata55660.2022.10020557 · Cited by 4 · see citing works
CrossModalNet: exploiting quality preoperative images for multimodal image registration
Sun, J., Liu, C., Li, C., Lu, Z., He, M., Gao, L., Lin, T., Sui, J., Xie, K., & Ni, X. (2021) · Physics in Medicine and Biology · doi:10.1088/1361-6560/ac195e · Cited by 4 · see citing works
Artificial Intelligence Surgery
(2021) · Artificial Intelligence Surgery · doi:10.20517/ais · Cited by 4 · see citing works
Burton, W. S., Myers, C. A., Stefanovic, M., Shelburne, K. B., & Rullkoetter, P. J. (2024) · Annals of Biomedical Engineering · doi:10.1007/s10439-024-03473-5 · Cited by 3 · see citing works
MMAR-Net: A Multi-Stride and Multi-Resolution Affine Registration Network for CT Images
Zhou, F., Luo, F., Kong, R., Chen, Y. P., & Liu, F. (2024) · Big Data Mining and Analytics · doi:10.26599/bdma.2024.9020005 · Cited by 3 · see citing works
Hagawi, W. S. A. A., Jubran, E. A. H., Almohammedsaleh, F. Y. A., AYED, A. S., HAMDI, A. N. A., Alnasser, A. M., Alosaimy, O. M., Alshammari, T. A., Alshammari, T. A., Alghamdi, A. H. J., et al. (2024) · Journal of Ecohumanism · doi:10.62754/joe.v3i8.5552 · Cited by 3 · see citing works
Jiang, Y., Li, P., Zhang, Y., Pei, Y., Guo, Y., Xu, T., & Yuan, X. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-59861-7_33 · Cited by 3 · see citing works
Registration made easy -- standalone orthopedic navigation with HoloLensPreprint
Liebmann, F., Roner, S., von Atzigen, M., Wanivenhaus, F., Neuhaus, C., Spirig, J. M., Scaramuzza, D., Sutter, R., Snedeker, J. G., Farshad, M., et al. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2001.06209 · Cited by 3 · see citing works
Jiang, Y., Yuan, X., & Pei, Y. (2023) · IEEE International Symposium on Biomedical Imaging (ISBI) · doi:10.1109/isbi53787.2023.10230425 · Cited by 2 · see citing works
Shape-Based Pose Estimation for Automatic Standard Views of the Knee
Kausch, L., Thomas, S., Kunze, H., Barbari, J. S. E., & Maier-Hein, K. H. (2023) · Lecture notes in computer science · doi:10.1007/978-3-031-43990-2_45 · Cited by 2 · see citing works
Jiang, Y., Pei, Y., Xu, T., Yuan, X., & Zha, H. (2024) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2024.3456251 · Cited by 2 · see citing works
Thibeault, S., Roy‐Beaudry, M., Stefan Parent, D., & Kadoury, S. (2024) · Medical Image Analysis · doi:10.1016/j.media.2024.103400 · Cited by 2 · see citing works
Li, S., Jia, B., Huang, W., Zhang, X., Wu, Z., Wang, C., & Teng, G. (2025) · Computerized Medical Imaging and Graphics · doi:10.1016/j.compmedimag.2025.102562 · Cited by 2 · see citing works
Pan, J., Min, Z., Zhang, A., Ma, H., & Meng, M. Q. ‐. (2021) · IEEE International Conference on Robotics and Biomimetics (ROBIO) · doi:10.1109/robio54168.2021.9739622 · Cited by 2 · see citing works
2D/3D-MGR: A 2D/3D Medical Image Registration Framework Based on DRR
Li, Z., Ji, X., Wang, C., Liu, W., Zhu, F., & Zhai, J. (2024) · IEEE Access · doi:10.1109/access.2024.3453661 · Cited by 1 · see citing works
O-Arm Imaging With Real-Time Control for Organ Motion Tracking: A Feasibility Study
Ghorbanian, A., Salehi, M., Sokout, M. S., & Beigzadeh, B. (2025) · IEEE Access · doi:10.1109/access.2025.3571989 · Cited by 1 · see citing works
Jiang, Y., Pei, Y., Li, P., Zhang, Y., Guo, Y., Fan, Y., Chen, G., Dai, F., Xu, T., Yuan, X., et al. (2022) · IEEE Transactions on Computational Imaging · doi:10.1109/tci.2022.3216202 · Cited by 1 · see citing works
A Systematic Literature Review on Multi-modal Medical Image Registration
Chaabane, M., & Koller, B. J. (2023) · Lecture notes in computer science · doi:10.1007/978-3-031-26507-5_8 · Cited by 1 · see citing works
A lightweight attention-guided semantic segmentation network using a group convolution pyramid
Yu, S., Huang, D., Wang, G., & Yao, A. (2023) · International Conference on Computing, Communication, Perception and Quantum Technology (CCPQT) · doi:10.1109/ccpqt60491.2023.00033 · Cited by 1 · see citing works
- Authors
- Miao, S., Piat, S., Fischer, P., Tuysuzoglu, A., Mewes, P., Mansi, T., & Liao, R.
- Affiliations
- Siemens Healthcare (United States) (US)
- Venue
- Proceedings of the AAAI Conference on Artificial Intelligence
- Published
- 2018-04-26
- Volume
- 32
- Issue
- 1
- ISSN
- 2159-5399 (ISSN-L)
- 2374-3468
- OpenAlex
- W2962759210
- Cited by
- 113
- References
- 30
- Citations by year
- 2026: 3 · 2025: 5 · 2024: 12 · 2023: 18 · 2022: 17 · 2021: 29 · 2020: 21 · 2019: 6 · 2018: 2
- FWCI
- 4.37
- Type
- conference paper
- Language
- en
- Open access
- diamond
- doi.org
- Locations
- Proceedings of the AAAI Conference on Artificial Intelligence (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- AAAI Publications (The Association for the Advancement of Artificial Intelligence (AAAI)) (submittedVersion)
- arXiv (Cornell University)
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Medical Image Segmentation Techniques
- Physical Sciences › Engineering › Biomedical Engineering › Medical Imaging and Analysis
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
- Keywords
- 2D-3D registration, multi-agent reinforcement learning, Markov decision process, X-ray images, cone-beam computed tomography, spinal surgery, metal artifact reduction, occlusion handling, minimally invasive spine surgery, low signal-to-noise ratio
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
