The DOI matches its Crossref record; 34 of the 37 references it cites were found in a registry or an index.
An Artificial Agent for Robust Image Registration
Open accessAbstract
3-D image registration, which involves aligning two or more images, is a critical step in a variety of medical applications from diagnosis to therapy. Image registration is commonly performed by optimizing an image matching metric as a cost function. However this task is challenging due to the non-convex nature of the matching metric over the plausible registration parameter space and insufficient approches for a robust optimization. As a result, current approaches are often customized to a specific problem and sensitive to image quality and artifacts. In this paper, we propose a completely different approach to image registration, inspired by how experts perform the task. We first cast the image registration problem as a "strategic learning" process, where the goal is to find the best sequence of motion actions (e.g. up, down, etc) that yields image alignment. Within this approach, an artificial agent is learned, modeled using deep convolutional neural networks, with 3D raw image data as the input, and the next optimal action as the output. To copy with the dimensionality of the problem, we propose a greedy supervised approach for an end-to-end training, coupled with attention-driven hierarchical strategy. The resulting registration approach inherently encodes both a data-driven matching metric and an optimal registration strategy (policy). We demonstrate on two 3-D/3-D medical image registration examples with drastically different nature of challenges, that the artificial agent outperforms several state-of-the-art registration methods by a large margin in terms of both accuracy and robustness.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Liao et al. (70) firstly proposed to train an artificial agent for the rigid registration of 3D CT images and cone-beam CT (CBCT) images for cardiac and abdominal images.”
From A review of deep learning-based deformable medical image registration (2022) · cites this paper as [70]
Citations
Cites
ImageNet classification with deep convolutional neural networks
Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017) · Communications of the ACM · doi:10.1145/3065386 · Cited by 112,155
Human-level control through deep reinforcement learning
Mnih, V., Kavukcuoglu, K., Silver, D., Rusu, A. A., Veness, J., Bellemare, M. G., Graves, A., Riedmiller, M., Fidjeland, A. K., Ostrovski, G., et al. (2015) · Nature · doi:10.1038/nature14236 · Cited by 30,357
A Method for Registration of 3-D Shapes
Besl, P., & Mckay, N. D. (1992) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/34.121791 · Cited by 22,163
Reducing the dimensionality of data with neural networks
Hinton, G. E., & Salakhutdinov, R. R. (2006) · Science · doi:10.1126/science.1127647 · Cited by 20,928
Mastering the game of Go with deep neural networks and tree search
Silver, D., Huang, A., Maddison, C. J., Guez, A., Sifre, L., van den Driessche, G., Schrittwieser, J., Antonoglou, I., Panneershelvam, V., Lanctot, M., et al. (2016) · Nature · doi:10.1038/nature16961 · Cited by 15,612
Practical Bayesian Optimization of Machine Learning Algorithms
Snoek, J., Larochelle, H., & Adams, R. P. (2012) · arXiv · doi:10.48550/arxiv.1206.2944 · Cited by 6,691
Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Maps
Simonyan, K., Vedaldi, A., & Zisserman, A. (2013) · arXiv · doi:10.48550/arxiv.1312.6034 · Cited by 5,370
Flownet: Learning optical flow with convolutional networks
Dosovitskiy, A., Fischer, P., Ilg, E., Hausser, P., Hazirbas, C., Golkov, V., Smagt, P. V. D., Cremers, D., & Brox, T. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.316 · Cited by 4,313
Bellman, R. (1957) · Indiana University Mathematics Journal · doi:10.1512/iumj.1957.6.56038 · Cited by 2,408
Derivative-free optimization: a review of algorithms and comparison of software implementations
Rios, L. M., & Sahinidis, N. V. (2013) · Journal of Global Optimization · doi:10.1007/s10898-012-9951-y · Cited by 1,276
Medical image fusion: A survey of the state of the art
James, A. P., & Dasarathy, B. V. (2014) · Information Fusion · doi:10.1016/j.inffus.2013.12.002 · Cited by 977
Medical image registration: a review
Oliveira, F. P. M., & Tavares, J. M. R. S. (2012) · Computer Methods in Biomechanics & Biomedical Engineering · doi:10.1080/10255842.2012.670855 · Cited by 782
A Review of 3d/2d Registration Methods for Image-Guided Interventions
Markelj, P., Tomaževič, D., Likar, B., & Pernuš, F. (2012) · Medical Image Analysis · doi:10.1016/j.media.2010.03.005 · Cited by 746
Optimization of mutual information for multiresolution image registration
Unser, M. A., & Thévenaz, P. (2000) · IEEE Transactions on Image Processing · doi:10.1109/83.887976 · Cited by 694
A CNN Regression Approach for Real-Time 2D/3D Registration
Miao, S., Wang, Z. J., & Liao, R. (2016) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2016.2521800 · Cited by 514
Self-citationLevine, S., & Koltun, V. (2013) · Cited by 481
Active Object Localization with Deep Reinforcement Learning
Caicedo, J. C., & Lazebnik, S. (2015) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.286 · Cited by 459
Learning descriptors for object recognition and 3D pose estimation
Wohlhart, P., & Lepetit, V. (2015) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2015.7298930 · Cited by 458
Evaluation of Registration Methods on Thoracic CT: The EMPIRE10 Challenge
Murphy, K., van Ginneken, B., Reinhardt, J. M., Kabus, S., Ding, K., Deng, X. H., Cao, K., Du, K., Christensen, G. E., Garcia, V., et al. (2011) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2011.2158349 · Cited by 441
Deep Reinforcement Learning in Parameterized Action SpacePreprint
Hausknecht, M., & Stone, P. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1511.04143 · Cited by 321
Recent advances in trust region algorithms
Yuan, Y.-X. (2015) · Mathematical Programming · doi:10.1007/s10107-015-0893-2 · Cited by 315
Guorong Wu, A., Kim, M., Wang, Q., Munsell, B. C., & Shen, D. G. (2015) · IEEE Transactions on Biomedical Engineering · doi:10.1109/tbme.2015.2496253 · Cited by 283
Automatic retinal image registration scheme using global optimization techniques
Matsopoulos, G. K., Mouravliansky, N. A., Delibasis, K. K., & Nikita, K. S. (1999) · IEEE Transactions on Information Technology in Biomedicine · doi:10.1109/4233.748975 · Cited by 189
An Artificial Agent for Anatomical Landmark Detection in Medical Images
Ghesu, F. C., Georgescu, B., Mansi, T., Neumann, D., Joachim Hornegger, A., & Comaniciu, D. (2016) · Lecture notes in computer science · doi:10.1007/978-3-319-46726-9_27 · Cited by 161
Self-citationImitation and Reinforcement Learning
Kober, J., & Peters, J. (2010) · IEEE Robotics & Automation Magazine · doi:10.1109/mra.2010.936952 · Cited by 132
Genetic algorithms for a robust 3-D MR-CT registration
Rouet, J.-M., Jacq, J.-J., & Roux, C. (2000) · IEEE Transactions on Information Technology in Biomedicine · doi:10.1109/4233.845205 · Cited by 112
A Review of Recent Advances in Registration Techniques Applied to Minimally Invasive Therapy
Liao, R., Zhang, L., Sun, Y., Miao, S., & Chefd'Hotel, C. (2013) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2013.2244869 · Cited by 71
Self-citationEvaluating similarity measures for brain image registration
Razlighi, Q. R., Kehtarnavaz, N., & Yousefi, S. (2013) · Journal of Visual Communication and Image Representation · doi:10.1016/j.jvcir.2013.06.010 · Cited by 43
A self-taught artificial agent for multi-physics computational model personalization
Neumann, D., Mansi, T., Lucian Mihai, I. T. U., Georgescu, B., Kayvanpour, E., Sedaghat‐Hamedani, F., Amr, A., Haas, J., Katus, H. A., Meder, B., et al. (2016) · Medical Image Analysis · doi:10.1016/j.media.2016.04.003 · Cited by 34
Self-citationBrounstein, A., Hacihaliloglu, I., Guy, P., Hodgson, A. J., & Rafeef Abugharbieh, A. (2011) · Lecture notes in computer science · doi:10.1007/978-3-642-23623-5_30 · Cited by 31
3D Meshless Prostate Segmentation and Registration in Image Guided Radiotherapy
Chen, T., Kim, S., Zhou, J., Metaxas, D., Rajagopal, G., & Yue, N. J. (2009) · Lecture notes in computer science · doi:10.1007/978-3-642-04268-3_6 · Cited by 26
Miao, S., Liao, R., Pfister, M., Zhang, L., & Ordy, V. (2013) · Lecture Notes in Computer Science · doi:10.1007/978-3-642-40811-3_48 · Cited by 17
Self-citationProbabilistic Sparse Matching for Robust 3D/3D Fusion in Minimally Invasive Surgery
Neumann, D., Grbić, S., John, M., Navab, N., Joachim Hornegger, A., & Ionasec, R. I. (2014) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2014.2343936 · Cited by 11
Lu, Y., Sun, Y., Liao, R., & Ong, S. H. (2014) · Computerized Medical Imaging and Graphics · doi:10.1016/j.compmedimag.2014.06.021 · Cited by 7
Self-citation
Cited by
Sorted by citations
Image Matching from Handcrafted to Deep Features: A Survey
Ma, J.-Y., Jiang, X., Fan, A., Jiang, J., & Yan, J. (2020) · International Journal of Computer Vision · doi:10.1007/s11263-020-01359-2 · Cited by 977
A deep learning framework for unsupervised affine and deformable image registration
de Vos, B. D., Berendsen, F. F., Viergever, M. A., Sokooti, H., Staring, M., & Išgum, I. (2019) · Medical Image Analysis · doi:10.1016/j.media.2018.11.010 · Cited by 818
Deep learning in medical image registration: a survey
Haskins, G., Kruger, U., & Yan, P. (2020) · Machine Vision and Applications · doi:10.1007/s00138-020-01060-x · Cited by 704
Artificial Intelligence in Cardiovascular Imaging
Dey, D., Slomka, P. J., Leeson, P., Comaniciu, D., Shrestha, S., Sengupta, P. P., & Marwick, T. H. (2019) · Journal of the American College of Cardiology · doi:10.1016/j.jacc.2018.12.054 · Cited by 720
Self-citationDeep learning in medical image registration: a review
Fu, Y., Lei, Y., Wang, T., Curran, W. J., Liu, T., & Yang, X. (2020) · Physics in Medicine & Biology · doi:10.1088/1361-6560/ab843e · Cited by 661
Metadata
- Authors
- Liao, R., Miao, S., De Tournemire, P., Grbic, S., Kamen, A., Mansi, T., & Comaniciu, D.
- Affiliations
- Siemens Healthcare (United States) (US)
- Venue
- Proceedings of the AAAI Conference on Artificial Intelligence
- Published
- 2017-02-12
- Volume
- 31
- Issue
- 1
- ISSN
- 2159-5399 (ISSN-L)
- 2374-3468
- OpenAlex
- W2963097316
- Cited by
- 199
- References
- 37
- Citations by year
- 2026: 2 · 2025: 10 · 2024: 24 · 2023: 28 · 2022: 20 · 2021: 36 · 2020: 26 · 2019: 29 · 2018: 19 · 2017: 5
- FWCI
- 6.32
- Type
- conference paper
- Language
- en
- Open access
- diamond
- doi.org
- Locations
- Proceedings of the AAAI Conference on Artificial Intelligence (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- AAAI Publications (The Association for the Advancement of Artificial Intelligence (AAAI)) (submittedVersion)
- arXiv (Cornell University)
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Medical Image Segmentation Techniques
- Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
- Physical Sciences › Computer Science › Artificial Intelligence › Domain Adaptation and Few-Shot Learning
- Keywords
- medical image registration, strategic learning, deep convolutional neural networks, non-convex optimization, robust optimization, end-to-end learning
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
