Cited by 102Medical Image Segmentation TechniquesAnatomy and Medical TechnologySurgical Simulation and Training

The DOI matches its Crossref record; 117 of the 131 references it cites were found in a registry or an index; 33 of the 83 sources its claims cite were checked for support.

The Impact of Machine Learning on 2D/3D Registration for Image-Guided Interventions: A Systematic Review and Perspective

Open access

Abstract

Image-based navigation is widely considered the next frontier of minimally invasive surgery. It is believed that image-based navigation will increase the access to reproducible, safe, and high-precision surgery as it may then be performed at acceptable costs and effort. This is because image-based techniques avoid the need of specialized equipment and seamlessly integrate with contemporary workflows. Furthermore, it is expected that image-based navigation techniques will play a major role in enabling mixed reality environments, as well as autonomous and robot-assisted workflows. A critical component of image guidance is 2D/3D registration, a technique to estimate the spatial relationships between 3D structures, e.g., preoperative volumetric imagery or models of surgical instruments, and 2D images thereof, such as intraoperative X-ray fluoroscopy or endoscopy. While image-based 2D/3D registration is a mature technique, its transition from the bench to the bedside has been restrained by well-known challenges, including brittleness with respect to optimization objective, hyperparameter selection, and initialization, difficulties in dealing with inconsistencies or multiple objects, and limited single-view performance. One reason these challenges persist today is that analytical solutions are likely inadequate considering the complexity, variability, and high-dimensionality of generic 2D/3D registration problems. The recent advent of machine learning-based approaches to imaging problems that, rather than specifying the desired functional mapping, approximate it using highly expressive parametric models holds promise for solving some of the notorious challenges in 2D/3D registration. In this manuscript, we review the impact of machine learning on 2D/3D registration to systematically summarize the recent advances made by introduction of this novel technology. Grounded in these insights, we then offer our perspective on the most pressing needs, significant open problems, and possible next steps.

Overview

AI snapshot

Written from the paper’s full text.

TL;DR

This systematic review and perspective examines how machine learning has been applied to image-based 2D/3D registration for image-guided interventions, reviewing 48 studies published from January 2012 through February 2021. It finds promising improvements in capture range, representation learning, and direct pose regression, but persistent gaps in domain shift, multi-object registration, verification/uncertainty, and standardized evaluation.

Key findings

  • Across 48 included studies, most addressed X-ray to CT/CBCT registration (34), followed by X-ray to 3D object models (10); clinical applications included orthopedics (19), angiography (9), radiation therapy (7), and cephalometry (4).
  • The review identified five machine learning themes: contextualization (14 studies), representation learning (12), similarity modeling (10), direct parameter regression (22), and verification (4). Most methods were single-object: 45 of 48 studies considered only one object, and the few multiple-object methods solved objects independently.
  • Training data were often synthetic: 16 of 48 studies used real clinical or cadaveric data to train, while among 32 deep learning studies, 24 trained on synthetic data, 7 on real data, and 1 used a pretrained network. Seven studies comparing synthetic and real performance found substantial drops under domain shift, and ten synthetic-trained studies never meaningfully tested on real data.
  • The authors conclude that machine learning has improved capture range and enabled fast direct regression, but ambiguity, high-dimensional optimization, verification/uncertainty, multi-object and foreign-object handling, and generalizability remain open problems requiring standardized metrics, public benchmarks, and more generalist components.

Methods

  • PRISMA systematic review searched PubMed, embase, Business Source Ultimate, and Compendex for January 2012–February 2021 using 2D/3D registration terms combined with learning/training/testing terms; 559 records were found, 495 unique studies screened, 447 excluded, and 48 included for full-text review.
  • Five authors extracted data using a template covering method summary, modalities and registration phase, spatial transformation, machine learning model and training setup, evaluation strategy including domain shift, and methodological concerns; each article was coded by at least two authors, with one author merging reports into a consensus document.
  • The paper synthesizes findings narratively around five themes and a perspective discussion, without a quantitative meta-analysis.

Limitations

  • Stated in the paper: the search may not be exhaustive because it omitted newer or less popular terminology such as “pose regression” and modality-specific terms such as “video to CT registration”; it may have excluded studies not explicitly characterized as machine/deep learning or from the general computer vision literature; and it was limited to published manuscripts, so publication bias may have excluded relevant work.
  • Evident from the text: the review is qualitative and narrative rather than meta-analytic, and the included studies use heterogeneous metrics, datasets, and evaluation protocols, with many relying on synthetic data and few reporting real-data performance, which limits comparability and real-world conclusions; the authors also note that private code and datasets inhibit reproducibility.

Model-written summary. Check it against the paper before relying on it.

Integrity signals

Registry

Crossref · 10.3389/frobt.2021.716007

Cites

117 of 117 references verified0 citation errors · 0 not checkable

Statistics

Nothing to recompute.

Verification

Claims the paper makes

33 of 83 cited sources checked · 9 supported · 3 flagged · accuracy not yet measured

Show all 134 claims

  • It may state two separate claims as one. 82
  • Probably not a substantial claim. 76
  • The quote only partly states this claim. 1
129 of 129
  • The authors state this as an assumption.

    The reduction in dimensionality in 2D/3D registration originates from a projective, not an affine, transformation.

    “For the purposes of this review, we will assume that the reduction in dimensionality originates from a projective, not an affine, transformation.”
  • The authors state this as an assumption.

    The 2D view modality is X-ray.

    “Since the vast majority of studies examined concern 2D X-ray images, we also assume that 2D view modality is X-ray.”
  • The paper relies on this without support.

    New researchers typically spend a great deal of time implementing a traditional registration pipeline.

    “New researchers typically spend a great deal of time implementing a traditional registration pipeline so that traditional methods may be leveraged in conjunction with the development of learning-based approaches.”
  • The paper relies on this without support.

    2D/3D registration leads a niche existence.

    “2D/3D registration already leads a niche existence”
  • The paper relies on this without support.

    Machine learning models have a compact domain assumption that confines solutions to the data domain they were developed on.

    “the compact domain assumption of machine learning models that confine solutions to the data domain they were developed on.”
5 of 5

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

Each contradiction between two passages listed here was confirmed by an independent check.

  • Full-text exclusion count conflicts with the reported 48 included studies

    Minor

    The text says 48 articles remained after screening and all 48 underwent full-text review. The figure caption instead says a further 12 were excluded after full-text review, yet still reports 48 included studies.

  • Representation-learning study list contains 13 studies, not 12

    Minor

    The cited list names 13 distinct studies, including two by Chou and Pizer and two by Zhang, despite the stated count of 12.

  • Table 2 assigns two different entries to Miao et al. (2016a)

    Minor

    Table 2 lists Miao et al. (2016a) twice with different training-domain-transfer entries. Table 1 identifies separate Miao et al. (2016a) and (2016b) studies, but Table 2 has no (2016b) entry.

3 of 3

Citations

Cites

Flagged first, then by citations

Show all 131 references

14 of the 131 references couldn’t be matched to a known paper.

Research works printed but not linked: 6

  1. Amrehn, 2017, Ui-net: Interactive Artificial Neural Networks for Iterative Image Segmentation Based on a User Model, Proc. Eurographics Workshop Vis. Comput. Biol. Med., 143
  2. Chen, 2018, Real-time 2d/3d Registration of Vertebra via Machine Learning and Geometric Transformation, Zidonghua Xuebao/Acta Automatica Sinica, 44, 1183
  3. Miao, Real-time 2d/3d Registration via Cnn Regression, 1430
  4. 2021, Artificial Intelligence/machine Learning (Ai/ml)-based Software as a Medical Device (Samd) Action Plan
  5. Varnavas, Fully Automated 2d-3d Registration and Verification, Med. image Anal., 26, 108, 10.1016/j.media.2015.08.005 · DOI · Paper page
  6. Xiangqian, 2020, 2d/3d Medical Image Registration Using Convolutional Neural Network, Chin. J. Biomed. Eng., 39, 394

Cited by

Sorted by citations

Items 101–101 of 101

Metadata

Authors
Unberath, M., Gao, C., Hu, Y., Judish, M., Taylor, R. H., Armand, M., & Grupp, R.
Affiliations
Johns Hopkins University (US)
Venue
Frontiers in Robotics and AI
Published
2021-08-30
Volume
8
Pages
716007-716007
ISSN
2296-9144 (ISSN-L)
OpenAlex
W3190313291
Cited by
102
References
131
Citations by year
2026: 19 · 2025: 23 · 2024: 33 · 2023: 17 · 2022: 9 · 2021: 1
FWCI
2.14
Type
journal article
Language
en
Open access
Locations
  • Frontiers in Robotics and AI (publishedVersion, cc-by)
  • PubMed (publishedVersion)
  • arXiv (Cornell University) (submittedVersion)
  • PubMed Central (submittedVersion, cc-by)
  • DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
  • arXiv (Cornell University) (cc-by)
Funders
National Institute of Biomedical Imaging and Bioengineering (R21 EB028505)
Topics
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Medical Image Segmentation Techniques
  • Physical Sciences › Engineering › Biomedical Engineering › Anatomy and Medical Technology
  • Health Sciences › Medicine › Surgery › Surgical Simulation and Training
Keywords
2D-3D registration, image-guided interventions, machine learning, image-based navigation, minimally invasive surgery, fluoroscopy, endoscopy, surgical navigation, open problems

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

A language model drew the claims from the paper on 2026-10-14, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).

A second model labelled each claim on 2026-10-14 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).

Cited sources are checked with a language model’s help (gpt-6-sol).

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review