CoChat
Sign Up
Cited by 16Medical Image Segmentation TechniquesMedical Imaging and AnalysisAdvanced Radiotherapy Techniques

Extended Capture Range of Rigid 2D/3D Registration by Estimating Riemannian Pose Gradients

Open access
Gu, Wenhao, Gao, Cong, Grupp, Robert, Fotouhi, Javad, Unberath, Mathias·2020·Lecture Notes in Computer Science
DOI 10.1007/978-3-030-59861-7_29PMC PMC7605345
Open this paper in CoChat — free account

Abstract

Traditional intensity-based 2D/3D registration requires near-perfect initialization in order for image similarity metrics to yield meaningful updates of X-ray pose and reduce the likelihood of getting trapped in a local minimum. The conventional approaches strongly depend on image appearance rather than content, and therefore, fail in revealing large pose offsets that substantially alter the appearance of the same structure. We complement traditional similarity metrics with a convolutional neural network-based (CNN-based) registration solution that captures large-range pose relations by extracting both local and contextual information, yielding meaningful X-ray pose updates without the need for accurate initialization. To register a 2D X-ray image and a 3D CT scan, our CNN accepts a target X-ray image and a digitally reconstructed radiograph at the current pose estimate as input and iteratively outputs pose updates in the direction of the pose gradient on the Riemannian Manifold. Our approach integrates seamlessly with conventional image-based registration frameworks, where long-range relations are captured primarily by our CNN-based method while short-range offsets are recovered accurately with an image similarity-based method. On both synthetic and real X-ray images of the human pelvis, we demonstrate that the proposed method can successfully recover large rotational and translational offsets, irrespective of initialization.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1007/978-3-030-59861-7_29

Cites

6 of 6 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Current learning-based iterative 2D/3D registration methods predict a pose update from 2D image observations by either framing it as a reinforcement learning task or direct gradient update regression.

    “The current learning-based iterative 2D/3D registration methods in literature aim at predicting a pose update from 2D image observations by either framing it as a reinforcement learning task or direct gradient update regression.”

    From A Fully Differentiable Framework for 2D/3D Registration and the Projective Spatial Transformers (2024) · cites this paper as [20]

    The Methods section describes an iterative CNN that compares two 2D images to predict relative pose, and the Experiments and Results section says the network predicts gradients during registration. These passages support the gradient-update part of the claim, but our copy does not describe reinforcement learning as the other approach used by current methods. (LLM, conf=0.85)

    Experimental — accuracy not yet measured

  • Learning from only a 2D domain without 3D parameters limits the network ability, likely to overfit the training domain.

    “Learning from only a 2D domain without 3D parameters limits the network ability, likely to overfit the training domain.”

    From A Fully Differentiable Framework for 2D/3D Registration and the Projective Spatial Transformers (2024) · cites this paper as [20]

    The Methods section describes a CNN that estimates relative pose from two 2D images, and the Discussion and Conclusion reports compromised performance on real data and a need to improve generalization to clinical X-rays (page numbers unavailable). The paper does not say that omitting 3D parameters limits the network’s ability or causes it to overfit the training domain. (LLM, conf=0.82)

    Experimental — accuracy not yet measured

  • Learning-based iterative 2D/3D registration methods that do not optimize a cost function have no guarantee of convergence to global or local minima.

    “Since the registration does not optimize a cost function, but the iterative pose prediction is extracted from 2D images, there is no guarantee for convergence to global or local minima.”

    From A Fully Differentiable Framework for 2D/3D Registration and the Projective Spatial Transformers (2024) · cites this paper as [20]

    The Methods section describes iterative 2D/3D registration in which a CNN predicts pose updates from two 2D images. It does not state that such methods lack a guarantee of convergence to global or local minima, or attribute that lack of guarantee to not optimizing a cost function. Instead, it says the estimated similarity metric makes the problem globally convex in theory. (LLM, conf=0.78)

    Experimental — accuracy not yet measured

  • For learning-based iterative 2D/3D registration methods, the prediction can change drastically if the input image's appearance alters by a small change.

    “The prediction can change drastically if the input image's appearance alters by a small change.”

    From A Fully Differentiable Framework for 2D/3D Registration and the Projective Spatial Transformers (2024) · cites this paper as [20]

    The Results section reports that 15 synthetic cases failed because their appearance was atypical of the training images, and the Discussion says performance was compromised when tools or implants appeared in real images (page numbers unavailable in our copy). These passages support sensitivity to image appearance, but they do not show that a small appearance change causes a drastic change in prediction. (LLM, conf=0.81)

    Experimental — accuracy not yet measured

  • Direct pose regression methods aim to fit a mapping function from 2D image observations to 3D pose or pose updates.

    “Although the architectures varied, all the above methods aim to directly fit a mapping function from 2D image observations to 3D pose or pose updates, taking deep networks as powerful encoders.”

    From A Fully Differentiable Framework for 2D/3D Registration and the Projective Spatial Transformers (2024) · cites this paper as [20]

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Flagged first, then by citations

Research works printed but not linked: 21

  1. [3]Bier B, Unberath M, Zaech JN, Fotouhi J, Armand M, Osgood G, Navab N, Maier A: X-ray-transform invariant anatomical landmark detection for pelvic trauma surgery In: International Conference on Medical Image Computing and Computer-Assisted Intervention. pp. 55– 63. Springer (2018) 2, 3, 8
  2. [4]Clark K, Vendt B, Smith K, Freymann J, Kirby J, Koppel P, Moore S, Phillips S, Maffitt D, Pringle M,: The cancer imaging archive (tcia): maintaining and operating a public information repository. Journal of digital imaging 26(6), 1045– 1057 (2013) 6 23884657 PMC3824915
  3. [5]Deng J, Dong W, Socher R, Li LJ, Li K, Fei-Fei L: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE conference on computer vision and pattern recognition pp. 248– 255. Ieee (2009) 5
  4. [7]Fotouhi J, Fuerst B, Johnson A, Lee SC, Taylor R, Osgood G, Navab N, Armand M: Pose-aware c-arm for automatic re-initialization of interventional 2d/3d image registration. International journal of computer assisted radiology and surgery 12(7), 1221– 1230 (2017) 3 28527025 PMC5898215
  5. [8]Gong RH, Stewart J, Abolmaesumi P: Multiple-object 2-d–3-d registration for noninvasive pose identification of fracture fragments. IEEE Transactions on Biomedical Engineering 58(6), 1592– 1601 (2011) 1 21233043
  6. [10]Grupp RB, Hegeman RA, Murphy RJ, Alexander CP, Otake Y, McArthur BA, Armand M, Taylor RH: Pose estimation of periacetabular osteotomy fragments with intraoperative x-ray navigation. arXiv preprint arXiv:1903.09339 (2019) 1, 3, 4, 6 PMC7297497 31059424 · arXiv
  7. [11]Hajek J, Unberath M, Fotouhi J, Bier B, Lee SC, Osgood G, Maier A, Armand M, Navab N: Closing the calibration loop: an inside-out-tracking paradigm for augmented reality in orthopedic surgery. In: International Conference on Medical Image Computing and Computer-Assisted Intervention pp. 299– 306. Springer (2018) 3
  8. [13]Huang G, Liu Z, Van Der Maaten L, Weinberger KQ: Densely connected convolutional networks. In: Proceedings of the IEEE conference on computer vision and pattern recognition pp. 4700– 4708 (2017) 5
  9. [14]Ketcha M, De Silva T, Uneri A, Jacobson M, Goerres J, Kleinszig G, Vogt S, Wolinsky J, Siewerdsen J: Multi-stage 3d–2d registration for correction of anatomical deformation in image-guided spine surgery. Physics in Medicine & Biology 62(11), 4604 (2017) 1, 3 28375139 PMC5755708
  10. [15]Lemieux L, Jagoe R, Fish D, Kitchen N, Thomas D: A patient-to-computed-tomography image registration method based on digitally reconstructed radiographs. Medical physics 21(11), 1749– 1760 (1994) 2 7891637
  11. [16]Liao H, Lin WA, Zhang J, Zhang J, Luo J, Zhou SK: Multiview 2d/3d rigid registration via a point-of-interest network for tracking and triangulation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition pp. 12638– 12647 (2019) 3, 8
  12. [17]Liu L, Ecker T, Schumann S, Siebenrock K, Nolte L, Zheng G: Computer assisted planning and navigation of periacetabular osteotomy with range of motion optimization. In: International Conference on Medical Image Computing and Computer-Assisted Intervention pp. 643– 650. Springer (2014) 1 25485434
  13. [18]Liu R, Lehman J, Molino P, Such FP, Frank E, Sergeev A, Yosinski J: An intriguing failing of convolutional neural networks and the coordconv solution. In: Advances in Neural Information Processing Systems. pp. 9628– 9639 (2018) 5
  14. [19]Markelj P, Tomaževič D, Likar B, Pernuš F: A review of 3d/2d registration methods for image-guided interventions. Medical image analysis 16(3), 642– 661 (2012) 1, 2, 3 20452269
  15. [22]Murray RM: A mathematical introduction to robotic manipulation. CRC press (2017), chapter 3.2. 5
  16. [23]Otake Y, Armand M, Armiger RS, Kutzer MD, Basafa E, Kazanzides P, Taylor RH: Intraoperative image-based multiview 2d/3d registration for image-guided orthopaedic surgery: incorporation of fiducial-based c-arm tracking and gpu-acceleration. IEEE transactions on medical imaging 31(4), 948– 962 (2012) 1 22113773 PMC4451116
  17. [24]Ruijters D, ter Haar Romeny BM, Suetens P: Vesselness-based 2d–3d registration of the coronary arteries. International journal of computer assisted radiology and surgery 4(4), 391– 397 (2009) 2 20033586
  18. [25]Troelsen A, Elmengaard B, Søballe K: A new minimally invasive transsartorial approach for periacetabular osteotomy. JBJS 90(3), 493– 498 (2008) 1 18310698
  19. [26]Unberath M, Zaech JN, Gao C, Bier B, Goldmann F, Lee SC, Fotouhi J, Taylor R, Armand M, Navab N: Enabling machine learning in x-ray-based procedures via realistic simulation of image formation. International journal of computer assisted radiology and surgery pp. 1– 12 (2019) 6 PMC7297499 31187399
  20. [28]Yaniv Z: Registration for orthopaedic interventions In: Computational radiology for orthopaedic interventions, pp. 41– 70. Springer (2016) 1
  21. [29]Yao J, Taylor RH, Goldberg RP, Kumar R, Bzostek A, Van Vorhis R, Kazanzides P, Gueziec A: A c-arm fluoroscopy-guided progressive cut refinement strategy using a surgical robot. Computer Aided Surgery: Official Journal of the International Society for Computer Aided Surgery (ISCAS) 5(6), 373– 390 (2000) 1 11295851

Cited by

Sorted by citations

Authors
Gu, W., Gao, C., Grupp, R., Fotouhi, J., & Unberath, M.
Affiliations
Johns Hopkins University (US)
Corresponding author
Gu, Wenhao
ORCID
Venue
Lecture Notes in Computer Science
Publisher
Springer International Publishing
Published
2020
Volume
12436
Pages
281-291
ISSN
  • 0302-9743 (ISSN-L)
  • 1611-3349
DOI
10.1007/978-3-030-59861-7_29
PMC
PMC7605345
OpenAlex
W3090629712
Cited by
16
References
6
Citations by year
2026: 4 · 2024: 5 · 2023: 6 · 2021: 1
FWCI
0.54
Type
conference paper
Language
en
Open access
Licence
Locations
  • Lecture notes in computer science (publishedVersion)
  • PubMed (publishedVersion)
  • PubMed Central (acceptedVersion)
Funders
National Institute of Biomedical Imaging and Bioengineering (R21EB028505)
Topics
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Medical Image Segmentation Techniques
  • Physical Sciences › Engineering › Biomedical Engineering › Medical Imaging and Analysis
  • Physical Sciences › Physics and Astronomy › Radiation › Advanced Radiotherapy Techniques
Keywords
2D-3D registration, Riemannian manifolds, digitally reconstructed radiographs, intensity-based registration, human pelvis, computed tomography

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review