CoChat
Sign Up
Cited by 37Medical Image Segmentation TechniquesAdvanced Radiotherapy TechniquesMedical Imaging and Analysis

A Fully Differentiable Framework for 2D/3D Registration and the Projective Spatial Transformers

Open access
Gao, Cong, Feng, Anqi, Liu, Xingtong, Taylor, Russell H., Armand, Mehran, Unberath, Mathias·2024·IEEE Transactions on Medical Imaging
DOI 10.1109/tmi.2023.3299588PMC PMC10879149
Open this paper in CoChat — free accountOpen PDF

Abstract

Image-based 2D/3D registration is a critical technique for fluoroscopic guided surgical interventions. Conventional intensity-based 2D/3D registration approa- ches suffer from a limited capture range due to the presence of local minima in hand-crafted image similarity functions. In this work, we aim to extend the 2D/3D registration capture range with a fully differentiable deep network framework that learns to approximate a convex-shape similarity function. The network uses a novel Projective Spatial Transformer (ProST) module that has unique differentiability with respect to 3D pose parameters, and is trained using an innovative double backward gradient-driven loss function. We compare the most popular learning-based pose regression methods in the literature and use the well-established CMAES intensity-based registration as a benchmark. We report registration pose error, target registration error (TRE) and success rate (SR) with a threshold of 10mm for mean TRE. For the pelvis anatomy, the median TRE of ProST followed by CMAES is 4.4mm with a SR of 65.6% in simulation, and 2.2mm with a SR of 73.2% in real data. The CMAES SRs without using ProST registration are 28.5% and 36.0% in simulation and real data, respectively. Our results suggest that the proposed ProST network learns a practical similarity function, which vastly extends the capture range of conventional intensity-based 2D/3D registration. We believe that the unique differentiable property of ProST has the potential to benefit related 3D medical imaging research applications. The source code is available at https://github.com/gaocong13/Projective-Spatial-Transformers.

AI snapshot

Written from the paper’s full text.

TL;DR

The paper introduces a fully differentiable 2D/3D registration framework built around a Projective Spatial Transformer (ProST) that learns a convex-shape image similarity function. It reports that ProST followed by CMAES substantially extends the capture range of conventional intensity-based registration on simulated and real pelvis X-rays.

Key findings

  • For pelvis, ProST followed by CMAES achieved median TRE 4.4 mm with 65.6% success rate in simulation, and median TRE 2.2 mm with 73.2% success rate on real standard AP X-rays; CMAES alone achieved 28.5% and 36.0% success rates, respectively.
  • On spine simulation, ProST followed by CMAES improved success rate from 23.3% to 39.4%, with median TRE 43.2 mm versus 101.6 mm for CMAES alone.
  • Ablations showed worse performance without key components: MSE loss training reached 27.9% simulation and 28.8% real success rates; removing the 3D network reached 48.3% and 49.2%; using full CT instead of segmented bone reached 59.4% real success rate.
  • For real images with surgical tool overlay, ProST prediction median TRE remained similar (96.5 mm tool overlay vs 96.1 mm baseline in Table II), but subsequent CMAES success rate dropped to 48.4% versus 73.2% for standard real images.

Methods

  • The framework uses a differentiable ProST module for DRR generation, a CrossViT 2D encoder, a 3D CNN, double backward gradient-driven training loss, and domain randomization; training targets were generated with DeepDRR from NMDID CT scans.
  • Simulation studies used 20 pelvis and 20 spine CT scans with 4-fold cross-validation; real-data testing used 100 standard AP and 100 challenging pelvis X-ray images, with five registrations per image.
  • Registration performance was compared against CMAES with P-Grad-NCC and against DeepIM and DMW, reporting pose error, target registration error (TRE), and success rate with a 10 mm TRE threshold.

Limitations

  • The paper states single-view 2D/3D registration has inherent ambiguity in depth translation and out-of-plane rotations, and the learned similarity has flattened minima near the target, limiting accuracy when the pose is close.
  • The paper states CT volumes and projections were downsampled to 128 voxels and 128×128 pixels, consuming about 15 GB GPU memory, which dropped information and limited registration accuracy; it also did not compare 3D CNN architectures due to memory and optimization constraints.
  • The paper states the method only concerns rigid registration, runtime is about 15 seconds versus 7 seconds for a landmark-based conventional method, and challenging-view real X-rays generalized poorly with highest success rate 45.8%; the text also makes evident that real-data ground-truth poses were estimated using another registration pipeline rather than independent ground truth.

Model-written summary. Check it against the paper before relying on it.

Integrity signals

Registry

Crossref · 10.1109/tmi.2023.3299588

Cites

55 of 55 references verified0 citation errors · 0 not checkable

Statistics

Nothing to recompute.

Claims the paper makes

26 of 43 cited sources checked · 7 supported · 1 flagged · accuracy not yet measured

  • Premisecitation

    Geodesic loss L_geo is convex with respect to pose transformations.

    “Geodesic loss, L geo , which is the square of the geodesic distance in SE(3), has been studied for registration problems due to its convexity with respect to pose transformations [32] [33].”
    Quote in the paperSubstantial claim1 of 2 sources checked
    • cites [32] Salehi, 2018, Real-time deep pose estimation with geodesic loss for image-to-template rigid registration, arXiv:1803.05982 source
    • cites [33] source

      In §3.4 (page unavailable), the paper says its geodesic-loss optimization encounters bad local minima and uses an initial MSE phase to avoid them, contrary to the claimed convexity. The paper discusses a loss on rotations in SO(3), not a squared geodesic distance in SE(3). (LLM, conf=0.77)

      Checked by a reasoning model — may be wrong.

      Experimental — accuracy not yet measured
  • Premisecitation

    Conventional intensity-based 2D/3D registration approaches suffer from a limited capture range due to local minima in hand-crafted image similarity functions.

    “Conventional intensity-based 2D/3D registration approaches suffer from a limited capture range due to the presence of local minima in hand-crafted image similarity functions.”
    • Also stated in: § I. INTRODUCTION · plain
      “One challenge of such approaches is that the capture range is very limited due to the presence of local minima of hand-crafted loss functions.”
    Quote in the paperSubstantial claim2 of 2 sources checked
    • cites [7] source

      The Introduction (page unavailable) says that correlation of sparse image features can be insensitive to misregistration at larger distances, supporting a limited-capture-range concern. A separate passage (‘Two random variables,’ page unavailable) describes local maxima and a smaller attraction basin for a mutual-information criterion after image intensities were spatially shuffled. Neither passage establishes that local minima in hand-crafted similarity functions cause limited capture range specifically in conventional 2D/3D registration. (LLM, conf=0.83)

      Experimental — accuracy not yet measured
    • cites [8] source

      The abstract (p. 1) says existing similarity metrics struggle with the mismatch between an intact preoperative pelvis and a fractured intraoperative pelvis. Section 2.1 (pp. 3–4) describes intensity-based registration and a limitation of normalized cross-correlation, but neither passage addresses limited capture range caused by local minima. Searches for “capture range” and “local minima” found no passage supporting that explanation. (LLM, conf=0.72)

      Experimental — accuracy not yet measured
  • Premisecitation

    Current learning-based iterative 2D/3D registration methods predict a pose update from 2D image observations by either framing it as a reinforcement learning task or direct gradient update regression.

    “The current learning-based iterative 2D/3D registration methods in literature aim at predicting a pose update from 2D image observations by either framing it as a reinforcement learning task or direct gradient update regression.”
    Quote in the paperSubstantial claim1 of 2 sources checked
    • cites [19] source
    • cites [20] source

      The Methods section describes an iterative CNN that compares two 2D images to predict relative pose, and the Experiments and Results section says the network predicts gradients during registration. These passages support the gradient-update part of the claim, but our copy does not describe reinforcement learning as the other approach used by current methods. (LLM, conf=0.85)

      Experimental — accuracy not yet measured
  • Premisereasoning

    Learning-based iterative 2D/3D registration methods that do not optimize a cost function have no guarantee of convergence to global or local minima.

    “Since the registration does not optimize a cost function, but the iterative pose prediction is extracted from 2D images, there is no guarantee for convergence to global or local minima.”
    Quote in the paperSubstantial claim1 of 2 sources checked
    • cites [19] source
    • cites [20] source

      The Methods section describes iterative 2D/3D registration in which a CNN predicts pose updates from two 2D images. It does not state that such methods lack a guarantee of convergence to global or local minima, or attribute that lack of guarantee to not optimizing a cost function. Instead, it says the estimated similarity metric makes the problem globally convex in theory. (LLM, conf=0.78)

      Experimental — accuracy not yet measured
  • Premisecitation

    DeepDRR is effective on learning-based X-ray imaging tasks compared to naive DRR.

    “Target images during training were generated using a physically-realistic X-ray simulation framework -DeepDRR [36], which is shown to be effective on learning-based X-ray imaging tasks compared to naive DRR [37], [38].”
    Quote in the paperSubstantial claim1 of 2 sources checked
    • cites [37] source
    • cites [38] Gao, 2022, SyntheX: Scaling up learning-based X-ray image analysis through in silico experiments, arXiv:2206.06127 source
  • Classified as probably not a substantial claim. 25
  • May combine two claims. 40
  • The quote only partly states this claim. 1

  • Classified as probably not a substantial claim.

    Premisecitation

    Direct pose regression methods aim to fit a mapping function from 2D image observations to 3D pose or pose updates.

    “Although the architectures varied, all the above methods aim to directly fit a mapping function from 2D image observations to 3D pose or pose updates, taking deep networks as powerful encoders.”
    Quote in the paperSubstantial claim1 of 3 sources checked
    • cites [18] source
    • cites [19] source
    • cites [20] source

  • May combine two claims.

    Premisecitation

    Feature-based 2D/3D registration methods require features or anatomical landmarks that are specific to every object and must be selected with domain knowledge and be clearly visible in target images.

    “However, these features and anatomical landmarks are specific to every object and must be selected with domain knowledge and such that they are clearly visible in the target images.”
    Quote in the paperSubstantial claim1 of 3 sources checked
    • cites [9] source

      Section 2.1 describes 2D/3D femur registration using matching feature points, supporting the claim that this method uses image features. But Sections 2.1 and 3.2 say SIFT detects and matches those points; the paper does not say they must be selected with domain knowledge, are specific to every object, or must be clearly visible in target images. Page numbers are unavailable in our copy. (LLM, conf=0.83)

      Experimental — accuracy not yet measured
    • cites [10] source
    • cites [11] source

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    The CMAES benchmark method for pose estimation of bone anatomy and surgical devices has been tested to meet clinical requirements in various orthopedic applications.

    “Due to its well-known robustness to local minima, the CMAES benchmark method for pose estimation of bone anatomy and surgical devices has been tested to meet clinical requirements in various orthopedic applications, including osteotomy [2], [13], femoroplasty [41], [4], core decompression of the hip [42], and transforaminal lumbar epidural injections [43], etc.”
    Quote in the paperNot a substantial claim2 of 3 sources checked
    • cites [2] source
    • cites [13] source

      The Methods (Training Dataset Creation and Intraoperative Registration; pages unavailable) use CMA-ES for pelvis and femur registration, and the Results evaluate registration on cadaveric hip fluoroscopy. The paper does not test CMA-ES for surgical-device pose estimation or show that it meets clinical requirements across the listed orthopedic applications; the Discussion describes registration of instruments as future research. (LLM, conf=0.88)

      Experimental — accuracy not yet measured
    • cites [41] source

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    DMW regresses pose updates from moving and target images.

    “DMW is also proposed to extend the capture range of 2D/3D registration by regressing pose updates from moving and target images [20].”
    Quote in the paperNot a substantial claim1 of 1 sources checked

  • May combine two claims.

    Premisecitation

    Learning from only a 2D domain without 3D parameters limits the network ability, likely to overfit the training domain.

    “Learning from only a 2D domain without 3D parameters limits the network ability, likely to overfit the training domain.”
    Quote in the paperSubstantial claim1 of 2 sources checked
    • cites [19] source
    • cites [20] source

      The Methods section describes a CNN that estimates relative pose from two 2D images, and the Discussion and Conclusion reports compromised performance on real data and a need to improve generalization to clinical X-rays (page numbers unavailable). The paper does not say that omitting 3D parameters limits the network’s ability or causes it to overfit the training domain. (LLM, conf=0.82)

      Experimental — accuracy not yet measured

  • Classified as probably not a substantial claim.

    Study design and conductcitation

    Groundtruth poses of the pelvis were estimated using the comprehensive image intensity-based 2D/3D registration pipeline described in [13].

    “Groundtruth poses of the pelvis were estimated using the comprehensive image intensity-based 2D/3D registration pipeline described in [13].”
    Quote in the paperSubstantial claim1 of 1 sources checked

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    There are popular feature-based 2D/3D registration methods, such as solving a PnP problem using corresponding anatomical landmarks, which may be complementary to intensity-based registration methods.

    “We are aware that there are popular feature-based 2D/3D registration methods, such as solving a PnP problem using corresponding anatomical landmarks [13], [16], [17], which may be complementary to the intensity-based registration methods.”
    Quote in the paperNot a substantial claim3 of 3 sources checked
    • cites [13] source
    • cites [16] source

      The Materials and methods section (2D/3D registration; page unavailable) describes using corresponding 2D X-ray and 3D CT landmarks to initialize registration by estimating a projection matrix. It specifies a direct linear transform, not a PnP solver, and does not explicitly discuss complementarity with intensity-based registration. The Discussion and conclusion likewise describes landmark-based initialization, supporting that part of the claim. (LLM, conf=0.85)

      Experimental — accuracy not yet measured
    • cites [17] source

  • May combine two claims.

    Premisecitation

    Feature-based 2D/3D registration methods need to define meaningful landmark features manually and do not learn registration in an end-to-end fashion.

    “However, these methods need to define meaningful landmark features manually and do not learn registration in an end-to-end fashion.”
    Quote in the paperSubstantial claim3 of 3 sources checked
    • cites [13] source

      Methods > Data Preprocessing states that 14 anatomical landmarks were manually annotated in 3D. Methods > Intraoperative Registration describes a separate optimization step rather than end-to-end learned registration. But the paper’s proposed approach uses a CNN to locate landmarks in fluoroscopy and requires no human assistance intraoperatively, so it does not support the unqualified claim that feature-based methods need manual landmark definition during registration. The supplied copy gives no page numbers. (LLM, conf=0.82)

      Experimental — accuracy not yet measured
    • cites [16] source

      In Materials and methods (Data generation), the authors say they manually selected clinically meaningful landmarks. In Materials and methods (2D/3D registration), they describe learned 2D landmark detection followed by a separate closed-form registration calculation, rather than end-to-end learning of registration. This supports the claim for the paper’s method, but not its implication that feature-based methods generally require manually defined landmarks. (LLM, conf=0.85)

      Experimental — accuracy not yet measured
    • cites [17] source

      The introduction (page not supplied) says a prior landmark-based method required manual annotation of CT landmarks for each new patient. That supports the manual-landmark concern for that method, but not the claim that feature-based methods generally require manually defined features. The paper also describes its own approach as enabling end-to-end registration; it does not state that feature-based methods generally cannot learn registration end to end. (LLM, conf=0.82)

      Experimental — accuracy not yet measured

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    Registration pose error, TRE, and success rate are general measures to report 2D/3D registration performance, commonly used in the literature.

    “Our results presented using registration pose error, TRE, and success rate are general measures to report the 2D/3D registration performance, which are commonly used in the literature [51], [18], [24].”
    Quote in the paperNot a substantial claim2 of 3 sources checked
    • cites [51] source

      Section 5 (Experiments and Results; page unavailable) uses TRE and success rate to evaluate registration performance. It evaluates 3-D/3-D registration, however, and does not report registration pose error or establish that all three measures are commonly used for 2-D/3-D registration. (LLM, conf=0.87)

      Experimental — accuracy not yet measured
    • cites [18] source
    • cites [24] source

      The Robotic Technique section describes a 2D/3D registration pipeline, but does not identify measures for evaluating its performance. Statistical Analysis reports needle-tip distance and angular-orientation errors for the injection procedure, not registration pose error, TRE, or registration success rate. The searched passages do not address whether those three measures are commonly used in the literature; page numbers are unavailable in our copy. (LLM, conf=0.75)

      Experimental — accuracy not yet measured

  • May combine two claims.

    Findingown data

    The ProST baseline model followed by CMAES performed the best across all ablation studies, achieving a median error of 0.26 degrees, 4.0 mm in simulation, and 13.6 degrees, 20.0 mm in real data, in rotation and translation, respectively.

    “The ProST baseline model followed by CMAES performed the best across all the ablation studies, which achieved a median error of 0.26 degrees, 4.0 mm in simulation, and 13.6 degrees, 20.0 mm in real data, in rotation and translation, respectively.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    CMAES from initialization achieved a median error of 22.4 degrees, 82.3 mm in simulation, and 57.9 degrees, 134.5 mm in real data, in rotation and translation, respectively.

    “The CMAES from initialization achieved a median error of 22.4 degrees, 82.3 mm in simulation, and 57.9 degrees, 134.5 mm in real data, in rotation and translation, respectively.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    On spine anatomy, the success rate was 39.4%, compared to 23.3% using CMAES from initialization.

    “The success rate was 39.4%, compared to 23.3% using CMAES from initialization.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    The network similarity is smoother and contains fewer local minima than Grad-NCC, especially in translations.

    “The similarity plots in Fig. 5 show that the network similarity is smoother and contains fewer local minima, especially in translations.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingreasoning

    Using the full CT volume as input performs worse than using the segmented bone volume as input, with a real data success rate of 59.4%.

    “We also performed a comparison study for using the full CT volume as input, and its performance is worse than using the segmented bone volume as input with real data success rate of 59.4%.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    The 3D pose rendering shows that the object poses were much closer to ground truth after ProST registration, which then successfully converged to the global optimum using CMAES.

    “The 3D pose rendering shows that the object poses were much closer to ground truth after ProST registration, which then successfully converged to the global optimum using CMAES.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    Using ProST baseline model, the CMAES registration success rate improved from 28.5% to 65.6% in simulation and 36.0% to 73.2% in real data.

    “Using our ProST baseline model, the CMAES registration success rate improved from 28.5% to 65.6% in simulation and 36.0% to 73.2% in real data.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    The median TRE for real images with tool overlays was 96.1 mm and without tool overlays was 96.5 mm.

    “The median TRE values for real images with and without tool overlays were 96.1 mm and 96.5 mm, respectively.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    For pelvis anatomy, median TRE of ProST followed by CMAES is 4.4 mm with SR 65.6% in simulation, and 2.2 mm with SR 73.2% in real data.

    “For the pelvis anatomy, the median TRE of ProST followed by CMAES is 4.4 mm with a SR of 65.6% in simulation, and 2.2 mm with a SR of 73.2% in real data.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    ProST shifted the error distribution from initialization closer to zero, which resulted in a much higher success rate for CMAES, compared to CMAES from initialization.

    “We clearly observe that ProST shifted the error distribution from initialization closer to zero, which resulted in a much higher success rate for CMAES, compared to CMAES from initialization.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingreasoning

    After CMAES registration, success rates were 73.2% for images with tool overlays and 48.4% for images without tool overlays.

    “However, the success rates decreased after performing the CMAES registration, with success rates of 73.2% and 48.4% observed for images with and without tool overlays, respectively.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    The correlation coefficient between similarity values and Riemannian distances is 0.56 for the network and 0.23 for Grad-NCC.

    “The correlation coefficient is 0.56 for the network and 0.23 for Grad-NCC.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    The success rates of the ProST baseline model are 65.6% in simulation and 73.2% in real data.

    “In our comparison experiments, the success rates of ProST baseline model in simulation and real are 65.6% and 73.2%, respectively.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    Registration results on challenging views are all deteriorated from standard view results, with the highest success rate of 45.8% for the ProST baseline model.

    “They are all deteriorated from the standard view image results with the highest success rate of 45.8%, corresponding to the ProST baseline model.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingreasoning

    ProST network registration is particularly effective when the object is far away, but its accuracy is limited by ambiguity when the pose is close to the target.

    “ProST network registration is particularly effective when the object is far away, but its accuracy is limited by this ambiguity when the pose is close to the target.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    The network similarity presents flattened bottoms while Grad-NCC has a sharp curve around global minima in the similarity loss shape plot.

    “This ambiguity can be observed in the similarity loss shape plot in Fig. 5, where the network similarity presents flattened bottoms while Grad-NCC has a sharp curve around global minima.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    The MICCAI architecture achieves success rates of 45.0% in simulation and 51.2% in real data, both worse than the CrossViT baseline model.

    “Our comparison experiment of MICCAI architecture achieves success rates of 45.0% in simulation and 51.2% in real data, which are both worse than the CrossViT baseline model.”
    Quote in the paperSubstantial claim

  • The quote only partly states this claim.

    Findingown data

    The success rate of ProST registration improves from 23.3% to 39.4% on the spine.

    “The success rate of ProST registration improves from 23.3% to 39.4%”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    The No DR results are slightly worse than the baseline model, with success rates of 65.4% in simulation and 71.4% in real.

    “The No DR results are slightly worse than the baseline model, with success rates of 65.4% in simulation and 71.4% in real.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data

    The proposed registration method largely extends the conventional intensity-based registration capture range and performs more robustly than other learning-based pose regression methods in the literature.

    “Our results suggest that the proposed registration method largely extends the convention intensity-based registration capture range and performs more robustly than the other learning-based pose regression methods in the literature.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    The ProST network registration takes about 10 seconds, the following CMAES intensity-based registration takes around 5 seconds, and the full registration workflow runtime is about 15 seconds.

    “The ProST network registration takes about 10 seconds, and the following CMAES intensity-based registration takes around 5 seconds, resulting in the full registration workflow runtime to be about 15 seconds.”
    Quote in the paperNot a substantial claim

  • Classified as probably not a substantial claim.

    Study design and conductnone given

    Target and moving image pairs of DeepIM and DMW comparison studies were generated using DeepDRR from the same CT data and pose distributions as ProST baseline training.

    “Target and moving image pairs of these two comparison studies were generated using DeepDRR from the same CT data and pose distributions as ProST baseline training.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Study design and conductnone given

    The similarity plots in Fig. 5 serve as an example and the convexity is related to the specific testing image.

    “The plots in Fig. 5 serves as an example and the convexity is related to the specific testing image.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductnone given

    K-fold cross-validation was used to improve the representation of the sample population.

    “We utilized K-fold cross-validation to improve the representation of the sample population.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Interpretationreasoning

    The bone is a rigid object with higher attenuation than soft tissue, which is better for feature extraction.

    “We attribute the reason to be that the bone is a rigid object with higher attenuation than soft tissue, which is better for feature extraction.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Interpretationown data

    The network generalizes well on unseen real cadaveric images, indicating superior generalization ability.

    “but generalizes well on unseen real cadaveric images, indicating its superior generalization ability.”
    Quote in the paperSubstantial claim

  • Classified as probably not a substantial claim.

    Study design and conductnone given

    The ablation study removed the 3D CNN part while keeping other training/testing settings the same.

    “We performed a controlled ablation study to demonstrate the importance of 3D parameters in the architecture by removing the 3D CNN part and keeping the other training/testing settings the same.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Interpretationown data

    The proposed ProST network learns a practical similarity function, which vastly extends the capture range of conventional intensity-based 2D/3D registration.

    “Our results suggest that the proposed ProST network learns a practical similarity function, which vastly extends the capture range of conventional intensity-based 2D/3D registration.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Interpretationnone given

    The proposed end-to-end DeepNet architecture uses ProST and learns to approximate the convex shape of L_geo to extend the capture range of 2D/3D registration.

    “Using ProST, we propose an end-to-end DeepNet architecture that learns to approximate the convex shape of L geo , aiming at extending the capture range of 2D/3D registration.”
    Quote in the paperSubstantial claim

  • Classified as probably not a substantial claim.

    Study design and conductnone given

    Coordinate frames of DeepDRR and xReg were calibrated to the ProST geometry convention.

    “The coordinate frames of DeepDRR and xReg were calibrated to the ProST geometry convention as introduced in Section II-A.1.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Interpretationreasoning

    The network's similarity computation relies on global image information, enabling it to be more resilient to local feature changes.

    “On the other hand, the network's similarity computation relies on global image information, enabling it to be more resilient to local feature changes.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S4 (based).

    Interpretationnone given

    ProST has unique differentiability with respect to 3D pose parameters.

    “The network uses a novel Projective Spatial Transformer (ProST) module that has unique differentiability with respect to 3D pose parameters”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductnone given

    Except for No DR, domain randomization was consistently applied in all other experiments.

    “Of note, except for No DR, domain randomization was consistently applied in all the other experiments.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisenone given

    Feature-based 2D/3D registration methods are an alternative to intensity-based approaches, where analytical pose solutions are computed from corresponding 2D and 3D geometric features.

    “Feature-based 2D/3D registration methods are an alternative to intensity-based approaches, where analytical pose solutions are computed from corresponding 2D and 3D geometric features.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Study design and conductnone given

    The training process took 15k different image pairs as input, which covers the sampling geometry and avoids overfitting.

    “The training process took 15k different image pairs as input, which covers the sampling geometry and avoids overfitting.”
    Quote in the paperNot a substantial claim

  • Classified as probably not a substantial claim. May combine two claims.

    Study design and conductnone given

    All experiments were fed with training data from the same distribution and applied the same training strategies.

    “All experiments were fed with training data from the same distribution and applied the same training strategies.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    DeepIM performs iterative pose updates using learned gradient predictions.

    “For example, Li et al. proposed DeepIM, which performs iterative pose updates using learned gradient predictions [19].”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [19] source

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductnone given

    The convexity of the network similarity function was not measured.

    “Of note, we do not measure the convexity of the network similarity function.”
    Quote in the paperNot a substantial claim

  • Classified as probably not a substantial claim. May combine two claims.

    Study design and conductnone given

    The hyper parameters were not optimized, but training settings were controlled to be the same for comparison.

    “Of note, we did not optimize the hyper parameters but we controlled the training settings to be the same for comparison.”
    Quote in the paperSubstantial claim

  • Classified as probably not a substantial claim.

    Premisecitation

    The CMAES benchmark method is well-known robust to local minima.

    “Due to its well-known robustness to local minima, the CMAES benchmark method”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [40] source

  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based). May combine two claims.

    Study design and conductnone given

    The CMAES optimizer was set with a pose sampling population size of 300 in each iteration, and sigmas of 45 degrees and 200 mm for rotation and translation, respectively, which was sufficient to cover the testing pose sampling space.

    “The CMAES optimizer was set with a pose sampling population size of 300 in each iteration, and sigmas of 45 degrees and 200 mm for rotation and translation, respectively, which was sufficient to cover the testing pose sampling space.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Study design and conductreasoning

    This work only concerns rigid registrations because the sampling grid design is rigid and fixed.

    “This work only concerns rigid registrations because the sampling grid design is rigid and fixed.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Study design and conductreasoning

    This work does not tie to a specific clinical application, and thus does not compare against specific clinical requirements.

    “This work does not tie to a specific clinical application, and thus we do not compare it against specific clinical requirements.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based). May combine two claims.

    Premisenone given

    Pose estimation of the spine is more difficult than the pelvis because the spine is symmetric and the vertebrae are smaller.

    “Pose estimation of the spine is more difficult than the pelvis because the spine is symmetric and the vertebrae are smaller.”
    Quote in the paperNot a substantial claim
    • Rests on an assumption: The spine is symmetric and its vertebrae are smaller than the pelvis.

  • May combine two claims.

    Interpretationnone given

    The paper presents a fully differentiable 2D/3D registration framework that learns a convex shape similarity function using a novel projective spatial transformer module.

    “In this work, we present a fully differentiable 2D/3D registration framework, which learns a convex shape similarity function using a novel projective spatial transformer module.”
    Quote in the paperSubstantial claim

  • Classified as probably not a substantial claim. May combine two claims.

    Study design and conductreasoning

    This work does not compare choices of 3D CNN architectures because the 3D gradient flow makes network optimization complicated and memory limitation restricts the choice of advanced networks.

    “This work does not compare choices of 3D CNN architectures, because the 3D gradient flow makes the network optimization complicated, and the memory limitation restricts the choice of advanced networks.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Interpretationreasoning

    Downsampling dropped information from the original data and limited the registration accuracy.

    “Such downsampling dropped information from the original data and limited the registration accuracy.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing).

    Interpretationnone given

    ProST has the potential to benefit learning-based 3D medical imaging research applications.

    “We believe that ProST has the potential to benefit learning-based 3D medical imaging research applications.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Interpretationreasoning

    The combination of ProST learning-based registration and CMAES registration effectively corrects the inherent ambiguity and facilitates accurate pose estimation.

    “This suggests that the combination of the ProST learning-based registration and the CMAES registration effectively corrects the inherent ambiguity and facilitates accurate pose estimation.”
    Quote in the paperSubstantial claim
58 of 58

  • The paper relies on this without support.

    The mapping from 2D image observations to 3D pose or pose updates is highly complex and ill-posed in transmission X-ray imaging.

    “However, mathematically, such mapping is a highly complex and illposed problem in transmission X-ray imaging.”

  • The paper relies on this without support.

    Fitting a neural network to a target function shape has lower problem complexity than mapping to precise pose values.

    “1) the problem complexity of fitting the neural network to a target function shape is lower than mapping to precise pose values”

  • The paper relies on this without support.

    Iterative optimization of a learned similarity function is more likely to converge to the global optimum than unconstrained prediction mapping.

    “2) the iterative optimization is more likely to converge to the global optimum than unconstrained prediction mapping.”

  • The paper relies on this without support.

    Differentiable gradient computation for DRR projection with respect to both input data and pose parameters has not yet been explored.

    “Unfortunately, this differentiable gradient computation for DRR projection with respect to both, the input data as well as the pose parameters, has not yet been explored.”

  • The authors state this as an assumption.

    The similarity function of X-ray image 2D/3D registration has not previously been learned to a target convex shape in an end-to-end fashion.

    “To our knowledge, this is the first time that the similarity function of X-ray image 2D/3D registration is learned to a target convex shape in an end-to-end fashion.”

  • The paper relies on this without support.

    The spine is symmetric and its vertebrae are smaller than the pelvis.

    “Pose estimation of the spine is more difficult than the pelvis because the spine is symmetric and the vertebrae are smaller.”

  • The paper relies on this without support.

    The mapping that the ProST method needs to learn is simple.

    “This design potentially makes generalization easier because the mapping that our method needs to learn is simple.”
7 of 7

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Gao et al. use a neural network to learn a convex image similarity function for iterative optimization.

    “Gao et al. [19] use a neural network to learn a convex image similarity function for iterative optimization”

    From Rapid patient-specific neural networks for intraoperative X-ray to volume registration (2025) · cites this paper as [19]

    The Introduction and “Approximating Convex Image Similarity” section say Gao et al. use a network to learn an image-similarity function toward a convex shape for iterative 2D/3D registration. The Discussion explicitly says they do not measure the learned function’s convexity, so describing it as a convex function overstates what the paper establishes. (LLM, conf=0.86)

    Experimental — accuracy not yet measured

  • Both Gao et al. and Grimm et al. evaluate their methods on real pelvic X-rays, with Gao et al. also using the DeepFluoro dataset.

    “Both papers evaluate their methods on real pelvic X-rays, with Gao et al. [19] also using the DeepFluoro dataset”

    From Rapid patient-specific neural networks for intraoperative X-ray to volume registration (2025) · cites this paper as [19]

    The Real Data Study section says Gao et al. tested on real hip X-rays selected from cadaveric X-rays released by Grupp et al., supporting the real-X-ray portion attributed to Gao. The copy does not identify those images as the DeepFluoro dataset; a search for “DeepFluoro” found no matching passage. This source cannot establish what Grimm et al. evaluated. (LLM, conf=0.84)

    Experimental — accuracy not yet measured

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

Each contradiction between two passages listed here was confirmed by an independent check.

  • Real-data ProST-plus-CMAES pose-error medians disagree with the reported results

    Major

    The results text gives real-data median errors of 13.6 degrees and 20.0 mm, while the accompanying pose-error results give 0.5 degrees and 1.7 mm for the same method.

  • Training image-pair count omits the batch size

    Minor

    At 50 iterations per epoch for 300 epochs with a batch size of 4, training processes 60,000 image-pair instances, not 15,000. The stated 15k equals the number of iterations rather than the number of pairs.

    Reported 15000 · recomputed 60000

  • Tool-overlay results are assigned to the wrong image groups

    Minor

    Table II assigns 96.1 mm median TRE and 73.2% post-CMAES success to the baseline without tool overlay, and 96.5 mm and 48.4% to tool-overlay images. The discussion reverses those assignments, affecting its account of how overlays change performance.

3 of 3

Cites

Flagged first, then by citations

8 of the 63 references couldn’t be matched to a known paper.

Research works printed but not linked: 1

  1. Edgar, 2020, New Mexico Decedent Image Database

Cited by

Sorted by citations

Related

Authors
Gao, C., Feng, A., Liu, X., Taylor, R. H., Armand, M., & Unberath, M.
Affiliations
Johns Hopkins University (US)
ORCID
Venue
IEEE Transactions on Medical Imaging
Publisher
Institute of Electrical and Electronics Engineers (IEEE)
Published
2024-01
Volume
43
Issue
1
Pages
275-285
ISSN
  • 0278-0062 (ISSN-L)
  • 1558-254X
DOI
10.1109/tmi.2023.3299588
PMC
PMC10879149
OpenAlex
W4385626871
Semantic Scholar
ea169a5951a75723725f2188210550891b5b7649
Cited by
36
References
63
Citations by year
2026: 14 · 2025: 7 · 2024: 14 · 2023: 1
FWCI
3.28
Type
journal article
Language
en
Open access
Licence
Locations
  • IEEE Transactions on Medical Imaging (publishedVersion)
  • PubMed (publishedVersion)
  • PubMed Central (submittedVersion)
Funders
  • U.S. National Institutes of Health (R21EB028505, R01EB016703, R01EB023939, R21EB020113)
  • Johns Hopkins Internal Funds
Topics
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Medical Image Segmentation Techniques
  • Physical Sciences › Physics and Astronomy › Radiation › Advanced Radiotherapy Techniques
  • Physical Sciences › Engineering › Biomedical Engineering › Medical Imaging and Analysis
Keywords
2D-3D registration, differentiable neural networks, intensity-based registration, capture range, pose regression, fluoroscopy-guided surgery, target registration error
MeSH terms
Algorithms, Fluoroscopy, Pelvis, Software, Imaging, Three-Dimensional

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Claims read by deepseek-v4.1-flash under definition version 3 (extract-v3) on 2026-10-14.

Classified by jev-1.13-20260917 (gold-guideline-v0.2) on 2026-10-14.

Cited sources are checked by gpt-6-sol.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review