Intraoperative 2D/3D Image Registration via Differentiable X-Ray Rendering

May combine two claims.

Intensity-based methods have limited capture range and sensitivity to the initial pose estimate.

“A major limitation of intensity-based methods is their limited capture range and sensitivity to the initial pose estimate [52].”

• cites [52] sourceSupported

May combine two claims.

Landmark-based localization requires expert knowledge of landmarks visible on X-ray and manual annotation of 3D landmarks for every new CT volume.

“Such methods require expert knowledge of landmarks that are visible on X-ray for the specific surgical application and manual annotation of 3D landmarks for every new CT volume [25].”

• cites [25] sourcePartly supported

  Methods > Data Preprocessing reports that 14 anatomical landmarks were manually annotated in 3D, and Experiments and Results > Data Collection and Training Dataset Creation says landmarks were manually digitized for each CT scan in the study. These passages support the manual-annotation part, but do not establish that landmark-based methods generally require expert knowledge of X-ray-visible landmarks for each surgical application or manual annotation for every new CT volume. (LLM, conf=0.83)

  Experimental — accuracy not yet measured

May combine two claims.

Neural networks trained on small datasets for 2D/3D registration fail on new patients or require impractical landmark supervision.

“neural networks trained on small datasets fail on new patients or require impractical landmark supervision”

• Also stated in: § Introduction · plain

  “Such methods require expert knowledge of landmarks that are visible on X-ray for the specific surgical application and manual annotation of 3D landmarks for every new CT volume [25].”

• Also stated in: § Introduction · plain

  “only a few small labeled datasets exist for training deep networks [24, 41, 50], preventing the generalization of these methods to new patients, procedures, and pathologies.”

• cites [25] sourcePartly supported

  The Discussion and Conclusion (page unavailable) reports poor registration on one held-out specimen whose C-arm poses differed from those in training, supporting a limited generalization concern—not a general claim that networks trained on small datasets fail on new patients. The Methods, Training Dataset Creation (page unavailable) describes automatically generating landmark labels and manually verifying them; it does not establish that landmark supervision is impractical. (LLM, conf=0.84)

  Experimental — accuracy not yet measured
• cites [24] sourceCould not verify

  The Discussion (page unavailable) mentions neural networks as a possible way to avoid manual landmark identification, but does not discuss their training datasets, performance on new patients, or landmark supervision during training. The Results > Cadaver Study passage measures manual annotation time for the paper’s registration method, not for a neural network. Searches for small datasets, unseen patients, and landmark supervision found no passage addressing the claim. (LLM, conf=0.76)

  Experimental — accuracy not yet measured
• cites [41] sourceNot checked — no full text of the source
• cites [50] sourceNot checked — no full text of the source

May combine two claims.

Traditional renderers are non-differentiable, so pose estimation is commonly driven by gradient-free optimization.

“As traditional renderers are non-differentiable, pose estimation is commonly driven by gradient-free optimization [24].”

• cites [24] sourcePartly supported

  The Intraoperative Registration Strategy section (page unavailable) says the paper uses CMA-ES for pose optimization and explicitly calls it derivative-free; it also uses BOBYQA at a finer resolution. Neither that section nor Related Work (page unavailable) says traditional renderers are non-differentiable or identifies that as the reason for using gradient-free optimization. The paper supports the optimization practice, but not the claimed explanation. (LLM, conf=0.86)

  Experimental — accuracy not yet measured

Classified as probably not a substantial claim.

A 2D intraoperative X-ray can be registered to a 3D preoperative CT volume by maximizing its similarity to a rendered X-ray with respect to the camera pose.

“A 2D intraoperative X-ray can be registered to a 3D preoperative CT volume by maximizing its similarity to a rendered X-ray with respect to the camera pose [8, 14, 34].”

• cites [8] sourceNot checked — no full text of the source
• cites [14] sourceNot checked — no full text of the source
• cites [34] sourcePartly supported

  The Introduction (page unavailable) describes registration by optimizing similarity between corresponding voxels, and C. Search (page unavailable) describes maximizing the registration criterion over transformation parameters. But the paper's experiments concern 3D CT, MR, and PET scans; the cited passages do not support registering a 2D intraoperative X-ray to a CT volume using a rendered X-ray and camera pose. (LLM, conf=0.82)

  Experimental — accuracy not yet measured

May combine two claims.

Only a few small labeled datasets exist for training deep networks for landmark localization, preventing generalization to new patients, procedures, and pathologies.

“While landmark localizers can be learned, only a few small labeled datasets exist for training deep networks [24, 41, 50], preventing the generalization of these methods to new patients, procedures, and pathologies.”

• cites [24] sourceCould not verify

  The Intraoperative Registration Strategy section describes manual landmark annotation, and the Discussion mentions neural-network landmark recognition as a possible alternative (page numbers unavailable). Neither passage addresses the availability or size of labeled training datasets, or says that dataset scarcity prevents generalization to new patients, procedures, or pathologies. Searches for labeled datasets, deep-network training, and generalization found no supporting passage. (LLM, conf=0.76)

  Experimental — accuracy not yet measured
• cites [41] sourceNot checked — no full text of the source
• cites [50] sourceNot checked — no full text of the source

May combine two claims.

Conventional optimization methods for 2D/3D registration are prohibitively slow and susceptible to local minima.

“conventional optimization methods are prohibitively slow and susceptible to local minima”

• Also stated in: § Introduction · plain

  “A major limitation of intensity-based methods is their limited capture range and sensitivity to the initial pose estimate [52].”

• Also stated in: § Intensity-based 2D/3D registration. · plain

  “This is both prohibitively slow for intraoperative use and often converges poorly in practice [58].”

• cites [52] sourcePartly supported

  Section 1.2 (page unavailable) says conventional intensity-based registration can get stuck in local minima without proper initialization. Section 3 (page unavailable) says machine-learning approaches are comparatively fast because they replace optimization with a few forward passes. That supports a relative speed disadvantage, but not the stronger claim that conventional optimization is prohibitively slow. (LLM, conf=0.87)

  Experimental — accuracy not yet measured
• cites [58] sourceNot checked — no full text of the source

May combine two claims.

DiffPose develops a sparse multiscale variant of local normalized cross correlation that is robust to local minima and faster to evaluate.

“we develop a sparse multiscale variant of local normalized cross correlation [6, 7] that is robust to local minima and faster to evaluate.”

• cites [6] Brian, 2011, A reproducible evaluation of ANTs similarity metric performance in brain image registration, Neuroimage sourceSupported
• cites [7] sourceNot checked — no full text of the source

Probably not a substantial claim: may not pass S3 (load-bearing).

Given supervision via anatomical landmarks, a camera pose can be estimated by applying a PnP solver to corresponding landmarks.

“Given supervision via anatomical landmarks, a camera pose can be estimated by applying a PnP solver [19] to corresponding landmarks extracted from 2D intraoperative and 3D preoperative imaging.”

• cites [19] sourceSupported

Probably not a substantial claim: may not pass S3 (load-bearing).

Higher-dimensional parameterizations of SO(3) have been proposed that do not suffer from discontinuities.

“In point cloud registration, higher-dimensional parameterizations of SO(3) have been proposed that do not suffer from this issue [32, 40, 57].”

• cites [32] Lin, 2023, Al-gebraically rigorous quaternion framework for the neural network pose estimation problem sourceNot checked — no full text of the source
• cites [40] sourcePartly supported

  The paper’s ‘Smooth Global Section of SO(3)’ section proves that its ten-dimensional rotation representation has a smooth, continuous mapping, supporting the claim about avoiding discontinuities. It tests rotation estimation on point clouds and mentions point-cloud registration in its ‘Learning-based Rotation Estimation’ section, but does not establish that its representation was proposed specifically for registration. (LLM, conf=0.84)

  Experimental — accuracy not yet measured
• cites [57] sourceNot checked — no full text of the source

May combine two claims.

Gradient-based optimization with DiffDRR recovers the true pose faster and more robustly than gradient-free methods on synthetic data.

“On synthetic data, gradient-based optimization with DiffDRR, a differentiable renderer, has been shown to recover the true pose faster and more robustly than gradient-free methods [1, 22].”

• cites [1] sourcePartly supported

  The Experimental Design and Results sections describe using DiffDRR with gradient descent to recover the target pose from simulated radiographs. The Limitations and Future Directions section says this was not a systematic comparison with other methods, so the claimed speed and robustness advantage over gradient-free methods is unsupported. Page numbers were not supplied in our copy. (LLM, conf=0.88)

  Experimental — accuracy not yet measured
• cites [22] Gopalakrishnan, 2022, Fast auto-differentiable digitally reconstructed radiographs for solving inverse problems in intraoperative imaging sourcePartly supported

  In Experiments, the paper reports that gradient descent with DiffDRR recovered the true pose for 745 of 1,000 synthetic initializations, with convergent runs taking 1.92 ± 0.43 seconds; 255 runs failed to converge. The Abstract says gradient-based registration promises to be faster than gradient-free strategies, but the paper does not establish that pose recovery is faster or more robust than gradient-free methods in a direct comparison. Page numbers were not provided in our copy. (LLM, conf=0.86)

  Experimental — accuracy not yet measured

Classified as probably not a substantial claim.

Recent approaches train neural networks to segment intraoperative X-rays and localize predefined landmarks from segmentation masks.

“Recent approaches train neural networks to segment intraoperative X-rays and localize predefined landmarks from the segmentation masks [17, 25, 27, 37, 39].”

• cites [17] sourceNot checked — no full text of the source
• cites [25] sourceSupported
• cites [27] sourcePartly supported

  Section 2.1 (p. 2) describes a deep-learning registration framework that takes a fluoroscopic X-ray image as input and estimates correspondences. It does not describe segmenting the X-ray or localizing predefined landmarks from segmentation masks; the mask in Section 2.3 (p. 5) is used to calculate optical-flow loss at projected contour points. (LLM, conf=0.84)

  Experimental — accuracy not yet measured
• cites [37] sourceNot checked — no full text of the source
• cites [39] Van, 2022, Automatic landmark detection and mapping for 2d/3d registration with bonenet, Frontiers in Veterinary Sci-enceNot matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

May combine two claims.

The gradient-based approach (DiffDRR) is more robust than gradient-free methods, but not to a surgically viable degree.

“The gradient-based approach (DiffDRR [22]) is more robust than gradient-free methods, but not to a surgically viable degree.”

• cites [22] Gopalakrishnan, 2022, Fast auto-differentiable digitally reconstructed radiographs for solving inverse problems in intraoperative imaging sourcePartly supported

  In Experiments > DRR-based Gradient Descent for Slice-to-Volume Registration (page unavailable), DiffDRR's gradient-descent example converged for 745 of 1,000 synthetic initializations and failed for 255, supporting a limitation in its reliability. The paper says gradient-based registration promises to be faster than gradient-free strategies, but it does not establish that it is more robust than them or assess whether its reliability is surgically viable. (LLM, conf=0.83)

  Experimental — accuracy not yet measured

May combine two claims.

Local NCC performs competitively but has high variance, demonstrating the instability of the loss.

“Local NCC [6] performs competitively but has high variance, demonstrating the instability of the loss.”

• cites [6] Brian, 2011, A reproducible evaluation of ANTs similarity metric performance in brain image registration, Neuroimage sourcePartly supported

  Results §4.2 reports competitive brain-extraction performance for registrations using CC and a relatively large standard deviation for the (CC, CC) pairing (0.937 ± 0.0210). But the (CC, MI) pairing has a much smaller standard deviation (0.958 ± 0.0054), and the paper attributes improved performance to MI affine initialization. It does not conclude that the local NCC loss itself is unstable; page numbers are unavailable in our copy. (LLM, conf=0.88)

  Experimental — accuracy not yet measured

Classified as probably not a substantial claim.

A registration is successful if mTRE ≤ 1 mm, following AAPM guidelines.

“Following guidelines from the American Association of Physicists in Medicine (AAPM) [11], a registration is successful if mTRE ≤ 1 mm.”

• cites [11] sourcePartly supported

  Section 4.C.4 specifies a 1 mm tolerance for stereotactic radiosurgery, but not for registration generally. Table III (section 4.D) lists the target registration error tolerance as the maximum voxel dimension, approximately 2–3 mm. The claim omits the condition under which the 1 mm tolerance applies. (LLM, conf=0.88)

  Experimental — accuracy not yet measured

Probably not a substantial claim: may not pass S3 (load-bearing).

Higher-dimensional alternatives (d>5) have theoretical advantages.

“This has motivated the development of several higher-dimensional (d>5) alternatives with theoretical advantages [32, 40, 57].”

• cites [32] Lin, 2023, Al-gebraically rigorous quaternion framework for the neural network pose estimation problem sourceNot checked — no full text of the source
• cites [40] sourceSupported
• cites [57] sourceNot checked — no full text of the source

Classified as probably not a substantial claim.

Augmenting training data with synthetically generated X-rays helps overcome small clinical sample sizes when training landmark extractors and pose regressors.

“Augmenting training data with synthetically generated X-rays helps overcome the small clinical sample sizes when training landmark extractors [17, 28, 46] and pose regressors [35, 56].”

• cites [17] sourceNot checked — no full text of the source
• cites [28] sourcePartly supported

  Section 3.2 describes training a 2D/3D registration network with simulated X-ray projections, and the Introduction says that collecting large paired, annotated datasets is prohibitive. This supports the general point that synthetic images can ease a training-data shortage, but the paper studies a registration network rather than training a landmark extractor or a pose regressor as such. It also says domain adaptation is needed because simulated and real X-rays differ. (LLM, conf=0.84)

  Experimental — accuracy not yet measured
• cites [46] sourceSupported
• cites [35] sourceNot checked — no full text of the source
• cites [56] sourceNot checked — no full text of the source

May combine two claims.

Traditional gradient-free optimization (xReg) successfully recovers the poses of some X-rays but frequently converges to suboptimal minima.

“Traditional gradient-free optimization (xReg [24]) is able to successfully recover the poses of some X-rays but frequently converges to suboptimal minima.”

• cites [24] sourcePartly supported

  The Intraoperative Registration Strategy describes derivative-free optimization, and the Results report successful pose estimates alongside some large errors. But the paper does not establish that the method frequently converges to suboptimal minima; it describes CMA-ES as robust to local minima and BOBYQA as less robust, without attributing the observed errors to such convergence. Page numbers are unavailable in our copy. (LLM, conf=0.84)

  Experimental — accuracy not yet measured

May combine two claims.

The rendering equation can be implemented as a series of vectorized tensor operations, enabling rendering of synthetic X-rays that are differentiable with respect to T.

“The rendering equation (4) can be implemented as a series of vectorized tensor operations [22], enabling the rendering of synthetic X-rays that are differentiable with respect to T.”

• cites [22] Gopalakrishnan, 2022, Fast auto-differentiable digitally reconstructed radiographs for solving inverse problems in intraoperative imaging sourceSupported

• AssumptionThe authors state this as an assumption: Second-order effects such as scattering and beam hardening can be ignored in the X-ray image formation model.
• AssumptionThe authors state this as an assumption: The ray has initial intensity I0.
• AssumptionThe authors state this as an assumption: The preoperative CT volume can be approximated as a discrete 3D voxel grid of linear attenuation coefficients.

Probably not a substantial claim: may not pass S3 (load-bearing).

A PnP solver can be subject to poor initialization especially if less than four landmarks are visible in the 2D image.

“These landmarks are used with a PnP solver to predict the camera pose, which can be subject to poor initialization especially if less than four landmarks are visible in the 2D image [25].”

• cites [25] sourceSupported

May combine two claims.

DiffPose pretrains a patient-specific CNN with a self-supervised pose regression task, removing the need for manually annotated training data.

“First, we pretrain a patient-specific CNN with our self-supervised pose regression task, removing the need for manually annotated training data.”

May combine two claims.

DiffPose does not use manual or estimated landmarks, operating directly on patient-specific image intensities.

“In contrast, our method does not use manual or estimated landmarks, operating directly on patient-specific image intensities.”

May combine two claims.

Multiscale NCC is both more accurate and more numerically stable than local NCC.

“Instead, we find averaging NCC over multiple patch sizes, a generalization known as multiscale NCC (ℒmNCC), to be both more accurate and more numerically stable.”

May combine two claims.

Multiscale NCC consistently achieves sub-millimeter registration accuracy while successfully converging more reliably than local NCC.

“This image similarity metric consistently achieves sub-millimeter registration accuracy while successfully converging more reliably than local NCC.”

May combine two claims.

On Subjects 3, 4, and 6, DiffPose consistently achieves sub-millimeter accuracy while PnP-Regularizer's median mTRE is around 10 mm.

“on Subjects 3, 4, and 6, DiffPose consistently achieves sub-millimeter accuracy while PnP-Regularizer’s median mTRE is around 10 mm.”

May combine two claims.

PnP methods provide better initial pose estimates on average, but their accuracy diminishes when only a few landmarks are present in the 2D image.

“PnP methods provide better initial pose estimates on average, but their accuracy diminishes when only a few landmarks are present in the 2D image.”

May combine two claims.

Local NCC is more unstable and frequently guides the test-time optimizer to a poor quality solution.

“However, the metric is also more unstable and frequently guides the test-time optimizer to a poor quality solution.”

May combine two claims.

Using mNCC, which averages global and local NCC, stabilizes the variance and results in the highest success rate.

“Using mNCC, which averages global and local NCC, stabilizes the variance and results in the highest success rate.”

Probably not a substantial claim: may not pass S4 (based).

Previous methods fail to generalize to new patients or procedures.

“While previous methods fail to generalize to new patients or procedures, DiffPose consistently achieves sub-millimeter registration accuracy across populations and anatomical structures.”

Classified as probably not a substantial claim.

Classical 2D/3D rigid registration methods apply ray tracing to CT volumes to render synthetic X-rays (DRRs).

“Classical 2D/3D rigid registration methods apply ray tracing to CT volumes to render synthetic X-rays, referred to as digitally reconstructed radiographs (DRRs) in medical imaging [4, 53].”

• cites [4] sourceNot checked — no full text of the source
• cites [53] sourceNot checked — no full text of the source

Classified as probably not a substantial claim.

Generating synthetic data for PnP registration pretraining still requires manual annotations of landmarks on each new CT.

“While generating synthetic data for PnP registration pretraining still requires manual annotations of landmarks on each new CT, rendered X-rays come with ground truth camera poses for free.”

May combine two claims.

These landmark-based approaches require expert knowledge of which landmarks to extract and manual annotation of a large 2D/3D dataset, precluding application to new anatomical structures.

“This approach requires expert knowledge of which landmarks to extract and manual annotation of a large 2D/3D dataset for the anatomy of interest, precluding application to new anatomical structures.”

May combine two claims.

The registration problem is more challenging because the subject can move between the acquisition of the preoperative and intraoperative images, so the 2D X-rays are no longer directly embedded in the 3D CT scan.

“This makes our registration problem more challenging because the subject can move between the acquisition of the preoperative and intraoperative images, meaning the 2D X-rays are no longer directly embedded in the 3D CT scan.”

• AssumptionRests on an assumption: The CTs in the DeepFluoro and Ljubljana datasets are not reconstructed from the acquired X-rays.
• AssumptionRests on an assumption: Subjects can move between the acquisition of the preoperative and intraoperative images.

May combine two claims.

Testing on datasets with independent and physically-acquired 2D and 3D modalities avoids test set leakage during evaluation and directly simulates the clinical use case.

“Testing on datasets with independent and physically-acquired 2D and 3D modalities avoids test set leakage during evaluation and directly simulates the clinical use case.”

• AssumptionRests on an assumption: The CTs in the DeepFluoro and Ljubljana datasets are not reconstructed from the acquired X-rays.

Probably not a substantial claim: may not pass S4 (based).

These landmark-based approaches often fail on patients with poses, anatomies, or pathologies substantially different from the training set.

“These approaches often fail on patients with poses, anatomies, or pathologies that are substantially different from those in the training set.”

Probably not a substantial claim: may not pass S4 (based).

Evaluating NCC across multiple scales is computationally expensive.

“Evaluating NCC across multiple scales is computationally expensive.”

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

Sample sizes are generally low in real 2D/3D surgical datasets due to difficulty of acquiring expert annotations with calibrated poses, limiting utility of supervised methods.

“Importantly, we note that sample sizes are generally low in real 2D/3D surgical datasets due to the difficulty of acquiring expert annotations with calibrated poses in surgical settings, limiting the utility of supervised methods.”

• AssumptionRests on an assumption: Sample sizes are generally low in real 2D/3D surgical datasets due to difficulty of acquiring expert annotations with calibrated poses in surgical settings.

Classified as probably not a substantial claim.

All models were trained with the same number of synthetic images.

“All models were trained with the same number of synthetic images.”

May combine two claims.

A purely supervised pose regressor (PoseNet) incurs large error because subjects are not registered to a common reference frame and there is a limited number of X-rays.

“As subjects are not registered to a common reference frame and there is a limited number of X-rays, a purely supervised pose regressor (PoseNet [29]) incurs large error.”

• cites [29] sourceNot checked — no full text of the source

• AssumptionRests on an assumption: Subjects are not registered to a common reference frame and there is a limited number of X-rays.

May combine two claims.

mTRE is an independent metric of registration accuracy because landmark supervision is never used for training or test-time optimization.

“Since we never use landmark supervision for training or test-time optimization, mTRE is an independent metric of registration accuracy.”

• AssumptionRests on an assumption: The intrinsic parameters of the imaging system are known.

Probably not a substantial claim: may not pass S3 (load-bearing).

Pose estimation can be formulated as a supervised regression problem.

“Pose estimation can also be formulated as a supervised regression problem [13, 35].”

• cites [13] sourceNot checked — no full text of the source
• cites [35] sourceNot checked — no full text of the source

May combine two claims.

Typically, a finite number of synthetic X-rays is generated for pretraining, failing to address the central limitation of CNN-based pose regression.

“Typically, a finite number of synthetic X-rays is generated for pretraining [35, 56], failing to address the central limitation of CNN-based pose regression”

• cites [35] sourceNot checked — no full text of the source
• cites [56] sourceNot checked — no full text of the source

May combine two claims.

Inter-subject variability arises from out-of-distribution failures when the pose or appearance of a particular X-ray does not match the training set, demonstrating the need for patient-specific methods.

“Inter-subject variability arises from out-of-distribution failures when the pose or appearance of a particular X-ray does not match the training set, demonstrating the need for patient-specific methods.”

Probably not a substantial claim: may not pass S3 (load-bearing).

DiffPose uses per-subject training, which may be too slow for emergency surgeries that do not allow for hours-long pretraining.

“Further, DiffPose uses per-subject training (analogous to NeRF [36]), which may be too slow for emergency surgeries that do not allow for hours-long pretraining.”

May combine two claims.

DiffPose estimates a single rigid transformation between pre- and intraoperative scans, precluding direct application to deformable target structures.

“DiffPose estimates a single rigid transformation between pre- and intraoperative scans, precluding direct application to deformable target structures.”