3D–2D image registration for target localization in spine surgery: investigation of similarity metrics providing robustness to content mismatch

May combine two claims.

Gradient orientation on its own has been widely used for feature matching, object detection, and image registration, and shown promise in robust matching under partial occlusion and view point changes.

“While all of the above metrics rely upon gradient magnitudes, gradient orientation on its own has been widely used for feature matching (Lowe 2004), object detection (Dalal and Triggs 2005), and image registration (De Nigris et al 2012), and shown promise in robust matching under partial occlusion and view point changes.”

• cites Lowe 2004 sourceNot checked — no full text of the source
• cites Dalal and Triggs 2005 sourcePartly supported

  The Abstract describes histograms of oriented gradients used for human detection, supporting the object-detection example attributed to Dalal and Triggs. It does not establish that gradient orientation was used on its own, independent of other gradient information. (LLM, conf=0.82)

  Experimental — accuracy not yet measured
• cites De Nigris et al 2012 sourceNot checked — no full text of the source

Classified as probably not a substantial claim.

The average thoracolumbar vertebral height in the superior-inferior direction is 22 mm.

“Considering an average thoracolumbar vertebral height (superior–inferior direction) of 22 mm (Busscher et al 2010)”

• cites Busscher et al 2010 sourceSupported

Probably not a substantial claim: may not pass S3 (load-bearing).

For the acquired radiographs, the magnification factor at the detector typically ranged from 1.4 to 1.6.

“For the acquired radiographs, the magnification factor at the detector typically ranged from 1.4–1.6.”

May combine two claims.

The GI, GC, and GS metrics demonstrated broad capture ranges: GI 192 mm ± 214 mm IQR, GC 345 mm ± 185 mm IQR, and GS 293 mm ± 180 mm IQR.

“The other metrics demonstrated a fairly broad capture range as well: GI capture range = 192 mm ± 214 mm IQR; GC capture range = 345 mm ± 185 mm IQR; and GS capture range = 293 mm ± 180 mm IQR.”

Probably not a substantial claim: may not pass S3 (load-bearing).

The PDE following initialization ranged from 11.3 to 111.1 mm (median PDE = 43.1 mm, interquartile range 20.9 mm).

“The PDE following initialization ranged from 11.3 to 111.1 mm (median PDE = 43.1 mm, interquartile range 20.9 mm).”

• AssumptionThe authors state this as an assumption: The transformation between preoperative CT and intraoperative radiograph can be modeled as a rigid 6D transformation.

May combine two claims.

In the R(r, 50) configuration, the GO metric maintained geometric accuracy and robustness.

“The GO metric, on the other hand, maintained geometric accuracy and robustness.”

May combine two claims.

The GC metric shows a more distinct optimum at the solution but also exhibits sharp local maxima attributed to strong gradient magnitudes far from the solution from surgical instruments.

“On the other hand, GC shows a more distinct optimum at the solution, but it also exhibits sharp local maxima attributed to strong gradient magnitudes far from the solution (surgical instruments).”

May combine two claims.

In the figure 3 simulation, while all metrics align corresponding square and circle elements at the desired solution, the mismatching triangle at the degenerate solution causes the GI and GC metrics to yield a larger relative contribution to the overall metric value.

“While all metrics align corresponding square and circle elements at the desired solution, the mismatching triangle at the degenerate solution causes the GI, GC metrics to yield a larger relative contribution to the overall metric value.”

May combine two claims.

GO provided robust registration (median PDE = 5.5 mm, 2.6 mm IQR) without manual masking and with an improved runtime (29.3 s).

“but GO provided robust registration (median PDE = 5.5 mm, 2.6 mm IQR) without manual masking and with an improved runtime (29.3 s).”

• AssumptionThe authors state this as an assumption: The transformation between preoperative CT and intraoperative radiograph can be modeled as a rigid 6D transformation.

May combine two claims.

In the R(r, 50) configuration, the performance of the GI, GC, and GS metrics degraded significantly, with the gross failure rate for GC increasing to 15%.

“The performance of the GI, GC, and GS metrics degraded significantly, e.g. the gross failure rate for GC increasing to 15%.”

May combine two claims.

For GI and GS, the peak at the solution is less distinct from surrounding local maxima, although GS appears improved over GI.

“For GI and GS, the peak at the solution is less distinct from surrounding local maxima, although GS appears improved over GI (consistent with improved robustness noted above).”

May combine two claims.

The average computation time ranged from 23-29 s for the S = 50 configurations, compared to 65-85 s for the S = 200 configuration.

“the average computation time ranged from 23–29 s for the S = 50 configurations, compared to 65–85 s for the S = 200 configuration.”

May combine two claims.

Among the RGO(p, 200), RGO(p, 50), and RGO(r, 50) distributions, the Wilcoxon signed rank test found p > 0.94, failing to reject the null hypothesis that the PDE distributions have the same median.

“Among distributions RGO(p, 200), RGO(p, 50), and RGO(r, 50), we found p-value >0.94 failing to reject the null hypothesis that the PDE distributions have the same median.”

May combine two claims.

For the GO metric, gradient magnitude values do not dominate the overall metric value, and the desired solution is reached with corresponding vertebrae gradients aligned robustly against extraneous gradients and occlusions from instrumentation.

“For the GO metric, the gradient magnitude values do not dominate the overall metric value, and the desired solution is reached wherein corresponding vertebrae gradients align in a manner that is robust against extraneous gradients and occlusions from instrumentation.”

May combine two claims.

The GO metric exhibits the most distinct global optimum at the solution and suppresses false maxima far from the solution.

“The GO metric exhibits the most distinct global optimum at the solution and suppresses false maxima far from the solution.”

May combine two claims.

With manual polygonal masking and 200 multistarts, the GC and GO metrics exhibited robust performance with 0% gross failures and median PDE < 6.4 mm (±4.4 mm interquartile range) and a median runtime of 84 s (plus upwards of 1-2 min for manual masking).

“With manual polygonal masking and 200 multistarts, the GC and GO metrics exhibited robust performance with 0% gross failures and median PDE < 6.4 mm (±4.4 mm interquartile range (IQR)) and a median runtime of 84 s (plus upwards of 1–2 min for manual masking).”

• AssumptionThe authors state this as an assumption: The transformation between preoperative CT and intraoperative radiograph can be modeled as a rigid 6D transformation.

May combine two claims.

The RGO(r, 50) configuration outperformed all of the GI, GS, and GC configurations in terms of geometric accuracy, robustness, and run time.

“Overall, the RGO(r, 50) configuration outperformed all of the GI, GS, and GC configurations in terms of geometric accuracy, robustness, and run time”

May combine two claims.

Reducing the number of multistarts to 50 in the R(p, 50) configuration compromises registration robustness and increases the failure rate for all similarity metrics.

“but compromises the robustness of the registration and results in an increasing failure rate for all of the similarity metrics.”

May combine two claims.

The mm-scale variations in PDE between multistart configurations were not statistically significant (p = 0.52, 0.72, 0.10, and 0.07 for GI, GC, GS, and GO, respectively) and are likely not clinically significant with respect to localizing a particular vertebral body.

“but such mm-scale variations in PDE in table 2 were not statistically significant (p = 0.52, 0.72, 0.10, and 0.07 for GI, GC, GS, and GO, respectively) and are likely not clinically significant with respect to localizing a particular vertebral body.”

May combine two claims.

Although the GS metric somewhat improved registration performance compared to GI, it did not exhibit particularly strong robustness to failure (>16%) over all tested configurations.

“Although the GS metric somewhat improved the registration performance compared to GI, it did not exhibit particularly strong robustness to failure (>16%) over all the tested configurations.”

May combine two claims.

In spine surgery, sources of mismatch such as surgical instrumentation in the intraoperative radiograph and non-rigid anatomical deformation or positioning differences are common occurrences and present particular challenges to 3D-2D registration.

“In spine surgery, for example, the following sources of mismatch are common occurrences (as shown in figure 1) and present particular challenges to 3D–2D registration: (a) The presence of surgical instrumentation in the intraoperative radiograph (e.g. retractors, screws, implants, clips, etc); and (b) non-rigid anatomical deformation”

May combine two claims.

Automatic or semi-automatic segmentation as a pre-processing step to mask extraneous objects increases computational burden and makes the registration algorithm susceptible to segmentation errors.

“However, such an approach increases the computational burden and makes the registration algorithm susceptible to segmentation errors.”

May combine two claims.

GC can accommodate differences in gradient images up to a linear factor and therefore offers some degree of robustness against mismatch.

“GC can accommodate such differences up to a linear factor and therefore offers some degree of robustness against mismatch:”

May combine two claims.

The broad capture ranges are partly attributable to the similarity metric and perhaps more so to the robust optimization strategy involving multiple parallel searches from multiple initial seed points.

“Such broad capture ranges are partly attributable to the similarity metric and perhaps more so to the robust optimization strategy involving multiple parallel searches from multiple initial seed points.”

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

Increasing the number of multistarts boosts the density of seed points positioned within the multidimensional space and therefore improves the robustness of finding the global optima at the cost of additional computation.

“Increasing the number of multistarts (denoted S in figure 2) boosts the density of seed points positioned within the multidimensional space and therefore improves the robustness of finding the global optima at the cost of additional computation in performing parallel searches (roughly proportional to the number of multi-starts).”

Classified as probably not a substantial claim.

The min operator in GI assumes that anatomical structures produce a constant gradient magnitude in both DRRs and radiographs.

“The min operator in GI assumes that anatomical structures produce a constant gradient magnitude in both DRRs and radiographs.”

May combine two claims.

Image gradients can filter out low spatial frequency differences such as soft tissue and act as a high-pass filter to focus registration on bony anatomy visible in x-ray images.

“Image gradients can be used to filter out low spatial frequency differences such as soft tissue (Penney et al 1998, Pluim et al 2000) and act as a high-pass filter to focus the registration on the bony anatomy clearly visible in x-ray images.”

• cites Penney et al 1998 sourceNot checked — no full text of the source
• cites Pluim et al 2000 sourceNot checked — no full text of the source

May combine two claims.

The GO similarity metric exhibits robust performance against content mismatch and enables the definition of a simple rectangular mask of the radiograph collimators to reduce or eliminate the need for polygonal masking.

“The GO similarity metric thereby exhibits robust performance against content mismatch and enables the definition of a simple rectangular mask of the radiograph collimators to reduce or eliminate the need for polygonal masking.”

May combine two claims.

The GO metric improved the registration accuracy and robustness in the presence of strong image content mismatch.

“The GO metric improved the registration accuracy and robustness in the presence of strong image content mismatch.”

• AssumptionThe authors state this as an assumption: The transformation between preoperative CT and intraoperative radiograph can be modeled as a rigid 6D transformation.

May combine two claims.

The definition of manual polygonal masks is subject to user variability and tends to exclude regions that may be disadvantageous to registration.

“the definition of manual polygonal masks is subject to user variability and tends to exclude regions that may be disadvantageous to registration.”

Probably not a substantial claim: may not pass S4 (based). May combine two claims.

Manual or semi-automatic methods to mask out extraneous or missing content are time consuming, user dependent, error prone, and disruptive to workflow.

“Manual/semi-automatic methods to mask out such extraneous (or missing) content are time consuming, user dependent, error prone, and disruptive to workflow.”

Probably not a substantial claim: may not pass S2 (checkable).

The GO metric capability could offer valuable assistance and decision support in spine level localization in a manner consistent with clinical workflow.

“This capability could offer valuable assistance and decision support in spine level localization in a manner consistent with clinical workflow.”

Probably not a substantial claim: may not pass S4 (based). May combine two claims.

The definition of polygons to mask extraneous objects is a slow, subjective step that is error prone and limits the practical utility of image-based registration within streamlined workflow.

“The definition of polygons to mask extraneous objects is a slow, subjective step that is error prone and limits the practical utility of image-based registration within streamlined workflow.”

Probably not a substantial claim: may not pass S4 (based).

Content mismatch caused by extraneous objects and tissue deformation can create a challenging search space with multiple local optima.

“Content mismatch caused by extraneous objects and tissue deformation can create a challenging search space with multiple local optima.”

May combine two claims.

For the GS metric, the effect of instrumentation gradients is mitigated partly by the scaling parameter α, but excluding extraneous gradients requires estimating α to accurately match corresponding gradients in the two images.

“For the GS metric, this effect is mitigated partly by the scaling parameter α; however, the exclusion of extraneous gradients requires estimating α to accurately match the corresponding gradients in the two images.”

May combine two claims.

The challenging search space shows multiple local maxima resulting from extraneous objects and stands to benefit from the multistart optimization strategy to improve robustness.

“The challenging search space illustrated in figure 6 shows multiple local maxima resulting from extraneous objects and stands to benefit from the multistart optimization strategy to improve robustness.”

May combine two claims.

Deformation of the spine is not specifically addressed in the current rigid registration approach, but the large clinical dataset presented realistic levels of anatomical deformation.

“Deformation of the spine is not specifically addressed in the current (rigid) registration approach, but the large clinical dataset did present realistic levels of anatomical deformation.”

May combine two claims.

The corresponding increase in robustness from more multistarts has diminishing return due to the curse of dimensionality in the 6D search space, whereas computation time increases approximately linearly with the number of multistarts.

“Moreover, the corresponding increase in robustness has diminishing return due to the ‘curse of dimensionality’ in the 6D search space, whereas the computation time increases approximately linearly with the number of multistarts.”

May combine two claims.

The GS parameter was evaluated by computing the ratio of maximum gradient magnitudes observed between the two images, but the parameter should be adjusted according to mismatch caused by strong gradient magnitudes, and automatic determination for any given pair of images is non-trivial.

“We evaluated this parameter by computing the ratio of maximum gradient magnitudes observed between the two images; however, the parameter should be adjusted according to mismatch caused by strong gradient magnitudes. Automatic determination of the parameter for any given pair of images is non-trivial.”

Classified as probably not a substantial claim.

The registration framework should rely less upon the number of multistarts, an attribute exhibited with the incorporation of the GO similarity metric.

“Therefore, it is desirable for the registration framework to rely less upon the number of multistarts, an attribute exhibited with the incorporation of the GO similarity metric.”

Probably not a substantial claim: may not pass S2 (checkable).

These improvements better enable clinical translation of the registration algorithm.

“These improvements better enable clinical translation of the registration algorithm”

Probably not a substantial claim: may not pass S3 (load-bearing).

The polygonal masking approach requires the user to anticipate and identify problematic gradients in a radiograph with many subtleties.

“Another drawback of the polygonal masking approach is that it requires the user to anticipate and identify problematic gradients in a radiograph with many subtleties.”

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

The small variations in PDE could be attributed to errors in localization of the vertebrae centroid (expert reader truth definition), the stochasticity of the CMA-ES optimizer, and/or fundamental limitations arising from the use of similarity metrics as surrogate measures of registration accuracy during the optimization process.

“The small variations in PDE could be attributed to errors in the localization of the vertebrae centroid (expert reader truth definition), the stochasticity of the CMA-ES optimizer, and/or fundamental limitations arising from the use of similarity metrics as surrogate measures of registration accuracy during the optimization process.”