Localizing dexterous surgical tools in X-ray for image-based navigation
Open accessAbstract
X-ray image based surgical tool navigation is fast and supplies accurate images of deep seated structures. Typically, recovering the 6 DOF rigid pose and deformation of tools with respect to the X-ray camera can be accurately achieved through intensity-based 2D/3D registration of 3D images or models to 2D X-rays. However, the capture range of image-based 2D/3D registration is inconveniently small suggesting that automatic and robust initialization strategies are of critical importance. This manuscript describes a first step towards leveraging semantic information of the imaged object to initialize 2D/3D registration within the capture range of image-based registration by performing concurrent segmentation and localization of dexterous surgical tools in X-ray images. We presented a learning-based strategy to simultaneously localize and segment dexterous surgical tools in X-ray images and demonstrate promising performance on synthetic and ex vivo data. We currently investigate methods to use semantic information extracted by the proposed network to reliably and robustly initialize image-based 2D/3D registration. While image-based 2D/3D registration has been an obvious focus of the CAI community, robust initialization thereof (albeit critical) has largely been neglected. This manuscript discusses learning-based retrieval of semantic information on imaged-objects as a stepping stone for such initialization and may therefore be of interest to the IPCAI community. Since results are still preliminary and only focus on localization, we target the Long Abstract category.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Applications of CNNs to fluoroscopy have been mostly limited to recognizing surgical instruments and tools.
“However, since existing large-scale hip datasets have focused on 3D image modalities and pre and postoperative radiography, rather than intraoperative fluoroscopy [12], applications of CNNs to fluoroscopy have been mostly limited to recognizing surgical instruments and tools [13-16].”
From Automatic Annotation of Hip Anatomy in Fluoroscopy for Robust and Efficient 2D/3D Registration (2020) · cites this paper as [16]
The Methods and Conclusions describe a ConvNet that localizes and segments a surgical tool in X-ray images, supporting this as an example of tool-focused work. The paper does not establish that CNN applications to fluoroscopy have been *mostly limited* to instruments and tools. (LLM, conf=0.82)
Experimental — accuracy not yet measuredUnlike [16,19,20], the authors train networks on smaller datasets of actual fluoroscopy and achieve state-of-the-art results on clinically relevant data.
“Contrary to [16,19,20], we train our networks using smaller datasets of actual fluoroscopy and achieve state-of-the-art results on clinically relevant data.”
From Automatic Annotation of Hip Anatomy in Fluoroscopy for Robust and Efficient 2D/3D Registration (2020) · cites this paper as [16]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
DeepDRR – A Catalyst for Machine Learning in Fluoroscopy-Guided Procedures
Unberath, M., Zaech, J.-N., Lee, S. C., Bier, B., Fotouhi, J., Armand, M., & Navab, N. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-00937-3_12 · Cited by 145 · cited 5× · see citing works
“Due to the unavailability of annotated X-ray images to train ConvNets, we rely on DeepDRR [10], a framework for physics-based rendering of digitally reconstructed radiographs (DRRs, i. e. synthetic fluoroscopic images) from 3D CT. DeepDRR accurately accounts for energy- and material-dependence of X-ray attenuation, sc…”
BackgroundMethodInfluentialSelf-citation
Flagged first, then by citations
Research works printed but not linked: 10
- [1]1. Alambeigi, F., et al.: A curved-drilling approach in core decompression of the femoral head osteonecrosis using a continuum manipulator. IEEE Robot Autom Lett 2 (3), 1480-1487 (2017)
- [2]2. Alambeigi, F., et al.: Inroads toward robot-assisted internal fixation of bone fractures using a bendable medical screw and the curved drilling technique. In: IEEE Conf on Biorob. pp. 595-600. IEEE (2018)
- [3]3. Bier, B., Unberath, M., et al.: X-ray-transform invariant anatomical landmark detection for pelvic trauma surgery. In: Proc MICCAI (2018)
- [4]4. Burgner-Kahrs, J., et al.: Continuum robots for medical applications: A survey. IEEE Trans Robot 31 (6), 1261-1280 (2015)
- [5]5. Kutzer, M.D., et al.: Design of a new cable-driven manipulator with a large open lumen: Preliminary applications in the minimally-invasive removal of osteolysis. In: Proc ICRA. pp. 2913-2920. IEEE (2011)
- [6]6. Laina, I., Rieke, N., et al.: Concurrent segmentation and localization for tracking of surgical instruments. In: Proc MICCAI. pp. 664-672 (2017)
- [7]7. Markelj, P., et al.: A review of 3d/2d registration methods for image-guided interventions. MediIA 16 (3), 642-661 (2012)
- [8]8. Murphy, R.J., et al.: Design and kinematic characterization of a surgical manipulator with a focus on treating osteolysis. Robotica 32 (6), 835-850 (2014)
- [9]9. Otake, Y., et al.: Piecewise-rigid 2d-3d registration for pose estimation of snake-like manipulator using an intraoperative x-ray projection. In: Proc SPIE Med Imag. vol. 9036, p. 90360Q (2014)
- [11]11. Walker, I.D., et al.: Snake-like and continuum robots. In: Springer Handbook of Robotics, pp. 481-498 (2016)
Cited by
Sorted by citations
Unberath, M., Gao, C., Hu, Y., Judish, M., Taylor, R. H., Armand, M., & Grupp, R. (2021) · Frontiers in Robotics and AI · doi:10.3389/frobt.2021.716007 · Cited by 102 · see citing works
“This is because image-based navigation techniques [86, 13, 87] as well as visual servoing of surgical robots [88, 89, 90] will also require high-precision 2D/3D registration at video frame-rates.”
BackgroundSelf-citationAutomatic Annotation of Hip Anatomy in Fluoroscopy for Robust and Efficient 2D/3D Registration
Grupp, R. B., Unberath, M., Gao, C., Hegeman, R. A., Murphy, R. J., Alexander, C. P., Otake, Y., McArthur, B. A., Armand, M., & Taylor, R. H. (2020) · International Journal of Computer Assisted Radiology and Surgery · doi:10.1007/s11548-020-02162-7 · Cited by 81 · see citing works
“Gao also leveraged this approach for the localization of a dexterous continuum manipulator in fluoroscopy [16].”
MethodInfluentialSelf-citation
- Authors
- Gao, C., Unberath, M., Taylor, R., & Armand, M.
- Affiliations
- Johns Hopkins University (US)
- Venue
- arXiv
- Publisher
- arXiv
- Published
- 2019
- ISSN
- 2331-8422 (ISSN-L)
- arXiv
- 1901.06672
- OpenAlex
- W2911585457
- Cited by
- 15
- References
- 1
- Citations by year
- 2023: 2 · 2022: 1 · 2021: 4 · 2020: 6 · 2019: 2
- Type
- posted content
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Funders
- Johns Hopkins University
- National Institutes of Health
- National Institute of Biomedical Imaging and Bioengineering (R01EB023939)
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Medical Image Segmentation Techniques
- Health Sciences › Medicine › Surgery › Surgical Simulation and Training
- Physical Sciences › Engineering › Biomedical Engineering › Medical Imaging and Analysis
- Keywords
- radiography, 2D-3D registration, semantic segmentation, learning-based segmentation, tool deformation
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
