CoChat

Detection and Localization of Robotic Tools in Robot-Assisted Surgery Videos Using Deep Neural Networks for Region Proposal and Detection

Open access
Sarikaya, Duygu, Corso, Jason J., Guru, Khurshid A. · 2017 · IEEE Transactions on Medical Imaging

The DOI matches its Crossref record; 32 of the 32 references it cites were found in a registry or an index.

Abstract

Video understanding of robot-assisted surgery (RAS) videos is an active research area. Modeling the gestures and skill level of surgeons presents an interesting problem. The insights drawn may be applied in effective skill acquisition, objective skill assessment, real-time feedback, and human-robot collaborative surgeries. We propose a solution to the tool detection and localization open problem in RAS video understanding, using a strictly computer vision approach and the recent advances of deep learning. We propose an architecture using multimodal convolutional neural networks for fast detection and localization of tools in RAS videos. To the best of our knowledge, this approach will be the first to incorporate deep neural networks for tool detection and localization in RAS videos. Our architecture applies a region proposal network (RPN) and a multimodal two stream convolutional network for object detection to jointly predict objectness and localization on a fusion of image and temporal motion cues. Our results with an average precision of 91% and a mean computation time of 0.1 s per test frame detection indicate that our study is superior to conventionally used methods for medical imaging while also emphasizing the benefits of using RPN for precision and efficiency. We also introduce a new data set, ATLAS Dione, for RAS video understanding. Our data set provides video data of ten surgeons from Roswell Park Cancer Institute, Buffalo, NY, USA, performing six different surgical tasks on the daVinci Surgical System (dVSS) with annotations of robotic tools per frame.

Overview

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1109/tmi.2017.2665671

Cites

32 of 32 references verified0 citation errors · 0 not checkable

Statistics

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Show all 32 references

Research works printed but not linked: 7

  1. reiter, 2012, Marker-less articulated surgical tool detection, Comput Assist Radiol Surgery
  2. ren, 2015, Faster R-CNN: Towards realtime object detection with region proposal networks, Proc Neural Inf Process Syst (NIPS), 1
  3. krizhevsky, 2012, ImageNet classification with deep convolutional neural networks, Proc Adv Neural Inf Process Syst (NIPS), 1
  4. kumar, 2015, Surgical performance assessment, ASME Dyn Syst Control Mag, 3, 7
  5. meadows, 2002, Robots lend a helping hand to surgeons, U S Food Drug Admin (FDA) Consum Mag, 36, 10
  6. sznitman, 2014, Fast part-based classification for instrument detection in minimally invasive surgery, Proc 17th Int Conf Med Image Comput Comput Assist Interventions (MICCAI), 692
  7. gao, 2014, JHU-ISI gesture and skill assessment working set (JIGSAWS): A surgical activity dataset for human motion modeling, Proceedings of the 5th Workshop on Modeling and Monitoring of Computer Assisted Interventions (M2CAI)

Cited by

See all 236

Metadata

Authors
Sarikaya, D., Corso, J. J., & Guru, K. A.
Affiliations
  • University at Buffalo, State University of New York (US)
  • University of Michigan (US)
  • Roswell Park Comprehensive Cancer Center (US)
Venue
IEEE Transactions on Medical Imaging
Published
2017-07
Volume
36
Issue
7
Pages
1542-1549
ISSN
  • 0278-0062 (ISSN-L)
  • 1558-254X
OpenAlex
W2587007321
Cited by
236
References
32
Citations by year
2026: 15 · 2025: 26 · 2024: 22 · 2023: 33 · 2022: 35 · 2021: 33 · 2020: 31 · 2019: 27 · 2018: 11 · 2017: 3
FWCI
20.16
Type
journal article
Language
en
Open access
Locations
  • IEEE Transactions on Medical Imaging (publishedVersion)
  • PubMed (publishedVersion)
  • arXiv (Cornell University) (submittedVersion)
  • arXiv (Cornell University)
Funders
Roswell Park Alliance Foundation Roswell Park Cancer Institute
Topics
  • Health Sciences › Medicine › Surgery › Surgical Simulation and Training
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Medical Image Segmentation Techniques
Keywords
robotic surgery, surgical tool detection, region proposal network, two-stream convolutional neural network, multimodal fusion, surgical skill assessment, da Vinci surgical system
MeSH terms
Humans, Motion, Robotics, Neural Networks, Computer, Robotic Surgical Procedures

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: Detection and Localization of Robotic Tools in Robot-Assisted Surgery Videos Using Deep Neural Networks for Region Proposal and Detection. https://app.cochat.ai/papers/10.1109/tmi.2017.2665671
The paper
Sarikaya, D., Corso, J. J., & Guru, K. A. (2017). Detection and Localization of Robotic Tools in Robot-Assisted Surgery Videos Using Deep Neural Networks for Region Proposal and Detection. IEEE Transactions on Medical Imaging. https://doi.org/10.1109/tmi.2017.2665671

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.