The DOI matches its Crossref record; 23 of the 23 references it cites were found in a registry or an index.
ToolNet: Holistically-nested real-time segmentation of robotic surgical tools
Open accessAbstract
Real-time tool segmentation from endoscopic videos is an essential part of many computer-assisted robotic surgical systems and of critical importance in robotic surgical data science. We propose two novel deep learning architectures for automatic segmentation of non-rigid surgical instruments. Both methods take advantage of automated deep-learning-based multi-scale feature extraction while trying to maintain an accurate segmentation quality at all resolutions. The two proposed methods encode the multi-scale constraint inside the network architecture. The first proposed architecture enforces it by cascaded aggregation of predictions and the second proposed network does it by means of a holistically-nested architecture where the loss at each scale is taken into account for the optimization process. As the proposed methods are for real-time semantic labeling, both present a reduced number of parameters. We propose the use of parametric rectified linear units for semantic labeling in these small architectures to increase the regularization of the network while maintaining the segmentation accuracy. We compare the proposed architectures against state-of-the-art fully convolutional networks. We validate our methods using existing benchmark datasets, including ex vivo cases with phantom tissue and different robotic surgical instruments present in the scene. Our results show a statistically significant improved Dice Similarity Coefficient over previous instrument segmentation methods. We analyze our design choices and discuss the key drivers for improving accuracy.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source says what the claim attributes to it.
In the citing paper
“Early studies have begun pursuing these objectives, primarily in video-based robotic and laparoscopic surgery-a number of works propose methods for detecting surgical tools and actions [118][119][120][121][122][123][124] .”
From Deep learning-enabled medical computer vision (2021) · cites this paper as [118]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Very Deep Convolutional Networks for Large-Scale Image Recognition
Simonyan, K., & Zisserman, A. (2014) · arXiv · doi:10.48550/arxiv.1409.1556 · Cited by 114,148 · cited 2×
“As the network and training data get smaller compared to the FCN-8s (which can enjoy transfer learning from very deep classification networks such as VGG16), we require a more robust regularization strategy that maintains the generalization ability of the network and protects the architecture from overfitting the trai…”
ResultImageNet classification with deep convolutional neural networks
Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017) · Communications of the ACM · doi:10.1145/3065386 · Cited by 112,155 · cited 1×
“In contrast to previous convolutional neural networks (CNNs) such as AlexNet [17] or VGG16 [18], FCNs do not have fully connected layers.”
ResultFully convolutional networks for semantic segmentation
Long, J., Shelhamer, E., & Darrell, T. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298965 · Cited by 38,834
Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification
He, K., Zhang, X., Ren, S., & Sun, J. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.123 · Cited by 19,741 · cited 3×
“We also introduce the parametric rectified linear unit (PReLU) [15] as a suitable adaptivelylearnt activation function for semantic labeling.”
BackgroundMethodInfluentialV-Net: Fully Convolutional Neural Networks for Volumetric Medical Image Segmentation
Milletari, F., Navab, N., & Ahmadi, S.-A. (2016) · 2016 Fourth International Conference on 3D Vision (3DV) · doi:10.1109/3dv.2016.79 · Cited by 11,931 · cited 1×
“This has been proposed by [20], [21] for segmentation of 3D volumes.”
MethodCaffe: Convolutional Architecture for Fast Feature Embedding
Jia, Y., Shelhamer, E., Donahue, J., Karayev, S., Long, J., Girshick, R., Guadarrama, S., & Darrell, T. (2014) · Proceedings of the 22nd ACM international conference on Multimedia · doi:10.1145/2647868.2654889 · Cited by 11,077 · cited 1×
“Our network architecture is implemented within the Caffe deep learning framework [25].”
MethodHolistically-Nested Edge Detection
Xie, S., & Tu, Z. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.164 · Cited by 3,096
Conditional Random Fields as Recurrent Neural Networks
Zheng, S., Jayasumana, S., Romera-Paredes, B., Vineet, V., Su, Z., Du, D., Huang, C., & Torr, P. H. S. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.179 · Cited by 2,386 · cited 1×
“FCNs have shown to perform well in general purpose segmentation tasks [19] as well as in instrument-background”
BackgroundEndoNet: A Deep Architecture for Recognition Tasks on Laparoscopic Videos
Twinanda, A. P., Shehata, S., Mutter, D., Marescaux, J., de Mathelin, M., & Padoy, N. (2017) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2016.2593957 · Cited by 1,096 · cited 1×
“EndoNet [12] has successfully shown the ability of Convolutional Neural Networks (CNNs) to perform instrument detection and surgical phase recognition.”
MethodHolistically-nested edge detection
Xie, S., & Tu, Z. (2017) · International Journal of Computer Vision · doi:10.1007/s11263-017-1004-z · Cited by 682 · cited 1×
“Instead, inspired by the holistically-nested edge detection [23], where the authors aggregate sigmoid cross-entropy losses from multiple scales, we propose a Multi-Scale Dice Loss (MSDL) for semantic segmentation that takes into account the accuracy”
Research works printed but not linked: 6
- kamnitsas, 2017, Efficient multiscale 3D CNN with fully connected CRF for accurate brain lesion segmentation, MedIA, 36, 61
- 0, MICCAI 2015 EndoVis Challenge
- maas, 2013, Rectifier Nonlinearities Improve Neural Network Acoustic Models, ICML
- li, 2017, On the Compactness, Efficency, and Representation of 3D Convolutional Networks: Brain Parcellation as a Pretext Task, IPMI
- smith, 2015, No more pesky learning rate guessing games
- pakhomov, 2017, Deep residual learning for instrument segmentation in robotic surgery
Cited by
Deep learning-enabled medical computer vision
Esteva, A., Chou, K., Yeung, S., Naik, N., Madani, A., Mottaghi, A., Liu, Y., Topol, E., Dean, J., & Socher, R. (2021) · npj Digital Medicine · doi:10.1038/s41746-020-00376-2 · Cited by 1,406
“Early studies have begun pursuing these objectives, primarily in video-based robotic and laparoscopic surgery — a number of works propose methods for detecting surgical tools and actions 118 – 124 .”
MethodNiftyNet: a deep-learning platform for medical imaging
Gibson, E., Li, W., Sudre, C., Fidon, L., Shakir, D. I., Wang, G., Eaton-Rosen, Z., Gray, R., Doel, T., Hu, Y., et al. (2018) · Computer Methods and Programs in Biomedicine · doi:10.1016/j.cmpb.2018.01.025 · Cited by 552
Self-citationWeakly-supervised convolutional neural networks for multimodal image registration
Hu, Y., Modat, M., Gibson, E., Li, W., Ghavami, N., Bonmati, E., Wang, G., Bandula, S., Moore, C. M., Emberton, M., et al. (2018) · Medical Image Analysis · doi:10.1016/j.media.2018.07.002 · Cited by 458
“, 2015) and medical image analysis (Dou et al., 2017; Garcia-Peraza-Herrera et al., 2017; Gibson et al., 2017a).”
BackgroundSelf-citationAutomatic Instrument Segmentation in Robot-Assisted Surgery using Deep Learning
Shvets, A. A., Rakhlin, A., Kalinin, A. A., & Iglovikov, V. I. (2018) · 2018 17th IEEE International Conference on Machine Learning and Applications (ICMLA) · doi:10.1109/icmla.2018.00100 · Cited by 379
Attanasio, A., Scaglioni, B., De Momi, E., Fiorini, P., & Valdastri, P. (2021) · Annual Review of Control, Robotics, and Autonomous Systems · doi:10.1146/annurev-control-062420-090543 · Cited by 263
“A 3D-CNN structure was implemented to account for the correlation between subsequent frames in [25].”
BackgroundMethod
Metadata
- Authors
- Garcia-Peraza-Herrera, L. C., Li, W., Fidon, L., Gruijthuijsen, C., Devreker, A., Attilakos, G., Deprest, J., Poorten, E. V., Stoyanov, D., Vercauteren, T., & Ourselin, S.
- Affiliations
- KU Leuven (BE)
- University College London (GB)
- Universitair Ziekenhuis Leuven (BE)
- Wellcome / EPSRC Centre for Interventional and Surgical Sciences (GB)
- ORCID
- Venue
- 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)
- Publisher
- IEEE
- Published
- 2017-09
- Pages
- 5717-5722
- ISSN
- 2153-0858 (ISSN-L)
- OpenAlex
- W2657144928
- Cited by
- 137
- References
- 22
- Citations by year
- 2026: 5 · 2025: 14 · 2024: 8 · 2023: 19 · 2022: 16 · 2021: 26 · 2020: 25 · 2019: 16 · 2018: 8
- FWCI
- 3.92
- Type
- conference paper
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- Research Portal (King's College London) (submittedVersion)
- Lirias (submittedVersion)
- arXiv (Cornell University)
- Funders
- Engineering and Physical Sciences Research Council (EP/H046410/1)
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Medical Image Segmentation Techniques
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
- Health Sciences › Medicine › Surgery › Surgical Simulation and Training
- Keywords
- robotic surgery, endoscopic video, multi-scale feature extraction, real-time semantic segmentation, computer-assisted surgery
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: ToolNet: Holistically-nested real-time segmentation of robotic surgical tools. https://app.cochat.ai/papers/10.1109/iros.2017.8206462
- The paper
- Garcia-Peraza-Herrera, L. C., Li, W., Fidon, L., Gruijthuijsen, C., Devreker, A., Attilakos, G., Deprest, J., Poorten, E. V., Stoyanov, D., Vercauteren, T., & Ourselin, S. (2017). ToolNet: Holistically-nested real-time segmentation of robotic surgical tools. 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). https://doi.org/10.1109/iros.2017.8206462
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.