The DOI matches its Crossref record; 31 of the 31 references it cites were found in a registry or an index.
EndoNet: A Deep Architecture for Recognition Tasks on Laparoscopic Videos
Open accessAbstract
Surgical workflow recognition has numerous potential medical applications, such as the automatic indexing of surgical video databases and the optimization of real-time operating room scheduling, among others. As a result, surgical phase recognition has been studied in the context of several kinds of surgeries, such as cataract, neurological, and laparoscopic surgeries. In the literature, two types of features are typically used to perform this task: visual features and tool usage signals. However, the used visual features are mostly handcrafted. Furthermore, the tool usage signals are usually collected via a manual annotation process or by using additional equipment. In this paper, we propose a novel method for phase recognition that uses a convolutional neural network (CNN) to automatically learn features from cholecystectomy videos and that relies uniquely on visual information. In previous studies, it has been shown that the tool usage signals can provide valuable information in performing the phase recognition task. Thus, we present a novel CNN architecture, called EndoNet, that is designed to carry out the phase recognition and tool presence detection tasks in a multi-task manner. To the best of our knowledge, this is the first work proposing to use a CNN for multiple recognition tasks on laparoscopic videos. Experimental comparisons to other methods show that EndoNet yields state-of-the-art results for both tasks.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
ImageNet classification with deep convolutional neural networks
Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017) · Communications of the ACM · doi:10.1145/3065386 · Cited by 112,155
ImageNet Large Scale Visual Recognition Challenge
Russakovsky, O., Deng, J., Su, H., Krause, J., Satheesh, S., Ma, S., Huang, Z., Karpathy, A., Khosla, A., Bernstein, M., et al. (2015) · International Journal of Computer Vision · doi:10.1007/s11263-015-0816-y · Cited by 39,380
Rich Feature Hierarchies for Accurate Object Detection and Semantic Segmentation
Girshick, R., Donahue, J., Darrell, T., & Malik, J. (2014) · 2014 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2014.81 · Cited by 32,684
Object Detection with Discriminatively Trained Part-Based Models
Felzenszwalb, P. F., Girshick, R. B., McAllester, D., & Ramanan, D. (2010) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2009.167 · Cited by 9,853
Error bounds for convolutional codes and an asymptotically optimum decoding algorithm
Viterbi, A. (1967) · IEEE Transactions on Information Theory · doi:10.1109/tit.1967.1054010 · Cited by 6,759
DeCAF: A Deep Convolutional Activation Feature for Generic Visual Recognition
Donahue, J., Jia, Y., Vinyals, O., Hoffman, J., Zhang, N., Tzeng, E., & Darrell, T. (2013) · arXiv · doi:10.48550/arxiv.1310.1531 · Cited by 4,019 · cited 2×
“Furthermore, in [30], it has been shown that the network trained in [10] is so powerful that it can be used as a blackbox feature extractor (without any modification) to success-”
BackgroundCaffe: Convolutional Architecture for Fast Feature Embedding
Jia, Y., Shelhamer, E., Donahue, J., Karayev, S., Long, J., Girshick, R., Guadarrama, S., & Darrell, T. J. (2014) · arXiv (Cornell University) · doi:10.48550/arxiv.1408.5093 · Cited by 3,752
Statistical modeling and recognition of surgical workflow
Padoy, N., Blum, T., Ahmadi, S.-A., Feussner, H., Berger, M.-O., & Navab, N. (2012) · Medical Image Analysis · doi:10.1016/j.media.2010.10.001 · Cited by 279
Self-citationHigh-dimensional signature compression for large-scale image classification
Sanchez, J., & Perronnin, F. (2011) · CVPR 2011 · doi:10.1109/cvpr.2011.5995504 · Cited by 272
Surgical gesture classification from video and kinematic data
Zappella, L., Béjar, B., Hager, G. D., & Vidal, R. (2013) · Medical Image Analysis · doi:10.1016/j.media.2013.04.007 · Cited by 154
Modeling and Segmentation of Surgical Workflow from Laparoscopic Video
Blum, T., Feußner, H., & Navab, N. (2010) · Lecture notes in computer science · doi:10.1007/978-3-642-15711-0_50 · Cited by 151
Lalys, F., Riffaud, L., Bouget, D., & Jannin, P. (2011) · IEEE Transactions on Biomedical Engineering · doi:10.1109/tbme.2011.2181168 · Cited by 131
Detecting Surgical Tools by Modelling Local Appearance and Global Shape
Bouget, D., Benenson, R., Omran, M., Riffaud, L., Schiele, B., & Jannin, P. (2015) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2015.2450831 · Cited by 128
On-line Recognition of Surgical Activity for Monitoring in the Operating Room
Padoy, N., Blum, T., Feußner, H., Berger, M., Navab, N., Lorraine, L. I., & Isar, K. R. D. (2008) · Cited by 86
Self-citationRandom Forests for Phase Detection in Surgical Workflow Analysis
Stauder, R., Okur, A., Peter, L., Schneider, A., Kranzfelder, M., Feußner, H., & Navab, N. (2014) · Lecture notes in computer science · doi:10.1007/978-3-319-07521-1_16 · Cited by 78
Feature Classification for Tracking Articulated Surgical Tools
Reiter, A., Allen, P. K., & Zhao, T. (2012) · Lecture notes in computer science · doi:10.1007/978-3-642-33418-4_73 · Cited by 76
Image Based Surgical Instrument Pose Estimation with Multi-class Labelling and Optical Flow
Allan, M., Chang, P.-L., Ourselin, S., Hawkes, D., Sridhar, A., Kelly, J. D., & Stoyanov, D. (2015) · International Conference on Medical Image Computing and Computer-Assisted Intervention · doi:10.1007/978-3-319-24553-9_41 · Cited by 73
Lea, C., Hager, G. D., & Vidal, R. (2015) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv.2015.154 · Cited by 72
Kranzfelder, M., Schneider, A., Fiolka, A., Schwan, E., Gillen, S., Wilhelm, D., Schirren, R., Reiser, S. B., Jensen, B., & Feußner, H. (2013) · Journal of Surgical Research · doi:10.1016/j.jss.2013.06.022 · Cited by 68
Automatic knowledge-based recognition of low-level tasks in ophthalmological procedures
Lalys, F., Bouget, D., Riffaud, L., & Jannin, P. (2012) · International Journal of Computer Assisted Radiology and Surgery · doi:10.1007/s11548-012-0685-6 · Cited by 59
Real-Time Task Recognition in Cataract Surgery Videos Using Adaptive Spatiotemporal Polynomials
Quellec, G., Lamard, M., Cochener, B., & Cazuguel, G. (2014) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2014.2366726 · Cited by 58
Fast Part-Based Classification for Instrument Detection in Minimally Invasive Surgery
Sznitman, R., Becker, C., & Fua, P. (2014) · Lecture notes in computer science · doi:10.1007/978-3-319-10470-6_86 · Cited by 58
Episode Classification for the Analysis of Tissue/Instrument Interaction with Multiple Visual Cues
Lo, B. P. L., Darzi, A. W., & Yang, G. (2003) · Lecture notes in computer science · doi:10.1007/978-3-540-39899-8_29 · Cited by 56
Workflow monitoring based on 3D motion features
Padoy, N., Mateus, D., Weinland, D., et al. (2009) · doi:10.1109/iccvw.2009.5457648 · Cited by 53
Self-citationAutomatic phase prediction from low-level surgical activities
Forestier, G., Riffaud, L., & Jannin, P. (2015) · International Journal of Computer Assisted Radiology and Surgery · doi:10.1007/s11548-015-1195-0 · Cited by 45
Automatic feature generation in endoscopic images
Klank, U., Padoy, N., Feußner, H., & Navab, N. (2008) · International Journal of Computer Assisted Radiology and Surgery · doi:10.1007/s11548-008-0223-8 · Cited by 44
Self-citationAutomatic classification of minimally invasive instruments based on endoscopic image sequences
Speidel, S., Benzko, J., Krappe, S., Sudra, G., Azad, P., Müller-Stich, B. P., Gutt, C. N., & Dillmann, R. (2009) · Proceedings of SPIE, the International Society for Optical Engineering/Proceedings of SPIE · doi:10.1117/12.811112 · Cited by 44
Online recognition of surgical instruments by information fusion
Neumuth, T., & Meißner, C. (2011) · International Journal of Computer Assisted Radiology and Surgery · doi:10.1007/s11548-011-0662-5 · Cited by 44
Katić, D., Wekerle, A.-L., Gärtner, F., Kenngott, H. G., Müller-Stich, B. P., Dillmann, R., & Speidel, S. (2014) · Lecture notes in computer science · doi:10.1007/978-3-319-07521-1_17 · Cited by 42
3D Sensing Algorithms Towards Building an Intelligent Intensive Care Unit.
Lea, C., Facker, J., Hager, G. D., Taylor, R. H., & Saria, S. (2013) · PubMed Central · Cited by 16
Towards Better Laparoscopic Video Database Organization by Automatic Surgery Classification
Twinanda, A. P., Marescaux, J., de Mathelin, M., et al. (2014) · Lecture notes in computer science · doi:10.1007/978-3-319-07521-1_20 · Cited by 10
Self-citation
Cited by
A survey on deep learning in medical image analysis
Litjens, G., Kooi, T., Bejnordi, B. E., Setio, A. A. A., Ciompi, F., Ghafoorian, M., van der Laak, J. A. W. M., van Ginneken, B., & Sánchez, C. I. (2017) · Medical Image Analysis · doi:10.1016/j.media.2017.07.005 · Cited by 15,332
Deep learning-enabled medical computer vision
Esteva, A., Chou, K., Yeung, S., Naik, N., Madani, A., Mottaghi, A., Liu, Y., Topol, E., Dean, J., & Socher, R. (2021) · npj Digital Medicine · doi:10.1038/s41746-020-00376-2 · Cited by 1,406
Madani, A., Namazi, B., Altieri, M. S., Hashimoto, D. A., Rivera, A. M., Pucher, P. H., Navarrete-Welton, A., Sankaranarayanan, G., Brunt, L. M., Okrainec, A., et al. (2022) · Annals of Surgery · doi:10.1097/sla.0000000000004594 · Cited by 391
SV-RCNet: Workflow Recognition From Surgical Videos Using Recurrent Convolutional Network
Jin, Y., Dou, Q., Chen, H., Yu, L., Qin, J., Fu, C.-W., & Heng, P.-A. (2018) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2017.2787657 · Cited by 316
Jin, A., Yeung, S., Jopling, J., Krause, J., Azagury, D., Milstein, A., & Fei-Fei, L. (2018) · 2018 IEEE Winter Conference on Applications of Computer Vision (WACV) · doi:10.1109/wacv.2018.00081 · Cited by 323
Metadata
- Authors
- Twinanda, A. P., Shehata, S., Mutter, D., Marescaux, J., de Mathelin, M., & Padoy, N.
- Venue
- IEEE Transactions on Medical Imaging
- Published
- 2017-01
- Volume
- 36
- Issue
- 1
- Pages
- 86-97
- ISSN
- 0278-0062 (ISSN-L)
- 1558-254X
- OpenAlex
- W2266464013
- Cited by
- 1,096
- References
- 31
- FWCI
- 46.80
- Type
- journal article
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.