The DOI matches its Crossref record; 57 of the 57 references it cites were found in a registry or an index.
A Dataset and Benchmarks for Segmentation and Recognition of Gestures in Robotic Surgery
Open accessAbstract
OBJECTIVE: State-of-the-art techniques for surgical data analysis report promising results for automated skill assessment and action recognition. The contributions of many of these techniques, however, are limited to study-specific data and validation metrics, making assessment of progress across the field extremely challenging. METHODS: In this paper, we address two major problems for surgical data analysis: First, lack of uniform-shared datasets and benchmarks, and second, lack of consistent validation processes. We address the former by presenting the JHU-ISI Gesture and Skill Assessment Working Set (JIGSAWS), a public dataset that we have created to support comparative research benchmarking. JIGSAWS contains synchronized video and kinematic data from multiple performances of robotic surgical tasks by operators of varying skill. We address the latter by presenting a well-documented evaluation methodology and reporting results for six techniques for automated segmentation and classification of time-series data on JIGSAWS. These techniques comprise four temporal approaches for joint segmentation and classification: hidden Markov model, sparse hidden Markov model (HMM), Markov semi-Markov conditional random field, and skip-chain conditional random field; and two feature-based ones that aim to classify fixed segments: bag of spatiotemporal features and linear dynamical systems. RESULTS: Most methods recognize gesture activities with approximately 80% overall accuracy under both leave-one-super-trial-out and leave-one-user-out cross-validation settings. CONCLUSION: Current methods show promising results on this shared dataset, but room for significant progress remains, particularly for consistent prediction of gesture activities across different surgeons. SIGNIFICANCE: The results reported in this paper provide the first systematic and uniform evaluation of surgical activity recognition techniques on the benchmark database.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
A tutorial on hidden Markov models and selected applications in speech recognition
Rabiner, L. R. (1989) · Proceedings of the IEEE · doi:10.1109/5.18626 · Cited by 22,743
Eigenfaces vs. Fisherfaces: recognition using class specific linear projection
Belhumeur, P. N., Hespanha, J. P., & Kriegman, D. J. (1997) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/34.598228 · Cited by 11,707
To Err is Human: Building a Safer Health System
Hendee, W. R. (2001) · Journal of Vascular and Interventional Radiology · doi:10.1016/s1051-0443(01)70072-3 · Cited by 10,103
$rm K$-SVD: An Algorithm for Designing Overcomplete Dictionaries for Sparse Representation
Aharon, M., Elad, M., & Bruckstein, A. (2006) · IEEE Transactions on Signal Processing · doi:10.1109/tsp.2006.881199 · Cited by 9,451
Video Google: a text retrieval approach to object matching in videos
Sivic, & Zisserman (2003) · Proceedings Ninth IEEE International Conference on Computer Vision · doi:10.1109/iccv.2003.1238663 · Cited by 6,367
Forney, G. D. (1973) · Proceedings of the IEEE · doi:10.1109/proc.1973.9030 · Cited by 5,647
Visual categorization with bags of keypoints
Csurka, G. (2004) · Cited by 4,379
Linear spatial pyramid matching using sparse coding for image classification
Yang, J., Yu, K., Gong, Y., & Huang, T. (2009) · 2009 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2009.5206757 · Cited by 2,828
Laptev, I. (2005) · International Journal of Computer Vision · doi:10.1007/s11263-005-1838-7 · Cited by 2,629
Objective structured assessment of technical skill (OSATS) for surgical residents
Martin, J. A., Regehr, G., Reznick, R., Macrae, H., Murnaghan, J., Hutchison, C., & Brown, M. (1997) · British Journal of Surgery · doi:10.1002/bjs.1800840237 · Cited by 2,527
Pictorial Structures for Object Recognition
Felzenszwalb, P. F., & Huttenlocher, D. P. (2005) · International Journal of Computer Vision · doi:10.1023/b:visi.0000042934.15159.49 · Cited by 2,180
Action recognition by dense trajectories
Wang, H., Klaser, A., Schmid, C., & Liu, C.-L. (2011) · CVPR 2011 · doi:10.1109/cvpr.2011.5995407 · Cited by 2,154
Online dictionary learning for sparse coding
Mairal, J., Bach, F., Ponce, J., & Sapiro, G. (2009) · Proceedings of the 26th Annual International Conference on Machine Learning · doi:10.1145/1553374.1553463 · Cited by 2,114
(2001) · Cited by 1,932
Zhang, J., Marszałek, M., Lazebnik, S., & Schmid, C. (2007) · International Journal of Computer Vision · doi:10.1007/s11263-006-9794-4 · Cited by 1,923
Large Margin Methods for Structured and Interdependent Output Variables
Tsochantaridis, I., Joachims, T., Hofmann, T., & Altün, Y. (2005) · MPG.PuRe (Max Planck Society) · Cited by 1,879
Evaluation of local spatio-temporal features for action recognition
Wang, H., Ullah, M. M., Klaser, A., Laptev, I., & Schmid, C. (2009) · Procedings of the British Machine Vision Conference 2009 · doi:10.5244/c.23.124 · Cited by 1,296
Cutting-plane training of structural SVMs
Joachims, T., Finley, T., & Yu, C.-N. J. (2009) · Machine Learning · doi:10.1007/s10994-009-5108-8 · Cited by 958
Doretto, G., Chiuso, A., Wu, Y. N., & Soatto, S. (2003) · International Journal of Computer Vision · doi:10.1023/a:1021669406132 · Cited by 885
An Efficient Dense and Scale-Invariant Spatio-Temporal Interest Point Detector
Willems, G., Tuytelaars, T., & Van Gool, L. (2008) · Lecture Notes in Computer Science · doi:10.1007/978-3-540-88688-4_48 · Cited by 869
An Introduction to Conditional Random Fields for Relational Learning
Sutton, C., & McCallum, A. (2007) · Introduction to Statistical Relational Learning · doi:10.7551/mitpress/7432.003.0006 · Cited by 841
Chaudhry, R., Ravichandran, A., Hager, G., & Vidal, R. (2009) · 2009 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2009.5206821 · Cited by 621
Self-citationGoh, A. C., Goldfarb, D. W., Sander, J. C., Miles, B. J., & Dunkin, B. J. (2012) · Journal of Urology · doi:10.1016/j.juro.2011.09.032 · Cited by 480
Datta, V., Mackay, S., Mandalia, M., & Darzi, A. (2001) · Journal of the American College of Surgeons · doi:10.1016/s1072-7515(01)01041-9 · Cited by 415
Statistical modeling and recognition of surgical workflow
Padoy, N., Blum, T., Ahmadi, S.-A., Feussner, H., Berger, M.-O., & Navab, N. (2012) · Medical Image Analysis · doi:10.1016/j.media.2010.10.001 · Cited by 279
Rosen, J., Hannaford, B., Richards, C. G., & Sinanan, M. N. (2001) · IEEE Transactions on Biomedical Engineering · doi:10.1109/10.918597 · Cited by 263
Towards automatic skill evaluation: Detection and segmentation of robot-assisted surgical motions
Lin, H. C., Shafran, I., Yuh, D., & Hager, G. D. (2006) · Computer Aided Surgery · doi:10.3109/10929080600989189 · Cited by 254
Self-citationSegmental Spatiotemporal CNNs for Fine-Grained Action Segmentation
Lea, C., Reiter, A., Vidal, R., & Hager, G. D. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-46487-9_3 · Cited by 246
Self-citationBlock-Coordinate Frank-Wolfe Optimization for Structural SVMsPreprint
Lacoste-Julien, S., Jaggi, M., Schmidt, M., & Pletscher, P. (2012) · arXiv (Cornell University) · doi:10.48550/arxiv.1207.4747 · Cited by 176
Hatala, R., Cook, D. A., Brydges, R., & Hawkins, R. (2015) · Advances in Health Sciences Education · doi:10.1007/s10459-015-9593-1 · Cited by 176
Sparse Hidden Markov Models for Surgical Gesture Classification and Skill Evaluation
Tao, L., Elhamifar, E., Khudanpur, S. P., Hager, G. D., & Vidal, R. (2012) · Lecture notes in computer science · doi:10.1007/978-3-642-30618-1_17 · Cited by 163
Self-citationSkills evaluation in minimally invasive surgery using force/torque signatures
Richards, C., Rosén, J., Hannaford, B., et al. (2000) · Surgical Endoscopy · doi:10.1007/s004640000230 · Cited by 161
Surgical gesture classification from video and kinematic data
Zappella, L., Béjar, B., Hager, G. D., & Vidal, R. (2013) · Medical Image Analysis · doi:10.1016/j.media.2013.04.007 · Cited by 146
Self-citationTask versus Subtask Surgical Skill Evaluation of Robotic Minimally Invasive Surgery
Reiley, C. E., & Hager, G. D. (2009) · Lecture notes in computer science · doi:10.1007/978-3-642-04268-3_54 · Cited by 141
Self-citationObjective evaluation of expert and novice performance during robotic surgical training tasks
Judkins, T. N., Oleynikov, D., & Stergiou, N. (2008) · Surgical Endoscopy · doi:10.1007/s00464-008-9933-9 · Cited by 141
Human Action Segmentation and Recognition Using Discriminative Semi-Markov Models
Shi, Q., Cheng, L., Wang, L., & Smola, A. J. (2010) · International Journal of Computer Vision · doi:10.1007/s11263-010-0384-0 · Cited by 138
Rosén, J., Solazzo, M., Hannaford, B., & D, M. N. S. M. (2002) · Computer Aided Surgery · doi:10.1002/igs.10026 · Cited by 138
MacKenzie, L., Ibbotson, J. A., Cao, C., & Lomax, A. (2001) · MITAT. Minimally invasive therapy & allied technologies · doi:10.1080/136457001753192222 · Cited by 131
Learning convolutional action primitives for fine-grained action recognition
Lea, C. S., Vidal, R., & Hager, G. (2016) · IEEE International Conference on Robotics and Automation · doi:10.1109/icra.2016.7487305 · Cited by 126
Self-citationSurgical Gesture Segmentation and Recognition
Tao, L., Zappella, L., Hager, G. D., & Vidal, R. (2013) · Lecture notes in computer science · doi:10.1007/978-3-642-40760-4_43 · Cited by 121
Self-citationData-Derived Models for Segmentation with Application to Surgical Assessment and Training
Varadarajan, B., Reiley, C. E., Lin, H., Khudanpur, S. P., & Hager, G. D. (2009) · Lecture notes in computer science · doi:10.1007/978-3-642-04268-3_53 · Cited by 105
Self-citationAssessment of surgical competence
Darzi, A. W., & MacKay, S. (2001) · BMJ Quality & Safety · doi:10.1136/qhc.0100064 · Cited by 92
Automatic Detection and Segmentation of Robot-Assisted Surgical Motions
Lin, H. C., Izhak Shafran, J., Murphy, T. E., Okamura, A. M., Yuh, D. D., & Hager, G. D. (2005) · Lecture notes in computer science · doi:10.1007/11566465_99 · Cited by 89
Self-citationWorkflow mining for visualization and analysis of surgeries
Blum, T., Padoy, N., Feußner, H., & Navab, N. (2008) · International Journal of Computer Assisted Radiology and Surgery · doi:10.1007/s11548-008-0239-0 · Cited by 87
Miyawaki, F., Masamune, K., Suzuki, S. V., Yoshimitsu, K., & Vain, J. (2005) · IEEE Transactions on Industrial Electronics · doi:10.1109/tie.2005.855692 · Cited by 84
Ahmidi, N., Poddar, P., Jones, J. D., et al. (2015) · International Journal of Computer Assisted Radiology and Surgery · doi:10.1007/s11548-015-1194-1 · Cited by 80
Self-citationAutomatic recognition of surgical motions using statistical modeling for capturing variability.
Reiley, C. E., Lin, H., Varadarajan, B., Vágvölgyi, B., Khudanpur, S., Yuh, D. D., & Hager, G. D. (2008) · PubMed · Cited by 78
Self-citationSurgical Gesture Classification from Video Data
Béjar, B., Zappella, L., & Vidal, R. (2012) · Lecture notes in computer science · doi:10.1007/978-3-642-33415-3_5 · Cited by 60
Self-citationAutomatic knowledge-based recognition of low-level tasks in ophthalmological procedures
Lalys, F., Bouget, D., Riffaud, L., et al. (2012) · International Journal of Computer Assisted Radiology and Surgery · doi:10.1007/s11548-012-0685-6 · Cited by 57
A Boosted Segmentation Method for Surgical Workflow Analysis
Padoy, N., Blum, T., Essa, I. A., Feußner, H., Berger, M., & Navab, N. (2007) · Lecture notes in computer science · doi:10.1007/978-3-540-75757-3_13 · Cited by 50
Effective Training and Assessment of Surgical Skills, and the Correlates of Performance
Hamstra, S. J., & Dubrowski, A. (2005) · Surgical Innovation · doi:10.1177/155335060501200110 · Cited by 48
Automatic feature generation in endoscopic images
Klank, U., Padoy, N., Feußner, H., & Navab, N. (2008) · International Journal of Computer Assisted Radiology and Surgery · doi:10.1007/s11548-008-0223-8 · Cited by 44
HMM assessment of quality of movement trajectory in laparoscopic surgery
Leong, J., Nicolaou, M., Atallah, L., et al. (2007) · Computer Aided Surgery · doi:10.3109/10929080701730979 · Cited by 44
Analysis of the Structure of Surgical Activity for a Suturing and Knot-Tying Task
Vedula, S. S., Malpani, A., Tao, L., Chen, G., Gao, Y., Poddar, P., Ahmidi, N., Paxton, C., Vidal, R., Khudanpur, S. P., et al. (2016) · PLoS ONE · doi:10.1371/journal.pone.0149174 · Cited by 39
Self-citationAn Application-Dependent Framework for the Recognition of High-Level Surgical Tasks in the OR
Lalys, F., Riffaud, L., Bouget, D., & Jannin, P. (2011) · Lecture notes in computer science · doi:10.1007/978-3-642-23623-5_42 · Cited by 31
Sensor substitution for video-based action recognition
Rupprecht, C., Lea, C., Tombari, F., Navab, N., & Hager, G. D. (2016) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros.2016.7759769 · Cited by 28
Self-citationInvited commentary: The ACGME “Outcomes Project”
Pellegrini, C. A. (2002) · Surgery · doi:10.1067/msy.2002.120666 · Cited by 21
Cited by
Wang, Z., & Majewicz Fey, A. (2018) · International Journal of Computer Assisted Radiology and Surgery · doi:10.1007/s11548-018-1860-1 · Cited by 275
Attanasio, A., Scaglioni, B., De Momi, E., Fiorini, P., & Valdastri, P. (2021) · Annual Review of Control, Robotics, and Autonomous Systems · doi:10.1146/annurev-control-062420-090543 · Cited by 263
A systematic review on artificial intelligence in robot-assisted surgery
Moglia, A., Georgiou, K., Georgiou, E., Satava, R. M., & Cuschieri, A. (2021) · International Journal of Surgery · doi:10.1016/j.ijsu.2021.106151 · Cited by 236
Application of artificial intelligence in surgery
Zhou, X.-Y., Guo, Y., Shen, M., & Yang, G.-Z. (2020) · Frontiers of Medicine · doi:10.1007/s11684-020-0770-0 · Cited by 181
Sharma, H., Drukker, L., Chatelain, P., Droste, R., Papageorghiou, A. T., & Noble, J. A. (2021) · Medical Image Analysis · doi:10.1016/j.media.2021.101973 · Cited by 48
Metadata
- Authors
- Ahmidi, N., Tao, L., Sefati, S., Gao, Y., Lea, C., Haro, B. B., Zappella, L., Khudanpur, S., Vidal, R., & Hager, G. D.
- Venue
- IEEE Transactions on Biomedical Engineering
- Published
- 2017-09
- Volume
- 64
- Issue
- 9
- Pages
- 2025-2041
- ISSN
- 0018-9294 (ISSN-L)
- 1558-2531
- OpenAlex
- W2568518337
- Cited by
- 289
- References
- 57
- FWCI
- 23.25
- Type
- journal article
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.