SV-RCNet: Workflow Recognition From Surgical Videos Using Recurrent Convolutional Network
The DOI matches its Crossref record; 48 of the 50 references it cites were found in a registry or an index.
Abstract
We propose an analysis of surgical videos that is based on a novel recurrent convolutional network (SV-RCNet), specifically for automatic workflow recognition from surgical videos online, which is a key component for developing the context-aware computer-assisted intervention systems. Different from previous methods which harness visual and temporal information separately, the proposed SV-RCNet seamlessly integrates a convolutional neural network (CNN) and a recurrent neural network (RNN) to form a novel recurrent convolutional architecture in order to take full advantages of the complementary information of visual and temporal features learned from surgical videos. We effectively train the SV-RCNet in an end-to-end manner so that the visual representations and sequential dynamics can be jointly optimized in the learning process. In order to produce more discriminative spatio-temporal features, we exploit a deep residual network (ResNet) and a long short term memory (LSTM) network, to extract visual features and temporal dependencies, respectively, and integrate them into the SV-RCNet. Moreover, based on the phase transition-sensitive predictions from the SV-RCNet, we propose a simple yet effective inference scheme, namely the prior knowledge inference (PKI), by leveraging the natural characteristic of surgical video. Such a strategy further improves the consistency of results and largely boosts the recognition performance. Extensive experiments have been conducted with the MICCAI 2016 Modeling and Monitoring of Computer Assisted Interventions Workflow Challenge dataset and Cholec80 dataset to validate SV-RCNet. Our approach not only achieves superior performance on these two datasets but also outperforms the state-of-the-art methods by a significant margin.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Another line of work uses CV to recognize distinct phases of surgery during operations, towards developing context-aware computer assistance systems 126,127 .”
From Deep learning-enabled medical computer vision (2021) · cites this paper as [126]
Citations
Cites
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474 · cited 4×
“Specifically, we exploit the very deep residual network, the ResNet [21], to extract highly discriminative visual features from the video frames.”
BackgroundMethodInfluentialGoing deeper with convolutions
Szegedy, C., Liu, W., Jia, Y., Sermanet, P., Reed, S., Anguelov, D., Erhan, D., Vanhoucke, V., & Rabinovich, A. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298594 · Cited by 48,562
A survey on deep learning in medical image analysis
Litjens, G., Kooi, T., Bejnordi, B. E., Setio, A. A. A., Ciompi, F., Ghafoorian, M., van der Laak, J. A. W. M., van Ginneken, B., & Sánchez, C. I. (2017) · Medical Image Analysis · doi:10.1016/j.media.2017.07.005 · Cited by 15,332
Learning Deep Features for Discriminative Localization
Zhou, B., Khosla, A., Lapedriza, A., Oliva, A., & Torralba, A. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.319 · Cited by 11,428 · cited 1×
“networks on an image and intercept the learned information of networks [37], [38].”
BackgroundCaffe: Convolutional Architecture for Fast Feature Embedding
Jia, Y., Shelhamer, E., Donahue, J., Karayev, S., Long, J., Girshick, R., Guadarrama, S., & Darrell, T. (2014) · Proceedings of the 22nd ACM international conference on Multimedia · doi:10.1145/2647868.2654889 · Cited by 11,077 · cited 1×
“Our framework is implemented with C++ and Python based on the Caffe [35] deep learning library, using a TITAN X GPU for acceleration.”
MethodShin, H.-C., Roth, H. R., Gao, M., Lu, L., Xu, Z., Nogues, I., Yao, J., Mollura, D., & Summers, R. M. (2016) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2016.2528162 · Cited by 5,759
Long-term recurrent convolutional networks for visual recognition and description
Donahue, J., Hendricks, L. A., Guadarrama, S., Rohrbach, M., Venugopalan, S., Darrell, T., & Saenko, K. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298878 · Cited by 5,228 · cited 2×
“HMM, we propose to tap into the temporal dimension of the surgical video data using the LSTM [31], [32], which has been demonstrated as a very powerful tool to model temporal concepts.”
MethodDeep learning in medical image analysis
Shen, D., Wu, G., & Suk, H.-I. (2017) · Annual Review of Biomedical Engineering · doi:10.1146/annurev-bioeng-071516-044442 · Cited by 4,968
Deep visual-semantic alignments for generating image descriptions
Karpathy, A., & Fei-Fei, L. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298932 · Cited by 4,902
Generating Sequences With Recurrent Neural Networks
Graves, A. (2013) · arXiv · doi:10.48550/arxiv.1308.0850 · Cited by 2,894 · cited 1×
“HMM, we propose to tap into the temporal dimension of the surgical video data using the LSTM [31], [32], which has been demonstrated as a very powerful tool to model temporal concepts.”
Method
Research works printed but not linked: 2
- olah, 2015, Understanding LSTM networks, Proc GitHub Blog
- dergachyova, 0, Data-driven surgical workflow detection: Technical report for M2CAI 2016 surgical workflow challenge, M2CAI Challenge
Cited by
Items 201–300 of 316
CMANet: A TCN-RMamba-Attention Network for Surgical Phase Online Recognition
Fan, W., Wang, Y., Liu, L., Zeng, J., & Liu, M. (2025) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2025.3599391 · Cited by 4
“Unlike previous methods that extract spatial or temporal features independently, SV-RCNet utilizes ResNet to obtain visual features from the current frame and employs LSTM to capture temporal features, thereby complementing each other [17].”
Deep Learning-Driven Models for Endoscopic Image Analysis
Jia, X., Xing, X., Yuan, Y., & Meng, M. Q. ‐. (2021) · Computational biology · doi:10.1007/978-3-030-69951-2_11 · Cited by 4
Surgical Action Recognition and Prediction with Transformers
Huang, G. (2022) · IEEE International Conference on Software Engineering and Artificial Intelligence (SEAI) · doi:10.1109/seai55746.2022.9832094 · Cited by 4
Zhang, B., Sturgeon, D., Shankar, A. R., Goel, V. K., Barker, J., Ghanem, A. G., Lee, P., Milecky, M., Stottler, N., & Petculescu, S. (2022) · Computer Methods in Biomechanics and Biomedical Engineering Imaging & Visualization · doi:10.1080/21681163.2022.2152371 · Cited by 5
“Examples of video-based surgical workflow analysis include surgical phase recognition (Jin et al. 2017, 2021; Zhang et al. 2021b,a; Ramesh et al. 2021; Wang et al. 2022; Kadkhodamohammadi et al. 2022), surgical gesture and action recognition (Zia et al. 2018; Qin et al. 2020; Kabata¸s and…”
MethodProposal-Based Graph Attention Networks for Workflow Detection
Zhang, M., Hu, H., Li, Z., & Chen, J. (2021) · Neural Processing Letters · doi:10.1007/s11063-021-10622-7 · Cited by 5
Surgical Workflow Recognition Using Two-Stream Mixed Convolution Network
Ding, Y., Fan, J.-F., Pang, K., Li, H., Fu, T.-Y., Song, H., Chen, L., & Yang, J. (2020) · 2020 3rd International Conference on Advanced Electronic Materials, Computers and Software Engineering (AEMCSE) · doi:10.1109/aemcse50948.2020.00064 · Cited by 5
Xu, Y., Ban, Y., Zhao, Y., Krauss, D. T., Bruns, C. J., Eckhoff, J. A., & Fuchs, H. F. (2025) · Computational and Structural Biotechnology Journal · doi:10.1016/j.csbj.2025.07.056 · Cited by 3
“Initial research on the application of AI in LC mainly concentrated on modeling surgical phase recognition [21 – 24] due to the standardized workflow and high frequency of execution.”
Yen, H., Ho, M., Yang, M.-H., Hsiao, Y.-H., Huang, H., Huang, J.-Y., Huang, C.-C., & Blue, J. (2025) · Surgical Endoscopy · doi:10.1007/s00464-025-12316-w · Cited by 3
“Segment-wise strategies are widely adopted in surgical video and workflow analysis [24, 25]; however, variations in surgical action sequences among different surgeons— even during the same procedural task—pose a significant challenge for temporal action analysis.”
Yousef, M., Aly, K. E., Ahmed, M., Ahmed, F. A., Jalham, K. A., & Balakrishnan, S. (2026) · Frontiers in Digital Health · doi:10.3389/fdgth.2026.1633888 · Cited by 4
“Similarly, Jin’s 2018 article “SV-RCNet: Workflow Recognition From Surgical Videos Using Recurrent Convolutional Network” (7) also focuses on workflow recognition, specifically in laparoscopic cholecystectomy videos.”
van Jaarsveld, R. C., Li, Y., de Jong, R. L. P. D., Badaloni, F., Kuiper, G. M., Jaspers, T. J. M., Breeuwer, M. M., van der Sommen, F., van Hillegersberg, R., Khalil, Y. A., et al. (2026) · Surgical Endoscopy · doi:10.1007/s00464-026-12985-1 · Cited by 3
“SV-RCNet [25] and TMRNet [26] are end-to-end architectures that integrate a ResNet-50 backbone with a Long Short-Term Memory (LSTM) module for joint spatial and temporal modeling.”
Multi-Granular Semantic Analysis Based on Nasal Endoscopic Video
Pan, X., Zhao, H., Liu, N., & Wang, H. (2020) · IEEE Access · doi:10.1109/access.2020.3017523 · Cited by 3
“In recent years, two-stream network [29], [30], [34]and C3D network [31], [32], [35] learns appearance and motion features.”
BackgroundChanges of Physiological parameters of the patient during laparoscopic gynaecology
Jalal, N. A., Alshirbaji, T. A., Docherty, P. D., Neumuth, T., Laufer, B., & Möeller, K. (2021) · Current Directions in Biomedical Engineering · doi:10.1515/cdbme-2021-2127 · Cited by 3
Oza, R. S., Mehta, M. A., Kotecha, K. V., & Lin, J. C. (2023) · Multimedia Tools and Applications · doi:10.1007/s11042-023-17344-z · Cited by 3
“Therefore, as out-lined in previous methods, numerous deep model-based spatiotemporal learning methods have been proposed to address the afermentioned challenges [14–21 The SV-RCNet method, introduced in [10], combines the CNN and RNN models for surgical phase classification.”
Lopez, A. B., Makarenko, M. O., Bonifazi, M., de Oliveira, B. N. M., Getman, F., Tian, Y., Mazzone, V., Li, N., Giammona, A., Liberale, C., et al. (2024) · Communications Biology · doi:10.1038/s42003-024-05839-w · Cited by 3
“Work in this area can leverage the large body of research developed in video understanding using deep-learning 34 – 36 .”
Frequency-Based Temporal Analysis Network for Accurate Phase Recognition from Surgical Videos
Zhang, S., Xu, T., Cao, Z. W., Liao, H., Ning, G., & Chen, F. (2024) · IEEE International Symposium on Biomedical Imaging (ISBI) · doi:10.1109/isbi56570.2024.10635806 · Cited by 3
Improving surgical phase recognition using self-supervised deep learning
López, A. C., González-Cebrián, Á., Paredes, I., Lagares, A., & de Toledo, P. (2025) · Scientific Reports · doi:10.1038/s41598-025-23697-2 · Cited by 4
Pradeep, C. S., & Sinha, N. (2023) · Lecture notes in computer science · doi:10.1007/978-3-031-25075-0_48 · Cited by 3
Deep learning‐based intraoperative video analysis for supporting surgery
Bai, J. (2023) · Concurrency and Computation Practice and Experience · doi:10.1002/cpe.7837 · Cited by 4
Brief Overview of Neural Networks for Medical Applications
Hireš, M., Bugata, P., Gazda, M., Hresko, D. J., Kanász, R., Vavrek, L., & Drotár, P. (2022) · Acta Electrotechnica et Informatica · doi:10.2478/aei-2022-0010 · Cited by 5
An Experimental Study of nmODE in Recognizing Endoscopic Submucosal Dissection Workflow
Huang, K., Yuan, X., Liu, R., Zhou, Y., Hu, B., & Zhang, Y. (2023) · International Annual Conference on Complex Systems and Intelligent Science (CSIS-IAC) · doi:10.1109/csis-iac60628.2023.10364057 · Cited by 3
“Jin et al. [3] proposed SV-RCNet, an end-to-end ResNet-LSTM network, for segmenting and classifying surgical video sequences.”
BackgroundLong, J., Hong, J., Wang, Z., Chen, T., Chen, Y., & Yang, L. (2023) · IEEE International Conference on Bioinformatics and Biomedicine (BIBM) · doi:10.1109/bibm58861.2023.10385579 · Cited by 4
CSMAE : Cataract Surgical Masked Autoencoder (MAE) Based Pre-Training
Shah, N. A., Bandara, C., Sikder, S., Vedula, S. S., & Patel, V. M. (2025) · IEEE International Symposium on Biomedical Imaging (ISBI) · doi:10.1109/isbi60581.2025.10981288 · Cited by 7
Oza, R. S., Mehta, M. A., & Siarry, P. (2025) · International Journal of Medical Robotics and Computer Assisted Surgery · doi:10.1002/rcs.70089 · Cited by 3
SurgPLAN++: Universal Surgical Phase Localization Network for Online and Offline Inference
Chen, Z., Luo, X., Wu, J., Bai, L., Lei, Z., Ren, H., Ourselin, S., & Liu, H. (2025) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra55743.2025.11127834 · Cited by 6
Hand Frame Extraction in Surgical Video Images Using Convolutional Neural Network
Sakib, S., Hossain, B., Hiranaka, T., & Kobashi, S. (2020) · Joint International Conference on Soft Computing and Intelligent Systems and International Symposium on Advanced Intelligent Systems (SCIS&ISIS) · doi:10.1109/scisisis50064.2020.9322759 · Cited by 3
Hard frame detection for the automated clipping of surgical nasal endoscopic video
Wang, H., Pan, X., Zhao, H., Gao, C., & Liu, N. (2021) · International Journal of Computer Assisted Radiology and Surgery · doi:10.1007/s11548-021-02311-6 · Cited by 3
Domain Adaptive Robotic Gesture Recognition with Unsupervised Kinematic-Visual Data Alignment
Shi, X., Jin, Y., Dou, Q., Qin, J., & Heng, P. (2021) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros51168.2021.9636578 · Cited by 3
Self-citationBilal, M., Alam, M., Bapu, D., Körsgen, S., Lal, N., Bach, S. P., Hajiyavand, A. M., Ali, M., Soomro, K., Qasim, I., et al. (2025) · Scientific Data · doi:10.1038/s41597-025-05646-w · Cited by 7
“Recent studies have focused on deriving the workflow solely from routinely collected endoscopic videos during surgery 15 .”
Surgical Phase Recognition in Laparoscopic Cholecystectomy
Li, Y., Gupta, H., Prasanna, P., Ramakrishnan, I. V., & Ling, H. (2024) · Procedia Computer Science · doi:10.1016/j.procs.2024.06.386 · Cited by 2
SurgX: Neuron-Concept Association for Explainable Surgical Phase Recognition
Kim, K. Y., Kim, H. B., & Kim, S. T. (2025) · Lecture notes in computer science · doi:10.1007/978-3-032-05127-1_51 · Cited by 2
DORC: Surgical Workflow Recognition Tool Using Fast R-CNN and Modified HMMs
Putalapattu, P. (2021) · Lecture notes in networks and systems · doi:10.1007/978-3-030-89880-9_47 · Cited by 2
Frontier of Smart Healthcare Engineering Management
Ding, S., Wu, D. D., Zhao, L., & Li, X. (2022) · AI for risks · doi:10.1007/978-981-19-2560-3_2 · Cited by 2
Kawamura, K., Ebata, R., Nakamura, R., & Otori, N. (2022) · International Journal of Computer Assisted Radiology and Surgery · doi:10.1007/s11548-022-02754-5 · Cited by 2
“reported a video analysis in laparoscopic surgery [11], and Yu et al.”
MethodTraining in robotic pancreatic surgery
Body, S., Kawka, M., & Gall, T. (2022) · Artificial Intelligence Surgery · doi:10.20517/ais.2022.28 · Cited by 2
Surgical phase recognition in laparoscopic videos using gated capsule autoencoder model
Konduri, P. S., & Rao, G. S. N. (2023) · Computer Methods in Biomechanics and Biomedical Engineering Imaging & Visualization · doi:10.1080/21681163.2023.2203280 · Cited by 2
OSFENet: Object Spatiotemporal Feature Enhanced Network for Surgical Phase Recognition
You, P., Zhang, Y., Hu, H., Wang, Y., & Fang, B. (2024) · Lecture notes in computer science · doi:10.1007/978-981-97-5615-5_19 · Cited by 2
Sun, Y., Shi, X., Zhai, S., Zhang, K., Pan, B., & Fu, Y. (2024) · Robotic Intelligence and Automation · doi:10.1108/ria-04-2024-0088 · Cited by 2
Multi-Teacher Temporal Regulation Network for Surgical Workflow Recognition
Huang, K., Yuan, X., Liu, R.-D., Ye, L., Zhou, Y., Hu, B., & Zhang, Y. (2025) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2025.3580082 · Cited by 2
“For example, Jin et al. proposed SV-RCNet [4], which seamlessly integrates a residual back-bone with a Long-Short-Term Memory (LSTM) module to recognize surgical phases from 10s video clips.”
InfluentialSTANet: A Surgical Gesture Recognition Method Based on Spatiotemporal Fusion
Jia, B., Wang, W., Tian, X., & Wang, X. (2025) · Annals of the New York Academy of Sciences · doi:10.1111/nyas.70053 · Cited by 3
SPRMamba: Surgical Phase Recognition for Endoscopic Submucosal Dissection With Mamba
Zhang, X., Zhang, Q. Y., Chen, J., Zhou, C., Wang, Y., Zhang, Z., Li, X., & Qian, D. (2026) · IEEE Sensors Journal · doi:10.1109/jsen.2025.3649151 · Cited by 12
“For the Cholec80 and AutoLaparo datasets, we benchmark our approach against a broader set of SOTA methods, including LAST [51], SV-RCNet [25], TeCNO [6], TMRNet [5], Trans-SVNet [7], SAHC [50], SKiT [52], and LoViT [53].”
InfluentialArtificial intelligence in surgery: A research team perspective
Mohamadipanah, H., Perumalla, C. A., Yang, S., Wise, B. J., Kearse, L. E., Goll, C. K., Witt, A. K., Korndorffer, J. R., & Pugh, C. M. (2022) · Current Problems in Surgery · doi:10.1016/j.cpsurg.2022.101125 · Cited by 3
Surgical Phase Recognition Method with a Sequential Consistency for CAOS-AI Navigation System
Nishio, S., Hossain, B., Yagi, N., Nii, M., Hiranaka, T., & Kobashi, S. (2020) · IEEE Global Conference on Life Sciences and Technologies (LifeTech) · doi:10.1109/lifetech48969.2020.1570619203 · Cited by 2
You, J., Wu, S., Wang, X., & Peng, B. (2021) · Annals of Surgery · doi:10.1097/sla.0000000000005263 · Cited by 2
Zhao, P., Sun, H., & Han, Q. (2025) · International Journal of Surgery · doi:10.1097/js9.0000000000002726 · Cited by 2
“We encourage future work to strengthen cross-scenario adaptability, build task-specific quantitative assessment frameworks, and conduct comparative analysis with existing models [ 2 ] (e.g., EndoNet [ 3 ] , SV-RCNet [ 4 ] ) to advance the clinical translation of such technologies.”
Surgical Action Triplet Recognition by Using A Multi-Task Prior-Reinforced and Cross-Sample Network
Li, Y., Zhao, Z., & Li, R. (2024) · International Conference on Electronic Engineering and Informatics (EEI) · doi:10.1109/eei63073.2024.10696468 · Cited by 1
“…understanding at a coarse-grained level, that is, they only focus on a single aspect of the surgery, such as the detection of surgical instruments [1], surgical phase recognition [2], which cannot provide a comprehensive and accurate description of fine-grained content in a given surgical video.”
BackgroundArthroPhase: a novel dataset and method for phase recognition in arthroscopic video
Malayeri, A. B., Seibold, M., Cavalcanti, N., Hein, J., Jecklin, S., Vlachopoulos, L., Fucentese, S., Hodel, S., & Fürnstahl, P. (2025) · Computer Assisted Surgery · doi:10.1080/24699322.2025.2508144 · Cited by 3
“These spatio-temporally coherent features are fed into two branches of transformers that differ in input length, enhancing temporal resolution and precision in identifying and classifying surgical phases.”
Lan, M., Si, W., Yan, X., & Li, X. (2025) · Medical Image Analysis · doi:10.1016/j.media.2025.103724 · Cited by 1
A technological convergence in hepatobiliary oncology: Evolving roles of smart surgical systems
Bai, X., Huang, R., Liu, Q., Jin, X., Wang, L., Tang, W., Karako, K., & Zhu, W. (2025) · BioScience Trends · doi:10.5582/bst.2025.01047 · Cited by 2
STSANet: Spatial Temporal-Self-Aggregation Network for surgical phase recognition
Li, Y., Zhao, G., li, C., Shi, W., Jiang, Z., Zhang, Z., & Feng, G. (2025) · Information Fusion · doi:10.1016/j.inffus.2025.103646 · Cited by 2
Artificial intelligence-driven phase recognition in lung surgery: a single-centre pilot study
Ohtani-Kim, S., Samejima, J., Wakabayashi, M., Tada, M., Miyoshi, T., Matsumura, Y., Tane, K., Aokage, K., Ishikawa, Y., Hayashi, K., et al. (2025) · ESMO Real World Data and Digital Oncology · doi:10.1016/j.esmorw.2025.100194 · Cited by 2
Pan, X.-Y., Zhang, Z.-Z., Chen, H., Wang, H., & He, X. (2026) · Biomedical Signal Processing and Control · doi:10.1016/j.bspc.2025.109052 · Cited by 1
Self-citationTemporal Propagation of Asymmetric Feature Pyramid for Surgical Scene Segmentation
Yuan, C., & Ban, Y. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-09784-2_19 · Cited by 1
Procedure-Aware Hierarchical Alignment for Open Surgery Video-Language Pretraining
Xu, B., Wu, J., Liang, J., Sun, Z., Liu, H., Luo, J., & Lei, Z. (2026) · IEEE Transactions on Image Processing · doi:10.1109/tip.2026.3659752 · Cited by 1
“These advances span various tasks, including surgical workflow analysis [20], [44], [49], instrument and anatomical structure segmentation [15], [31], [38], and depth estimation [53], among others.”
Wang, Y., Yang, X., Wang, T., Wang, T., Wang, Y., & Zhu, X. (2026) · Journal of Robotic Surgery · doi:10.1007/s11701-026-04095-w · Cited by 1
Research on workflow recognition for liver rupture repair surgery
Men, Y., Zhao, Z., Chen, W., Wu, H., Zhang, G., Luo, F., & Yu, M. Z. (2024) · Mathematical Biosciences & Engineering · doi:10.3934/mbe.2024080 · Cited by 1
Label-Guided Teacher for Surgical Phase Recognition via Knowledge Distillation
Guan, J., Zou, X., Tao, R., & Guoyan Zheng, A. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-72089-5_33 · Cited by 2
Step Detection in Cataract Surgery Videos
Shah, N. A., Sikder, S., Vedula, S. S., & Patel, V. M. (2025) · IEEE International Symposium on Biomedical Imaging (ISBI) · doi:10.1109/isbi60581.2025.10980885 · Cited by 2
ViPeR: Vision-Based Surgical Phase Recognition
Chandra, S., Chowdhury, S. S., Yong, C., Jeng, G., Mahenthiran, A. K., Morris, K., Love, H. L., Sundaram, C. P., & Roy, K. (2025) · IEEE Access · doi:10.1109/access.2025.3588428 · Cited by 1
“Following the convention established by previous research [20], [23], [24], [26], [29], [38], the dataset is divided into two parts, with the first 40 videos used for training and the remaining 40 for testing.”
Liu, M., Duan, F., Ling, L., Hu, J., Ge, M., Zhang, X., & Cai, S. (2025) · International Journal of Computer Assisted Radiology and Surgery · doi:10.1007/s11548-025-03509-8 · Cited by 1
Nishio, S., Hossain, B., Nii, M., Yagi, N., Hiranaka, T., & Kobashi, S. (2020) · International Journal of Affective Engineering · doi:10.5057/ijae.ijae-d-19-00018 · Cited by 1
“To recognize the surgical phase from video data, SV-RC Net [10] and B. Namazi et al .”
MethodEbata, R., Nakamura, R., Kawamura, K., & Otori, N. (2021) · Journal of Japan Society of Computer Aided Surgery · doi:10.5759/jscas.23.48 · Cited by 1
SUrgical PRediction GAN for Events Anticipation.Preprint
Ban, Y., Rosman, G., Ward, T. M., Hashimoto, D. A., Kondo, T., Iwaki, H., Meireles, O. R., & Rus, D. L. (2021) · arXiv (Cornell University) · Cited by 1
Exploiting Segment-level Semantics for Online Phase Recognition from Surgical VideosPreprint
Ding, X., & Li, X. (2021) · arXiv (Cornell University) · Cited by 1
Interventional procedures training
Ungi, T., Holden, M., Zevin, B., & Gabor Fichtinger, A. (2019) · Elsevier eBooks · doi:10.1016/b978-0-12-816176-0.00042-9
Ninth air force service command
Lasch, M. C., & Heins, E. M. (1945)
Guzmán-García, C. (2021) · UPM Digital Archive (Technical University of Madrid) · doi:10.20868/upm.thesis.69221 · Cited by 1
Gu, S. (2022) · Sensors and Materials · doi:10.18494/sam3929
Multi-agent Neural-Like Models for the Integration of Multimodal Medical Examination Data
Вячеславович, Н. З., Nagoeva, O., Ауесовна, П. И., Bzhikhatlov, K. C., Gurtueva, I. A., & Kankulov, S. (2023) · Lecture notes in computer science · doi:10.1007/978-3-031-25191-7_24
Automatic Step Recognition with Video and Kinematic Data for Intelligent Operating Room and Beyond
Chng, C.-B., Lin, W., Hu, Y., Hu, Y., Liu, J., & Chui, C. (2023) · International Symposium on Information and Communication Technology · doi:10.1145/3628797.3628999
“Jin et al. [9] proposed an end-to-end recurrent convolutional model including a CNN network to capture frame-wise visual information and a LSTM to model the clip-wise sequential dynamics.”
MethodZhao, X., de Perez, A. R., Dimitrova, E., Kemp, M. L., & Anderson, P. E. (2024) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2024.03.11.584308
“The integration of Convolutional Neural Networks (CNNs) with Recurrent Neural Networks (RNN) [15]–[19] and the adoption of Transformer models for sequential image and video classification [20]– [23] exemplify the significant potential of sequential modeling in image classification tasks.”
MethodComputer Assisted Phase Recognition of Micro-Neurosurgical Intraoperative Videos
Vadali, P. S., Jayam, B., Deepika, P., Rao, M., & Vazhayil, V. V. (2024) · Proceedings - IEEE Symposium on Computer-Based Medical Systems · doi:10.1109/cbms61543.2024.00081
“SV-RCNet, a novel recurrent convolutional network that integrates a ResNet backbone with an LSTM to predict phases on the endoscopy videos for cholecystectomy surgery [7].”
BackgroundMethodCurrent application of artificial intelligence in laparoscopic cholecystectomy
Chooklin, S., & Chuklin, S. S. (2024) · EMERGENCY MEDICINE · doi:10.22141/2224-0586.20.6.2024.1764
PCNet: Accurate Surgical Workflow Recognition with Phase Consistency
Tran, B., Kulić, D., Harrison, S. M., Pilgrim, C. H. C., & Abdi, E. (2024) · Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC) · doi:10.1109/embc53108.2024.10782329
“To better encapsulate the intricate temporal relationships along the input scenes, later works such as SV-RCNet [6] adopted Long Short-term Memory (LSTM) [7] and achieved superior accuracy compared to HMM-based models.”
Automated Assessment of Simulated Laparoscopic Surgical Performance using 3DCNN
Power, D., & Ullah, I. (2024) · Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC) · doi:10.1109/embc53108.2024.10782160
“The first successful use of deep learning (DL) in MIS was in 2016 with the Convolutional Neural Network (CNN), EndoNet [11].”
Image Processing and Machine Vision in Surgery and Its Training
Ahmadi, M. J., Allahkaram, M. S., Abdi, P., Mohammadi, S.-F., & Taghirad, H. D. (2023) · Journal of Control · doi:10.61186/joc.17.2.25 · Cited by 1
Surgical video analysis using an ensemble of multitask recurrent convolutional neural networks
Oliveira, B., Torres, H. R., Morais, P. G., & Vilaça, J. L. (2025) · SPIE Medical Imaging: Image-Guided Procedures, Robotic Interventions, and Modeling · doi:10.1117/12.3047310
“To take advantage of the temporal information, recent works proposed learning high-level spatio-temporal features using a 3D convolutional neural network (CNN) or recurrent neural networks (RNN) [4]– [6].”
Yao, G., Hayashi, Y., Oda, M., Misawa, K., & Mori, K. (2025) · SPIE Medical Imaging: Image-Guided Procedures, Robotic Interventions, and Modeling · doi:10.1117/12.3047731
Xia, Y., Song, S., Yang, Y., Lü, T., & Xiao, R. (2025) · International Conference on Sensors and Information Technology (ICSI) · doi:10.1109/icsi64877.2025.11009385
Deshpande, A. H., Borkar, S. S., & Baheti, J. (2025) · Algorithms for intelligent systems · doi:10.1007/978-981-96-3648-8_3 · Cited by 1
Ophora: A Large-Scale Data-Driven Text-Guided Ophthalmic Surgical Video Generation Model
Li, W., Hu, M., Wang, G., Liu, L., Zhou, K., Ning, J., Guo, X., Ge, Z., Gu, L., & He, J. (2025) · Lecture notes in computer science · doi:10.1007/978-3-032-05114-1_41 · Cited by 5
Towards Robust Algorithms for Surgical Phase Recognition via Digital Twin Representation
Ding, H., Zhang, Y., Cheng, W., Wang, X., Xu, L., Yu, C., Shu, H., Kim, J. W., Krieger, A., & Unberath, M. (2025) · Lecture notes in computer science · doi:10.1007/978-3-032-07694-6_12 · Cited by 10
“With the rise of deep learning, end-to-end feed-forward networks [23,13,27,10,17,26] have demonstrated strong performance on SPR benchmarks [23,24].”
Surgical Phase Detection with Deep Learning for Thoracoscopic Pulmonary Lobectomy Surgery
Yang, Q., Liu, C., Zhou, Z., Feng, Y., Zheng, H., Wang, J., Tang, J., Hou, Y., Su, B., & Zhang, X. (2025) · WRC Symposium on Advanced Robotics and Automation (WRC SARA) · doi:10.1109/wrcsara68202.2025.11195045
“In this study, we adopt the SV-RCNet [21] model as the foundational architecture (see Figure 1), which is an end-to-end spatiotemporal feature learning framework specifically designed for surgical workflow recognition.”
ESIP: Explicit Surgical Instrument Prompting for Surgical Workflow Recognition
Qiu, Y., Liu, M., He, S.-Y., Zhou, G., Lyu, F., Chen, Y., & Zhou, P. (2025) · IEEE journal of biomedical and health informatics · doi:10.1109/jbhi.2025.3625420
“2) Evaluation Metrics: To comprehensively evaluate SWR performance, we follow previous works [6], [7], [8], [11], [15] to employ four evaluation metrics including Accuracy, Precision, Recall and Jaccard.”
InfluentialOpen-world surgical video generation via dual-visual diffusion and dual-annealed generation
Ma, N., Yang, S., Zhou, Y., Zhang, C., Chen, J., & He, X. (2025) · Neural Networks · doi:10.1016/j.neunet.2025.108281
Li, L., Deng, Z.-X., Chen, J.-C., Li, X.-J., & Wang, Q.-Y. (2025) · International Conference on Informatics and Analytics · doi:10.1109/icia64617.2025.11277854
“SV-RCNet was the first to propose an end-to-end general phase recognition model for surgical visual question answering [15].”
STFMamba: A Neurocognitive-Inspired Dual-Path State Space Model for Surgical Phase Recognition
Wang, J., Wang, W., Wen, Y., Liao, H., Huang, T., & Chen, F. (2025) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2025.3643273
“Against SV-RCNet [15], we achieve +8.3% higher accuracy while reducing parameters by 31.4% and FLOPs by ON THE A UTO L APARO D ATASET 40.3%.”
InfluentialSurgical Gesture Recognition Method for Visual Feedback of Surgical Robots
Jia, B., Wang, W., Tian, X., & Wang, X. (2025) · International Conference on Biomedical Signal and Image Processing (ICBIP) · doi:10.1109/icbip66535.2025.11284386
“Commonly employed architectures in-This clude temporal convolutional networks [12], recurrent neural networks [13], and 3D convolutional neural networks [14].”
Liu, Q., Chen, Z., Fan, K., Musi, G., Mistretta, F. A., Luzzago, S., Ferrigno, G., & Momi, E. D. (2025) · IEEE Transactions on Medical Robotics and Bionics · doi:10.1109/tmrb.2025.3646753
“PhaseNet [8] and SV-RCNet [9] employ AlexNet [16] and ResNet-50 [17], respectively, for spatial feature extraction, followed by LSTM to capture temporal dynamics.”
Han, S., Lu, M., Liu, Y., Pei, Y., & Ma, K. (2026) · Applied Soft Computing · doi:10.1016/j.asoc.2026.114654
Sstanet: Surgical Spatio-Temporal Aggregation Network for Phase Recognition
Li, Y., Li, C., Jin, C., Zhao, G., Miao, Y., & Jiang, Z. (2025) · IEEE International Conference on Parallel and Distributed Systems (ICPADS) · doi:10.1109/icpads67057.2025.11323072
“Subsequently, Jin et al. adopted LSTM [9]–[11] and proposed TMRNet, which integrates global information through a non-local operator [12].”
InfluentialB2Q-Net: Bidirectional Branch Query Network for Surgical Phase Recognition
Zhang, W., Li, Z., Bi, Y., Jia, X., Song, R., Zhang, Y., & Zhang, W. (2026) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2026.3654795 · Cited by 1
“Among these, a number of methods [4], [5], [6], [7] employ convolutional or recurrent neural networks to encode temporal dynamics.”
InfluentialGDPA: A Parameter-Efficient Gated Dual-Path Spatiotemporal Adapter for Surgical Workflow Recognition
Wang, S., Guo, Y., Shao, H., Liu, Y.-S., Xiong, J., Wang, L., Yan, X., & Yan, Y. (2025) · IEEE International Conference on Bioinformatics and Biomedicine · doi:10.1109/bibm66473.2025.11356335 · Cited by 1
“For example, SV-RCNet [31] tightly couples CNNs and RNNs to predict phases directly from video sequences, reducing the error accumulation of two-stage pipelines.”
He, H., Yin, Z., & Song, C. (2026) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2026.3662737
“Depending on the application needs, surgical video analysis can extract information at varying levels of granularity, including instrument detection, surgical phase recognition, and step segmentation [5], [6], [7].”
Surgical Activity Recognition with Frame-Action Cross-Attention Temporal Modeling
Kamabattula, S., Perreault, C., & Bhattacharyya, K. (2026) · Journal of Medical Robotics Research · doi:10.1142/s2424905x26500029
Yao, G., Hayashi, Y., Oda, M., Misawa, K., & Mori, K. (2026) · SPIE Medical Imaging: Image-Guided Procedures, Robotic Interventions, and Modeling · doi:10.1117/12.3086006
Analyse automatique et assistance à l'annotation des processus chirurgicaux basées Deep Learning
Lécuyer, G. (2020) · theses.fr (ABES) · doi:10.70675/f909088dz2b19z40e9z884ez8364ebad1f0b
Javaheri, H., Ghamarnejad, O., Schwarzkopf, P. D. K., Karolus, J., Stavrou, G. A., & Lukowicz, P. (2026) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3772318.3790327
“…temporal prediction models, such as phase recognition networks and temporal convolutional architectures such as those used in Cholec80 and related benchmarks [17, 31], can determine which surgical phase is currently underway, how the procedure is progressing, and how much time is likely remaining.”
Surgical-Clip: A Dual-Branch Temporal Clip for Surgical Video Recognition
He, M., Zhang, M., & Yuan, W. (2026) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp55912.2026.11465022
Niu, J., Xiong, Z., Liu, C., Li, S., Ye, J., Lv, C., 束国栋, Zhu, Y., Chen, S., Long, D., et al. (2026) · Biomedical Signal Processing and Control · doi:10.1016/j.bspc.2026.110494
Artificial intelligence for the future of digital surgery
Jin, Y., Dou, Q., & Kudsi, O. Y. (2026) · npj Digital Surgery · doi:10.1038/s44484-026-00008-4
Self-citation
Metadata
- Authors
- Jin, Y., Dou, Q., Chen, H., Yu, L., Qin, J., Fu, C.-W., & Heng, P.-A.
- Affiliations
- Chinese University of Hong Kong (HK)
- Hong Kong Polytechnic University (HK)
- Venue
- IEEE Transactions on Medical Imaging
- Published
- 2018-05
- Volume
- 37
- Issue
- 5
- Pages
- 1114-1126
- ISSN
- 0278-0062 (ISSN-L)
- 1558-254X
- OpenAlex
- W2777273430
- Cited by
- 316
- References
- 50
- Citations by year
- 2026: 32 · 2025: 55 · 2024: 37 · 2023: 45 · 2022: 45 · 2021: 50 · 2020: 30 · 2019: 17 · 2018: 4
- FWCI
- 14.22
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- IEEE Transactions on Medical Imaging (publishedVersion)
- PubMed (publishedVersion)
- Rare & Special e-Zone (The Hong Kong University of Science and Technology) (submittedVersion)
- PolyU Institutional Research Archive (Hong Kong Polytechnic University) (submittedVersion)
- Funders
- Research Grants Council of Hong Kong Special Administrative Region (CUHK 14202514, CUHK 14203115)
- Shenzhen Science and Technology Program (JCYJ20170413162617606)
- Hong Kong Polytechnic University (1-ZE8J)
- Topics
- Health Sciences › Medicine › Surgery › Surgical Simulation and Training
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Medical Image Segmentation Techniques
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Human Pose and Action Recognition
- Keywords
- surgical workflow recognition, surgical video analysis, recurrent convolutional networks, computer-assisted surgery, spatio-temporal features, Cholec80 dataset, ResNet, LSTM
- MeSH terms
- Algorithms, Humans, Image Processing, Computer-Assisted, Software, Databases, Factual, Neural Networks, Computer, Video-Assisted Surgery, Workflow
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.