CoChat

Data-driven spatio-temporal RGBD feature encoding for action recognition in operating rooms

Twinanda, Andru P., Alkan, Emre O., Gangi, Afshin, de Mathelin, Michel, Padoy, Nicolas · 2015 · International Journal of Computer Assisted Radiology and Surgery

The DOI matches its Crossref record; the references it cites are not checked yet.

Abstract

Purpose Context-aware systems for the operating room (OR) provide the possibility to significantly improve surgical workflow through various applications such as efficient OR scheduling, context-sensitive user interfaces, and automatic transcription of medical procedures. Being an essential element of such a system, surgical action recognition is thus an important research area. In this paper, we tackle the problem of classifying surgical actions from video clips that capture the activities taking place in the OR.Methods We acquire recordings using a multi-view RGBD camera system mounted on the ceiling of a hybrid OR dedicated to X-ray-based procedures and annotate clips of the recordings with the corresponding actions. To recognize the surgical actions from the video clips, we use a classification pipeline based on the bag-of-words (BoW) approach. We propose a novel feature encoding method that extends the classical BoW approach. Instead of using the typical rigid grid layout to divide the space of the feature locations, we propose to learn the layout from the actual 4D spatio-temporal locations of the visual features. This results in a data-driven and non-rigid layout which retains more spatio-temporal information compared to the rigid counterpart.Results We classify multi-view video clips from a new dataset generated from 11-day recordings of real operations. This dataset is composed of 1734 video clips of 15 actions. These include generic actions (e.g., moving patient to the OR bed) and actions specific to the vertebroplasty procedure (e.g., hammering). The experiments show that the proposed non-rigid feature encoding method performs better than the rigid encoding one. The classifier's accuracy is increased by over 4 %, from 81.08 to 85.53 %.Conclusion The combination of both intensity and depth information from the RGBD data provides more discriminative power in carrying out the surgical action recognition task as compared to using either one of them alone. Furthermore, the proposed non-rigid spatio-temporal feature encoding scheme provides more discriminative histogram representations than the rigid counterpart. To the best of our knowledge, this is also the first work that presents action recognition results on multi-view RGBD data recorded in the OR.

Overview

Integrity signals

Registry

Crossref · 10.1007/s11548-015-1186-1

Cites

Statistics

Recomputing p-values needs the paper’s full text.

Verification

Claims the paper makes

This paper’s full text isn’t openly available, so its claims and self-check weren’t run. The registry and citation checks still apply.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Citations

The citation lists are queued

They are fetched later and appear here then.

References 22 · Cited by 45

Metadata

Authors
Twinanda, A. P., Alkan, E. O., Gangi, A., de Mathelin, M., & Padoy, N.
Affiliations
  • Centre National de la Recherche Scientifique (FR)
  • Université de Bourgogne (FR)
Corresponding author
Twinanda, Andru P.
Venue
International Journal of Computer Assisted Radiology and Surgery
Published
2015-04-07
Volume
10
Issue
6
Pages
737-747
ISSN
  • 1861-6410 (ISSN-L)
  • 1861-6429
OpenAlex
W2104449113
Cited by
45
References
22
Citations by year
2026: 1 · 2025: 5 · 2024: 2 · 2023: 2 · 2022: 10 · 2021: 5 · 2020: 4 · 2019: 6 · 2018: 2 · 2017: 5 · 2016: 3
FWCI
2.89
Type
journal article
Language
en
Open access
closed
Licence
www.springer.com/tdm (tdm)
Locations
  • International Journal of Computer Assisted Radiology and Surgery (publishedVersion)
  • PubMed (publishedVersion)
Funders
  • Agence Nationale de la Recherche (ANR-11-LABX-0004, ANR-10-IDEX-0002)
  • Agence Nationale pour le Développement de la Recherche Universitaire
Topics
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Human Pose and Action Recognition
  • Health Sciences › Medicine › Surgery › Surgical Simulation and Training
  • Physical Sciences › Computer Science › Human-Computer Interaction › Hand Gesture Recognition Systems
Keywords
surgical action recognition, operating room, RGB-D video, bag-of-words, multi-view video, vertebroplasty, surgical workflow, context-aware systems
MeSH terms
Algorithms, Computer Systems, Humans, Operating Rooms, Video Recording, Surgery, Computer-Assisted

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: Data-driven spatio-temporal RGBD feature encoding for action recognition in operating rooms. https://app.cochat.ai/papers/10.1007/s11548-015-1186-1
The paper
Twinanda, A. P., Alkan, E. O., Gangi, A., de Mathelin, M., & Padoy, N. (2015). Data-driven spatio-temporal RGBD feature encoding for action recognition in operating rooms. International Journal of Computer Assisted Radiology and Surgery. https://doi.org/10.1007/s11548-015-1186-1

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.