Cited by 48Surgical Simulation and TrainingArtificial Intelligence in Healthcare and EducationMedical Image Segmentation Techniques

Knowledge representation and learning of operator clinical workflow from full-length routine fetal ultrasound scan videos

Open access
Sharma, Harshita, Drukker, Lior, Chatelain, Pierre, Droste, Richard, Papageorghiou, Aris T., Noble, J. Alison·2021·Medical Image Analysis

Abstract

Ultrasound is a widely used imaging modality, yet it is well-known that scanning can be highly operator-dependent and difficult to perform, which limits its wider use in clinical practice. The literature on understanding what makes clinical sonography hard to learn and how sonography varies in the field is sparse, restricted to small-scale studies on the effectiveness of ultrasound training schemes, the role of ultrasound simulation in training, and the effect of introducing scanning guidelines and standards on diagnostic image quality. The Big Data era, and the recent and rapid emergence of machine learning as a more mainstream large-scale data analysis technique, presents a fresh opportunity to study sonography in the field at scale for the first time. Large-scale analysis of video recordings of full-length routine fetal ultrasound scans offers the potential to characterise differences between the scanning proficiency of experts and trainees that would be tedious and time-consuming to do manually due to the vast amounts of data. Such research would be informative to better understand operator clinical workflow when conducting ultrasound scans to support skills training, optimise scan times, and inform building better user-machine interfaces. This paper is to our knowledge the first to address sonography data science, which we consider in the context of second-trimester fetal sonography screening. Specifically, we present a fully-automatic framework to analyse operator clinical workflow solely from full-length routine second-trimester fetal ultrasound scan videos. An ultrasound video dataset containing more than 200 hours of scan recordings was generated for this study. We developed an original deep learning method to temporally segment the ultrasound video into semantically meaningful segments (the video description). The resulting semantic annotation was then used to depict operator clinical workflow (the knowledge representation). Machine learning was applied to the knowledge representation to characterise operator skills and assess operator variability. For video description, our best-performing deep spatio-temporal network shows favourable results in cross-validation (accuracy: 91.7%), statistical analysis (correlation: 0.98, p < 0.05) and retrospective manual validation (accuracy: 76.4%). For knowledge representation of operator clinical workflow, a three-level abstraction scheme consisting of a Subject-specific Timeline Model (STM), Summary of Timeline Features (STF), and an Operator Graph Model (OGM), was introduced that led to a significant decrease in dimensionality and computational complexity compared to raw video data. The workflow representations were learnt to discriminate between operator skills, where a proposed convolutional neural network-based model showed most promising performance (cross-validation accuracy: 98.5%, accuracy on unseen operators: 76.9%). These were further used to derive operator-specific scanning signatures and operator variability in terms of type, order and time distribution of constituent tasks.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1016/j.media.2021.101973

Cites

76 of 76 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Flagged first, then by citations

Cited by

Sorted by citations

Authors
Sharma, H., Drukker, L., Chatelain, P., Droste, R., Papageorghiou, A. T., & Noble, J. A.
Affiliations
University of Oxford (GB)
Corresponding author
Sharma, Harshita
Venue
Medical Image Analysis
Publisher
Elsevier BV
Published
2021-04
Volume
69
Pages
101973
ISSN
1361-8415 (ISSN-L)
OpenAlex
W3121390535
Cited by
48
References
76
Citations by year
2026: 6 · 2025: 8 · 2024: 8 · 2023: 8 · 2022: 12 · 2021: 6
FWCI
7.99
Type
journal article
Language
en
Open access
Locations
  • Medical Image Analysis (publishedVersion, cc-by-nc-nd)
  • PubMed (publishedVersion)
  • St George's Online Research Archive (St George's University of London) (acceptedVersion)
Funders
  • Engineering and Physical Sciences Research Council
  • NIHR Oxford Biomedical Research Centre
  • European Research Council
Topics
  • Health Sciences › Medicine › Surgery › Surgical Simulation and Training
  • Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Medical Image Segmentation Techniques
Keywords
prenatal ultrasound, ultrasound video analysis, video semantic segmentation, spatiotemporal deep learning, knowledge representation, operator variability, ultrasound training
MeSH terms
Computer Simulation, Female, Humans, Pregnancy, Retrospective Studies, Ultrasonography, Prenatal, Neural Networks, Computer, Workflow

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review