CoChat

Automatic Detection of the Pharyngeal Phase in Raw Videos for the Videofluoroscopic Swallowing Study Using Efficient Data Collection and 3D Convolutional Networks

Open access

The DOI matches its Crossref record; 34 of the 34 references it cites were found in a registry or an index.

Abstract

Videofluoroscopic swallowing study (VFSS) is a standard diagnostic tool for dysphagia. To detect the presence of aspiration during a swallow, a manual search is commonly used to mark the time intervals of the pharyngeal phase on the corresponding VFSS image. In this study, we present a novel approach that uses 3D convolutional networks to detect the pharyngeal phase in raw VFSS videos without manual annotations. For efficient collection of training data, we propose a cascade framework which no longer requires time intervals of the swallowing process nor the manual marking of anatomical positions for detection. For video classification, we applied the inflated 3D convolutional network (I3D), one of the state-of-the-art network for action classification, as a baseline architecture. We also present a modified 3D convolutional network architecture that is derived from the baseline I3D architecture. The classification and detection performance of these two architectures were evaluated for comparison. The experimental results show that the proposed model outperformed the baseline I3D model in the condition where both models are trained with random weights. We conclude that the proposed method greatly reduces the examination time of the VFSS images with a low miss rate.

Overview

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.3390/s19183873

Cites

34 of 34 references verified0 citation errors · 0 not checkable

Statistics

Figures

Figure 1 (a) Structural anatomy and normal physiologic swallowing of a thick liquid bolus in (b) the oral phase; (c) the pharyngeal phase; and (d) the esophageal phase.
Figure 1 (a) Structural anatomy and normal physiologic swallowing of a thick liquid bolus in (b) the oral phase; (c) the pharyngeal phase; and (d) the esophageal phase.CC BY · PubMed Central
Figure 2 An overview of our framework.
Figure 2 An overview of our framework.CC BY · PubMed Central
Figure 3 The box-plots of the video length of the raw videofluoroscopic swallowing study (VFSS) videos, the number of occurrences, and the video length of the pharyngeal phases in each raw VFSS video (from left to right). On the horizontal axes, n, k, ss, and s indicate thin liquid, thick liquid, semi-sold, and solid, respectively. The labels 3, 6, 9 indicate the amount of substance used in the experiment in milliliters.
Figure 3 The box-plots of the video length of the raw videofluoroscopic swallowing study (VFSS) videos, the number of occurrences, and the video length of the pharyngeal phases in each raw VFSS video (from left to right). On the horizontal axes, n, k, ss, and s indicate thin liquid, thick liquid, semi-sold, and solid, respectively. The labels 3, 6, 9 indicate the amount of substance used in the experiment in milliliters.CC BY · PubMed Central
Figure 4 RGB (top) and optical flow (bottom) visualization of (a) positive sample—pharyngeal phase; and (b) negative sample—other action (e.g., head motions due to coughing).
Figure 4 RGB (top) and optical flow (bottom) visualization of (a) positive sample—pharyngeal phase; and (b) negative sample—other action (e.g., head motions due to coughing).CC BY · PubMed Central
Figure 5 An evaluation of the sample detection results.
Figure 5 An evaluation of the sample detection results.CC BY · PubMed Central
Figure 6 Detection results on a raw VFSS video. The detected occurrences of the pharyngeal phase are indicated by the red colored right angle bracket. From a total of 4103 frames, 160 frames are selectively displayed.
Figure 6 Detection results on a raw VFSS video. The detected occurrences of the pharyngeal phase are indicated by the red colored right angle bracket. From a total of 4103 frames, 160 frames are selectively displayed.CC BY · PubMed Central

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Show all 34 references

Research works printed but not linked: 3

  1. Leonard, 2017, Two methods for quantifying pharyngeal residue on fluoroscopic swallow studies: Reliability assessment, Ann. Otolaryngol. Rhinol., 4, 1168
  2. Kay, W., Carreira, J., Simonyan, K., Zhang, B., Hillier, C., Vijayanarasimhan, S., Viola, F., Green, T., Back, T., and Natsev, P. (2017). The kinetics human action video dataset. arXiv.
  3. Soomro, K., Zamir, A.R., and Shah, M. (2012). UCF101: A dataset of 101 human actions classes from videos in the wild. arXiv.

Cited by

Metadata

Authors
Lee, J. T., Park, E., & Jung, T.-D.
Affiliations
  • Electronics and Telecommunications Research Institute (KR)
  • Kyungpook National University (KR)
Corresponding author
Park, Eunhee
Venue
Sensors
Publisher
MDPI AG
Published
2019-09-07
Volume
19
Issue
18
Pages
3873
ISSN
1424-8220 (ISSN-L)
OpenAlex
W2971813254
Cited by
30
References
34
Citations by year
2026: 2 · 2025: 4 · 2024: 2 · 2023: 10 · 2022: 3 · 2021: 4 · 2020: 5
FWCI
5.07
Type
journal article
Language
en
Open access
Locations
  • Sensors (publishedVersion, cc-by)
  • PubMed Central (publishedVersion, cc-by)
  • PubMed (publishedVersion)
  • DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
Funders
Biomedical Research Institute grant, Kyungpook National University Hospital (2018) (NA)
Topics
  • Health Sciences › Health Professions › Speech and Hearing › Dysphagia Assessment and Management
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Human Pose and Action Recognition
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
Keywords
videofluoroscopic swallowing study, dysphagia, aspiration detection, 3D convolutional networks, I3D, training data collection

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.