Automatic Detection of Airway Invasion from Videofluoroscopy via Deep Learning Technology
Open accessThe DOI matches its Crossref record; 32 of the 32 references it cites were found in a registry or an index.
Abstract
In dysphagia, food materials frequently invade the laryngeal airway, potentially resulting in serious consequences, such as asphyxia or pneumonia. The VFSS (videofluoroscopic swallowing study) procedure can be used to visualize the occurrence of airway invasion, but its reliability is limited by human errors and fatigue. Deep learning technology may improve the efficiency and reliability of VFSS analysis by reducing the human effort required. A deep learning model has been developed that can detect airway invasion from VFSS images in a fully automated manner. The model consists of three phases: (1) image normalization, (2) dynamic ROI (region of interest) determination, and (3) airway invasion detection. Noise induced by movement and learning from unintended areas is minimized by defining a “dynamic” ROI with respect to the center of the cervical spinal column as segmented using U-Net. An Xception module, trained on a dataset consisting of 267,748 image frames obtained from 319 VFSS video files, is used for the detection of airway invasion. The present model shows an overall accuracy of 97.2% in classifying image frames and 93.2% in classifying video files. It is anticipated that the present model will enable more accurate analysis of VFSS data.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Videofluoroscopy of swallowing was used as the reference examination in most studies.”
From Artificial intelligence in the diagnosis and management of dysphagia: a scoping review (2025) · cites this paper as [23]
In the citing paper
“Studies on AI in dysphagia primarily rely on imaging resources such as VFSS for comparative analysis due to its high reliability(16,23-28).”
From Artificial intelligence in the diagnosis and management of dysphagia: a scoping review (2025) · cites this paper as [23]
Citations
Cites
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474 · cited 2×
“U-Net is an end-to-end FCN (fully connected network)-based model consisting of a convolutional encoder and a convolutional decoder connected by skip connections [24].”
MethodU-Net: Convolutional Networks for Biomedical Image Segmentation
Ronneberger, O., Fischer, P., & Brox, T. (2015) · Lecture notes in computer science · doi:10.1007/978-3-319-24574-4_28 · Cited by 99,404
Rethinking the Inception Architecture for Computer Vision
Szegedy, C., Vanhoucke, V., Ioffe, S., Shlens, J., & Wojna, Z. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.308 · Cited by 32,138
Focal Loss for Dense Object Detection
Lin, T.-Y., Goyal, P., Girshick, R., He, K., & Dollar, P. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.324 · Cited by 30,581 · cited 1×
“To alleviate the problem of data imbalance between these classes, we used the focal loss function [31].”
MethodPan, S. J., & Yang, Q. (2010) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2009.191 · Cited by 23,387 · cited 2×
“To effectively train the network on a small amount of data, we used the transfer learning [25] technique.”
MethodXception: Deep Learning with Depthwise Separable Convolutions
Chollet, F. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.195 · Cited by 20,380 · cited 2×
“Therefore, among the models that can be effectively trained on limited data, in this paper, we choose a CNN based on an Xception module [27].”
MethodContrast Limited Adaptive Histogram Equalization
Zuiderveld, K. (1994) · Graphics Gems · doi:10.1016/b978-0-12-336156-1.50061-6 · Cited by 4,192 · cited 1×
“Therefore, to ensure that all other conditions in all images are identical and that the convolution filters in the deep learning network are effectively trained, we first normalized all of the input images using CLAHE (contrast limited adaptive histogram equalization) [22].”
MethodA penetration-aspiration scale
Rosenbek, J. C., Robbins, J. A., Roecker, E. B., Coyle, J. L., & Wood, J. L. (1996) · Dysphagia · doi:10.1007/bf00417897 · Cited by 2,803 · cited 1×
“…was found to be misinterpreted as normal, and all false negative images showed penetration corresponding to a PAS (penetration aspiration score) of two, in which the bolus invades the laryngeal vestibule, but does not cross the true vocal folds and is expelled immediately after swallowing [35].”
BackgroundAn overview of deep learning in medical imaging focusing on MRI
Lundervold, A. S., & Lundervold, A. (2019) · Zeitschrift für Medizinische Physik · doi:10.1016/j.zemedi.2018.11.002 · Cited by 2,058 · cited 1×
“Recently, advances in deep learning technology have dramatically improved the accuracy of image classification by computers to a level exceeding that of human eyes [17].”
BackgroundBag of Tricks for Image Classification with Convolutional Neural Networks
He, T., Zhang, Z., Zhang, H., Zhang, Z., Xie, J., & Li, M. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00065 · Cited by 1,680 · cited 1×
“1 ∗ b/256 by using the linear scaling learning rate method [30] to determine the learning rate in accordance with the batch size (b is 32 for the U-Net model in Section 2.”
MethodAn overview of deep learning in medical imaging focusing on MRI
Lundervold, A. S., & Lundervold, A. (2018) · NORA - Norwegian Open Research Archives · Cited by 1,012
Evaluation and Treatment of Swallowing Disorders
Logemann, J. A. (1994) · American Journal of Speech-Language Pathology · doi:10.1044/1058-0360.0303.41 · Cited by 915
Dysphagia in the elderly: management and nutritional considerations
Crary, M., Sura, L., Madhavan, A., & Carnaby-Mann, G. (2012) · Clinical Interventions in Aging · doi:10.2147/cia.s23404 · Cited by 813 · cited 1×
“Dysphagia can lead to serious and life-threatening consequences, such as aspiration pneumonia or malnutrition, and has become a great concern in health care, especially among the aged population [2].”
BackgroundUsing Deep Learning for Classification of Lung Nodules on Computed Tomography Images
Song, Q., Zhao, L., Luo, X., & Dou, X. (2017) · Journal of Healthcare Engineering · doi:10.1155/2017/8314740 · Cited by 422
The Videofluorographic Swallowing Study
Martin-Harris, B., & Jones, B. (2008) · Physical Medicine and Rehabilitation Clinics of North America · doi:10.1016/j.pmr.2008.06.004 · Cited by 356 · cited 3×
“The VFSS (videofluoroscopic swallowing study) procedure, also known as an MBS (modified barium swallowing) examination, is regarded as the gold standard for the evaluation of dysphagia and is the examination method that is most commonly used by clinicians for this purpose [3].”
BackgroundMethodNonlinear Signal Processing: A Statistical Approach
Arce, G. R. (2004) · doi:10.1002/0471691852 · Cited by 356 · cited 1×
“To capture the temporal context of each video in the final decision on whether airway invasion has occurred, we applied two types of filters, namely a median filter [29] and a convolution filter, to the classification results for the individual images, as shown in Figure 6.”
MethodInterrater Reliability of Videofluoroscopic Swallow Evaluation
Stoeckli, S. J., Huisman, T. A. G. M., Seifert, B. A. G. M., & Martin-Harris, B. J. W. (2003) · Dysphagia · doi:10.1007/s00455-002-0085-0 · Cited by 198
Le, M. H., Chen, J., Wang, L., Wang, Z., Liu, W., Cheng, K.-T. (., & Yang, X. (2017) · Physics in Medicine & Biology · doi:10.1088/1361-6560/aa7731 · Cited by 161 · cited 1×
“Due to the accuracy and capability of automation, deep learning is expected to reduce errors in reading medical images, including conventional radiography, MRI, and CT images [18–20].”
BackgroundInter- and Intrajudge Reliability for Videofluoroscopic Swallowing Evaluation Measures
McCullough, G. H., Wertz, R. T., Rosenbek, J. C., Mills, R. H., Webb, W. G., & Ross, K. (2001) · Dysphagia · doi:10.1007/pl00021291 · Cited by 144
A Study of Interrater Reliability when Using Videofluoroscopy as an Assessment of Swallowing
Scott, A., Perry, A. R., & Bench, J. (1998) · Dysphagia · doi:10.1007/pl00009576 · Cited by 129 · cited 1×
“However, the analysis of VFSS data is vulnerable to human bias because the swallowing process occurs in a very short time of less than a few seconds and includes several events occurring simultaneously, complicating the necessary judgments [4].”
BackgroundIntra- and Interrater Variation in the Evaluation of Videofluorographic Swallowing Studies
Kuhlemeier, K. V., Yates, P., & Palmer, J. B. (1998) · Dysphagia · doi:10.1007/pl00009564 · Cited by 124
Learning to Read Chest X-Ray Images from 16000+ Examples Using CNN
Dong, Y., Pan, Y., Zhang, J., & Xu, W. (2017) · 2017 IEEE/ACM International Conference on Connected Health: Applications, Systems and Engineering Technologies (CHASE) · doi:10.1109/chase.2017.59 · Cited by 77
Inter-rater Reliability of Videofluoroscopic Dysphagia Scale
Kim, D. H., Choi, K. H., Kim, H. M., Koo, J. H., Kim, B. R., Kim, T. W., Ryu, J. S., Im, S., Choi, I. S., Pyun, S. B., et al. (2012) · Annals of Rehabilitation Medicine · doi:10.5535/arm.2012.36.6.791 · Cited by 73
Self-citationIntrarater and interrater reliability for measurements in videofluoroscopy of swallowing
Baijens, L. W. J., Barikroo, A., & Pilz, W. (2013) · European Journal of Radiology · doi:10.1016/j.ejrad.2013.05.009 · Cited by 72 · cited 1×
“The reviewing process frequently becomes time consuming and even exhausting because it requires repeated frame-by-frame or slow-motion examination [9].”
BackgroundAutomatic hyoid bone detection in fluoroscopic images using deep learning
Zhang, Z., Coyle, J. L., & Sejdić, E. (2018) · Scientific Reports · doi:10.1038/s41598-018-30182-6 · Cited by 64 · cited 3×
“One of those studies investigated the accuracy of a deep learning model for the automatic identification of the hyoid bone [16].”
BackgroundMethodLee, J. T., Park, E., & Jung, T.-D. (2019) · Sensors · doi:10.3390/s19183873 · Cited by 30
Semi-automatic tracking of hyolaryngeal coordinates in videofluoroscopic swallowing studies
Natarajan, R., Stavness, I., & Pearson, W. G. (2015) · Computer Methods in Biomechanics and Biomedical Engineering Imaging & Visualization · doi:10.1080/21681163.2015.1046190 · Cited by 21 · cited 2×
“Most previous studies have used obsolete semiautomated identification and/or tracking algorithms [12–15], in which salient landmarks must first be demarcated by humans.”
MethodAung, M. H., Goulermas, J. Y., Hamdy, S., & Power, M. (2009) · IEEE Transactions on Biomedical Engineering · doi:10.1109/tbme.2009.2019828 · Cited by 18 · cited 1×
“measured the oral and pharyngeal transit time by demarcation of anatomical landmarks using the user-steered delineation algorithm live-wire and straight-line annotators; although they demonstrated a higher correlation coefficient than that between human observers, anatomical landmarks had to be demarcated by users [12…”
MethodSemi-automatic assessment of hyoid bone motion in digital videofluoroscopic images
Hossain, I., Roberts-South, A., Jog, M., & El-Sakka, M. R. (2014) · Computer Methods in Biomechanics and Biomedical Engineering: Imaging & Visualization · doi:10.1080/21681163.2013.833859 · Cited by 17 · cited 1×
“attempted to track the motion of the hyoid bone semi-automatically by determining the region of interest using the Haar classifier, but manual identification of templates was required to increase accuracy [13].”
MethodAung, M. S. H., Goulermas, J. Y., Stanschus, S., Hamdy, S., & Power, M. (2010) · Medical Engineering & Physics · doi:10.1016/j.medengphy.2010.08.008 · Cited by 17 · cited 2×
“Aung et al. in another study attempted automatic demarcation of salient landmarks by applying a 16 point active shape model with high reliability; however, manual initialization was required to determine the co-ordinates of the reference points from the first frame of the video [10].”
MethodChang, M. W., Lin, E., & Hwang, J.-N. (1999) · Dysphagia · doi:10.1007/pl00009609 · Cited by 14 · cited 1×
“Chang et al. successfully reconstructed the three-dimensional pharyngeal bolus movement from the VHSvideos of normal subjects using a knowledge-based snake algorithm; however, its clinical usefulness has not been verified [11].”
MethodSTAMPS: development and verification of swallowing kinematic analysis software
Lee, W. H., Chun, C., Seo, H. G., Lee, S. H., & Oh, B.-M. (2017) · BioMedical Engineering OnLine · doi:10.1186/s12938-017-0412-1 · Cited by 13
Self-citation
Research works printed but not linked: 4
- Simonyan, K., and Zisserman, A. (2014). Very deep convolutional networks for large-scale image recognition. arXiv.
- Ioffe, S., and Szegedy, C. (2015). Batch normalization: Accelerating deep network training by reducing internal covariate shift. arXiv.
- Chen, L.C., Papandreou, G., Schroff, F., and Adam, H. (2017). Hartwig Rethinking atrous convolution for semantic image segmentation. arXiv.
- Zoph, B., Ghiasi, G., Lin, T.Y., Cui, Y., Liu, H., Cubuk, E.D., and Le, Q.V. (2020). Rethinking pre-training and self-training. arXiv.
Cited by
Kim, H.-I., Kim, Y., Kim, B., Shin, D. Y., Lee, S. J., & Choi, S. (2021) · Diagnostics · doi:10.3390/diagnostics11071147 · Cited by 35
“…D Rx , D Ry ) of ( H x , H y ) to ( C x , C y ) from the center point ( C x , C y ) and the position of the hyoid bone ( H x , H y ) as follows [21,25]: where D Rx and D Ry are the distances in the horizontal and vertical directions between the center points of the hyoid bone and the cervical…”
BackgroundSelf-citationJeong, S. Y., Kim, J. M., Park, J. E., Baek, S. J., & Yang, S. N. (2023) · Scientific Reports · doi:10.1038/s41598-023-44802-3 · Cited by 31
“Hence, studies have been conducted using deep learning to analyze VFSS videos 22–26 .”
BackgroundMethodGirardi, A. M., Cardell, E. A., & Bird, S. P. (2023) · Big Data and Cognitive Computing · doi:10.3390/bdcc7040178 · Cited by 16
Dysphagia foods: Current research, technological innovations, and future directions
Dong, Y., Dai, J., Han, J., Zhang, Y., Xu, Z., Jiang, L., Lan, T., & Sui, X. (2025) · Trends in Food Science & Technology · doi:10.1016/j.tifs.2025.105330 · Cited by 18
Jeong, C., Lee, C.-S., Lim, D.-W., Noh, S.-H., Moon, H. K., Park, C., & Kim, M.-S. (2024) · Brain Sciences · doi:10.3390/brainsci14060546 · Cited by 16
Metadata
- Authors
- Lee, S. J., Ko, J. Y., Kim, H. I., & Choi, S.-I.
- Affiliations
- Dankook University Hospital (KR)
- Dankook University (KR)
- Corresponding author
- Choi, Sang-Il
- Venue
- Applied Sciences
- Publisher
- MDPI AG
- Published
- 2020-09-05
- Volume
- 10
- Issue
- 18
- Pages
- 6179
- ISSN
- 2076-3417 (ISSN-L)
- OpenAlex
- W3083842555
- Cited by
- 28
- References
- 32
- Citations by year
- 2026: 3 · 2025: 11 · 2024: 4 · 2023: 7 · 2021: 3
- FWCI
- 3.55
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- Locations
- Applied Sciences (publishedVersion, cc-by)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Funders
- National Research Foundation of Korea (2018R1A2B6001400, 2018R1D1A3B07049300)
- Topics
- Health Sciences › Health Professions › Speech and Hearing › Dysphagia Assessment and Management
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Medical Image Segmentation Techniques
- Physical Sciences › Computer Science › Artificial Intelligence › AI in cancer detection
- Keywords
- videofluoroscopic swallowing study, dysphagia, laryngeal airway, region of interest segmentation, U-Net, Xception, image denoising
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.