Automatic Detection of the Pharyngeal Phase in Raw Videos for the Videofluoroscopic Swallowing Study Using Efficient Data Collection and 3D Convolutional Networks
Open accessThe DOI matches its Crossref record; 34 of the 34 references it cites were found in a registry or an index.
Abstract
Videofluoroscopic swallowing study (VFSS) is a standard diagnostic tool for dysphagia. To detect the presence of aspiration during a swallow, a manual search is commonly used to mark the time intervals of the pharyngeal phase on the corresponding VFSS image. In this study, we present a novel approach that uses 3D convolutional networks to detect the pharyngeal phase in raw VFSS videos without manual annotations. For efficient collection of training data, we propose a cascade framework which no longer requires time intervals of the swallowing process nor the manual marking of anatomical positions for detection. For video classification, we applied the inflated 3D convolutional network (I3D), one of the state-of-the-art network for action classification, as a baseline architecture. We also present a modified 3D convolutional network architecture that is derived from the baseline I3D architecture. The classification and detection performance of these two architectures were evaluated for comparison. The experimental results show that the proposed model outperformed the baseline I3D model in the condition where both models are trained with random weights. We conclude that the proposed method greatly reduces the examination time of the VFSS images with a low miss rate.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Figures






Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“VFSS is the only technique able to show the bolus throughout the three phases of the swallow”
From Automated pharyngeal phase detection and bolus localization in videofluoroscopic swallowing study: killing two birds with one stone? (2022) · cites this paper as [9]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“From an algorithmic perspective, these two problems have always been tackled independently, for example: the pharyngeal phase has usually been detected in a video segmentation fashion using convolutional neural networks (CNNs) [9,14,15], whereas the bolus has been localized via image segmentation algorithms [16].”
From Automated pharyngeal phase detection and bolus localization in videofluoroscopic swallowing study: killing two birds with one stone? (2022) · cites this paper as [9]
Experimental — accuracy not yet measuredIn the citing paper
“Recent results obtained by Lee et al. [9,15] demonstrated that this phase can be detected with accuracy higher than 90%.”
From Automated pharyngeal phase detection and bolus localization in videofluoroscopic swallowing study: killing two birds with one stone? (2022) · cites this paper as [9]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474
Learning Spatiotemporal Features with 3D Convolutional Networks
Tran, D., Bourdev, L., Fergus, R., Torresani, L., & Paluri, M. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.510 · Cited by 9,593
Quo Vadis, Action Recognition? A New Model and the Kinetics Dataset
Carreira, J., & Zisserman, A. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.502 · Cited by 9,570
SGDR: Stochastic Gradient Descent with Warm Restarts
Loshchilov, I., & Hutter, F. (2016) · arXiv · doi:10.48550/arxiv.1608.03983 · Cited by 5,309
Hmdb: a large video database for human motion recognition
Kuehne, H., Jhuang, H., Garrote, E., Poggio, T., & Serre, T. (2011) · 2011 International Conference on Computer Vision · doi:10.1109/iccv.2011.6126543 · Cited by 3,965 · cited 1×
“While the Inflated Inception-V1 architecture achieved a state-of-the-art performance in action classifications such as UCF-101 [34], HMDB-51 [35], and Kinetics [30], we modified the architecture to improve the performance in classification on the VFSS videos as shown in Table 1.”
MethodA Duality Based Approach for Realtime TV-L 1 Optical Flow
Zach, C., Pock, T., & Bischof, H. (2007) · Lecture Notes in Computer Science · doi:10.1007/978-3-540-74936-3_22 · Cited by 1,658 · cited 1×
“First, we applied a TV-L1 optical flow algorithm [31,32] on gray-scale images.”
MethodAspiration Pneumonitis and Aspiration Pneumonia
Marik, P. E. (2001) · New England Journal of Medicine · doi:10.1056/nejm200103013440908 · Cited by 1,627
Spectral–Spatial Classification of Hyperspectral Imagery with 3D Convolutional Neural Network
Li, Y., Zhang, H., & Shen, Q. (2017) · Remote Sensing · doi:10.3390/rs9010067 · Cited by 1,320
Dysphagia: current reality and scope of the problem
Clavé, P., & Shaker, R. (2015) · Nature Reviews Gastroenterology & Hepatology · doi:10.1038/nrgastro.2015.49 · Cited by 577
Ertekin, C., & Aydogdu, I. (2003) · Clinical Neurophysiology · doi:10.1016/s1388-2457(03)00237-2 · Cited by 536
Research works printed but not linked: 3
- Leonard, 2017, Two methods for quantifying pharyngeal residue on fluoroscopic swallow studies: Reliability assessment, Ann. Otolaryngol. Rhinol., 4, 1168
- Kay, W., Carreira, J., Simonyan, K., Zhang, B., Hillier, C., Vijayanarasimhan, S., Viola, F., Green, T., Back, T., and Natsev, P. (2017). The kinetics human action video dataset. arXiv.
- Soomro, K., Zamir, A.R., and Shah, M. (2012). UCF101: A dataset of 101 human actions classes from videos in the wild. arXiv.
Cited by
2023 IEEE International Conference on Image Processing (ICIP)
(2023) · doi:10.1109/icip49359.2023 · Cited by 353
Lee, J. T., Park, E., Hwang, J.-M., Jung, T.-D., & Park, D. (2020) · Scientific Reports · doi:10.1038/s41598-020-71713-4 · Cited by 49
Self-citationKim, J. K., Choo, Y. J., Choi, G. S., Shin, H., Chang, M. C., & Park, D. (2022) · Journal of Korean Medical Science · doi:10.3346/jkms.2022.37.e42 · Cited by 48
Kim, H.-I., Kim, Y., Kim, B., Shin, D. Y., Lee, S. J., & Choi, S. (2021) · Diagnostics · doi:10.3390/diagnostics11071147 · Cited by 35
Automatic detection of airway invasion from videofluoroscopy via deep learning technology
Lee, S. J., Ko, J. Y., Kim, H. I., & Choi, S.-I. (2020) · Applied Sciences · doi:10.3390/app10186179 · Cited by 28
Jeong, S. Y., Kim, J. M., Park, J. E., Baek, S. J., & Yang, S. N. (2023) · Scientific Reports · doi:10.1038/s41598-023-44802-3 · Cited by 31
Lee, K.-S., Lee, E., Choi, B., & Pyun, S.-B. (2021) · Diagnostics · doi:10.3390/diagnostics11020300 · Cited by 25
Deep Learning for Automatic Hyoid Tracking in Videofluoroscopic Swallow Studies
Hsiao, M., Weng, C.-H., Wang, Y.-C., Cheng, S.-H., Wei, K., Tung, P.-Y., Chen, J., Yeh, C., & Wang, T. (2022) · Dysphagia · doi:10.1007/s00455-022-10438-0 · Cited by 27
Detection of aspiration from images of a videofluoroscopic swallowing study adopting deep learning
Iida, Y., Näppi, J. J., Kitano, T., Hironaka, T., Katsumata, A., & Yoshida, H. (2023) · Oral Radiology · doi:10.1007/s11282-023-00669-8 · Cited by 22
Bandini, A., Smaoui, S., & Steele, C. M. (2022) · Computer Methods and Programs in Biomedicine · doi:10.1016/j.cmpb.2022.107058 · Cited by 16
“Likewise, temporal analysis of VFSS has benefited from the development of powerful 3DCNNs [38] that allowed detection of the pharyngeal phase in VFSS recordings [12], [13].”
BackgroundMethodInfluentialShu, K., Mao, S., Zhang, Z., Coyle, J. L., & Sejdić, E. (2024) · Computer Methods and Programs in Biomedicine · doi:10.1016/j.cmpb.2024.108505 · Cited by 16
Artificial Intelligence in Videofluoroscopy Swallow Study Analysis: A Comprehensive Review
Sanjeevi, G., Gopalakrishnan, U., Pathinarupothi, R. K., & Iyer, K. S. (2025) · Dysphagia · doi:10.1007/s00455-025-10812-8 · Cited by 12
A Deep Learning Approach to Video Fluoroscopic Swallowing Exam Classification
Wilhelm, P., Reinhardt, J. M., & Daele, D. J. V. (2020) · IEEE International Symposium on Biomedical Imaging (ISBI) · doi:10.1109/isbi45749.2020.9098510 · Cited by 12
Video-SwinUNet: Spatio-temporal Deep Learning Framework for VFSS Instance Segmentation
Zeng, C., Yang, X., Smithard, D. G., Mirmehdi, M., Gambaruto, A. M., & Burghardt, T. (2023) · Proceedings - International Conference on Image Processing · doi:10.1109/icip49359.2023.10222212 · Cited by 14
Reddy, C. S., Park, E., & Lee, J. T. (2023) · IEEE Access · doi:10.1109/access.2023.3315342 · Cited by 11
Self-citationMachine learning in the evaluation of voice and swallowing in the head and neck cancer patient
Srinivasan, Y., Liu, A. Y., & Rameau, A. (2023) · Current Opinion in Otolaryngology & Head & Neck Surgery · doi:10.1097/moo.0000000000000948 · Cited by 10
Temporal Micro-Action Localization for Videofluoroscopic Swallowing Study
Ruan, X., Dai, M., Chen, Z., You, Z., Zhang, Y., Li, Y., Dou, Z., & Tan, M. (2023) · IEEE Journal of Biomedical and Health Informatics · doi:10.1109/jbhi.2023.3313255 · Cited by 10
Bandini, A., & Steele, C. M. (2021) · Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC) · doi:10.1109/embc46164.2021.9629562 · Cited by 10
Fakhry, A., Antony, S. M., Park, E., & Lee, J. T. (2025) · IEEE Access · doi:10.1109/access.2025.3573282 · Cited by 8
Self-citationFujinaka, A., Mekata, K., Takizawa, H., & Kudo, H. (2020) · International Journal of Computer Assisted Radiology and Surgery · doi:10.1007/s11548-020-02145-8 · Cited by 5
Unsupervised Action Proposals Using Support Vector Classifiers for Online Video Processing
Baptista-Ríos, M., López-Sastre, R. J., Acevedo-Rodríguez, F. J., Martin-Martin, P., & Maldonado-Bascón, S. (2020) · Sensors · doi:10.3390/s20102953 · Cited by 3
Rapid Detection of Penetration-Aspiration from Fluoroscopic Videos in Dysphagia Patients
Sanjeevi, G., Gopalakrishnan, U., Pathinarupothi, R. K., Arya, C. J., & Subramania Iyer, K. (2023) · IEEE International Conference on Bioinformatics and Bioengineering (BIBE) · doi:10.1109/bibe60311.2023.00073 · Cited by 2
Artificial Intelligence in Laryngology, Broncho-Esophagology, and Sleep Surgery
Nwosu, O. I., & Naunheim, M. R. (2024) · Otolaryngologic Clinics of North America · doi:10.1016/j.otc.2024.04.002 · Cited by 1
Jeong, C., Lim, D.-W., Noh, S.-H., Moon, H., Park, C., Ko, N., & Kim, M. (2025) · Diagnostics · doi:10.3390/diagnostics16010045 · Cited by 1
Dynamic Recognition and Tracking of Barium Flow Field Based on Deglutition Video
Qin, G., Zou, J., Xia, Q., & Qin, J. (2021) · Journal of Web Engineering · doi:10.13052/jwe1540-9589.20214 · Cited by 1
Video-SwinUNet: Spatio-temporal Deep Learning Framework for VFSS Instance SegmentationPreprint
Zeng, C., Yang, X., Smithard, D. G., Mirmehdi, M., Gambaruto, A. M., & Burghardt, T. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2302.11325 · Cited by 3
Luo, S., Takizawa, H., Mekata, K., & Kudo, H. (2023) · International Conference on Intelligent Systems and Image Processing (ICISIP) · doi:10.12792/icisip2023.044
da Rocha, M. M. L., van der Molen, L., Neijman, M., van Alphen, M. J. A., van den Brekel, M. W. M., & Siepel, F. J. (2025) · International Journal of Computer Assisted Radiology and Surgery · doi:10.1007/s11548-025-03556-1 · Cited by 1
Chuang, W.-K., Lin, B., Lee, Y., Su, P.-H., Kao, Y., & Lu, C.-F. (2026) · Archives of Physical Medicine and Rehabilitation · doi:10.1016/j.apmr.2026.03.016
Yuan, Z., Qian, S., Wang, Z., Wu, D., Liu, L., Dong, Y., Chen, Y., & Xue, C. (2026) · Journal of Imaging Informatics in Medicine · doi:10.1007/s10278-026-02324-0
Metadata
- Authors
- Lee, J. T., Park, E., & Jung, T.-D.
- Affiliations
- Electronics and Telecommunications Research Institute (KR)
- Kyungpook National University (KR)
- Corresponding author
- Park, Eunhee
- Venue
- Sensors
- Publisher
- MDPI AG
- Published
- 2019-09-07
- Volume
- 19
- Issue
- 18
- Pages
- 3873
- ISSN
- 1424-8220 (ISSN-L)
- PMC
- PMC6767274
- OpenAlex
- W2971813254
- Cited by
- 30
- References
- 34
- Citations by year
- 2026: 2 · 2025: 4 · 2024: 2 · 2023: 10 · 2022: 3 · 2021: 4 · 2020: 5
- FWCI
- 5.07
- Type
- journal article
- Language
- en
- Open access
- gold
- dx.doi.org
- Licence
- Locations
- Sensors (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Funders
- Biomedical Research Institute grant, Kyungpook National University Hospital (2018) (NA)
- Topics
- Health Sciences › Health Professions › Speech and Hearing › Dysphagia Assessment and Management
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Human Pose and Action Recognition
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
- Keywords
- videofluoroscopic swallowing study, dysphagia, aspiration detection, 3D convolutional networks, I3D, training data collection
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.