Automatic Detection of the Pharyngeal Phase in Raw Videos for the Videofluoroscopic Swallowing Study Using Efficient Data Collection and 3D Convolutional Networks
Open accessThe DOI matches its Crossref record; 34 of the 34 references it cites were found in a registry or an index.
Abstract
Videofluoroscopic swallowing study (VFSS) is a standard diagnostic tool for dysphagia. To detect the presence of aspiration during a swallow, a manual search is commonly used to mark the time intervals of the pharyngeal phase on the corresponding VFSS image. In this study, we present a novel approach that uses 3D convolutional networks to detect the pharyngeal phase in raw VFSS videos without manual annotations. For efficient collection of training data, we propose a cascade framework which no longer requires time intervals of the swallowing process nor the manual marking of anatomical positions for detection. For video classification, we applied the inflated 3D convolutional network (I3D), one of the state-of-the-art network for action classification, as a baseline architecture. We also present a modified 3D convolutional network architecture that is derived from the baseline I3D architecture. The classification and detection performance of these two architectures were evaluated for comparison. The experimental results show that the proposed model outperformed the baseline I3D model in the condition where both models are trained with random weights. We conclude that the proposed method greatly reduces the examination time of the VFSS images with a low miss rate.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Figures






Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“VFSS is the only technique able to show the bolus throughout the three phases of the swallow”
From Automated pharyngeal phase detection and bolus localization in videofluoroscopic swallowing study: killing two birds with one stone? (2022) · cites this paper as [9]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“From an algorithmic perspective, these two problems have always been tackled independently, for example: the pharyngeal phase has usually been detected in a video segmentation fashion using convolutional neural networks (CNNs) [9,14,15], whereas the bolus has been localized via image segmentation algorithms [16].”
From Automated pharyngeal phase detection and bolus localization in videofluoroscopic swallowing study: killing two birds with one stone? (2022) · cites this paper as [9]
Experimental — accuracy not yet measuredIn the citing paper
“Recent results obtained by Lee et al. [9,15] demonstrated that this phase can be detected with accuracy higher than 90%.”
From Automated pharyngeal phase detection and bolus localization in videofluoroscopic swallowing study: killing two birds with one stone? (2022) · cites this paper as [9]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474
Learning Spatiotemporal Features with 3D Convolutional Networks
Tran, D., Bourdev, L., Fergus, R., Torresani, L., & Paluri, M. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.510 · Cited by 9,593
Quo Vadis, Action Recognition? A New Model and the Kinetics Dataset
Carreira, J., & Zisserman, A. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.502 · Cited by 9,570
SGDR: Stochastic Gradient Descent with Warm Restarts
Loshchilov, I., & Hutter, F. (2016) · arXiv · doi:10.48550/arxiv.1608.03983 · Cited by 5,309
Hmdb: a large video database for human motion recognition
Kuehne, H., Jhuang, H., Garrote, E., Poggio, T., & Serre, T. (2011) · 2011 International Conference on Computer Vision · doi:10.1109/iccv.2011.6126543 · Cited by 3,965 · cited 1×
“While the Inflated Inception-V1 architecture achieved a state-of-the-art performance in action classifications such as UCF-101 [34], HMDB-51 [35], and Kinetics [30], we modified the architecture to improve the performance in classification on the VFSS videos as shown in Table 1.”
MethodA Duality Based Approach for Realtime TV-L 1 Optical Flow
Zach, C., Pock, T., & Bischof, H. (2007) · Lecture Notes in Computer Science · doi:10.1007/978-3-540-74936-3_22 · Cited by 1,658 · cited 1×
“First, we applied a TV-L1 optical flow algorithm [31,32] on gray-scale images.”
MethodAspiration Pneumonitis and Aspiration Pneumonia
Marik, P. E. (2001) · New England Journal of Medicine · doi:10.1056/nejm200103013440908 · Cited by 1,627
Spectral–Spatial Classification of Hyperspectral Imagery with 3D Convolutional Neural Network
Li, Y., Zhang, H., & Shen, Q. (2017) · Remote Sensing · doi:10.3390/rs9010067 · Cited by 1,320
Dysphagia: current reality and scope of the problem
Clavé, P., & Shaker, R. (2015) · Nature Reviews Gastroenterology & Hepatology · doi:10.1038/nrgastro.2015.49 · Cited by 577
Ertekin, C., & Aydogdu, I. (2003) · Clinical Neurophysiology · doi:10.1016/s1388-2457(03)00237-2 · Cited by 536
Sánchez Pérez, J., Meinhardt-Llopis, E., & Facciolo, G. (2013) · Image Processing On Line · doi:10.5201/ipol.2013.26 · Cited by 416
Prevalence of Impaired Swallowing in Institutionalized Older People in Taiwan
Lin, L., Wu, S., Chen, H. S., Wang, T., & Chen, M. (2002) · Journal of the American Geriatrics Society · doi:10.1046/j.1532-5415.2002.50270.x · Cited by 247
Oropharyngeal Swallow in Younger and Older Women: Videofluoroscopic Analysis
Logemann, J. A., Pauloski, B. R., Rademaker, A. W., & Kahrilas, P. J. (2002) · Journal of Speech, Language, and Hearing Research · doi:10.1044/1092-4388(2002/034 · Cited by 224
Schmidt, J., Holas, M., Halvorson, K., & Reding, M. (1994) · Dysphagia · doi:10.1007/bf00262752 · Cited by 222
A protocol for the videofluorographic swallowing study
Palmer, J. B., Kuhlemeier, K. V., Tippett, D. C., & Lynch, C. (1993) · Dysphagia · doi:10.1007/bf01354540 · Cited by 206
Change Detection in Hyperspectral Images Using Recurrent 3D Fully Convolutional Networks
Song, A., Choi, J., Han, Y., & Kim, Y. (2018) · Remote Sensing · doi:10.3390/rs10111827 · Cited by 188
Monitoring eating habits using a piezoelectric sensor-based necklace
Kalantarian, H., Alshurafa, N. I., Le, T., & Sarrafzadeh, M. (2015) · Computers in Biology and Medicine · doi:10.1016/j.compbiomed.2015.01.005 · Cited by 169
Oropharyngeal Swallow Efficiency as a Representative Measure of Swallowing Function
Rademaker, A. W., Pauloski, B. R., Logemann, J. A., & Shanahan, T. K. (1994) · Journal of Speech, Language, and Hearing Research · doi:10.1044/jshr.3702.314 · Cited by 167
Inter- and Intrajudge Reliability for Videofluoroscopic Swallowing Evaluation Measures
McCullough, G. H., Wertz, R. T., Rosenbek, J. C., Mills, R. H., Webb, W. G., & Ross, K. (2001) · Dysphagia · doi:10.1007/pl00021291 · Cited by 144
Kinematic and Temporal Factors Associated with Penetration–Aspiration in Swallowing Liquids
Molfenter, S. M., & Steele, C. M. (2014) · Dysphagia · doi:10.1007/s00455-013-9506-5 · Cited by 102
Interactive computer program for biomechanical analysis of videoradiographic studies of swallowing
Logemann, J., Kahrilas, P., Begelman, J., Dodds, W., & Pauloski, B. (1989) · American Journal of Roentgenology · doi:10.2214/ajr.153.2.277 · Cited by 77
Automatic hyoid bone detection in fluoroscopic images using deep learning
Zhang, Z., Coyle, J. L., & Sejdić, E. (2018) · Scientific Reports · doi:10.1038/s41598-018-30182-6 · Cited by 70
Clinical and videofluoroscopic evaluation of swallowing disorders.
Ott, D. J., & Pikna, L. A. (1993) · American Journal of Roentgenology · doi:10.2214/ajr.161.3.8352094 · Cited by 65
Computer-Assisted Assessment of Hyoid Bone Motion from Videofluoroscopic Swallow Studies
Kellen, P. M., Becker, D., Reinhardt, J. M., & Daele, D. J. V. (2009) · Dysphagia · doi:10.1007/s00455-009-9261-9 · Cited by 48
Deep learning for classification of normal swallows in adults
Dudik, J. M., Coyle, J. L., El-Jaroudi, A., Mao, Z.-H., Sun, M., & Sejdić, E. (2018) · Neurocomputing · doi:10.1016/j.neucom.2017.12.059 · Cited by 42
Jardine, M., Miles, A., & Allen, J. E. (2018) · Geriatrics · doi:10.3390/geriatrics3040066 · Cited by 41
Automated Acoustic Analysis in Detection of Spontaneous Swallows in Parkinson’s Disease
Golabbakhsh, M., Rajaei, A., Derakhshan, M., Sadri, S., Taheri, M., & Adibi, P. (2014) · Dysphagia · doi:10.1007/s00455-014-9547-4 · Cited by 33
Shieh, W., Wang, C., Cheng, H. K., & Wang, C.-H. (2019) · Sensors · doi:10.3390/s19112624 · Cited by 32
Kim, D., Cho, H., Shin, H., Lim, S., & Hwang, W. (2019) · Sensors · doi:10.3390/s19163579 · Cited by 29
Image processing in swallowing and speech research
Dengel, G. A., Robbins, J., & Rosenbek, J. C. (1991) · Dysphagia · doi:10.1007/bf02503461 · Cited by 28
Aung, M. H., Goulermas, J. Y., Stanschus, S., Hamdy, S., & Power, M. (2010) · Medical Engineering & Physics · doi:10.1016/j.medengphy.2010.08.008 · Cited by 18
Imtiaz, U., Yamamura, K., Kong, W., Sessa, S., Lin, Z., Bartolomeo, L. D., Ishii, H., Zecca, M., Yamada, Y., & Takanishi, A. (2014) · Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC) · doi:10.1109/embc.2014.6944842 · Cited by 9
Lee, J. T., & Park, E. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-00919-9_38 · Cited by 7 · cited 3×
“In a previous study [29], a system was able to classify whether a short VFSS video clip was in the pharyngeal phase or not.”
BackgroundMethodSelf-citationNoorwali, S. (2013) · Scholarship@Western (Western University) · Cited by 7
Research works printed but not linked: 3
- Leonard, 2017, Two methods for quantifying pharyngeal residue on fluoroscopic swallow studies: Reliability assessment, Ann. Otolaryngol. Rhinol., 4, 1168
- Kay, W., Carreira, J., Simonyan, K., Zhang, B., Hillier, C., Vijayanarasimhan, S., Viola, F., Green, T., Back, T., and Natsev, P. (2017). The kinetics human action video dataset. arXiv.
- Soomro, K., Zamir, A.R., and Shah, M. (2012). UCF101: A dataset of 101 human actions classes from videos in the wild. arXiv.
Cited by
2023 IEEE International Conference on Image Processing (ICIP)
(2023) · doi:10.1109/icip49359.2023 · Cited by 353
Lee, J. T., Park, E., Hwang, J.-M., Jung, T.-D., & Park, D. (2020) · Scientific Reports · doi:10.1038/s41598-020-71713-4 · Cited by 49
Self-citationKim, J. K., Choo, Y. J., Choi, G. S., Shin, H., Chang, M. C., & Park, D. (2022) · Journal of Korean Medical Science · doi:10.3346/jkms.2022.37.e42 · Cited by 48
Kim, H.-I., Kim, Y., Kim, B., Shin, D. Y., Lee, S. J., & Choi, S. (2021) · Diagnostics · doi:10.3390/diagnostics11071147 · Cited by 35
Automatic detection of airway invasion from videofluoroscopy via deep learning technology
Lee, S. J., Ko, J. Y., Kim, H. I., & Choi, S.-I. (2020) · Applied Sciences · doi:10.3390/app10186179 · Cited by 28
Metadata
- Authors
- Lee, J. T., Park, E., & Jung, T.-D.
- Affiliations
- Electronics and Telecommunications Research Institute (KR)
- Kyungpook National University (KR)
- Corresponding author
- Park, Eunhee
- Venue
- Sensors
- Publisher
- MDPI AG
- Published
- 2019-09-07
- Volume
- 19
- Issue
- 18
- Pages
- 3873
- ISSN
- 1424-8220 (ISSN-L)
- PMC
- PMC6767274
- OpenAlex
- W2971813254
- Cited by
- 30
- References
- 34
- Citations by year
- 2026: 2 · 2025: 4 · 2024: 2 · 2023: 10 · 2022: 3 · 2021: 4 · 2020: 5
- FWCI
- 5.07
- Type
- journal article
- Language
- en
- Open access
- gold
- dx.doi.org
- Licence
- Locations
- Sensors (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Funders
- Biomedical Research Institute grant, Kyungpook National University Hospital (2018) (NA)
- Topics
- Health Sciences › Health Professions › Speech and Hearing › Dysphagia Assessment and Management
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Human Pose and Action Recognition
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
- Keywords
- videofluoroscopic swallowing study, dysphagia, aspiration detection, 3D convolutional networks, I3D, training data collection
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.