Transformer Models and Convolutional Networks with Different Activation Functions for Swallow Classification Using Depth Video Data
Open accessThe DOI matches its Crossref record; 76 of the 76 references it cites were found in a registry or an index.
Abstract
Dysphagia is a common geriatric syndrome that might induce serious complications and death. Standard diagnostics using the Videofluoroscopic Swallowing Study (VFSS) or Fiberoptic Evaluation of Swallowing (FEES) are expensive and expose patients to risks, while bedside screening is subjective and might lack reliability. An affordable and accessible instrumented screening is necessary. This study aimed to evaluate the classification performance of Transformer models and convolutional networks in identifying swallowing and non-swallowing tasks through depth video data. Different activation functions (ReLU, LeakyReLU, GELU, ELU, SiLU, and GLU) were then evaluated on the best-performing model. Sixty-five healthy participants (n = 65) were invited to perform swallowing (eating a cracker and drinking water) and non-swallowing tasks (a deep breath and pronouncing vowels: “/eɪ/”, “/iː/”, “/aɪ/”, “/oʊ/”, “/u:/”). Swallowing and non-swallowing were classified by Transformer models (TimeSFormer, Video Vision Transformer (ViViT)), and convolutional neural networks (SlowFast, X3D, and R(2+1)D), respectively. In general, convolutional neural networks outperformed the Transformer models. X3D was the best model with good-to-excellent performance (F1-score: 0.920; adjusted F1-score: 0.885) in classifying swallowing and non-swallowing conditions. Moreover, X3D with its default activation function (ReLU) produced the best results, although LeakyReLU performed better in deep breathing and pronouncing “/aɪ/” tasks. Future studies shall consider collecting more data for pretraining and developing a hyperparameter tuning strategy for activation functions and the high dimensionality video data for Transformer models.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“with stroke being the most cited in 12 studies(16,18,21,22,27,32,36,40,46,48-51)”
From Artificial intelligence in the diagnosis and management of dysphagia: a scoping review (2025) · cites this paper as [18]
Citations
Cites
Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification
He, K., Zhang, X., Ren, S., & Sun, J. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.123 · Cited by 19,741
Learning long-term dependencies with gradient descent is difficult
Bengio, Y., Simard, P., & Frasconi, P. (1994) · IEEE Transactions on Neural Networks · doi:10.1109/72.279181 · Cited by 8,542
Saito, T., & Rehmsmeier, M. (2015) · PLOS ONE · doi:10.1371/journal.pone.0118432 · Cited by 5,309
SlowFast Networks for Video Recognition
Feichtenhofer, C., Fan, H., Malik, J., & He, K. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00630 · Cited by 4,007
Temporal Segment Networks: Towards Good Practices for Deep Action Recognition
Wang, L., Xiong, Y., Wang, Z., Qiao, Y., Lin, D., Tang, X., & Van Gool, L. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-46484-8_2 · Cited by 3,958
A Closer Look at Spatiotemporal Convolutions for Action Recognition
Tran, D., Wang, H., Torresani, L., Ray, J., LeCun, Y., & Paluri, M. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00675 · Cited by 3,516
ViViT: A Video Vision Transformer
Arnab, A., Dehghani, M., Heigold, G., Sun, C., Lucic, M., & Schmid, C. (2021) · 2021 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00676 · Cited by 2,767
Sigmoid-Weighted Linear Units for Neural Network Function Approximation in Reinforcement Learning
Elfwing, S., Uchibe, E., & Doya, K. (2018) · Neural Networks · doi:10.1016/j.neunet.2017.12.012 · Cited by 2,119
Activation functions in deep learning: a comprehensive survey and benchmark
Dubey, S. R., Singh, S. K., & Chaudhuri, B. B. (2022) · Neurocomputing · doi:10.1016/j.neucom.2022.06.111 · Cited by 1,259
X3D: Expanding Architectures for Efficient Video Recognition
Feichtenhofer, C. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.00028 · Cited by 1,233
Research works printed but not linked: 17
- Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, L., and Polosukhin, I. (2017). Attention is All You Need. arXiv.
- Nwankpa, C., Ijomah, W., Gachagan, A., and Marshall, S. (2018). Activation Functions: Comparison of trends in Practice and Research for Deep Learning. arXiv.
- Hendrycks, D., and Gimpel, K. (2016). Gaussian Error Linear Units (GELUs). arXiv.
- Clevert, D.-A., Unterthiner, T., and Hochreiter, S. (2015). Fast and Accurate Deep Network Learning by Exponential Linear Units (ELUs). arXiv.
- Dauphin, Y.N., Fan, A., Auli, M., and Grangier, D. (2016). Language Modeling with Gated Convolutional Networks. arXiv.
- Bertasius, G., Wang, H., and Torresani, L. (2021). Is Space-Time Attention All You Need for Video Understanding?. arXiv.
- Paszke, A., Gross, S., Massa, F., Lerer, A., Bradbury, J., Chanan, G., Killeen, T., Lin, Z., Gimelshein, N., and Antiga, L. (2019). PyTorch: An Imperative Style, High-Performance Deep Learning Library. arXiv.
- Park, N., and Kim, S. (2022). How do vision transformers work?. arXiv.
- Yadav, 2018, Dynamic time warping (dtw) algorithm in speech: A review, Int. J. Res. Electron. Comput. Eng., 6, 524
- Dubbs, A. (2021). Test Set Sizing Via Random Matrix Theory. arXiv.
- Afendras, G., and Markatou, M. (2015). Optimality of Training/Test Size and Resampling Effectiveness of Cross-Validation Estimators of the Generalization Error. arXiv.
- Marchisio, A., Abdullah Hanif, M., Rehman, S., Martina, M., and Shafique, M. (2018). A Methodology for Automatic Selection of Activation Functions to Design Hybrid Deep Neural Networks. arXiv.
- Basirat, M., and Roth, P.M. (2018). The Quest for the Golden Activation Function. arXiv.
- Hayou, S., Doucet, A., and Rousseau, J. (2018). On the Selection of Initialization and Activation Function for Deep Neural Networks. arXiv.
- Sinha, S., Ohashi, H., and Nakamura, K. (2020). Class-wise difficulty-balanced loss for solving class-imbalance. arXiv.
- Abraham, 2013, A review of class imbalance problem, J. Netw. Innov. Comput., 1, 332
- Chawla, N.V., Bowyer, K.W., Hall, L.O., and Kegelmeyer, W.P. (2011). SMOTE: Synthetic Minority Over-sampling Technique. arXiv.
Cited by
Three Decades of Activations: A Comprehensive Survey of 400 Activation Functions for Neural NetworksPreprint
Kunc, V., & Kléma, J. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2402.09092 · Cited by 104
Current Technological Advances in Dysphagia Screening: Systematic Scoping Review
Wong, D. W., Wang, J., Cheung, S. M.-Y., Lai, D. K.-H., Chiu, A. T. S., Pu, D., Cheung, J. C., & Kwok, T. C. Y. (2025) · Journal of Medical Internet Research · doi:10.2196/65551 · Cited by 22
Self-citationWang, J., Zheng, Y., Luo, J., Lee, T. T., Li, P., Zhang, Y., Cheung, J. C., Wong, D. W., & Ni, M. (2024) · IEEE Access · doi:10.1109/access.2024.3389669 · Cited by 16
Self-citationArtificial intelligence in the diagnosis and management of dysphagia: a scoping review
Silva, R. D. D., Almeida, S. B., Gonçalves, F. M., Zeigelboim, B. S., Stechman-Neto, J., Schroder, A. G. D., Nascimento, W. V., Santos, R. S., & Araujo, C. M. D. (2025) · CoDAS · doi:10.1590/2317-1782/e20240305en · Cited by 6
“Literature search flowchart and selection criteria cited in 12 studies (16,18,21,22,27,32,36,40,46,48-51) , neurodegenerative diseases like Parkinson’s were present in 3 studies (22,24,27) , and two studies mentioned esophageal alterations (42,52) .”
Tsujimoto, M., Hisajima, T., Matsuda, S., Tanaka, S., Suzuki, K., Shimokakimoto, T., & Toyama, Y. (2024) · Digital Health · doi:10.1177/20552076241264640 · Cited by 5
Li, Y., Wu, Y., Tang, Q., Lin, H., Fu, J., Yang, Y., & Zhang, F. (2025) · Biomedizinische Technik/Biomedical Engineering · doi:10.1515/bmt-2025-0028 · Cited by 2
Tsujimoto, M., Hisajima, T., Matsuda, S., Suzuki, K., Shimokakimoto, T., & Toyama, Y. (2025) · Journal of Aging Research and Lifestyle · doi:10.1016/j.jarlif.2025.100038 · Cited by 2
Rathore, R., Lakdawala, H., Zade, N., Vora, D. R., Gupte, A. A., & Patil, S. (2026) · MethodsX · doi:10.1016/j.mex.2026.103808 · Cited by 2
A Human Tissue Complex Impedance Measurement System for Swallowing Action Recognition
Liu, B., Teng, Z.-S., Tang, Q., Deng, T.-Y., Lan, H.-Q., & Zhong, H. (2025) · IEEE Transactions on Instrumentation and Measurement · doi:10.1109/tim.2025.3541660 · Cited by 2
Zhang, Y., Tian, M., & Liu, B. (2025) · Frontiers in Neuroinformatics · doi:10.3389/fninf.2025.1526259 · Cited by 2
Harrison, H., Mamat, M. B., Wong, F., Yew, H. T., Lim, R., & Zaki, W. M. D. W. (2025) · Journal of Vibroengineering · doi:10.21595/jve.2025.25002 · Cited by 1
Energy Saving Based on Transformer Models with LeakyReLU Activation Function
Wang, J., Li, X., & Wang, J. (2023) · International Conference on Information Science and Technology · doi:10.1109/icist59754.2023.10367091 · Cited by 1
Chu, X., Yu, S., Zhang, F., Yang, Y., Fu, L., & Liu, Q. (2024) · IEEE Sensors Journal · doi:10.1109/jsen.2024.3519559
28 Diagnosis and Differential Diagnosis of Dysphagia
Angerstein, W., Farahat, M., Farneti, D., Graf, S., Jungheim, M., Kiagiadaki, D. E., Malki, K. H., Moerman, M., Müller, F., Nienstedt, J. C., et al. (2025) · European manual of medicine · doi:10.1007/978-3-031-48091-1_13
Lai, D. K.-H., Zhao, Z.-A., Tam, A. Y.-C., Li, J., Zhang, J. Z.-S., Wong, D. W., & Cheung, J. C. (2025) · AI · doi:10.3390/ai6110276
Self-citationChuang, W.-K., Lin, B., Lee, Y., Su, P.-H., Kao, Y., & Lu, C.-F. (2026) · Archives of Physical Medicine and Rehabilitation · doi:10.1016/j.apmr.2026.03.016
Manasa, N., S, A. G. A. S. G., & Bindu, N. M. (2026) · International Conference on Interactive Systems & Information Society Technologies (InterSys) · doi:10.1109/intersys69825.2026.11565265
Yang, R., & Wu, J. (2026) · Journal of Hydrology Regional Studies · doi:10.1016/j.ejrh.2026.103703
MMVX-Net: Multispectral Mamba Representation Learning for Dysphagia ScreeningPreprint
Lai, D. K.-H., Li, J., Po, M.-Y., Chiu, A. T. S., & Cheung, J. C. (2026) · Research Square · doi:10.21203/rs.3.rs-11262558/v1
Self-citation
Metadata
- Authors
- Lai, D. K.-H., Cheng, E. S.-W., So, B. P.-H., Mao, Y.-J., Cheung, S. M.-Y., Cheung, D. S. K., Wong, D. W.-C., & Cheung, J. C.-W.
- Affiliations
- Hong Kong Polytechnic University (HK)
- Hong Kong University of Science and Technology (HK)
- Corresponding author
- Wong, Duo Wai-Chi
- Cheung, James Chung-Wai
- ORCID
- Venue
- Mathematics
- Publisher
- MDPI AG
- Published
- 2023-07-12
- Volume
- 11
- Issue
- 14
- Pages
- 3081
- ISSN
- 2227-7390 (ISSN-L)
- OpenAlex
- W4384130932
- Cited by
- 18
- References
- 75
- Citations by year
- 2026: 5 · 2025: 9 · 2024: 3 · 2023: 1
- FWCI
- 6.60
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- Locations
- Mathematics (publishedVersion, cc-by)
- PolyU Institutional Research Archive (Hong Kong Polytechnic University) (submittedVersion, cc-by)
- RePEc: Research Papers in Economics (submittedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Figshare (submittedVersion, cc-by)
- Rare & Special e-Zone (The Hong Kong University of Science and Technology) (submittedVersion)
- Funders
- Health Bureau, Hong Kong (19200461)
- Research Institute for Smart Ageing, The Hong Kong Polytechnic University (19200461)
- Topics
- Health Sciences › Health Professions › Speech and Hearing › Dysphagia Assessment and Management
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Human Pose and Action Recognition
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
- Keywords
- dysphagia, depth video, activation functions, transformer models, convolutional neural networks, X3D, SlowFast, ViViT, TimeSformer, videofluoroscopic swallowing study
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.