CoChat

Transformer Models and Convolutional Networks with Different Activation Functions for Swallow Classification Using Depth Video Data

Open access

The DOI matches its Crossref record; 76 of the 76 references it cites were found in a registry or an index.

Abstract

Dysphagia is a common geriatric syndrome that might induce serious complications and death. Standard diagnostics using the Videofluoroscopic Swallowing Study (VFSS) or Fiberoptic Evaluation of Swallowing (FEES) are expensive and expose patients to risks, while bedside screening is subjective and might lack reliability. An affordable and accessible instrumented screening is necessary. This study aimed to evaluate the classification performance of Transformer models and convolutional networks in identifying swallowing and non-swallowing tasks through depth video data. Different activation functions (ReLU, LeakyReLU, GELU, ELU, SiLU, and GLU) were then evaluated on the best-performing model. Sixty-five healthy participants (n = 65) were invited to perform swallowing (eating a cracker and drinking water) and non-swallowing tasks (a deep breath and pronouncing vowels: “/eɪ/”, “/iː/”, “/aɪ/”, “/oʊ/”, “/u:/”). Swallowing and non-swallowing were classified by Transformer models (TimeSFormer, Video Vision Transformer (ViViT)), and convolutional neural networks (SlowFast, X3D, and R(2+1)D), respectively. In general, convolutional neural networks outperformed the Transformer models. X3D was the best model with good-to-excellent performance (F1-score: 0.920; adjusted F1-score: 0.885) in classifying swallowing and non-swallowing conditions. Moreover, X3D with its default activation function (ReLU) produced the best results, although LeakyReLU performed better in deep breathing and pronouncing “/aɪ/” tasks. Future studies shall consider collecting more data for pretraining and developing a hyperparameter tuning strategy for activation functions and the high dimensionality video data for Transformer models.

Overview

Integrity signals

Registry

Crossref · 10.3390/math11143081

Cites

76 of 76 references verified0 citation errors · 0 not checkable

Statistics

Recomputing p-values needs the paper’s full text.

Verification

Claims the paper makes

We couldn’t retrieve this paper’s full text, so its claims and self-check weren’t run. The registry and citation checks still apply.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Citations

Cites

Research works printed but not linked: 17

  1. Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, L., and Polosukhin, I. (2017). Attention is All You Need. arXiv.
  2. Nwankpa, C., Ijomah, W., Gachagan, A., and Marshall, S. (2018). Activation Functions: Comparison of trends in Practice and Research for Deep Learning. arXiv.
  3. Hendrycks, D., and Gimpel, K. (2016). Gaussian Error Linear Units (GELUs). arXiv.
  4. Clevert, D.-A., Unterthiner, T., and Hochreiter, S. (2015). Fast and Accurate Deep Network Learning by Exponential Linear Units (ELUs). arXiv.
  5. Dauphin, Y.N., Fan, A., Auli, M., and Grangier, D. (2016). Language Modeling with Gated Convolutional Networks. arXiv.
  6. Bertasius, G., Wang, H., and Torresani, L. (2021). Is Space-Time Attention All You Need for Video Understanding?. arXiv.
  7. Paszke, A., Gross, S., Massa, F., Lerer, A., Bradbury, J., Chanan, G., Killeen, T., Lin, Z., Gimelshein, N., and Antiga, L. (2019). PyTorch: An Imperative Style, High-Performance Deep Learning Library. arXiv.
  8. Park, N., and Kim, S. (2022). How do vision transformers work?. arXiv.
  9. Yadav, 2018, Dynamic time warping (dtw) algorithm in speech: A review, Int. J. Res. Electron. Comput. Eng., 6, 524
  10. Dubbs, A. (2021). Test Set Sizing Via Random Matrix Theory. arXiv.
  11. Afendras, G., and Markatou, M. (2015). Optimality of Training/Test Size and Resampling Effectiveness of Cross-Validation Estimators of the Generalization Error. arXiv.
  12. Marchisio, A., Abdullah Hanif, M., Rehman, S., Martina, M., and Shafique, M. (2018). A Methodology for Automatic Selection of Activation Functions to Design Hybrid Deep Neural Networks. arXiv.
  13. Basirat, M., and Roth, P.M. (2018). The Quest for the Golden Activation Function. arXiv.
  14. Hayou, S., Doucet, A., and Rousseau, J. (2018). On the Selection of Initialization and Activation Function for Deep Neural Networks. arXiv.
  15. Sinha, S., Ohashi, H., and Nakamura, K. (2020). Class-wise difficulty-balanced loss for solving class-imbalance. arXiv.
  16. Abraham, 2013, A review of class imbalance problem, J. Netw. Innov. Comput., 1, 332
  17. Chawla, N.V., Bowyer, K.W., Hall, L.O., and Kegelmeyer, W.P. (2011). SMOTE: Synthetic Minority Over-sampling Technique. arXiv.

Cited by

See all 19

Metadata

Authors
Lai, D. K.-H., Cheng, E. S.-W., So, B. P.-H., Mao, Y.-J., Cheung, S. M.-Y., Cheung, D. S. K., Wong, D. W.-C., & Cheung, J. C.-W.
Affiliations
  • Hong Kong Polytechnic University (HK)
  • Hong Kong University of Science and Technology (HK)
Corresponding author
  • Wong, Duo Wai-Chi
  • Cheung, James Chung-Wai
Venue
Mathematics
Publisher
MDPI AG
Published
2023-07-12
Volume
11
Issue
14
Pages
3081
ISSN
2227-7390 (ISSN-L)
OpenAlex
W4384130932
Cited by
18
References
75
Citations by year
2026: 5 · 2025: 9 · 2024: 3 · 2023: 1
FWCI
6.60
Type
journal article
Language
en
Open access
Locations
  • Mathematics (publishedVersion, cc-by)
  • PolyU Institutional Research Archive (Hong Kong Polytechnic University) (submittedVersion, cc-by)
  • RePEc: Research Papers in Economics (submittedVersion)
  • DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
  • Figshare (submittedVersion, cc-by)
  • Rare & Special e-Zone (The Hong Kong University of Science and Technology) (submittedVersion)
Funders
  • Health Bureau, Hong Kong (19200461)
  • Research Institute for Smart Ageing, The Hong Kong Polytechnic University (19200461)
Topics
  • Health Sciences › Health Professions › Speech and Hearing › Dysphagia Assessment and Management
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Human Pose and Action Recognition
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
Keywords
dysphagia, depth video, activation functions, transformer models, convolutional neural networks, X3D, SlowFast, ViViT, TimeSformer, videofluoroscopic swallowing study

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.