CoChat

Hybrid LSTM-Transformer Model for Emotion Recognition From Speech Audio Files

Open access
Andayani, Felicia, Theng, Lau Bee, Tsun, Mark Teekit, Chua, Caslon · 2022 · IEEE Access
Cited by 178

The DOI matches its Crossref record; 3 of the 3 references it cites were found in a registry or an index.

Abstract

Emotion is a vital component in daily human communication and it helps people understand each other. Emotion recognition plays a crucial role in developing human-computer interaction and computer-based speech emotion recognition. In a nutshell, Speech Emotion Recognition (SER) recognizes emotion signals transmitted through human speech or daily conversation where the emotions in a speech strongly depend on temporal information. Despite the fact that much existing research showed that a hybrid system performs better than traditional single classifiers used in SER, there are some limitations in each of them. As a result, this paper discussed a proposed hybrid Long Short-Term Memory (LSTM) Network and Transformer Encoder to learn the long-term dependencies in speech signals and classify emotions. Speech features are extracted with Mel Frequency Cepstral Coefficient (MFCC) and fed into the proposed hybrid LSTM-Transformer classifier. A range of performance evaluations was conducted on the proposed LSTM-Transformer model. The results indicate that it achieves a significant recognition improvement compared with existing models offered by other published works. The proposed hybrid model reached 75.62%, 85.55%, and 72.49% recognition success with the RAVDESS, Emo-DB, and language-independent datasets.

Overview

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1109/access.2022.3163856

Cites

3 of 3 references verified0 citation errors · 0 not checkable

Statistics

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Show all 3 references

Research works printed but not linked: 29

  1. 10.14569/IJACSA.2021.0120422 · doi:10.14569/ijacsa.2021.0120422
  2. 10.1016/j.procs.2021.01.098 · doi:10.1016/j.procs.2021.01.098
  3. 10.1016/j.apacoust.2021.108046 · DOI · Paper page
  4. 10.1109/ICSIMA50015.2021.9526323 · DOI
  5. tanikawa, 2020, Text Generation With LSTM+Transformer Model (Japanese)
  6. sakatani, 2021, Combining RNN with transformer for modeling multi-leg trips, Proc ACM WSDM WebTour, 50
  7. mcfee, 2020, Librosa/librosa
  8. 10.1371/journal.pone.0196391 · DOI · Paper page
  9. 10.21437/Interspeech.2019-2753 · DOI
  10. md jalal, 2019, Learning temporal clusters using capsule routing for speech emotion recognition, Proc INTERSPEECH, 1701
  11. 10.1007/s11042-017-5539-3 · DOI · Paper page
  12. huang, 2020, TRANS-BLSTM: Transformer with bidirectional LSTM for language understanding, arXiv 2003 07000
  13. 10.1109/ICTC46691.2019.8939830 · DOI
  14. 10.5120/17201-7409 · doi:10.5120/17201-7409
  15. lech, 2020, Real-time speech emotion recognition using a pre-trained image classification network: Effects of bandwidth reduction and companding, Frontiers Comput Sci, 2, 14, 10.3389/fcomp.2020.00014 · DOI
  16. 10.1109/TASLP.2019.2925934 · DOI · Paper page
  17. 10.1109/ICASSP40776.2020.9053581 · DOI
  18. 10.1109/ICASSP.2018.8462677 · DOI · Paper page
  19. heusser, 2019, Bimodal speech emotion recognition using pre-trained language models, arXiv 1912 02610
  20. vaswani, 2017, Attention is all you need, arXiv 1706 03762
  21. 10.1109/ATSIP.2017.8075575 · DOI
  22. 10.3390/s20226688 · DOI
  23. pedregosa, 2012, Scikit-learn: Machine learning in Python, J Mach Learn Res, 12, 2825
  24. kerkeni, 2019, Automatic speech emotion recognition using machine learning, Proc Social Media Mach Learn, 1
  25. 10.5220/0006611601750182 · DOI
  26. craft, 2018, Dialectical Behavior Therapy Control Your Emotions Overcome Mood Swings and Balance Your Life With DBT
  27. jing, 2021, Transformer-like model with linear attention for speech emotion recognition, J Southeast Univ, 37, 164
  28. 10.1145/3382507.3417964 · DOI
  29. huang, 2020, Speech emotion recognition based on three-chanel feature fusion of CNN and BiLSTM, Proc ICIAI, 52

Cited by

See all 178

Metadata

Authors
Andayani, F., Theng, L. B., Tsun, M. T., & Chua, C.
Venue
IEEE Access
Published
2022
Volume
10
Pages
36018-36027
ISSN
2169-3536 (ISSN-L)
OpenAlex
W4226153164
Cited by
178
References
3
FWCI
30.64
Type
journal article
Language
en
Open access
gold
Funders
Swinburne University of Technology Sarawak Higher Degree by Research (HDR) Support Fund

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: Hybrid LSTM-Transformer Model for Emotion Recognition From Speech Audio Files. https://app.cochat.ai/papers/10.1109/access.2022.3163856
The paper
Andayani, F., Theng, L. B., Tsun, M. T., & Chua, C. (2022). Hybrid LSTM-Transformer Model for Emotion Recognition From Speech Audio Files. IEEE Access. https://doi.org/10.1109/access.2022.3163856

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.