Hybrid LSTM-Transformer Model for Emotion Recognition From Speech Audio Files
Open accessThe DOI matches its Crossref record; 3 of the 3 references it cites were found in a registry or an index.
Abstract
Emotion is a vital component in daily human communication and it helps people understand each other. Emotion recognition plays a crucial role in developing human-computer interaction and computer-based speech emotion recognition. In a nutshell, Speech Emotion Recognition (SER) recognizes emotion signals transmitted through human speech or daily conversation where the emotions in a speech strongly depend on temporal information. Despite the fact that much existing research showed that a hybrid system performs better than traditional single classifiers used in SER, there are some limitations in each of them. As a result, this paper discussed a proposed hybrid Long Short-Term Memory (LSTM) Network and Transformer Encoder to learn the long-term dependencies in speech signals and classify emotions. Speech features are extracted with Mel Frequency Cepstral Coefficient (MFCC) and fed into the proposed hybrid LSTM-Transformer classifier. A range of performance evaluations was conducted on the proposed LSTM-Transformer model. The results indicate that it achieves a significant recognition improvement compared with existing models offered by other published works. The proposed hybrid model reached 75.62%, 85.55%, and 72.49% recognition success with the RAVDESS, Emo-DB, and language-independent datasets.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Transformer-XL: Attentive language models beyond a fixed-length context
Dai, Z., Yang, Z., Yang, Y., Carbonell, J., Le, Q., & Salakhutdinov, R. (2019) · Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/p19-1285 · Cited by 3,169 · cited 3×
“Moreover, LSTM has been developed to solve the short-term memory problem; however, the LSTM alone is insufficient to solve the longer-term dependency problem [15].”
BackgroundA database of German emotional speech
Burkhardt, F., Paeschke, A., Rolfes, M., Sendlmeier, W. F., & Weiss, B. (2005) · Interspeech 2005 · doi:10.21437/interspeech.2005-446 · Cited by 2,365 · cited 1×
“Another dataset used was the Emo-DB [14].”
Class-level spectral features for emotion recognition
Bitouk, D., Verma, R., & Nenkova, A. (2010) · Speech Communication · doi:10.1016/j.specom.2010.02.010 · Cited by 219 · cited 1×
“Besides, the mean values of MFCCs were calculated to have a fixed-length feature vector [28].”
Research works printed but not linked: 29
- 10.14569/IJACSA.2021.0120422 · doi:10.14569/ijacsa.2021.0120422
- 10.1016/j.procs.2021.01.098 · doi:10.1016/j.procs.2021.01.098
- 10.1016/j.apacoust.2021.108046 · DOI · Paper page
- 10.1109/ICSIMA50015.2021.9526323 · DOI
- tanikawa, 2020, Text Generation With LSTM+Transformer Model (Japanese)
- sakatani, 2021, Combining RNN with transformer for modeling multi-leg trips, Proc ACM WSDM WebTour, 50
- mcfee, 2020, Librosa/librosa
- 10.1371/journal.pone.0196391 · DOI · Paper page
- 10.21437/Interspeech.2019-2753 · DOI
- md jalal, 2019, Learning temporal clusters using capsule routing for speech emotion recognition, Proc INTERSPEECH, 1701
- 10.1007/s11042-017-5539-3 · DOI · Paper page
- huang, 2020, TRANS-BLSTM: Transformer with bidirectional LSTM for language understanding, arXiv 2003 07000
- 10.1109/ICTC46691.2019.8939830 · DOI
- 10.5120/17201-7409 · doi:10.5120/17201-7409
- lech, 2020, Real-time speech emotion recognition using a pre-trained image classification network: Effects of bandwidth reduction and companding, Frontiers Comput Sci, 2, 14, 10.3389/fcomp.2020.00014 · DOI
- 10.1109/TASLP.2019.2925934 · DOI · Paper page
- 10.1109/ICASSP40776.2020.9053581 · DOI
- 10.1109/ICASSP.2018.8462677 · DOI · Paper page
- heusser, 2019, Bimodal speech emotion recognition using pre-trained language models, arXiv 1912 02610
- vaswani, 2017, Attention is all you need, arXiv 1706 03762
- 10.1109/ATSIP.2017.8075575 · DOI
- 10.3390/s20226688 · DOI
- pedregosa, 2012, Scikit-learn: Machine learning in Python, J Mach Learn Res, 12, 2825
- kerkeni, 2019, Automatic speech emotion recognition using machine learning, Proc Social Media Mach Learn, 1
- 10.5220/0006611601750182 · DOI
- craft, 2018, Dialectical Behavior Therapy Control Your Emotions Overcome Mood Swings and Balance Your Life With DBT
- jing, 2021, Transformer-like model with linear attention for speech emotion recognition, J Southeast Univ, 37, 164
- 10.1145/3382507.3417964 · DOI
- huang, 2020, Speech emotion recognition based on three-chanel feature fusion of CNN and BiLSTM, Proc ICIAI, 52
Cited by
Multimodal Emotion Recognition Based on Facial Expressions, Speech, and EEG
Pan, J., Fang, W., Zhang, Z., Chen, B., Zhang, Z., & Wang, S. (2024) · IEEE Open Journal of Engineering in Medicine and Biology · doi:10.1109/ojemb.2023.3240280 · Cited by 121
Metadata
- Authors
- Andayani, F., Theng, L. B., Tsun, M. T., & Chua, C.
- Venue
- IEEE Access
- Published
- 2022
- Volume
- 10
- Pages
- 36018-36027
- ISSN
- 2169-3536 (ISSN-L)
- OpenAlex
- W4226153164
- Cited by
- 178
- References
- 3
- FWCI
- 30.64
- Type
- journal article
- Language
- en
- Open access
- gold
- Funders
- Swinburne University of Technology Sarawak Higher Degree by Research (HDR) Support Fund
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Hybrid LSTM-Transformer Model for Emotion Recognition From Speech Audio Files. https://app.cochat.ai/papers/10.1109/access.2022.3163856
- The paper
- Andayani, F., Theng, L. B., Tsun, M. T., & Chua, C. (2022). Hybrid LSTM-Transformer Model for Emotion Recognition From Speech Audio Files. IEEE Access. https://doi.org/10.1109/access.2022.3163856
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.