Clustering-Based Speech Emotion Recognition by Incorporating Learned Features and Deep BiLSTM
Open accessThe DOI matches its Crossref record; 64 of the 64 references it cites were found in a registry or an index.
Abstract
Emotional state recognition of a speaker is a difficult task for machine learning algorithms which plays an important role in the field of speech emotion recognition (SER). SER plays a significant role in many real-time applications such as human behavior assessment, human-robot interaction, virtual reality, and emergency centers to analyze the emotional state of speakers. Previous research in this field is mostly focused on handcrafted features and traditional convolutional neural network (CNN) models used to extract high-level features from speech spectrograms to increase the recognition accuracy and overall model cost complexity. In contrast, we introduce a novel framework for SER using a key sequence segment selection based on redial based function network (RBFN) similarity measurement in clusters. The selected sequence is converted into a spectrogram by applying the STFT algorithm and passed into the CNN model to extract the discriminative and salient features from the speech spectrogram. Furthermore, we normalize the CNN features to ensure precise recognition performance and feed them to the deep bi-directional long short-term memory (BiLSTM) to learn the temporal information for recognizing the final state of emotion. In the proposed technique, we process the key segments instead of the whole utterance to reduce the computational complexity of the overall model and normalize the CNN features before their actual processing, so that it can easily recognize the Spatio-temporal information. The proposed system is evaluated over different standard dataset including IEMOCAP, EMO-DB, and RAVDESS to improve the recognition accuracy and reduce the processing time of the model, respectively. The robustness and effectiveness of the suggested SER model is proved from the experimentations when compared to state-of-the-art SER methods with an achieve up to 72.25%, 85.57%, and 77.02% accuracy over IEMOCAP, EMO-DB, and RAVDESS dataset, respectively.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474
Very Deep Convolutional Networks for Large-Scale Image Recognition
Simonyan, K., & Zisserman, A. (2014) · arXiv · doi:10.48550/arxiv.1409.1556 · Cited by 114,148
ImageNet classification with deep convolutional neural networks
Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017) · Communications of the ACM · doi:10.1145/3065386 · Cited by 112,155
Hochreiter, S., & Schmidhuber, J. (1997) · Neural Computation · doi:10.1162/neco.1997.9.8.1735 · Cited by 102,318
Adam: A Method for Stochastic Optimization
Kingma, D. P., & Ba, J. (2014) · arXiv · doi:10.48550/arxiv.1412.6980 · Cited by 98,946
IEMOCAP: Interactive emotional dyadic motion capture database
Busso, C., Bulut, M., Lee, C.-C., Kazemzadeh, A., Mower, E., Kim, S., Chang, J. N., Lee, S., & Narayanan, S. S. (2008) · Language Resources and Evaluation · doi:10.1007/s10579-008-9076-6 · Cited by 3,869
Revisiting Unreasonable Effectiveness of Data in Deep Learning Era
Sun, C., Shrivastava, A., Singh, S., & Gupta, A. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.97 · Cited by 2,484
A database of German emotional speech
Burkhardt, F., Paeschke, A., Rolfes, M., Sendlmeier, W. F., & Weiss, B. (2005) · Interspeech 2005 · doi:10.21437/interspeech.2005-446 · Cited by 2,365 · cited 2×
“In this paper, we utilize the three public speech emotions datasets, the IEMOCAP [49] interactive emotional dyadic motion capture dataset, Emo-DB [50] berlin emotional dataset, and RAVDESS [51] Ryerson audio-visual dataset of emotional speech and songs.”
MethodLivingstone, S. R., & Russo, F. A. (2018) · PLOS ONE · doi:10.1371/journal.pone.0196391 · Cited by 1,883
Convolutional, Long Short-Term Memory, fully connected Deep Neural Networks
Sainath, T. N., Vinyals, O., Senior, A., & Sak, H. (2015) · 2015 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) · doi:10.1109/icassp.2015.7178838 · Cited by 1,678
Cited by
Items 401–439 of 439
EEG-based emotion recognition: A state-of-the-art review of dataset and method
Hariyady, H., Ibrahim, A. A. A., Teo, J., Suharso, W., Barlaman, M. B. F., Bitaqwa, M. A., Ng, G. W., Ahmad, A., Yassin, F. M., & Salimun, C. (2025) · AIP conference proceedings · doi:10.1063/5.0267598
Real-Time Speech Emotion Recognition
Raisi, K., & Mourhir, A. (2024) · International Conference on Intelligent Computing in Data Sciences (ICDS) · doi:10.1109/icds62089.2024.10756323
Annotation practices in affective computing: What are these algorithms actually trained on?
Backer, S. D. (2023) · Research Repository (Delft University of Technology)
Liao, J. (2025) · Internet Technology Letters · doi:10.1002/itl2.70108
Leveraging Differently Layered Convolutional Neural Network for Speech Emotion Recognition
Kumar, G. P., Iqubal, A., Hasan, M., & Agrawal, A. P. (2025) · Lecture notes in electrical engineering · doi:10.1007/978-981-97-9507-9_3
TIM-GCN Speech Emotion Recognition Network Based on Generative Adversarial Network
Zhang, L., Du, J., Li, J., & Chen, S. (2025) · Lecture notes of the Institute for Computer Sciences, Social Informatics and Telecommunications Engineering · doi:10.1007/978-3-032-00218-1_11
Chen, Y.-L., Ciou, J.-F., Lin, C.-H., & Lien, S.-S. (2024) · Communications in computer and information science · doi:10.1007/978-981-97-9890-2_13
An intelligent classification of emotions in human speech using CNN and LSTM
Nithyashri, J., Goel, H., & Hada, M. (2026) · AIP conference proceedings · doi:10.1063/5.0298091
Exploring Spiking-Vision Transformer Fusion for Speech Emotion Recognition
Shahin, I. M. A., Alsabek, M. B., & Ahmed, A. H. (2025) · Advances in Science and Engineering Technology International Conferences (ASET) · doi:10.1109/aset66891.2025.11427934
Dikbıyık, E. (2025) · Dspace Repository (Marmara Üniversitesi)
LF-GANet: Local Frame-Level Global Dynamic Attention Network for Speech Emotion Recognition
Dou, S., Han, T., Liu, R., Xia, W., & Zhong, H. (2024) · Lecture notes in electrical engineering · doi:10.1007/978-981-99-7505-1_13
Optimization of Speech Emotion Recognition Based on Deep Learning Techniques
Tonge, S., & Sondur, S. (2025) · Lecture notes in networks and systems · doi:10.1007/978-981-96-5958-6_28
Koohestan-Mahalian, F., & Cotter, N. E. (2020) · IEEE Access · doi:10.1109/access.2020.3034353
VoSE: An algorithm to Separate and Enhance Voices from Mixed Signals using Gradient BoostingPreprint
Gupta, M., Singh, R., & Sinha, S. (2020) · Research Square · doi:10.21203/rs.3.rs-93561/v1
IVSE: Indian Voice Separation and Enhancement from a cocktail party scenarioPreprint
Gupta, M., Singh, R. K., & Singh, S. (2023) · Research Square · doi:10.21203/rs.3.rs-1484209/v1
Emotion Recognition in Speech Using CNN Architecture for Enhanced Human-Machine InteractionPreprint
Singh, P., Raut, S., Chaudhari, T., & Gautam, N. (2025) · SSRN Electronic Journal · doi:10.2139/ssrn.5088907
Speech Emotion Recognition for Psychiatric Care Using Vision Transformers and Predictive Analysis
Uyyala, R., Shelly, M. S., Reddy, S. T., & Dugyala, R. (2024) · International Conference on Communication and Electronics Systems (ICCES) · doi:10.1109/icces63552.2024.10860014
Emotion recognition in deep learning across diverse speech and text corpora
Premakumari, S. B. N., Raja, A. C., Sumathi, K. K., Arimurthy, M., & Sampathkumar, Y. (2025) · AIP conference proceedings · doi:10.1063/5.0262274
Singh, S., & Rizvi, S. W. A. (2025) · International Journal of Computer Applications · doi:10.5120/ijca2025924860
Chaugule, I., & Sankaye, S. R. (2025) · International Journal of Research in Management · doi:10.33545/26648792.2025.v7.i1m.398
Prajapati, Y. J., Degadwala, P. G., & Degadwala, S. (2025) · International Journal of Scientific Research in Science and Technology · doi:10.32628/ijsrst25123148
Shelly, M. S., Reddy, S. T., Uyyala, R., & Vurubindi, P. (2025) · International Conference on Self Sustainable Artificial Intelligence Systems (ICSSAS) · doi:10.1109/icssas66150.2025.11080733
Emotion Detection from Speech Using Deep Neural Networks
Rani, S. S., Chandana, C., Naidu, P. H., Muzammil, T., & Tadepalli, S. K. (2025) · Advances in computer science research · doi:10.2991/978-94-6463-858-5_61
Evaluation of Noise Filtering Techniques for Speech Emotion Recognition in Educational Environment
Sunaryo, F., & Porle, R. R. (2025) · IEEE International Conference on Artificial Intelligence in Engineering and Technology (IICAIET) · doi:10.1109/iicaiet67254.2025.11265308
Irmawan, I., Dwijayanti, S., & Suprapto, B. Y. (2025) · Jurnal Nasional Teknik Elektro · doi:10.25077/jnte.v14n3.1310.2025
Optimized intelligent speech signal verification system for identifying authorized users
Ghate, P. M., Jadhav, B. D., Kota, P. N., Chavan, S. D., & Chopade, P. B. (2023) · Engineering and Applied Science Research · doi:10.14456/easr.2023.55
Explainable Machine Learning for Emotion Recognition from Speech Signals
Jeevapriya, G. M., Malya, A. R., Subbulakshmi, B., & Prasanna, S. (2025) · Advances in intelligent systems research/Advances in Intelligent Systems Research · doi:10.2991/978-94-6463-852-3_8
Palo, H. K., & Pattanaik, N. (2026) · Measurement · doi:10.1016/j.measurement.2026.120414
Robust Speech Emotion Recognition with Spectral Features Using Deep Convolutional Networks
Agarwal, S., Gulati, A., Kumar, G., & Bhardwaj, V. (2025) · IEEE Global Conference for Advancement in Technology (GCAT) · doi:10.1109/gcat66372.2025.11368460
Karthik, M. S., Tomar, S., & Koolagudi, S. G. (2025) · International Conference on Computational Intelligence and Communication Networks (CICN) · doi:10.1109/cicn67655.2025.11368129
A Hybrid BiLSTM–Transformer–Attention Model for Speech Emotion Recognition
Tahmid, S. M. A., & Lu, L. (2026) · International Conference on Communication Networks and Machine Learning (CNML) · doi:10.1109/cnml68938.2026.11452437
An Assistant System for Speaker and Sentiment Recognition Using RAM and a Hybrid AI Model
Bozyiğit, F., Aygün, İ., Sağlam, O., Özcan, E., Borandağ, E., & Karasulu, B. (2026) · Electronics · doi:10.3390/electronics15081731
Discourse Modeling for Text-based Emotion Recognition Using Fusion Approaches
Padhy, M., Rautray, R., & Sahu, P. (2026) · International Conference on Emerging Trends in Advancements and Applications of Computational Intelligence Techniques (ETAACT) · doi:10.1109/etaact69135.2026.11541522
Jahangir, R., Nauman, M. A., Saidani, O., & Ramzan, F. (2026) · Computers, materials & continua/Computers, materials & continua (Print) · doi:10.32604/cmc.2026.080025
Voice-Automated Emergency Response System
Jeyaprabha, T. J., S, H. K. G., T, H. V., & Balasaraswathy, B. (2026) · International Journal on Science and Technology · doi:10.71097/ijsat.v17.i2.11306
Implementation of Deep Learning in a Voice Recognition System for Virtual Assistants
Apriyanto, A., Sahirin, R., & Bradford, S. (2024) · Journal of Computer Science Advancements · doi:10.70177/jsca.v2i6.1533
AI-Driven Facial Emotion Detection using Convolutional Neural Network
Bamber, S. S. (2026) · International Journal of Advanced Research in Science Communication and Technology · doi:10.48175/ijarsct-38035
An Evaluation Study for Worthwhile Research in Speech Emotion Recognition Using Deep Learning
Abbas, S., & Abbas, J. M. A.-T. (2026) · Academic Science Journal · doi:10.24237/04.03.626
Peruka, B. (2024) · Journal of Electrical Systems
Metadata
- Authors
- Mustaqeem, Sajjad, M., & Kwon, S.
- Affiliations
- Islamia College University (PK)
- Sejong University (KR)
- Venue
- IEEE Access
- Published
- 2020
- Volume
- 8
- Pages
- 79861-79875
- ISSN
- 2169-3536 (ISSN-L)
- OpenAlex
- W3022013598
- Cited by
- 402
- References
- 64
- Citations by year
- 2026: 30 · 2025: 58 · 2024: 85 · 2023: 91 · 2022: 68 · 2021: 54 · 2020: 16
- FWCI
- 55.30
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Locations
- IEEE Access (publishedVersion, cc-by)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Funders
- Institute for Information and Communications Technology Planning and Evaluation (IITP) funded by the Korea Government through the Ministry of Science and ICT (MSIT) (2017-0-00189)
- 2020 Faculty Research Fund of Sejong University
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Emotion and Mood Recognition
- Physical Sciences › Computer Science › Artificial Intelligence › Speech Recognition and Synthesis
- Physical Sciences › Computer Science › Signal Processing › Speech and Audio Processing
- Keywords
- speech emotion recognition, clustering, bidirectional LSTM, temporal information, computational complexity reduction, IEMOCAP, EMO-DB, RAVDESS
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.