The DOI matches its Crossref record; 23 of the 23 references it cites were found in a registry or an index.
3-D Convolutional Recurrent Neural Networks With Attention Model for Speech Emotion Recognition
Abstract
Speech emotion recognition (SER) is a difficult task due to the complexity of emotions. The SER performances are heavily dependent on the effectiveness of emotional features extracted from the speech. However, most emotional features are sensitive to emotionally irrelevant factors, such as the speaker, speaking styles, and environment. In this letter, we assume that calculating the deltas and delta-deltas for personalized features not only preserves the effective emotional information but also reduces the influence of emotionally irrelevant factors, leading to reduce misclassification. In addition, SER often suffers from the silent frames and emotionally irrelevant frames. Meanwhile, attention mechanism has exhibited outstanding performances in learning relevant feature representations for specific tasks. Inspired by this, we propose a three-dimensional attention-based convolutional recurrent neural networks to learn discriminative features for SER, where the Mel-spectrogram with deltas and delta-deltas are used as input. Experiments on IEMOCAP and Emo-DB corpus demonstrate the effectiveness of the proposed method and achieve the state-of-the-art performance in terms of unweighted average recall.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Another issue faced by the time-frequency representation of speech emotion is that the distribution of emotion information in an utterance is sparse in both frequency domain and time domain [19], [23].”
From Speech Emotion Recognition via an Attentive Time–Frequency Neural Network (2023) · cites this paper as [19]
In the citing paper
“However, these studies mainly embed attention into highlevel time-frequency features, leading to introduce the redundant information in the feature extraction and roughly locate the emotion-related regions in the time-frequency domain.”
From Speech Emotion Recognition via an Attentive Time–Frequency Neural Network (2023) · cites this paper as [19]
In the citing paper
“In addition, they mainly focus on the selection on the time frames while ignoring the selection on the frequency bands.”
From Speech Emotion Recognition via an Attentive Time–Frequency Neural Network (2023) · cites this paper as [19]
In the citing paper
“ignoring to capture the specific frequency bands associated with emotions”
From Speech Emotion Recognition via an Attentive Time–Frequency Neural Network (2023) · cites this paper as [19]
Citations
Cites
Adam: A Method for Stochastic Optimization
Kingma, D. P., & Ba, J. (2014) · arXiv · doi:10.48550/arxiv.1412.6980 · Cited by 98,946 · cited 1×
“toolkit [24], and the parameters of the model were optimized by minimizing the cross-entropy objective function, with a minibatch of 40 samples, using the Adam optimizer [25] with Nestorov momentum.”
MethodNeural Machine Translation by Jointly Learning to Align and Translate
Bahdanau, D., Cho, K., & Bengio, Y. (2014) · arXiv · doi:10.48550/arxiv.1409.0473 · Cited by 16,213 · cited 1×
“proposed in [9] for machine translation and later applied for”
MethodTensorFlow: Large-Scale Machine Learning on Heterogeneous Distributed Systems
Abadi, M., Agarwal, A., Barham, P., Brevdo, E., Chen, Z., Citro, C., Corrado, G. S., Davis, A., Dean, J., Devin, M., et al. (2016) · arXiv · doi:10.48550/arxiv.1603.04467 · Cited by 9,408
Hierarchical Attention Networks for Document Classification
Yang, Z., Yang, D., Dyer, C., He, X., Smola, A., & Hovy, E. (2016) · Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies · doi:10.18653/v1/n16-1174 · Cited by 4,712 · cited 1×
“document classification [10] and speech recognition [11].”
Self-citationIEMOCAP: Interactive emotional dyadic motion capture database
Busso, C., Bulut, M., Lee, C.-C., Kazemzadeh, A., Mower, E., Kim, S., Chang, J. N., Lee, S., & Narayanan, S. S. (2008) · Language Resources and Evaluation · doi:10.1007/s10579-008-9076-6 · Cited by 3,869 · cited 12×
“D ACRNN obtains an absolute improvement of 2.34% and 3.44% for IEMOCAP and Emo-DB, respectively.”
MethodInfluentialA database of German emotional speech
Burkhardt, F., Paeschke, A., Rolfes, M., Sendlmeier, W. F., & Weiss, B. (2005) · Interspeech 2005 · doi:10.21437/interspeech.2005-446 · Cited by 2,365 · cited 1×
“form speaker-independent SER experiments on the Interactive Emotional Dyadic Motion Capture database (IEMOCAP) [21] and the Berlin Emotional Database (Emo-DB) [22].”
MethodAttention-Based Models for Speech Recognition
Chorowski, J., Bahdanau, D., Serdyuk, D., Cho, K., & Bengio, Y. (2015) · arXiv · doi:10.48550/arxiv.1506.07503 · Cited by 2,021
Bidirectional LSTM Networks for Improved Phoneme Classification and Recognition
Graves, A., Fernández, S., & Schmidhuber, J. (2005) · Lecture Notes in Computer Science · doi:10.1007/11550907_126 · Cited by 925 · cited 1×
“bidirectional RNN with long short-term memory cells [19] for temporal summarization, and each direction contains 128 cells; then, we can obtain a sequence of 256-dimensional high-level”
MethodAdieu features? End-to-end speech emotion recognition using a deep convolutional recurrent network
Trigeorgis, G., Ringeval, F., Brueckner, R., Marchi, E., Nicolaou, M. A., Schuller, B., & Zafeiriou, S. (2016) · 2016 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) · doi:10.1109/icassp.2016.7472669 · Cited by 904 · cited 1×
“In recent years, CNN has exhibited excellent performances in the field of SER [1], [4], [6].”
BackgroundSpeech emotion recognition using deep neural network and extreme learning machine
Han, K., Yu, D., & Tashev, I. (2014) · Interspeech 2014 · doi:10.21437/interspeech.2014-57 · Cited by 794 · cited 4×
“1) DNN-extreme learning machine (DNN-ELM) ( see [3]).”
MethodInfluential
Research works printed but not linked: 2
- wei, 2016, Attention assisted discovery of sub-utterance structure in speech emotion recognition, Proc INTERSPEECH, 1387
- frano, 2016, Normalization and batch correction methods for high-throughput glycomics, Proc Joint Meet Soc Glycobiol, 1160
Cited by
Akçay, M. B., & Oğuz, K. (2020) · Speech Communication · doi:10.1016/j.specom.2019.12.001 · Cited by 721
Speech Emotion Recognition Using Deep Learning Techniques: A Review
Khalil, R. A., Jones, E., Babar, M. I., Jan, T., Zafar, M. H., & Alhussain, T. (2019) · IEEE Access · doi:10.1109/access.2019.2936124 · Cited by 702
Speech emotion recognition with deep convolutional neural networks
Issa, D., Fatih Demirci, M., & Yazici, A. (2020) · Biomedical Signal Processing and Control · doi:10.1016/j.bspc.2020.101894 · Cited by 619
A systematic review on affective computing: emotion models, databases, and recent advances
Wang, Y., Song, W., Tao, W., Liotta, A., Yang, D., Li, X., Gao, S., Sun, Y., Ge, W., Zhang, W., et al. (2022) · Information Fusion · doi:10.1016/j.inffus.2022.03.009 · Cited by 615
Clustering-Based Speech Emotion Recognition by Incorporating Learned Features and Deep BiLSTM
Mustaqeem, Sajjad, M., & Kwon, S. (2020) · IEEE Access · doi:10.1109/access.2020.2990405 · Cited by 402
A CNN-Assisted Enhanced Audio Signal Processing for Speech Emotion Recognition
Mustaqeem, & Kwon, S. (2019) · Sensors · doi:10.3390/s20010183 · Cited by 392
“Chen et al. [42] developed a system for SER using 3D CNN architecture and trained model to increase the accuracy of SER, but he also used the pooling scheme to develop the network.”
MethodSpeech Emotion Recognition From 3D Log-Mel Spectrograms With Deep Learning Network
Meng, H., Yan, T., Yuan, F., & Wei, H. (2019) · IEEE Access · doi:10.1109/access.2019.2938007 · Cited by 356
“[12] proposed a 3-D attention-based convolutional recurrent neural networks(ACRNN), which was drafted by the learning ability of relevant feature representations for specific tasks from attention mechanism that was raised firstly in networks by Bahdanau et al.”
BackgroundMethodInfluentialSurvey on Deep Neural Networks in Speech and Vision Systems
Alam, M., Samad, M. D., Vidyaratne, L., Glandon, A., & Iftekharuddin, K. M. (2020) · Neurocomputing · doi:10.1016/j.neucom.2020.07.053 · Cited by 252
A Review on Speech Emotion Recognition Using Deep Learning and Attention Mechanism
Lieskovská, E., Jakubec, M., Jarina, R., & Chmulík, M. (2021) · Electronics · doi:10.3390/electronics10101163 · Cited by 230
MLT-DNet: Speech emotion recognition using 1D dilated CNN based on multi-learning trick approach
Mustaqeem, & Kwon, S. (2021) · Expert Systems with Applications · doi:10.1016/j.eswa.2020.114177 · Cited by 204
An ensemble 1D-CNN-LSTM-GRU model with data augmentation for speech emotion recognition
Rayhan Ahmed, M., Islam, S., Muzahidul Islam, A. K. M., & Shatabda, S. (2023) · Expert Systems with Applications · doi:10.1016/j.eswa.2023.119633 · Cited by 196
Abdelhamid, A. A., El-Kenawy, E.-S. M., Alotaibi, B., Amer, G. M., Abdelkader, M. Y., Ibrahim, A., & Eid, M. M. (2022) · IEEE Access · doi:10.1109/access.2022.3172954 · Cited by 194
“Chenet al. [21] hypothesized that measuring deltas and delta-deltas for customized characteristics not only retains successful emotional information but also reduces the impact of emotionally irrelevant variables, resulting in less mis-classification.”
BackgroundData Augmentation Using GANs for Speech Emotion Recognition
Chatziagapi, A., Paraskevopoulos, G., Sgouropoulos, D., Πανταζόπουλος, Γ., Nikandrou, M., Γιαννακόπουλος, Θ., Katsamanis, A., Potamianos, A., & Narayanan, S. S. (2019) · Interspeech (USB) · doi:10.21437/interspeech.2019-2561 · Cited by 191
“Furthermore, it has been proved that Recurrent Neural Networks (RNNs), such as Long Short-Term Memory units (LSTM), are able to take into account the temporal information in speech, resulting in a more robust modeling of the speech signals [5, 6, 7, 4].”
BackgroundSeen and Unseen Emotional Style Transfer for Voice Conversion with A New Emotional Speech Dataset
Zhou, K., Sisman, B., Liu, R., & Li, H. (2021) · ICASSP 2021 - 2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) · doi:10.1109/icassp39728.2021.9413391 · Cited by 190
“The SER architecture is as the same as that in [35], which includes: 1) a three-dimensional (3-”
BackgroundEmotion Intensity and its Control for Emotional Voice Conversion
Zhou, K., Şişman, B., Rana, R. K., Schuller, B. W., & Li, H. (2022) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2022.3175578 · Cited by 181
Improved End-to-End Speech Emotion Recognition Using Self Attention Mechanism and Multitask Learning
Li, Y., Zhao, T., & Kawahara, T. (2019) · Interspeech 2019 · doi:10.21437/interspeech.2019-2594 · Cited by 180
“Finally, the spectrogram with deltas and delta-deltas for better capturing personalized features [22], were extracted as the input to the neural network.”
MethodSpeech Emotion Recognition Based on Multiple Acoustic Features and Deep Convolutional Neural Network
Bhangale, K. B., & Kothandaraman, M. (2023) · Electronics · doi:10.3390/electronics12040839 · Cited by 177
Deep-Net: A Lightweight CNN-Based Speech Emotion Recognition System Using Deep Frequency Features
Anvarjon, T., Mustaqeem, & Kwon, S. (2020) · Sensors · doi:10.3390/s20185212 · Cited by 174
“Models IEMOCAP Accuracy EMO-DB Accuracy ACRNN [52] 13,487 s 64.”
MethodInfluentialA systematic literature review of speech emotion recognition approaches
Singh, Y. B., & Goel, S. (2022) · Neurocomputing · doi:10.1016/j.neucom.2022.04.028 · Cited by 174
Automated emotion recognition: Current trends and future perspectives
Maithri, M., Raghavendra, U., Gudigar, A., Samanth, J., Barua, P. D., Murugappan, M., Chakole, Y., & Acharya, U. R. (2022) · Computer Methods and Programs in Biomedicine · doi:10.1016/j.cmpb.2022.106646 · Cited by 172
Speech emotion recognition using recurrent neural networks with directional self-attention
Li, D., Liu, J., Yang, Z., Sun, L., & Wang, Z. (2021) · Expert Systems with Applications · doi:10.1016/j.eswa.2021.114683 · Cited by 171
CLSTM: Deep Feature-Based Speech Emotion Recognition Using the Hierarchical ConvLSTM Network
Mustaqeem, M., & Kwon, S. (2020) · Mathematics · doi:10.3390/math8122133 · Cited by 168
“The suggested function calculates the loss between the deep features and their consistent class center [50].”
BackgroundSpeech Emotion Recognition with Co-Attention Based Multi-Level Acoustic Information
Zou, H., Si, Y., Chen, C., Rajan, D., & Chng, E. S. (2022) · ICASSP 2022 - 2022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) · doi:10.1109/icassp43922.2022.9747095 · Cited by 166
Mu, W., Yin, B., Huang, X., Xu, J., & Du, Z. (2021) · Scientific Reports · doi:10.1038/s41598-021-01045-4 · Cited by 162
A survey of speech emotion recognition in natural environment
Shah Fahad, M., Ranjan, A., Yadav, J., & Deepak, A. (2021) · Digital Signal Processing · doi:10.1016/j.dsp.2020.102951 · Cited by 161
Att-Net: Enhanced emotion recognition system using lightweight self-attention module
Mustaqeem, M., & Kwon, S. (2021) · Applied Soft Computing · doi:10.1016/j.asoc.2021.107101 · Cited by 160
Farooq, M., Hussain, F., Baloch, N. K., Raja, F. R., Yu, H., & Zikria, Y. B. (2020) · Sensors · doi:10.3390/s20216008 · Cited by 159
“In [46], a 3-”
MethodResultInfluentialXu, M., Zhang, F., & Zhang, W. (2021) · IEEE Access · doi:10.1109/access.2021.3067460 · Cited by 153
“Chen et al. [2] combines convolutional neural networks (CNN) and Long Short-Term Memory(LSTM) in their SER model.”
BackgroundMethodImproving Speech Emotion Recognition With Adversarial Data Augmentation Network
Lu, Y., & Mak, M. (2020) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2020.3027600 · Cited by 142
“[55], which uses a 3-D convolutional recurrent neural networks to generate discriminative features.”
BackgroundLIGHT-SERNET: A Lightweight Fully Convolutional Neural Network for Speech Emotion Recognition
Aftab, A., Morsali, A., Ghaemmaghami, S., & Champagne, B. (2022) · ICASSP 2022 - 2022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) · doi:10.1109/icassp43922.2022.9746679 · Cited by 130
Deep Learning Techniques for Speech Emotion Recognition: A Review
Pandey, S., Shekhawat, H. S., & Prasanna, S. R. M. (2019) · International Conference Radioelektronika · doi:10.1109/radioelek.2019.8733432 · Cited by 130
“The work in [6], [7], [8] and [9] utilizes attention layer in addition to the conventional RNN or CNN layers to increase the emotion capturing capability of the models.”
MethodA multimodal hierarchical approach to speech emotion recognition from audio and text
Singh, P., Srivastava, R., Rana, K. P. S., & Kumar, V. (2021) · Knowledge-Based Systems · doi:10.1016/j.knosys.2021.107316 · Cited by 130
Multimodal Emotion Recognition Based on Facial Expressions, Speech, and EEG
Pan, J., Fang, W., Zhang, Z., Chen, B., Zhang, Z., & Wang, S. (2024) · IEEE Open Journal of Engineering in Medicine and Biology · doi:10.1109/ojemb.2023.3240280 · Cited by 121
Zhao, Z., Li, Q., Zhang, Z., Cummins, N., Wang, H., Tao, J., & Schuller, B. W. (2021) · Neural Networks · doi:10.1016/j.neunet.2021.03.013 · Cited by 121
“A critical underlying issue is the loss of dynamic temporal information, which can strongly reflect changes in emotional states (Han, Ruan, Chen, Wang, Li, & Schuller, 2018).”
InfluentialAn enhanced speech emotion recognition using vision transformer
Akinpelu, S. A., Viriri, S., & Adegun, A. A. (2024) · Scientific Reports · doi:10.1038/s41598-024-63776-4 · Cited by 120
Jiang, P., Fu, H., Tao, H., Lei, P., & Zhao, L. (2019) · IEEE Access · doi:10.1109/access.2019.2927384 · Cited by 118
“GENERATION OF PCRN INPUT Since spectral features can better describe the time-frequency correlation of emotional details, more and more researchers have applied spectral features to speech emotion recognition [28], [12], [13].”
BackgroundMethodInfluentialAttention Driven Fusion for Multi-Modal Emotion Recognition
Priyasad, D., Fernando, T., Denman, S., Sridharan, S., & Fookes, C. (2020) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp40776.2020.9054441 · Cited by 117
“However, both methods use manually calculated audio features which limit their accuracy and the robustness of the acoustic model [13, 14].”
MethodMustaqeem, M., & Kwon, S. (2021) · International Journal of Intelligent Systems · doi:10.1002/int.22505 · Cited by 117
Learning multi-scale features for speech emotion recognition with connection attention mechanism
Chen, Z., Li, J., Liu, H., Wang, X., Hu, W., & Zheng, Q. (2022) · Expert Systems with Applications · doi:10.1016/j.eswa.2022.118943 · Cited by 113
Robust Multimodal Emotion Recognition from Conversation with Transformer-Based Crossmodality Fusion
Xie, B., Sidulova, M., & Park, C. H. (2021) · Sensors · doi:10.3390/s21144913 · Cited by 108
Speech Synthesis With Mixed Emotions
Zhou, K., Şişman, B., Rana, R. K., Schuller, B. W., & Li, H. (2022) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2022.3233324 · Cited by 108
“The SER architecture is the same as that in [122], which includes: 1) a three-dimensional (3-”
BackgroundAttention-augmented End-to-end Multi-task Learning for Emotion Prediction from Speech
Zhang, Z., Wu, B., & Schuller, B. W. (2019) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp.2019.8682896 · Cited by 105
Chamishka, S., Madhavi, I., Nawaratne, R., Alahakoon, D., Silva, D. D., Chilamkurti, N., & Nanayakkara, V. (2022) · Multimedia Tools and Applications · doi:10.1007/s11042-022-13363-4 · Cited by 103
Liu, Z., Han, M., Wu, B.-H., & Rehman, A. (2022) · Applied Acoustics · doi:10.1016/j.apacoust.2022.109178 · Cited by 100
Expressive TTS Training With Frame and Style Reconstruction Loss
Liu, R., Şişman, B., Gao, G., & Li, H. (2021) · IEEE/ACM Transactions on Audio Speech and Language Processing · doi:10.1109/taslp.2021.3076369 · Cited by 97
“D CNN for dimension reduction.”
MethodInfluentialEnd-to-End Speech Emotion Recognition With Gender Information
Sun, T.-W. (2020) · IEEE Access · doi:10.1109/access.2020.3017462 · Cited by 96
“By contrast, [22]–[25] focus on discrete emotion recognition.”
BackgroundMethodSpeech Emotion Recognition with Multiscale Area Attention and Data Augmentation
Xu, M., Zhang, F., Cui, X., & Zhāng, W. (2021) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp39728.2021.9414635 · Cited by 96
Bangla Speech Emotion Recognition and Cross-Lingual Study Using Deep CNN and BLSTM Networks
Sultana, S., Iqbal, M. Z., Selim, M. R., Rashid, M. M., & Rahman, M. S. (2021) · IEEE Access · doi:10.1109/access.2021.3136251 · Cited by 95
“D attention-based convolutional recurrent neural networks (ACRNN), Chen et al. used deltas and delta deltas of log mel-spectrogram for emotion identification [16].”
Research on Bearing Fault Diagnosis of Wind Turbine Gearbox Based on 1DCNN-PSO-SVM
Zhang, X., Han, P., Li, X., Zhang, F., Wang, Y., & Gao, L. (2020) · IEEE Access · doi:10.1109/access.2020.3032719 · Cited by 94
Adversarial Invariant Feature Learning with Accuracy Constraint for Domain Generalization
Akuzawa, K., Iwasawa, Y., & Matsuo, Y. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-46147-8_19 · Cited by 92
Attention-LSTM-Attention Model for Speech Emotion Recognition and Analysis of IEMOCAP Database
Yu, Y., & Kim, Y.-J. (2020) · Electronics · doi:10.3390/electronics9050713 · Cited by 92
Peng, Z., Li, X., Zhu, Z., Unoki, M., Dang, J., & Akagi, M. (2020) · IEEE Access · doi:10.1109/access.2020.2967791 · Cited by 91
“For example, [25], [29], [30] did not merge happy and excited into one class, while [33] just reported weighted accuracy.”
BackgroundMethodLuo, Z., Xu, H., & Chen, F. (2018) · EasyChair preprint · doi:10.29007/7mhj · Cited by 89
Speech Databases, Speech Features, and Classifiers in Speech Emotion Recognition: A Review
Dar, G. H. M., & Delhibabu, R. (2024) · IEEE Access · doi:10.1109/access.2024.3476960 · Cited by 88
Daneshfar, F., Kabudian, S. J., & Neekabadi, A. (2020) · Applied Acoustics · doi:10.1016/j.apacoust.2020.107360 · Cited by 87
Liu, Z., Kang, X., & Ren, F. (2023) · IEEE/ACM Transactions on Audio Speech and Language Processing · doi:10.1109/taslp.2023.3282092 · Cited by 87
Spatiotemporal and frequential cascaded attention networks for speech emotion recognition
Li, S., Xing, X., Fan, W., Cai, B., Fordson, P., & Xu, X. (2021) · Neurocomputing · doi:10.1016/j.neucom.2021.02.094 · Cited by 86
Speaker Normalization for Self-Supervised Speech Emotion Recognition
Gat, I., Aronowitz, H., Zhu, W., Morais, E. D. S., & Hoory, R. (2022) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp43922.2022.9747460 · Cited by 86
Zhou, H., Du, J., Zhang, Y., Wang, Q., Liu, Q.-F., & Lee, C.-H. (2021) · IEEE/ACM Transactions on Audio, Speech, and Language Processing · doi:10.1109/taslp.2021.3096037 · Cited by 85
“In [11], a 3-”
BackgroundExploring Emotion Features and Fusion Strategies for Audio-Video Emotion Recognition
Zhou, H., Meng, D., Zhang, Y., Peng, X., Du, J., Wang, K., & Qiao, Y. (2019) · INTERNATIONAL CONFERENCE ON MULTIMODAL INTERACTION · doi:10.1145/3340555.3355713 · Cited by 85
“For each audio, the speech spectrogram and log Mel-spectrogram extraction process is consistent with [32] and [3] respectively.”
MethodImprove Accuracy of Speech Emotion Recognition with Attention Head Fusion
Xu, M., Zhang, F., & Khan, S. U. (2020) · IEEE Annual Computing and Communication Workshop and Conference (CCWC) · doi:10.1109/ccwc47524.2020.9031207 · Cited by 84
“In the field of speech recognition, attention mechanism has been used in many tasks such as ASR [11], speaker recognition [12] and SER [3], [13]–[15], also in our work.”
MethodSpeech Emotion Recognition with Global-Aware Fusion on Multi-Scale Feature Representation
Zhu, W., & Li, X. (2022) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp43922.2022.9747517 · Cited by 84
1D-CNN: Speech Emotion Recognition System Using a Stacked Network with Dilated CNN Features
Mustaqeem, M., & Kwon, S. (2021) · Computers, materials & continua/Computers, materials & continua (Print) · doi:10.32604/cmc.2021.015070 · Cited by 82
Survey on multimodal approaches to emotion recognition
Aruna Gladys, A., & Vetriselvi, V. (2023) · Neurocomputing · doi:10.1016/j.neucom.2023.126693 · Cited by 82
Gudmalwar, A. P., Basel, B., Dutta, A., & Rao, C. V. R. (2023) · IEEE Transactions on Consumer Electronics · doi:10.1109/tce.2023.3236972 · Cited by 80
“In [27], authors used 3D CNN for SER using log-Mel spectrum, its delta and double delta values as features.”
BackgroundMethodLearning Attentive Representations for Environmental Sound Classification
Zhang, Z., Xu, S., Zhang, S., Qiao, T., & Cao, S. C. (2019) · IEEE Access · doi:10.1109/access.2019.2939495 · Cited by 77
“Attention mechanisms have also been widely explored in sound recognition area such as speech recognition, speech emotion recognition and speaker recognition [24]–[29] and obtained promising performance in several datasets.”
BackgroundSpeech Emotion Recognition with Local-Global Aware Deep Representation Learning
Liu, J., Liu, Z., Wang, L., Guo, L., & Dang, J. (2020) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp40776.2020.9053192 · Cited by 75
“At the same time, Happiness is always mistaken into Neturality in IEMOCAP [22].”
BackgroundA novel spatio-temporal convolutional neural framework for multimodal emotion recognition
Sharafi, M., Yazdchi, M., Rasti, R., & Nasimi, F. (2022) · Biomedical Signal Processing and Control · doi:10.1016/j.bspc.2022.103970 · Cited by 73
Contextual and Cross-Modal Interaction for Multi-Modal Speech Emotion Recognition
Yang, D., Huang, S., Liu, Y., & Zhang, L. (2022) · IEEE Signal Processing Letters · doi:10.1109/lsp.2022.3210836 · Cited by 73
“…methods in SER, various neural network models have been developed to extract emotion-related information from either handcrafted acoustic features or raw audio signals, such as Convolutional Neural Networks (CNN) [4], [5], Recurrent Neural Networks (RNN) [6], [7], and their variants [8], [9].”
MethodHao, C., Zhi, H., Dong, L., & Liu, Y. (2018) · Computational Intelligence and Neuroscience · doi:10.1155/2018/9750904 · Cited by 72
“Emotional recognition can be performed by external features such as facial expressions and voice intonation [2–6].”
BackgroundPham, N. T., Dang, D. N. M., Nguyen, N. D., Nguyen, T. T., Nguyen, V., Manavalan, B., Lim, C. P., & Nguyen, S. D. (2023) · Expert Systems with Applications · doi:10.1016/j.eswa.2023.120608 · Cited by 71
Semantic Alignment Network for Multi-Modal Emotion Recognition
Hou, M.-X., Zhang, Z., Liu, C., & Lu, G. (2023) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2023.3247822 · Cited by 71
“The audio signal of an utterance was split into frames with Hamming windows of 25-ms and a 10-ms shift [12].”
BackgroundMethodKhan, M., Saddik, A. E., Alotaibi, F. S., & Pham, N. T. (2023) · Knowledge-Based Systems · doi:10.1016/j.knosys.2023.110525 · Cited by 71
Efficient speech emotion recognition using modified feature extraction
Langari, S., Marvi, H., & Zahedi, M. (2020) · Informatics in Medicine Unlocked · doi:10.1016/j.imu.2020.100424 · Cited by 70
Deepfake Speech Detection Through Emotion Recognition: A Semantic Approach
Conti, E., Salvi, D., Borrelli, C., Hosler, B., Bestagini, P., Antonacci, F., Sarti, A., Stamm, M. C., & Tubaro, S. (2022) · ICASSP 2022 - 2022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) · doi:10.1109/icassp43922.2022.9747186 · Cited by 69
A Novel end-to-end Speech Emotion Recognition Network with Stacked Transformer Layers
Wang, X., Wang, M., Qi, W., Su, W., Wang, X., & Zhou, H. (2021) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp39728.2021.9414314 · Cited by 69
Hybrid Network Feature Extraction for Depression Assessment from Speech
Zhao, Z., Li, Q., Cummins, N., Liu, B., Wang, H., Tao, J., & Schuller, B. W. (2020) · Interspeech (USB) · doi:10.21437/interspeech.2020-2396 · Cited by 68
“Inspired by the successful use of 3D log-Mel spectrograms in SER [20], our hybrid system also uses log-Mels, together with deltas and delta-deltas, as the input to the DCNN.”
BackgroundMethodInfluentialA novel dual attention-based BLSTM with hybrid features in speech emotion recognition
Chen, Q., & Huang, G. (2021) · Engineering Applications of Artificial Intelligence · doi:10.1016/j.engappai.2021.104277 · Cited by 68
Sparse temporal aware capsule network for robust speech emotion recognition
Zhang, H., Huang, H., Zhao, P., & Yu, Z. (2025) · Engineering Applications of Artificial Intelligence · doi:10.1016/j.engappai.2025.110060 · Cited by 68
Self-citationIs Cross-Attention Preferable to Self-Attention for Multi-Modal Emotion Recognition?
Rajan, V., Brutti, A., & Cavallaro, A. (2022) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp43922.2022.9746924 · Cited by 66
Poorna, S. S., Menon, V., & Gopalan, S. (2024) · Biomedical Signal Processing and Control · doi:10.1016/j.bspc.2024.106967 · Cited by 66
Cross-Corpus Speech Emotion Recognition Based on Few-Shot Learning and Domain Adaptation
Ahn, Y., Lee, S. J., & Shin, J. W. (2021) · IEEE Signal Processing Letters · doi:10.1109/lsp.2021.3086395 · Cited by 65
Social contagion of cyberbullying via online perpetrator and victim networks
Yokotani, K., & Takano, M. (2021) · Computers in Human Behavior · doi:10.1016/j.chb.2021.106719 · Cited by 64
Lin, W.-C., & Busso, C. (2021) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2021.3083821 · Cited by 63
“One essential component of the proposed framework is the fusion of information in the chunks through attention models, which have been widely used in SER [35], [36], [37], [38].”
MethodAcoustic and Lexical Sentiment Analysis for Customer Service Calls
Li, B., Dimitriadis, D., & Stolcke, A. (2019) · ICASSP 2019 - 2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) · doi:10.1109/icassp.2019.8683679 · Cited by 62
“[2] propose a frame-based 3DCRNN, and report results of 64.”
BackgroundMethodHao, C., & Liu, Y. (2020) · IEEE Access · doi:10.1109/access.2020.2974009 · Cited by 62
“Emotion recognition can be implemented by external information such as voice intonation [2], [3], facial expression [4] and body posture [5], as well as by changes in the nervous system such as EEG signals.”
BackgroundCao, Q. Z., Hou, M.-X., Chen, B., Zhang, Z., & Lu, G. (2021) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp39728.2021.9414540 · Cited by 61
“Work [10] provides 3 channels to examine static and dynamic features separately, but it ignores the fusion of multiple features.”
BackgroundMethodResultInfluentialReinforcement Learning for Emotional Text-to-Speech Synthesis with Improved Emotion Discriminability
Liu, R., Şişman, B., & Li, H. (2021) · Interspeech (USB) · doi:10.21437/interspeech.2021-1236 · Cited by 61
An octonion-based nonlinear echo state network for speech emotion recognition in Metaverse
Daneshfar, F., & Jamshidi, M. B. (2023) · Neural Networks · doi:10.1016/j.neunet.2023.03.026 · Cited by 59
Context-Aware Emotion Recognition Based on Visual Relationship Detection
Hoang, M.-H., Kim, S.-H., Yang, H.-J., & Lee, G.-S. (2021) · IEEE Access · doi:10.1109/access.2021.3091169 · Cited by 57
Detection of Emotion of Speech for RAVDESS Audio Using Hybrid Convolution Neural Network
Puri, T., Soni, M., Dhiman, G., Khalaf, O. I., Alazzam, M. B., & Khan, I. R. (2022) · Journal of Healthcare Engineering · doi:10.1155/2022/8472947 · Cited by 56
“Rasmus et al. [26] developed the idea of pure unattended networks that conserve adequate information, in comparison with strictly supervised networks, to recreate input instances that only hold specific classification information. kind of architecture assumes that all of these features are a…”
BackgroundRetractedJIANG, P., Xu, X., Tao, H., Zhao, L., & Zou, C. (2021) · IEEE Transactions on Cognitive and Developmental Systems · doi:10.1109/tcds.2021.3123979 · Cited by 55
“The highly related work [17] uses the serial structure of CNN and LSTM, different from our paralleled structure, while [22], [23] present similar models as ours, yet we propose a design of the attention layers.”
BackgroundMethodInfluentialMaji, B., Swain, M., & Mustaqeem, M. (2022) · Electronics · doi:10.3390/electronics11091328 · Cited by 55
EmoMix: Emotion Mixing via Diffusion Models for Emotional Speech Synthesis
Tang, H., Zhang, X., Wang, J., Cheng, N., & Xiao, J. (2023) · Interspeech (USB) · doi:10.21437/interspeech.2023-1317 · Cited by 55
Pre-trained Deep Convolution Neural Network Model With Attention for Speech Emotion Recognition
Zhang, H., Gou, R., Shang, J., Shen, F., Wu, Y., & Dai, G. (2021) · Frontiers in Physiology · doi:10.3389/fphys.2021.643202 · Cited by 53
Self-citationChen, Z., Lin, M., Wang, Z., Zheng, Q., & Liu, C. (2023) · Knowledge-Based Systems · doi:10.1016/j.knosys.2023.111077 · Cited by 53
A Lightweight Model Based on Separable Convolution for Speech Emotion Recognition
Zhong, Y., Hu, Y., Huang, H. M., & Silamu, W. (2020) · Interspeech (USB) · doi:10.21437/interspeech.2020-2408 · Cited by 53
“Chen et al. [9] employed attention-based convolutional RNN (ACRNN) network to extract high-level emotional feature representations from the log Mel-spectrogram.”
BackgroundMethodAn Ensemble Model for Multi-Level Speech Emotion Recognition
Zheng, C., Wang, C., & Jia, N. (2019) · Applied Sciences · doi:10.3390/app10010205 · Cited by 52
“In [22], a three-dimensional attention-based convolution recurrent neural network is proposed to learn the discriminative features of SER, which not only retains effective emotional information, but also reduces the influence of emotional independence.”
BackgroundLi, W., Xue, J., Tan, R., Wang, C., Deng, Z., Li, S., Guo, G., & Cao, D. (2023) · IEEE Transactions on Intelligent Vehicles · doi:10.1109/tiv.2023.3259988 · Cited by 52
“[25]extractedthelog-melspectrumfeaturemapin speech, calculated the first-order and second-order differences, and then combined the three sets of features to form a set of feature maps with 3 channels, through a set of CRNN networks to test the robustness of features.”
BackgroundSpeech Emotion Recognition via an Attentive Time–Frequency Neural Network
Lu, C., Zheng, W., Lian, H., Zong, Y., Tang, C., Li, S., & Zhao, Y. (2023) · IEEE Transactions on Computational Social Systems · doi:10.1109/tcss.2022.3219825 · Cited by 51
“For the purpose of capturing these emotion-related regions, Chen et al. [19] investigated adding an attention layer after CNN and LSTM to capture specific temporal frames for the speech emotion representation.”
BackgroundMethodInfluential
Metadata
- Authors
- Chen, M., He, X., Yang, J., & Zhang, H.
- Affiliations
- Central South University (CN)
- Venue
- IEEE Signal Processing Letters
- Published
- 2018-10
- Volume
- 25
- Issue
- 10
- Pages
- 1440-1444
- ISSN
- 1070-9908 (ISSN-L)
- 1558-2361
- OpenAlex
- W2885005742
- Cited by
- 503
- References
- 23
- Citations by year
- 2026: 21 · 2025: 50 · 2024: 70 · 2023: 85 · 2022: 91 · 2021: 94 · 2020: 60 · 2019: 27 · 2018: 5
- FWCI
- 60.91
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- IEEE Signal Processing Letters (publishedVersion)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Emotion and Mood Recognition
- Physical Sciences › Computer Science › Artificial Intelligence › Speech Recognition and Synthesis
- Physical Sciences › Computer Science › Signal Processing › Speech and Audio Processing
- Keywords
- speech emotion recognition, attention mechanism, Mel spectrogram, IEMOCAP, EMO-DB
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.