3-D Convolutional Recurrent Neural Networks With Attention Model for Speech Emotion Recognition
The DOI matches its Crossref record; 23 of the 23 references it cites were found in a registry or an index.
Abstract
Speech emotion recognition (SER) is a difficult task due to the complexity of emotions. The SER performances are heavily dependent on the effectiveness of emotional features extracted from the speech. However, most emotional features are sensitive to emotionally irrelevant factors, such as the speaker, speaking styles, and environment. In this letter, we assume that calculating the deltas and delta-deltas for personalized features not only preserves the effective emotional information but also reduces the influence of emotionally irrelevant factors, leading to reduce misclassification. In addition, SER often suffers from the silent frames and emotionally irrelevant frames. Meanwhile, attention mechanism has exhibited outstanding performances in learning relevant feature representations for specific tasks. Inspired by this, we propose a three-dimensional attention-based convolutional recurrent neural networks to learn discriminative features for SER, where the Mel-spectrogram with deltas and delta-deltas are used as input. Experiments on IEMOCAP and Emo-DB corpus demonstrate the effectiveness of the proposed method and achieve the state-of-the-art performance in terms of unweighted average recall.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Another issue faced by the time-frequency representation of speech emotion is that the distribution of emotion information in an utterance is sparse in both frequency domain and time domain [19], [23].”
From Speech Emotion Recognition via an Attentive Time–Frequency Neural Network (2023) · cites this paper as [19]
In the citing paper
“However, these studies mainly embed attention into highlevel time-frequency features, leading to introduce the redundant information in the feature extraction and roughly locate the emotion-related regions in the time-frequency domain.”
From Speech Emotion Recognition via an Attentive Time–Frequency Neural Network (2023) · cites this paper as [19]
In the citing paper
“In addition, they mainly focus on the selection on the time frames while ignoring the selection on the frequency bands.”
From Speech Emotion Recognition via an Attentive Time–Frequency Neural Network (2023) · cites this paper as [19]
In the citing paper
“ignoring to capture the specific frequency bands associated with emotions”
From Speech Emotion Recognition via an Attentive Time–Frequency Neural Network (2023) · cites this paper as [19]
Citations
Cites
Adam: A Method for Stochastic Optimization
Kingma, D. P., & Ba, J. (2014) · arXiv · doi:10.48550/arxiv.1412.6980 · Cited by 98,946 · cited 1×
“toolkit [24], and the parameters of the model were optimized by minimizing the cross-entropy objective function, with a minibatch of 40 samples, using the Adam optimizer [25] with Nestorov momentum.”
MethodNeural Machine Translation by Jointly Learning to Align and Translate
Bahdanau, D., Cho, K., & Bengio, Y. (2014) · arXiv · doi:10.48550/arxiv.1409.0473 · Cited by 16,213 · cited 1×
“proposed in [9] for machine translation and later applied for”
MethodTensorFlow: Large-Scale Machine Learning on Heterogeneous Distributed Systems
Abadi, M., Agarwal, A., Barham, P., Brevdo, E., Chen, Z., Citro, C., Corrado, G. S., Davis, A., Dean, J., Devin, M., et al. (2016) · arXiv · doi:10.48550/arxiv.1603.04467 · Cited by 9,408
Hierarchical Attention Networks for Document Classification
Yang, Z., Yang, D., Dyer, C., He, X., Smola, A., & Hovy, E. (2016) · Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies · doi:10.18653/v1/n16-1174 · Cited by 4,712 · cited 1×
“document classification [10] and speech recognition [11].”
Self-citationIEMOCAP: Interactive emotional dyadic motion capture database
Busso, C., Bulut, M., Lee, C.-C., Kazemzadeh, A., Mower, E., Kim, S., Chang, J. N., Lee, S., & Narayanan, S. S. (2008) · Language Resources and Evaluation · doi:10.1007/s10579-008-9076-6 · Cited by 3,869 · cited 12×
“D ACRNN obtains an absolute improvement of 2.34% and 3.44% for IEMOCAP and Emo-DB, respectively.”
MethodInfluentialA database of German emotional speech
Burkhardt, F., Paeschke, A., Rolfes, M., Sendlmeier, W. F., & Weiss, B. (2005) · Interspeech 2005 · doi:10.21437/interspeech.2005-446 · Cited by 2,365 · cited 1×
“form speaker-independent SER experiments on the Interactive Emotional Dyadic Motion Capture database (IEMOCAP) [21] and the Berlin Emotional Database (Emo-DB) [22].”
MethodAttention-Based Models for Speech Recognition
Chorowski, J., Bahdanau, D., Serdyuk, D., Cho, K., & Bengio, Y. (2015) · arXiv · doi:10.48550/arxiv.1506.07503 · Cited by 2,021
Bidirectional LSTM Networks for Improved Phoneme Classification and Recognition
Graves, A., Fernández, S., & Schmidhuber, J. (2005) · Lecture Notes in Computer Science · doi:10.1007/11550907_126 · Cited by 925 · cited 1×
“bidirectional RNN with long short-term memory cells [19] for temporal summarization, and each direction contains 128 cells; then, we can obtain a sequence of 256-dimensional high-level”
MethodAdieu features? End-to-end speech emotion recognition using a deep convolutional recurrent network
Trigeorgis, G., Ringeval, F., Brueckner, R., Marchi, E., Nicolaou, M. A., Schuller, B., & Zafeiriou, S. (2016) · 2016 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) · doi:10.1109/icassp.2016.7472669 · Cited by 904 · cited 1×
“In recent years, CNN has exhibited excellent performances in the field of SER [1], [4], [6].”
BackgroundSpeech emotion recognition using deep neural network and extreme learning machine
Han, K., Yu, D., & Tashev, I. (2014) · Interspeech 2014 · doi:10.21437/interspeech.2014-57 · Cited by 794 · cited 4×
“1) DNN-extreme learning machine (DNN-ELM) ( see [3]).”
MethodInfluential
Research works printed but not linked: 2
- wei, 2016, Attention assisted discovery of sub-utterance structure in speech emotion recognition, Proc INTERSPEECH, 1387
- frano, 2016, Normalization and batch correction methods for high-throughput glycomics, Proc Joint Meet Soc Glycobiol, 1160
Cited by
Items 201–300 of 549
Speech emotion recognition for the Urdu language: Dataset and evaluation
Zaheer, N., Ahmad, O. U., Shabbir, M., & Raza, A. A. (2022) · Language Resources and Evaluation · doi:10.1007/s10579-022-09610-7 · Cited by 20
“The weighted mean is taken of the result using a dense neural network layer (Chen et al., 2018; Zhao et al., 2019).”
MethodDeep Learning Framework for Speech Emotion Classification: A Survey of the State-of-the-Art
Akinpelu, S. A., & Viriri, S. (2024) · IEEE Access · doi:10.1109/access.2024.3474553 · Cited by 20
“Chen et al. [206] utilized three-dimensional attention-based CRNN for selecting discriminating features in the classification of speech emotion.”
MethodA novel feature engineering method for severity prediction of marine accidents
Li, T., Wang, X., Zhang, Z., & Feng, Y. (2025) · Journal of Marine Engineering & Technology · doi:10.1080/20464177.2025.2469336 · Cited by 20
“( 2022 ) as it assigns equal weight to each class, independent of class size (Kim et al. 2015; Chen et al. 2018).”
3D Convolutional Recurrent Global Neural Network for Speech Emotion Recognition
Zayene, B., Jlassi, C., & Arous, N. (2020) · International Conference on Advanced Technologies for Signal and Image Processing · doi:10.1109/atsip49331.2020.9231597 · Cited by 20
Pérez-Toro, P. A., Vásquez-Correa, J. C., Bocklet, T., Nöth, E., & Orozco‐Arroyave, J. R. (2021) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2021.3112543 · Cited by 21
“…the first have shown to be suitable to capture spatial patterns and the energy content in the different frequency bands of the spectrogram [55], [56], and the second one allows to model prosody aspects and temporal information of speech, which are useful to characterize affective patterns [57],…”
BackgroundSpeech Emotion Recognition using XGBoost and CNN BLSTM with Attention
He, J., & Ren, L. (2021) · IEEE SmartWorld, Ubiquitous Intelligence & Computing, Advanced & Trusted Computing, Scalable Computing & Communications, Cloud & Big Data Computing, Internet of People and Smart City Innovation (SmartWorld/SCALCOM/UIC/ATC/CBDCom/IOP/SCI) · doi:10.1109/swc50871.2021.00030 · Cited by 20
Rethinking Auditory Affective Descriptors Through Zero-Shot Emotion Recognition in Speech
Xu, X., Deng, J., Zhang, Z., Fan, X., Zhao, L., Devillers, L. Y., & Schuller, B. W. (2021) · IEEE Transactions on Computational Social Systems · doi:10.1109/tcss.2021.3130401 · Cited by 20
“Considering these settings’ differences, existing SER topics usually aim at exploring fully supervised [6], [9], [10], semisupervised [11], and data-driven transfer learning [12]–[14] as cases.”
BackgroundJafarzadeh, P., Rostami, A. M., & Choobdar, P. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2411.02964 · Cited by 21
“Some meth-ods apply CNNs to extract local features from the input, then use LSTM layers to learn long-term dependencies from the extracted local features [57, 9].”
BackgroundMethodLearning Local to Global Feature Aggregation for Speech Emotion Recognition
Lu, C., Lian, H., Zheng, W., Zong, Y., Zhao, Y., & Li, S. (2023) · INTERSPEECH 2023 · doi:10.21437/interspeech.2023-543 · Cited by 19
Advanced Sequence Learning Approaches for Emotion Recognition Using Speech Signals
Khan, M., Ishaq, M., Swain, M., & Kwon, S. (2023) · Springer eBooks · doi:10.1007/978-3-031-34873-0_13 · Cited by 19
Development of Speech Emotion Recognition Algorithm using MFCC and Prosody
Koo, H., Jeong, S., Yoon, S., & Kim, W. (2020) · International Conference on Electronics, Information, and Communication (ICEIC) · doi:10.1109/iceic49074.2020.9051281 · Cited by 19
“[12] Chen et al. extracted log-Mels (static, deltas and delta-deltas) as the 3-dimensional CNN input, and the input was fed into attention-based convolutional recurrent neural networks (ACRNN) [13].”
MethodNfissi, A., Bouachir, W., Bouguila, N., & Mishara, B. L. (2022) · International Conference on Machine Learning and Applications · doi:10.1109/icmla55696.2022.00116 · Cited by 19
“In [5], another principle was employed to address the temporal representation of the speech, which consists of a 3D-CNN after computing deltas and deltadeltas for log-Mels attention CRNN.”
MethodAi, X., Sheng, V. S., Fang, W., Ling, C. X., & Li, C. (2020) · IEEE Access · doi:10.1109/access.2020.3035910 · Cited by 19
“In 2018, Chen et al. [18] proposed to feed Mel-spectrogram, Mel-spectrogram with deltas and Mel-spectrogram with delta-deltas into an attention model integrated CNN.”
MethodDeep Learning-Based Speech Emotion Recognition: Evaluating CNN, CLSTM, and LSTM Models
Jannu, C., Bharath, D., Sanjay, V., Kothakotamaana, H., Adapa, S., & Parisae, V. (2025) · International Conference on Inventive Systems and Control (ICISC) · doi:10.1109/icisc65841.2025.11188229 · Cited by 18
“Still different, researchers calculated delta and delta-delta features in the context of log-Mel coefficients to represent temporal speech before processing action for a 3D-CNN [5].”
A multi-dilated convolution network for speech emotion recognition
Madanian, S., Adeleye, O., Templeton, J. M., Chen, T., Poellabauer, C., Zhang, E., & Schneider, S. L. (2025) · Scientific Reports · doi:10.1038/s41598-025-92640-2 · Cited by 18
“In addition to the 40 log-mel spectrogram features, we also extract the delta and delta-delta features from the log-mel spectrum, following the approach described in 17,21 .”
InfluentialPrivacy and Utility Preserving Data Transformation for Speech Emotion Recognition
Feng, T., & Narayanan, S. S. (2021) · International Conference on Affective Computing and Intelligent Interaction and workshops · doi:10.1109/acii52823.2021.9597433 · Cited by 18
“This result is consistent with previous works in predicting happy emotion [4], [11].”
ResultRajapakshe, T., Rana, R. K., Khalifa, S., Şişman, B., Schuller, B. W., & Busso, C. (2024) · IEEE Access · doi:10.1109/access.2024.3439604 · Cited by 18
“In recent years, the literature has highlighted the use of attention networks in SER, which has provided superior outcomes [27], [28].”
BackgroundMeta Multi-Task Learning for Speech Emotion Recognition
Cai, R., Guo, K., Xu, B., Yang, X., & Zhang, Z. (2020) · Interspeech (USB) · doi:10.21437/interspeech.2020-2624 · Cited by 18
“However, studies [7, 8] have shown that different emotions have different characteristics in valence-activation space.”
BackgroundUtterance-Based Audio Sentiment Analysis Learned by a Parallel Combination of CNN and LSTM.Preprint
Luo, Z., Xu, H., & Chen, F. (2018) · arXiv (Cornell University) · Cited by 18
Is Cross-Attention Preferable to Self-Attention for Multi-Modal Emotion Recognition?Preprint
Rajan, V., Brutti, A., & Cavallaro, A. (2022) · arXiv (Cornell University) · doi:10.48550/arxiv.2202.09263 · Cited by 18
“A self-attention mechanism computes the representation of a uni-modal sequence by relating different positions of the same sequence [6, 12].”
Background", Mihalache, Ş., & Burileanu, D. (2023) · Romanian Journal of Information Science and Technology · doi:10.59277/romjist.2023.3-4.10 · Cited by 17
Combining wav2vec 2.0 Fine-Tuning and ConLearnNet for Speech Emotion Recognition
Sun, C., Zhou, Y., Huang, X. M., Yang, J., & Hou, X. (2024) · Electronics · doi:10.3390/electronics13061103 · Cited by 17
Self-citationSpeech-based emotion recognition using a hybrid RNN-CNN network
Ning, J., & Zhang, W. (2024) · Signal Image and Video Processing · doi:10.1007/s11760-024-03574-7 · Cited by 17
“Moreover, Chen et al.’s work [20] examined the use of attention-based convolution RN N to represent emotional aspects in the E MO DB dataset, obtaining a SE R accuracy of 82 .”
Creation and Analysis of Emotional Speech Database for Multiple Emotions Recognition
Sato, R., Sasaki, R., Suga, N., & Furukawa, T. (2020) · International Conference on Speech Database and Assessments · doi:10.1109/o-cocosda50338.2020.9295041 · Cited by 17
“The other is to estimate which emotions belong to a specific emotion category such as “joy”, “anger”, or “sadness” [4], [7].”
BackgroundMethodInfluentialImproving Convolutional Recurrent Neural Networks for Speech Emotion Recognition
Meyer, P., Xu, Z., & Fingscheidt, T. (2021) · IEEE Spoken Language Technology Workshop (SLT) · doi:10.1109/slt48900.2021.9383513 · Cited by 17
“First, we extend the static log-mel spectrogram by determining the first and second order derivatives of the J mel =40 log-mel coeffi-cients as proposed by Chen et al. [24].”
MethodSpeech emotion recognition based on convolutional neural network
Chen, J. (2021) · International Conference on Networking, Communications and Information Technology (NetCIT) · doi:10.1109/netcit54147.2021.00028 · Cited by 17
Improving Speech Emotion Recognition Through Focus and Calibration Attention Mechanisms
Kim, J., An, Y., & Kim, J. (2022) · Interspeech (USB) · doi:10.21437/interspeech.2022-299 · Cited by 17
Ramaswamy, S. L., & Chinnappan, J. (2023) · Journal of Intelligent & Fuzzy Systems · doi:10.3233/jifs-230917 · Cited by 17
Cross-Domain Transfer Learning for Complex Emotion Recognition
Nagarajan, B., & Murthy, O. V. R. (2019) · IEEE Region 10 Symposium (TENSYMP) · doi:10.1109/tensymp46218.2019.8971023 · Cited by 17
“[10] hypothesized that Mel-spectrogram with deltas and delta-deltas preserves the effective emotional information and reduces the effect of emotionally irrelevant frames.”
BackgroundSpeech emotion classification using feature-level and classifier-level fusion
Mishra, S. P., Warule, P., & Deb, S. (2023) · Evolving Systems · doi:10.1007/s12530-023-09550-9 · Cited by 16
“…2022), medical signal processing (Fu et al. 2010; Warule et al. 2022), etc. Similarly, the spectro-gram (Sajjad and Kwon 2020; Badshah et al. 2017) and mel-spectrogram (Chen et al. 2018; Issa et al. 2020) are popular time-frequency (TF) features used by many researchers to classify speech emotion.”
Single- and Cross-Lingual Speech Emotion Recognition Based on WavLM Domain Emotion Embedding
Yang, J., Liu, J., Huang, K., Xia, J., Zhu, Z., & Zhang, H. (2024) · Electronics · doi:10.3390/electronics13071380 · Cited by 16
“LLDs have been used in studies [12–14], while the mel spectrogram has been used in studies such as [15–19].”
MethodSelf-citationAPIN: Amplitude- and phase-aware interaction network for speech emotion recognition
Guo, L., Li, J., Ding, S., & Dang, J. (2025) · Speech Communication · doi:10.1016/j.specom.2025.103201 · Cited by 16
A Survey of Classification Techniques in Speech Emotion Recognition
Roy, T., Marwala, T., & Chakraverty, S. (2020) · Wiley eBooks · doi:10.1002/9781119585640.ch3 · Cited by 16
Analysis of constant-Q filterbank based representations for speech emotion recognition
Singh, P., Waldekar, S., Sahidullah, M., & Saha, G. (2022) · Digital Signal Processing · doi:10.1016/j.dsp.2022.103712 · Cited by 16
“In [70], an attention-based convolutional recurrent neural network (ACRNN) was proposed for SER. Inspired by this, we designed an architecture consisting of only CNN and LSTM layers.”
MethodYalamanchili, B., Kumar, S. S., & Rao, A. K. (2023) · Journal of Information and Communication Technology · doi:10.32890/jict2023.22.1.3 · Cited by 16
Multi-cultural speech emotion recognition using language and speaker cues
Pandey, S., Shekhawat, H. S., & Prasanna, S. R. M. (2023) · Biomedical Signal Processing and Control · doi:10.1016/j.bspc.2023.104679 · Cited by 16
Supervised Contrastive Learning with Nearest Neighbor Search for Speech Emotion Recognition
Wang, X., Zhao, S., & Qin, Y. (2023) · Interspeech (USB) · doi:10.21437/interspeech.2023-842 · Cited by 16
“After fine-tuning with the help of SCL loss, the boundaries between different emotions become clearer, especially for happy and neutral emotions, which are recognized as more difficult emotions to distinguish in previous researches [24].”
BackgroundNon-Uniform Attention Network for Multi-modal Sentiment Analysis
Wang, B., Dong, G., Zhao, Y., Li, R., Cao, Q., & Chao, Y. (2022) · Lecture notes in computer science · doi:10.1007/978-3-030-98358-1_48 · Cited by 16
PulseEmoNet: Pulse emotion network for speech emotion recognition
Zhang, H., Tang, G., Huang, H., Zhu, Y., & Li, Z. (2025) · Biomedical Signal Processing and Control · doi:10.1016/j.bspc.2025.107687 · Cited by 15
Self-citationA wide dynamic range gas analysis model with deep learning based on cavity ring-down spectroscopy
Tang, R., Song, Y., Zhang, H., & Zhou, S. (2025) · Sensors and Actuators B Chemical · doi:10.1016/j.snb.2025.137575 · Cited by 15
Self-citationA Method upon Deep Learning for Speech Emotion Recognition
Pham, N. T., Dang, D. N. M., & Nguyen, S. D. (2020) · Journal of Advanced Engineering and Computation · doi:10.25073/jaec.202044.311 · Cited by 15
“Besides, our proposed method was more accurate 12%, 36%, 1%, 17%, and 7% than [17] in the angry, happy, fear, bored, and neutral emotional states, respectively.”
BackgroundMethodResultInfluentialUnsupervised Feature Learning for Speech Emotion Recognition Based on Autoencoder
Ying, Y., Tu, Y., & Zhou, H. (2021) · Electronics · doi:10.3390/electronics10172086 · Cited by 15
“[21] proposed to use three dimensions—MFCC, deltas, and delta-deltas of MFCC—as input, and adopted RCNN to extract emotional features from 3-D dimensional input.”
MethodTowards Noise Robust Speech Emotion Recognition Using Dynamic Layer Customization
Wilf, A., & Provost, E. M. (2021) · International Conference on Affective Computing and Intelligent Interaction and workshops · doi:10.1109/acii52823.2021.9597437 · Cited by 15
“Here we use Unweighted Average Recall (UAR) as our metric, in line with work in the field [10], [24].”
MethodGraph based emotion recognition with attention pooling for variable-length utterances
Liu, J., Wang, H., Sun, M., & Wei, Y. (2022) · Neurocomputing · doi:10.1016/j.neucom.2022.05.007 · Cited by 15
Guo, Y., Yong-ping, Z., Xiong, X., Jiang, X., Tian, H., & Zhang, Q. (2023) · IEEE Access · doi:10.1109/access.2023.3299822 · Cited by 15
“Chen et al. [11] proposed a 3D CNN-LSTM model by inputting the MFCC coefficients and first-order and second-order differences together as multi-channel inputs to the model, thus bringing the unweighted average recall rate of IEMOCAP and EMODB data sets to 64.74% and 82.82% respectively.”
MethodInfluentialLearning Salient Segments for Speech Emotion Recognition Using Attentive Temporal Pooling
Xia, X., Jiang, D., & Sahli, H. (2020) · IEEE Access · doi:10.1109/access.2020.3014733 · Cited by 15
“Chen et al. [27] first split each utterance into equal-length segments of 3 seconds and computed the log-Mels with first and second derivatives to obtain a 3D feature representation for each segment.”
MethodInfluentialSingh, Y. B., & Goel, S. (2021) · Multimedia Tools and Applications · doi:10.1007/s11042-020-10399-2 · Cited by 15
Retracted: Controlling Email System Using Audio with Speech Recognition and Text to Speech
Shah, H. D., Sundas, A., & Sharma, S. (2021) · International Conference on Reliability, Infocom Technologies and Optimization (Trends and Future Directions) (ICRITO) · doi:10.1109/icrito51393.2021.9596293 · Cited by 15
RetractedTime-Frequency Attention for Speech Emotion Recognition with Squeeze-and-Excitation Blocks
Liu, K., Wang, C., Chen, J., & Feng, J. H. (2022) · Lecture notes in computer science · doi:10.1007/978-3-030-98358-1_42 · Cited by 15
Exploring Sequential Feature Selection in Deep Bi-LSTM Models for Speech Emotion Recognition
Harby, F., Alohali, M., Thaljaoui, A., & Talaat, A. S. (2024) · Computers, materials & continua/Computers, materials & continua (Print) · doi:10.32604/cmc.2024.046623 · Cited by 14
Enhanced Speech Emotion Recognition Using Conditional-DCGAN-Based Data Augmentation
Roh, K.-M., & Lee, S.-P. (2024) · Applied Sciences · doi:10.3390/app14219890 · Cited by 14
Speaker-Independent Emotional Voice Conversion via Disentangled Representations
Chen, X., Xu, X., Chen, J., Zhang, Z., Takiguchi, T., & Hancock, E. R. (2022) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2022.3222646 · Cited by 14
“For a fair comparison, we used a third-party pre-trained speech emotion recognition model [56] to classify the emotion from the converted speech.”
MethodBillah, M., Sarker, M. L., Akhand, M. A. H., & Kamal, M. A. S. (2024) · International Journal of Advanced Computer Science and Applications · doi:10.14569/ijacsa.2024.0150460 · Cited by 14
“Chen et al. [8] used a portion of the IEMOCAP dataset for three different models: a shallow CNN combined with a Bi-LSTM, a deep CNN with a shallow Bi-LSTM, and a deep CNN with a deep Bi-LSTM.”
MethodMeng, H., Yan, T., Wei, H., & Ji, X. (2021) · Bulletin of the Polish Academy of Sciences Technical Sciences · doi:10.24425/bpasts.2020.136300 · Cited by 14
“Chen [6] used the functional variables that included deltas and delta-deltas of the log mel-spectrogram (Log-Mels) as the CNN input based on the attention mechanism to help to link the time-frequency relationship.”
MethodTripartite interaction representation learning for multi-modal sentiment analysis
Wang, B., Dong, G., Zhao, Y., Li, R., Yin, W., & Lu, L. (2025) · Expert Systems with Applications · doi:10.1016/j.eswa.2024.126279 · Cited by 13
Karthikeyan, V., Priyadharsini, S. S., & Balamurugan, K. (2025) · Multimedia Tools and Applications · doi:10.1007/s11042-025-20694-5 · Cited by 13
“Because of these benefits, the attention strategy is well-suited for incorporation into CNN frameworks [26, 27].”
Gudivaka, S. R., Polipogu, P., Kodali, R., Dhulipalla, V. R., Tatavarty, V. S. K., Penumudi, J., & Gogulamudi, P. R. (2026) · International Journal of Speech Technology · doi:10.1007/s10772-025-10229-6 · Cited by 13
“GRUs achieve com - parable performance to LSTMs in SER while being compu - tationally more efficient.”
InfluentialSpeaker-Independent Speech Emotion Recognition Based on CNN-BLSTM and Multiple SVMs
Liu, Z., Xiao, P., Li, D., & Hao, M. (2019) · Lecture notes in computer science · doi:10.1007/978-3-030-27535-8_43 · Cited by 13
Dimensional Emotion Prediction Based on Interactive Context in Conversation
Shi, X., Li, S., & Dang, J. (2020) · Interspeech (USB) · doi:10.21437/interspeech.2020-1820 · Cited by 13
“We use the Mel filterbanks [9][10] to extract the spectrogram feature.”
MethodSDTF-Net: Static and dynamic time–frequency network for Speech Emotion Recognition
Liu, L., Liu, W., & Feng, L. (2023) · Speech Communication · doi:10.1016/j.specom.2023.01.008 · Cited by 13
Hierarchical Network with Decoupled Knowledge Distillation for Speech Emotion Recognition
Zhao, Z., Wang, H., Wang, H., & Schuller, B. W. (2023) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp49357.2023.10095045 · Cited by 13
“[8] proposed a 3-D ACRNN model that combines CNN, Bi-LSTM, and attention mechanisms.”
MethodEmotion Classification from Speech by an Ensemble Strategy
Novais, R., Cardoso, P. J. S., & Rodrigues, J. M. F. (2022) · International Conference on Software Development and Technologies for Enhancing Accessibility and Fighting Info-exclusion (DSAI) · doi:10.1145/3563137.3563170 · Cited by 12
“[16] presented a 3D attention-based convolutional recurrent neural network to learn discriminative features for SER, where the Mel-spectrogram with deltas and delta-deltas are used as input.”
MethodPark, S., Mark, M., Park, B., & Hong, H. (2023) · Computers, materials & continua/Computers, materials & continua (Print) · doi:10.32604/cmc.2023.041332 · Cited by 12
MVIB-DVA: Learning minimum sufficient multi-feature speech emotion embeddings under dual-view aware
Li, G., Hou, J., Liu, Y., & Wei, J. (2024) · Expert Systems with Applications · doi:10.1016/j.eswa.2023.123110 · Cited by 12
Diemerling, H., Stresemann, L., Braun, T., & von Oertzen, T. (2024) · Frontiers in Psychology · doi:10.3389/fpsyg.2024.1300996 · Cited by 12
SEMOUR: A Scripted Emotional Speech Repository for Urdu
Zaheer, N., Ahmad, O. U., Ahmed, A., Khan, M. S., & Shabbir, M. (2021) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3411764.3445171 · Cited by 12
“In the end, the weighted average is taken of the output using a fully connected neural network layer [11, 54].”
MethodXue, J., Li, W., Zhang, Y., Xiao, H., Tan, R., Xing, Y., Guo, G., & Cao, D. (2021) · CAA International Conference on Vehicular Control and Intelligence (CVCI) · doi:10.1109/cvci54083.2021.9661268 · Cited by 12
Goel, D. P., Mahajan, K., Nguyen, N. D., Natesan, S., & Lim, C. P. (2022) · Neural Computing and Applications · doi:10.1007/s00521-022-07723-2 · Cited by 12
Speech Emotion Recognition Based on Swin-Transformer
Liao, Z., & Shen, S. (2023) · Journal of Physics Conference Series · doi:10.1088/1742-6596/2508/1/012056 · Cited by 12
Gated Variational AutoEncoders: Incorporating Weak Supervision to Encourage Disentanglement
Vowels, M. J., Camgöz, N. C., & Bowden, R. (2020) · International Conference on Automatic Face and Gesture Recognition · doi:10.1109/fg47880.2020.00016 · Cited by 12
“These include Separated Attribute Predictability [5], Mutual Information Gap [15], FactorVAE metric [11], Modularity [16], the β-vae metric [4] and the later relative of the β-vae metric [9].”
Feature-based hybrid strategies for gradient descent optimization in end-to-end speech recognition
DOKUZ, Y., & TÜFEKCİ, Z. (2022) · Multimedia Tools and Applications · doi:10.1007/s11042-022-12304-5 · Cited by 12
“Also, new hybrid deep learning architectures were proposed to improve speech recognition performance, such as CNNBLSTM-CTC [32], CLDNN [28], 3D attention-based CRNN [2], DCNN [7, 29] CNNLSTM [14, 30], and LSTM-Convolutional-BiLSTM Encoder Decoder [33].”
MethodSigWavNet: Learning Multiresolution Signal Wavelet Network for Speech Emotion Recognition
Nfissi, A., Bouachir, W., Bouguila, N., & Mishara, B. L. (2025) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2025.3537991 · Cited by 11
“The study by [62] introduces a 3-”
Kim, B., & Kwon, Y. (2025) · Scientific Reports · doi:10.1038/s41598-025-19887-7 · Cited by 11
“M. Chen et al. [15] proposed a CNN-LSTM-Attention model to aggregate the hidden states in each time step.”
Shi, X., He, J., Li, X., & Toda, T. (2026) · IEEE Transactions on Audio Speech and Language Processing · doi:10.1109/taslpro.2026.3654273 · Cited by 11
“For the SSL baseline (Wav2Vec 2.0), UAR and F1 rise from 45.63% and 45.22% at –10 dB to 65.83% and 64.80% at 10 dB, corresponding to improvements of 20.20% and 19.58%.”
InfluentialInvestigation of the Effect of Increased Dimension Levels in Speech Emotion Recognition
Wang, H., Zhao, X., & Zhao, Y. (2022) · IEEE Access · doi:10.1109/access.2022.3194039 · Cited by 11
Speech Emotion Recognition Via Two-Stream Pooling Attention With Discriminative Channel Weighting
Liu, K., Wang, D., Wu, D., & Feng, J. (2023) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp49357.2023.10095588 · Cited by 11
“[20] 61.32 60.43 CMRN [1] − 66.64 Wav2vec2-PT [12] 67.90 − Cross-representation [21] 70.10 70.70 Co-attention [2] 69.80 71.05 Ours 72.08 72.17 3D ACRNN [22] − 64.74 BLSTM-CTC-CA [23] 69.00 67.00 HNSD [24] 70.05 72.50 e2e ASR [25] 71.72 72.56 Co-attention [2] 71.64 72.70 Ours 73.18 74.26”
MethodInfluentialRizhinashvili, D., Sham, A. H., & Anbarjafari, G. (2024) · Signal Image and Video Processing · doi:10.1007/s11760-024-03406-8 · Cited by 11
“In the subsequent years, this hybrid modeling approach has been extensively explored and refined, as evidenced by various studies [17–22].”
Mixed-EVC: Mixed Emotion Synthesis and Control in Voice Conversion
Zhou, K., Şişman, B., Busso, C., Ma, B., & Li, H. (2024) · The Speaker and Language Recognition Workshop (Odyssey) · doi:10.21437/odyssey.2024-26 · Cited by 11
“This SER model shares a similar structure to the one presented in [41], encompassing components such as a 3-”
MethodA Comparative Study of Data Augmentation Techniques for Deep Learning Based Emotion RecognitionPreprint
Shankar, R., Kenfack, A. H., Somayazulu, A., & Venkataraman, A. (2022) · arXiv (Cornell University) · doi:10.48550/arxiv.2211.05047 · Cited by 11
“D convolutional model using the first and second order derivative of MFCCs has also been proposed in the literature by [15].”
MethodOne-shot emotional voice conversion based on feature separation
Lu, W., Zhao, X., Guo, N., Li, Y., Wei, J., Tao, J., & Dang, J. (2022) · Speech Communication · doi:10.1016/j.specom.2022.07.001 · Cited by 10
Sequential Modeling by Leveraging Non-Uniform Distribution of Speech Emotion
Lin, W.-C., & Busso, C. (2023) · IEEE/ACM Transactions on Audio Speech and Language Processing · doi:10.1109/taslp.2023.3244527 · Cited by 10
“information in the features [30], [38].”
Semisupervised Complex Network With Spatial Statistics Fusion for PolSAR Image Classification
Jiang, Y., Li, M., Zhang, P., & Song, W. (2023) · IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing · doi:10.1109/jstars.2023.3325365 · Cited by 10
“…hardware, deep learning models have achieved great success in many fields, such as image processing [17], [18], [19], [20], speech recognition [21], [22], [23], natural language processing [24], [25], [26], etc. Deep learning models can automatically extract representative deep features through…”
BackgroundConvoluational Transformer With Adaptive Position Embedding For Covid-19 Detection From Cough Sounds
Yan, T., Meng, H., Liu, S., Parada‐Cabaleiro, E., Ren, Z. X., & Schuller, B. W. (2022) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp43922.2022.9747513 · Cited by 10
“Hence, we extract delta and delta-deltas based on the static spectrogram and jointly form a 3-channel input feature M ∈ R t,f,c [ 20 , 21 ], where t denotes time, f describes the number of filters, and c represents the number of channels, which is set to three: (cid:4) (cid:5) where d stands for the…”
Adapting Multiple Distributions for Bridging Emotions from Different Speech Corpora
Zong, Y., Lian, H., Chang, H., Lu, C., & Tang, C. (2022) · Entropy · doi:10.3390/e24091250 · Cited by 10
Attention-based Interactive Disentangling Network for Instance-level Emotional Voice Conversion
Chen, Y., Yang, L., Chen, Q., Lai, J., & Xie, X. (2023) · Interspeech (USB) · doi:10.21437/interspeech.2023-39 · Cited by 10
“Besides, we use a pre-trained emotion classifier ACRNN [29] to evaluate the accuracy (ACC cls ) of converted speech.”
MethodLanguage Reinforced Superposition Multimodal Fusion for Sentiment Analysis
He, J., & Hu, H. (2022) · IEEE Signal Processing Letters · doi:10.1109/lsp.2022.3180668 · Cited by 10
Light-SERNet: A lightweight fully convolutional neural network for speech emotion recognitionPreprint
Aftab, A., Morsali, A., Ghaemmaghami, S., & Champagne, B. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2110.03435 · Cited by 10
MSAC: Multiple Speech Attribute Control Method for Reliable Speech Emotion RecognitionPreprint
Pan, Y., Yang, Y., Huang, Y., Yao, J., Yin, J., Hu, Y., Lu, H., Ma, L., & Zhao, J. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2308.04025 · Cited by 10
“For example, [18] proposed an attention-based convolutional recurrent neural network to extract discriminative reprsentations for emotion recognition using Mel-spectrogram with deltas and delta-deltas.”
BackgroundMethodNaturalVoices: A Large-Scale, Spontaneous and Emotional Podcast Dataset for Voice ConversionPreprint
Du, Z., Chandra, S. S., Ülgen, İ. R., Mahapatra, A., Salman, A. N., Busso, C., & Şişman, B. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2511.00256 · Cited by 10
“To assess how well the converted speech preserved the target emotional state, we used two complementary metrics: • Emotion Category Accuracy (ECA) [107]: The emotion of generated speech was classified using a pre-trained model (emotion2vec) [108] and compared to the reference label.”
Li, X., & Zhang, Z. (2024) · Digital Communications and Networks · doi:10.1016/j.dcan.2024.10.007 · Cited by 9
Wang, F., Jiang, Z., Jiang, J., Yao, Y., & Liu, Y. (2025) · Quantitative InfraRed Thermography Journal · doi:10.1080/17686733.2025.2592191 · Cited by 9
“Three-dimensional (3D) convolution, extensively used in video and medical image processing, addressed this limitation by enabling simultaneous spatio-temporal feature extraction [21][21– 23].”
Short video fingerprint extraction: from audio–visual fingerprint fusion to multi-index hashing
Zhang, S., Zhang, J., Wang, Y., & Li, Z. (2022) · Multimedia Systems · doi:10.1007/s00530-022-01031-4 · Cited by 9
Converting Anyone's Voice: End-to-End Expressive Voice Conversion with A Conditional Diffusion Model
Du, Z., Lu, J., Zhou, K., Kaushik, L., & Şişman, B. (2024) · The Speaker and Language Recognition Workshop (Odyssey) · doi:10.21437/odyssey.2024-25 · Cited by 9
A Lightweight Multi-Scale Model for Speech Emotion Recognition
Li, H., Zhao, D., Wang, J., & Wang, D. (2024) · IEEE Access · doi:10.1109/access.2024.3432813 · Cited by 9
“The results of the proposed model and the baselines considered [10], [11], [13], [29], [30], [34], [36], [37], [38] on IEMOCAP, EMO-DB and SAVEE datasets are given in Table 10, Table 11 Firstly, we compare the number of parameters of the proposed model with those of the baselines according to Table…”
MethodInfluentialNavigating the Diverse Challenges of Speech Emotion Recognition: A Deep Learning Perspective
Luo, S. (2024) · International Academic Mindtrek Conference (Mindtrek) · doi:10.1145/3681716.3681725 · Cited by 9
Spectro-Temporal Modulations Incorporated Two-Stream Robust Speech Emotion Recognition
Shen, Y.-L., Hsieh, P.-C., & Chi, T.-S. (2025) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2025.3531638 · Cited by 9
“The ACRNN architecture has been demonstrated to effectively extract information from log Mel-spectrograms [14], while spectro-temporal modulation features can be considered as fil-tered spectrograms.”
InfluentialHierarchical Control of Emotion Rendering in Speech Synthesis
Inoue, S., Zhou, K., Wang, S., & Li, H. (2025) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2025.3582715 · Cited by 9
“We trained the SER model [68] on the ESD dataset, with data augmentation of Gaussian noise and time and frequency masking strategy [69], [70], [71], [72].”
Emotional State Modeling for the Assessment of Depression in Parkinson’s Disease
Pérez-Toro, P. A., Vásquez-Correa, J. C., Arias‐Vergara, T., Klumpp, P., Schuster, M., Nöth, E., & Orozco‐Arroyave, J. R. (2021) · Lecture notes in computer science · doi:10.1007/978-3-030-83527-9_39 · Cited by 9
Speech Emotion Recognition Using Semi-Supervised Learning with Efficient Labeling Strategies
Zhu, Z., & Sato, Y. (2021) · IEEE Workshop on Automatic Speech Recognition and Understanding (ASRU) · doi:10.1109/asru51503.2021.9687938 · Cited by 9
Adibi, P., Kalani, S., Zahabi, S. J., Asadi, H., Bakhtiar, M., Heidarpour, M. R., Roohafza, H., Shahoon, H., & Amouzadeh, M. (2023) · World Journal of Psychiatry · doi:10.5498/wjp.v13.i1.1 · Cited by 9
Metadata
- Authors
- Chen, M., He, X., Yang, J., & Zhang, H.
- Affiliations
- Central South University (CN)
- Venue
- IEEE Signal Processing Letters
- Published
- 2018-10
- Volume
- 25
- Issue
- 10
- Pages
- 1440-1444
- ISSN
- 1070-9908 (ISSN-L)
- 1558-2361
- OpenAlex
- W2885005742
- Cited by
- 503
- References
- 23
- Citations by year
- 2026: 21 · 2025: 50 · 2024: 70 · 2023: 85 · 2022: 91 · 2021: 94 · 2020: 60 · 2019: 27 · 2018: 5
- FWCI
- 60.91
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- IEEE Signal Processing Letters (publishedVersion)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Emotion and Mood Recognition
- Physical Sciences › Computer Science › Artificial Intelligence › Speech Recognition and Synthesis
- Physical Sciences › Computer Science › Signal Processing › Speech and Audio Processing
- Keywords
- speech emotion recognition, attention mechanism, Mel spectrogram, IEMOCAP, EMO-DB
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.