3-D Convolutional Recurrent Neural Networks With Attention Model for Speech Emotion Recognition
The DOI matches its Crossref record; 23 of the 23 references it cites were found in a registry or an index.
Abstract
Speech emotion recognition (SER) is a difficult task due to the complexity of emotions. The SER performances are heavily dependent on the effectiveness of emotional features extracted from the speech. However, most emotional features are sensitive to emotionally irrelevant factors, such as the speaker, speaking styles, and environment. In this letter, we assume that calculating the deltas and delta-deltas for personalized features not only preserves the effective emotional information but also reduces the influence of emotionally irrelevant factors, leading to reduce misclassification. In addition, SER often suffers from the silent frames and emotionally irrelevant frames. Meanwhile, attention mechanism has exhibited outstanding performances in learning relevant feature representations for specific tasks. Inspired by this, we propose a three-dimensional attention-based convolutional recurrent neural networks to learn discriminative features for SER, where the Mel-spectrogram with deltas and delta-deltas are used as input. Experiments on IEMOCAP and Emo-DB corpus demonstrate the effectiveness of the proposed method and achieve the state-of-the-art performance in terms of unweighted average recall.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Another issue faced by the time-frequency representation of speech emotion is that the distribution of emotion information in an utterance is sparse in both frequency domain and time domain [19], [23].”
From Speech Emotion Recognition via an Attentive Time–Frequency Neural Network (2023) · cites this paper as [19]
In the citing paper
“However, these studies mainly embed attention into highlevel time-frequency features, leading to introduce the redundant information in the feature extraction and roughly locate the emotion-related regions in the time-frequency domain.”
From Speech Emotion Recognition via an Attentive Time–Frequency Neural Network (2023) · cites this paper as [19]
In the citing paper
“In addition, they mainly focus on the selection on the time frames while ignoring the selection on the frequency bands.”
From Speech Emotion Recognition via an Attentive Time–Frequency Neural Network (2023) · cites this paper as [19]
In the citing paper
“ignoring to capture the specific frequency bands associated with emotions”
From Speech Emotion Recognition via an Attentive Time–Frequency Neural Network (2023) · cites this paper as [19]
Citations
Cites
Adam: A Method for Stochastic Optimization
Kingma, D. P., & Ba, J. (2014) · arXiv · doi:10.48550/arxiv.1412.6980 · Cited by 98,946 · cited 1×
“toolkit [24], and the parameters of the model were optimized by minimizing the cross-entropy objective function, with a minibatch of 40 samples, using the Adam optimizer [25] with Nestorov momentum.”
MethodNeural Machine Translation by Jointly Learning to Align and Translate
Bahdanau, D., Cho, K., & Bengio, Y. (2014) · arXiv · doi:10.48550/arxiv.1409.0473 · Cited by 16,213 · cited 1×
“proposed in [9] for machine translation and later applied for”
MethodTensorFlow: Large-Scale Machine Learning on Heterogeneous Distributed Systems
Abadi, M., Agarwal, A., Barham, P., Brevdo, E., Chen, Z., Citro, C., Corrado, G. S., Davis, A., Dean, J., Devin, M., et al. (2016) · arXiv · doi:10.48550/arxiv.1603.04467 · Cited by 9,408
Hierarchical Attention Networks for Document Classification
Yang, Z., Yang, D., Dyer, C., He, X., Smola, A., & Hovy, E. (2016) · Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies · doi:10.18653/v1/n16-1174 · Cited by 4,712 · cited 1×
“document classification [10] and speech recognition [11].”
Self-citationIEMOCAP: Interactive emotional dyadic motion capture database
Busso, C., Bulut, M., Lee, C.-C., Kazemzadeh, A., Mower, E., Kim, S., Chang, J. N., Lee, S., & Narayanan, S. S. (2008) · Language Resources and Evaluation · doi:10.1007/s10579-008-9076-6 · Cited by 3,869 · cited 12×
“D ACRNN obtains an absolute improvement of 2.34% and 3.44% for IEMOCAP and Emo-DB, respectively.”
MethodInfluentialA database of German emotional speech
Burkhardt, F., Paeschke, A., Rolfes, M., Sendlmeier, W. F., & Weiss, B. (2005) · Interspeech 2005 · doi:10.21437/interspeech.2005-446 · Cited by 2,365 · cited 1×
“form speaker-independent SER experiments on the Interactive Emotional Dyadic Motion Capture database (IEMOCAP) [21] and the Berlin Emotional Database (Emo-DB) [22].”
MethodAttention-Based Models for Speech Recognition
Chorowski, J., Bahdanau, D., Serdyuk, D., Cho, K., & Bengio, Y. (2015) · arXiv · doi:10.48550/arxiv.1506.07503 · Cited by 2,021
Bidirectional LSTM Networks for Improved Phoneme Classification and Recognition
Graves, A., Fernández, S., & Schmidhuber, J. (2005) · Lecture Notes in Computer Science · doi:10.1007/11550907_126 · Cited by 925 · cited 1×
“bidirectional RNN with long short-term memory cells [19] for temporal summarization, and each direction contains 128 cells; then, we can obtain a sequence of 256-dimensional high-level”
MethodAdieu features? End-to-end speech emotion recognition using a deep convolutional recurrent network
Trigeorgis, G., Ringeval, F., Brueckner, R., Marchi, E., Nicolaou, M. A., Schuller, B., & Zafeiriou, S. (2016) · 2016 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) · doi:10.1109/icassp.2016.7472669 · Cited by 904 · cited 1×
“In recent years, CNN has exhibited excellent performances in the field of SER [1], [4], [6].”
BackgroundSpeech emotion recognition using deep neural network and extreme learning machine
Han, K., Yu, D., & Tashev, I. (2014) · Interspeech 2014 · doi:10.21437/interspeech.2014-57 · Cited by 794 · cited 4×
“1) DNN-extreme learning machine (DNN-ELM) ( see [3]).”
MethodInfluential
Research works printed but not linked: 2
- wei, 2016, Attention assisted discovery of sub-utterance structure in speech emotion recognition, Proc INTERSPEECH, 1387
- frano, 2016, Normalization and batch correction methods for high-throughput glycomics, Proc Joint Meet Soc Glycobiol, 1160
Cited by
Items 301–400 of 549
Deep ensemble models for speech emotion classification
Pravin, S. C., Sivaraman, V. B., & Saranya, J. D. (2023) · Microprocessors and Microsystems · doi:10.1016/j.micpro.2023.104790 · Cited by 9
Liu, Y., Sun, H., Guan, W., Xia, Y., & Zhao, Z. (2023) · Machine Intelligence Research · doi:10.1007/s11633-022-1356-x · Cited by 9
Speech Emotion Recognition with Discriminative Feature Learning
Zhou, H., & Liu, K. (2020) · Interspeech (USB) · doi:10.21437/interspeech.2020-2237 · Cited by 9
“LSTM [11] 57.1 LSTM + Att [12] 59.6 RNN + Att [13] 59.7 CNN + BLSTM [14] 59.4 TDNN + LSTM [4] 60.7 ResNet + MHA [3] 67.4 3D CRNN [1] 64.7 The proposed baseline 62.3 From the Tab.1, it can be observed that our proposed base-line achieves satisfying results by outperforming most prior art.”
MethodResultInfluentialDaneshfar, F., & Kabudian, S. J. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2111.07094 · Cited by 9
A deep tensor-based approach for automatic depression recognition from speech utterances
Pandey, S., Shekhawat, H. S., Prasanna, S. R. M., Bhasin, S., & Jasuja, R. (2022) · PLoS ONE · doi:10.1371/journal.pone.0272659 · Cited by 8
Multimodal attention for lip synthesis using conditional generative adversarial networks
Vidal, A., & Busso, C. (2023) · Speech Communication · doi:10.1016/j.specom.2023.102959 · Cited by 8
Unlocking the Emotional States of High-Risk Suicide Callers through Speech Analysis
Nfissi, A., Bouachir, W., Bouguila, N., & Mishara, B. L. (2024) · IEEE International Conference on Semantic Computing (ICSC) · doi:10.1109/icsc59802.2024.00012 · Cited by 8
“In [16], the authors propose a three-dimensional attention-based convolutional recurrent neural network (CRNN) that uses Mel-spectrogram with deltas and delta-deltas as input, and an attention mechanism to learn discriminative features.”
BackgroundMethodResultInfluentialKawade, R. R., & Jagtap, S. K. (2024) · Revue d intelligence artificielle · doi:10.18280/ria.380318 · Cited by 8
“…dataset, higher recognition rate, etc. various deep learning techniques has been presented successfully for the SER such as Auto encoder (AE) [19], Convolution Neural Network (CNN) [20], Deep Belief Network (DBN) [21], Long-Short Term Memory (LSTM) [22], Recurrent Neural Network (RNN) [23], etc.”
MethodRen, F., Liu, Z., & Kang, X. (2022) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2022.3192899 · Cited by 8
“The papers [23] [48] [49] [50] [51] use high-level features extracted from acoustic features, [50] directly extract original audio, and [51] [52] [53] reconstruct the features by using representation learning.”
BackgroundChauhan, K., Sharma, K. K., & Varma, T. (2022) · Multimedia Tools and Applications · doi:10.1007/s11042-022-13463-1 · Cited by 8
“The initial development of speech emotion recognition(SER) systems was based on hand-crafted features from the audio signal, for example, Mel Frequency cepstrum coefficients (MFCC), Linear predictive coefficients (LPC), Teager energy-related features (TEO), Fourier parameters (FP) etc. [4, 15, 43].”
BackgroundMethodInfluentialMFCC-based deep convolutional neural network for audio depression recognition
Wang, Y., Lu, X., & Shi, D. (2022) · International Conference on Asian Language Processing · doi:10.1109/ialp57159.2022.9961267 · Cited by 8
“According to previous studies, the results obtained from 4s speech fragments are relatively objective [19][20].”
ResultSpeech Emotion Recognition Based on Joint Self-Assessment Manikins and Emotion Labels
Chen, J., Chang, P., & Liang, K.-W. (2019) · IEEE International Symposium on Multimedia (ISM) · doi:10.1109/ism46123.2019.00073 · Cited by 8
“Like [4], we also use Delta and Delta-Delta features as input, they are calculated by the following formula (1) and (2). surveyThus, the dimensions of our input has 3 channels (log-Mels, Delta, and Delta-Delta), and the size of each channel is N (frames) × 40 (features).”
MethodSpeech Emotion Recognition Based on BLSTM and CNN Feature Fusion
Huilian, L., Hu, W., & Wang, Y. (2020) · IEEE International Conference on Digital Signal Processing (DSP) · doi:10.1145/3408127.3408192 · Cited by 8
“BLSTM network combines the advantages of LSTM network and Bidirectional Recurrent Neural Network (BRNN) to learn the temporal context information of speech sequence data [1, 4].”
MethodInfluentialSpeaker Attentive Speech Emotion RecognitionPreprint
Veillon, C. L. M., Obin, N., & Röebel, A. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2104.07288 · Cited by 8
“[10] as input of their ACRNN model and later used in [11].”
BackgroundMethodInfluentialPham, N. T., Dang, D. N. M., & Nguyen, S. D. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2109.09026 · Cited by 8
“The other studies applied an attention mechanism to the SER to obtain the most utterance features that represent the corresponding emotion [24]–[26].”
MethodInfluentialLearning Sign Language Representation using CNN LSTM, 3DCNN, CNN RNN LSTM and CCN TDPreprint
Louison, N., Goodridge, W. S., & Khan, K. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2412.18187 · Cited by 8
Performance Analysis of a Chunk-Based Speech Emotion Recognition Model Using RNN
Shin, H.-S., & Hong, J.-K. (2022) · Intelligent Automation & Soft Computing · doi:10.32604/iasc.2023.033082 · Cited by 7
Multimodal Emotion Classification Supported in the Aggregation of Pre-trained Classification Models
Cardoso, P. J. S., Rodrigues, J. M. F., & Novais, R. (2023) · Lecture notes in computer science · doi:10.1007/978-3-031-36030-5_35 · Cited by 7
“The Mel-spectrogram with deltas and delta-deltas is utilized as input by Chen et al. [16] in a 3D attention-based convolutional recurrent neural network to learn discriminative features for speech emotion recognition.”
Speech emotion recognition using the novel SwinEmoNet (Shifted Window Transformer Emotion Network)
Ramesh, R., Prahaladhan, V. B., Nithish, P., & Mohanaprasad, K. (2024) · International Journal of Speech Technology · doi:10.1007/s10772-024-10123-7 · Cited by 7
Soltani, R., Benmohamed, E., & Ltifi, H. (2024) · Neural Computing and Applications · doi:10.1007/s00521-024-10519-1 · Cited by 7
“Moreover, our approach outperforms the methods presented in [57–59, 68, 69] on the EMO-DB.”
EmoFusionNet: A unified approach for robust speech emotion recognition
Vijayan, B., & Judy, M. V. (2025) · Digital Signal Processing · doi:10.1016/j.dsp.2025.105173 · Cited by 7
Pre-attentive speech signal processing with adaptive routing for emotion recognition
Zhang, H., Pang, Z., Zhao, P., Tang, G., Shen, L., & Wang, G. (2025) · Biomedical Signal Processing and Control · doi:10.1016/j.bspc.2025.108782 · Cited by 7
Self-citationBaumann, J., Meyer, P., Lohrenz, T., Roy, A., Papendieck, M., & Fingscheidt, T. (2021) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp39728.2021.9414254 · Cited by 7
Convolution neural network with multiple pooling strategies for speech emotion recognition
JIANG, P., & Zou, C. (2022) · International Symposium on Computer Science and Intelligent Control (ISCSIC) · doi:10.1109/iscsic57216.2022.00029 · Cited by 7
“Because spectral features [4], and [5] have more parameters than other manual features, they are usually used as input features.”
BackgroundSpeech Emotion Recognition via Heterogeneous Feature Learning
Liu, K., Wu, D., Wang, D., & Feng, J. (2023) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp49357.2023.10095566 · Cited by 7
“…60.43 CMRN [2] − 66.64 Wav2vec2-PT [11] 67.90 − Cross-representation [21] 70.10 70.70 Co-attention [3] 69.80 71.05 Ours 72.91 73.42 LOSpO 3D ACRNN [22] − 64.74 BLSTM-CTC-CA [23] 69.00 67.00 HNSD [24] 70.05 72.50 e2e ASR [25] 71.72 72.56 Co-attention [3] 71 In our work, each audio utterance is…”
MethodInfluentialStudy of Speech Emotion Recognition Using Blstm with Attention
Sherman, D., Hazan, G., & Gannot, S. (2023) · European Signal Processing Conference (EUSIPCO) · doi:10.23919/eusipco58844.2023.10289866 · Cited by 7
“Log-mel filterbank energies and log-mel spectrogram were used as features in [4] and [10], [12], respectively.”
BackgroundMethodSpeech Emotion Recognition by Combining a Unified First-Order Attention Network With Data Balance
Chen, G., Zhang, S., Tao, X., & Zhao, X. (2020) · IEEE Access · doi:10.1109/access.2020.3038493 · Cited by 7
“(13) [38].”
BackgroundInfluentialXie, B., & Park, C. H. (2023) · ACM/IEEE International Conference on Human-Robot Interaction (HRI) · doi:10.1145/3568294.3580170 · Cited by 7
“Previous studies recognized emotions through facial expressions [16, 34], acoustic features [10, 37], and texts [5, 9].”
BackgroundImproved Speech Emotion Recognition Using Channel-wise Global Head Pooling (CwGHP)
Chauhan, K., Sharma, K. K., & Varma, T. (2023) · Circuits Systems and Signal Processing · doi:10.1007/s00034-023-02367-6 · Cited by 7
“It has been observed in numerous past researches that with the inclusion of the attention algorithm in SER, the recognition task gets better [1, 12, 39].”
BackgroundMethodAutomatic Speech Emotion Recognition Using Hybrid Deep Learning Techniques
Rasheed, B. H., Yuvaraj, D., Alnuaimi, S. S., & Priya, S. S. (2024) · International Journal of Intelligent Systems and Applications in Engineering · Cited by 7
PAD 3-D Speech Emotion Recognition Based on Feature Fusion
Pengfei, X., Houpan, Z., & Zhou, W. (2020) · Journal of Physics Conference Series · doi:10.1088/1742-6596/1616/1/012106 · Cited by 7
“At present, deep learning algorithms such as CNN, LSTM and Attention are widely used in the field of speech emotion recognition [8-12].”
MethodMulti-features learning via Attention-BLSTM for Speech Emotion Recognition
Chakhtouna, A., Sekkate, S., & Adib, A. (2024) · International Symposium on Signal, Image, Video and Communications (ISIVC) · doi:10.1109/isivc61350.2024.10577886 · Cited by 6
“Chen et al. [17] introduced a 3-”
Affective Computing Emotional Body Gesture Recognition: Evolution and the Cream of the Crop
Vaz, P. J., Rodrigues, J. M. F., & Cardoso, P. J. S. (2025) · IEEE Access · doi:10.1109/access.2025.3630563 · Cited by 7
Fusion of Log-Mel spectrogram and acoustic features for driver speech emotion recognition
Lu, L., Liu, C. K., Li, D., Liu, X., Li, Y., Cheng, L., & Zheng, X. (2025) · Expert Systems with Applications · doi:10.1016/j.eswa.2025.131021 · Cited by 6
Speech emotion recognition: A systematic mega-review of techniques and pipelines
Chakhtouna, A., Sekkate, S., & Adib, A. (2026) · Information Fusion · doi:10.1016/j.inffus.2026.104161 · Cited by 6
ML Based Comprehensive Application to Enhance Soft Skills
Bura, A., Joshi, S., Deshpande, S., Chaudhari, R., & Deshpande, K. (2023) · International Conference on Emerging Trends in Engineering and Technology (ICETET) · doi:10.1109/icetet-sip58143.2023.10151569 · Cited by 6
“The method uses personalized features calculated by deltas and delta-deltas and a three-dimensional attention-based convolutional recurrent neural network to learn discriminative features for SER.”
MethodInfluentialUnimodal and Crossmodal Refinement Network for Multimodal Sequence Fusion
Guo, X., Kong, A. W., Zhou, H., Wang, X., & Wang, M. (2021) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2021.emnlp-main.720 · Cited by 6
“…on modality representations in language (Pennington et al., 2014; Devlin et al., 2019; Brown et al., 2020), audio (Degottex et al., 2014; Chen et al., 2018; Li et al., 2019), and vision (He et al., 2016; Woo et al., 2018; Li and Deng, 2020), multimodal learning (Baltrušaitis et al.,…”
BackgroundSpeech Emotion Recognition using Time Distributed 2D-Convolution layers for CAPSULENETS
Yalamanchili, B., Anne, K. R., & Samayamantula, S. K. (2022) · Multimedia Tools and Applications · doi:10.1007/s11042-022-12112-x · Cited by 6
Multiple attention convolutional-recurrent neural networks for speech emotion recognition
Zhang, Z., & Wang, K. (2022) · International Conference on Affective Computing and Intelligent Interaction Workshops and Demos (ACIIW) · doi:10.1109/aciiw57231.2022.10086021 · Cited by 6
“In the EMODB dataset, the UA of this model is 4.8% higher than that of ACRNN [21] using the temporal attention mechanism; 2.0% and 5.6% higher than the WA and UA of BLSTM-DSA [32], respectively; and In the comparison of MA-CRNN [37], although WA did not improve, but UA increased by 1.2%, indicating that this model is…”
BackgroundMethodInfluentialSurana, A., Rathod, M., Gite, S., Patil, S., Kotecha, K. V., Selvachandran, G., Quek, S. G., & Abraham, A. (2023) · Multimedia Tools and Applications · doi:10.1007/s11042-023-16815-7 · Cited by 6
“Chen et al. [13] proposed a three-dimensional attention-based recurrent neural network (RNN) using the Mel spectrogram with deltas as input to learning the unique discriminative features.”
Preethi, V., & Jesi, V. E. (2024) · IEEE Access · doi:10.1109/access.2024.3428336 · Cited by 6
Contactless Human Emotion Analysis Across Different Modalities
Nahid, N., Rahman, A., & Ahad, M. A. R. (2021) · Intelligent systems reference library · doi:10.1007/978-3-030-68590-4_9 · Cited by 6
Suzuki, M., & Qi, J. (2022) · Procedia Computer Science · doi:10.1016/j.procs.2022.09.123 · Cited by 6
Attention Driven Fusion For Multi-Modal Emotion RecognitionPreprint
Priyasad, D., Fernando, T., Denman, S., Sridharan, S., & Fookes, C. (2020) · arXiv (Cornell University) · doi:10.17023/hqsn-6n05 · Cited by 6
CNN-BLSTM with Attention Model for Speech Emotion RecognitionPreprint
Xia, Y., & Zhao, L. (2023) · Research Square · doi:10.21203/rs.3.rs-3392008/v1 · Cited by 6
EmoDescriptor: A hybrid feature for emotional classification in dance movements
Bai, J., Dai, R., Dai, J. P., & Pan, J. (2021) · Computer Animation and Virtual Worlds · doi:10.1002/cav.1996 · Cited by 5
Detection of Road Rage in Vehicle Drivers Based on Speech Feature Fusion
Feng, X., Liu, C., & Chen, Y. (2024) · IEEE Access · doi:10.1109/access.2024.3496652 · Cited by 5
“The widely used speech emotion feature recognition models currently include Gaussian Mixture Model (GMM) [33], SVM [34], Convolu-tional Neural Network and Attention Mechanism Model [30], [31], [32], etc.”
Kodali, M., Ansari, L., Kadiri, S. R., Narayanan, S. S., & Alku, P. (2025) · Speech Communication · doi:10.1016/j.specom.2025.103288 · Cited by 5
Zhu, S., Xie, Y., & Wang, Z. (2025) · Sensors · doi:10.3390/s25237378 · Cited by 5
Karthikeyan, V., & Priyadharsini, S. S. (2026) · Evolving Systems · doi:10.1007/s12530-026-09797-y · Cited by 5
“The 3D-CNN method offers a thorough way out by integrating the advantages of CNNs with an attention mechanism for super-resolution (Chen et al. 2018).”
Building Emotion Identification System from Speech Using CNN-GRU Model
Gopi, A. P., Deepika, P., Gayathri, G., Keerthana, C., & Mounika, B. V. (2025) · Advances in Science, Technology & Innovation/Advances in science, technology & innovation · doi:10.1007/978-3-031-72747-4_8 · Cited by 5
Nfissi, A., Bouachir, W., Bouguila, N., & Mishara, B. L. (2025) · Applied Intelligence · doi:10.1007/s10489-025-06620-0 · Cited by 5
Coughing-Based Recognition of Covid-19 with Spatial Attentive ConvLSTM Recurrent Neural Networks
Yan, T., Meng, H., Parada‐Cabaleiro, E., Liu, S., Song, M., & Schuller, B. W. (2021) · Interspeech (USB) · doi:10.21437/interspeech.2021-630 · Cited by 5
“Then, the proportion of sub-segments successfully predicted (Voting Rule) is computed [13].”
MethodInfluentialSpeech Emotion Recognition Using RA-Gmlp Model on Time–Frequency Domain Features Extracted by TFCM
Sha, M., Yang, W., Wei, F., Lu, Z., Chen, M., Ma, C., Zhang, L., & Shi, H. (2024) · Electronics · doi:10.3390/electronics13030588 · Cited by 5
Self-citationEnhanced speech emotion understanding using advanced attention-centric convolutional networks
Qi, Y., Huang, H., & Zhang, H. (2025) · Biomedical Signal Processing and Control · doi:10.1016/j.bspc.2025.107936 · Cited by 5
Self-citationDeep Learning Approaches for Speech Emotion Recognition
Bhavan, A., Sharma, M., Piplani, M., Chauhan, P., Hitkul, & Shah, R. R. (2020) · Algorithms for intelligent systems · doi:10.1007/978-981-15-1216-2_10 · Cited by 5
Social rhythms measured via social media use for predicting psychiatric symptoms
Yokotani, K., & Takano, M. (2021) · APSIPA Transactions on Signal and Information Processing · doi:10.1017/atsip.2021.17 · Cited by 5
Speech Emotion Recognition using Time-frequency Random Circular Shift and Deep Neural Networks
Xia, S., Fourer, D., García, L., Rouas, J.-L., & Shochi, T. (2022) · Speech prosody · doi:10.21437/speechprosody.2022-119 · Cited by 5
“Commonly used neural architectures for SER are Long-Short Term Memory (LSTM) [8], Auto-Encoder (AE) [9], Deep Neural Network (DNN) [10, 11], Deep Convolutional Neural Network (DCNN) [12] and also attention mechanisms [13, 14].”
Generative Adversarial Network-Based Neural Audio Caption Model for Oral Evaluation
Zhang, L., Shu, C., Guo, J., Zhang, H., Xie, C., & Liu, Q. (2020) · Electronics · doi:10.3390/electronics9030424 · Cited by 5
“Next, according to the MFCC features of the first N frames and the next N frames of each frame, we calculate the deltas and delta-deltas for personalized features that preserve dynamic information effectively [32].”
MethodSelf-citationInferring Association Between Alcohol Addiction and Defendant's Emotion Based on Sound at Court
Song, Y. S., & Wei, Z. (2021) · Frontiers in Psychology · doi:10.3389/fpsyg.2021.669780 · Cited by 5
“Many existing studies have shown that the performance of SER task can be improved by converting the speech signal sequence data into log-Mels spectrogram data (Zhang et al., 2017; Chen et al., 2018; Dai et al., 2019; Zhao et al., 2019; Zhou et al., 2019; Chen and Zhao, 2020; Zayene et al., 2020).”
BackgroundMethodInfluentialSpeech Emotion Recognition with Global-Aware Fusion on Multi-scale Feature RepresentationPreprint
Zhu, W., & Li, X. (2022) · arXiv (Cornell University) · doi:10.48550/arxiv.2204.05571 · Cited by 4
“Then, by viewing feature segments as images, convolutional networks show the great effects on feature representation for SER [2, 4, 6, 8, 9, 11, 12].”
BackgroundMethodResearch on lightweight speech emotion recognition in vehicle noisy environment
Tang, C., Qi, A., & Xie, B. (2024) · Advances in Mechanical Engineering · doi:10.1177/16878132241260585 · Cited by 4
“Chen et al. 11 utilize Mel spectrograms, deltas, and delta-deltas as feature inputs for the model.”
Lesyk, E., Arias‐Vergara, T., Nöth, E., Maier, A., Orozco‐Arroyave, J. R., & Pérez-Toro, P. A. (2024) · Human-Centric Intelligent Systems · doi:10.1007/s44230-024-00088-w · Cited by 4
Generalizable Dynamic Domain Adaptation for Speaker-Independent Speech Emotion Recognition
Lu, C., Zong, Y., Lian, H., Li, S., Zhao, Y. D., Schuller, B. W., & Zheng, W. (2025) · IEEE Transactions on Audio Speech and Language Processing · doi:10.1109/taslpro.2025.3636055 · Cited by 4
Development of a Speech Emotion Recognition Dataset for Indonesian
Santoso, T. B., Budianto, A. N., & Dutono, T. (2026) · IEEE Access · doi:10.1109/access.2026.3652377 · Cited by 4
“The quality of the dataset formed was also analyzed through inter-rater reliability [9], statistical behaviour [10], and feature evaluation [11], [12] and tested using Convolutional Neural Network (CNN) clustering [13], [14].”
Residual attention network for audio-based speech emotion recognition
Zhao, P., Zhang, H., Yue, Z., Li, Z., Mi, N., & Liu, X. (2026) · Applied Soft Computing · doi:10.1016/j.asoc.2026.115637 · Cited by 4
Self-citationMultimodal Speech Emotion Recognition Based on Large Language Model
Congcong, F. A. N. G., Jin, Y., Guanlin, C. H. E. N., ZHANG, Y., Li, S., Ma, Y., & Xie, Y. (2024) · IEICE Transactions on Information and Systems · doi:10.1587/transinf.2024edl8034 · Cited by 4
“Due to the unique structure of the attention mechanism, models can assign varying levels of importance to each word in a sentence based on the current task, making them well-suited for tasks such as emotion recognition and related en-deavors[11].”
BackgroundFei, S., Feng, Q., & Gao, F. (2025) · IEEE Access · doi:10.1109/access.2025.3567954 · Cited by 4
“With the rapid development of deep learning in recent years, deep neural networks (DNNs) have been widely used in emotion recognition by relying on their powerful nonlinear modeling representation capability [12], [13], [14], [15], [16].”
Heydari, S., & Shahidi, S. M. (2025) · Multimedia Tools and Applications · doi:10.1007/s11042-025-21009-4 · Cited by 4
“Furthermore, it outperforms the models presented in [ 6 ] and [78] in terms of accuracy, while maintaining performance comparable to other works.”
Zhu, L. (2022) · Occupational Therapy International · doi:10.1155/2022/3909734 · Cited by 4
“In terms of educational means, it is necessary to carry out various trainings and lectures to increase employees’ awareness of emotional management; in terms of scientific assessment, employees can be tested by authoritative institutions to grasp the true state of employees’ emotions [31, 32].”
BackgroundRetractedInferring influence of people's emotions at court on defendant's emotions using a prediction model
Song, Y., & Zhao, T. (2023) · Frontiers in Psychology · doi:10.3389/fpsyg.2023.1131724 · Cited by 4
“Through Llog-Mels spectrogram, the difficulty of speech recognition due to differences in style and intonation can be reduced (Chen et al., 2018).”
Facial Emotion Recognition with InceptionV3 CNN Architecture
Timothy, J., Prasetio, R., Tandi, D., Lucky, H., & Iswanto, I. A. (2023) · International Conference on Informatics, Multimedia, Cyber and Information System (ICIMCIS) · doi:10.1109/icimcis60089.2023.10348994 · Cited by 4
“[5], [11], [12] use long short-term memory (LSTM) classifiers on their model and achieved 99.81% and 96.99% accuracy, respectively.”
BackgroundInfluentialA Residual Multi-Scale Convolutional Neural Network With Transformers for Speech Emotion Recognition
Yan, T., Meng, H., Parada‐Cabaleiro, E., Tao, J., Li, T., & Schuller, B. W. (2024) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2024.3481253 · Cited by 4
“LSTM − Attention : The model utilizes reshaping techniques provided by Chen et al. [41] to transform Transformer outputs into chunk-level features.”
InfluentialWurst, A., Hopwood, M. W., Wu, S., Li, F., & Yao, Y. (2023) · Wireless and Optical Communications Conference · doi:10.1109/wocc58016.2023.10139686 · Cited by 4
Zhang, X. (2024) · Scalable Computing Practice and Experience · doi:10.12694/scpe.v25i1.2296 · Cited by 4
“The experimental findings indicated that the method had high application effectiveness, and the recall rate was high [14].”
FPGA Implementation of CNN-LSTM Classifier in Speech Emotion Recognition System
Gao, Z., Xiao, W., Zhou, W., & Yang, Z. (2023) · International Conference on High Performance Big Data and Intelligent Systems (HPBD&IS) · doi:10.1109/hdis60872.2023.10499604 · Cited by 4
“The human ear perceives sound on a logarithmic scale[5], so taking logarithms can better match the perception of the human ear[6].”
BackgroundAdvancements and Role of Emotion Recognition in the 4th Industrial Revolution
Roy, T., Marwala, T., & Chakraverty, S. (2020) · Lecture notes in electrical engineering · doi:10.1007/978-3-030-48230-5_8 · Cited by 4
Applying Multitask Deep Learning to Emotion Recognition in Speech
Ryabinov, A., Uzdiaev, M., & Vatamaniuk, I. V. (2021) · Proceedings of Southwest State University · doi:10.21869/2223-1560-2021-25-1-82-109 · Cited by 4
“Однако лишь в последние годы были предложены сквозные (end-to-end) модели [12-17], использующие для обучения непосредственно запись человеческого голоса.”
A Perspective Study on Speech Emotion Recognition: Databases, Features and Classification Models
Raghu, K., & Sadanandam, M. (2021) · Traitement du signal · doi:10.18280/ts.380631 · Cited by 4
Research on Speech Emotion Recognition Method Based A-CapsNet
Qi, Y., Huang, H., & Zhang, H. (2022) · Applied Sciences · doi:10.3390/app122412983 · Cited by 4
Self-citationEmotion Recognition Audio Database for Service Robots
Telembici, T., Muscar, L., Grama, L., & Rusu, C. (2022) · International Symposium on Electronics and Telecommunications (ISETC) · doi:10.1109/isetc56213.2022.10009933 · Cited by 4
“Given this context, Speech Emotion Recognition (SER), one of the most recent challenges in human-computer interaction, has gained attention [2, 3].”
BackgroundMulti-Level Attention-Based Categorical Emotion Recognition Using Modulation-Filtered Cochleagram
Peng, Z., Wen-hua, H., Li, Y., Du, Y., & Dang, J. (2023) · Applied Sciences · doi:10.3390/app13116749 · Cited by 4
“In [40], the authors used a 3D attention-based CRNN for learning discriminative features for SER, utilizing a Mel spectrogram with deltas and delta-deltas as inputs.”
MethodZhao, J. M., Zhang, Z., Qiu, J., Shi, L., Kuang, Z., Jing, W., & Zhu, Z.-P. (2021) · Research Square · doi:10.21203/rs.3.rs-1085276/v1 · Cited by 4
“In order to analyze people’s emotions well, researchers need to collect and recognize the emotions expressed in speech [2, 3] facial expressions [4]peripheral physiological signals [5]and behavioral actions [6].”
BackgroundSpeaker Normalization for Self-supervised Speech Emotion RecognitionPreprint
Gat, I., Aronowitz, H., Zhu, W., Morais, E. D. S., & Hoory, R. (2022) · arXiv (Cornell University) · doi:10.48550/arxiv.2202.01252 · Cited by 4
“Later, attention-based models outperformed the CNN and LSTM combination due to their ability to focus on specific parts of the input [22, 23, 24].”
Arezzo, A., & Berretti, S. (2022) · arXiv (Cornell University) · doi:10.48550/arxiv.2211.02366 · Cited by 4
“To this end, a particular architecture known as ACRNN, was presented for the first time in SER in [6].”
MethodAnalysis Of Emotions Through Speech Recognition
Anandappa, M., & Mudnal, M. K. (2024) · Journal of Scientific Research and Technology · doi:10.61808/jsrt95 · Cited by 3
Alahmari, S., Al-shathry, N. I., Eltahir, M. M. E., Alzaidi, M. S. A., ALGHAMDI, A. A., & Ahmed, M. (2024) · Fractals · doi:10.1142/s0218348x25400225 · Cited by 3
Zhao, H., Han, K., Fan, W., Ye, N., & Wang, R. (2025) · Expert Systems with Applications · doi:10.1016/j.eswa.2025.128982 · Cited by 3
Application of an Improved LSTM Model to Emotion Recognition
Li, Y. (2022) · Journal of Electrical and Computer Engineering · doi:10.1155/2022/3271074 · Cited by 3
“At present, in the field of speech emotion recognition research, there are many kinds of corpora available for research, such as EMO-DB German database, DES Danish database [29], CASIA (the Institute of Automation of the Chinese Academy of Sciences) database [30], and IEMOCAP English database [23].”
MethodRetractedIntroducing ISAP and MATSS: Mental stress induced speech utterance procedure and obtained dataset
Singh, B., & Phartiyal, G. S. (2022) · Speech Communication · doi:10.1016/j.specom.2022.09.005 · Cited by 3
Srilakshmi, R., Kamma, V., Choudhary, S., Kumar, S., & Kumar, M. (2023) · International Conference on Technological Advancements in Computational Sciences (ICTACS) · doi:10.1109/ictacs59847.2023.10390529 · Cited by 3
“To recognize feelings from audio signals, you need to extract features and train a classifier [3].”
BackgroundRetractedA Review on Language-Independent Search on Speech and its Applications
Kulkarni, S. V., & Pal, S. (2024) · IEEE Access · doi:10.1109/access.2024.3520394 · Cited by 3
“Some of the deep models used techniques are initiated for cross-modal hashing, and supervised deep models utilize semantic labels to enrich the cross-modal data correlation, whereas the feature transformation and hash functions can be mutually learned in an end-to-end manner [7], [8], [51].”
FF-GLAM-cs: a fusion framework based on GLAM with channel shuffle for speech emotion recognition
Wang, J., Zheng, Z., Liang, Y., Qin, J., & Wang, W. (2023) · International Journal of Machine Learning and Cybernetics · doi:10.1007/s13042-023-01948-w · Cited by 3
“Since convolutional neural networks have shown excellent performance in picture classification, some researchers have introduced them to speech emotion recognition tasks [3–6].”
An End-to-End Model for Mental Disorders Detection by Spontaneous Physical Activity Data
Xu, D., Wang, Z., Kitajima, T., Nakamura, T., Shimura, H., Takeuchi, H., Tan, Y., Ge, R., Qian, K., Hu, B., et al. (2023) · IEEE ... International Conference on Data Mining workshops · doi:10.1109/icdmw60847.2023.00168 · Cited by 3
“From Table II, we observe promising results in terms of Unweighted Average Recall (UAR) for the binary classifi-cation tasks [34], [35].”
MethodEMOVA: Emotion-driven neural volumetric avatar
Hwang, J., Kim, B.-G., Kim, T., Oh, H., & Kang, J. (2024) · Image and Vision Computing · doi:10.1016/j.imavis.2024.105043 · Cited by 3
Hosain, M. S., Sugiura, Y., Haque, M., Rahman, M. S., & Shimamura, T. (2024) · International Conference on Computer and Information Technology (ICCIT) · doi:10.1109/iccit64611.2024.11021978 · Cited by 3
“The researchers in [13] employed 3-”
Bird Song Recognition Based on Deep Transfer Learning with XGBoost
Zhang, F., Liu, W., Zhang, J., & Wang, K. (2022) · International Conference on Robotics and Computer Vision (ICRCV) · doi:10.1109/icrcv55858.2022.9953226 · Cited by 3
“By calculating the delta and delta-deltas of log-Mels to obtain dynamic features, the dynamic features as the supplementary features of log-Mels can reduce the influence of extraneous factors such as environmental noise, while retaining the useful species information in the audio[11].”
BackgroundAutomatic Classification of Vocal Intensity Category from Speech
Kodali, M., Kadiri, S. R., Laaksonen, L., & Alku, P. (2023) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp49357.2023.10097160 · Cited by 3
“It was also found that intensity category information is not uniformly distributed in time throughout the speech signal; hence, advanced neural networks like attention-based models could also be used in the future to improve the classification performance [43].”
MethodShuffleaugment: A Data Augmentation Method Using Time Shuffling
Sato, Y., Ikeda, N., & Takahashi, H. (2023) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp49357.2023.10096927 · Cited by 3
“The base model used in our experiments for speech emotion recognition was the attention-based convolutional recurrent neural network [32, 33].”
MethodChen, X., Liu, Z., Xiao, J., Liu, T., & Zhao, Y. (2023) · Cognitive Systems Research · doi:10.1016/j.cogsys.2023.101150 · Cited by 3
Metadata
- Authors
- Chen, M., He, X., Yang, J., & Zhang, H.
- Affiliations
- Central South University (CN)
- Venue
- IEEE Signal Processing Letters
- Published
- 2018-10
- Volume
- 25
- Issue
- 10
- Pages
- 1440-1444
- ISSN
- 1070-9908 (ISSN-L)
- 1558-2361
- OpenAlex
- W2885005742
- Cited by
- 503
- References
- 23
- Citations by year
- 2026: 21 · 2025: 50 · 2024: 70 · 2023: 85 · 2022: 91 · 2021: 94 · 2020: 60 · 2019: 27 · 2018: 5
- FWCI
- 60.91
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- IEEE Signal Processing Letters (publishedVersion)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Emotion and Mood Recognition
- Physical Sciences › Computer Science › Artificial Intelligence › Speech Recognition and Synthesis
- Physical Sciences › Computer Science › Signal Processing › Speech and Audio Processing
- Keywords
- speech emotion recognition, attention mechanism, Mel spectrogram, IEMOCAP, EMO-DB
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.