3-D Convolutional Recurrent Neural Networks With Attention Model for Speech Emotion Recognition
The DOI matches its Crossref record; 23 of the 23 references it cites were found in a registry or an index.
Abstract
Speech emotion recognition (SER) is a difficult task due to the complexity of emotions. The SER performances are heavily dependent on the effectiveness of emotional features extracted from the speech. However, most emotional features are sensitive to emotionally irrelevant factors, such as the speaker, speaking styles, and environment. In this letter, we assume that calculating the deltas and delta-deltas for personalized features not only preserves the effective emotional information but also reduces the influence of emotionally irrelevant factors, leading to reduce misclassification. In addition, SER often suffers from the silent frames and emotionally irrelevant frames. Meanwhile, attention mechanism has exhibited outstanding performances in learning relevant feature representations for specific tasks. Inspired by this, we propose a three-dimensional attention-based convolutional recurrent neural networks to learn discriminative features for SER, where the Mel-spectrogram with deltas and delta-deltas are used as input. Experiments on IEMOCAP and Emo-DB corpus demonstrate the effectiveness of the proposed method and achieve the state-of-the-art performance in terms of unweighted average recall.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Another issue faced by the time-frequency representation of speech emotion is that the distribution of emotion information in an utterance is sparse in both frequency domain and time domain [19], [23].”
From Speech Emotion Recognition via an Attentive Time–Frequency Neural Network (2023) · cites this paper as [19]
In the citing paper
“However, these studies mainly embed attention into highlevel time-frequency features, leading to introduce the redundant information in the feature extraction and roughly locate the emotion-related regions in the time-frequency domain.”
From Speech Emotion Recognition via an Attentive Time–Frequency Neural Network (2023) · cites this paper as [19]
In the citing paper
“In addition, they mainly focus on the selection on the time frames while ignoring the selection on the frequency bands.”
From Speech Emotion Recognition via an Attentive Time–Frequency Neural Network (2023) · cites this paper as [19]
In the citing paper
“ignoring to capture the specific frequency bands associated with emotions”
From Speech Emotion Recognition via an Attentive Time–Frequency Neural Network (2023) · cites this paper as [19]
Citations
Cites
Adam: A Method for Stochastic Optimization
Kingma, D. P., & Ba, J. (2014) · arXiv · doi:10.48550/arxiv.1412.6980 · Cited by 98,946 · cited 1×
“toolkit [24], and the parameters of the model were optimized by minimizing the cross-entropy objective function, with a minibatch of 40 samples, using the Adam optimizer [25] with Nestorov momentum.”
MethodNeural Machine Translation by Jointly Learning to Align and Translate
Bahdanau, D., Cho, K., & Bengio, Y. (2014) · arXiv · doi:10.48550/arxiv.1409.0473 · Cited by 16,213 · cited 1×
“proposed in [9] for machine translation and later applied for”
MethodTensorFlow: Large-Scale Machine Learning on Heterogeneous Distributed Systems
Abadi, M., Agarwal, A., Barham, P., Brevdo, E., Chen, Z., Citro, C., Corrado, G. S., Davis, A., Dean, J., Devin, M., et al. (2016) · arXiv · doi:10.48550/arxiv.1603.04467 · Cited by 9,408
Hierarchical Attention Networks for Document Classification
Yang, Z., Yang, D., Dyer, C., He, X., Smola, A., & Hovy, E. (2016) · Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies · doi:10.18653/v1/n16-1174 · Cited by 4,712 · cited 1×
“document classification [10] and speech recognition [11].”
Self-citationIEMOCAP: Interactive emotional dyadic motion capture database
Busso, C., Bulut, M., Lee, C.-C., Kazemzadeh, A., Mower, E., Kim, S., Chang, J. N., Lee, S., & Narayanan, S. S. (2008) · Language Resources and Evaluation · doi:10.1007/s10579-008-9076-6 · Cited by 3,869 · cited 12×
“D ACRNN obtains an absolute improvement of 2.34% and 3.44% for IEMOCAP and Emo-DB, respectively.”
MethodInfluentialA database of German emotional speech
Burkhardt, F., Paeschke, A., Rolfes, M., Sendlmeier, W. F., & Weiss, B. (2005) · Interspeech 2005 · doi:10.21437/interspeech.2005-446 · Cited by 2,365 · cited 1×
“form speaker-independent SER experiments on the Interactive Emotional Dyadic Motion Capture database (IEMOCAP) [21] and the Berlin Emotional Database (Emo-DB) [22].”
MethodAttention-Based Models for Speech Recognition
Chorowski, J., Bahdanau, D., Serdyuk, D., Cho, K., & Bengio, Y. (2015) · arXiv · doi:10.48550/arxiv.1506.07503 · Cited by 2,021
Bidirectional LSTM Networks for Improved Phoneme Classification and Recognition
Graves, A., Fernández, S., & Schmidhuber, J. (2005) · Lecture Notes in Computer Science · doi:10.1007/11550907_126 · Cited by 925 · cited 1×
“bidirectional RNN with long short-term memory cells [19] for temporal summarization, and each direction contains 128 cells; then, we can obtain a sequence of 256-dimensional high-level”
MethodAdieu features? End-to-end speech emotion recognition using a deep convolutional recurrent network
Trigeorgis, G., Ringeval, F., Brueckner, R., Marchi, E., Nicolaou, M. A., Schuller, B., & Zafeiriou, S. (2016) · 2016 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) · doi:10.1109/icassp.2016.7472669 · Cited by 904 · cited 1×
“In recent years, CNN has exhibited excellent performances in the field of SER [1], [4], [6].”
BackgroundSpeech emotion recognition using deep neural network and extreme learning machine
Han, K., Yu, D., & Tashev, I. (2014) · Interspeech 2014 · doi:10.21437/interspeech.2014-57 · Cited by 794 · cited 4×
“1) DNN-extreme learning machine (DNN-ELM) ( see [3]).”
MethodInfluential
Research works printed but not linked: 2
- wei, 2016, Attention assisted discovery of sub-utterance structure in speech emotion recognition, Proc INTERSPEECH, 1387
- frano, 2016, Normalization and batch correction methods for high-throughput glycomics, Proc Joint Meet Soc Glycobiol, 1160
Cited by
Items 501–549 of 549
Recognizing Speech Emotions Using AI
Ramachandramurthyraju, V. (2024) · International Journal for Research in Applied Science and Engineering Technology · doi:10.22214/ijraset.2024.63569
Speech Emotion Recognition Using CNN Classifier Based on Deep Learning Model
Archana, M., Shanthi, D., & Vadrevu, P. K. (2024) · Communications in computer and information science · doi:10.1007/978-3-031-73065-8_21
Multi-Model Parkinson’s Speech Classification Model (MM-Parkinson)
Sun, B., Wang, N., Liu, Y., & Zhao, A. (2023) · International Conference on Computer, Internet of Things and Smart City (CIoTSC) · doi:10.1109/ciotsc60428.2023.00019
ASEGAA: Leveraging Attention in Emotion, Gender and Age Analysis from Speech
Sigicharla, I., Ritesh, A., Tiwari, S., & Ibrahim, S. P. S. (2024) · International Conference on Computing, Communication and Networking Technologies (ICCCNT) · doi:10.1109/icccnt61001.2024.10725221
“Reference [14] describes a way for attention mechanisms that work with 3D Convolutional Recurrent Neural Networks and achieves an Unweighted Average Recall of 64.74 on IEMOCAP and 82.82 on EMO-DB for emotion classification.”
MethodInfluentialWang, Q., Li, L., Li, F., Zhou, G., Chen, N., Meng, X., Li, X., Liu, Y., & Zhao, Z. (2024) · Procedia Computer Science · doi:10.1016/j.procs.2024.10.107
Speech Emotion Recognition Using Attention Gate
Tran-Viet, A., Hao, D., & Chau, D. T. (2024) · International Conference on Control, Robotics and Informatics (ICCRI) · doi:10.1109/iccri64298.2024.00019
“Chen et al. [6] proposed a method to calculate deltas and delta-deltas for log-mel of sound called 3-”
Emotion-Aware Synthetic Speech Detection with Multi-level Fusion Features
Xiang, L., Zhou, Z., & Liu, Y. (2025) · Lecture notes in networks and systems · doi:10.1007/978-981-96-3161-2_9
Luitel, S., Liu, Y., & Anwar, M. M. (2026) · Data & Knowledge Engineering · doi:10.1016/j.datak.2026.102649
Exploring Emotional Signals: A Study on Audio-based Emotion Classification
Pinto, J. P., Balasubramani, R., & Martis, J. E. (2025) · Computer Science & Engineering An International Journal · doi:10.5121/cseij.2025.15132
Exploring Machine Learning Approaches Speech Emotion Recognition System
Yaswanth, G., & Manjunath, D. J. (2025) · INTERANTIONAL JOURNAL OF SCIENTIFIC RESEARCH IN ENGINEERING AND MANAGEMENT · doi:10.55041/ijsrem44466
Enhanced Speech Emotion RecognitionUsing AudioSignal Processing with CNN Assistance
Deepika, C., & Kuchibhotla, S. (2025) · Data & Metadata · doi:10.56294/dm2025715
Enhancing Speech Emotion Recognition through Bone-Conducted Speech
Hosain, M. S., & Shimamura, T. (2025) · IntechOpen eBooks · doi:10.5772/intechopen.1008939
Adagale, S. S., Gupta, P. K., & Vairagar, S. (2025) · International Journal of Data Science and Analytics · doi:10.1007/s41060-025-00822-y
“The RMS of the amplitudes of the voice is taken into consideration to compute the RMS of the loudness of voice, which is referred to as the RMS ( x rms ) [43, 44].”
InfluentialGan, C., Zhou, D., Wang, K., Zhu, Q., Jain, D. K., & Štruc, V. (2025) · Computer Vision and Image Understanding · doi:10.1016/j.cviu.2025.104483
TIM-GCN Speech Emotion Recognition Network Based on Generative Adversarial Network
Zhang, L., Du, J., Li, J., & Chen, S. (2025) · Lecture notes of the Institute for Computer Sciences, Social Informatics and Telecommunications Engineering · doi:10.1007/978-3-032-00218-1_11
Wu, S., Wen, J., Wen, C., Qian, J., Zhang, X., Song, Z., & Zhu, Z. (2025) · CAA Symposium on Fault Detection, Supervision and Safety for Technical Processes (SAFEPROCESS) · doi:10.1109/safeprocess67117.2025.11267866
“The first-order Delta is calculated as (2). where t c represents the Log-Mel coefficient at frame t , and N defines the window length for smoothing temporal variations, typically set to 2[20].”
Robust speech emotion recognition using conditional transformer-based architecture
Chung, H., & Yoo, H. D. (2025) · ACM International Conference on Multimedia in Asia (MMAsia) · doi:10.1145/3743093.3771083
“For example, the use of Mel-spectral coefficients with their derivatives in training a 3D attention-based CNN [6], and CNN-based autoencoders with emotion embeddings [9, 43].”
Peng, Z. (2020) · Institutional Repositories DataBase (IRDB)
Deep regression models for spatio-temporal expression recognition in videos
Rajasekhar, G. P. (2023) · Espace ÉTS (ETS)
Deep learning for automatic classification of speech intensity modes
Ansari, L. (2023) · Aaltodoc (Aalto University)
Annotation practices in affective computing: What are these algorithms actually trained on?
Backer, S. D. (2023) · Research Repository (Delft University of Technology)
Pérez-Toro, P. A. (2021) · Institutional Repository University of Antioquia (University of Antioquia)
Zha, C., & Tang, Y. (2025) · Journal of Electrical and Computer Engineering · doi:10.1155/jece/5899808
“D ACRNN) [30] and NO2GESNet [31], an Echo State Network enhanced with octonion algebra [32].”
Speech Emotion Recognition and Personalized Mental Health Recommendations Using DSCNN and BERT
Padamati, P. K., Rao, D. R., & Kanamarlapudi, L. L. (2025) · Lecture notes in networks and systems · doi:10.1007/978-981-96-8343-7_51
Li, X., & Li, J. (2025) · IEEE International Conference on Big Data (BigData) · doi:10.1109/bigdata66926.2025.11401009
“The advent of deep learning has substantially advanced VA recognition, with models such as attention-based recurrent neural networks [9] and convolutional recurrent neural networks [10] demonstrating improved capacity to model emotional dynamics from speech.”
Speech Emotion Recognition with Continuous Representation using Valence and Arousal
Zhang, Y., Riggan, B. S., Nakamura, K., & Bhattacharyya, S. S. (2025) · IEEE Annual Congress on Artificial Intelligence of Things (AIoT) · doi:10.1109/aiot66900.2025.00090
“Chen et al. exploit mel spectrograms together with deltas (first difference) and delta-deltas (second difference) of signal features as input [19].”
Kaur, K., & Singh, P. (2026) · Neural Computing and Applications · doi:10.1007/s00521-026-11900-y
陳, 訓., チン, ク., & Chen, X. (2024) · Institutional Repositories DataBase (IRDB)
Impact of accent on urdu speech emotion recognition: Deep learning based an experimental study
Ali, S., Murtaza, H., & Ahmed, M. (2026) · Multimedia Tools and Applications · doi:10.1007/s11042-026-21484-3
Fujimura, H., & 藤村, 浩. (2022) · Institutional Repositories DataBase (IRDB) · doi:10.15002/00025229
Research on Speech Emotion Recognition Based on Machine Learning Approaches
Liu, Z. (2023) · Institutional Repositories DataBase (IRDB)
自己注意型トランスフォーマーと相互注意型トランスフォーマーを用いた音声感情認識に関する研究
He, Y. (2022) · Institutional Repositories DataBase (IRDB)
Srivastava, G., Sursakar, G. W., & Jain, P. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-18480-1_84
Cuccovillo, L., Pianese, A., Gerhardt, M., Cozzolino, D., Poggi, G., & Aichroth, P. (2026) · Springer eBooks · doi:10.1007/978-3-032-11782-3_7
Rajesh, M. V., Subrahmanayeswari, N. M. S., Priyanka, D. H. S., Chandra, K. R., Prasad, K. L. V., & Chandravirat, I. (2026) · International Conference on Augmented Intelligence and Sustainable Systems (ICAISS) · doi:10.1109/icaiss68683.2026.11526360
“Recent work uses attention mechanisms [8], transformer-based architectures [9], and self-supervised pre-training using models like wav2vec 2.0 [10] and WavLM [11].”
InfluentialSFM-Adapter: Style-Aware Feature Manipulation Adapter for Speech Style Editing
Chen, Y., Liu, H. D., Chen, Q., Singh, A., Zhao, J., Wang, W., Jackson, P. J. B., & Plumbley, M. D. (2026) · IEEE Transactions on Audio Speech and Language Processing · doi:10.1109/taslpro.2026.3698957
Speech Emotion Recognition through Acoustic Data Augmentation and Attention-Driven CRNN-BiGRU
Kacamarga, F., & Aprianto, K. A. (2026) · Ultimatics Jurnal Teknik Informatika · doi:10.31937/ti.v18i1.4250
A Fast and Practical Algorithm for Clustering via Multi-Block Division and SamplingPreprint
Feng, Q., Ji, H., & Li, S. (2026) · SSRN Electronic Journal · doi:10.2139/ssrn.7150019
Multi-feature wavelet attention network for audio deepfake detection
Wang, B., Wang, R., Wang, W., & Ba, Z. (2026) · Knowledge-Based Systems · doi:10.1016/j.knosys.2026.116757
Özseven, T., & Arpacioglu, M. (2026) · Circuits Systems and Signal Processing · doi:10.1007/s00034-026-03700-5
Prema, J. M., Lalini, M. S. A., Babu, T. N., & Suhasini, N. (2026) · doi:10.65647/msp/126/2026
Bhutekar, S., Borole, Y., & Chandurkar, S. (2026) · International Journal For Multidisciplinary Research · doi:10.36948/ijfmr.2026.v08i05.87002
“Speech emotion recognition has benefited from hybrid convolutional-recurrent architectures [4, 5].”
A Temporal-Aware Wasserstein Generative Adversarial Network for Speech Emotion Recognition
Yan, Y., & Yang, R. (2026) · International Conference on Machine Learning and Intelligent Computing · doi:10.1145/3829441.3829456
Balanced Speech Emotion Recognition for Edge and Cloud: A Practical Accuracy-Efficiency Tradeoff
Ahmed, Y., Elsherif, A., Darwish, F., & Mohammed, A. (2026) · Intelligent Methods, Systems, and Applications (IMSA) · doi:10.1109/imsa70415.2026.11700355
“In many benchmarks, CNN and CRNN architectures outperform traditional pipelines[4] [5] [6].”
Audio-Visual Emotion Recognition Using Multilevel Fusion
Shoaib, M., Haq, S. U., Shah, M. S., Rasool, I., & Farooq, M. O. (2024) · The Sciencetech
ETC-TTS: Emotion Trajectory Learning for Controllable Emotional Text-to-Speech
Kim, G., Lee, J., & Chang, J.-H. (2026) · Interspeech (USB) · doi:10.21437/interspeech.2026-3088
Metadata
- Authors
- Chen, M., He, X., Yang, J., & Zhang, H.
- Affiliations
- Central South University (CN)
- Venue
- IEEE Signal Processing Letters
- Published
- 2018-10
- Volume
- 25
- Issue
- 10
- Pages
- 1440-1444
- ISSN
- 1070-9908 (ISSN-L)
- 1558-2361
- OpenAlex
- W2885005742
- Cited by
- 503
- References
- 23
- Citations by year
- 2026: 21 · 2025: 50 · 2024: 70 · 2023: 85 · 2022: 91 · 2021: 94 · 2020: 60 · 2019: 27 · 2018: 5
- FWCI
- 60.91
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- IEEE Signal Processing Letters (publishedVersion)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Emotion and Mood Recognition
- Physical Sciences › Computer Science › Artificial Intelligence › Speech Recognition and Synthesis
- Physical Sciences › Computer Science › Signal Processing › Speech and Audio Processing
- Keywords
- speech emotion recognition, attention mechanism, Mel spectrogram, IEMOCAP, EMO-DB
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.