Attention Based Fully Convolutional Network for Speech Emotion Recognition
Open accessThe DOI matches its Crossref record; 27 of the 27 references it cites were found in a registry or an index.
Abstract
Speech emotion recognition is a challenging task for three main reasons: 1) human emotion is abstract, which means it is hard to distinguish; 2) in general, human emotion can only be detected in some specific moments during a long utterance; 3) speech data with emotional labeling is usually limited. In this paper, we present a novel attention based fully convolutional network for speech emotion recognition. We employ fully convolutional network as it is able to handle variable-length speech, free of the demand of segmentation to keep critical information not lost. The proposed attention mechanism can make our model be aware of which time-frequency region of speech spectrogram is more emotion-relevant. Considering limited data, the transfer learning is also adapted to improve the accuracy. Especially, it's interesting to observe obvious improvement obtained with natural scene image based pre-trained model. Validated on the publicly available IEMOCAP corpus, the proposed model outperformed the state-of-the-art methods with a weighted accuracy of 70.4% and an unweighted accuracy of 63.9% respectively.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source says what the claim attributes to it.
In the citing paper
“Another issue faced by the time-frequency representation of speech emotion is that the distribution of emotion information in an utterance is sparse in both frequency domain and time domain [19], [23].”
From Speech Emotion Recognition via an Attentive Time–Frequency Neural Network (2023) · cites this paper as [23]
In the citing paper
“In addition, they mainly focus on the selection on the time frames while ignoring the selection on the frequency bands.”
From Speech Emotion Recognition via an Attentive Time–Frequency Neural Network (2023) · cites this paper as [23]
In the citing paper
“However, these studies mainly embed attention into highlevel time-frequency features, leading to introduce the redundant information in the feature extraction and roughly locate the emotion-related regions in the time-frequency domain.”
From Speech Emotion Recognition via an Attentive Time–Frequency Neural Network (2023) · cites this paper as [23]
In the citing paper
“ignoring to capture the specific frequency bands associated with emotions”
From Speech Emotion Recognition via an Attentive Time–Frequency Neural Network (2023) · cites this paper as [23]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474
Very Deep Convolutional Networks for Large-Scale Image Recognition
Simonyan, K., & Zisserman, A. (2014) · arXiv · doi:10.48550/arxiv.1409.1556 · Cited by 114,148
ImageNet classification with deep convolutional neural networks
Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017) · Communications of the ACM · doi:10.1145/3065386 · Cited by 112,155
ImageNet Large Scale Visual Recognition Challenge
Russakovsky, O., Deng, J., Su, H., Krause, J., Satheesh, S., Ma, S., Huang, Z., Karpathy, A., Khosla, A., Bernstein, M., et al. (2015) · International Journal of Computer Vision · doi:10.1007/s11263-015-0816-y · Cited by 39,380
Fully convolutional networks for semantic segmentation
Long, J., Shelhamer, E., & Darrell, T. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298965 · Cited by 38,834
Rich Feature Hierarchies for Accurate Object Detection and Semantic Segmentation
Girshick, R., Donahue, J., Darrell, T., & Malik, J. (2014) · 2014 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2014.81 · Cited by 32,684
Neural Machine Translation by Jointly Learning to Align and Translate
Bahdanau, D., Cho, K., & Bengio, Y. (2014) · arXiv · doi:10.48550/arxiv.1409.0473 · Cited by 16,213
Effective Approaches to Attention-based Neural Machine Translation
Luong, T., Pham, H., & Manning, C. D. (2015) · Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d15-1166 · Cited by 8,382
Hierarchical Attention Networks for Document Classification
Yang, Z., Yang, D., Dyer, C., He, X., Smola, A., & Hovy, E. (2016) · Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies · doi:10.18653/v1/n16-1174 · Cited by 4,712
DeCAF: A Deep Convolutional Activation Feature for Generic Visual Recognition
Donahue, J., Jia, Y., Vinyals, O., Hoffman, J., Zhang, N., Tzeng, E., & Darrell, T. (2013) · arXiv · doi:10.48550/arxiv.1310.1531 · Cited by 4,019 · cited 1×
“…current task with the help of the knowledge obtained from related problems [10], which can be operated as finetuning the parameters of network from a pre-trained model, It has been widely used when the training data is insufficient [16], [17], [18], [19], especially when the model is based on CNN.”
Method
Cited by
Items 101–160 of 160
Multimodal Depression Severity Detection Using Deep Neural Networks and Depression Assessment Scale
Sudhan, H. V. M., & Kumar, S. S. (2022) · Lecture notes on data engineering and communications technologies · doi:10.1007/978-981-16-7182-1_29 · Cited by 5
Selvi, R., & Vijayakumaran, C. (2023) · International Conference on Advanced Computing and Communication Systems (ICACCS) · doi:10.1109/icaccs57279.2023.10113124 · Cited by 6
An Algorithm of Emotion Recognition And Valence of Drivers on Multimodal Data
Guo, L., Shen, Y. C., & Ding, P. (2022) · IEEE International Symposium on Broadband Multimedia Systems and Broadcasting (BMSB) · doi:10.1109/bmsb55706.2022.9828712 · Cited by 5
Chauhan, K., Sharma, K. K., & Varma, T. (2022) · Multimedia Tools and Applications · doi:10.1007/s11042-022-13463-1 · Cited by 8
Santoso, J., Yamada, T., & Makino, S. (2019) · Asia-Pacific Signal and Information Processing Association Annual Summit and Conference (APSIPA ASC) · doi:10.1109/apsipaasc47483.2019.9023288 · Cited by 7
Improving Noise Robustness In Speaker Identification Using A Two-Stage Attention ModelPreprint
Shi, Y., Huang, Q., & Hain, T. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1909.11200 · Cited by 9
WITHDRAWN: Intelligent system for imposter detection: Asurvey
Ali, A. T., Abdullah, H. S., & Fadhil, M. N. (2021) · Materials Today Proceedings · doi:10.1016/j.matpr.2021.04.074 · Cited by 6
RetractedGFRN-SEA: Global-Aware Feature Representation Network for Speech Emotion Analysis
Pan, L., & Wang, Q. (2024) · IEEE Access · doi:10.1109/access.2024.3490186 · Cited by 6
A comparison study of widespread CNN architectures for speech emotion recognition on spectrogram
Ryabinov, A., & Uzdiaev, M. (2022) · AIP conference proceedings · doi:10.1063/5.0092612 · Cited by 3
Sparse Spatial-Temporal Emotion Graph Convolutional Network for Video Emotion Recognition
Liu, X., Xu, H., & Wang, M. (2022) · Computational Intelligence and Neuroscience · doi:10.1155/2022/3518879 · Cited by 4
A Residual Multi-Scale Convolutional Neural Network With Transformers for Speech Emotion Recognition
Yan, T., Meng, H., Parada‐Cabaleiro, E., Tao, J., Li, T., & Schuller, B. W. (2024) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2024.3481253 · Cited by 4
Maji, B., Swain, M., & Panda, R. (2022) · Research Square · doi:10.21203/rs.3.rs-1529387/v1 · Cited by 3
CNN-BLSTM with Attention Model for Speech Emotion RecognitionPreprint
Xia, Y., & Zhao, L. (2023) · Research Square · doi:10.21203/rs.3.rs-3392008/v1 · Cited by 6
LANCET: Lightweight Attention-Enhanced Network for Robust Speech Emotion Recognition
Terraf, Y., & Iraqi, Y. (2025) · European Signal Processing Conference (EUSIPCO) · doi:10.23919/eusipco63237.2025.11226051 · Cited by 2
Feature Fusion for Emotion Recognition †
Mahmood, A. (2023) · doi:10.3390/engproc2023046020 · Cited by 2
Sharma, S., Phoga, A., & Kadyan, V. (2023) · OITS International Conference on Information Technology (OCIT) · doi:10.1109/ocit59427.2023.10431114 · Cited by 2
Privacy Preserving Federated Learning Approach for Speech Emotion Recognition
Chowdhury, M. R. Z., Afiat, M., Hore, A. R., Akhter, R., Sarker, A., Mehedi, M. H. K., Hossain, A., & Rasel, A. A. (2023) · International Conference on Computer and Information Technology (ICCIT) · doi:10.1109/iccit60459.2023.10441577 · Cited by 4
Manohar, K., Sravani, K., & Reddy, U. S. (2023) · International Conference on New Frontiers in Communication, Automation, Management and Security (ICCAMS) · doi:10.1109/iccams60113.2023.10525931 · Cited by 2
Improving Valence-Arousal Estimation with Spatiotemporal Relationship Learning and Multimodal Fusion
Yu, J., Zhao, G., Wang, Y., Wei, Z., Zhang, Z., Cai, Z., Xie, G., Zhu, J., Zhu, W., Yang, S., et al. (2024) · IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw63382.2024.00785 · Cited by 2
Integrating gating and learned queries in audiovisual emotion recognition
Zhang, Z., Guo, Q., Lu, S., Su, J., & Tang, T. (2024) · Multimedia Systems · doi:10.1007/s00530-024-01551-1 · Cited by 2
CoAffinity: A Multimodal Dataset for Cognitive Load and Affect Assessment in Remote Collaboration
Gunasekaran, T. S., Gupta, K., Pai, Y. S., Bai, H., & Billinghurst, M. (2025) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2025.3562559 · Cited by 5
Attentions for Short Duration Speech Classification
Modi, H., Patel, M., & Patil, H. A. (2023) · European Signal Processing Conference (EUSIPCO) · doi:10.23919/eusipco58844.2023.10290005 · Cited by 2
Zhu, Y., & Liang, X. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2008.07052 · Cited by 4
Yan, T., Meng, H., Parada‐Cabaleiro, E., Tao, J., & Schuller, B. W. (2022) · doi:10.36227/techrxiv.21309600.v1 · Cited by 3
Generalizable Dynamic Domain Adaptation for Speaker-Independent Speech Emotion Recognition
Lu, C., Zong, Y., Lian, H., Li, S., Zhao, Y. D., Schuller, B. W., & Zheng, W. (2025) · IEEE Transactions on Audio Speech and Language Processing · doi:10.1109/taslpro.2025.3636055 · Cited by 4
72‐3: Fully Convolutional Transformer‐Based Speech Emotion Recognition for Automotive Systems
Chung, H., & Yoo, H. (2025) · SID Symposium Digest of Technical Papers · doi:10.1002/sdtp.18336 · Cited by 1
Optimizing Speech Emotion Recognition with Attention-Driven Deep CRNN
Popescu, V.-D., & Giurgiu, M. (2025) · Telecommunications Forum (TELFOR) · doi:10.1109/telfor67910.2025.11314307 · Cited by 1
Emotion Estimation Using ConvNeXt Based on Feature Extraction from Imaged MFCC
Noda, R., & Hashimoto, T. (2025) · International Conference on Robotics and Computer Vision (ICRCV) · doi:10.1109/icrcv67407.2025.11349139 · Cited by 1
A speech understanding-based method for recognizing psychological medical speech feelings
Zhang, S., Li, J., Sufyan, M., Xiao, Y., & Dong, X. (2022) · IEEE Annual International Computer Software and Applications Conference (COMPSAC) · doi:10.1109/compsac54236.2022.00228 · Cited by 1
Machine Learning Based Integrated Audio and Text Modalities for Enhanced Emotional Analysis
Agarkhed, J. V., & Vishalakshmi (2023) · International Conference on Inventive Research in Computing Applications (ICIRCA) · doi:10.1109/icirca57980.2023.10220776 · Cited by 1
An emotion recognition system: bridging the gap between humans-machines interaction
Nouman, A., Iqbal, H., & Saleem, M. K. (2023) · IAES International Journal of Robotics and Automation (IJRA) · doi:10.11591/ijra.v12i4.pp315-324 · Cited by 2
Speech Emotion Recognition Based on Feature Fusion and Residual Graph Convolutional Network
Zhang, Y., Cui, L., Sun, X., & Xue, K. (2023) · IEEE International Conference on Signal Processing, Communications and Computing (ICSPCC) · doi:10.1109/icspcc59353.2023.10400357 · Cited by 1
Self-citationWang, Z.-R., & Du, J. (2019) · arXiv (Cornell University) · Cited by 1
Self-citationEfficient Arabic emotion recognition using deep neural networksPreprint
Ali, A. M. A., & Hifny, Y. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2011.00346 · Cited by 2
LSSED: a large-scale dataset and benchmark for speech emotion recognitionPreprint
Fan, W., Xu, X., Xing, X., Chen, W., & Huang, D. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2102.01754 · Cited by 1
Attention and Memory Augmented Networks
Kamath, U., Liu, J., & Whitaker, J. (2019) · Springer eBooks · doi:10.1007/978-3-030-14596-5_9
Customer Reviews’ Sentiments Analysis using Deep Learning
Mehta, K., & Panda, S. P. (2020) · International Journal of Computer Applications · doi:10.5120/ijca2020920842
Deep audio and video emotion detection
Aminbeidokhti, M. (2020) · Espace ÉTS (ETS)
Sentiment Perception in Articulation
Kancherla, G. N. M. (2021) · International Journal for Research in Applied Science and Engineering Technology · doi:10.22214/ijraset.2021.36973
Speech error recognition based on broad learning
Yan, Y., Wang, L., & Guo, W. (2021) · Journal of Physics Conference Series · doi:10.1088/1742-6596/2031/1/012030
Improved Speech Emotion Recognition Using LAM and CTC
Meng, L., Sun, Z., Liu, Y., Zhao, Z., & Li, Y. (2021) · Lecture notes of the Institute for Computer Sciences, Social Informatics and Telecommunications Engineering · doi:10.1007/978-3-030-89814-4_55
Zhou, H., Du, J., Zhang, Y., Wang, Q., Liu, Q., & Lee, C. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2111.08910
Self-citationIdentification of Emotions in Spoken Language Using Deep Learning
de Oliveira, Y. M. M., & Figueiredo, C. M. S. (2021) · doi:10.5753/eniac.2021.18245
Yan, T., Meng, H., Parada‐Cabaleiro, E., Tao, J., & Schuller, B. W. (2022) · doi:10.36227/techrxiv.21309600
Dense Attention Memory Network for Multi-modal emotion recognition
Gl, G., & Guo, X. J. (2022) · International Conference on Machine Learning and Natural Language Processing (MLNLP) · doi:10.1145/3578741.3578751
Manohar, K., & Logashanmugam, E. (2023) · SSRN Electronic Journal · doi:10.2139/ssrn.4455697
Spectrogram Generation for Music Game Based on Attention-BiLSTM
Zhang, M., Jiang, Z., Zhao, Z., Lv, X., & Zhang, L. (2022) · Asian Conference on Artificial Intelligence Technology (ACAIT) · doi:10.1109/acait56212.2022.10137939
Ascertaining Speech Emotion using Attention-based Convolutional Neural Network Framework
Arya, A., Arya, V., Kohli, N., Sukhija, N., Ibrahim, A. O., Bharany, S., Binzagr, F., Muchtar, F., & Mamoun, M. (2023) · International Journal of Advanced Computer Science and Applications · doi:10.14569/ijacsa.2023.0141163
Erquiza, A. J. B., Mamaclay, R. J. E., Platon, E., & Acula, D. D. (2024) · International Conference on Smart Computing and Communications (ICSCC) · doi:10.1109/icscc62041.2024.10690712
Conditional Transformer-Based U-Net Architecture for Speech Emotion Recognition
Chung, H., & Yoo, H. (2025) · IEEE Intelligent Vehicles Symposium · doi:10.1109/iv64158.2025.11097450 · Cited by 2
Boomika, S., Nagur, S. V., Badakali, B., Chikkamath, S., & Budihal, S. V. (2025) · International Conference for Emerging Technology (INCET) · doi:10.1109/incet64471.2025.11140866
Robust speech emotion recognition using conditional transformer-based architecture
Chung, H., & Yoo, H. D. (2025) · ACM International Conference on Multimedia in Asia (MMAsia) · doi:10.1145/3743093.3771083
Edge-Based Audio Emotion Detection Using STM32 and Nanoedge AI
Bansode, N. V., Shaikh, N., Kawadkar, J., Tade, S. L., Sawant, S., Rajarapollu, P. R., & Patil, N. A. (2025) · International Conference on Computing Communication Control and automation · doi:10.1109/iccubea65967.2025.11284200
A Deep Voice Emotion Recognition CNN- BiLSTM Model for Intelligent Tutoring Systems
Sawan, V., Mahajan, P., Jose, D., Lendale, P. K., Sajiv, G., & Subramanyam, M. V. (2026) · International Conference on Expert Clouds and Applications (ICOECA) · doi:10.1109/icoeca68095.2026.11485545
Tyagi, S., & Szénási, S. (2026) · Algorithms · doi:10.3390/a19050416 · Cited by 1
An Interpretable Collaborative Acoustic Parameter Modeling Network for Speech Emotion Recognition
Lian, H., Lu, C., Yang, H. S., Zhao, Y. D., Li, S., & Zong, Y. (2026) · IEEE Transactions on Computational Social Systems · doi:10.1109/tcss.2026.3692891 · Cited by 1
Motiejauskas, M. (2026) · doi:10.15388/vu.thesis.923 · Cited by 1
Canuto, T., Sardenberg, E., Mann, P., Utino, M. Y. R., COUTINHO, D. J. B., Uchôa, A., & Pereira, J. A. (2026) · International Workshop on Cooperative and Human Aspects of Software Engineering (CHASE) · doi:10.1145/3794860.3794910
Özseven, T., & Arpacioglu, M. (2026) · Circuits Systems and Signal Processing · doi:10.1007/s00034-026-03700-5
Kumar, A. P., N, A. T., & Dhanya, R. (2026) · International Conference on Trends in Engineering Systems and Technologies (ICTEST) · doi:10.1109/ictest69239.2026.11722902
Metadata
- Authors
- Zhang, Y., Du, J., Wang, Z., Zhang, J., & Tu, Y.
- Affiliations
- University of Science and Technology of China (CN)
- Venue
- 2018 Asia-Pacific Signal and Information Processing Association Annual Summit and Conference (APSIPA ASC)
- Publisher
- IEEE
- Published
- 2018-11
- Pages
- 1771-1775
- OpenAlex
- W2806051338
- Cited by
- 161
- References
- 23
- Citations by year
- 2026: 7 · 2025: 16 · 2024: 17 · 2023: 27 · 2022: 32 · 2021: 31 · 2020: 17 · 2019: 14
- FWCI
- 18.88
- Type
- conference paper
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- Asia-Pacific Signal and Information Processing Association Annual Summit and Conference (APSIPA ASC) (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University) (public-domain)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Emotion and Mood Recognition
- Physical Sciences › Computer Science › Artificial Intelligence › Speech Recognition and Synthesis
- Physical Sciences › Computer Science › Signal Processing › Speech and Audio Processing
- Keywords
- speech emotion recognition, attention mechanism, fully convolutional network, time-frequency spectrograms, transfer learning, IEMOCAP
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.