The DOI matches its Crossref record; 84 of the 84 references it cites were found in a registry or an index.
Deep Learning Techniques for Speech Emotion Recognition, from Databases to Models
Open accessAbstract
The advancements in neural networks and the on-demand need for accurate and near real-time Speech Emotion Recognition (SER) in human–computer interactions make it mandatory to compare available methods and databases in SER to achieve feasible solutions and a firmer understanding of this open-ended problem. The current study reviews deep learning approaches for SER with available datasets, followed by conventional machine learning techniques for speech emotion recognition. Ultimately, we present a multi-aspect comparison between practical neural network approaches in speech emotion recognition. The goal of this study is to provide a survey of the field of discrete speech emotion recognition.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
ImageNet classification with deep convolutional neural networks
Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017) · Communications of the ACM · doi:10.1145/3065386 · Cited by 112,155
Hochreiter, S., & Schmidhuber, J. (1997) · Neural Computation · doi:10.1162/neco.1997.9.8.1735 · Cited by 102,318
Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
Ioffe, S., & Szegedy, C. (2015) · arXiv · doi:10.48550/arxiv.1502.03167 · Cited by 27,818
A tutorial on hidden Markov models and selected applications in speech recognition
Rabiner, L. R. (1989) · Proceedings of the IEEE · doi:10.1109/5.18626 · Cited by 22,743
Generative adversarial networks
Goodfellow, I., Pouget-Abadie, J., Mirza, M., Xu, B., Warde-Farley, D., Ozair, S., Courville, A., & Bengio, Y. (2020) · Communications of the ACM · doi:10.1145/3422622 · Cited by 18,384
Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
Ioffe, S. (2024) · arXiv (Cornell University) · doi:10.57702/o9raffed · Cited by 14,214
Conditional Generative Adversarial Nets
Mirza, M., & Osindero, S. (2014) · arXiv · doi:10.48550/arxiv.1411.1784 · Cited by 8,863
Greff, K., Srivastava, R. K., Koutnik, J., Steunebrink, B. R., & Schmidhuber, J. (2017) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2016.2582924 · Cited by 6,955
Receptive fields and functional architecture of monkey striate cortex
Hubel, D. H., & Wiesel, T. N. (1968) · The Journal of Physiology · doi:10.1113/jphysiol.1968.sp008455 · Cited by 6,568
Fast and Accurate Deep Network Learning by Exponential Linear Units (ELUs)
Clevert, D.-A., Unterthiner, T., & Hochreiter, S. (2015) · arXiv · doi:10.48550/arxiv.1511.07289 · Cited by 4,513
IEMOCAP: Interactive emotional dyadic motion capture database
Busso, C., Bulut, M., Lee, C.-C., Kazemzadeh, A., Mower, E., Kim, S., Chang, J. N., Lee, S., & Narayanan, S. S. (2008) · Language Resources and Evaluation · doi:10.1007/s10579-008-9076-6 · Cited by 3,869
A database of German emotional speech
Burkhardt, F., Paeschke, A., Rolfes, M., Sendlmeier, W. F., & Weiss, B. (2005) · Interspeech 2005 · doi:10.21437/interspeech.2005-446 · Cited by 2,365
Livingstone, S. R., & Russo, F. A. (2018) · PLOS ONE · doi:10.1371/journal.pone.0196391 · Cited by 1,883
Mehrabian, A. (1996) · Current Psychology · doi:10.1007/bf02686918 · Cited by 1,493
Speech emotion recognition using deep 1D & 2D CNN LSTM networks
Zhao, J., Mao, X., & Chen, L. (2019) · Biomedical Signal Processing and Control · doi:10.1016/j.bspc.2018.08.035 · Cited by 1,044
Toward detecting emotions in spoken dialogs
Lee, C. M., & Narayanan, S. S. (2005) · IEEE Transactions on Speech and Audio Processing · doi:10.1109/tsa.2004.838534 · Cited by 988
Speech emotion recognition using hidden Markov models
Nwe, T. L., Foo, S. W., & De Silva, L. C. (2003) · Speech Communication · doi:10.1016/s0167-6393(03)00099-2 · Cited by 920
CREMA-D: Crowd-Sourced Emotional Multimodal Actors Dataset
Cao, H., Cooper, D. G., Keutmann, M. K., Gur, R. C., Nenkova, A., & Verma, R. (2014) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2014.2336244 · Cited by 914
Adieu features? End-to-end speech emotion recognition using a deep convolutional recurrent network
Trigeorgis, G., Ringeval, F., Brueckner, R., Marchi, E., Nicolaou, M. A., Schuller, B., & Zafeiriou, S. (2016) · 2016 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) · doi:10.1109/icassp.2016.7472669 · Cited by 904
Speech emotion recognition using deep neural network and extreme learning machine
Han, K., Yu, D., & Tashev, I. (2014) · Interspeech 2014 · doi:10.21437/interspeech.2014-57 · Cited by 794
The role of voice quality in communicating emotion, mood and attitude
Gobl, C. (2003) · Speech Communication · doi:10.1016/s0167-6393(02)00082-1 · Cited by 787
Automatic speech emotion recognition using recurrent neural networks with local attention
Mirsamadi, S., Barsoum, E., & Zhang, C. (2017) · 2017 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) · doi:10.1109/icassp.2017.7952552 · Cited by 770
Emotions and speech: some acoustical correlates.
Williams, C. E., & Stevens, K. N. (1972) · The Journal of the Acoustical Society of America · doi:10.1121/1.1913238 · Cited by 735 · cited 1×
“Understanding the feelings in correlation with the message comprehended will be an essential key to a fruitful conversation [34].”
BackgroundAkçay, M. B., & Oğuz, K. (2020) · Speech Communication · doi:10.1016/j.specom.2019.12.001 · Cited by 721
Speech Emotion Recognition Using Deep Learning Techniques: A Review
Khalil, R. A., Jones, E., Babar, M. I., Jan, T., Zafar, M. H., & Alhussain, T. (2019) · IEEE Access · doi:10.1109/access.2019.2936124 · Cited by 702
Acoustical properties of speech as indicators of depression and suicidal risk
France, D. J., Shiavi, R. G., Silverman, S., Silverman, M., & Wilkes, M. (2000) · IEEE Transactions on Biomedical Engineering · doi:10.1109/10.846676 · Cited by 504
Zhang, S., Zhang, S., Huang, T., & Gao, W. (2018) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2017.2766843 · Cited by 441
SenticNet 6: Ensemble Application of Symbolic and Subsymbolic AI for Sentiment Analysis
Cambria, E., Li, Y., Xing, F. Z., Poria, S., & Kwok, K. (2020) · Proceedings of the 29th ACM International Conference on Information & Knowledge Management · doi:10.1145/3340531.3412003 · Cited by 437
Schuller, B., Rigoll, G., & Lang, M. (2004) · 2004 IEEE International Conference on Acoustics, Speech, and Signal Processing · doi:10.1109/icassp.2004.1326051 · Cited by 426
The Vera am Mittag German audio-visual emotional speech database
Grimm, M., Kroschel, K., & Narayanan, S. (2008) · 2008 IEEE International Conference on Multimedia and Expo · doi:10.1109/icme.2008.4607572 · Cited by 407
Databases, features and classifiers for speech emotion recognition: a review
Swain, M., Routray, A., & Kabisatpathy, P. (2018) · International Journal of Speech Technology · doi:10.1007/s10772-018-9491-z · Cited by 356
Emotion Recognition in Speech Using Neural Networks
Nicholson, J., Takahashi, K., & Nakatsu, R. (2000) · Neural Computing & Applications · doi:10.1007/s005210070006 · Cited by 344
Ensemble methods for spoken emotion recognition in call-centres
Morrison, D., Wang, R., & De Silva, L. C. (2007) · Speech Communication · doi:10.1016/j.specom.2006.11.004 · Cited by 320
LSTM-Modeling of continuous emotions in an audiovisual affect recognition framework
Wöllmer, M., Kaiser, M., Eyben, F., Schuller, B., & Rigoll, G. (2013) · Image and Vision Computing · doi:10.1016/j.imavis.2012.03.001 · Cited by 288 · cited 3×
“An early proposal for using LSTM networks in 2013 was in the work of Martin Wöllmer et al. [28].”
MethodDeep neural networks for acoustic emotion recognition: Raising the benchmarks
Stuhlsatz, A., Meyer, C., Eyben, F., Zielke, T., Meier, G., & Schuller, B. (2011) · 2011 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) · doi:10.1109/icassp.2011.5947651 · Cited by 288
Speech Emotion Classification Using Attention-Based LSTM
Xie, Y., Liang, R., Liang, Z., Huang, C., Zou, C., & Schuller, B. (2019) · IEEE/ACM Transactions on Audio, Speech, and Language Processing · doi:10.1109/taslp.2019.2925934 · Cited by 282
EMOTION IN SPEECH: RECOGNITION AND APPLICATION TO CALL CENTERS
Petrushin, V. A. (1999) · Cited by 276
Speech emotion recognition based on HMM and SVM
Lin, Y.-L., & Wei, G. (2005) · 2005 International Conference on Machine Learning and Cybernetics · doi:10.1109/icmlc.2005.1527805 · Cited by 260
Deep Learning Detection of Electricity Theft Cyber-Attacks in Renewable Distributed Generation
Ismail, M., Shaaban, M. F., Naidu, M., & Serpedin, E. (2020) · IEEE Transactions on Smart Grid · doi:10.1109/tsg.2020.2973681 · Cited by 257
Akhtar, M. S., Ekbal, A., & Cambria, E. (2020) · IEEE Computational Intelligence Magazine · doi:10.1109/mci.2019.2954667 · Cited by 246
Speech emotion recognition using hidden Markov models
Nogueiras, A., Moreno, A., Bonafonte, A., & Mariño, J. B. (2001) · 7th European Conference on Speech Communication and Technology (Eurospeech 2001) · doi:10.21437/eurospeech.2001-627 · Cited by 232
Introduction to Human-Computer Interaction
(1995) · Readings in Human–Computer Interaction · doi:10.1016/b978-0-08-051574-8.50005-4 · Cited by 225
Improved End-to-End Speech Emotion Recognition Using Self Attention Mechanism and Multitask Learning
Li, Y., Zhao, T., & Kawahara, T. (2019) · Interspeech 2019 · doi:10.21437/interspeech.2019-2594 · Cited by 180
Deep-Net: A Lightweight CNN-Based Speech Emotion Recognition System Using Deep Frequency Features
Anvarjon, T., Mustaqeem, & Kwon, S. (2020) · Sensors · doi:10.3390/s20185212 · Cited by 174
Speech Emotion Recognition Using Spectrogram & Phoneme Embedding
Yenigalla, P., Kumar, A., Tripathi, S., Singh, C., Kar, S., & Vepa, J. (2018) · Interspeech 2018 · doi:10.21437/interspeech.2018-1811 · Cited by 164
Data Augmentation Using GANs for Speech Emotion Recognition
Chatziagapi, A., Paraskevopoulos, G., Sgouropoulos, D., Πανταζόπουλος, Γ., Nikandrou, M., Γιαννακόπουλος, Θ., Katsamanis, A., Potamianos, A., & Narayanan, S. S. (2019) · Interspeech (USB) · doi:10.21437/interspeech.2019-2561 · Cited by 162
The Belfast Induced Natural Emotion Database
Sneddon, I., McRorie, M., McKeown, G. J., & Hanratty, J. (2011) · IEEE Transactions on Affective Computing · doi:10.1109/t-affc.2011.26 · Cited by 154
Speech Emotion Recognition using Support Vector Machine
Chavhan, Y., Dhore, M. L., & Yesaware, P. (2010) · International Journal of Computer Applications · doi:10.5120/431-636 · Cited by 144
A multi-layer and multi-ensemble stock trader using deep learning and deep reinforcement learning
Carta, S., Corriga, A., Ferreira, A., Podda, A. S., & Recupero, D. R. (2020) · Applied Intelligence · doi:10.1007/s10489-020-01839-5 · Cited by 136
Speech emotion recognition using Support Vector Machines
Seehapoch, T., & Wongthanavasu, S. (2013) · International Conference on Knowledge and Smart Technology · doi:10.1109/kst.2013.6512793 · Cited by 130
Design, recording and verification of a danish emotional speech database
Engberg, I. S., Hansen, A. V., Andersen, O., & Dalsgaard, P. (1997) · European Conference on Speech Communication and Technology (Eurospeech) · doi:10.21437/eurospeech.1997-482 · Cited by 130
Huang, C.-W., & Narayanan, S. S. (2017) · IEEE International Conference on Multimedia and Expo (ICME) · doi:10.1109/icme.2017.8019296 · Cited by 130
The Relevance of Voice Quality Features in Speaker Independent Emotion Recognition
Lugger, M., & Yang, B. (2007) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp.2007.367152 · Cited by 126
A review on emotion recognition using speech
Basu, S., Chakraborty, J., Bag, A., & Aftabuddin, M. (2017) · International Conference Inventive Communication and Computational Technologies · doi:10.1109/icicct.2017.7975169 · Cited by 119
A first look into a Convolutional Neural Network for speech emotion detection
Bertero, D., & Fung, P. (2017) · 2017 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) · doi:10.1109/icassp.2017.7953131 · Cited by 117
Variational Autoencoders for Learning Latent Representations of Speech Emotion: A Preliminary Study
Latif, S., Rana, R. K., Qadir, J., & Epps, J. (2018) · Interspeech (USB) · doi:10.21437/interspeech.2018-1568 · Cited by 116
Hybrid Deep Learning Approach for Multi-Step-Ahead Daily Rainfall Prediction Using GCM Simulations
Khan, M. I., & Maity, R. (2020) · IEEE Access · doi:10.1109/access.2020.2980977 · Cited by 115
Sentic Computing: Exploitation of Common Sense for the Development of Emotion-Sensitive Systems
Cambria, E., Hussain, A., Havasi, C., & Eckl, C. (2010) · Lecture notes in computer science · doi:10.1007/978-3-642-12397-9_12 · Cited by 111
Speech emotion recognition with deep learning
Harár, P., Bürget, R., & Dutta, M. K. (2017) · International Conference on Signal Processing and Integrated Networks (SPIN) · doi:10.1109/spin.2017.8049931 · Cited by 111
Kim, J., Englebienne, G., Truong, K. P., & Evers, V. (2017) · Interspeech (USB) · doi:10.21437/interspeech.2017-736 · Cited by 94
Transfer Linear Subspace Learning for Cross-Corpus Speech Emotion Recognition
Song, P. (2017) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2017.2705696 · Cited by 93
Cresceptron: a self-organizing neural network which grows adaptively
Weng, J., Ahuja, N., & Huang, T. S. (2003) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn.1992.287150 · Cited by 92
Timing levels in segment-based speech emotion recognition
Schuller, B. W., & Rigoll, G. (2006) · Interspeech (USB) · doi:10.21437/interspeech.2006-502 · Cited by 86
Analysis of Emotional Speech—A Review
Gangamohan, P., Kadiri, S. R., & Yegnanarayana, B. (2016) · Intelligent systems reference library · doi:10.1007/978-3-319-31056-5_11 · Cited by 75
Hansen, J. H. L., & Cairns, D. A. (1995) · Speech Communication · doi:10.1016/0167-6393(95)00007-b · Cited by 71
Unsupervised Learning Approach to Feature Analysis for Automatic Speech Emotion Recognition
Eskimez, Ş. E., Duan, Z., & Heinzelman, W. (2018) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp.2018.8462685 · Cited by 69
On Enhancing Speech Emotion Recognition Using Generative Adversarial Networks
Sahu, S., Gupta, R., & Espy-Wilson, C. (2018) · Interspeech (USB) · doi:10.21437/interspeech.2018-1883 · Cited by 63
Speech-based recognition of self-reported and observed emotion in a dimensional space
Truong, K. P., van Leeuwen, D. A., & de Jong, F. M. G. (2012) · Speech Communication · doi:10.1016/j.specom.2012.04.006 · Cited by 62
Negative Emotion Recognition using Deep Learning for Thai Language
Mekruksavanich, S., Jitpattanakul, A., & Hnoohom, N. (2020) · Joint International Conference on Digital Arts, Media and Technology with ECTI Northern Section Conference on Electrical, Electronics, Computer and Telecommunications Engineering (ECTI DAMT & NCON) · doi:10.1109/ectidamtncon48261.2020.9090768 · Cited by 62
Multi-class and hierarchical SVMs for emotion recognition
Hassan, A., & Damper, R. I. (2010) · Interspeech (USB) · doi:10.21437/interspeech.2010-644 · Cited by 57
Sentiment extraction from natural audio streams
Kaushik, L., Sangwan, A., & Hansen, J. H. L. (2013) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp.2013.6639321 · Cited by 57
Dupuis, K., & Pichora-Fuller, M. K. (2011) · Canadian acoustics · Cited by 55
Speech Emotion Recognition Using Transfer Learning
Song, P., Jin, Y., Zhao, L., & Xin, M. (2014) · IEICE Transactions on Information and Systems · doi:10.1587/transinf.2014edl8038 · Cited by 55
Tiwari, U., Soni, M. H., Chakraborty, R., Panda, A., & Kopparapu, S. K. (2020) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp40776.2020.9053581 · Cited by 54
Effective Attention Mechanism in Dynamic Models for Speech Emotion Recognition
Hsiao, P.-W., & Chen, C.-P. (2018) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp.2018.8461431 · Cited by 52
Emotion recognition and its application to computer agents with spontaneous interactive capabilities
Nakatsu, R., Nicholson, J., & Tosa, N. (2000) · Knowledge-Based Systems · doi:10.1016/s0950-7051(00)00070-8 · Cited by 46
Emotion detection in speech using deep networks
Amer, M. R., Siddiquie, B., Richey, C., & Divakaran, A. (2014) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp.2014.6854297 · Cited by 41
Duan, P., He, Z., He, Y., Liu, F., Zhang, A., & Zhou, D. (2020) · Computers & Industrial Engineering · doi:10.1016/j.cie.2020.106643 · Cited by 38
Emotion Recognition and Classification in Speech using Artificial Neural Networks
Shaw, A., Kumar, R. S., & Saxena, S. (2016) · International Journal of Computer Applications · doi:10.5120/ijca2016910710 · Cited by 37
Speech Emotion Recognition Using Support Vector Machines
Yu, C., Tian, Q., Cheng, F. X., & Zhang, S. (2011) · Communications in computer and information science · doi:10.1007/978-3-642-21402-8_35 · Cited by 36
Emotion recognition from Marathi speech database using adaptive artificial neural network
Darekar, R. V., & Dhande, A. (2018) · Biologically Inspired Cognitive Architectures · doi:10.1016/j.bica.2018.01.002 · Cited by 33
Vlasenko, B., Prylipko, D., Philippou-Hübner, D., & Wendemuth, A. (2011) · Interspeech (USB) · doi:10.21437/interspeech.2011-474 · Cited by 32
Kim, J., Englebienne, G., Truong, K. P., et al. (2017) · University of Twente Research Information · doi:10.48550/arxiv.1708.03920 · Cited by 15
Emotion analysis from speech using temporal contextual trajectories
Balti, H., & Elmaghraby, A. S. (2014) · IEEE Symposium on Computers and Communications (ISCC) · doi:10.1109/iscc.2014.6912616 · Cited by 7
Self-citation
Cited by
Alam, M. S. B., Lameesa, A., Sharmin, S., Afrin, S., Ahmed, S. F., Nikoo, M. R., & Gandomi, A. H. (2025) · Digital Communications and Networks · doi:10.1016/j.dcan.2025.04.001 · Cited by 9
A Neural Network Architecture for Children’s Audio–Visual Emotion Recognition
(2023) · doi:10.3390/math11224573 · Cited by 3
Metadata
- Authors
- Abbaschian, B. J., Sierra-Sosa, D., & Elmaghraby, A.
- Venue
- Sensors
- Publisher
- MDPI AG
- Published
- 2021-02-10
- Volume
- 21
- Issue
- 4
- Pages
- 1249
- ISSN
- 1424-8220 (ISSN-L)
- OpenAlex
- W3126625480
- Cited by
- 293
- References
- 84
- FWCI
- 50.15
- Type
- journal article
- Language
- en
- Open access
- gold
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.