Deep neural networks for acoustic emotion recognition: Raising the benchmarks
Open accessAbstract
Deep Neural Networks (DNNs) denote multilayer artificial neural networks with more than one hidden layer and millions of free parameters. We propose a Generalized Discriminant Analysis (GerDA) based on DNNs to learn discriminative features of low dimension optimized with respect to a fast classification from a large set of acoustic features for emotion recognition. On nine frequently used emotional speech corpora, we compare the performance of GerDA features and their subsequent linear classification with previously reported benchmarks obtained using the same set of acoustic features classified by Support Vector Machines (SVMs). Our results impressively show that low-dimensional GerDA features capture hidden information from the acoustic features leading to a significantly raised unweighted average recall and considerably raised weighted average recall.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Dimensionality reduction of handcrafted features with DNNs can lead to improvements in accuracy over various databases.
“Dimensionality reduction of handcrafted features with DNNs can lead to improvements in accuracy over various databases [161].”
From Deep learning for human affect recognition: Insights and new developments (2021) · cites this paper as [161]
Cites
Reducing the dimensionality of data with neural networks
Hinton, G. E., & Salakhutdinov, R. R. (2006) · Science · doi:10.1126/science.1127647 · Cited by 20,928 · see citing works
A database of German emotional speech
Burkhardt, F., Paeschke, A., Rolfes, M., Sendlmeier, W. F., & Weiss, B. (2005) · Interspeech 2005 · doi:10.21437/interspeech.2005-446 · Cited by 2,365 · see citing works
The INTERSPEECH 2009 emotion challenge
Schuller, B., Steidl, S., & Batliner, A. (2009) · Interspeech 2009 · doi:10.21437/interspeech.2009-103 · Cited by 938 · see citing works
Self-citationThe eNTERFACE’05 Audio-Visual Emotion Database
Martin, O., Kotsia, I., Macq, B., & Pitas, I. (2006) · 22nd International Conference on Data Engineering Workshops (ICDEW'06) · doi:10.1109/icdew.2006.145 · Cited by 615 · see citing works
OpenEAR — Introducing the munich open-source emotion and affect recognition toolkit
Eyben, F., Wollmer, M., & Schuller, B. (2009) · 2009 3rd International Conference on Affective Computing and Intelligent Interaction and Workshops · doi:10.1109/acii.2009.5349350 · Cited by 418 · see citing works
Self-citationThe Vera am Mittag German audio-visual emotional speech database
Grimm, M., Kroschel, K., & Narayanan, S. S. (2008) · IEEE International Conference on Multimedia and Expo (ICME) · doi:10.1109/icme.2008.4607572 · Cited by 413 · see citing works
Cross-Corpus Acoustic Emotion Recognition: Variances and Strategies
Schuller, B. W., Vlasenko, B., Eyben, F., Wöllmer, M., Stuhlsatz, A., Wendemuth, A., & Rigoll, G. (2010) · IEEE Transactions on Affective Computing · doi:10.1109/t-affc.2010.8 · Cited by 386 · see citing works
Self-citationWöllmer, M., Eyben, F., Reiter, S. A., Schuller, B. W., Cox, C., Douglas‐Cowie, E., & Cowie, R. (2008) · Interspeech (USB) · doi:10.21437/interspeech.2008-192 · Cited by 329 · see citing works
Self-citationGetting started with SUSAS: a speech under simulated and actual stress database
Hansen, J. H. L., & Bou-Ghazale, S. E. (1997) · European Conference on Speech Communication and Technology (Eurospeech) · doi:10.21437/eurospeech.1997-494 · Cited by 301 · see citing works
Acoustic emotion recognition: A benchmark comparison of performances
Schuller, B. W., Vlasenko, B., Eyben, F., Rigoll, G., & Wendemuth, A. (2009) · IEEE Workshop on Automatic Speech Recognition and Understanding (ASRU) · doi:10.1109/asru.2009.5372886 · Cited by 275 · see citing works
Self-citationSchuller, B. W., Müller, R., Eyben, F., Gast, J., Hörnler, B., Wöllmer, M., Rigoll, G., Hoethker, A., & Konosu, H. (2009) · Image and Vision Computing · doi:10.1016/j.imavis.2009.02.013 · Cited by 194 · see citing works
Self-citationAudiovisual Behavior Modeling by Combined Feature Spaces
Schuller, B. W., Arsić, D., Rigoll, G., Wimmer, M., & Radig, B. (2007) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp.2007.366340 · Cited by 82 · see citing works
Self-citationDevelopment of the UserState Conventions for the Multimodal Corpus in SmartKom
Steininger, S., Rabold, S., Dioubina, O. I., & Schiel, F. (2002) · Cited by 32 · see citing works
Discriminative feature extraction with Deep Neural Networks
Stuhlsatz, A., Lippel, J., & Zielke, T. (2010) · 2022 International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn.2010.5596329 · Cited by 20 · see citing works
Self-citation
Flagged first, then by citations
Cited by
Sorted by citations
Speech emotion recognition using deep 1D & 2D CNN LSTM networks
Zhao, J., Mao, X., & Chen, L. (2019) · Biomedical Signal Processing and Control · doi:10.1016/j.bspc.2018.08.035 · Cited by 1,044 · see citing works
Automatic speech emotion recognition using recurrent neural networks with local attention
Mirsamadi, S., Barsoum, E., & Zhang, C. (2017) · 2017 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) · doi:10.1109/icassp.2017.7952552 · Cited by 770 · see citing works
Mao, Q., Dong, M., Huang, Z., & Zhan, Y. (2014) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2014.2360798 · Cited by 568 · see citing works
Evaluating deep learning architectures for Speech Emotion Recognition
Fayek, H. M., Lech, M., & Cavedon, L. (2017) · Neural Networks · doi:10.1016/j.neunet.2017.02.013 · Cited by 554 · see citing works
Deep learning for robust feature generation in audiovisual emotion recognition
Kim, Y., Lee, H., & Provost, E. M. (2013) · 2013 IEEE International Conference on Acoustics, Speech and Signal Processing · doi:10.1109/icassp.2013.6638346 · Cited by 401 · see citing works
Deep learning for human affect recognition: Insights and new developments
Rouast, P. V., Adam, M. T. P., & Chiong, R. (2021) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2018.2890471 · Cited by 243 · see citing works
Attention Based Fully Convolutional Network for Speech Emotion Recognition
Zhang, Y., Du, J., Wang, Z., Zhang, J., & Tu, Y. (2018) · 2018 Asia-Pacific Signal and Information Processing Association Annual Summit and Conference (APSIPA ASC) · doi:10.23919/apsipa.2018.8659587 · Cited by 161 · see citing works
Yao, Z., Wang, Z., Liu, W., Liu, Y., & Pan, J. (2020) · Speech Communication · doi:10.1016/j.specom.2020.03.005 · Cited by 152 · see citing works
LIGHT-SERNET: A Lightweight Fully Convolutional Neural Network for Speech Emotion Recognition
Aftab, A., Morsali, A., Ghaemmaghami, S., & Champagne, B. (2022) · ICASSP 2022 - 2022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) · doi:10.1109/icassp43922.2022.9746679 · Cited by 130 · see citing works
Towards real-time Speech Emotion Recognition using deep neural networks
Fayek, H. M., Lech, M., & Cavedon, L. (2015) · 2015 9th International Conference on Signal Processing and Communication Systems (ICSPCS) · doi:10.1109/icspcs.2015.7391796 · Cited by 103 · see citing works
DNN-based Emotion Recognition Based on Bottleneck Acoustic Features and Lexical Features
Kim, E., & Shin, J. W. (2019) · ICASSP 2019 - 2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) · doi:10.1109/icassp.2019.8683077 · Cited by 54 · see citing works
Emotion recognition from thermal infrared images using deep Boltzmann machine
Wang, S., He, M., Gao, Z., He, S., & Ji, Q. (2014) · Frontiers of Computer Science · doi:10.1007/s11704-014-3295-3 · Cited by 51 · see citing works
Speech Emotion Recognition via an Attentive Time–Frequency Neural Network
Lu, C., Zheng, W., Lian, H., Zong, Y., Tang, C., Li, S., & Zhao, Y. (2023) · IEEE Transactions on Computational Social Systems · doi:10.1109/tcss.2022.3219825 · Cited by 51 · see citing works
“…the extraction of deep features, generally, spectrogram features (e.g., MFCC, magnitude spectrogram, and Mel-spectrogram) with rich time– frequency information are often used as input features of DNNs to learn the high(er)-level time–frequency patterns of speech signal for SER [7], [8], [9], [13].”
BackgroundMethodInfluentialUsing denoising autoencoder for emotion recognition
Xia, R., & Liu, Y. (2013) · Interspeech 2013 · doi:10.21437/interspeech.2013-256 · Cited by 28
- Authors
- Stuhlsatz, A., Meyer, C., Eyben, F., Zielke, T., Meier, G., & Schuller, B.
- Affiliations
- Hochschule Düsseldorf University of Applied Sciences (DE)
- Technical University of Munich (DE)
- Venue
- 2011 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)
- Publisher
- IEEE
- Published
- 2011-05
- Pages
- 5688-5691
- ISSN
- 1520-6149 (ISSN-L)
- OpenAlex
- W2099767163
- Cited by
- 288
- References
- 14
- Citations by year
- 2026: 2 · 2025: 7 · 2024: 18 · 2023: 24 · 2022: 23 · 2021: 34 · 2020: 28 · 2019: 34 · 2018: 25 · 2017: 20 · 2016: 13 · 2015: 18 · 2014: 19 · 2013: 11 · 2012: 10
- FWCI
- 21.36
- Type
- conference paper
- Language
- en
- Open access
- green
- mediatum.ub.tum.de
- Licence
- other-oa
- Locations
- IEEE International Conference on Acoustics Speech and Signal Processing (publishedVersion)
- mediaTUM – the media and publications repository of the Technical University Munich (Technical University Munich) (submittedVersion, other-oa)
- OPUS (Augsburg University) (submittedVersion)
- Topics
- Social Sciences › Psychology › Experimental and Cognitive Psychology › Emotion and Mood Recognition
- Physical Sciences › Computer Science › Artificial Intelligence › Speech Recognition and Synthesis
- Physical Sciences › Computer Science › Signal Processing › Speech and Audio Processing
- Keywords
- acoustic emotion recognition, deep neural network, generalized discriminant analysis, feature extraction, linear classification, support vector machine, benchmark comparison
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
