The DOI matches its Crossref record; 47 of the 47 references it cites were found in a registry or an index.
Speech emotion recognition with deep convolutional neural networks
Abstract
The speech emotion recognition (or, classification) is one of the most challenging topics in data science. In this work, we introduce a new architecture, which extracts mel-frequency cepstral coefficients, chromagram, mel-scale spectrogram, Tonnetz representation, and spectral contrast features from sound files and uses them as inputs for the one-dimensional Convolutional Neural Network for the identification of emotions using samples from the Ryerson Audio-Visual Database of Emotional Speech and Song (RAVDESS), Berlin (EMO-DB), and Interactive Emotional Dyadic Motion Capture (IEMOCAP) datasets. We utilize an incremental method for modifying our initial model in order to improve classification accuracy. All of the proposed models work directly with raw sound data without the need for conversion to visual representations, unlike some previous approaches. Based on experimental results, our best-performing model outperforms existing frameworks for RAVDESS and IEMOCAP, thus setting the new state-of-the-art. For the EMO-DB dataset, it outperforms all previous works except one but compares favorably with that one in terms of generality, simplicity, and applicability. Specifically, the proposed framework obtains 71.61% for RAVDESS with 8 classes, 86.1% for EMO-DB with 535 samples in 7 classes, 95.71% for EMO-DB with 520 samples in 7 classes, and 64.3% for IEMOCAP with 4 classes in speaker-independent audio classification tasks.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
1 retracted
Demircan, S., & Kahramanlı, H. (2016) · Neural Computing and Applications · doi:10.1007/s00521-016-2712-y · Cited by 65
RetractedDeep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474
IEMOCAP: Interactive emotional dyadic motion capture database
Busso, C., Bulut, M., Lee, C.-C., Kazemzadeh, A., Mower, E., Kim, S., Chang, J. N., Lee, S., & Narayanan, S. S. (2008) · Language Resources and Evaluation · doi:10.1007/s10579-008-9076-6 · Cited by 3,869
librosa: Audio and Music Signal Analysis in Python
McFee, B., Raffel, C., Liang, D., Ellis, D., McVicar, M., Battenberg, E., & Nieto, O. (2015) · Proceedings of the Python in Science Conference · doi:10.25080/majora-7b98e3ed-003 · Cited by 3,033
A database of German emotional speech
Burkhardt, F., Paeschke, A., Rolfes, M., Sendlmeier, W. F., & Weiss, B. (2005) · Interspeech 2005 · doi:10.21437/interspeech.2005-446 · Cited by 2,365
Livingstone, S. R., & Russo, F. A. (2018) · PLOS ONE · doi:10.1371/journal.pone.0196391 · Cited by 1,883
The Geneva Minimalistic Acoustic Parameter Set (GeMAPS) for Voice Research and Affective Computing
Eyben, F., Scherer, K. R., Schuller, B. W., Sundberg, J., Andre, E., Busso, C., Devillers, L. Y., Epps, J., Laukka, P., Narayanan, S. S., et al. (2016) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2015.2457417 · Cited by 1,818
A Scale for the Measurement of the Psychological Magnitude Pitch
Stevens, S. S., Volkmann, J., & Newman, E. B. (1937) · The Journal of the Acoustical Society of America · doi:10.1121/1.1915893 · Cited by 1,107
Speech emotion recognition using deep 1D & 2D CNN LSTM networks
Zhao, J., Mao, X., & Chen, L. (2019) · Biomedical Signal Processing and Control · doi:10.1016/j.bspc.2018.08.035 · Cited by 1,044
Adieu features? End-to-end speech emotion recognition using a deep convolutional recurrent network
Trigeorgis, G., Ringeval, F., Brueckner, R., Marchi, E., Nicolaou, M. A., Schuller, B., & Zafeiriou, S. (2016) · 2016 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) · doi:10.1109/icassp.2016.7472669 · Cited by 904
Speech emotion recognition using deep neural network and extreme learning machine
Han, K., Yu, D., & Tashev, I. (2014) · Interspeech 2014 · doi:10.21437/interspeech.2014-57 · Cited by 794
3-D Convolutional Recurrent Neural Networks With Attention Model for Speech Emotion Recognition
Chen, M., He, X., Yang, J., & Zhang, H. (2018) · IEEE Signal Processing Letters · doi:10.1109/lsp.2018.2860246 · Cited by 503
Speech Emotion Recognition from Spectrograms with Deep Convolutional Neural Network
Badshah, A. M., Ahmad, J., Rahim, N., & Baik, S. W. (2017) · 2017 International Conference on Platform Technology and Service (PlatCon) · doi:10.1109/platcon.2017.7883728 · Cited by 413
Speech Emotion Recognition Using Fourier Parameters
Wang, K., An, N., Li, B. N., Zhang, Y., & Li, L. (2015) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2015.2392101 · Cited by 408
Speech emotion recognition using convolutional and Recurrent Neural Networks
Lim, W., Jang, D., & Lee, T. (2016) · 2016 Asia-Pacific Signal and Information Processing Association Annual Summit and Conference (APSIPA) · doi:10.1109/apsipa.2016.7820699 · Cited by 405
Deep learning for robust feature generation in audiovisual emotion recognition
Kim, Y., Lee, H., & Provost, E. M. (2013) · 2013 IEEE International Conference on Acoustics, Speech and Signal Processing · doi:10.1109/icassp.2013.6638346 · Cited by 401
Speech Emotion Recognition Using CNN
Huang, Z., Dong, M., Mao, Q., & Zhan, Y. (2014) · Proceedings of the 22nd ACM international conference on Multimedia · doi:10.1145/2647868.2654984 · Cited by 398
Automatic speech emotion recognition using modulation spectral features
Wu, S., Falk, T. H., & Chan, W.-Y. (2011) · Speech Communication · doi:10.1016/j.specom.2010.08.013 · Cited by 389
High-level feature representation using recurrent neural network for speech emotion recognition
Lee, J., & Tashev, I. (2015) · Interspeech 2015 · doi:10.21437/interspeech.2015-336 · Cited by 388
Multimodal Speech Emotion Recognition Using Audio and Text
Yoon, S., Byun, S., & Jung, K. (2018) · 2018 IEEE Spoken Language Technology Workshop (SLT) · doi:10.1109/slt.2018.8639583 · Cited by 351
Music type classification by spectral contrast feature
Jiang, D.-N., Lu, L., Zhang, H.-J., Tao, J.-H., & Cai, L.-H. (2003) · Proceedings. IEEE International Conference on Multimedia and Expo · doi:10.1109/icme.2002.1035731 · Cited by 339
Fundamentals of Speaker Recognition
Beigi, H. (2011) · Springer eBooks · doi:10.1007/978-0-387-77592-0 · Cited by 305
Detecting harmonic change in musical audio
Harte, C., Sandler, M., & Gasser, M. (2006) · Proceedings of the 1st ACM workshop on Audio and music computing multimedia · doi:10.1145/1178723.1178727 · Cited by 276
The INTERSPEECH 2016 Computational Paralinguistics Challenge: Deception, Sincerity & Native Language
Schuller, B., Steidl, S., Batliner, A., Hirschberg, J., Burgoon, J. K., Baird, A., Elkins, A., Zhang, Y., Coutinho, E., & Evanini, K. (2016) · Interspeech 2016 · doi:10.21437/interspeech.2016-129 · Cited by 241
An experimental study of speech emotion recognition based on deep convolutional neural networks
Zheng, W. Q., Yu, J. S., & Zou, Y. X. (2015) · 2015 International Conference on Affective Computing and Intelligent Interaction (ACII) · doi:10.1109/acii.2015.7344669 · Cited by 187
Spectrogram based multi-task audio classification
Zeng, Y., Mao, H., Peng, D., & Yi, Z. (2019) · Multimedia Tools and Applications · doi:10.1007/s11042-017-5539-3 · Cited by 178
Data Augmentation Using GANs for Speech Emotion Recognition
Chatziagapi, A., Paraskevopoulos, G., Sgouropoulos, D., Πανταζόπουλος, Γ., Nikandrou, M., Γιαννακόπουλος, Θ., Katsamanis, A., Potamianos, A., & Narayanan, S. S. (2019) · Interspeech (USB) · doi:10.21437/interspeech.2019-2561 · Cited by 162
Self-Attention for Speech Emotion Recognition
Tarantino, L., Garner, P. N., & Lazaridis, A. (2019) · Interspeech (USB) · doi:10.21437/interspeech.2019-2822 · Cited by 137
Multi-Modal Emotion recognition on IEMOCAP Dataset using Deep Learning
Tripathi, S., Tripathi, S., & Beigi, H. (2018) · arXiv · doi:10.48550/arxiv.1804.05788 · Cited by 132
Speech Emotion Recognition Using Deep Neural Network Considering Verbal and Nonverbal Speech Sounds
Huang, K.-Y., Wu, C., Hong, Q.-B., Su, M.-H., & Chen, Y. (2019) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp.2019.8682283 · Cited by 125
2018 IEEE Spoken Language Technology Workshop (SLT)
(2018) · doi:10.1109/slt43100.2018 · Cited by 96
Towards Robust Speech Emotion Recognition Using Deep Residual Networks for Speech Enhancement
Triantafyllopoulos, A., Keren, G., Wagner, J., Steiner, I., & Schuller, B. W. (2019) · Interspeech (USB) · doi:10.21437/interspeech.2019-1811 · Cited by 77
Cross-corpus acoustic emotion recognition from singing and speaking: A multi-task learning approach
Zhang, B., Provost, E. M., & Georg, E. (2016) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp.2016.7472790 · Cited by 68
Voice recognition based on adaptive MFCC and deep learning
Bae, H.-S., Lee, H., & Lee, S.-G. (2016) · IEEE Conference on Industrial Electronics and Applications (ICIEA) · doi:10.1109/iciea.2016.7603830 · Cited by 62
Popova, A. S., Rassadin, A. G., & Ponomarenko, A. (2017) · Studies in computational intelligence · doi:10.1007/978-3-319-66604-4_18 · Cited by 44
Reusing Neural Speech Representations for Auditory Emotion RecognitionPreprint
Lakomkin, E., Weber, C., Magg, S., & Wermter, S. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1803.11508 · Cited by 39
Evaluation of MPEG-7 Descriptors for Speech Emotional Recognition
Lampropoulos, A. S., & Tsihrintzis, G. A. (2012) · International Conference on Intelligent Information Hiding and Multimedia Signal Processing (IIH-MSP) · doi:10.1109/iih-msp.2012.29 · Cited by 34
Improvement on Speech Emotion Recognition Based on Deep Convolutional Neural Networks
Niu, Y., Zou, D., Niu, Y., He, Z., & Tan, H. M. (2018) · International Conference on Computing and Artificial Intelligence (ICCAI) · doi:10.1145/3194452.3194460 · Cited by 27
Recognition of emotional speech with convolutional neural networks by means of spectral estimates
Weißkirchen, N., Böck, R., & Wendemuth, A. (2017) · International Conference on Affective Computing and Intelligent Interaction Workshops and Demos (ACIIW) · doi:10.1109/aciiw.2017.8272585 · Cited by 25
Real Time Multiple Face Recognition
Mittal, S., Agarwal, S., & Nigam, M. J. (2018) · International Conference on Digital Medicine and Image Processing (DMIP) · doi:10.1145/3299852.3299853 · Cited by 22
Cited by
Khare, S. K., Blanes-Vidal, V., Nadimi, E. S., & Acharya, U. R. (2024) · Information Fusion · doi:10.1016/j.inffus.2023.102019 · Cited by 469
Emotion Recognition from Speech Using wav2vec 2.0 Embeddings
Pepino, L., Riera, P., & Ferrer, L. (2021) · Interspeech 2021 · doi:10.21437/interspeech.2021-703 · Cited by 389
Pan, B., Hirota, K., Jia, Z., & Dai, Y. (2023) · Neurocomputing · doi:10.1016/j.neucom.2023.126866 · Cited by 142
Zhang, J., Wu, J., Qiu, Y., Song, A., Li, W., Li, X., & Liu, Y. (2023) · Computers in Biology and Medicine · doi:10.1016/j.compbiomed.2022.106517 · Cited by 69
State-of-the-art Multimodal Emotion Recognition: A comprehensive survey and taxonomy
Yazici, A., Kucukyilmaz, T., Dokeroglu, T., Sharipbay, A., Lee, M.-H., & Tyler, B. (2026) · Intelligent Systems with Applications · doi:10.1016/j.iswa.2026.200642 · Cited by 11
Metadata
- Authors
- Issa, D., Fatih Demirci, M., & Yazici, A.
- Venue
- Biomedical Signal Processing and Control
- Publisher
- Elsevier BV
- Published
- 2020-05
- Volume
- 59
- Pages
- 101894
- ISSN
- 1746-8094 (ISSN-L)
- OpenAlex
- W3008039831
- Cited by
- 619
- References
- 47
- FWCI
- 73.79
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.