Hidden Markov model-based speech emotion recognition
The DOI matches its Crossref record; 4 of the 4 references it cites were found in a registry or an index.
Abstract
In this contribution we introduce speech emotion recognition by use of continuous hidden Markov models. Two methods are propagated and compared throughout the paper. Within the first method a global statistics framework of an utterance is classified by Gaussian mixture models using derived features of the raw pitch and energy contour of the speech signal. A second method introduces increased temporal complexity applying continuous hidden Markov models considering several states using low-level instantaneous features instead of global statistics. The paper addresses the design of working recognition engines and results achieved with respect to the alluded alternatives. A speech corpus consisting of acted and spontaneous emotion samples in German and English language is described in detail. Both engines have been tested and trained using this equivalent speech corpus. Results in recognition of seven discrete emotions exceeded 86% recognition rate. As a basis of comparison the similar judgment of human deciders classifying the same corpus at 79.8% recognition rate was analyzed.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Emotion recognition in human-computer interaction
Cowie, R., Douglas-Cowie, E., Tsapatsoulis, N., Votsis, G., Kollias, S., Fellenz, W., & Taylor, J. G. (2001) · IEEE Signal Processing Magazine · doi:10.1109/79.911197 · Cited by 2,603
Speech emotion recognition using hidden Markov models
Nogueiras, A., Moreno, A., Bonafonte, A., & Mariño, J. B. (2001) · 7th European Conference on Speech Communication and Technology (Eurospeech 2001) · doi:10.21437/eurospeech.2001-627 · Cited by 232
Multimodal emotion recognition in audiovisual communication
Schuller, B., Lang, M., & Rigoll, G. (2003) · Proceedings. IEEE International Conference on Multimedia and Expo · doi:10.1109/icme.2002.1035889 · Cited by 48
Self-citationVerbal and non-verbal cues in the communication of emotions
Polzin, T. (2002) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp.2000.859332 · Cited by 26
Cited by
Survey on speech emotion recognition: Features, classification schemes, and databases
El Ayadi, M., Kamel, M. S., & Karray, F. (2011) · Pattern Recognition · doi:10.1016/j.patcog.2010.09.020 · Cited by 2,073
Speech emotion recognition using deep neural network and extreme learning machine
Han, K., Yu, D., & Tashev, I. (2014) · Interspeech 2014 · doi:10.21437/interspeech.2014-57 · Cited by 794
Schuller, B., Batliner, A., Steidl, S., & Seppi, D. (2011) · Speech Communication · doi:10.1016/j.specom.2011.01.011 · Cited by 758
Self-citationAkçay, M. B., & Oğuz, K. (2020) · Speech Communication · doi:10.1016/j.specom.2019.12.001 · Cited by 721
Sharma, N., & Gedeon, T. (2012) · Computer Methods and Programs in Biomedicine · doi:10.1016/j.cmpb.2012.07.003 · Cited by 510
EmotionSense: a mobile phones based adaptive platform for experimental social psychology research
Rachuri, K. K., Musolesi, M., Mascolo, C., Rentfrow, P. J., Longworth, C., & Aucinas, A. (2010) · Proceedings of the 12th ACM international conference on Ubiquitous computing · doi:10.1145/1864349.1864393 · Cited by 508
Features and classifiers for emotion recognition from speech: a survey from 2000 to 2011
Anagnostopoulos, C.-N., Iliou, T., & Giannoukos, I. (2015) · Artificial Intelligence Review · doi:10.1007/s10462-012-9368-5 · Cited by 428
“Hidden Markov Models (HMMs) are considered to be a common classification/modeling technique for speech emotion recognition (Schuller et al. 2003; Pao et al. 2007a; Ijima et al. 2009; Yun and Yoo 2009; Yu 2008; Fu et al. 2008b; Nogueiras et al. 2001).”
BackgroundMethodInfluentialSchuller, B., Rigoll, G., & Lang, M. (2004) · 2004 IEEE International Conference on Acoustics, Speech, and Signal Processing · doi:10.1109/icassp.2004.1326051 · Cited by 426
Self-citationSpeech emotion recognition using convolutional and Recurrent Neural Networks
Lim, W., Jang, D., & Lee, T. (2016) · 2016 Asia-Pacific Signal and Information Processing Association Annual Summit and Conference (APSIPA) · doi:10.1109/apsipa.2016.7820699 · Cited by 405
Emotion Recognition from Speech Using wav2vec 2.0 Embeddings
Pepino, L., Riera, P., & Ferrer, L. (2021) · Interspeech 2021 · doi:10.21437/interspeech.2021-703 · Cited by 389
Cross-Corpus Acoustic Emotion Recognition: Variances and Strategies
Schuller, B., Vlasenko, B., Eyben, F., Wollmer, M., Stuhlsatz, A., Wendemuth, A., & Rigoll, G. (2010) · IEEE Transactions on Affective Computing · doi:10.1109/t-affc.2010.8 · Cited by 388
Self-citationDatabases, features and classifiers for speech emotion recognition: a review
Swain, M., Routray, A., & Kabisatpathy, P. (2018) · International Journal of Speech Technology · doi:10.1007/s10772-018-9491-z · Cited by 356
Multimodal Speech Emotion Recognition Using Audio and Text
Yoon, S., Byun, S., & Jung, K. (2018) · 2018 IEEE Spoken Language Technology Workshop (SLT) · doi:10.1109/slt.2018.8639583 · Cited by 351
LSTM-Modeling of continuous emotions in an audiovisual affect recognition framework
Wöllmer, M., Kaiser, M., Eyben, F., Schuller, B., & Rigoll, G. (2013) · Image and Vision Computing · doi:10.1016/j.imavis.2012.03.001 · Cited by 288
Self-citationSpeech emotion recognition based on HMM and SVM
Lin, Y.-L., & Wei, G. (2005) · 2005 International Conference on Machine Learning and Cybernetics · doi:10.1109/icmlc.2005.1527805 · Cited by 260
Affective state estimation for human–robot Interaction
Kulic, D., & Croft, E. A. (2007) · IEEE Transactions on Robotics · doi:10.1109/tro.2007.904899 · Cited by 230
An experimental study of speech emotion recognition based on deep convolutional neural networks
Zheng, W. Q., Yu, J. S., & Zou, Y. X. (2015) · 2015 International Conference on Affective Computing and Intelligent Interaction (ACII) · doi:10.1109/acii.2015.7344669 · Cited by 187
“Some researchers also explored generative models, such as Gaussian mixture models (GMMs) and Hidden Markov models (HMMs), to learn the distribution of the extracted speech features, and then apply the Bayesian classifier or the maximum likelihood principle for emotion recognition [11][12].”
BackgroundFear-type emotion recognition for future audio-based surveillance systems
Clavel, C., Vasilescu, I., Devillers, L., Richard, G., & Ehrette, T. (2008) · Speech Communication · doi:10.1016/j.specom.2008.03.012 · Cited by 180
“…of interaction), recording conditions; – the emotional classes (number and type); – the training and test conditions (speaker-dependent or not) (Schuller et al., 2003) ; – the techniques for acoustic feature extraction which are more or less dependent of prior knowledge of the linguistic…”
BackgroundInfluentialError weighted semi-coupled hidden Markov model for audio-visual emotion recognition
Lin, J.-C., Wu, C.-H., & Wei, W.-L. (2012) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2011.2171334 · Cited by 121
Revisiting Hidden Markov Models for Speech Emotion Recognition
Mao, S., Tao, D., Zhang, G., Ching, P. C., & Lee, T. (2019) · ICASSP 2019 - 2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) · doi:10.1109/icassp.2019.8683172 · Cited by 83
Strength modelling for real-worldautomatic continuous affect recognition from audiovisual signals
Han, J., Zhang, Z., Cummins, N., Ringeval, F., & Schuller, B. (2017) · Image and Vision Computing · doi:10.1016/j.imavis.2016.11.020 · Cited by 53
Self-citationAdaptive Domain-Aware Representation Learning for Speech Emotion Recognition
Fan, W., Xu, X., Xing, X., & Huang, D. (2020) · Interspeech 2020 · doi:10.21437/interspeech.2020-2572 · Cited by 14
Metadata
- Authors
- Schuller, B., Rigoll, G., & Lang, M.
- Venue
- 2003 International Conference on Multimedia and Expo. ICME '03. Proceedings (Cat. No.03TH8698)
- Publisher
- IEEE
- Published
- 2003
- Pages
- I-401
- OpenAlex
- W2110052520
- Cited by
- 570
- References
- 4
- FWCI
- 29.22
- Type
- conference paper
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.