The DOI matches its Crossref record.
Recognizing emotion in speech
Abstract
This paper explores several statistical pattern recognition techniques to classify utterances according to their emotional content.We have recorded a corpus containing emotional speech with over a 1000 utterances from different speakers.We present a new method of extracting prosodic features from speech, based on a smoothing spline approximation of the pitch contour.To make maximal use of the limited amount of training data available, we introduce a novel pattern recognition technique: majority voting of subspace specialists.Using this technique, we obtain classification performance that is close to human performance on the task.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cited by
A Survey of Affect Recognition Methods: Audio, Visual, and Spontaneous Expressions
Zeng, Z., Pantic, M., Roisman, G. I., & Huang, T. S. (2009) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2008.52 · Cited by 2,797
A review of affective computing: From unimodal analysis to multimodal fusion
Poria, S., Cambria, E., Bajpai, R., & Hussain, A. (2017) · Information Fusion · doi:10.1016/j.inffus.2017.02.003 · Cited by 1,644
Toward detecting emotions in spoken dialogs
Lee, C. M., & Narayanan, S. S. (2005) · IEEE Transactions on Speech and Audio Processing · doi:10.1109/tsa.2004.838534 · Cited by 988
Emotional speech recognition: Resources, features, and methods
Ververidis, D., & Kotropoulos, C. (2006) · Speech Communication · doi:10.1016/j.specom.2006.04.003 · Cited by 925
Speech emotion recognition using hidden Markov models
Nwe, T. L., Foo, S. W., & De Silva, L. C. (2003) · Speech Communication · doi:10.1016/s0167-6393(03)00099-2 · Cited by 920
Analysis of emotion recognition using facial expressions, speech and multimodal information
Busso, C., Deng, Z., Yildirim, S., Bulut, M., Lee, C. M., Kazemzadeh, A., Lee, S., Neumann, U., & Narayanan, S. (2004) · Proceedings of the 6th international conference on Multimodal interfaces · doi:10.1145/1027933.1027968 · Cited by 840
Toward an affect-sensitive multimodal human-computer interaction
Pantic, M., & Rothkrantz, L. J. M. (2003) · Proceedings of the IEEE · doi:10.1109/jproc.2003.817122 · Cited by 791
Schuller, B., Batliner, A., Steidl, S., & Seppi, D. (2011) · Speech Communication · doi:10.1016/j.specom.2011.01.011 · Cited by 758
Speech Emotion Recognition Using Deep Learning Techniques: A Review
Khalil, R. A., Jones, E., Babar, M. I., Jan, T., Zafar, M. H., & Alhussain, T. (2019) · IEEE Access · doi:10.1109/access.2019.2936124 · Cited by 702
Characteristic-Based Clustering for Time Series Data
Wang, X., Smith, K., & Hyndman, R. (2006) · Data Mining and Knowledge Discovery · doi:10.1007/s10618-005-0039-x · Cited by 679
Emotion recognition from speech: a review
Koolagudi, S. G., & Rao, K. S. (2012) · International Journal of Speech Technology · doi:10.1007/s10772-011-9125-1 · Cited by 641
A survey of multimodal sentiment analysis
Soleymani, M., Garcia, D., Jou, B., Schuller, B., Chang, S.-F., & Pantic, M. (2017) · Image and Vision Computing · doi:10.1016/j.imavis.2017.08.003 · Cited by 601
Fusing audio, visual and textual clues for sentiment analysis from multimodal content
Poria, S., Cambria, E., Howard, N., Huang, G.-B., & Hussain, A. (2016) · Neurocomputing · doi:10.1016/j.neucom.2015.01.095 · Cited by 534
Speech emotion recognition: two decades in a nutshell, benchmarks, and ongoing trends
Schuller, B. W. (2018) · Communications of the ACM · doi:10.1145/3129340 · Cited by 541
Tao, J., & Tan, T. (2005) · Lecture Notes in Computer Science · doi:10.1007/11573548_125 · Cited by 494
Conversational Memory Network for Emotion Recognition in Dyadic Dialogue Videos
Hazarika, D., Poria, S., Zadeh, A., Cambria, E., Morency, L.-P., & Zimmermann, R. (2018) · Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers) · doi:10.18653/v1/n18-1193 · Cited by 463
Zhang, S., Zhang, S., Huang, T., & Gao, W. (2018) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2017.2766843 · Cited by 441
Cross-Corpus Acoustic Emotion Recognition: Variances and Strategies
Schuller, B., Vlasenko, B., Eyben, F., Wollmer, M., Stuhlsatz, A., Wendemuth, A., & Rigoll, G. (2010) · IEEE Transactions on Affective Computing · doi:10.1109/t-affc.2010.8 · Cited by 388
Prosody-based automatic detection of annoyance and frustration in human-computer dialog
Ang, J., Dhillon, R., Krupski, A., Shriberg, E., & Stolcke, A. (2002) · 7th International Conference on Spoken Language Processing (ICSLP 2002) · doi:10.21437/icslp.2002-559 · Cited by 386
Databases, features and classifiers for speech emotion recognition: a review
Swain, M., Routray, A., & Kabisatpathy, P. (2018) · International Journal of Speech Technology · doi:10.1007/s10772-018-9491-z · Cited by 356
Challenges in real-life emotion annotation and machine learning based detection
Devillers, L., Vidrascu, L., & Lamel, L. (2005) · Neural Networks · doi:10.1016/j.neunet.2005.03.007 · Cited by 342
Ensemble methods for spoken emotion recognition in call-centres
Morrison, D., Wang, R., & De Silva, L. C. (2007) · Speech Communication · doi:10.1016/j.specom.2006.11.004 · Cited by 320
Primitives-based evaluation and estimation of emotions in speech
Grimm, M., Kroschel, K., Mower, E., & Narayanan, S. (2007) · Speech Communication · doi:10.1016/j.specom.2007.01.010 · Cited by 291
Affective understanding in film
Wang, H. L., & Cheong, L.-F. (2006) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2006.873781 · Cited by 267
Emotion recognition using brain activity
Horlings, R., Datcu, D., & Rothkrantz, L. J. M. (2008) · Proceedings of the 9th International Conference on Computer Systems and Technologies and Workshop for PhD Students in Computing - CompSysTech '08 · doi:10.1145/1500879.1500888 · Cited by 262
Passive and In-Situ assessment of mental and physical well-being using mobile sensors
Rabbi, M., Ali, S., Choudhury, T., & Berke, E. (2011) · Proceedings of the 13th international conference on Ubiquitous computing · doi:10.1145/2030112.2030164 · Cited by 243
“, pitch, speaking rate, loudness) and are supported by previous research [44, 45, 46].”
ResultTowards an intelligent framework for multimodal affective data analysis
Poria, S., Cambria, E., Hussain, A., & Huang, G.-B. (2015) · Neural Networks · doi:10.1016/j.neunet.2014.10.005 · Cited by 236
Class-level spectral features for emotion recognition
Bitouk, D., Verma, R., & Nenkova, A. (2010) · Speech Communication · doi:10.1016/j.specom.2010.02.010 · Cited by 219
“Following this tradition, most of the work on automatic recognition of emotion has made use of utterance-level statistics (mean, min, max, std) of prosodic features such as F 0, formant frequencies and intensity (Dellaert et al., 1996; McGilloway et al., 2000).”
InfluentialA Framework for Automatic Human Emotion Classification Using Emotion Profiles
Mower, E., Mataric, M. J., & Narayanan, S. (2011) · IEEE Transactions on Audio, Speech, and Language Processing · doi:10.1109/tasl.2010.2076804 · Cited by 215
MULTIMODAL EMOTION RECOGNITION
Sebe, N., Cohen, I., & Huang, T. S. (2005) · Handbook of Pattern Recognition and Computer Vision · doi:10.1142/9789812775320_0021 · Cited by 201
Multimodal human emotion/expression recognition
Chen, L. S., Huang, T. S., Miyasato, T., & Nakatsu, R. (2002) · Proceedings Third IEEE International Conference on Automatic Face and Gesture Recognition · doi:10.1109/afgr.1998.670976 · Cited by 183
Fear-type emotion recognition for future audio-based surveillance systems
Clavel, C., Vasilescu, I., Devillers, L., Richard, G., & Ehrette, T. (2008) · Speech Communication · doi:10.1016/j.specom.2008.03.012 · Cited by 180
“…contexts and extreme emotions are almost exclusively illustrated in acted databases (Mozziconacci , 1998; Kienast and Sendlmeier, 2000; van Bezooijen, 1984; Abelin and Allwood, 2000; Yacoub et al., 2003; McGilloway, 1997; Dellaert et al., 1996; Banse and Scherer, 1996; Banziger and Pirker, 2006).”
BackgroundA survey of affect recognition methods: audio, visual and spontaneous expressions
Zeng, Z., Pantic, M., Roisman, G. I., & Huang, T. S. (2007) · Proceedings of the 9th international conference on Multimodal interfaces · doi:10.1145/1322192.1322216 · Cited by 144
Survey on AI-Based Multimodal Methods for Emotion Detection
Marechal, C., Mikołajewski, D., Tyburek, K., Prokopowicz, P., Bougueroua, L., Ancourt, C., & Węgrzyn-Wolska, K. (2019) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-16272-6_11 · Cited by 137
Emotion recognition using Kinect motion capture data of human gaits
Li, S., Cui, L., Zhu, C., Li, B., Zhao, N., & Zhu, T. (2016) · PeerJ · doi:10.7717/peerj.2364 · Cited by 71
(2002) · doi:10.1109/icme.2002.1035887 · Cited by 69
Arruti, A., Cearreta, I., Álvarez, A., Lazkano, E., & Sierra, B. (2014) · PLoS ONE · doi:10.1371/journal.pone.0108975 · Cited by 21
“ML paradigms take a principal role in some works related to SER found in the literature [31].”
Background
Metadata
- Authors
- Dellaert, F., Polzin, T., & Waibel, A.
- Venue
- 4th International Conference on Spoken Language Processing (ICSLP 1996)
- Publisher
- ISCA
- Published
- 1996-10-03
- Pages
- 1970-1973
- OpenAlex
- W4300985914
- Cited by
- 158
- References
- 0
- FWCI
- 0.00
- Type
- conference paper
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.