The DOI matches its Crossref record; 31 of the 31 references it cites were found in a registry or an index.
On the Acoustics of Emotion in Audio: What Speech, Music, and Sound have in Common
Open accessAbstract
WITHOUT DOUBT, THERE IS EMOTIONAL INFORMATION IN ALMOST ANY KIND OF SOUND RECEIVED BY HUMANS EVERY DAY: be it the affective state of a person transmitted by means of speech; the emotion intended by a composer while writing a musical piece, or conveyed by a musician while performing it; or the affective state connected to an acoustic event occurring in the environment, in the soundtrack of a movie, or in a radio play. In the field of affective computing, there is currently some loosely connected research concerning either of these phenomena, but a holistic computational model of affect in sound is still lacking. In turn, for tomorrow's pervasive technical systems, including affective companions and robots, it is expected to be highly beneficial to understand the affective dimensions of "the sound that something makes," in order to evaluate the system's auditory environment and its own audio output. This article aims at a first step toward a holistic computational model: starting from standard acoustic feature extraction schemes in the domains of speech, music, and sound analysis, we interpret the worth of individual features across these three domains, considering four audio databases with observer annotations in the arousal and valence dimensions. In the results, we find that by selection of appropriate descriptors, cross-domain arousal, and valence regression is feasible achieving significant correlations with the observer annotations of up to 0.78 for arousal (training on sound and testing on enacted speech) and 0.60 for valence (training on enacted speech and testing on music). The high degree of cross-domain consistency in encoding the two main dimensions of affect may be attributable to the co-evolution of speech and music from multimodal affect bursts, including the integration of nature sounds for expressive effects.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
The WEKA data mining software: an update
Hall, M., Frank, E., Holmes, G., Pfahringer, B., Reutemann, P., & Witten, I. H. (2009) · ACM SIGKDD Explorations Newsletter · doi:10.1145/1656274.1656278 · Cited by 17,750 · cited 1×
“To foster reproducibility of our research, we use the open-source machine learning toolkit Weka (Hall et al., 2009).”
Method(2010) · Intellectum (Universidad de La Sabana) · doi:10.1111/j.1467- · Cited by 16,131
A tutorial on support vector regression
Smola, A. J., & Schölkopf, B. (2004) · Statistics and Computing · doi:10.1023/b:stco.0000035301.49549.88 · Cited by 13,387
Davis, S., & Mermelstein, P. (1980) · IEEE Transactions on Acoustics, Speech, and Signal Processing · doi:10.1109/tassp.1980.1163420 · Cited by 5,465
Fast Training of Support Vector Machines Using Sequential Minimal Optimization
Platt, J. C. (1998) · Advances in Kernel Methods · doi:10.7551/mitpress/1130.003.0016 · Cited by 5,437
Psychoacoustics: facts and models
(1991) · Choice Reviews Online · doi:10.5860/choice.28-5706 · Cited by 2,402
A database of German emotional speech
Burkhardt, F., Paeschke, A., Rolfes, M., Sendlmeier, W. F., & Weiss, B. (2005) · Interspeech 2005 · doi:10.21437/interspeech.2005-446 · Cited by 2,365
Communication of emotions in vocal expression and music performance: different channels, same code?
Juslin, P. N., & Laukka, P. (2003) · Psychological Bulletin · doi:10.1037/0033-2909.129.5.770 · Cited by 1,273
The world of emotions is not two-dimensional
Fontaine, J. R. J., Scherer, K. R., Roesch, E. B., & Ellsworth, P. C. (2007) · Psychological Science · doi:10.1111/j.1467-9280.2007.02024.x · Cited by 1,179
Self-citationThe INTERSPEECH 2009 emotion challenge
Schuller, B., Steidl, S., & Batliner, A. (2009) · Interspeech 2009 · doi:10.21437/interspeech.2009-103 · Cited by 938
Self-citationEmotional speech recognition: Resources, features, and methods
Ververidis, D., & Kotropoulos, C. (2006) · Speech Communication · doi:10.1016/j.specom.2006.04.003 · Cited by 925
Schuller, B., Batliner, A., Steidl, S., & Seppi, D. (2011) · Speech Communication · doi:10.1016/j.specom.2011.01.011 · Cited by 758
Self-citationSchuller, B., Steidl, S., Batliner, A., Vinciarelli, A., Scherer, K., Ringeval, F., Chetouani, M., Weninger, F., Eyben, F., Marchi, E., et al. (2013) · Interspeech 2013 · doi:10.21437/interspeech.2013-56 · Cited by 731
Self-citationMcKeown, G., Valstar, M., Cowie, R., Pantic, M., & Schroder, M. (2012) · IEEE Transactions on Affective Computing · doi:10.1109/t-affc.2011.20 · Cited by 665
Young, S. J., Evermann, G., Gales, M., Kershaw, D., Moore, G. L., Odell, J. J., Ollason, D., Povey, D., Valtchev, & Woodland, P. (2006) · Cambridge University Engineering Department Publications Database · Cited by 619
Introducing the Geneva Multimodal expression corpus for experimental research on emotion perception.
Bänziger, T., Mortillaro, M., & Scherer, K. R. (2012) · Emotion · doi:10.1037/a0025827 · Cited by 439
Self-citationScherer, K. R., Johnstone, & Klasmeyer, G. (2002) · Handbook of Affective Sciences · doi:10.1093/oso/9780195126013.003.0023 · Cited by 426
Self-citationThe Vera am Mittag German audio-visual emotional speech database
Grimm, M., Kroschel, K., & Narayanan, S. (2008) · 2008 IEEE International Conference on Multimedia and Expo · doi:10.1109/icme.2008.4607572 · Cited by 407
Machine Recognition of Music Emotion: A Review
Yang, Y.-H., & Chen, H. H. (2012) · ACM Transactions on Intelligent Systems and Technology · doi:10.1145/2168752.2168754 · Cited by 378
A Comparison of Acoustic Cues in Music and Speech for Three Dimensions of Affect
Ilie, G., & Thompson, W. F. (2006) · Music Perception · doi:10.1525/mp.2006.23.4.319 · Cited by 355
Decoding speech prosody: Do music lessons help?
Thompson, W. F., Schellenberg, E. G., & Husain, G. (2004) · Emotion · doi:10.1037/1528-3542.4.1.46 · Cited by 351
Primitives-based evaluation and estimation of emotions in speech
Grimm, M., Kroschel, K., Mower, E., & Narayanan, S. (2007) · Speech Communication · doi:10.1016/j.specom.2007.01.010 · Cited by 291
Evaluation of natural emotions using self assessment manikins
Grimm, M., & Kroschel, K. (2005) · IEEE Workshop on Automatic Speech Recognition and Understanding, 2005. · doi:10.1109/asru.2005.1566530 · Cited by 179
Support Vector Regression for Automatic Recognition of Spontaneous Emotions in Speech
Grimm, M., Kroschel, K., & Narayanan, S. S. (2007) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp.2007.367262 · Cited by 114
Advocating a Componential Appraisal Model to Guide Emotion Recognition
Mortillaro, M., Meuleman, B., & Scherer, K. R. (2012) · International Journal of Synthetic Emotions · doi:10.4018/jse.2012010102 · Cited by 53
Self-citationDetermination of Nonprototypical Valence and Arousal in Popular Music: Features and Performances
Schuller, B., Dorfner, J., & Rigoll, G. (2010) · EURASIP Journal on Audio, Speech, and Music Processing · doi:10.1155/2010/735854 · Cited by 50
Self-citationAutomatic recognition of emotion evoked by general sound events
Schuller, B. W., Hantke, S., Weninger, F. J., Han, W., Zhang, Z., & Narayanan, S. S. (2012) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp.2012.6287886 · Cited by 45
Self-citationEmotion expression in speech and music
Scherer, K. R. (1991) · doi:10.1007/978-1-349-12670-5_13 · Cited by 38
Self-citationEmotion in Action, Interaction, Music, and Speech
Scherer, K. R. (2013) · The MIT Press eBooks · doi:10.7551/mitpress/9548.003.0010 · Cited by 28
Self-citationA robust unsupervised arousal rating framework using prosody with cross-corpora evaluation
Bone, D., Lee, C.-C., & Narayanan, S. S. (2012) · Interspeech (USB) · doi:10.21437/interspeech.2012-123 · Cited by 15
Towards evaluation of example-based audio retrieval system using affective dimensions
Sundaram, S., & Schleicher, R. (2010) · IEEE International Conference on Multimedia and Expo (ICME) · doi:10.1109/icme.2010.5583001 · Cited by 10
Cited by
The Geneva Minimalistic Acoustic Parameter Set (GeMAPS) for Voice Research and Affective Computing
Eyben, F., Scherer, K. R., Schuller, B. W., Sundberg, J., Andre, E., Busso, C., Devillers, L. Y., Epps, J., Laukka, P., Narayanan, S. S., et al. (2016) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2015.2457417 · Cited by 1,818
Self-citationRecent developments in openSMILE, the munich open-source multimedia feature extractor
Eyben, F., Weninger, F., Gross, F., & Schuller, B. (2013) · Proceedings of the 21st ACM international conference on Multimedia · doi:10.1145/2502081.2502224 · Cited by 1,241
Self-citationSpeech emotion recognition: two decades in a nutshell, benchmarks, and ongoing trends
Schuller, B. W. (2018) · Communications of the ACM · doi:10.1145/3129340 · Cited by 541
Self-citationPrediction of asynchronous dimensional emotion ratings from audiovisual and physiological data
Ringeval, F., Eyben, F., Kroupi, E., Yuce, A., Thiran, J.-P., Ebrahimi, T., Lalanne, D., & Schuller, B. (2015) · Pattern Recognition Letters · doi:10.1016/j.patrec.2014.11.007 · Cited by 202
Self-citationRickard, N., Arjmand, H.-A., Bakker, D., & Seabrook, E. (2016) · JMIR Mental Health · doi:10.2196/mental.6202 · Cited by 118
“In addition, a weekly prompt was delivered that requested a short voice recording to serve as an implicit measure of emotional state [51,53].”
Metadata
- Authors
- Weninger, F., Eyben, F., Schuller, B. W., Mortillaro, M., & Scherer, K. R.
- Venue
- Frontiers in Psychology
- Publisher
- Frontiers Media SA
- Published
- 2013
- Volume
- 4
- Pages
- 292-292
- ISSN
- 1664-1078 (ISSN-L)
- OpenAlex
- W2082927806
- Cited by
- 278
- References
- 31
- FWCI
- 25.37
- Type
- journal article
- Language
- en
- Open access
- gold
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.