The DOI matches its Crossref record; 31 of the 31 references it cites were found in a registry or an index.
On the Acoustics of Emotion in Audio: What Speech, Music, and Sound have in Common
Open accessAbstract
WITHOUT DOUBT, THERE IS EMOTIONAL INFORMATION IN ALMOST ANY KIND OF SOUND RECEIVED BY HUMANS EVERY DAY: be it the affective state of a person transmitted by means of speech; the emotion intended by a composer while writing a musical piece, or conveyed by a musician while performing it; or the affective state connected to an acoustic event occurring in the environment, in the soundtrack of a movie, or in a radio play. In the field of affective computing, there is currently some loosely connected research concerning either of these phenomena, but a holistic computational model of affect in sound is still lacking. In turn, for tomorrow's pervasive technical systems, including affective companions and robots, it is expected to be highly beneficial to understand the affective dimensions of "the sound that something makes," in order to evaluate the system's auditory environment and its own audio output. This article aims at a first step toward a holistic computational model: starting from standard acoustic feature extraction schemes in the domains of speech, music, and sound analysis, we interpret the worth of individual features across these three domains, considering four audio databases with observer annotations in the arousal and valence dimensions. In the results, we find that by selection of appropriate descriptors, cross-domain arousal, and valence regression is feasible achieving significant correlations with the observer annotations of up to 0.78 for arousal (training on sound and testing on enacted speech) and 0.60 for valence (training on enacted speech and testing on music). The high degree of cross-domain consistency in encoding the two main dimensions of affect may be attributable to the co-evolution of speech and music from multimodal affect bursts, including the integration of nature sounds for expressive effects.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
The WEKA data mining software: an update
Hall, M., Frank, E., Holmes, G., Pfahringer, B., Reutemann, P., & Witten, I. H. (2009) · ACM SIGKDD Explorations Newsletter · doi:10.1145/1656274.1656278 · Cited by 17,750 · cited 1×
“To foster reproducibility of our research, we use the open-source machine learning toolkit Weka (Hall et al., 2009).”
Method(2010) · Intellectum (Universidad de La Sabana) · doi:10.1111/j.1467- · Cited by 16,131
A tutorial on support vector regression
Smola, A. J., & Schölkopf, B. (2004) · Statistics and Computing · doi:10.1023/b:stco.0000035301.49549.88 · Cited by 13,387
Davis, S., & Mermelstein, P. (1980) · IEEE Transactions on Acoustics, Speech, and Signal Processing · doi:10.1109/tassp.1980.1163420 · Cited by 5,465
Fast Training of Support Vector Machines Using Sequential Minimal Optimization
Platt, J. C. (1998) · Advances in Kernel Methods · doi:10.7551/mitpress/1130.003.0016 · Cited by 5,437
Psychoacoustics: facts and models
(1991) · Choice Reviews Online · doi:10.5860/choice.28-5706 · Cited by 2,402
A database of German emotional speech
Burkhardt, F., Paeschke, A., Rolfes, M., Sendlmeier, W. F., & Weiss, B. (2005) · Interspeech 2005 · doi:10.21437/interspeech.2005-446 · Cited by 2,365
Communication of emotions in vocal expression and music performance: different channels, same code?
Juslin, P. N., & Laukka, P. (2003) · Psychological Bulletin · doi:10.1037/0033-2909.129.5.770 · Cited by 1,273
The world of emotions is not two-dimensional
Fontaine, J. R. J., Scherer, K. R., Roesch, E. B., & Ellsworth, P. C. (2007) · Psychological Science · doi:10.1111/j.1467-9280.2007.02024.x · Cited by 1,179
Self-citationThe INTERSPEECH 2009 emotion challenge
Schuller, B., Steidl, S., & Batliner, A. (2009) · Interspeech 2009 · doi:10.21437/interspeech.2009-103 · Cited by 938
Self-citation
Cited by
The Geneva Minimalistic Acoustic Parameter Set (GeMAPS) for Voice Research and Affective Computing
Eyben, F., Scherer, K. R., Schuller, B. W., Sundberg, J., Andre, E., Busso, C., Devillers, L. Y., Epps, J., Laukka, P., Narayanan, S. S., et al. (2016) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2015.2457417 · Cited by 1,818
“…is that affective processes differentially change autonomic arousal and the tension of the striate musculature and thereby affect voice and speech production on the phonatory and articulatory level and that these changes can be estimated by different parameters of the acoustic waveform…”
Self-citationRecent developments in openSMILE, the munich open-source multimedia feature extractor
Eyben, F., Weninger, F., Gross, F., & Schuller, B. (2013) · Proceedings of the 21st ACM international conference on Multimedia · doi:10.1145/2502081.2502224 · Cited by 1,241
Self-citationSpeech emotion recognition: two decades in a nutshell, benchmarks, and ongoing trends
Schuller, B. W. (2018) · Communications of the ACM · doi:10.1145/3129340 · Cited by 541
Self-citationPrediction of asynchronous dimensional emotion ratings from audiovisual and physiological data
Ringeval, F., Eyben, F., Kroupi, E., Yuce, A., Thiran, J.-P., Ebrahimi, T., Lalanne, D., & Schuller, B. (2015) · Pattern Recognition Letters · doi:10.1016/j.patrec.2014.11.007 · Cited by 202
Self-citationRickard, N., Arjmand, H.-A., Bakker, D., & Seabrook, E. (2016) · JMIR Mental Health · doi:10.2196/mental.6202 · Cited by 118
“In addition, a weekly prompt was delivered that requested a short voice recording to serve as an implicit measure of emotional state [51,53].”
Schuller, B. W., Batliner, A., Bergler, C., Mascolo, C., Han, J., Lefter, I., Kaya, H., Amiriparian, S., Baird, A., Stappen, L., et al. (2021) · Interspeech 2021 · doi:10.21437/interspeech.2021-19 · Cited by 89
Self-citationHecker, P., Steckhan, N., Eyben, F., Schuller, B. W., & Arnrich, B. (2022) · Frontiers in Digital Health · doi:10.3389/fdgth.2022.842301 · Cited by 78
Self-citationTransfer learning emotion manifestation across music and speech
Coutinho, E., Deng, J., & Schuller, B. (2014) · 2014 International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn.2014.6889814 · Cited by 36
Self-citationLiu, Z., Guo, Z., Ling, Z., & Li, Y. (2021) · ICASSP 2021 - 2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) · doi:10.1109/icassp39728.2021.9413566 · Cited by 34
Lee, J. B., & Lee, H. G. (2024) · Biomedical Signal Processing and Control · doi:10.1016/j.bspc.2024.106014 · Cited by 15
Metadata
- Authors
- Weninger, F., Eyben, F., Schuller, B. W., Mortillaro, M., & Scherer, K. R.
- Venue
- Frontiers in Psychology
- Publisher
- Frontiers Media SA
- Published
- 2013
- Volume
- 4
- Pages
- 292-292
- ISSN
- 1664-1078 (ISSN-L)
- OpenAlex
- W2082927806
- Cited by
- 278
- References
- 31
- FWCI
- 25.37
- Type
- journal article
- Language
- en
- Open access
- gold
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.