The DOI matches its Crossref record; 54 of the 54 references it cites were found in a registry or an index.
Building Naturalistic Emotionally Balanced Speech Corpus by Retrieving Emotional Speech from Existing Podcast Recordings
Abstract
The lack of a large, natural emotional database is one of the key barriers to translate results on speech emotion recognition in controlled conditions into real-life applications. Collecting emotional databases is expensive and time demanding, which limits the size of existing corpora. Current approaches used to collect spontaneous databases tend to provide unbalanced emotional content, which is dictated by the given recording protocol (e.g., positive for colloquial conversations, negative for discussion or debates). The size and speaker diversity are also limited. This paper proposes a novel approach to effectively build a large, naturalistic emotional database with balanced emotional content, reduced cost and reduced manual labor. It relies on existing spontaneous recordings obtained from audio-sharing websites. The proposed approach combines machine learning algorithms to retrieve recordings conveying balanced emotional content with a cost effective annotation process using crowdsourcing, which make it possible to build a large scale speech emotional database. This approach provides natural emotional renditions from multiple speakers, with different channel conditions and conveying balanced emotional content that are difficult to obtain with alternative data collection protocols.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Wu, Y. (1999) · Technometrics · doi:10.1080/00401706.1999.10485951 · Cited by 27,221
The WEKA data mining software: an update
Hall, M., Frank, E., Holmes, G., Pfahringer, B., Reutemann, P., & Witten, I. H. (2009) · ACM SIGKDD Explorations Newsletter · doi:10.1145/1656274.1656278 · Cited by 17,750
IEMOCAP: Interactive emotional dyadic motion capture database
Busso, C., Bulut, M., Lee, C.-C., Kazemzadeh, A., Mower, E., Kim, S., Chang, J. N., Lee, S., & Narayanan, S. S. (2008) · Language Resources and Evaluation · doi:10.1007/s10579-008-9076-6 · Cited by 3,869
Self-citationOpensmile: the munich versatile and fast open-source audio feature extractor
Eyben, F., Wöllmer, M., & Schuller, B. (2010) · Proceedings of the 18th ACM international conference on Multimedia · doi:10.1145/1873951.1874246 · Cited by 2,642
A database of German emotional speech
Burkhardt, F., Paeschke, A., Rolfes, M., Sendlmeier, W. F., & Weiss, B. (2005) · Interspeech 2005 · doi:10.21437/interspeech.2005-446 · Cited by 2,365
Training linear SVMs in linear time
Joachims, T. (2006) · Proceedings of the 12th ACM SIGKDD international conference on Knowledge discovery and data mining · doi:10.1145/1150402.1150429 · Cited by 1,920
The Geneva Minimalistic Acoustic Parameter Set (GeMAPS) for Voice Research and Affective Computing
Eyben, F., Scherer, K. R., Schuller, B. W., Sundberg, J., Andre, E., Busso, C., Devillers, L. Y., Epps, J., Laukka, P., Narayanan, S. S., et al. (2016) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2015.2457417 · Cited by 1,818
Self-citationToward detecting emotions in spoken dialogs
Lee, C. M., & Narayanan, S. S. (2005) · IEEE Transactions on Speech and Audio Processing · doi:10.1109/tsa.2004.838534 · Cited by 988
CREMA-D: Crowd-Sourced Emotional Multimodal Actors Dataset
Cao, H., Cooper, D. G., Keutmann, M. K., Gur, R. C., Nenkova, A., & Verma, R. (2014) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2014.2336244 · Cited by 914
(2024) · Multivariate Biomarker Discovery · doi:10.1017/9781009006767.012 · Cited by 809
Introducing the RECOLA multimodal corpus of remote collaborative and affective interactions
Ringeval, F., Sonderegger, A., Sauer, J., & Lalanne, D. (2013) · 2013 10th IEEE International Conference and Workshops on Automatic Face and Gesture Recognition (FG) · doi:10.1109/fg.2013.6553805 · Cited by 771
Zhang, F., & O'Donnell, L. J. (2020) · Machine Learning · doi:10.1016/b978-0-12-815739-8.00007-9 · Cited by 737
Schuller, B., Steidl, S., Batliner, A., Vinciarelli, A., Scherer, K., Ringeval, F., Chetouani, M., Weninger, F., Eyben, F., Marchi, E., et al. (2013) · Interspeech 2013 · doi:10.21437/interspeech.2013-56 · Cited by 731
McKeown, G., Valstar, M., Cowie, R., Pantic, M., & Schroder, M. (2012) · IEEE Transactions on Affective Computing · doi:10.1109/t-affc.2011.20 · Cited by 665
Towards multimodal sentiment analysis: harvesting opinions from the web
Morency, L.-P., Mihalcea, R., & Doshi, P. (2011) · Proceedings of the 13th international conference on multimodal interfaces · doi:10.1145/2070481.2070509 · Cited by 470
The Fisher Corpus: a Resource for the Next Generations of Speech-to-Text
Cieri, C., Miller, D. A. B., & Walker, K. P. (2004) · Language Resources and Evaluation Conference (LREC) · doi:10.63317/5q862povzthr · Cited by 464
Emotional speech: Towards a new generation of databases
Douglas-Cowie, E., Campbell, N., Cowie, R., & Roach, P. (2003) · Speech Communication · doi:10.1016/s0167-6393(02)00070-5 · Cited by 461
Support vector learning for ordinal regression
Herbrich, R. (1999) · 9th International Conference on Artificial Neural Networks: ICANN '99 · doi:10.1049/cp:19991091 · Cited by 448
The Vera am Mittag German audio-visual emotional speech database
Grimm, M., Kroschel, K., & Narayanan, S. (2008) · 2008 IEEE International Conference on Multimedia and Expo · doi:10.1109/icme.2008.4607572 · Cited by 407
MSP-IMPROV: An Acted Corpus of Dyadic Interactions to Study Emotion Perception
Busso, C., Parthasarathy, S., Burmania, A., AbdelWahab, M., Sadoughi, N., & Provost, E. M. (2017) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2016.2515617 · Cited by 391
Self-citationPreference learning with Gaussian processes
Chu, W., & Ghahramani, Z. (2005) · Proceedings of the 22nd international conference on Machine learning - ICML '05 · doi:10.1145/1102351.1102369 · Cited by 361
Challenges in real-life emotion annotation and machine learning based detection
Devillers, L., Vidrascu, L., & Lamel, L. (2005) · Neural Networks · doi:10.1016/j.neunet.2005.03.007 · Cited by 342
An empirical study of machine learning techniques for affect recognition in human–robot interaction
Rani, P., Liu, C., Sarkar, N., & Vanman, E. (2006) · Pattern Analysis and Applications · doi:10.1007/s10044-006-0025-y · Cited by 313
Primitives-based evaluation and estimation of emotions in speech
Grimm, M., Kroschel, K., Mower, E., & Narayanan, S. (2007) · Speech Communication · doi:10.1016/j.specom.2007.01.010 · Cited by 291
Speech emotion recognition based on HMM and SVM
Lin, Y.-L., & Wei, G. (2005) · 2005 International Conference on Machine Learning and Cybernetics · doi:10.1109/icmlc.2005.1527805 · Cited by 260
Schuller, B., Müller, R., Eyben, F., Gast, J., Hörnler, B., Wöllmer, M., Rigoll, G., Höthker, A., & Konosu, H. (2009) · Image and Vision Computing · doi:10.1016/j.imavis.2009.02.013 · Cited by 194
Robust signal-to-noise ratio estimation based on waveform amplitude distribution analysis
Kim, C., & Stern, R. M. (2008) · Interspeech 2008 · doi:10.21437/interspeech.2008-644 · Cited by 165
Real-life emotions detection with lexical and paralinguistic cues on human-human call center dialogs
Devillers, L., & Vidrascu, L. (2006) · Interspeech 2006 · doi:10.21437/interspeech.2006-275 · Cited by 160
A new emotion database: considerations, sources and scope
Douglas‐Cowie, E., Cowie, R., & Schroeder, N. V. (2000) · Cited by 156
Jointly Predicting Arousal, Valence and Dominance with Multi-Task Learning
Parthasarathy, S., & Busso, C. (2017) · Interspeech (USB) · doi:10.21437/interspeech.2017-1494 · Cited by 142
Self-citationBeyond emotion archetypes: Databases for emotion modelling using neural networks
Cowie, R., Douglas‐Cowie, E., & Cox, C. (2005) · Neural Networks · doi:10.1016/j.neunet.2005.03.002 · Cited by 136
Using Actor Portrayals to Systematically Study Multimodal Emotion Expression: The GEMEP Corpus
Bänziger, T., & Scherer, K. R. (2007) · Lecture notes in computer science · doi:10.1007/978-3-540-74889-2_42 · Cited by 135
Speaker-sensitive emotion recognition via ranking: Studies on acted and spontaneous speech
Cao, H., Verma, R., & Nenkova, A. (2014) · Computer Speech & Language · doi:10.1016/j.csl.2014.01.003 · Cited by 133
Interpreting ambiguous emotional expressions
Mower, E., Metallinou, A., Lee, C.-C., Kazemzadeh, A., Busso, C., Lee, S., & Narayanan, S. S. (2009) · International Conference on Affective Computing and Intelligent Interaction and workshops · doi:10.1109/acii.2009.5349500 · Cited by 132
Self-citationDon’t Classify Ratings of Affect; Rank Them!
Martínez, H. P., Yannakakis, G. N., & Hallam, J. C. T. (2014) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2014.2352268 · Cited by 116
Increasing the Reliability of Crowdsourcing Evaluations Using Online Quality Assessment
Burmania, A., Parthasarathy, S., & Busso, C. (2015) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2015.2493525 · Cited by 111
Self-citationHuang, T. S., & Chen, L. S.-H. (2000) · Cited by 109
Doyle, J. (2004) · Computational Intelligence · doi:10.1111/j.0824-7935.2004.00233.x · Cited by 107
Toward Effective Automatic Recognition Systems of Emotion in Speech
Busso, C., Bulut, M., & Narayanan, S. S. (2013) · Oxford University Press eBooks · doi:10.1093/acprof:oso/9780195387643.003.0008 · Cited by 90
Self-citationA State of the Art Review on Emotional Speech Databases
Ververidis, D., & Kotropoulos, K. (2003) · Cited by 90
Multimodal databases of everyday emotion: facing up to complexity
Douglas‐Cowie, E., Devillers, L. Y., Martin, J., et al. (2005) · Interspeech (USB) · doi:10.21437/interspeech.2005-381 · Cited by 82
Douglas‐Cowie, E., Cowie, R., Cox, C., Amier, N., & Heylen, D. (2008) · University of Twente Research Information · Cited by 81
Mariooryad, S., Lotfian, R., & Busso, C. (2014) · Interspeech (USB) · doi:10.21437/interspeech.2014-60 · Cited by 58
Self-citationLearning Preference Relations for Information Retrieval
Herbrich, R., Graepel, T., Bollmann-Sdorra, P., & Obermayer, K. (1998) · Cited by 51
Unveiling the acoustic properties that describe the valence dimension
Busso, C., & Rahman, T. (2012) · Interspeech (USB) · doi:10.21437/interspeech.2012-124 · Cited by 38
Self-citationPractical considerations on the use of preference learning for ranking emotional speech
Lotfian, R., & Busso, C. (2016) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp.2016.7472670 · Cited by 35
Self-citationA personalized emotion recognition system using an unsupervised feature adaptation scheme
Rahman, T., & Busso, C. (2012) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp.2012.6289072 · Cited by 34
Self-citationA Framework for Eliciting Emotional Speech: Capitalizing on the Actor’s Process
Hirschberg, J., & Enos, F. (2006) · HAL AMU · doi:10.7916/d88s4zcg · Cited by 31
Desperately Seeking Emotions Or: Actors, Wizards, And Human Beings
Batliner, A., Fischer, K., Huber, R., Spilker, J., & Nöth, E. (2000) · OPUS (Augsburg University) · Cited by 31
Using Agreement on Direction of Change to Build Rank-Based Emotion Classifiers
Parthasarathy, S., Cowie, R., & Busso, C. (2016) · IEEE/ACM Transactions on Audio Speech and Language Processing · doi:10.1109/taslp.2016.2593944 · Cited by 29
Self-citationLotfian, R., & Busso, C. (2016) · Interspeech (USB) · doi:10.21437/interspeech.2016-1052 · Cited by 27
Self-citationA study of speaker verification performance with expressive speech
Parthasarathy, S., Zhang, C., Hansen, J. H. L., & Busso, C. (2017) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp.2017.7953216 · Cited by 26
Self-citationRanking emotional attributes with deep neural networks
Parthasarathy, S., Lotfian, R., & Busso, C. (2017) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp.2017.7953107 · Cited by 25
Self-citationMariooryad, S., & Busso, C. (2015) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2015.2508454 · Cited by 21
Self-citation
Cited by
A Survey of Deep Active Learning
Ren, P., Xiao, Y., Chang, X., Huang, P.-Y., Li, Z., Gupta, B. B., Chen, X., & Wang, X. (2022) · ACM Computing Surveys · doi:10.1145/3472291 · Cited by 1,169
Dawn of the Transformer Era in Speech Emotion Recognition: Closing the Valence Gap
Wagner, J., Triantafyllopoulos, A., Wierstorf, H., Schmitt, M., Burkhardt, F., Eyben, F., & Schuller, B. W. (2023) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2023.3263585 · Cited by 395
Emotion Recognition from Speech Using wav2vec 2.0 Embeddings
Pepino, L., Riera, P., & Ferrer, L. (2021) · Interspeech 2021 · doi:10.21437/interspeech.2021-703 · Cited by 389
Domain Adversarial for Acoustic Emotion Recognition
Abdelwahab, M., & Busso, C. (2018) · IEEE/ACM Transactions on Audio, Speech, and Language Processing · doi:10.1109/taslp.2018.2867099 · Cited by 178
Self-citationFeng, T., & Narayanan, S. (2023) · 2023 11th International Conference on Affective Computing and Intelligent Interaction (ACII) · doi:10.1109/acii59096.2023.10388152 · Cited by 34
Metadata
- Authors
- Lotfian, R., & Busso, C.
- Venue
- IEEE Transactions on Affective Computing
- Published
- 2019-10-01
- Volume
- 10
- Issue
- 4
- Pages
- 471-483
- ISSN
- 1949-3045 (ISSN-L)
- 2371-9850
- OpenAlex
- W2742542661
- Cited by
- 329
- References
- 54
- FWCI
- 14.61
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.