The DOI matches its Crossref record; 51 of the 51 references it cites were found in a registry or an index.
CREMA-D: Crowd-Sourced Emotional Multimodal Actors Dataset
Open accessAbstract
People convey their emotional state in their face and voice. We present an audio-visual data set uniquely suited for the study of multi-modal emotion expression and perception. The data set consists of facial and vocal emotional expressions in sentences spoken in a range of basic emotional states (happy, sad, anger, fear, disgust, and neutral). 7,442 clips of 91 actors with diverse ethnic backgrounds were rated by multiple raters in three modalities: audio, visual, and audio-visual. Categorical emotion labels and real-value intensity values for the perceived emotion were collected using crowd-sourcing from 2,443 raters. The human recognition of intended emotion for the audio-only, visual-only, and audio-visual data are 40.9%, 58.2% and 63.6% respectively. Recognition rates are highest for neutral, followed by happy, anger, disgust, fear, and sad. Average intensity levels of emotion are rated highest for visual-only perception. The accurate recognition of disgust and fear requires simultaneous audio-visual cues, while anger and happiness can be well recognized based on evidence from a single modality. The large dataset we introduce can be used to probe other questions concerning the audio-visual perception of emotion.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
An argument for basic emotions
Ekman, P. (1992) · Cognition and Emotion · doi:10.1080/02699939208411068 · Cited by 9,646
IEMOCAP: Interactive emotional dyadic motion capture database
Busso, C., Bulut, M., Lee, C.-C., Kazemzadeh, A., Mower, E., Kim, S., Chang, J. N., Lee, S., & Narayanan, S. S. (2008) · Language Resources and Evaluation · doi:10.1007/s10579-008-9076-6 · Cited by 3,869
Psychological Assessment Resources
Abidin, R. R. (1995) · Medical Entomology and Zoology · Cited by 3,140
Comprehensive database for facial expression analysis
Kanade, T., Cohn, J. F., & Tian, Y. (2002) · Proceedings Fourth IEEE International Conference on Automatic Face and Gesture Recognition (Cat. No. PR00580) · doi:10.1109/afgr.2000.840611 · Cited by 2,591
A database of German emotional speech
Burkhardt, F., Paeschke, A., Rolfes, M., Sendlmeier, W. F., & Weiss, B. (2005) · Interspeech 2005 · doi:10.21437/interspeech.2005-446 · Cited by 2,365
A Multimodal Database for Affect Recognition and Implicit Tagging
Soleymani, M., Lichtenauer, J., Pun, T., & Pantic, M. (2012) · IEEE Transactions on Affective Computing · doi:10.1109/t-affc.2011.25 · Cited by 1,664
Acoustic profiles in vocal emotion expression
Banse, R., & Scherer, K. R. (1996) · Journal of Personality and Social Psychology · doi:10.1037/0022-3514.70.3.614 · Cited by 1,546
(2017) · The SAGE Encyclopedia of Abnormal and Clinical Psychology · doi:10.4135/9781483365817.n1487 · Cited by 1,252
Murray, I. R., & Arnott, J. L. (1993) · The Journal of the Acoustical Society of America · doi:10.1121/1.405558 · Cited by 1,098
Web-Based Database for Facial Expression Analysis
Pantic, M., Valstar, M., Rademaker, R., & Maat, L. (2005) · 2005 IEEE International Conference on Multimedia and Expo · doi:10.1109/icme.2005.1521424 · Cited by 1,039
Reliability in content analysis: some common misconceptions and recommendations.
Krippendorff, K. (2004) · Human Communication Research · doi:10.1093/hcr/30.3.411 · Cited by 1,027
Facial Expressions of Emotion: New Findings, New Questions
Ekman, P. (1992) · Psychological Science · doi:10.1111/j.1467-9280.1992.tb00253.x · Cited by 917
Analysis of emotion recognition using facial expressions, speech and multimodal information
Busso, C., Deng, Z., Yildirim, S., Bulut, M., Lee, C. M., Kazemzadeh, A., Lee, S., Neumann, U., & Narayanan, S. (2004) · Proceedings of the 6th international conference on Multimodal interfaces · doi:10.1145/1027933.1027968 · Cited by 840 · cited 1×
“[52] found, the recognition rate at each intensity level is worst for audio-only and best for audio-visual.”
BackgroundCross-cultural recognition of basic emotions through nonverbal emotional vocalizations
Sauter, D. A., Eisner, F., Ekman, P., & Scott, S. K. (2010) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.0908239106 · Cited by 789
Introducing the RECOLA multimodal corpus of remote collaborative and affective interactions
Ringeval, F., Sonderegger, A., Sauer, J., & Lalanne, D. (2013) · 2013 10th IEEE International Conference and Workshops on Automatic Face and Gesture Recognition (FG) · doi:10.1109/fg.2013.6553805 · Cited by 771
Emotion Inferences from Vocal Expression Correlate Across Languages and Cultures
Scherer, K. R., Banse, R., & Wallbott, H. G. (2001) · Journal of Cross-Cultural Psychology · doi:10.1177/0022022101032001009 · Cited by 751
The perception of emotions by ear and by eye
de Gelder, B., & Vroomen, J. (2000) · Cognition & Emotion · doi:10.1080/026999300378824 · Cited by 629
Wide Range Achievement Test 3 (wrat3)
Snelbaker, A. J., Wilkinson, G. S., Robertson, G. J., & Glutting, J. J. (2001) · Understanding Psychological Assessment · doi:10.1007/978-1-4615-1185-4_13 · Cited by 546
Estimating the Reliability, Systematic Error and Random Error of Interval Data
Krippendorff, K. (1970) · Educational and Psychological Measurement · doi:10.1177/001316447003000105 · Cited by 491
Emotional speech: Towards a new generation of databases
Douglas-Cowie, E., Campbell, N., Cowie, R., & Roach, P. (2003) · Speech Communication · doi:10.1016/s0167-6393(02)00070-5 · Cited by 461
Scherer, K. R., Johnstone, & Klasmeyer, G. (2002) · Handbook of Affective Sciences · doi:10.1093/oso/9780195126013.003.0023 · Cited by 426
The Vera am Mittag German audio-visual emotional speech database
Grimm, M., Kroschel, K., & Narayanan, S. (2008) · 2008 IEEE International Conference on Multimedia and Expo · doi:10.1109/icme.2008.4607572 · Cited by 407
Audio-visual integration of emotion expression
Collignon, O., Girard, S., Gosselin, F., Roy, S., Saint-Amour, D., Lassonde, M., & Lepore, F. (2008) · Brain Research · doi:10.1016/j.brainres.2008.04.023 · Cited by 353
Douglas-Cowie, E., Cowie, R., Sneddon, I., Cox, C., Lowry, O., McRorie, M., Martin, J.-C., Devillers, L., Abrilian, S., Batliner, A., et al. (2007) · Lecture Notes in Computer Science · doi:10.1007/978-3-540-74889-2_43 · Cited by 321
Facial emotion recognition using multi-modal information
De Silva, L. C., Miyasato, T., & Nakatsu, R. (2002) · Proceedings of ICICS, 1997 International Conference on Information, Communications and Signal Processing. Theme: Trends in Information Systems Engineering and Wireless Multimedia Communications (Cat. No.97TH8237) · doi:10.1109/icics.1997.647126 · Cited by 319
The argument and evidence about universals in facial expressions of emotion.
Ekman, P. (1989) · Cited by 312
Cross-Cultural Emotion Recognition among Canadian Ethnic Groups
Beaupré, M. G., & Hess, U. (2005) · Journal of Cross-Cultural Psychology · doi:10.1177/0022022104273656 · Cited by 292
Emotion recognition in spontaneous speech using GMMs
Neiberg, D., Elenius, K., & Laskowski, K. (2006) · Interspeech 2006 · doi:10.21437/interspeech.2006-277 · Cited by 247
A video database of moving faces and people
O'Toole, A. J., Ha, J., Snow, S. L., Hurst, D. R., Pappas, M. R., Ayyad, J. H., & Abdi, H. (2005) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2005.90 · Cited by 212
Acoustic feature selection for automatic emotion recognition from speech
Rong, J., Li, G., & Chen, Y.-P. P. (2009) · Information Processing & Management · doi:10.1016/j.ipm.2008.09.003 · Cited by 210
GMM Supervector Based SVM with Spectral Features for Speech Emotion Recognition
Hu, H., Xu, M.-X., & Wu, W. (2007) · 2007 IEEE International Conference on Acoustics, Speech and Signal Processing - ICASSP '07 · doi:10.1109/icassp.2007.366937 · Cited by 198
(2008) · Cited by 193
Introducing the Geneva Multimodal Emotion Portrayal (GEMEP) corpus
Bänziger, & Scherer, T. A. (2010) · Cited by 170
Increasing cheat robustness of crowdsourcing tasks
Eickhoff, C., & de Vries, A. P. (2013) · Information Retrieval · doi:10.1007/s10791-011-9181-9 · Cited by 167
The ISL meeting corpus: the impact of meeting type on speech style
Burger, S., MacLaren, V., & Yu, H. X. (2002) · International Conference on Spoken Language Processing (ICSLP) · doi:10.21437/icslp.2002-140 · Cited by 158
The Belfast Induced Natural Emotion Database
Sneddon, I., McRorie, M., McKeown, G. J., & Hanratty, J. (2011) · IEEE Transactions on Affective Computing · doi:10.1109/t-affc.2011.26 · Cited by 154
Multisensory emotions: perception, combination and underlying neural processes
Klasen, M., Chen, Y.-H., & Mathiak, K. (2012) · Reviews in the Neurosciences · doi:10.1515/revneuro-2012-0040 · Cited by 154
Fundamental frequency of phonation and perceived emotional stress
Protopapas, A., & LIEBERMAN, P. H. (1997) · The Journal of the Acoustical Society of America · doi:10.1121/1.418247 · Cited by 136
Beyond emotion archetypes: Databases for emotion modelling using neural networks
Cowie, R., Douglas‐Cowie, E., & Cox, C. (2005) · Neural Networks · doi:10.1016/j.neunet.2005.03.002 · Cited by 136
Supramodal Representation of Emotions
Klasen, M., Kenworthy, C. A., Mathiak, K. A., Kircher, T. T. J., & Mathiak, K. (2011) · Journal of Neuroscience · doi:10.1523/jneurosci.2833-11.2011 · Cited by 122
Incongruence effects in crossmodal emotional integration
Müller, V. I., Habel, U., Derntl, B., Schneider, F. W., Zilles, K., Turetsky, B. I., & Eickhoff, S. B. (2010) · NeuroImage · doi:10.1016/j.neuroimage.2010.10.047 · Cited by 119
Huang, T. S., & Chen, L. S.-H. (2000) · Cited by 109
Analysis of the glottal excitation of emotionally styled and stressed speech
Cummings, K. E., & Clements, M. A. (1995) · The Journal of the Acoustical Society of America · doi:10.1121/1.413664 · Cited by 107
Batliner, A. M., Steidl, S., Hacker, C., & Nöth, E. (2007) · User Modeling and User-Adapted Interaction · doi:10.1007/s11257-007-9039-4 · Cited by 98
Mechanical cheat: Spamming schemes and adversarial techniques on crowdsourcing platforms
Difallah, D. E., Demartini, G., & Cudré-Mauroux, P. (2012) · Cited by 90
Crossmodal interactions in audiovisual emotion processing
Müller, V. I., Cieslik, E. C., Turetsky, B. I., & Eickhoff, S. B. (2012) · NeuroImage · doi:10.1016/j.neuroimage.2011.12.007 · Cited by 65
Mower, E., Matarić, M. J., & Narayanan, S. S. (2009) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2009.2021722 · Cited by 60
Cost-Optimal Validation Mechanisms and Cheat-Detection for Crowdsourcing Platforms
Hirth, M., Hoßfeld, T., & Tran‐Gia, P. (2011) · International Conference on Innovative Mobile and Internet Services in Ubiquitous Computing (IMIS) · doi:10.1109/imis.2011.91 · Cited by 57
Validating a Multilingual and Multimodal Affective Database
López, J. M., Cearreta, I., Fajardo, I., & Garay-Vitoria, N. (2007) · Lecture notes in computer science · doi:10.1007/978-3-540-73289-1_49 · Cited by 8
CHAD: A Chinese Affective Database
You, M., Chen, C., & Bu, J. (2005) · Lecture notes in computer science · doi:10.1007/11573548_70 · Cited by 7
Using emotional noise to uncloud audio-visual emotion perceptual evaluation
Provost, E. M., Zhu, I., & Narayanan, S. S. (2013) · IEEE International Conference on Multimedia and Expo (ICME) · doi:10.1109/icme.2013.6607537 · Cited by 7
Cited by
Training deep networks for facial expression recognition with crowd-sourced label distribution
Barsoum, E., Zhang, C., Ferrer, C. C., & Zhang, Z. (2016) · Proceedings of the 18th ACM International Conference on Multimodal Interaction · doi:10.1145/2993148.2993165 · Cited by 926
MSP-IMPROV: An Acted Corpus of Dyadic Interactions to Study Emotion Perception
Busso, C., Parthasarathy, S., Burmania, A., AbdelWahab, M., Sadoughi, N., & Provost, E. M. (2017) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2016.2515617 · Cited by 391
Realistic Speech-Driven Facial Animation with GANs
Vougioukas, K., Petridis, S., & Pantic, M. (2020) · International Journal of Computer Vision · doi:10.1007/s11263-019-01251-8 · Cited by 298
ForgeryNet: A Versatile Benchmark for Comprehensive Forgery Analysis
He, Y., Gan, B., Chen, S., Zhou, Y., Yin, G., Song, L., Sheng, L., Shao, J., & Liu, Z. (2021) · 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr46437.2021.00434 · Cited by 161
Zhang, Y., Park, D. S., Han, W., Qin, J., Gulati, A., Shor, J., Jansen, A., Xu, Y., Huang, Y., Wang, S., et al. (2022) · IEEE Journal of Selected Topics in Signal Processing · doi:10.1109/jstsp.2022.3182537 · Cited by 161
Metadata
- Authors
- Cao, H., Cooper, D. G., Keutmann, M. K., Gur, R. C., Nenkova, A., & Verma, R.
- Venue
- IEEE Transactions on Affective Computing
- Published
- 2014-10-01
- Volume
- 5
- Issue
- 4
- Pages
- 377-390
- ISSN
- 1949-3045 (ISSN-L)
- OpenAlex
- W2030931454
- Cited by
- 914
- References
- 51
- FWCI
- 4.88
- Type
- journal article
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.