The OMG-Emotion Behavior Dataset
Open accessThe DOI matches its Crossref record; 20 of the 21 references it cites were found in a registry or an index.
Abstract
This paper is the basis paper for the accepted IJCNN challenge One-Minute Gradual-Emotion Recognition (OMG-Emotion)1by which we hope to foster long-emotion classification using neural models for the benefit of the IJCNN community. The proposed corpus has as novelty the data collection and annotation strategy based on emotion expressions which evolve over time into a specific context. Different from other corpora, we propose a novel multimodal corpus for emotion expression recognition, which uses gradual annotations with a focus on contextual emotion expressions. Our dataset was collected from Youtube videos using a specific search strategy based on restricted keywords and filtering which guaranteed that the data follow a gradual emotion expression transition, i.e. emotion expressions evolve over time in a natural and continuous fashion. We also provide an experimental protocol and a series of unimodal baseline experiments which can be used to evaluate deep and recurrent neural models in a fair and standard manner.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source says what the claim attributes to it.
In the citing paper
“The annotation strategy incorporates both discrete (anger, disgust, fear, happiness, sadness, surprise, and neutral) and continuous (valence and arousal) measures.”
From A Survey of Deep Learning-Based Multimodal Emotion Recognition: Speech, Text, and Face (2023) · cites this paper as [31]
The cited source says what the claim attributes to it.
In the citing paper
“As part of their assignment, annotators watch the video segments continuously and annotate each one according to the valence/arousal scale and Ekman’s theory of universal emotions.”
From A Survey of Deep Learning-Based Multimodal Emotion Recognition: Speech, Text, and Face (2023) · cites this paper as [31]
The cited source says what the claim attributes to it.
In the citing paper
“The unique selection process of these videos, centered around the term “monologue”, is designed to depict the gradual evolution of emotions.”
From A Survey of Deep Learning-Based Multimodal Emotion Recognition: Speech, Text, and Face (2023) · cites this paper as [31]
The cited source says what the claim attributes to it.
In the citing paper
“The One-Minute Gradual-Emotion Recognition (OMG) dataset [31], which incorporates videos from a multitude of YouTube channels, provides a comprehensive view of various emotional behaviors within different contexts.”
From A Survey of Deep Learning-Based Multimodal Emotion Recognition: Speech, Text, and Face (2023) · cites this paper as [31]
The cited source says what the claim attributes to it.
In the citing paper
“Every video within the OMG dataset is segmented based on speech, and each of these segments is annotated by at least five independent individuals.”
From A Survey of Deep Learning-Based Multimodal Emotion Recognition: Speech, Text, and Face (2023) · cites this paper as [31]
1 more claim about this paper
Create a free account to see every claim other papers make about this one.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
A concordance correlation coefficient to evaluate reproducibility.
Lin, L. I.-K. (1989) · Biometrics · doi:10.2307/2532051 · Cited by 8,785
Constants across cultures in the face and emotion.
Ekman, P., & Friesen, W. V. (1971) · Journal of Personality and Social Psychology · doi:10.1037/h0030377 · Cited by 5,509
IEMOCAP: Interactive emotional dyadic motion capture database
Busso, C., Bulut, M., Lee, C.-C., Kazemzadeh, A., Mower, E., Kim, S., Chang, J. N., Lee, S., & Narayanan, S. S. (2008) · Language Resources and Evaluation · doi:10.1007/s10579-008-9076-6 · Cited by 3,869
Recent developments in openSMILE, the munich open-source multimedia feature extractor
Eyben, F., Weninger, F., Gross, F., & Schuller, B. (2013) · Proceedings of the 21st ACM international conference on Multimedia · doi:10.1145/2502081.2502224 · Cited by 1,241
Cognitive Emotion Regulation: Insights From Social Cognitive and Affective Neuroscience
Ochsner, K. N., & Gross, J. J. (2008) · Current Directions in Psychological Science · doi:10.1111/j.1467-8721.2008.00566.x · Cited by 988
Basic emotions, relations among emotions, and emotion-cognition relations.
Izard, C. E. (1992) · Psychological Review · doi:10.1037/0033-295x.99.3.561 · Cited by 972
Schuller, B., Steidl, S., Batliner, A., Vinciarelli, A., Scherer, K., Ringeval, F., Chetouani, M., Weninger, F., Eyben, F., Marchi, E., et al. (2013) · Interspeech 2013 · doi:10.21437/interspeech.2013-56 · Cited by 731
A survey of multimodal sentiment analysis
Soleymani, M., Garcia, D., Jou, B., Schuller, B., Chang, S.-F., & Pantic, M. (2017) · Image and Vision Computing · doi:10.1016/j.imavis.2017.08.003 · Cited by 601
A Note On "A Concordance Correlation Coefficient to Evaluate Reproducibility"
Nickerson, C. A. E. (1997) · Biometrics · doi:10.2307/2533516 · Cited by 510
MOSI: Multimodal Corpus of Sentiment Intensity and Subjectivity Analysis in Online Opinion Videos
Zadeh, A., Zellers, R., Pincus, E., & Morency, L.-P. (2016) · arXiv · doi:10.48550/arxiv.1606.06259 · Cited by 350
Mapping discrete and dimensional emotions onto the brain: controversies and consensus
Hamann, S. (2012) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2012.07.006 · Cited by 316
An empirical study of machine learning techniques for affect recognition in human–robot interaction
Rani, P., Liu, C., Sarkar, N., & Vanman, E. (2006) · Pattern Analysis and Applications · doi:10.1007/s10044-006-0025-y · Cited by 313
Aff-Wild: Valence and Arousal ‘In-the-Wild’ Challenge
Zafeiriou, S., Kollias, D., Nicolaou, M. A., Papaioannou, A., Zhao, G., & Kotsia, I. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw.2017.248 · Cited by 297
From individual to group-level emotion recognition: EmotiW 5.0
Dhall, A., Goecke, R., Ghosh, S., Joshi, J., Hoey, J., & Gedeon, T. (2017) · Proceedings of the 19th ACM International Conference on Multimodal Interaction · doi:10.1145/3136755.3143004 · Cited by 218
Evaluation of natural emotions using self assessment manikins
Grimm, M., & Kroschel, K. (2005) · IEEE Workshop on Automatic Speech Recognition and Understanding, 2005. · doi:10.1109/asru.2005.1566530 · Cited by 179
EmoReact: a multimodal approach and dataset for recognizing emotional responses in children
Nojavanasghari, B., Baltrušaitis, T., Hughes, C. E., & Morency, L.-P. (2016) · Proceedings of the 18th ACM International Conference on Multimodal Interaction · doi:10.1145/2993148.2993168 · Cited by 119
Han, J., Zhang, Z., Schmitt, M., Pantić, M., & Schuller, B. W. (2017) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3123266.3123383 · Cited by 118
Developing crossmodal expression recognition based on a deep neural model
Barros, P., & Wermter, S. (2016) · Adaptive Behavior · doi:10.1177/1059712316664017 · Cited by 72
Self-citationMethods and measures in developmental emotions research: Some assembly required
Thompson, R. A. (2011) · Journal of Experimental Child Psychology · doi:10.1016/j.jecp.2011.04.007 · Cited by 54
GIFGIF+: Collecting emotional animated GIFs with clustered multi-task learning
Chen, W. '., Rudovic, O. O., & Picard, R. W. (2017) · International Conference on Affective Computing and Intelligent Interaction and workshops · doi:10.1109/acii.2017.8273647 · Cited by 23
Cited by
Multimodal Emotion Recognition with Deep Learning: Advancements, challenges, and future directions
A.V., G., T., M., D., P., & E., U. (2024) · Information Fusion · doi:10.1016/j.inffus.2023.102218 · Cited by 225
A Survey of Deep Learning-Based Multimodal Emotion Recognition: Speech, Text, and Face
Lian, H., Lu, C., Li, S., Zhao, Y., Tang, C., & Zong, Y. (2023) · Entropy · doi:10.3390/e25101440 · Cited by 224
“The One-Minute Gradual-Emotion Recognition (OMG) dataset [31], which incorporates videos from a multitude of YouTube channels, provides a comprehensive view of various emotional behaviors within different contexts.”
BackgroundAnalysing Affective Behavior in the second ABAW2 Competition
Kollias, D., & Zafeiriou, S. (2021) · 2021 IEEE/CVF International Conference on Computer Vision Workshops (ICCVW) · doi:10.1109/iccvw54120.2021.00408 · Cited by 170
Affect Analysis in-the-wild: Valence-Arousal, Expressions, Action Units and a Unified FrameworkPreprint
Kollias, D., & Zafeiriou, S. P. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2103.15792 · Cited by 297
Multitask Emotion Recognition with Incomplete Labels
Deng, D., Chen, Z., & Shi, B. E. (2020) · International Conference on Automatic Face and Gesture Recognition · doi:10.1109/fg47880.2020.00131 · Cited by 120
Metadata
- Authors
- Barros, P., Churamani, N., Lakomkin, E., Siqueira, H., Sutherland, A., & Wermter, S.
- Affiliations
- Universität Hamburg (DE)
- Venue
- 2018 International Joint Conference on Neural Networks (IJCNN)
- Publisher
- IEEE
- Published
- 2018-07
- Pages
- 1-7
- ISSN
- 2161-4407 (ISSN-L)
- OpenAlex
- W2792522983
- Cited by
- 121
- References
- 21
- Citations by year
- 2026: 6 · 2025: 4 · 2024: 16 · 2023: 20 · 2022: 16 · 2021: 18 · 2020: 16 · 2019: 13 · 2018: 12
- FWCI
- 17.89
- Type
- conference paper
- Language
- en
- Open access
- green
- arxiv.org
- Licence
- cc-by
- Locations
- International Joint Conference on Neural Networks (IJCNN) (publishedVersion)
- arXiv (Cornell University) (submittedVersion, cc-by)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University) (cc-by)
- Funders
- European Commission (642667, 721619)
- Deutsche Forschungsgemeinschaft (TRR169, CML (TRR 169))
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Emotion and Mood Recognition
- Physical Sciences › Computer Science › Artificial Intelligence › Sentiment Analysis and Opinion Mining
- Physical Sciences › Computer Science › Signal Processing › Speech and Audio Processing
- Keywords
- emotion recognition, multimodal corpus, emotion annotation, benchmark dataset
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.