The OMG-Emotion Behavior Dataset
Open accessThe DOI matches its Crossref record; 20 of the 21 references it cites were found in a registry or an index.
Abstract
This paper is the basis paper for the accepted IJCNN challenge One-Minute Gradual-Emotion Recognition (OMG-Emotion)1by which we hope to foster long-emotion classification using neural models for the benefit of the IJCNN community. The proposed corpus has as novelty the data collection and annotation strategy based on emotion expressions which evolve over time into a specific context. Different from other corpora, we propose a novel multimodal corpus for emotion expression recognition, which uses gradual annotations with a focus on contextual emotion expressions. Our dataset was collected from Youtube videos using a specific search strategy based on restricted keywords and filtering which guaranteed that the data follow a gradual emotion expression transition, i.e. emotion expressions evolve over time in a natural and continuous fashion. We also provide an experimental protocol and a series of unimodal baseline experiments which can be used to evaluate deep and recurrent neural models in a fair and standard manner.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source says what the claim attributes to it.
In the citing paper
“The annotation strategy incorporates both discrete (anger, disgust, fear, happiness, sadness, surprise, and neutral) and continuous (valence and arousal) measures.”
From A Survey of Deep Learning-Based Multimodal Emotion Recognition: Speech, Text, and Face (2023) · cites this paper as [31]
The cited source says what the claim attributes to it.
In the citing paper
“As part of their assignment, annotators watch the video segments continuously and annotate each one according to the valence/arousal scale and Ekman’s theory of universal emotions.”
From A Survey of Deep Learning-Based Multimodal Emotion Recognition: Speech, Text, and Face (2023) · cites this paper as [31]
The cited source says what the claim attributes to it.
In the citing paper
“The unique selection process of these videos, centered around the term “monologue”, is designed to depict the gradual evolution of emotions.”
From A Survey of Deep Learning-Based Multimodal Emotion Recognition: Speech, Text, and Face (2023) · cites this paper as [31]
The cited source says what the claim attributes to it.
In the citing paper
“The One-Minute Gradual-Emotion Recognition (OMG) dataset [31], which incorporates videos from a multitude of YouTube channels, provides a comprehensive view of various emotional behaviors within different contexts.”
From A Survey of Deep Learning-Based Multimodal Emotion Recognition: Speech, Text, and Face (2023) · cites this paper as [31]
The cited source says what the claim attributes to it.
In the citing paper
“Every video within the OMG dataset is segmented based on speech, and each of these segments is annotated by at least five independent individuals.”
From A Survey of Deep Learning-Based Multimodal Emotion Recognition: Speech, Text, and Face (2023) · cites this paper as [31]
1 more claim about this paper
Create a free account to see every claim other papers make about this one.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
A concordance correlation coefficient to evaluate reproducibility.
Lin, L. I.-K. (1989) · Biometrics · doi:10.2307/2532051 · Cited by 8,785
Constants across cultures in the face and emotion.
Ekman, P., & Friesen, W. V. (1971) · Journal of Personality and Social Psychology · doi:10.1037/h0030377 · Cited by 5,509
IEMOCAP: Interactive emotional dyadic motion capture database
Busso, C., Bulut, M., Lee, C.-C., Kazemzadeh, A., Mower, E., Kim, S., Chang, J. N., Lee, S., & Narayanan, S. S. (2008) · Language Resources and Evaluation · doi:10.1007/s10579-008-9076-6 · Cited by 3,869
Recent developments in openSMILE, the munich open-source multimedia feature extractor
Eyben, F., Weninger, F., Gross, F., & Schuller, B. (2013) · Proceedings of the 21st ACM international conference on Multimedia · doi:10.1145/2502081.2502224 · Cited by 1,241
Cognitive Emotion Regulation: Insights From Social Cognitive and Affective Neuroscience
Ochsner, K. N., & Gross, J. J. (2008) · Current Directions in Psychological Science · doi:10.1111/j.1467-8721.2008.00566.x · Cited by 988
Basic emotions, relations among emotions, and emotion-cognition relations.
Izard, C. E. (1992) · Psychological Review · doi:10.1037/0033-295x.99.3.561 · Cited by 972
Schuller, B., Steidl, S., Batliner, A., Vinciarelli, A., Scherer, K., Ringeval, F., Chetouani, M., Weninger, F., Eyben, F., Marchi, E., et al. (2013) · Interspeech 2013 · doi:10.21437/interspeech.2013-56 · Cited by 731
A survey of multimodal sentiment analysis
Soleymani, M., Garcia, D., Jou, B., Schuller, B., Chang, S.-F., & Pantic, M. (2017) · Image and Vision Computing · doi:10.1016/j.imavis.2017.08.003 · Cited by 601
A Note On "A Concordance Correlation Coefficient to Evaluate Reproducibility"
Nickerson, C. A. E. (1997) · Biometrics · doi:10.2307/2533516 · Cited by 510
MOSI: Multimodal Corpus of Sentiment Intensity and Subjectivity Analysis in Online Opinion Videos
Zadeh, A., Zellers, R., Pincus, E., & Morency, L.-P. (2016) · arXiv · doi:10.48550/arxiv.1606.06259 · Cited by 350
Cited by
Multimodal Emotion Recognition with Deep Learning: Advancements, challenges, and future directions
A.V., G., T., M., D., P., & E., U. (2024) · Information Fusion · doi:10.1016/j.inffus.2023.102218 · Cited by 225
A Survey of Deep Learning-Based Multimodal Emotion Recognition: Speech, Text, and Face
Lian, H., Lu, C., Li, S., Zhao, Y., Tang, C., & Zong, Y. (2023) · Entropy · doi:10.3390/e25101440 · Cited by 224
“The One-Minute Gradual-Emotion Recognition (OMG) dataset [31], which incorporates videos from a multitude of YouTube channels, provides a comprehensive view of various emotional behaviors within different contexts.”
BackgroundAnalysing Affective Behavior in the second ABAW2 Competition
Kollias, D., & Zafeiriou, S. (2021) · 2021 IEEE/CVF International Conference on Computer Vision Workshops (ICCVW) · doi:10.1109/iccvw54120.2021.00408 · Cited by 170
Affect Analysis in-the-wild: Valence-Arousal, Expressions, Action Units and a Unified FrameworkPreprint
Kollias, D., & Zafeiriou, S. P. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2103.15792 · Cited by 297
Multitask Emotion Recognition with Incomplete Labels
Deng, D., Chen, Z., & Shi, B. E. (2020) · International Conference on Automatic Face and Gesture Recognition · doi:10.1109/fg47880.2020.00131 · Cited by 119
Intensity-Aware Loss for Dynamic Facial Expression Recognition in the Wild
Li, H., Niu, H., Zhu, Z., & Zhao, F. (2023) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v37i1.25077 · Cited by 136
Modeling Emotion in Complex Stories: The Stanford Emotional Narratives Dataset
Ong, D. C., Wu, Z., Zhi‐Xuan, T., Reddan, M. C., Kahhalé, I., Mattek, A. M., & Zaki, J. (2019) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2019.2955949 · Cited by 98
The CANDOR corpus: Insights from a large multimodal dataset of naturalistic conversation
Reece, A., Cooney, G., Bull, P. E., Chung, C., Dawson, B., Fitzpatrick, C. A., Glazer, T., Knox, D., Liebscher, A., & Marin, S. (2023) · Science Advances · doi:10.1126/sciadv.adf3197 · Cited by 206
EmoBed: Strengthening Monomodal Emotion Recognition via Training with Crossmodal Emotion Embeddings
Han, J., Zhang, Z., Ren, Z. X., & Schuller, B. W. (2019) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2019.2928297 · Cited by 74
Exploring the Contextual Factors Affecting Multimodal Emotion Recognition in Videos
Bhattacharya, P., Gupta, R. K., & Yang, Y. (2021) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2021.3071503 · Cited by 71
MIMAMO Net: Integrating Micro- and Macro-Motion for Video Emotion Recognition
Deng, D., Chen, Z., Zhou, Y., & Shi, B. E. (2020) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v34i03.5646 · Cited by 65
Do I Have a Personality? Endowing Care Robots with Context-Dependent Personality Traits
Andriella, A., Siqueira, H. L., Fu, D., Magg, S., Barros, P. V. A., Wermter, S., Torras, C., & Alenyà, G. (2020) · International Journal of Social Robotics · doi:10.1007/s12369-020-00690-5 · Cited by 64
Self-citationGarcía-Hernández, R. A., Luna-García, H., Celaya-Padilla, J. M., García-Hernández, A., Reveles-Gómez, L. C., Flores-Chaires, L. A., Delgado-Contreras, J. R., Rondon, D., & Villalba‐Condori, K. O. (2024) · Applied Sciences · doi:10.3390/app14167165 · Cited by 85
The unbearable (technical) unreliability of automated facial emotion recognition
Cabitza, F., Campagner, A., & Mattioli, M. (2022) · Big Data & Society · doi:10.1177/20539517221129549 · Cited by 72
From Pixels to Affect: A Study on Games and Player Experience
Makantasis, K., Liapis, A., & Yannakakis, G. N. (2019) · International Conference on Affective Computing and Intelligent Interaction and workshops · doi:10.1109/acii.2019.8925493 · Cited by 50
Implicit Fusion by Joint Audiovisual Training for Emotion Recognition in Mono Modality
Han, J., Zhang, Z., Ren, Z. X., & Schuller, B. W. (2019) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp.2019.8682773 · Cited by 54
MEmoR: A Dataset for Multimodal Emotion Reasoning in Videos
Shen, G., Wang, X. Y., Duan, X., Li, H., & Zhu, W. (2020) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3394171.3413909 · Cited by 73
Du, Z., Wu, S., Huang, D., Li, W., & Wang, Y. (2019) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2019.2940224 · Cited by 44
Affect-Driven Learning of Robot Behaviour for Collaborative Human-Robot Interactions
Churamani, N., Barros, P. V. A., Güneş, H., & Wermter, S. (2022) · Frontiers in Robotics and AI · doi:10.3389/frobt.2022.717193 · Cited by 50
Self-citationThe FaceChannel: A Fast and Furious Deep Neural Network for Facial Expression Recognition
Barros, P. V. A., Churamani, N., & Sciutti, A. (2020) · SN Computer Science · doi:10.1007/s42979-020-00325-6 · Cited by 51
Self-citationZhou, S., Wu, X., Jiang, F., Huang, Q., & Huang, C. (2023) · International Journal of Environmental Research and Public Health · doi:10.3390/ijerph20021400 · Cited by 48
Affect2MM: Affective Analysis of Multimedia Content Using Emotion Causality
Mittal, T., Mathur, P., Bera, A., & Manocha, D. (2021) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr46437.2021.00561 · Cited by 53
A Multi-component CNN-RNN Approach for Dimensional Emotion Recognition in-the-wildPreprint
Kollias, D., & Zafeiriou, S. P. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1805.01452 · Cited by 52
Survey of deep emotion recognition in dynamic data using facial, speech and textual cues
Zhang, T., & Tan, Z. (2024) · Multimedia Tools and Applications · doi:10.1007/s11042-023-17944-9 · Cited by 46
Emotion Recognition System from Speech and Visual Information based on Convolutional Neural Networks
Ristea, N.-C., Dutu, L. C., & Radoi, A. (2019) · International Conference on Speech Technology and Human-Computer Dialogue (SpeD) · doi:10.1109/sped.2019.8906538 · Cited by 45
Mattioli, M., & Cabitza, F. (2024) · Machine Learning and Knowledge Extraction · doi:10.3390/make6040109 · Cited by 59
Antoniadis, P., Pikoulis, I., Filntisis, P. P., & Maragos, P. A. (2021) · IEEE International Conference on Computer Vision Workshops · doi:10.1109/iccvw54120.2021.00407 · Cited by 37
Training Deep Neural Networks with Different Datasets In-the-wild: The Emotion Recognition Paradigm
Kollias, D., & Zafeiriou, S. P. (2018) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn.2018.8489340 · Cited by 57
MGEED: A Multimodal Genuine Emotion and Expression Detection Database
Wang, Y., Yu, H. L., Gao, W., Xia, Y., & Nduka, C. C. (2023) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2023.3286351 · Cited by 40
Multimodal Utterance-level Affect Analysis using Visual, Audio and Text FeaturesPreprint
Deng, D., Zhou, Y., Pi, J., & Shi, B. E. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1805.00625 · Cited by 43
On the Gap between Domestic Robotic Applications and Computational Intelligence
Zhong, J., Ling, C., Cangelosi, A., Lotfi, A., & Liu, X. (2021) · Electronics · doi:10.3390/electronics10070793 · Cited by 37
The FaceChannel: A Light-weight Deep Neural Network for Facial Expression Recognition
Barros, P., Churamani, N., & Sciutti, A. (2020) · International Conference on Automatic Face and Gesture Recognition · doi:10.1109/fg47880.2020.00070 · Cited by 33
Self-citationA Multimodal Non-Intrusive Stress Monitoring From the Pleasure-Arousal Emotional Dimensions
Dahmane, M., Alam, J., St-Charles, P.-L., Lalonde, M., Heffner, K., & Foucher, S. (2020) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2020.2988455 · Cited by 29
Kollias, D., & Zafeiriou, S. (2021) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2020.3014171 · Cited by 179
A Personalized Affective Memory Model for Improving Emotion Recognition
Barros, P. V. A., Parisi, G. I., & Wermter, S. (2019) · International Conference on Machine Learning · Cited by 36
Self-citationGenerative technology for human emotion recognition: A scoping review
Ma, F., Yuan, Y., Xie, Y., Ren, H., Liu, I., He, Y., Ren, F., Yu, F. R., & Ni, S. (2024) · Information Fusion · doi:10.1016/j.inffus.2024.102753 · Cited by 61
EmoSeC: Emotion recognition from scene context
Thuseethan, S., Rajasegarar, S., & Yearwood, J. L. (2022) · Neurocomputing · doi:10.1016/j.neucom.2022.04.019 · Cited by 27
Multimodal Emotion Recognition for One-Minute-Gradual Emotion ChallengePreprint
Zheng, Z., Cao, C., Chen, X., & Xu, G. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1805.01060 · Cited by 23
A Deep Network for Arousal-Valence Emotion Prediction with Acoustic-Visual CuesPreprint
Peng, S., Zhang, L., Ban, Y., Fang, M., & WINKLER, S. S. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1805.00638 · Cited by 20
HICEM: A High-Coverage Emotion Model for Artificial Emotional Intelligence
Wortman, B., & Wang, J. Z. (2023) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2023.3324902 · Cited by 27
Kerzel, M., Rosin, T. P., Strahl, E., Heinrich, S., & Wermter, S. (2020) · Frontiers in Neurorobotics · doi:10.3389/fnbot.2020.00028 · Cited by 19
Self-citationVEATIC: Video-based Emotion and Affect Tracking in Context Dataset
Ren, Z., Ortega, J., Wang, Y., Chen, Z., Guo, Y., Yu, S. X., & Whitney, D. C. (2024) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv57701.2024.00441 · Cited by 37
MUSER: MUltimodal Stress detection using Emotion Recognition as an Auxiliary Task
Yao, Y., Papakostas, M., Burzo, M. G., Abouelenien, M., & Mihalcea, R. F. (2021) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/2021.naacl-main.216 · Cited by 30
Reece, A. G., Cooney, G., Bull, P. E., Chung, C., Dawson, B., Fitzpatrick, C. A., Glazer, T., Knox, D., Liebscher, A., & Marin, S. (2022) · doi:10.31234/osf.io/ts43f · Cited by 17
Context-based emotion recognition: A survey
Abbas, R., Ni, B., Ma, R., Li, T., Lu, Y., & Li, X. (2024) · Neurocomputing · doi:10.1016/j.neucom.2024.129073 · Cited by 24
KT-Speech-Crawler: Automatic Dataset Construction for Speech Recognition from YouTube Videos
Lakomkin, E., Magg, S., Weber, C., & Wermter, S. (2018) · Conference on Empirical Methods in Natural Language Processing: System Demonstrations (EMNLP) · doi:10.18653/v1/d18-2016 · Cited by 17
Self-citationHuman Emotion: A Survey focusing on Languages, Ontologies, Datasets, and Systems
Elkobaisi, M. R., Machot, F. A., & Mayr, H. C. (2022) · SN Computer Science · doi:10.1007/s42979-022-01116-x · Cited by 21
Barros, P. V. A., Barakova, E. I., & Wermter, S. (2020) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2020.3002657 · Cited by 17
Self-citationMapping Discrete Emotions in the Dimensional Space: An Acoustic Approach
Trnka, M., Darjaa, S., Ritomský, M., Sabo, R., Rusko, M., Schaper, M., & Stelkens-Kobsch, T. H. (2021) · Electronics · doi:10.3390/electronics10232950 · Cited by 15
Gaze-enhanced Crossmodal Embeddings for Emotion Recognition
Abdou, A., Sood, E., Müller, P., & Bulling, A. (2022) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3530879 · Cited by 14
Disentangling Prosody Representations With Unsupervised Speech Reconstruction
Qu, L., Li, T., Weber, C., Rosin, T. P., Ren, F., & Wermter, S. (2023) · IEEE/ACM Transactions on Audio Speech and Language Processing · doi:10.1109/taslp.2023.3320864 · Cited by 30
Self-citationA Multimodal LSTM for Predicting Listener Empathic Responses Over Time
Zhi‐Xuan, T., Goel, A., Nguyen, T.-S., & Ong, D. C. (2019) · International Conference on Automatic Face and Gesture Recognition · doi:10.1109/fg.2019.8756577 · Cited by 15
Learning from Label Relationships in Human Affect
Foteinopoulou, N. M., & Patras, I. (2022) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3503161.3548373 · Cited by 19
The OMG-Empathy Dataset: Evaluating the Impact of Affective Behavior in Storytelling
Barros, P. V. A., Churamani, N., Lim, A., & Wermter, S. (2019) · International Conference on Affective Computing and Intelligent Interaction and workshops · doi:10.1109/acii.2019.8925530 · Cited by 12
Self-citationIntegrating Multimodal Affective Signals for Stress Detection from Audio-Visual Data
Ghose, D., Gitelson, O., & Scassellati, B. M. (2024) · INTERNATIONAL CONFERENCE ON MULTIMODAL INTERACTION · doi:10.1145/3678957.3685717 · Cited by 19
Affective Computing for eHealth Using Low-Cost Remote Internet of Things-Based EMG Platform
Tamulis, Ž., Vasiljevas, M., Damaševičius, R., Maskeliūnas, R., & Misra, S. (2022) · Internet of things · doi:10.1007/978-3-030-81473-1_3 · Cited by 10
Across the Universe: Biasing Facial Representations Toward Non-Universal Emotions With the Face-STN
Barros, P. V. A., & Sciutti, A. (2022) · IEEE Access · doi:10.1109/access.2022.3210183 · Cited by 10
Self-citationTikTok for good: Creating a diverse emotion expression database
Surabhi, S., Shah, B., Washington, P. Y., Mutlu, O. C., Leblanc, É., Mohite, P., Husic, A., Kline, A., Dunlap, K., McNealis, M., et al. (2022) · IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw56347.2022.00279 · Cited by 10
A Survey on the Semi Supervised Learning Paradigm in the Context of Speech Emotion Recognition
Andrade, G., Rodrigues, M., & Nováis, P. (2021) · Lecture notes in networks and systems · doi:10.1007/978-3-030-82196-8_57 · Cited by 9
A Deep Neural Model Of Emotion AppraisalPreprint
Barros, P. V. A., Barakova, E. I., & Wermter, S. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1808.00252 · Cited by 9
Self-citationZhang, H., Yin, L., & Zhang, H. (2024) · Ergonomics · doi:10.1080/00140139.2024.2359123 · Cited by 7
Dimensional emotion recognition from camera-based PRV features
Zhou, K., Schinle, M., & Stork, W. (2023) · Methods · doi:10.1016/j.ymeth.2023.08.014 · Cited by 9
Semi-supervised Model for Emotion Recognition in Speech
Pereira, I., dos Santos, D. P., Maciel, A. M. A., & Barros, P. V. A. (2018) · Lecture notes in computer science · doi:10.1007/978-3-030-01418-6_77 · Cited by 6
Self-citationCIAO! A Contrastive Adaptation Mechanism for Non-Universal Facial Expression Recognition
Barros, P. V. A., & Sciutti, A. (2022) · International Conference on Affective Computing and Intelligent Interaction and workshops · doi:10.1109/acii55700.2022.9953863 · Cited by 11
Self-citationDeep Emotion Recognition using Facial, Speech and Textual Cues: A SurveyPreprint
Zhang, T., & Tan, Z. (2021) · doi:10.36227/techrxiv.15184302.v2 · Cited by 9
Kerkeni, L., & Gueuret, T. (2024) · ELCVIA Electronic Letters on Computer Vision and Image Analysis · doi:10.5565/rev/elcvia.1811 · Cited by 6
Recent Trends in Artificial Intelligence for Emotion Detection using Facial Image Analysis
Jain, P. R., Quadri, S. M. K., & Lalit, M. (2021) · International Conference on Contemporary Computing · doi:10.1145/3474124.3474205 · Cited by 10
Arousal and Valence Estimation for Visual Non-Intrusive Stress Monitoring
Dahmane, M., St-Charles, P.-L., Lalonde, M., Heffner, K., & Foucher, S. (2019) · International Conference on Image Processing Theory, Tools and Applications (IPTA) · doi:10.1109/ipta.2019.8936110 · Cited by 5
A Personalized Affective Memory Neural Model for Improving Emotion RecognitionPreprint
Barros, P. V. A., Parisi, G. I., & Wermter, S. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1904.12632 · Cited by 8
Self-citationTechnical Report for Valence-Arousal Estimation on Affwild2 DatasetPreprint
Li, I.-H. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2105.01502 · Cited by 13
A Comprehensive Survey on Datasets for Affective Computing and Mental Disorder
Sze, R. B., & Brandão, W. C. (2025) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2025.3624354 · Cited by 6
An Enroll-to-Verify Approach for Cross-Task Unseen Emotion Class Recognition
Li, J.-L., & Lee, C.-C. (2022) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2022.3183166 · Cited by 4
Brügge, N. S., Mohammadi, E., Münchau, A. M., Bäumer, T., Frings, C., Beste, C., Roessner, V., & Handels, H. (2023) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp49357.2023.10095309 · Cited by 4
Datasets for Valence and Arousal Inference: A Survey
Schneider, H. M., Pavlitska, S., Gremmelmaier, H., & Zöllner, J. M. (2025) · IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw67362.2025.00563 · Cited by 4
Multimodal Russian Corpus and its use in emotional studies
Savchuk, S. O., & Makhova, A. (2021) · Russian Journal of Communication · doi:10.1080/19409419.2021.1887991 · Cited by 7
An Overview of Emotion Recognition from Body Movement
Takalloo, L. E., Li, K. F., & Takano, K. (2022) · Lecture notes in networks and systems · doi:10.1007/978-3-031-08812-4_11 · Cited by 3
A Multi-Modal Approach to Sensing Human Emotion
Gibilisco, H., Laubenberger, M., Spiridonov, V., Belga, J., Hallstrom, J. O., & Peluso, P. R. (2018) · IEEE International Conference on Big Data (BigData) · doi:10.1109/bigdata.2018.8622451 · Cited by 3
Deep Emotion Recognition using Facial, Speech and Textual Cues: A SurveyPreprint
Zhang, T., & Tan, Z. (2021) · doi:10.36227/techrxiv.15184302 · Cited by 4
TACL: A Trusted Action-enhanced Curriculum Learning Approach to Multimodal Affective Computing
Yu, T., Wang, J., Luo, J., Wang, J., & Zhou, G. (2024) · Neurocomputing · doi:10.1016/j.neucom.2024.129195 · Cited by 5
Dimensional emotion recognition using visual and textual cuesPreprint
Ferreira, P. M., Pernes, D., Fernandes, K., Rebelo, A., & Cardoso, J. S. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1805.01416 · Cited by 2
OMG Emotion Challenge - ExCouple TeamPreprint
Pereira, I., & dos Santos, D. P. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1805.01576 · Cited by 2
The Contextual Dynamics of Multimodal Emotion Recognition in VideosPreprint
Bhattacharya, P., Gupta, R. K., & Yang, Y. (2020) · arXiv (Cornell University) · Cited by 2
Technical Report for Valence-Arousal Estimation in ABAW2 ChallengePreprint
Xie, H.-X., Li, I.-H., Lo, L., Shuai, H.-H., & Cheng, W.-H. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2107.03891 · Cited by 10
Sparse Maximum Margin Learning from Multimodal Human Behavioral Patterns
Zheng, E., Yu, Q., & Zheng, Z. (2023) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v37i4.25676 · Cited by 1
Annotations from speech and heart rate: impact on multimodal emotion recognition
Sharma, K., & Chanel, G. (2023) · INTERNATIONAL CONFERENCE ON MULTIMODAL INTERACTION · doi:10.1145/3577190.3614165 · Cited by 3
Effects of Facial Expressions and Gestures on the Trustworthiness of a Person
Lutfi, S. L., Lahasan, B. M., Luna-Jiménez, C., Bamasood, Z. A., & Akhtar, Z. (2023) · IEEE Access · doi:10.1109/access.2023.3334270 · Cited by 5
VATE: A Large Scale Multimodal Spontaneous Dataset for Affective Evaluation
Agnelli, F., Grossi, G., D’Amelio, A., Paoli, M. D., & Lanzarotti, R. (2025) · Lecture notes in computer science · doi:10.1007/978-3-031-91575-8_13 · Cited by 2
Jamil, S., Ali, T., Nawaz, A., & Shahid, A. (2026) · PLoS ONE · doi:10.1371/journal.pone.0346770 · Cited by 1
Almotlak, H., Weber, C., Qu, L., & Wermter, S. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-61609-0_42 · Cited by 1
Self-citationEmotional Meta-annotation of Educational Videos. A Review of Emotional Database Characterization
Astudillo, G. J., Sanz, C. V., & Baldassarri, S. (2022) · Communications in computer and information science · doi:10.1007/978-3-031-22210-8_4 · Cited by 1
Cross-Modality Learning by Exploring Modality Interactions for Emotion Reasoning
Tran, T.-D., Ho, N.-H., Pant, S., Yang, H.-J., Kim, S.-H., & Lee, G.-S. (2023) · IEEE Access · doi:10.1109/access.2023.3283597 · Cited by 1
Multimodal Emotion Recognition based on 2D Kernel Density Estimation for Multiple Labels Fusion
Luo, Z., & Komatani, K. (2023) · Asia-Pacific Signal and Information Processing Association Annual Summit and Conference (APSIPA ASC) · doi:10.1109/apsipaasc58517.2023.10317262 · Cited by 1
Affective analysis in machine learning using AMIGOS with Gaussian expectation-maximization model
Kaliappan, B., Sudalaiyadumperumal, B. V., & Sudalaimuthu, T. (2023) · International Journal of Reconfigurable and Embedded Systems (IJRES) · doi:10.11591/ijres.v13.i1.pp201-209 · Cited by 1
Towards a Better Understanding of Human Emotions: Challenges of Dataset Labeling
Guerdelli, H., Ferrari, C., Neto, J. B. C., Berretti, S., Barhoumi, W., & Bimbo, A. D. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-51023-6_21 · Cited by 4
Refining Valence-Arousal Estimation with Dual-Stream Label Density Smoothing
Xie, H., Li, I.-H., Lo, L., Shuai, H.-H., & Cheng, W.-H. (2024) · International Conference on Consumer Electronics · doi:10.1109/icce59016.2024.10444180 · Cited by 1
A Multimodal, Multilabel Approach to Recognize Emotions in Oral History Interviews
Viswanath, A., Gref, M., Hassan, T., & Schmidt, C. (2024) · International Conference on Affective Computing and Intelligent Interaction and workshops · doi:10.1109/acii63134.2024.00038 · Cited by 1
A weightless regression system for predicting multi-modal empathy
Filho, L. L., Oliveira, L. F. R., Carneiro, H., Guarisa, G. P., Filho, A. L., França, F. M. G., & Lima, P. M. V. (2020) · International Conference on Automatic Face and Gesture Recognition · doi:10.1109/fg47880.2020.00086 · Cited by 1
Analyzing the Influence of Dataset Composition for Emotion RecognitionPreprint
Sutherland, A., Magg, S., Weber, C., & Wermter, S. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2103.03700 · Cited by 1
Self-citationMUSER: MUltimodal Stress Detection using Emotion Recognition as an Auxiliary TaskPreprint
Yao, Y., Papakostas, M., Burzo, M. G., Abouelenien, M., & Mihalcea, R. F. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2105.08146 · Cited by 1
AffecTube — Chrome extension for YouTube video affective annotations
Kulas, D., & Wróbel, M. R. (2023) · SoftwareX · doi:10.1016/j.softx.2023.101504 · Cited by 1
Metadata
- Authors
- Barros, P., Churamani, N., Lakomkin, E., Siqueira, H., Sutherland, A., & Wermter, S.
- Affiliations
- Universität Hamburg (DE)
- Venue
- 2018 International Joint Conference on Neural Networks (IJCNN)
- Publisher
- IEEE
- Published
- 2018-07
- Pages
- 1-7
- ISSN
- 2161-4407 (ISSN-L)
- OpenAlex
- W2792522983
- Cited by
- 121
- References
- 21
- Citations by year
- 2026: 6 · 2025: 4 · 2024: 16 · 2023: 20 · 2022: 16 · 2021: 18 · 2020: 16 · 2019: 13 · 2018: 12
- FWCI
- 17.89
- Type
- conference paper
- Language
- en
- Open access
- green
- arxiv.org
- Licence
- cc-by
- Locations
- International Joint Conference on Neural Networks (IJCNN) (publishedVersion)
- arXiv (Cornell University) (submittedVersion, cc-by)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University) (cc-by)
- Funders
- European Commission (642667, 721619)
- Deutsche Forschungsgemeinschaft (TRR169, CML (TRR 169))
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Emotion and Mood Recognition
- Physical Sciences › Computer Science › Artificial Intelligence › Sentiment Analysis and Opinion Mining
- Physical Sciences › Computer Science › Signal Processing › Speech and Audio Processing
- Keywords
- emotion recognition, multimodal corpus, emotion annotation, benchmark dataset
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.