Prediction of asynchronous dimensional emotion ratings from audiovisual and physiological data

Ringeval, Fabien, Eyben, Florian, Kroupi, Eleni, Yuce, Anil, Thiran, Jean-Philippe, Ebrahimi, Touradj, Lalanne, Denis, Schuller, Björn·2015·Pattern Recognition Letters
DOI 10.1016/j.patrec.2014.11.007

Cited by (201)

Items 101–200 of 201

  1. Predicting Emotionally Salient Regions Using Qualitative Agreement of Deep Neural Network Regressors
    Srinivas Parthasarathy, Carlos Busso · 2018 · IEEE Transactions on Affective Computing
  2. Dynamic Multi-Rater Gaussian Mixture Regression Incorporating Temporal Dependencies of Emotion Uncertainty Using Kalman Filters
    Ting Dang, Vidhyasaharan Sethu, Eliathamby Ambikairajah A · 2018 · IEEE International Conference on Acoustics Speech and Signal Processing
  3. Automatic Recognition of Affective Laughter in Spontaneous Dyadic Interactions from Audiovisual Signals
    Reshmashree B. Kantharaju, Fabien Ringeval, Laurent Besacier · 2018 · INTERNATIONAL CONFERENCE ON MULTIMODAL INTERACTION
  4. Gaze-enhanced Crossmodal Embeddings for Emotion Recognition
    Ahmed Abdou, Ekta Sood, Philipp Müller, Andreas Bulling · 2022 · Proceedings of the ACM on Human-Computer Interaction
  5. Face reading from speech — predicting facial action units from audio cues
    Fabien Ringeval, Erik Marchi, Marc Méhu, Klaus R. Scherer, Björn Wolfgang Schuller · 2015 · Interspeech (USB)
  6. The Conflict Escalation Resolution (CONFER) Database
    Christos Georgakis, Yannis Panagakis, Stefanos P. Zafeiriou, Maja Pantić · 2016 · Image and Vision Computing
  7. AI-Driven Detection of Obstructive Sleep Apnea Using Dual-Branch CNN and Machine Learning Models
    Manjur Kolhar, Manahil Muhammad Alfridan, Rayan A. Siraj · 2025 · Biomedicines
  8. Towards a Better Gold Standard
    Chen Wang, Phil Lopes, Thierry Pun, Guillaume Chanel · 2018
  9. Decoding emotions through personalized multi-modal fNIRS-EEG Systems: Exploring deterministic fusion techniques
    Alireza Farrokhi Nia, Vanessa Tang, Gonzalo D. Maso Talou, Mark Billinghurst · 2025 · Biomedical Signal Processing and Control
  10. An array of physical sensors and an adaptive regression strategy for emotion recognition in a noisy scenario
    F. Mosciano, A. Mencattini, F. Ringeval, Björn Schuller, E. Martinelli, C. Di Natale · 2017 · Sensors and Actuators A Physical
  11. Affect-salient event sequence modelling for continuous speech emotion recognition
    Yizhuo Dong, Xinyu Yang · 2021 · Neurocomputing
  12. An Affect-Based Multimodal Video Recommendation System
    Artūras Kaklauskas, Renaldas Gudauskas, Matas Kozlovas, Lina Pečiūrė, Natalija Lepkova, Justas Čerkauskas, Audrius Banaitis · 2016 · Studies in Informatics and Control
  13. What Can Self-Reports and Acoustic Data Analyses on Emotions Tell Us?
    Samaneh Soleimani, Effie Lai‐Chong Law · 2017 · Designing Interactive Systems Conference
  14. An Improved Multimodal Dimension Emotion Recognition Based on Different Fusion Methods
    Su, Haiyang, Liu, Bin, Tao, Jianhua, Dong, Yongfeng, Huang, Jian, Lian, Zheng, Song, Leichao · 2020 · 2020 15th IEEE International Conference on Signal Processing (ICSP)
  15. Bags in Bag: Generating Context-Aware Bags for Tracking Emotions from Speech
    Jing Han, Zixing Zhang, Maximilian Schmitt, Zhao Xu Ren, Fabien Ringeval, Björn Wolfgang Schuller · 2018 · Interspeech (USB)
  16. Eye-based Continuous Affect Prediction
    Jonny O’Dwyer, Niall Murray, Ronan Flynn · 2019 · International Conference on Affective Computing and Intelligent Interaction and workshops
  17. Multimodal Emotion Recognition Based on Speech and Physiological Signals Using Deep Neural Networks
    Ali Bakhshi, Stephan K. Chalup · 2021 · Lecture notes in computer science
  18. Learning Spatio-Temporal Features With Partial Expression Sequences for On-the-Fly Prediction
    Wissam J. Baddar, Yong Man Ro · 2018 · Proceedings of the AAAI Conference on Artificial Intelligence
  19. Speech emotion recognition based on dynamic convolutional neural network
    Ziyao Lin, Zhangfang Hu, Kuilin Zhu · 2023 · Journal of Computing and Electronic Information Management
  20. Humor Detection System for MuSE 2023: Contextual Modeling, Pesudo Labelling, and Post-smoothing
    Mingyu Xu, Shun Chen, Zheng Lian, Bin Liu · 2023
  21. A Bayesian Filtering Approach for Tracking Sympathetic Arousal and Cortisol-Related Energy From Marked Point Process and Continuous-Valued Observations
    Dilranjan S. Wickramasuriya, Saman Khazaei, Roozbeh Kiani, Rose T. Faghih · 2023 · IEEE Access
  22. MAFONN-EP: A Minimal Angular Feature Oriented Neural Network based Emotion Prediction system in image processing
    L. B. Krithika, G. G. Lakshmi Priya · 2018 · Journal of King Saud University - Computer and Information Sciences
  23. Prediction of Continuous Emotional Measures through Physiological and Visual Data
    Itaf Omar Joudeh, Ana-Maria Creţu, Stéphane Bouchard, Synthia Guimond · 2023 · Sensors
  24. Factor Analysis Based Speaker Normalisation for Continuous Emotion Prediction
    Ting Dang, Vidhyasaharan Sethu, Eliathamby Ambikairajah A · 2016 · Interspeech (USB)
  25. Facial Emotion Recognition
    Vikas Maurya, Ms. Anjali Awasthi, Rajat Kumar Singh, Deepak Maurya, Manorma Dwivedi · 2023 · International Journal for Research in Applied Science and Engineering Technology
  26. Acquisition of Affect
    Björn Schuller · 2017 · Emotions and Personality in Personalized Services
  27. Predicting Affective Dimensions Based on Self Assessed Depression Severity
    Rahul Gupta, Shrikanth Shri Narayanan · 2016 · Interspeech (USB)
  28. Continuous affect prediction using eye gaze and speech
    Jonny O’Dwyer, Ronan Flynn, Niall Murray · 2017 · IEEE International Conference on Bioinformatics and Biomedicine (BIBM)
  29. Multimodal stability-sensitive emotion recognition based on brainwave and physiological signals
    Nattapong Thammasan, Juan Lorenzo Hagad, Ken–ichi Fukui, Masayuki Numao · 2017 · International Conference on Affective Computing and Intelligent Interaction Workshops and Demos (ACIIW)
  30. Understanding auditory representations of emotional expressions with neural networks
    Iris Wieser, Pablo V. A. Barros, Stefan Heinrich, Stefan Wermter · 2018 · Neural Computing and Applications
  31. A Bayesian Filtering Framework for Continuous Affect Recognition From Facial Images
    Ercheng Pei, Yong Hui Zhao, Meshia Cédric Oveneke, Dongmei Jiang, Hichem Sahli B · 2022 · IEEE Transactions on Multimedia
  32. Towards Estimating Emotions and Satisfaction Level of Tourist Based on Eye Gaze and Head Movement
    Dmitrii Fedotov, Yuki Matsuda, Yuta Takahashi, Yutaka Arakawa, Keiichi Yasumoto, Wolfgang Minker · 2018 · IEEE International Conference on Smart Computing
  33. Contextual Dependencies in Time-Continuous Multidimensional Affect Recognition
    Dmitrii Fedotov, Denis Ivanko, Sidorov Maxim Yuryevich, Wolfgang Minker · 2018 · Language Resources and Evaluation Conference (LREC)
  34. Predicting the Arousal and Valence Values of Emotional States Using Learned, Predesigned, and Deep Visual Features
    Itaf Omar Joudeh, Ana-Maria Creţu, Stéphane Bouchard · 2024 · Sensors
  35. Understanding the Impact of Emotions on the Quality of Software Artifacts
    Khaled M. Khan, Moutaz Saleh · 2021 · IEEE Access
  36. Strength Is in Numbers: Can Concordant Artificial Listeners Improve Prediction of Emotion from Speech?
    Eugenio Martinelli, Arianna Mencattini, Elena Daprati, Corrado Di Natale · 2016 · PLoS ONE
  37. Physiological Signals Fusion Oriented to Diagnosis - A Review
    Y. F. Uribe, Karla C. Álvarez-Uribe, Diego Hernán Peluffo-Ordóñez, Miguel A. Becerra · 2018 · Communications in computer and information science
  38. Prediction of Emotion Change From Speech
    Zhaocheng Huang, Julien Epps · 2018 · Frontiers in ICT
  39. A Systematic Review of Experimental Protocols: Towards a Uniform Framework in Virtual Reality Affective Research
    Allison Bayro, Heejin Jeong · 2025 · IEEE Transactions on Affective Computing
  40. A Multi-Scale Multi-Task Learning Model for Continuous Dimensional Emotion Recognition from Audio
    Xia Li, Guanming Lu, Jingjie Yan, Zhengyan Zhang · 2022 · Electronics
  41. Ensemble Learning to Assess Dynamics of Affective Experience Ratings and Physiological Change
    Felix Dollack, Kiyoshi Kiyokawa, Huakun Liu, Monica Perusquía-Hernández, Chirag Raman, Hideaki Uchiyama, Xin Wei · 2023 · International Conference on Affective Computing and Intelligent Interaction Workshops and Demos (ACIIW)
  42. Hybrid models for opinion analysis in speech interactions
    Valentin Barrière · 2017 · INTERNATIONAL CONFERENCE ON MULTIMODAL INTERACTION
  43. Generating labels for regression of subjective constructs using triplet embeddings
    Karel Mundnich, Brandon M. Booth, Benjamin Girault, Shrikanth S. Narayanan · 2019 · Pattern Recognition Letters
  44. Multimodal Classification Algorithms for Emotional Stress Analysis with an ECG-Centered Framework: A Comprehensive Review
    Xinyang Zhang, Hai-Min Zhang, Min Xu · 2026 · Applied Informatics
  45. Multimodal Affective Analysis combining Regularized Linear Regression and Boosted Regression Trees
    Aleksandar Milchevski, Alessandro Rozza, Dimitar Taškovski · 2015 · International Workshop on Audio/Visual Emotion Challenge (AVEC)
  46. Tracking the Expression of Annoyance in Call Centers
    Jon Irastorza, María Inés Torres · 2018 · Topics in intelligent engineering and informatics
  47. Continuous Monitoring of Emotions by a Multimodal Cooperative Sensor System
    Arianna Mencattini, Fabien Ringeval, Björn Wolfgang Schuller, Eugenio Martinelli, Corrado Di Natale · 2015 · Procedia Engineering
  48. Automatic speech discrete labels to dimensional emotional values conversion method
    Shaoling Jing, Xia Mao, Lijiang Chen · 2018 · IET Biometrics
  49. Continuous valence prediction using recurrent neural networks with facial expressions and EEG signals
    Dogancan Sen, Mustafa Sert · 2018 · Signal Processing and Communications Applications Conference (SIU)
  50. Emotion Classification Using Advanced Machine Learning Techniques Applied to Wearable Physiological Signals Data
    Bahareh Nakisa · 2019 · Queensland University of Technology
  51. Estimation of affective dimensions using CNN-based features of audiovisual data
    Ramesh Basnet, Mohammad Tariqul Islam, Tamanna Howlader, S. M. Mahbubur Rahman, Dimitrios Hatzinakos · 2019 · Pattern Recognition Letters
  52. User Emotion Recognition Method Based on Facial Expression and Speech Signal Fusion
    Fei Lu, Long Zhang, Guohui Tian · 2021 · IEEE Conference on Industrial Electronics and Applications (ICIEA)
  53. An Open Dataset for Impression Recognition from Multimodal Bodily Responses
    Chen Wang, Guillaume Chanel · 2021 · International Conference on Affective Computing and Intelligent Interaction and workshops
  54. Multivariate Output-Associative Rvm For Multi-Dimensional Affect Predictions
    Achut Manandhar, Kenneth D. Morton, Peter A. Torrione, Leslie M. Collins · 2016 · Zenodo (CERN European Organization for Nuclear Research)
  55. Tracking Authentic and In-the-wild Emotions Using Speech
    Vedhas Pandit, Nicholas Cummins, Maximilian Schmitt, Simone Hantke, Franz Graf, Lucas Paletta, Björn Wolfgang Schuller · 2018 · Asian Conference on Affective Computing and Intelligent Interaction (ACII Asia)
  56. Performance Analysis of Unimodal and Multimodal Models in Valence-Based Empathy Recognition
    Adria Mallol-Ragolta, Maximilian Schmitt, Alice Baird, Nicholas Cummins, Björn Wolfgang Schuller · 2019 · International Conference on Automatic Face and Gesture Recognition
  57. Speech, Head, and Eye-based Cues for Continuous Affect Prediction
    Jonny O’Dwyer · 2019 · International Conference on Affective Computing and Intelligent Interaction Workshops and Demos (ACIIW)
  58. Prediction of Sports Aggression Behavior and Analysis of Sports Intervention Based on Swarm Intelligence Model
    Huijian Deng, Cao Shi-jian, Jingen Tang · 2022 · Scientific Programming
  59. Adversarial-based neural networks for affect estimations in the wild
    Decky Aspandi, Adria Mallol-Ragolta, Björn Wolfgang Schuller, Xavier Binefa · 2020 · arXiv (Cornell University)
  60. Combining feature selection and representation for speech emotion recognition
    Wenjing Han, Huabin Ruan, Xiaojie Yu, Xuan Zhu · 2016 · IEEE International Conference on Multimedia & Expo Workshops (ICMEW)
  61. Context-Awared Models in Time-Continuous Multidimensional Affect Recognition
    Dmitrii Fedotov, Sidorov Maxim Yuryevich, Wolfgang Minker · 2017 · Lecture notes in computer science
  62. Multimodal Emotion Recognition using Deep Neural Network- A Survey
    Haritha C. V, Pillai Praveen Thulasidharan · 2018 · International Journal of Computer Sciences and Engineering
  63. Mobile Game Evaluation Method Based on Data Mining of Affective Time Series
    Jeremi K. Ochab, Paweł Wȩgrzyn, Przemysław Witaszczyk, Dominika Drążyk, Grzegorz Jacek Nalepa · 2025 · Sensors
  64. Mutual information-based selection of audiovisual affective features to predict instantaneous emotional state
    Sudipta Paul, Nurani Saoda, S. M. Mahbubur Rahman, Dimitrios Hatzinakos · 2016 · International Conference on Computer and Information Technology (ICCIT)
  65. AST-Net: An Attribute-based Siamese Temporal Network for Real-Time Emotion Recognition
    Shuhui Wang, Chiou-Ting Hsu · 2017 · British Machine Vision Conference (BMVC)
  66. SUNVE: Distributed Message Middleware towards Heterogeneous Database Synchronization
    Xiao Ying Wu, Hui Wang, Ying Xiong, Qiang Qian, Gaochao Li, Xin Jin, Tao Qin, Lidong Wang · 2018 · IEEE International Conference on Advanced Infocomm Technology
  67. FI-Net: A Speech Emotion Recognition Framework with Feature Integration and Data Augmentation
    Guangmin Xia, Fan Li, Dongdi Zhao, Qian Zhang, Song Yang · 2019 · International Conference on Big Data Computing and Communications (BigCom)
  68. Estimating User Satisfaction Impact in Cities Using Physical Reaction Sensing and Multimodal Dialogue System
    Yuki Matsuda, Dmitrii Fedotov, Yuta Takahashi, Yutaka Arakawa, Keiichi Yasumoto, Wolfgang Minker · 2019 · Lecture notes in electrical engineering
  69. Stock Price Prediction Using Deep Learning
    Ping Liu · 2022 · Applied economics and policy studies
  70. Recent Advances in Nonlinear Speech Processing: Directions and Challenges
    Anna Aurelia Esposito, Marcos Faúndez-Zanuy, Antonietta M. Esposito, Gennaro Cordasco, Thomas Drugman A, Jordi Solé‐Casals, Francesco Carlo Morabito · 2016 · Smart innovation, systems and technologies
  71. Emotion Modelling via Speech Content and Prosody: In Computer Games and Elsewhere
    Björn Wolfgang Schuller · 2016 · Socio-affective computing
  72. Bio-acoustic emotion recognition using continuous conditional recurrent neural fields
    Ntombikayise Banda, Lang He, Andries Petrus Engelbrecht · 2017 · IEEE Symposium Series on Computational Intelligence (SSCI)
  73. Time-Continuous Emotion Recognition Using Spectrogram Based CNN-RNN Modelling
    Dmitrii Fedotov, Bobae Kim, Alexey Anatolievich Karpov, Wolfgang Minker · 2019 · Lecture notes in computer science
  74. Cross-Modal Diversity-Based Active Learning for Multi-Modal Emotion Estimation
    Yifan Xu, Lubin Meng, Ruimin Peng, Yingjie Yin, Jingting Ding, Liang Li, Dongrui Wu · 2023 · International Joint Conference on Neural Networks (IJCNN)
  75. Optimizable Ensemble Regression for Arousal and Valence Predictions from Visual Features
    Itaf Omar Joudeh, Ana-Maria Creţu, Stéphane Bouchard · 2023 · International Electronic Conference on Sensors and Applications (ECSA)
  76. A Combined Reinforcement Regression Model Based on Weighted Feedback for Multimodal Emotion Recognition
    Ge Zhang, Gang Yang, Shengli Qu, Tianxiang Luo, Xiaoyu Han · 2019 · IEEE Conference on Industrial Electronics and Applications (ICIEA)
  77. Personalized physiological-based emotion recognition and implementation on hardware
    Wenlu Yang · 2018
  78. Annotations from speech and heart rate: impact on multimodal emotion recognition
    Kaushal Sharma, Guillaume Chanel · 2023 · INTERNATIONAL CONFERENCE ON MULTIMODAL INTERACTION
  79. A Review of Machine Learning-Based Assessment of Depression
    Zhao Wang, Ziyi Cai, Shuya Dong, Weihe Hei · 2025 · Communications in computer and information science
  80. A review study of human-affection knowledge on usability engineering
    D. Lakshmi, Ramalingam Ponnusamy · 2016 · International Conference on Advances in Human Machine Interaction (HMI)
  81. Convolutional Neural Network for Identifying Human Emotions with Different Head Poses
    Wafa Mellouk, Wahida Handouzi · 2021 · Lecture notes in networks and systems
  82. Multi-Rater Consensus Learning for Modeling Multiple Sparse Ratings of Affective Behaviour
    Luca Romeo, Temitayo Olugbade, Massimiliano Pontil, Nadia Bianchi‐Berthouze · 2023 · IEEE Transactions on Affective Computing
  83. Speech-based affective computing using attention with multimodal fusion
    Yue Gu · 2020
  84. A Comprehensive Investigation on Emotional Detection in Deep Learning
    Mamta Anand, S. Babu · 2022 · International Journal of Scientific Research in Computer Science Engineering and Information Technology
  85. Hybrid approaches for context recognition in Ambient Assisted Living systems : application to emotion recognition and human activity recognition and anticipation
    Hazem Abdelkawy · 2021 · theses.fr (ABES)
  86. Decoupling Temporal Dynamics for Naturalistic Affect Recognition in a Two-Stage Regression Framework
    Yona Falinie A. Gaus, Hongying Meng, Asim Jan · 2017 · IEEE International Conference on Cybernetics (CYBCONF)
  87. Linear and Non-Linear Multimodal Fusion for Continuous Affect Estimation In-the-Wild
    Yona Falinie A. Gaus, Hongying Meng · 2018 · International Conference on Automatic Face and Gesture Recognition
  88. Hyper-Parameter Optimization for Emotion Detection using Physiological Signals
    Amani Abdulrahman Albraikan, Diana P. Tobón, Abdulmotaleb El Saddik · 2018 · IEEE International Conference on Pervasive Computing and Communications Workshops (PerCom Workshops)
  89. Decision Rule Ensemble Formation Via a Multicriteria Evolutionary Algorithm for the Problem of Human Emotion Analysis in Audio Data
    A S Polyakova, L. V. Lipinskiy · 2019 · Herald of the Bauman Moscow State Technical University Series Instrument Engineering
  90. Deep audio and video emotion detection
    Masih Aminbeidokhti · 2020 · Espace ÉTS (ETS)
  91. Improving the Generalizability of Speech Emotion Recognition: Methods for Handling Data and Label Variability
    Biqiao Zhang · 2018 · Deep Blue (University of Michigan)
  92. DeepTMH: Multimodal Semi-supervised framework leveraging Affective and Cognitive engagement for Telemental Health.
    Pooja Guhan, Naman Awasthi, Ritwika Das, Manas Agarwal, Kathryn McDonald, Kristin Bussell, Dinesh Manocha, Gloria M. Reeves, Aniket Bera · 2021 · arXiv (Cornell University)
  93. Reconocimiento multimodal del estado emocional de un niño en un contexto educativo
    Javier Alcalá Vásquez · 2020
  94. Artificial Intelligence in Airway Management
    Ming Xia · 2023 · Springer eBooks
  95. Review on Facial Emotion Recognition Using Deep Learning With Multiple Databases
    Mal Hari Prasad, P. Swarnalatha, Kaneswaran Anantharajah, S. Prabu · 2024 · Advances in medical technologies and clinical practice book series
  96. Enhancing Empathic Accuracy: Penalized Functional Alignment Method to Correct Temporal Misalignment in Real-Time Emotional Perception
    Linh H. Nghiem, Jing Cao, Chrystyna D. Kouros, Chul Moon · 2025 · Psychometrika
  97. Voice and Face-Based Mood Recognition for Personalized Recommendations
    A.M.N.P Jayasekara, H.W.C. Ganishka, A.A.I. Udani, H.M.I. Udayangani, Shalini Upeksha, Indika Karunaratne · 2025 · International Conference on Information Technology Research (ICITR)
  98. Learning Sleep Apnea Patterns from Raw ECG via Convolutional–Recurrent Architectures
    Nija Asif, Haseeb Amjad, Ahmad Tariq, Umar Khan, Hassan Elahi · 2025 · International Conference on Information & Communication Technology and System (ICTS)
  99. First seconds matter : Mangaing first impressions for a more engaging virtual agent
    Béatrice Biancardi · 2019
  100. KTU-MEDAFE: A Newly Developed Multimodal Dataset for Emotion Recognition Using EEG–Speech Decision-Level Fusion
    Bahar Hatipoğlu Yılmaz, Betul Mumcu, Busra Ozkellekci · 2026 · Sensors