Temporal Bayesian Fusion for Affect Sensing: Combining Video, Audio, and Lexical Modalities
Abstract
The affective state of people changes in the course of conversations and these changes are expressed externally in a variety of channels, including facial expressions, voice, and spoken words. Recent advances in automatic sensing of affect, through cues in individual modalities, have been remarkable; yet emotion recognition is far from a solved problem. Recently, researchers have turned their attention to the problem of multimodal affect sensing in the hope that combining different information sources would provide great improvements. However, reported results fall short of the expectations, indicating only modest benefits and occasionally even degradation in performance. We develop temporal Bayesian fusion for continuous real-value estimation of valence, arousal, power, and expectancy dimensions of affect by combining video, audio, and lexical modalities. Our approach provides substantial gains in recognition performance compared to previous work. This is achieved by the use of a powerful temporal prediction model as prior in Bayesian fusion as well as by incorporating uncertainties about the unimodal predictions. The temporal prediction model makes use of time correlations on the affect sequences and employs estimated temporal biases to control the affect estimations at the beginning of conversations. In contrast to other recent methods for combination of modalities our model is simpler, since it does not model relationships between modalities and involves only a few interpretable parameters to be estimated from the training data.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Cites
Distribution of the Estimators for Autoregressive Time Series with a Unit Root
Dickey, D. A., & Fuller, W. A. (1979) · Journal of the American Statistical Association · doi:10.1080/01621459.1979.10482531 · Cited by 23,211 · see citing works
Sequential Monte Carlo Methods in Practice
Doucet, A. (2001) · Springer eBooks · doi:10.1007/978-1-4757-3437-9 · Cited by 7,445 · see citing works
Ekman, P., & Friesen, W. V. (2019) · PsycTESTS Dataset · doi:10.1037/t27734-000 · Cited by 6,019 · see citing works
A Survey of Affect Recognition Methods: Audio, Visual, and Spontaneous Expressions
Zeng, Z., Pantic, M., Roisman, G. I., & Huang, T. S. (2009) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2008.52 · Cited by 2,797 · see citing works
Opensmile: the munich versatile and fast open-source audio feature extractor
Eyben, F., Wöllmer, M., & Schuller, B. (2010) · Proceedings of the 18th ACM international conference on Multimedia · doi:10.1145/1873951.1874246 · Cited by 2,642 · see citing works
Multisensor data fusion: A review of the state-of-the-art
Khaleghi, B., Khamis, A. M., Karray, F. O., & Razavi, S. (2011) · Information Fusion · doi:10.1016/j.inffus.2011.08.001 · Cited by 2,097 · see citing works
Affect Detection: An Interdisciplinary Review of Models, Methods, and Their Applications
Calvo, R. A., & D'Mello, S. (2010) · IEEE Transactions on Affective Computing · doi:10.1109/t-affc.2010.1 · Cited by 1,893 · cited 1× · see citing works
“ent disciplines and sub-fields, each primarily interested in a given subset of modalities [3], [5], [6].”
BackgroundAutomatic facial expression analysis: a survey
Fasel, B., & Luettin, J. (2003) · Pattern Recognition · doi:10.1016/s0031-3203(02)00052-3 · Cited by 1,810 · see citing works
The world of emotions is not two-dimensional
Fontaine, J. R. J., Scherer, K. R., Roesch, E. B., & Ellsworth, P. C. (2007) · Psychological Science · doi:10.1111/j.1467-9280.2007.02024.x · Cited by 1,179 · see citing works
Emotional speech recognition: Resources, features, and methods
Ververidis, D., & Kotropoulos, C. L. (2006) · Speech Communication · doi:10.1016/j.specom.2006.04.003 · Cited by 930 · see citing works
Multimodal Emotion Recognition in Response to Videos
Soleymani, M., Pantić, M., & Pun, T. (2011) · IEEE Transactions on Affective Computing · doi:10.1109/t-affc.2011.37 · Cited by 725 · see citing works
McKeown, G., Valstar, M., Cowie, R., Pantic, M., & Schroder, M. (2012) · IEEE Transactions on Affective Computing · doi:10.1109/t-affc.2011.20 · Cited by 665 · see citing works
Toward detecting emotions in spoken dialogs
Lee, C. M., & Narayanan, S. S. (2005) · IEEE Transactions on Speech and Audio Processing · doi:10.1109/tsa.2004.838534 · Cited by 655 · see citing works
(2017) · The SAGE Encyclopedia of Communication Research Methods · doi:10.4135/9781483381411.n178 · Cited by 559 · see citing works
Automatic, Dimensional and Continuous Emotion Recognition
Gunes, H., & Pantic, M. (2010) · International Journal of Synthetic Emotions · doi:10.4018/jse.2010101605 · Cited by 466 · see citing works
Schuller, B. W., Rigoll, G., & Lang, M. (2004) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp.2004.1326051 · Cited by 429 · see citing works
Multimodal affect recognition in learning environments
Kapoor, A., & Picard, R. W. (2005) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/1101149.1101300 · Cited by 342 · see citing works
LSTM-Modeling of continuous emotions in an audiovisual affect recognition framework
Wöllmer, M., Kaiser, M., Eyben, F., Schuller, B., & Rigoll, G. (2013) · Image and Vision Computing · doi:10.1016/j.imavis.2012.03.001 · Cited by 288 · see citing works
Acoustic emotion recognition: A benchmark comparison of performances
Schuller, B. W., Vlasenko, B., Eyben, F., Rigoll, G., & Wendemuth, A. (2009) · IEEE Workshop on Automatic Speech Recognition and Understanding (ASRU) · doi:10.1109/asru.2009.5372886 · Cited by 275 · see citing works
AVEC 2011–The First International Audio/Visual Emotion Challenge
Schuller, B. W., Valstar, M., Eyben, F., McKeown, G. J., Cowie, R., & Pantić, M. (2011) · Lecture notes in computer science · doi:10.1007/978-3-642-24571-8_53 · Cited by 270 · see citing works
Wu, C., & Liang, W.-B. (2010) · IEEE Transactions on Affective Computing · doi:10.1109/t-affc.2010.16 · Cited by 265 · see citing works
Kessous, L., Castellano, G., & Caridakis, G. (2010) · Journal on Multimodal User Interfaces · doi:10.1007/s12193-009-0025-5 · Cited by 256 · see citing works
Schuller, B. W., Valstar, M., Eyben, F., Cowie, R., & Pantić, M. (2012) · INTERNATIONAL CONFERENCE ON MULTIMODAL INTERACTION · doi:10.1145/2388676.2388776 · Cited by 225 · see citing works
Automatic Temporal Segment Detection and Affect Recognition From Face and Body Display
Gunes, H., & Piccardi, M. (2009) · IEEE Transactions on Systems Man and Cybernetics Part B (Cybernetics) · doi:10.1109/tsmcb.2008.927269 · Cited by 225 · see citing works
Schuller, B. W., Müller, R., Lang, M. K., & Rigoll, G. (2005) · Interspeech (USB) · doi:10.21437/interspeech.2005-379 · Cited by 141 · see citing works
Regression-based intensity estimation of facial action units
Savran, A., Sankur, B., & Taha Bilge, M. (2012) · Image and Vision Computing · doi:10.1016/j.imavis.2011.11.008 · Cited by 129 · see citing works
Self-citationFacial Action Recognition Combining Heterogeneous Features via Multikernel Learning
Sénéchal, T., Rapp, V., Salam, H., Séguier, R., Bailly, K., & Prévost, L. (2012) · IEEE Transactions on Systems Man and Cybernetics Part B (Cybernetics) · doi:10.1109/tsmcb.2012.2193567 · Cited by 123 · see citing works
D'Mello, S. K., & Kory, J. (2012) · International Conference on Multimodal Interaction · doi:10.1145/2388676.2388686 · Cited by 122 · see citing works
Error weighted semi-coupled hidden Markov model for audio-visual emotion recognition
Lin, J.-C., Wu, C.-H., & Wei, W.-L. (2012) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2011.2171334 · Cited by 121 · see citing works
Robust continuous prediction of human emotions using multiscale dynamic cues
Nicolle, J., Rapp, V., Bailly, K., Prévost, L., & Chétouani, M. (2012) · INTERNATIONAL CONFERENCE ON MULTIMODAL INTERACTION · doi:10.1145/2388676.2388783 · Cited by 113 · see citing works
Comparative evaluation of 3D vs. 2D modality for automatic detection of facial action units
Savran, A., Sankur, B., & Taha Bilge, M. (2012) · Pattern Recognition · doi:10.1016/j.patcog.2011.07.022 · Cited by 111 · see citing works
Self-citationRobust Speech Recognition and Understanding
Grimm, M., & Kroschel, K. (2007) · doi:10.5772/35 · Cited by 98 · see citing works
Savran, A., Cao, H., Shah, M., Nenkova, A., & Verma, R. (2012) · INTERNATIONAL CONFERENCE ON MULTIMODAL INTERACTION · doi:10.1145/2388676.2388781 · Cited by 59 · see citing works
Self-citationString-based audiovisual fusion of behavioural events for the assessment of dimensional affect
Eyben, F., Wöllmer, M., Valstar, M., Güneş, H., Schuller, B. W., & Pantić, M. (2011) · International Conference on Automatic Face and Gesture Recognition · doi:10.1109/fg.2011.5771417 · Cited by 54 · see citing works
Soladié, C., Salam, H., Pelachaud, C., Stoiber, N., & Séguier, R. (2012) · Proceedings of the 14th ACM international conference on Multimodal interaction · doi:10.1145/2388676.2388782 · Cited by 33 · see citing works
Güneş, H., & Piccardi, M. (2005) · Lecture notes in computer science · doi:10.1007/11573548_14 · Cited by 32 · see citing works
Desperately Seeking Emotions Or: Actors, Wizards, And Human Beings
Batliner, A., Fischer, K., Huber, R., Spilker, J., & Nöth, E. (2000) · OPUS (Augsburg University) · Cited by 31 · see citing works
Step-wise emotion recognition using concatenated-HMM
Özkan, D., Scherer, S. J., & Morency, L. (2012) · INTERNATIONAL CONFERENCE ON MULTIMODAL INTERACTION · doi:10.1145/2388676.2388780 · Cited by 27 · see citing works
Balancing spoken content adaptation and unit length in the recognition of emotion and interest
Vlasenko, B., Schuller, B. W., Mengistu, K. T., Rigoll, G., & Wendemuth, A. (2008) · Interspeech (USB) · doi:10.21437/interspeech.2008-246 · Cited by 20 · see citing works
Automatic Detection of Emotion Valence on Faces Using Consumer Depth Cameras
Savran, A., Gur, R. C., & Verma, R. (2013) · IEEE International Conference on Computer Vision Workshops · doi:10.1109/iccvw.2013.17 · Cited by 15 · see citing works
Self-citationAcoustic and lexical representations for affect prediction in spontaneous conversations
Cao, H., Savran, A., Verma, R., & Nenkova, A. (2014) · Computer Speech & Language · doi:10.1016/j.csl.2014.04.002 · Cited by 8 · see citing works
Self-citation
Flagged first, then by citations
Cited by
Sorted by citations
Corneanu, C. A., Simon, M. O., Cohn, J. F., & Guerrero, S. E. (2016) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2016.2515606 · Cited by 527 · see citing works
Machine learning in the Internet of Things: Designed techniques for smart cities
Din, I. U., Guizani, M., Rodrigues, J. J. P. C., Hassan, S., & Korotaev, V. V. (2019) · Future Generation Computer Systems · doi:10.1016/j.future.2019.04.017 · Cited by 168 · see citing works
Kalateh, S., Estrada-Jimenez, L. A., Nikghadam-Hojjati, S., & Barata, J. (2024) · IEEE Access · doi:10.1109/access.2024.3430850 · Cited by 151 · see citing works
SPHA: Smart Personal Health Advisor Based on Deep Analytics
Chen, M., Zhang⋆, Y., Meikang Qiu, A., Guizani, N., & Hao, Y. (2018) · IEEE Communications Magazine · doi:10.1109/mcom.2018.1700274 · Cited by 136 · see citing works
Discriminative deep multi-task learning for facial expression recognition
Zheng, H., Wang, R., Ji, W., Zong, M., Wong, W. K., Lai, Z., & Lv, H. (2020) · Information Sciences · doi:10.1016/j.ins.2020.04.041 · Cited by 90 · see citing works
A Review on Facial Expression Recognition: Feature Extraction and Classification
Zhao, X., & Zhang, S. (2016) · IETE Technical Review · doi:10.1080/02564602.2015.1117403 · Cited by 75 · see citing works
Automatic Recognition of Facial Displays of Unfelt Emotions
Kulkarni, K., Corneanu, C. A., Ofodile, I., Escalera, S., Baró, X., Hyniewska, S., Allïk, J., & Anbarjafari, G. (2018) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2018.2874996 · Cited by 61 · see citing works
Deb, S., & Dandapat, S. (2019) · IEEE Transactions on Cybernetics · doi:10.1109/tcyb.2017.2787717 · Cited by 55 · see citing works
Dynamic Objectives Learning for Facial Expression Recognition
Wen, G., Chang, T., Li, H., & Jiang, L. (2020) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2020.2966858 · Cited by 53 · see citing works
Two-Stage Fuzzy Fusion Based-Convolution Neural Network for Dynamic Emotion Recognition
Wu, M., Su, W., Chen, L., Pedrycz, W., & Hirota, K. (2020) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2020.2966440 · Cited by 48 · see citing works
Lei, Y., & Cao, H. (2023) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2023.3234777 · Cited by 45 · see citing works
Self-citationAn Emergency Evacuation Behavior Simulation Method Combines Personality Traits and Emotion Contagion
Zhou, R., Ou, Y., Tang, W., Wang, Q., & Yu, B. C. (2020) · IEEE Access · doi:10.1109/access.2020.2985987 · Cited by 37 · see citing works
Liu, Y., Li, Y., Tan, X., Wang, P., & Zhang, Y. (2020) · Biomedical Signal Processing and Control · doi:10.1016/j.bspc.2020.102165 · Cited by 38 · see citing works
A Novel Facial Expression Intelligent Recognition Method Using Improved Convolutional Neural Network
Shi, M., Xu, L., & Chen, X. (2020) · IEEE Access · doi:10.1109/access.2020.2982286 · Cited by 35 · see citing works
Thirumuru, R., Gurugubelli, K., & Vuppala, A. K. (2021) · Digital Signal Processing · doi:10.1016/j.dsp.2021.103293 · Cited by 35 · see citing works
Facial Sentiment Classification Based on Resnet-18 Model
Zhou, Y., Ren, F., Nishide, S., & Kang, X. (2019) · International Conference on Electronic Engineering and Informatics (EEI) · doi:10.1109/eei48997.2019.00106 · Cited by 30 · see citing works
Faria, D. R., Godkin, A. L., & Ayrosa, P. P. S. (2025) · Sensors · doi:10.3390/s25041161 · Cited by 27 · see citing works
Zaman, K., Zengkang, G., Sun, Z., Shah, S. M., Riaz, W., Ji, J., Hussain, T., & Attar, R. W. (2025) · International Journal of Intelligent Systems · doi:10.1155/int/6611276 · Cited by 31 · see citing works
Rivas, J. J., Orihuela‐Espina, F., Palafox, L., Bianchi‐Berthouze, N., Lara, M. D. C., Hernández-Franco, J., & Sucar, L. E. (2018) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2018.2808295 · Cited by 27 · see citing works
Detection of physical stress using multispectral imaging
Hong, K., Liu, X., Liu, G., & Chen, W. (2018) · Neurocomputing · doi:10.1016/j.neucom.2018.10.011 · Cited by 25 · see citing works
Mishra, S., Warule, P., & Deb, S. (2023) · Speech Communication · doi:10.1016/j.specom.2023.102986 · Cited by 26 · see citing works
A Visual–Audio-Based Emotion Recognition System Integrating Dimensional Analysis
Tian, J., & She, Y. (2022) · IEEE Transactions on Computational Social Systems · doi:10.1109/tcss.2022.3200060 · Cited by 24 · see citing works
A Review of Affective Computing Research Based on Function-Component-Representation Framework
Ma, H., & Yarosh, S. (2021) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2021.3104512 · Cited by 23 · see citing works
Janssoone, T., Clavel, C., Bailly, K., & Gaël Richard, A. (2016) · Lecture notes in computer science · doi:10.1007/978-3-319-47665-0_16 · Cited by 20 · see citing works
Data fusion methods in multimodal human computer dialog
Yang, M., & Tao, J. (2019) · Virtual Reality & Intelligent Hardware · doi:10.3724/sp.j.2096-5796.2018.0010 · Cited by 20 · see citing works
Xu, X., Luo, C., Luo, X. R., & Wang, Z. (2024) · Information & Management · doi:10.1016/j.im.2024.104031 · Cited by 19 · see citing works
Audio Tracking in Noisy Environments by Acoustic Map and Spectral Signature
Crocco, M., Martelli, S., Trucco, A., Zunino, A., & Murino, V. (2017) · IEEE Transactions on Cybernetics · doi:10.1109/tcyb.2017.2711497 · Cited by 18 · see citing works
Local and correlation attention learning for subtle facial expression recognition
Wang, S., Yuan, Y., Zheng, X., & Lu, X. (2020) · Neurocomputing · doi:10.1016/j.neucom.2020.07.120 · Cited by 15 · see citing works
Emotion Recognition from Sensory and Bio-Signals: A Survey
Vora, K., Shah, S., Harsoda, H., Sheth, J., Agarwal, S., Thakkar, A., & Mankad, S. H. (2018) · Proceedings of the 2nd International Conference on Data Engineering and Communication Technology · doi:10.1007/978-981-13-1610-4_35 · Cited by 11 · see citing works
Intelligence methods of multi-modal information fusion in human-computer interaction
Yang, M., & Tao, J. (2018) · Scientia Sinica Informationis · doi:10.1360/n112017-00211 · Cited by 10 · see citing works
A Bayesian Filtering Framework for Continuous Affect Recognition From Facial Images
Pei, E., Zhao, Y. H., Oveneke, M. C., Jiang, D., & Hichem Sahli, B. (2022) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2022.3164248 · Cited by 9 · see citing works
Li, H., Han, H., Xu, C., Chen, T., Liu, X., & Wen, G. (2026) · Engineering Applications of Artificial Intelligence · doi:10.1016/j.engappai.2026.114127 · Cited by 8 · see citing works
Facial expression recognition sensing the complexity of testing samples
Chang, T., Li, H., Wen, G., Hu, Y., & Ma, J. (2019) · Applied Intelligence · doi:10.1007/s10489-019-01491-8 · Cited by 8 · see citing works
Continuous affect prediction using eye gaze and speech
O’Dwyer, J., Flynn, R., & Murray, N. (2017) · IEEE International Conference on Bioinformatics and Biomedicine (BIBM) · doi:10.1109/bibm.2017.8217968 · Cited by 8 · see citing works
An adaptive weighted fusion model with two subspaces for facial expression recognition
Sun, Z., Hu, Z., Chiong, R., Wang, M., & Zhao, S. (2018) · Signal Image and Video Processing · doi:10.1007/s11760-017-1226-0 · Cited by 8 · see citing works
Pei, E., Hu, Z., He, L., Ning, H., & Berenguer, A. D. (2023) · Expert Systems with Applications · doi:10.1016/j.eswa.2023.121290 · Cited by 8 · see citing works
Exploring Temporal Dependencies in Multimodal Referring Expressions with Mixed Reality
Sibirtseva, E., Ghadirzadeh, A., Leite, I., Björkman, M., & Kragić, D. (2019) · Lecture notes in computer science · doi:10.1007/978-3-030-21565-1_8 · Cited by 7 · see citing works
Multi-timescale boosting for efficient and improved event camera face pose alignment
Savran, A. (2023) · Computer Vision and Image Understanding · doi:10.1016/j.cviu.2023.103817 · Cited by 6 · see citing works
Self-citationTemporal Association Rules for Modelling Multimodal Social Signals
Janssoone, T. (2015) · INTERNATIONAL CONFERENCE ON MULTIMODAL INTERACTION · doi:10.1145/2818346.2823305 · Cited by 6 · see citing works
Maheswari, V. U., Aluvalu, R., Mudrakola, S., Chithaluru, P., & Kumar, M. (2025) · Multimedia Tools and Applications · doi:10.1007/s11042-025-20880-5 · Cited by 5 · see citing works
Emotion Classification using Dual-Tree Complex Wavelet Transform
Deb, S., & Dandapat, S. (2017) · IEEE India Conference · doi:10.1109/indicon.2017.8487854 · Cited by 5 · see citing works
Monitoring Smart Devices for Personal Health Care Using Deep Analytics
Tharageswari, K., Pandian, D. S., Raja, L., & Dhanapal, R. (2020) · International Conference on I-SMAC (IoT in Social, Mobile, Analytics and Cloud) (I-SMAC) · doi:10.1109/i-smac49090.2020.9243413 · Cited by 4 · see citing works
Wang, K., Feng, Z., Li, J., & Han, R. (2019) · IEEE Access · doi:10.1109/access.2019.2945330 · Cited by 4 · see citing works
Face recognition system using soft-output classifier fusion method
Toufiq, R., & Isalm, M. R. (2016) · International Conference on Electrical, Computer & Telecommunication Engineering (ICECTE) · doi:10.1109/icecte.2016.7879582 · Cited by 3 · see citing works
AST-Net: An Attribute-based Siamese Temporal Network for Real-Time Emotion Recognition
Wang, S., & Hsu, C.-T. (2017) · British Machine Vision Conference (BMVC) · doi:10.5244/c.31.70 · Cited by 3 · see citing works
Analyzing the Affect of a Group of People Using Multi-modal FrameworkPreprint
Huang, X., Dhall, A., Liu, X., Zhao, G., Shi, J., Goecke, R., & Pietikäinen, M. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1610.03640 · Cited by 3 · see citing works
An Intention Understanding Algorithm Based on Multimodal Information Fusion
Dou, S., Feng, Z., Tian, J., Fan, X., Hou, Y., & Zhang, X. (2021) · Scientific Programming · doi:10.1155/2021/8354015 · Cited by 2 · see citing works
Facial expression recognition based on anomaly feature
Hong, K. (2022) · Optical Review · doi:10.1007/s10043-022-00734-3 · Cited by 2 · see citing works
Internet of Things based Framework for Smart Healthcare Using Hybrid Machine LearningPreprint
Okemiri, H. A., Uzoma, A., & Nnamene, C. C. (2022) · Research Square · doi:10.21203/rs.3.rs-1428239/v1 · Cited by 2 · see citing works
Spatial–Spectral–Temporal Framework for Emotion Recognition
Hong, K. (2020) · IEEE Access · doi:10.1109/access.2020.3000066 · Cited by 1 · see citing works
Multimodal continuous emotion analysis
Zhang, S. (2023) · doi:10.32657/10356/166783
Vu, M. T., Vu, H. N., & Nguyen, H. L. (2026) · Pattern Recognition and Image Analysis · doi:10.1134/s1054661825701287
Dynamic graph interaction for feature-augmented multimodal sentiment analysis
Li, Z., Mao, Y., Jiang, M., Yu, J., Chen, H., Gao, Y., Huang, X., Lei, T., & Pan, Y. (2026) · Multimedia Systems · doi:10.1007/s00530-026-02703-1
- Authors
- Savran, A., Cao, H., Nenkova, A., & Verma, R.
- Affiliations
- University of Pennsylvania (US)
- Venue
- IEEE Transactions on Cybernetics
- Published
- 2015-09
- Volume
- 45
- Issue
- 9
- Pages
- 1927-1941
- ISSN
- 2168-2267 (ISSN-L)
- 2168-2275
- OpenAlex
- W2086655908
- Cited by
- 53
- References
- 41
- Citations by year
- 2026: 3 · 2025: 3 · 2024: 2 · 2023: 5 · 2022: 4 · 2021: 3 · 2020: 9 · 2019: 6 · 2018: 8 · 2017: 4 · 2016: 5 · 2015: 1
- FWCI
- 4.64
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- IEEE Transactions on Cybernetics (publishedVersion)
- PubMed (publishedVersion)
- Funders
- NIH Clinical Center (R01-MH-073174)
- Topics
- Social Sciences › Psychology › Experimental and Cognitive Psychology › Emotion and Mood Recognition
- Physical Sciences › Computer Science › Artificial Intelligence › Sentiment Analysis and Opinion Mining
- Physical Sciences › Computer Science › Signal Processing › Speech and Audio Processing
- Keywords
- affective computing, multimodal emotion recognition, Bayesian fusion, valence, arousal, power, expectancy, video, audio, lexical cues, uncertainty estimation, temporal bias
- MeSH terms
- Bayes Theorem, Emotions, Facial Expression, Humans, Pattern Recognition, Automated, Video Recording
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
