A Survey of Affect Recognition Methods: Audio, Visual, and Spontaneous Expressions
Abstract
Automated analysis of human affective behavior has attracted increasing attention from researchers in psychology, computer science, linguistics, neuroscience, and related disciplines. However, the existing methods typically handle only deliberately displayed and exaggerated expressions of prototypical emotions despite the fact that deliberate behaviour differs in visual appearance, audio profile, and timing from spontaneously occurring behaviour. To address this problem, efforts to develop algorithms that can process naturally occurring human affective behaviour have recently emerged. Moreover, an increasing number of efforts are reported toward multimodal fusion for human affect analysis including audiovisual fusion, linguistic and paralinguistic fusion, and multi-cue visual fusion based on facial expressions, head movements, and body gestures. This paper introduces and surveys these recent advances. We first discuss human emotion perception from a psychological perspective. Next we examine available approaches to solving the problem of machine understanding of human affective behavior, and discuss important issues like the collection and availability of training and test data. We finally outline some of the scientific and engineering challenges to advancing human affect sensing technology.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Linking two modalities into a single system for affective computing analysis is not new and has been studied before.
“Linking two modalities into a single system for affective computing analysis is not new and has been studied before (Chen, 2000; Zeng, Pantic, Roisman, & Huang, 2009).”
From Towards multimodal emotion recognition in e-learning environments (2016) · cites this paper as Zeng, Pantic, Roisman, & Huang, 2009
It is generally accepted in the literature that multimodal sensor combinations have complementary effects and thus may increase model accuracy.
“It is generally accepted in the literature that multimodal (e.g., audiovisual) sensor combinations have complementary effects and thus may increase model accuracy [11].”
From Deep learning for human affect recognition: Insights and new developments (2021) · cites this paper as [11]
Cites
Development and validation of brief measures of positive and negative affect: The PANAS scales.
Watson, D., Clark, L. A., & Tellegen, A. (1988) · Journal of Personality and Social Psychology · doi:10.1037/0022-3514.54.6.1063 · Cited by 32,433 · see citing works
Picard, R. W. (1997) · MIT Press · doi:10.7551/mitpress/1140.001.0001 · Cited by 7,382 · see citing works
Evidence for a three-factor theory of emotions
Russell, J. A., & Mehrabian, A. (1977) · Journal of Research in Personality · doi:10.1016/0092-6566(77)90037-x · Cited by 1,940 · see citing works
Automatic analysis of facial expressions: the state of the art
Pantie, M., & Rothkrantz, L. J. M. (2000) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/34.895976 · Cited by 1,778 · see citing works
Strong evidence for universals in facial expressions: A reply to Russell's mistaken critique.
Ekman, P. (1994) · Psychological Bulletin · doi:10.1037//0033-2909.115.2.268 · Cited by 1,130 · see citing works
Analysis of emotion recognition using facial expressions, speech and multimodal information
Busso, C., Deng, Z., Yildirim, S., Bulut, M., Lee, C. M., Kazemzadeh, A., Lee, S., Neumann, U., & Narayanan, S. (2004) · Proceedings of the 6th international conference on Multimodal interfaces · doi:10.1145/1027933.1027968 · Cited by 840 · see citing works
Emotions and speech: some acoustical correlates.
Williams, C. E., & Stevens, K. N. (1972) · The Journal of the Acoustical Society of America · doi:10.1121/1.1913238 · Cited by 735 · see citing works
Signs of appeasement: Evidence for the distinct displays of embarrassment, amusement, and shame.
Keltner, D. (1995) · Journal of Personality and Social Psychology · doi:10.1037/0022-3514.68.3.441 · Cited by 718 · see citing works
Handbook of Emotion Elicitation and Assessment
(2007) · doi:10.1093/oso/9780195169157.001.0001 · Cited by 348 · see citing works
Affective multimodal human-computer interaction
Pantic, M., Sebe, N., Cohn, J. F., & Huang, T. (2005) · Proceedings of the 13th annual ACM international conference on Multimedia · doi:10.1145/1101149.1101299 · Cited by 234 · see citing works
Self-citationAffect Recognition from Face and Body: Early Fusion vs. Late Fusion
Gunes, H., & Piccardi, M. (2006) · 2005 IEEE International Conference on Systems, Man and Cybernetics · doi:10.1109/icsmc.2005.1571679 · Cited by 191 · see citing works
Multimodal human emotion/expression recognition
Chen, L. S., Huang, T. S., Miyasato, T., & Nakatsu, R. (2002) · Proceedings Third IEEE International Conference on Automatic Face and Gesture Recognition · doi:10.1109/afgr.1998.670976 · Cited by 183 · see citing works
Self-citationSpontaneous vs. posed facial behavior: automatic analysis of brow actions
Valstar, M. F., Pantic, M., Ambadar, Z., & Cohn, J. F. (2006) · Proceedings of the 8th international conference on Multimodal interfaces · doi:10.1145/1180995.1181031 · Cited by 168 · see citing works
A survey of affect recognition methods: audio, visual and spontaneous expressions
Zeng, Z., Pantic, M., Roisman, G. I., & Huang, T. S. (2007) · Proceedings of the 9th international conference on Multimodal interfaces · doi:10.1145/1322192.1322216 · Cited by 144 · see citing works
Self-citation
Flagged first, then by citations
Cited by
Sorted by citations
Multimodal Machine Learning: A Survey and Taxonomy
Baltrusaitis, T., Ahuja, C., & Morency, L.-P. (2019) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2018.2798607 · Cited by 5,018 · see citing works
Lucey, P., Cohn, J. F., Kanade, T., Saragih, J., Ambadar, Z., & Matthews, I. (2010) · 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition - Workshops · doi:10.1109/cvprw.2010.5543262 · Cited by 4,303 · see citing works
A Multimodal Database for Affect Recognition and Implicit Tagging
Soleymani, M., Lichtenauer, J., Pun, T., & Pantic, M. (2012) · IEEE Transactions on Affective Computing · doi:10.1109/t-affc.2011.25 · Cited by 1,664 · see citing works
Self-citationA review of affective computing: From unimodal analysis to multimodal fusion
Poria, S., Cambria, E., Bajpai, R., & Hussain, A. (2017) · Information Fusion · doi:10.1016/j.inffus.2017.02.003 · Cited by 1,644 · see citing works
The INTERSPEECH 2009 emotion challenge
Schuller, B., Steidl, S., & Batliner, A. (2009) · Interspeech 2009 · doi:10.21437/interspeech.2009-103 · Cited by 938 · see citing works
Social signal processing: Survey of an emerging domain
Vinciarelli, A., Pantic, M., & Bourlard, H. (2009) · Image and Vision Computing · doi:10.1016/j.imavis.2008.11.007 · Cited by 842 · see citing works
Self-citationBP4D-Spontaneous: a high-resolution spontaneous 3D dynamic facial expression database
Zhang, X., Yin, L., Cohn, J. F., Canavan, S., Reale, M., Horowitz, A., Liu, P., & Girard, J. M. (2014) · Image and Vision Computing · doi:10.1016/j.imavis.2014.06.002 · Cited by 745 · see citing works
Mao, Q., Dong, M., Huang, Z., & Zhan, Y. (2014) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2014.2360798 · Cited by 568 · see citing works
Corneanu, C. A., Simon, M. O., Cohn, J. F., & Guerrero, S. E. (2016) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2016.2515606 · Cited by 527 · see citing works
Static facial expression analysis in tough conditions: Data, evaluation protocol and benchmark
Dhall, A., Goecke, R., Lucey, S., & Gedeon, T. (2011) · 2011 IEEE International Conference on Computer Vision Workshops (ICCV Workshops) · doi:10.1109/iccvw.2011.6130508 · Cited by 541 · see citing works
Towards multimodal sentiment analysis: harvesting opinions from the web
Morency, L.-P., Mihalcea, R., & Doshi, P. (2011) · Proceedings of the 13th international conference on multimodal interfaces · doi:10.1145/2070481.2070509 · Cited by 470 · see citing works
MoodScope: building a mood sensor from smartphone usage patterns
LiKamWa, R., Liu, Y., Lane, N. D., & Zhong, L. (2013) · Proceeding of the 11th annual international conference on Mobile systems, applications, and services · doi:10.1145/2462456.2464449 · Cited by 444 · see citing works
OpenEAR — Introducing the munich open-source emotion and affect recognition toolkit
Eyben, F., Wollmer, M., & Schuller, B. (2009) · 2009 3rd International Conference on Affective Computing and Intelligent Interaction and Workshops · doi:10.1109/acii.2009.5349350 · Cited by 418 · see citing works
Recognition of emotions using multimodal physiological signals and an ensemble deep learning model
Yin, Z., Zhao, M., Wang, Y., Yang, J., & Zhang, J. (2017) · Computer Methods and Programs in Biomedicine · doi:10.1016/j.cmpb.2016.12.005 · Cited by 416 · see citing works
Youtube movie reviews: Sentiment analysis in an audio-visual context
Wollmer, M., Weninger, F., Knaup, T., Schuller, B., Sun, C., Sagae, K., & Morency, L.-P. (2013) · IEEE Intelligent Systems · doi:10.1109/mis.2013.34 · Cited by 403 · see citing works
The first facial expression recognition and analysis challenge
Valstar, M. F., Jiang, B., Mehu, M., Pantic, M., & Scherer, K. (2011) · Face and Gesture 2011 · doi:10.1109/fg.2011.5771374 · Cited by 385 · see citing works
Self-citationGunes, H., & Schuller, B. (2013) · Image and Vision Computing · doi:10.1016/j.imavis.2012.06.016 · Cited by 360 · see citing works
Static and dynamic 3D facial expression recognition: A comprehensive survey
Sandbach, G., Zafeiriou, S., Pantic, M., & Yin, L. (2012) · Image and Vision Computing · doi:10.1016/j.imavis.2012.06.005 · Cited by 357 · see citing works
Self-citationLearning Affective Features With a Hybrid Deep Model for Audio–Visual Emotion Recognition
Zhang, S., Zhang, S., Huang, T., Gao, W., & Tian, Q. (2018) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2017.2719043 · Cited by 346 · see citing works
Self-citationThe influence of empathy in human-robot relations
Leite, I., Pereira, A., Mascarenhas, S., Martinho, C., Prada, R., & Paiva, A. (2013) · International Journal of Human-Computer Studies · doi:10.1016/j.ijhcs.2012.09.005 · Cited by 344 · see citing works
Universals and cultural variations in 22 emotional expressions across five cultures.
Cordaro, D. T., Sun, R., Keltner, D., Kamble, S., Huddar, N., & McNeil, G. (2018) · Emotion · doi:10.1037/emo0000302 · Cited by 343 · see citing works
Zhi, R., Flierl, M., Ruan, Q., & Kleijn, W. B. (2011) · IEEE Transactions on Systems Man and Cybernetics Part B (Cybernetics) · doi:10.1109/tsmcb.2010.2044788 · Cited by 260 · see citing works
Reading people’s minds from emotion expressions in interdependent decision making.
de Melo, C. M., Carnevale, P. J., Read, S. J., & Gratch, J. (2014) · Journal of Personality and Social Psychology · doi:10.1037/a0034251 · Cited by 260 · see citing works
FERA 2015 - second Facial Expression Recognition and Analysis challenge
Valstar, M. F., Almaev, T., Girard, J. M., McKeown, G., Mehu, M., Yin, L., Pantic, M., & Cohn, J. F. (2015) · 2015 11th IEEE International Conference and Workshops on Automatic Face and Gesture Recognition (FG) · doi:10.1109/fg.2015.7284874 · Cited by 252 · see citing works
Nonverbal social withdrawal in depression: Evidence from manual and automatic analyses
Girard, J. M., Cohn, J. F., Mahoor, M. H., Mavadati, S. M., Hammal, Z., & Rosenwald, D. P. (2014) · Image and Vision Computing · doi:10.1016/j.imavis.2013.12.007 · Cited by 255 · see citing works
Facial Expression Recognition Using Facial Movement Features
Zhang, L., & Tjondronegoro, D. (2011) · IEEE Transactions on Affective Computing · doi:10.1109/t-affc.2011.13 · Cited by 253 · see citing works
Automatic analysis of affective postures and body motion to detect engagement with a game companion,
Sanghvi, J., Castellano, G., Leite, I., Pereira, A., McOwan, P. W., & Paiva, A. (2011) · Proceedings of the 6th international conference on Human-robot interaction · doi:10.1145/1957656.1957781 · Cited by 250 · see citing works
Sewa db: A rich database for audio-visual emotion and sentiment research in the wild
Kossaifi, J., Walecki, R., Panagakis, Y., Shen, J., Schmitt, M., Ringeval, F., Han, J., Pandit, V., Toisoul, A., Schuller, B., et al. (2021) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2019.2944808 · Cited by 243 · see citing works
Self-citationAFEW-VA database for valence and arousal estimation in-the-wild
Kossaifi, J., Tzimiropoulos, G., Todorovic, S., & Pantic, M. (2017) · Image and Vision Computing · doi:10.1016/j.imavis.2017.02.001 · Cited by 236 · see citing works
Self-citationDeep learning for human affect recognition: Insights and new developments
Rouast, P. V., Adam, M. T. P., & Chiong, R. (2021) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2018.2890471 · Cited by 243 · see citing works
Video-based emotion recognition in the wild using deep transfer learning and score fusion
Kaya, H., Gürpınar, F., & Salah, A. A. (2017) · Image and Vision Computing · doi:10.1016/j.imavis.2017.01.012 · Cited by 235 · see citing works
Building autonomous sensitive artificial listeners
Schroder, M., Bevacqua, E., Cowie, R., Eyben, F., Gunes, H., Heylen, D., ter Maat, M., McKeown, G., Pammi, S., Pantic, M., et al. (2012) · IEEE Transactions on Affective Computing · doi:10.1109/t-affc.2011.34 · Cited by 185 · see citing works
Learning Race from Face: A Survey
Fu, S., He, H., & Hou, Z.-G. (2014) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2014.2321570 · Cited by 178 · see citing works
Automatic Detection of Learning-Centered Affective States in the Wild
Bosch, N., D'Mello, S., Baker, R., Ocumpaugh, J., Shute, V., Ventura, M., Wang, L., & Zhao, W. (2015) · Proceedings of the 20th International Conference on Intelligent User Interfaces · doi:10.1145/2678025.2701397 · Cited by 178 · see citing works
Yannakakis, G. N. (2012) · Proceedings of the 9th conference on Computing Frontiers · doi:10.1145/2212908.2212954 · Cited by 179 · see citing works
Facial Sentiment Analysis Using AI Techniques: State-of-the-Art, Taxonomies, and Challenges
Patel, K., Mehta, D., Mistry, C., Gupta, R., Tanwar, S., Kumar, N., & Alazab, M. (2020) · IEEE Access · doi:10.1109/access.2020.2993803 · Cited by 157 · see citing works
Pan, B., Hirota, K., Jia, Z., & Dai, Y. (2023) · Neurocomputing · doi:10.1016/j.neucom.2023.126866 · Cited by 142 · see citing works
Towards multimodal emotion recognition in e-learning environments
Bahreini, K., Nadolski, R., & Westera, W. (2016) · Interactive Learning Environments · doi:10.1080/10494820.2014.908927 · Cited by 137 · see citing works
Recognizing Emotions in Video Using Multimodal DNN Feature Fusion
Williams, J., Kleinegesse, S., Comanescu, R., & Radu, O. (2018) · Proceedings of Grand Challenge and Workshop on Human Multimodal Language (Challenge-HML) · doi:10.18653/v1/w18-3302 · Cited by 132 · see citing works
Facial expression recognition using Gabor motion energy filters
Wu, T., Bartlett, M. S., & Movellan, J. R. (2010) · 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition - Workshops · doi:10.1109/cvprw.2010.5543267 · Cited by 126 · see citing works
Smiling AI agents: How anthropomorphism and broad smiles increase charitable giving
Baek, T. H., Bakpayev, M., Yoon, S., & Kim, S. (2022) · International Journal of Advertising · doi:10.1080/02650487.2021.2011654 · Cited by 125 · see citing works
Error weighted semi-coupled hidden Markov model for audio-visual emotion recognition
Lin, J.-C., Wu, C.-H., & Wei, W.-L. (2012) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2011.2171334 · Cited by 121 · see citing works
Multiple kernel learning for emotion recognition in the wild
Sikka, K., Dykstra, K., Sathyanarayana, S., Littlewort, G., & Bartlett, M. (2013) · Proceedings of the 15th ACM on International conference on multimodal interaction · doi:10.1145/2522848.2531741 · Cited by 119 · see citing works
HoloNet: towards robust emotion recognition in the wild
Yao, A., Cai, D., Hu, P., Wang, S., Sha, L., & Chen, Y. (2016) · Proceedings of the 18th ACM International Conference on Multimodal Interaction · doi:10.1145/2993148.2997639 · Cited by 115 · see citing works
A hybrid latent space data fusion method for multimodal emotion recognition
Nemati, S., Rohani, R., Basiri, M. E., Abdar, M., Yen, N. Y., & Makarenkov, V. (2019) · IEEE Access · doi:10.1109/access.2019.2955637 · Cited by 98 · see citing works
The more the merrier: Analysing the affect of a group of people in images
Dhall, A., Joshi, J., Sikka, K., Goecke, R., & Sebe, N. (2015) · 2015 11th IEEE International Conference and Workshops on Automatic Face and Gesture Recognition (FG) · doi:10.1109/fg.2015.7163151 · Cited by 93 · see citing works
Khezri, M., Firoozabadi, M., & Sharafat, A. R. (2015) · Computer Methods and Programs in Biomedicine · doi:10.1016/j.cmpb.2015.07.006 · Cited by 89 · see citing works
Chen, S., Tian, Y., Liu, Q., & Metaxas, D. N. (2013) · Image and Vision Computing · doi:10.1016/j.imavis.2012.06.014 · Cited by 72 · see citing works
Expression recognition from 3D dynamic faces using robust spatio-temporal shape features
Le, V., Tang, H., & Huang, T. S. (2011) · Face and Gesture 2011 · doi:10.1109/fg.2011.5771435 · Cited by 67 · see citing works
Self-citationApplying multiple classifiers and non-linear dynamics features for detecting sleepiness from speech
Krajewski, J., Schnieder, S., Sommer, D., Batliner, A., & Schuller, B. (2012) · Neurocomputing · doi:10.1016/j.neucom.2011.12.021 · Cited by 67 · see citing works
Walecki, R., Rudovic, O., Pavlovic, V., & Pantic, M. (2015) · 2015 11th IEEE International Conference and Workshops on Automatic Face and Gesture Recognition (FG) · doi:10.1109/fg.2015.7163137 · Cited by 65 · see citing works
Self-citationCaroppo, A., Leone, A., & Siciliano, P. (2020) · Journal of Computer Science and Technology · doi:10.1007/s11390-020-9665-4 · Cited by 60 · see citing works
Temporal Bayesian Fusion for Affect Sensing: Combining Video, Audio, and Lexical Modalities
Savran, A., Cao, H., Nenkova, A., & Verma, R. (2015) · IEEE Transactions on Cybernetics · doi:10.1109/tcyb.2014.2362101 · Cited by 53 · see citing works
Automatic measurement of ad preferences from facial responses gathered over the Internet
McDuff, D., El Kaliouby, R., Senechal, T., Demirdjian, D., & Picard, R. (2014) · Image and Vision Computing · doi:10.1016/j.imavis.2014.01.004 · Cited by 52 · see citing works
Emotion recognition from thermal infrared images using deep Boltzmann machine
Wang, S., He, M., Gao, Z., He, S., & Ji, Q. (2014) · Frontiers of Computer Science · doi:10.1007/s11704-014-3295-3 · Cited by 51 · see citing works
Hagendorff, T. (2021) · Minds and Machines · doi:10.1007/s11023-021-09573-8 · Cited by 47 · see citing works
Estimating smile intensity: A better way
Girard, J. M., Cohn, J. F., & De la Torre, F. (2015) · Pattern Recognition Letters · doi:10.1016/j.patrec.2014.10.004 · Cited by 44 · see citing works
Human Emotion Recognition With Relational Region-Level Analysis
Li, W., Dong, X., & Wang, Y. (2023) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2021.3064918 · Cited by 43 · see citing works
Abadi, M. K., Correa, J. A. M., Wache, J., Yang, H., Patras, I., & Sebe, N. (2015) · 2015 11th IEEE International Conference and Workshops on Automatic Face and Gesture Recognition (FG) · doi:10.1109/fg.2015.7163100 · Cited by 36 · see citing works
Soladié, C., Salam, H., Pelachaud, C., Stoiber, N., & Séguier, R. (2012) · Proceedings of the 14th ACM international conference on Multimodal interaction · doi:10.1145/2388676.2388782 · Cited by 33 · see citing works
Virtual Affective Agents and Therapeutic Games
Hudlicka, E. (2016) · Artificial Intelligence in Behavioral and Mental Health Care · doi:10.1016/b978-0-12-420248-1.00004-0 · Cited by 34 · see citing works
UX Heatmaps: Mapping User Experience on Visual Interfaces
Georges, V., Courtemanche, F., Senecal, S., Baccino, T., Fredette, M., & Leger, P.-M. (2016) · Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems · doi:10.1145/2858036.2858271 · Cited by 29 · see citing works
Zhao, X., Dellandréa, E., Zou, J., & Chen, L. (2013) · Image and Vision Computing · doi:10.1016/j.imavis.2012.10.001 · Cited by 27 · see citing works
Perception of emotional expressions in different representations using facial feature points
Afzal, S., Sezgin, T. M., Gao, Y., & Robinson, P. (2009) · 2009 3rd International Conference on Affective Computing and Intelligent Interaction and Workshops · doi:10.1109/acii.2009.5349549 · Cited by 25 · see citing works
Dynamic facial expression recognition by joint static and multi-time gap transition classification
Dapogny, A., Bailly, K., & Dubuisson, S. (2015) · 2015 11th IEEE International Conference and Workshops on Automatic Face and Gesture Recognition (FG) · doi:10.1109/fg.2015.7163111 · Cited by 24 · see citing works
Identification of affective state change in adults with aphasia using speech acoustics
Gillespie, S., Laures-Gore, J., Moore, E., Farina, M., Russell, S., & Haaland, B. (2018) · Journal of Speech, Language, and Hearing Research · doi:10.1044/2018_jslhr-s-17-0057 · Cited by 20 · see citing works
Hierarchical fusion of visual and physiological signals for emotion recognition
Fang, Y., Rong, R., & Huang, J. (2021) · Multidimensional Systems and Signal Processing · doi:10.1007/s11045-021-00774-z · Cited by 17 · see citing works
Multimodal interaction, interfaces, and analytics
Oviatt, S. (2022) · Handbook of Human Computer Interaction · doi:10.1007/978-3-319-27648-9_22-1 · Cited by 12 · see citing works
Design of a hierarchy modular neural network and its application in multimodal emotion recognition
Li, W., Chu, M., & Qiao, J. (2019) · Soft Computing · doi:10.1007/s00500-018-03735-0 · Cited by 13 · see citing works
Towards Analyzing and Predicting the Experience of Live Performances with Wearable Sensing
Gedik, E., Cabrera-Quiros, L., Martella, C., Englebienne, G., & Hung, H. (2021) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2018.2875987 · Cited by 10 · see citing works
Inferring mental overload based on postural behavior and gestures
Hihn, H., Meudt, S., & Schwenker, F. (2016) · Proceedings of the 2nd workshop on Emotion Representations and Modelling for Companion Systems · doi:10.1145/3009960.3009961 · Cited by 8 · see citing works
Ji, M. (2024) · Poster Volume Ⅰ The 2024 Twentieth International Conference on Intelligent Computing August 5-8, 2024 Tianjin, China · doi:10.65286/icic.v20i2.94055
- Authors
- Zeng, Z., Pantic, M., Roisman, G. I., & Huang, T. S.
- Affiliations
- University of Illinois Urbana-Champaign (US)
- Venue
- IEEE Transactions on Pattern Analysis and Machine Intelligence
- Published
- 2009-01
- Volume
- 31
- Issue
- 1
- Pages
- 39-58
- ISSN
- 0162-8828 (ISSN-L)
- OpenAlex
- W2156503193
- Cited by
- 2,797
- References
- 14
- Citations by year
- 2026: 72 · 2025: 65 · 2024: 58 · 2023: 79 · 2022: 119 · 2021: 145 · 2020: 153 · 2019: 171 · 2018: 162 · 2017: 225 · 2016: 213 · 2015: 271 · 2014: 226 · 2013: 238 · 2012: 189
- FWCI
- 15.44
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- IEEE Transactions on Pattern Analysis and Machine Intelligence (publishedVersion)
- PubMed (publishedVersion)
- University of Twente Research Information (publishedVersion)
- University of Twente Research Information (submittedVersion)
- Topics
- Social Sciences › Psychology › Experimental and Cognitive Psychology › Emotion and Mood Recognition
- Physical Sciences › Computer Science › Artificial Intelligence › Sentiment Analysis and Opinion Mining
- Physical Sciences › Computer Science › Signal Processing › Speech and Audio Processing
- Keywords
- affect recognition, emotion recognition, audiovisual integration, multimodal fusion, paralinguistic cues, facial expressions, body gestures, head movement, training data collection
- MeSH terms
- Affect, Algorithms, Artificial Intelligence, Emotions, Facial Expression, Monitoring, Physiologic, Pattern Recognition, Automated, Sound Spectrography
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
