Cited by 234Emotion and Mood RecognitionSocial Robot Interaction and HRIInnovative Human-Technology Interaction

The DOI matches its Crossref record; 49 of the 50 references it cites were found in a registry or an index.

Affective multimodal human-computer interaction

Pantic, Maja, Sebe, Nicu, Cohn, Jeffrey F., Huang, Thomas·2005·Proceedings of the 13th annual ACM international conference on Multimedia

DOI 10.1145/1101149.1101299

Open this paper in CoChat — free account

Abstract

Social and emotional intelligence are aspects of human intelligence that have been argued to be better predictors than IQ for measuring aspects of success in life, especially in social interactions, learning, and adapting to what is important. When it comes to machines, not all of them will need such skills. Yet to have machines like computers, broadcast systems, and cars, capable of adapting to their users and of anticipating their wishes, endowing them with the ability to recognize user's affective states is necessary. This article discusses the components of human affect, how they might be integrated into computers, and how far are we from realizing affective multimodal human-computer interaction.

Integrity signals

Registry

Crossref · 10.1145/1101149.1101299

Cites

49 of 49 references verified0 citation errors · 0 not checkable

Statistics

Recomputing p-values needs the paper’s full text.

Claims the paper makes

This paper’s full text isn’t openly available, so its claims and self-check weren’t run. The registry and citation checks still apply.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Affective Computing
Picard, R. W. (1997) · MIT Press · doi:10.7551/mitpress/1140.001.0001 · Cited by 7,382
The Repertoire of Nonverbal Behavior: Categories, Origins, Usage, and Coding
EKMAN, P., & FRIESEN, W. V. (1969) · Semiotica · doi:10.1515/semi.1969.1.1.49 · Cited by 3,316
Comprehensive database for facial expression analysis
Takeo Kanade, A., Cohn, J. F., & Tian, Y. (2002) · International Conference on Automatic Face and Gesture Recognition · doi:10.1109/afgr.2000.840611 · Cited by 2,644
Self-citation
Toward machine emotional intelligence: analysis of affective physiological state
Picard, R. W., Vyzas, E., & Healey, J. (2001) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/34.954607 · Cited by 2,334
A Survey on Visual Surveillance of Object Motion and Behaviors
Hu, W., Tan, T. N., Wang, L., & Maybank, S. J. (2004) · IEEE Transactions on Systems Man and Cybernetics Part C (Applications and Reviews) · doi:10.1109/tsmcc.2004.829274 · Cited by 2,156
Thin slices of expressive behavior as predictors of interpersonal consequences: A meta-analysis.
Ambady, N., & Rosenthal, R. (1992) · Psychological Bulletin · doi:10.1037/0033-2909.111.2.256 · Cited by 2,003
Is there universal recognition of emotion from facial expression? A review of the cross-cultural studies.
Russell, J. A. (1994) · Psychological Bulletin · doi:10.1037/0033-2909.115.1.102 · Cited by 1,640
Acoustic profiles in vocal emotion expression
Banse, R., & Scherer, K. R. (1996) · Journal of Personality and Social Psychology · doi:10.1037/0022-3514.70.3.614 · Cited by 1,546
Testing a tripartite model: I. Evaluating the convergent and discriminant validity of anxiety and depression symptom scales.
Watson, D., Weber, K., Assenheimer, J. S., Clark, L. A., Strauss, M. E., & McCormick, R. A. (1995) · Journal of Abnormal Psychology · doi:10.1037/0021-843x.104.1.3 · Cited by 1,423
Web-Based Database for Facial Expression Analysis
Pantić, M., Valstar, M., Rademaker, R. M., & Maat, L. (2005) · IEEE International Conference on Multimedia and Expo (ICME) · doi:10.1109/icme.2005.1521424 · Cited by 1,124

Show all 50 references

Research works printed but not linked: 9

10.5555/795661.796218 · DOI
Garg , A. , Naphade , M. , Huang , T.S. Modeling video using input/output Markov models with application to multimodal event detection. Handbook of Video Databases: Design and Applications, B. Furth, O. Marques, and B. Furth , Eds. , 2003 . Garg, A., Naphade, M., Huang, T.S. Modeling video using input/output Markov models with application to multimodal event detection. Handbook of Video Databases: Design and Applications, B. Furth, O. Marques, and B. Furth, Eds., 2003.
10.5555/795661.796240 · DOI
10.5555/795661.796155 · DOI
Larsen , R.J. and Diener , E . Promises and problems with the circumplex model of emotion. Emotion, vol. 13, Review of Personality and Social Psychology, M. S. Clark, Ed., Sage Publications , Newbury Park, USA , 1992 , 25 -- 59 . Larsen, R.J. and Diener, E. Promises and problems with the circumplex model of emotion. Emotion, vol. 13, Review of Personality and Social Psychology, M. S. Clark, Ed., Sage Publications, Newbury Park, USA, 1992, 25--59.
10.5555/265013 · DOI
10.5555/1949767.1949859 · DOI
10.5555/1896300.1896449 · DOI
10.1037/0021-843X.104.1.15 · DOI

Sorted by citations

A Survey of Affect Recognition Methods: Audio, Visual, and Spontaneous Expressions
Zeng, Z., Pantic, M., Roisman, G. I., & Huang, T. S. (2009) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2008.52 · Cited by 2,797
Self-citation
Affective Computing and Sentiment Analysis
Cambria, E. (2016) · IEEE Intelligent Systems · doi:10.1109/mis.2016.31 · Cited by 1,308
Facial Expression Analysis under Partial Occlusion
Zhang, L., Verma, B., Tjondronegoro, D., & Chandran, V. (2018) · ACM Computing Surveys · doi:10.1145/3158369 · Cited by 1,338
Multimodal human–computer interaction: A survey
Jaimes, A., & Sebe, N. (2007) · Computer Vision and Image Understanding · doi:10.1016/j.cviu.2006.10.019 · Cited by 965
Self-citation
Dynamics of facial expression: recognition of facial actions and their temporal segments from face profile image sequences
Pantic, M., & Patras, I. (2006) · IEEE Transactions on Systems, Man and Cybernetics, Part B (Cybernetics) · doi:10.1109/tsmcb.2005.859075 · Cited by 580
A high-resolution 3D dynamic facial expression database
Yin, L., Chen, X., Sun, Y., Worm, T., & Reale, M. (2008) · 2008 8th IEEE International Conference on Automatic Face & Gesture Recognition · doi:10.1109/afgr.2008.4813324 · Cited by 488
AVEC 2019 Workshop and Challenge: State-of-Mind, Detecting Depression with AI, and Cross-Cultural Affect Recognition
Ringeval, F., Schuller, B., Valstar, M., Cummins, N., Cowie, R., Tavabi, L., Schmitt, M., Alisamir, S., Amiriparian, S., Messner, E.-M., et al. (2019) · Proceedings of the 9th International on Audio/Visual Emotion Challenge and Workshop · doi:10.1145/3347320.3357688 · Cited by 409
Bi-modal emotion recognition from expressive face and body gestures
Gunes, H., & Piccardi, M. (2007) · Journal of Network and Computer Applications · doi:10.1016/j.jnca.2006.09.007 · Cited by 382
Emotion recognition using a hierarchical binary decision tree approach
Lee, C.-C., Mower, E., Busso, C., Lee, S., & Narayanan, S. S. (2011) · Speech Communication · doi:10.1016/j.specom.2011.06.004 · Cited by 366
Emotions Don't Lie: An Audio-Visual Deepfake Detection Method using Affective Cues
Mittal, T., Bhattacharya, U., Chandra, R., Bera, A., & Manocha, D. (2020) · Proceedings of the 28th ACM International Conference on Multimedia · doi:10.1145/3394171.3413570 · Cited by 319
Emotion Recognition through Multiple Modalities: Face, Body Gesture, Speech
Castellano, G., Kessous, L., & Caridakis, G. (2008) · Lecture notes in computer science · doi:10.1007/978-3-540-85099-1_8 · Cited by 267
Theories, methods and current research on emotions in library and information science, information retrieval and human–computer interaction
Lopatovska, I., & Arapakis, I. (2010) · Information Processing & Management · doi:10.1016/j.ipm.2010.09.001 · Cited by 279
Multimodal emotion recognition in speech-based interaction using facial expression, body gesture and acoustic analysis
Kessous, L., Castellano, G., & Caridakis, G. (2010) · Journal on Multimodal User Interfaces · doi:10.1007/s12193-009-0025-5 · Cited by 256
Toward Emotion Aware Computing: An Integrated Approach Using Multichannel Neurophysiological Recordings and Affective Visual Stimuli
Frantzidιs, C. A., Bratsas, C., Papadelis, C. L., Konstantinidis, E. I., Pappas, C., & Bamidis, P. D. (2010) · IEEE Transactions on Information Technology in Biomedicine · doi:10.1109/titb.2010.2041553 · Cited by 254
Machine Analysis of Facial Expressions
Pantic, M., & Stewart, M. (2007) · Face Recognition · doi:10.5772/4847 · Cited by 248
Deep learning for human affect recognition: Insights and new developments
Rouast, P. V., Adam, M. T. P., & Chiong, R. (2021) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2018.2890471 · Cited by 243
Facial Expression Analysis
Takeo Kanade, A. (2005) · Lecture notes in computer science · doi:10.1007/11564386_1 · Cited by 226
Artificial Emotional Intelligence in Socially Assistive Robots for Older Adults: A Pilot Study
Abdollahi, H., Mahoor, M. H., Zandie, R., Siewierski, J., & Qualls, S. H. (2022) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2022.3143803 · Cited by 209
A Bimodal Face and Body Gesture Database for Automatic Analysis of Human Nonverbal Affective Behavior
Gunes, H., & Piccardi, M. (2006) · Proceedings - International Conference on Pattern Recognition/Proceedings/International Conference on Pattern Recognition · doi:10.1109/icpr.2006.39 · Cited by 206
Emotion Recognition Based on Joint Visual and Audio Cues
Sebe, N., Cohen, I., Gevers, T., & Huang, T. S. (2006) · 18th International Conference on Pattern Recognition (ICPR'06) · doi:10.1109/icpr.2006.489 · Cited by 202
Self-citation
How to distinguish posed from spontaneous smiles using geometric features
Valstar, M., Güneş, H., & Pantić, M. (2007) · INTERNATIONAL CONFERENCE ON MULTIMODAL INTERACTION · doi:10.1145/1322192.1322202 · Cited by 203
Multimodal emotion recognition from expressive faces, body gestures and speech
Caridakis, G., Castellano, G., Kessous, L., Raouzaiou, A., Malatesta, L., Asteriadis, S., & Karpouzis, K. (2007) · IFIP The International Federation for Information Processing · doi:10.1007/978-0-387-74161-1_41 · Cited by 183
AVEC 2018 Workshop and Challenge
Ringeval, F., Schuller, B. W., Valstar, M., Cowie, R., Kaya, H., Schmitt, M., Amiriparian, S., Cummins, N., Lalanne, D., Michaud, A., et al. (2018) · doi:10.1145/3266302.3266316 · Cited by 149
A survey of affect recognition methods: audio, visual and spontaneous expressions
Zeng, Z., Pantic, M., Roisman, G. I., & Huang, T. S. (2007) · Proceedings of the 9th international conference on Multimodal interfaces · doi:10.1145/1322192.1322216 · Cited by 144
Self-citation
Towards multimodal emotion recognition in e-learning environments
Bahreini, K., Nadolski, R., & Westera, W. (2016) · Interactive Learning Environments · doi:10.1080/10494820.2014.908927 · Cited by 137
Deep learning for video classification and captioning
Wu, Z., Yao, T., Fu, Y., & Jiang, Y.-G. (2017) · doi:10.1145/3122865.3122867 · Cited by 149
Artificial intelligence moving serious gaming: Presenting reusable game AI components
Westera, W., Prada, R., Mascarenhas, S., Santos, P. A., Dias, J., Guimarães, M., Georgiadis, K., Nyamsuren, E., Bahreini, K., Yumak, Z., et al. (2020) · Education and Information Technologies · doi:10.1007/s10639-019-09968-2 · Cited by 136
Audio–Visual Affective Expression Recognition Through Multistream Fused HMM
Zeng, Z., Tu, J., Pianfetti, B., & Huang, T. S. (2008) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2008.921737 · Cited by 134
Self-citation
Modeling naturalistic affective states via facial and vocal expressions recognition
Caridakis, G., Malatesta, L., Kessous, L., Amir, N., Raouzaiou, A. T., & Karpouzis, K. C. (2006) · INTERNATIONAL CONFERENCE ON MULTIMODAL INTERACTION · doi:10.1145/1180995.1181029 · Cited by 133
Interpreting ambiguous emotional expressions
Mower, E., Metallinou, A., Lee, C.-C., Kazemzadeh, A., Busso, C., Lee, S., & Narayanan, S. S. (2009) · International Conference on Affective Computing and Intelligent Interaction and workshops · doi:10.1109/acii.2009.5349500 · Cited by 127
AffectButton: A method for reliable and valid affective self-report
Broekens, J., & Brinkman, W. (2013) · International Journal of Human-Computer Studies · doi:10.1016/j.ijhcs.2013.02.003 · Cited by 133
Beyond Facial Expressions: Learning Human Emotion from Body Gestures
Shan, C., Gong, S., & McOwan, P. W. (2007) · British Machine Vision Conference (BMVC) · doi:10.5244/c.21.43 · Cited by 125
A Multimodal Emotion Detection System during Human–Robot Interaction
Alonso-Martín, F., Malfáz, M., Sequeira, J. S., Gorostiza, J. F., & Salichs, M. A. (2013) · Sensors · doi:10.3390/s131115549 · Cited by 117
Multimodal Human Computer Interaction: A Survey
Jaimes, A., & Sebe, N. (2005) · Lecture notes in computer science · doi:10.1007/11573425_1 · Cited by 115
Self-citation
A Quantum-Like multimodal network framework for modeling interaction dynamics in multiparty conversational sentiment analysis
Zhang, Y., Song, D., Li, X., Zhang, P., Wang, P., Rong, L., Yu, G., & Wang, B. (2020) · Information Fusion · doi:10.1016/j.inffus.2020.04.003 · Cited by 116
New Measures of Mental State and Behavior Based on Data Collected From Sensors, Smartphones, and the Internet
Glenn, T., & Monteith, S. (2014) · Current Psychiatry Reports · doi:10.1007/s11920-014-0523-3 · Cited by 104
Machine analysis of facial behaviour: naturalistic and dynamic behaviour
Pantić, M. (2009) · Philosophical Transactions of the Royal Society B Biological Sciences · doi:10.1098/rstb.2009.0135 · Cited by 107
Foundations of human computing
Cohn, J. F. (2006) · INTERNATIONAL CONFERENCE ON MULTIMODAL INTERACTION · doi:10.1145/1180995.1181043 · Cited by 106
Self-citation
Modeling Latent Discriminative Dynamic of Multi-dimensional Affective Signals
Ramírez, G. A., Baltrušaitis, T., & Morency, L. (2011) · Lecture notes in computer science · doi:10.1007/978-3-642-24571-8_51 · Cited by 91
Adversarial Training in Affective Computing and Sentiment Analysis: Recent Advances and Perspectives [Review Article]
Han, J., Zhang, Z., Cummins, N., & Schuller, B. W. (2019) · IEEE Computational Intelligence Magazine · doi:10.1109/mci.2019.2901088 · Cited by 86
Appearance Manifold of Facial Expression
Shan, C., Gong, S., & McOwan, P. W. (2005) · Lecture notes in computer science · doi:10.1007/11573425_22 · Cited by 86
Using facial expressions and peripheral physiological signals as implicit indicators of topical relevance
Arapakis, I., Konstas, I., & Jose, J. M. (2009) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/1631272.1631336 · Cited by 78
Multimodal deep representation learning for video classification
Tian, H., Tao, Y., Pouyanfar, S., Chen, S., & Shyu, M. (2018) · World Wide Web · doi:10.1007/s11280-018-0548-3 · Cited by 80
Foundations of Human Computing: Facial Expression and Emotion
Cohn, J. F. (2007) · Springer eBooks · doi:10.1007/978-3-540-72348-6_1 · Cited by 77
Self-citation
Segmenting into Adequate Units for Automatic Recognition of Emotion-Related Episodes: A Speech-Based Approach
Batliner, A. M., Seppi, D., Steidl, S., & Schuller, B. W. (2010) · Advances in Human-Computer Interaction · doi:10.1155/2010/782802 · Cited by 72
Modeling Naturalistic Affective States Via Facial, Vocal, and Bodily Expressions Recognition
Karpouzis, K. C., Caridakis, G., Kessous, L., Amir, N., Raouzaiou, A. T., Malatesta, L., & Kollias, S. (2007) · Springer eBooks · doi:10.1007/978-3-540-72348-6_5 · Cited by 72
Emotion Recognition by Fusing Time Synchronous and Time Asynchronous Representations
Wu, W., Zhang, C., & Woodland, P. C. (2021) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp39728.2021.9414880 · Cited by 70
Analyzing Free-standing Conversational Groups
Alameda-Pineda, X., Yan, Y., Ricci, E., Lanz, O., & Sebe, N. (2015) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/2733373.2806238 · Cited by 68
Self-citation
Temporal Segmentation of Facial Behavior
la Torre, F. D., Campoy, J., Ambadar, Z., & Cohn, J. F. (2007) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2007.4408961 · Cited by 66
Self-citation
Speaker-independent emotion recognition exploiting a psychologically-inspired binary cascade classification schema
Kotti, M., & Paternò, F. (2012) · International Journal of Speech Technology · doi:10.1007/s10772-012-9127-7 · Cited by 74
Boosting selection of speech related features to improve performance of multi-class SVMs in emotion detection
Altun, H., & Polat, G. (2008) · Expert Systems with Applications · doi:10.1016/j.eswa.2008.10.005 · Cited by 64
MultiSense—Context-Aware Nonverbal Behavior Analysis Framework: A Psychological Distress Use Case
Stratou, G., & Morency, L. (2016) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2016.2614300 · Cited by 64
A computational model of empathy for interactive agents
Yalcin, O. N., & DiPaola, S. (2018) · Biologically Inspired Cognitive Architectures · doi:10.1016/j.bica.2018.07.010 · Cited by 71
Multimodal interfaces: Challenges and perspectives
Sebe, N. (2009) · Journal of Ambient Intelligence and Smart Environments · doi:10.3233/ais-2009-0003 · Cited by 57
Self-citation
Dynamic Cascades with Bidirectional Bootstrapping for Action Unit Detection in Spontaneous Facial Behavior
Zhu, Y., la Torre, F. D., Cohn, J. F., & Zhang, Y. (2011) · IEEE Transactions on Affective Computing · doi:10.1109/t-affc.2011.10 · Cited by 53
Self-citation
Emotion recognition using facial expressions with active appearance models
Ratliff, M. S., & Patterson, E. K. (2008) · Cited by 53
The emerging roles of performance within HCI and interaction design
Macaulay, C., Jacucci, G., O’Neill, S. J., Kankaineen, T., & Simpson, M. (2006) · Interacting with Computers · doi:10.1016/j.intcom.2006.07.001 · Cited by 52
User and context adaptive neural networks for emotion recognition
Caridakis, G., Karpouzis, K. C., & Kollias, S. (2008) · Neurocomputing · doi:10.1016/j.neucom.2007.11.043 · Cited by 51
Audio-Visual Spontaneous Emotion Recognition
Zeng, Z., Hu, Y., Roisman, G. I., Wen, Z., Fu, Y., & Huang, T. S. (2007) · Springer eBooks · doi:10.1007/978-3-540-72348-6_4 · Cited by 51
Self-citation
EMODA
Ez-Zaouia, M., & Lavoué, É. (2017) · International Conference on Learning Analytics and Knowledge (LAK) · doi:10.1145/3027385.3027434 · Cited by 50
Recursive Joint Cross-Modal Attention for Multimodal Fusion in Dimensional Emotion Recognition
Praveen, R. G., & Alam, M. J. (2024) · IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw63382.2024.00483 · Cited by 54
A fuzzy logic approach to reliable real-time recognition of facial emotions
Bahreini, K., van der Vegt, W., & Westera, W. (2019) · Multimedia Tools and Applications · doi:10.1007/s11042-019-7250-z · Cited by 51
Audio-visual emotion recognition in adult attachment interview
Zeng, Z., Hu, Y., Fu, Y., Huang, T. S., Roisman, G. I., & Wen, Z. (2006) · INTERNATIONAL CONFERENCE ON MULTIMODAL INTERACTION · doi:10.1145/1180995.1181028 · Cited by 46
Self-citation
A comparison of general vs personalised affective models for the prediction of topical relevance
Arapakis, I., Αθανασάκος, Κ., & Jose, J. M. (2010) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/1835449.1835512 · Cited by 45
Recognizing partial facial action units based on 3D dynamic range data for facial expression recognition
Sun, Y., Reale, M. J., & Yin, L. (2008) · International Conference on Automatic Face and Gesture Recognition · doi:10.1109/afgr.2008.4813336 · Cited by 46
Gaze-X: Adaptive, Affective, Multimodal Interface for Single-User Office Scenarios
Maat, L., & Pantic, M. (2007) · Lecture Notes in Computer Science · doi:10.1007/978-3-540-72348-6_13 · Cited by 44
Image-based memes as sentiment predictors
French, J. H. (2017) · International Conference on Information Society · doi:10.23919/i-society.2017.8354676 · Cited by 42
Context-Aware Automated Analysis and Annotation of Social Human--Agent Interactions
Baur, T., Mehlmann, G., Damian, I., Lingenfelser, F., Wagner, J., Lugrin, B., André, E., & Gebhard, P. (2015) · ACM Transactions on Interactive Intelligent Systems · doi:10.1145/2764921 · Cited by 40
Adaptive eye-gaze tracking using neural-network-based user profiles to assist people with motor disability
Sesin, A. (2008) · The Journal of Rehabilitation Research and Development · doi:10.1682/jrrd.2007.05.0075 · Cited by 39
Sentiment analysis of COVID-19 cases in Greece using Twitter data
Samaras, L., García‐Barriocanal, E., & Sicilia, M. (2023) · Expert Systems with Applications · doi:10.1016/j.eswa.2023.120577 · Cited by 38
Multimodal user’s affective state analysis in naturalistic interaction
Caridakis, G., Karpouzis, K. C., Wallace, M., Kessous, L., & Amir, N. (2009) · Journal on Multimodal User Interfaces · doi:10.1007/s12193-009-0030-8 · Cited by 38
Driver frustration detection from audio and video in the wild
Abdić, I., Fridman, L., McDuff, D., Marchi, E., Reimer, B., & Schuller, B. W. (2016) · OPUS (Augsburg University) · Cited by 35
Multimodal Transformer with Learnable Frontend and Self Attention for Emotion Recognition
Dutta, S., & Ganapathy, S. (2022) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp43922.2022.9747723 · Cited by 34
Joint Multi-Scale Multimodal Transformer for Emotion Using Consumer Devices
Khan, M., Ahmad, J., Gueaieb, W., Masi, G., Karray, F. O., & Saddik, A. E. (2025) · IEEE Transactions on Consumer Electronics · doi:10.1109/tce.2025.3532322 · Cited by 37
Competitive affective gaming
Mourão, A., & Magalhães, J. (2013) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/2502081.2502115 · Cited by 33
StimuVAR: Spatiotemporal Stimuli-Aware Video Affective Reasoning with Multimodal Large Language Models
Guo, Y., Siddiqui, F., Zhao, Y., Chellappa, R., & Lo, S.-Y. (2025) · International Journal of Computer Vision · doi:10.1007/s11263-025-02495-3 · Cited by 39
Building a three-level multimodal emotion recognition framework
Garcia-Garcia, J. M., Lozano, M. D., Penichet, V. M. R., & Law, E. L. (2022) · Multimedia Tools and Applications · doi:10.1007/s11042-022-13254-8 · Cited by 32
Gaze-X
Maat, L., & Pantić, M. (2006) · INTERNATIONAL CONFERENCE ON MULTIMODAL INTERACTION · doi:10.1145/1180995.1181032 · Cited by 32
Multimodal Affect Recognition: Current Approaches and Challenges
Osman, H. A., & Falk, T. H. (2017) · InTech eBooks · doi:10.5772/65683 · Cited by 37
LLM supervised Pre-training for Multimodal Emotion Recognition in Conversations
Dutta, S., & Ganapathy, S. (2025) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp49660.2025.10889998 · Cited by 38
Recognizing Facial Expressions Automatically from Video
Shan, C., & Braspenning, R. (2009) · Springer eBooks · doi:10.1007/978-0-387-93808-0_18 · Cited by 30
Dynamic cascades with bidirectional bootstrapping for spontaneous facial action unit detection
Zhu, Y., la Torre, F. D., Cohn, J. F., & Zhang, Y. (2009) · International Conference on Affective Computing and Intelligent Interaction and workshops · doi:10.1109/acii.2009.5349603 · Cited by 30
Self-citation
A Report on the First International Workshop on Research Methods in Animal-Computer Interaction
Zamansky, A., Roshier, M., Mancini, C., Collins, E. C., Hall, C., Grillaert, K., Morrison, A., North, S., & Wirman, H. (2017) · CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA) · doi:10.1145/3027063.3052759 · Cited by 28
Multimodal Emotion Recognition from Low-Level Cues
Pantić, M., Caridakis, G., André, E., Kim, J., Karpouzis, K. C., & Kollias, S. (2010) · Cognitive technologies · doi:10.1007/978-3-642-15184-2_8 · Cited by 28
Representation of facial expression categories in continuous arousal–valence space: Feature and correlation
Zhang, L., Tjondronegoro, D., & Chandran, V. (2014) · Image and Vision Computing · doi:10.1016/j.imavis.2014.09.005 · Cited by 27
The affect-aware city
Guthier, B., Abaalkhail, R., Alharthi, R., & Saddik, A. E. (2015) · 2012 International Conference on Computing, Networking and Communications (ICNC) · doi:10.1109/iccnc.2015.7069418 · Cited by 26
EEG Emotion Copilot: Optimizing lightweight LLMs for emotional EEG interpretation with assisted medical record generation
Chen, H., Zeng, W., Chen, C., Cai, L., Wang, F., Shi, Y., Wang, L., Zhang, W., Li, Y., Yan, H., et al. (2025) · Neural Networks · doi:10.1016/j.neunet.2025.107848 · Cited by 32
Frontiers of Multimedia Research
Chang, S.-F. (2017) · doi:10.1145/3122865 · Cited by 26
Affective multimodal mirror
Melder, W. A., Truong, K. P., den Uyl, M. J., van Leeuwen, D. A., Neerincx, M. A., Loos, L. R., & Plum, B. S. (2007) · doi:10.1145/1290128.1290134 · Cited by 24
Red Hen Lab: Dataset and Tools for Multimodal Human Communication Research
Joo, J., Steen, F. F., & Turner, M. B. (2017) · Künstliche Intell. · doi:10.1007/s13218-017-0505-9 · Cited by 26
Vision-Based Game Interface Using Human Gesture
Park, H. S., Jung, D. J., & Kim, H. J. (2006) · Lecture notes in computer science · doi:10.1007/11949534_66 · Cited by 24
M3ER: Multiplicative Multimodal Emotion Recognition using Facial, Textual, and Speech Cues
Mittal, T., Bhattacharya, U., Chandra, R., Bera, A., & Manocha, D. (2020) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v34i02.5492 · Cited by 23
Applied Affective Computing
Tian, L., Oviatt, S. L., Muszyński, M., Chamberlain, B. C., Healey, J., & Sano, A. (2022) · ACM eBooks · doi:10.1145/3502398 · Cited by 25
A Survey to Understand Emotional Situations on the Road and What They Mean for Affective Automotive UIs
Braun, M., Pfleging, B., & Alt, F. (2018) · Multimodal Technologies and Interaction · doi:10.3390/mti2040075 · Cited by 23
Towards robust automatic affective classification of images using facial expressions for practical applications
Zhang, L., Tjondronegoro, D., Chandran, V., & Eggink, J. (2015) · Multimedia Tools and Applications · doi:10.1007/s11042-015-2497-5 · Cited by 22
Incongruity-Aware Cross-Modal Attention for Audio-Visual Fusion in Dimensional Emotion Recognition
Praveen, R. G., & Alam, J. (2024) · IEEE Journal of Selected Topics in Signal Processing · doi:10.1109/jstsp.2024.3422823 · Cited by 21
Empirical analysis of multiple modalities for emotion recognition using convolutional neural network
Jaswal, R. A., & Dhingra, S. (2023) · Measurement Sensors · doi:10.1016/j.measen.2023.100716 · Cited by 20
Face for Ambient Interface
Pantić, M. (2006) · Lecture notes in computer science · doi:10.1007/11825890_2 · Cited by 20
Evaluating the Emotional State of a User Using a Webcam
Magdin, M., Turčáni, M., & Hudec, L. (2016) · International Journal of Interactive Multimedia and Artificial Intelligence · doi:10.9781/ijimai.2016.4112 · Cited by 20
Multimodal databases
Valstar, M. (2019) · ACM eBooks · doi:10.1145/3233795.3233807 · Cited by 18

Authors: Pantic, M., Sebe, N., Cohn, J. F., & Huang, T.
Affiliations: Delft University of Technology, The Netherlands
University of Amsterdam (NL)
University of Pittsburgh (US)
University of Illinois Urbana-Champaign (US)
ORCID: Sebe, Nicu
Cohn, Jeffrey F.
Huang, Thomas
Venue: Proceedings of the 13th annual ACM international conference on Multimedia
Publisher: ACM
Published: 2005-11-06
Pages: 669-676
DOI: 10.1145/1101149.1101299
OpenAlex: W2028975938
Cited by: 234
References: 50
Citations by year: 2026: 9 · 2025: 11 · 2024: 8 · 2023: 8 · 2022: 23 · 2021: 6 · 2020: 10 · 2019: 9 · 2018: 11 · 2017: 9 · 2016: 7 · 2015: 12 · 2014: 6 · 2013: 14 · 2012: 8
FWCI: 25.50
Type: conference paper
Language: en
Open access: closed
Licence: www.acm.org/publications/policies/copyright_policy#Background (vor)
Locations: ACM International Conference on Multimedia (ACM MM) (publishedVersion)
UvA-DARE (University of Amsterdam) (publishedVersion)
Institutional Research Information System (Università degli Studi di Trento) (submittedVersion)
Data Archiving and Networked Services (DANS) (submittedVersion)
Topics: Physical Sciences › Computer Science › Artificial Intelligence › Emotion and Mood Recognition
Social Sciences › Psychology › Social Psychology › Social Robot Interaction and HRI
Physical Sciences › Computer Science › Human-Computer Interaction › Innovative Human-Technology Interaction
Keywords: affective computing, multimodal human-computer interaction, affective states, social-emotional intelligence, user modeling, adaptive systems, emotion recognition

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

Affective multimodal human-computer interaction

Abstract

Overview

Integrity signals

Verification

Claims the paper makes

Claims other papers make about this one

Citations

Cites

Cited by

Related

Metadata

Record

About these checks