The DOI matches its Crossref record; 50 of the 50 references it cites were found in a registry or an index.
Affective multimodal human-computer interaction
Abstract
Social and emotional intelligence are aspects of human intelligence that have been argued to be better predictors than IQ for measuring aspects of success in life, especially in social interactions, learning, and adapting to what is important. When it comes to machines, not all of them will need such skills. Yet to have machines like computers, broadcast systems, and cars, capable of adapting to their users and of anticipating their wishes, endowing them with the ability to recognize user's affective states is necessary. This article discusses the components of human affect, how they might be integrated into computers, and how far are we from realizing affective multimodal human-computer interaction.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Previous software primarily dealt with offline emotion recognition that cause post-processing of the learner's data.”
From Towards multimodal emotion recognition in e-learning environments (2016) · cites this paper as Pantic, Sebe, Cohn, & Huang, 2005
In the citing paper
“The application context is restricted by the fact that such software can only manage a small set of expressions from frontal view faces without facial hair, glasses provided that there is constant illumination.”
From Towards multimodal emotion recognition in e-learning environments (2016) · cites this paper as Pantic, Sebe, Cohn, & Huang, 2005
Citations
Cites
Picard, R. W. (1997) · MIT Press · doi:10.7551/mitpress/1140.001.0001 · Cited by 7,382
The Repertoire of Nonverbal Behavior: Categories, Origins, Usage, and Coding
EKMAN, P., & FRIESEN, W. V. (1969) · Semiotica · doi:10.1515/semi.1969.1.1.49 · Cited by 3,316
Comprehensive database for facial expression analysis
Takeo Kanade, A., Cohn, J. F., & Tian, Y. (2002) · International Conference on Automatic Face and Gesture Recognition · doi:10.1109/afgr.2000.840611 · Cited by 2,644
Self-citationToward machine emotional intelligence: analysis of affective physiological state
Picard, R. W., Vyzas, E., & Healey, J. (2001) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/34.954607 · Cited by 2,334
A Survey on Visual Surveillance of Object Motion and Behaviors
Hu, W., Tan, T. N., Wang, L., & Maybank, S. J. (2004) · IEEE Transactions on Systems Man and Cybernetics Part C (Applications and Reviews) · doi:10.1109/tsmcc.2004.829274 · Cited by 2,156
Thin slices of expressive behavior as predictors of interpersonal consequences: A meta-analysis.
Ambady, N., & Rosenthal, R. (1992) · Psychological Bulletin · doi:10.1037/0033-2909.111.2.256 · Cited by 2,003
Russell, J. A. (1994) · Psychological Bulletin · doi:10.1037/0033-2909.115.1.102 · Cited by 1,640
Acoustic profiles in vocal emotion expression
Banse, R., & Scherer, K. R. (1996) · Journal of Personality and Social Psychology · doi:10.1037/0022-3514.70.3.614 · Cited by 1,546
Watson, D., Weber, K., Assenheimer, J. S., Clark, L. A., Strauss, M. E., & McCormick, R. A. (1995) · Journal of Abnormal Psychology · doi:10.1037/0021-843x.104.1.3 · Cited by 1,423
Web-Based Database for Facial Expression Analysis
Pantić, M., Valstar, M., Rademaker, R. M., & Maat, L. (2005) · IEEE International Conference on Multimedia and Expo (ICME) · doi:10.1109/icme.2005.1521424 · Cited by 1,124
Watson, D., Clark, L. A., Weber, K., Assenheimer, J. S., Strauss, M. E., & McCormick, R. A. (1995) · Journal of Abnormal Psychology · doi:10.1037//0021-843x.104.1.15 · Cited by 1,002
Speech emotion recognition using hidden Markov models
Nwe, T. L., Foo, S. W., & De Silva, L. C. (2003) · Speech Communication · doi:10.1016/s0167-6393(03)00099-2 · Cited by 920
Facial expression recognition from video sequences: temporal and static modeling
Cohen, I., Sebe, N., Garg, A., Chen, L. S., & Huang, T. S. (2003) · Computer Vision and Image Understanding · doi:10.1016/s1077-3142(03)00081-x · Cited by 874
Self-citationToward an affect-sensitive multimodal human-computer interaction
Pantic, M., & Rothkrantz, L. J. M. (2003) · Proceedings of the IEEE · doi:10.1109/jproc.2003.817122 · Cited by 791
Watson, D., Clark, L. A., Weber, K., et al. (1995) · Journal of Abnormal Psychology · doi:10.1037/0021-843x.104.1.15 · Cited by 745
Emotional Intelligence. New York (Bantam) 1995.
Goleman, D. (1995) · Cited by 666
The Psychology of Facial Expression
Mandler, G. (1997) · Cambridge University Press eBooks · doi:10.1017/cbo9780511659911 · Cited by 656
Cultural similarities and differences in display rules
Matsumoto, D. (1990) · Motivation and Emotion · doi:10.1007/bf00995569 · Cited by 652
Behavioral markers and recognizability of the smile of enjoyment.
Frank, M. G., Ekman, P., & Friesen, W. V. (1993) · Journal of Personality and Social Psychology · doi:10.1037/0022-3514.64.1.83 · Cited by 555
Keltner, D. J., Ekman, P., Gonzaga, G. C., & Beer, J. S. (2002) · doi:10.1093/oso/9780195126013.003.0022 · Cited by 555
Handbook of methods in nonverbal behavior research
Scherer, K. R., & Ekman, P. (1982) · Cited by 521
Human-centered design considered harmful
Norman, D. A. (2005) · interactions · doi:10.1145/1070960.1070976 · Cited by 487
Behavioral Markers and Recognizability of the Smile of Enjoyment
Frank, M. G., Ekman, P., & Friesen, W. V. (2005) · Oxford University Press eBooks · doi:10.1093/acprof:oso/9780195179644.003.0011 · Cited by 412
Looking at people: sensing for ubiquitous and wearable computing
Pentland, A. (2000) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/34.824823 · Cited by 406
The New Handbook of Methods in Nonverbal Behavior Research
Harrigan, J. A., Rosenthal, R., & Scherer, K. R. (2008) · Oxford University Press eBooks · doi:10.1093/acprof:oso/9780198529620.001.0001 · Cited by 359
Juslin, P. N., & Scherer, K. R. (2005) · Oxford University Press eBooks · doi:10.1093/oso/9780198529613.003.0003 · Cited by 339
THE TIMING OF FACIAL MOTION IN POSED AND SPONTANEOUS SMILES
Cohn, J. F., & Schmidt, K. L. (2004) · International Journal of Wavelets Multiresolution and Information Processing · doi:10.1142/s021969130400041x · Cited by 308
Self-citationAudio-visual integration in multimodal communication
Chen, T., & Rao, R. R. (1998) · Proceedings of the IEEE · doi:10.1109/5.664274 · Cited by 256
Socially aware, computation and communication
Pentlan, A. (2005) · Computer · doi:10.1109/mc.2005.104 · Cited by 255
Has the Internet become indispensable?
Hoffman, D. L., Novak, T., & Venkatesh, A. (2004) · Communications of the ACM · doi:10.1145/1005817.1005818 · Cited by 229
De Silva, L. C., & Ng, P. C. (2002) · Proceedings Fourth IEEE International Conference on Automatic Face and Gesture Recognition (Cat. No. PR00580) · doi:10.1109/afgr.2000.840655 · Cited by 196
The new ethology of human facial expressions
Fridlund, A. J. (1997) · Cambridge University Press eBooks · doi:10.1017/cbo9780511659911.007 · Cited by 194
Yoshitomi, Y., Kim, S.-I., Kawano, T., & Kilazoe, T. (2002) · Proceedings 9th IEEE International Workshop on Robot and Human Interactive Communication. IEEE RO-MAN 2000 (Cat. No.00TH8499) · doi:10.1109/roman.2000.892491 · Cited by 144
Emotion Analysis in Man-Machine Interaction Systems
Balomenos, T., Raouzaiou, A. T., Ioannou, S. V., Drosopoulos, A., Karpouzis, K. C., & Kollias, S. (2005) · Lecture notes in computer science · doi:10.1007/978-3-540-30568-2_27 · Cited by 129
Audio-visual based emotion recognition-a new approach
Song, M., Bu, J., Chen, C., & Li, N. (2004) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2004.1315276 · Cited by 126
Huang, T. S., & Chen, L. S.-H. (2000) · Cited by 108
Self-citationEmbodied contextual agent in information delivering application
Pélachaud, C., Carofiglio, V., Carolis, B. N. D., de Rosis, F., & Poggi, I. (2002) · Adaptive Agents and Multi-Agents Systems · doi:10.1145/544862.544921 · Cited by 105
Authentic facial expression analysis
Sebe, N., Lew, M. S., Cohen, I., Sun, Y., Gevers, T., & Huang, T. S. (2004) · International Conference on Automatic Face and Gesture Recognition · doi:10.1109/afgr.2004.1301585 · Cited by 101
Self-citationEmotional expressions in audiovisual human computer interaction
Chen, L., & Huang, T. S. (2002) · IEEE International Conference on Multimedia and Expo (ICME) · doi:10.1109/icme.2000.869630 · Cited by 83
Self-citationUser-centered modeling and evaluation of multimodal interfaces
Oviatt, S. L. (2003) · Proceedings of the IEEE · doi:10.1109/jproc.2003.817127 · Cited by 83
Emotion recognition from the facial image and speech signal
Go, H.-J., Kwak, K.-C., Lee, D.-J., & Chun, M.-G. (2003) · Cited by 63
Bimodal expression of emotion by face and voice
Cohn, J. F., & Katz, G. S. (1998) · doi:10.1145/306668.306683 · Cited by 54
Self-citationBoosted learning in dynamic bayesian networks for multimodal speaker detection
Garg, A., Pavlović, V., & Rehg, J. M. (2003) · Proceedings of the IEEE · doi:10.1109/jproc.2003.817119 · Cited by 51
Audio-visual speaker detection using dynamic Bayesian networks
Garg, A., Pavlović, V., & Rehg, J. M. (2002) · International Conference on Automatic Face and Gesture Recognition · doi:10.1109/afgr.2000.840663 · Cited by 42
Feature analysis for automatic speechreading
Scanlon, P., & Reilly, R. B. (2002) · IEEE Workshop on Multimedia Signal Processing (MMSP) · doi:10.1109/mmsp.2001.962802 · Cited by 41
Wierzbicka, A. (1993) · Pragmatics & Cognition · doi:10.1075/pc.1.1.03wie · Cited by 31
Case-based reasoning for user-profiled recognition of emotions from face images
Pantić, M., & Rothkrantz, L. J. M. (2005) · IEEE International Conference on Multimedia and Expo (ICME) · doi:10.1109/icme.2004.1394211 · Cited by 23
Exploiting the dependencies in information fusion
Pan, H., Liang, Z., & Huang, T. S. (2003) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.1999.784713 · Cited by 21
Self-citationWierzbicka, A. (1999) · Cambridge University Press eBooks · doi:10.1017/cbo9780511521256.004 · Cited by 5
(1993)
Flagged first, then by citations
Research works printed but not linked: 8
- 10.5555/795661.796218 · DOI
- Garg , A. , Naphade , M. , Huang , T.S. Modeling video using input/output Markov models with application to multimodal event detection. Handbook of Video Databases: Design and Applications, B. Furth, O. Marques, and B. Furth , Eds. , 2003 . Garg, A., Naphade, M., Huang, T.S. Modeling video using input/output Markov models with application to multimodal event detection. Handbook of Video Databases: Design and Applications, B. Furth, O. Marques, and B. Furth, Eds., 2003.
- 10.5555/795661.796240 · DOI
- 10.5555/795661.796155 · DOI
- Larsen , R.J. and Diener , E . Promises and problems with the circumplex model of emotion. Emotion, vol. 13, Review of Personality and Social Psychology, M. S. Clark, Ed., Sage Publications , Newbury Park, USA , 1992 , 25 -- 59 . Larsen, R.J. and Diener, E. Promises and problems with the circumplex model of emotion. Emotion, vol. 13, Review of Personality and Social Psychology, M. S. Clark, Ed., Sage Publications, Newbury Park, USA, 1992, 25--59.
- 10.5555/265013 · DOI
- 10.5555/1949767.1949859 · DOI
- 10.5555/1896300.1896449 · DOI
Cited by
Sorted by citations
A Survey of Affect Recognition Methods: Audio, Visual, and Spontaneous Expressions
Zeng, Z., Pantic, M., Roisman, G. I., & Huang, T. S. (2009) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2008.52 · Cited by 2,797
Self-citationAffective Computing and Sentiment Analysis
Cambria, E. (2016) · IEEE Intelligent Systems · doi:10.1109/mis.2016.31 · Cited by 1,308
Facial Expression Analysis under Partial Occlusion
Zhang, L., Verma, B., Tjondronegoro, D., & Chandran, V. (2018) · ACM Computing Surveys · doi:10.1145/3158369 · Cited by 1,338
Multimodal human–computer interaction: A survey
Jaimes, A., & Sebe, N. (2007) · Computer Vision and Image Understanding · doi:10.1016/j.cviu.2006.10.019 · Cited by 965
Self-citationPantic, M., & Patras, I. (2006) · IEEE Transactions on Systems, Man and Cybernetics, Part B (Cybernetics) · doi:10.1109/tsmcb.2005.859075 · Cited by 580
Metadata
- Authors
- Pantic, M., Sebe, N., Cohn, J. F., & Huang, T.
- Affiliations
- Delft University of Technology, The Netherlands
- University of Amsterdam (NL)
- University of Pittsburgh (US)
- University of Illinois Urbana-Champaign (US)
- Venue
- Proceedings of the 13th annual ACM international conference on Multimedia
- Publisher
- ACM
- Published
- 2005-11-06
- Pages
- 669-676
- OpenAlex
- W2028975938
- Cited by
- 234
- References
- 50
- Citations by year
- 2026: 9 · 2025: 11 · 2024: 8 · 2023: 8 · 2022: 23 · 2021: 6 · 2020: 10 · 2019: 9 · 2018: 11 · 2017: 9 · 2016: 7 · 2015: 12 · 2014: 6 · 2013: 14 · 2012: 8
- FWCI
- 25.50
- Type
- conference paper
- Language
- en
- Open access
- closed
- Locations
- ACM International Conference on Multimedia (ACM MM) (publishedVersion)
- UvA-DARE (University of Amsterdam) (publishedVersion)
- Institutional Research Information System (Università degli Studi di Trento) (submittedVersion)
- Data Archiving and Networked Services (DANS) (submittedVersion)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Emotion and Mood Recognition
- Social Sciences › Psychology › Social Psychology › Social Robot Interaction and HRI
- Physical Sciences › Computer Science › Human-Computer Interaction › Innovative Human-Technology Interaction
- Keywords
- affective computing, multimodal human-computer interaction, affective states, social-emotional intelligence, user modeling, adaptive systems, emotion recognition
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
