Affect Recognition from Face and Body: Early Fusion vs. Late Fusion
Open accessAbstract
This paper presents an approach to automatic visual emotion recognition from two modalities: face and body. Firstly, individual classifiers are trained from individual modalities. Secondly, we fuse facial expression and affective body gesture information first at a feature-level, in which the data from both modalities are combined before classification, and later at a decision-level, in which we integrate the outputs of the monomodal systems by the use of suitable criteria. We then evaluate these two fusion approaches, in terms of performance over monomodal emotion recognition based on facial expression modality only. In the experiments performed the emotion classification using the two modalities achieved a better recognition accuracy outperforming the classification using the individual facial modality. Moreover, fusion at the feature-level proved better recognition than fusion at the decision-level.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Weight criterion assigns a confidence to each classifier output, outputting a weighted linear combination of the predictions.
“Weight criterion solves this by assigning a confidence to each classifier output, otputting a weighted linear combination of the predictions [161], [162], [167], [200].”
From Survey on RGB, 3D, Thermal, and Multimodal Approaches for Facial Expression Recognition: History, Trends, and Affect-Related Applications (2016) · cites this paper as [161]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
(1981) · Cited by 11,963 · see citing works
Ekman, P., & Friesen, W. V. (2019) · PsycTESTS Dataset · doi:10.1037/t27734-000 · Cited by 6,019 · see citing works
Kittler, J., Hatef, M., Duin, R. P. W., & Matas, J. (1998) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/34.667881 · Cited by 5,312 · see citing works
Ekman, P., & Friesen, W. V. (1975) · Cited by 3,569 · see citing works
Computer Vision Face Tracking For Use in a Perceptual User Interface
Gary, R. B. (1998) · Cited by 1,400 · see citing works
Mehrabian, A. (2017) · communication theory · doi:10.4324/9781315080918-15 · Cited by 879 · see citing works
Toward an affect-sensitive multimodal human-computer interaction
Pantic, M., & Rothkrantz, L. J. M. (2003) · Proceedings of the IEEE · doi:10.1109/jproc.2003.817122 · Cited by 791 · see citing works
A theoretical study on six classifier fusion strategies
Kuncheva, L. I. (2002) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/34.982906 · Cited by 724 · see citing works
von Laban, R., & Ullmann, L. (1950) · Medical Entomology and Zoology · Cited by 429 · see citing works
The contribution of general features of body movement to the attribution of emotions
de Meijer, M. (1989) · Journal of Nonverbal Behavior · doi:10.1007/bf00990296 · Cited by 408 · see citing works
To feel or not to feel: The role of affect in human–computer interaction
Hudlická, E. (2003) · International Journal of Human-Computer Studies · doi:10.1016/s1071-5819(03)00047-8 · Cited by 392 · see citing works
A novel method for automatic face segmentation, facial feature extraction and tracking
Sobottka, K., & Pitas, I. (1998) · Signal Processing: Image Communication · doi:10.1016/s0923-5965(97)00042-8 · Cited by 306 · see citing works
Children's decoding of emotion in expressive body movement: The development of cue attunement.
Boone, R. T., & Cunningham, J. G. (1998) · Developmental Psychology · doi:10.1037//0012-1649.34.5.1007 · Cited by 225 · see citing works
De Silva, L. C., & Ng, P. C. (2002) · Proceedings Fourth IEEE International Conference on Automatic Face and Gesture Recognition (Cat. No. PR00580) · doi:10.1109/afgr.2000.840655 · Cited by 196 · see citing works
Multimodal integration-a statistical view
Wu, L., Oviatt, S. L., & Cohen, P. R. (1999) · IEEE Transactions on Multimedia · doi:10.1109/6046.807953 · Cited by 190 · see citing works
A Framework for Gesture Generation and Interpretation
Cassell, J. (1998) · Cambridge University Press eBooks · doi:10.1017/cbo9780511569937.013 · Cited by 173 · see citing works
Wood, A. (1986) · Cited by 171 · see citing works
How universal and specific is emotional experience? Evidence from 27 countries on five continents
Wallbott, H. G., & Scherer, K. R. (1986) · Social Science Information · doi:10.1177/053901886025004001 · Cited by 168 · see citing works
Machine learning methods for fully automatic recognition of facial expressions and facial actions
Bartlett, M. S., Littlewort, G. C., Lainscsek, C. S. M., Fasel, I., & Movellan, J. R. (2005) · Conference proceedings/Conference proceedings - IEEE International Conference on Systems, Man, and Cybernetics · doi:10.1109/icsmc.2004.1398364 · Cited by 168 · see citing works
Emotional experience in cultural context: A comparison between Europe, Japan, and the United States.
Scherer, K. R., Matsumoto, D., Wallbott, H. G., & Kudoh, T. (1988) · doi:10.4324/9781315808123-6 · Cited by 137 · see citing works
Nonverbal behavior in psychotherapy research.
Ekman, P., & Friesen, W. V. (1968) · American Psychological Association eBooks · doi:10.1037/10546-011 · Cited by 133 · see citing works
Emotion Analysis in Man-Machine Interaction Systems
Balomenos, T., Raouzaiou, A. T., Ioannou, S. V., Drosopoulos, A., Karpouzis, K. C., & Kollias, S. (2005) · Lecture notes in computer science · doi:10.1007/978-3-540-30568-2_27 · Cited by 126 · see citing works
Probabilistic combination of multiple modalities to detect interest
Kapoor, A., Picard, R. W., & Ivanov, Y. A. (2004) · Proceedings - International Conference on Pattern Recognition/Proceedings/International Conference on Pattern Recognition · doi:10.1109/icpr.2004.1334690 · Cited by 100 · see citing works
Identification of Emotion from Body Movements: A Cross-Cultural Study of Americans and Japanese
Sogon, S., & Masutani, M. (1989) · Psychological Reports · doi:10.2466/pr0.1989.65.1.35 · Cited by 66 · see citing works
Facial expression recognition using thermal image processing and neural network
Yoshitomi, Y., Miyawaki, N., Tomita, S., & Kimura, S. (2002) · Proceedings 6th IEEE International Workshop on Robot and Human Communication. RO-MAN'97 SENDAI · doi:10.1109/roman.1997.647016 · Cited by 58 · see citing works
Emotion recognition from audiovisual information
Chen, L. S., Tao, H., Huang, T. S., Miyasato, T., & Nakatsu, R. (2002) · IEEE International Workshop on Multimedia Signal Processing (MMSP) · doi:10.1109/mmsp.1998.738917 · Cited by 54 · see citing works
Computer Vision and Image Processing
(2021) · Communications in computer and information science · doi:10.1007/978-981-16-1092-9 · Cited by 43 · see citing works
Automatic Visual Recognition of Face and Body Action Units
Gunes, H., & Piccardi, M. (2005) · International Conference on Information Technology and Applications · doi:10.1109/icita.2005.83 · Cited by 9 · see citing works
Self-citation
Flagged first, then by citations
1 of the 29 references couldn’t be matched to a known paper.
Cited by
Sorted by citations
A Survey of Affect Recognition Methods: Audio, Visual, and Spontaneous Expressions
Zeng, Z., Pantic, M., Roisman, G. I., & Huang, T. S. (2009) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2008.52 · Cited by 2,797 · see citing works
A review of affective computing: From unimodal analysis to multimodal fusion
Poria, S., Cambria, E., Bajpai, R., & Hussain, A. (2017) · Information Fusion · doi:10.1016/j.inffus.2017.02.003 · Cited by 1,644 · see citing works
Schuller, B. W., Batliner, A. M., Steidl, S., & Seppi, D. (2011) · Speech Communication · doi:10.1016/j.specom.2011.01.011 · Cited by 765 · see citing works
Corneanu, C. A., Simon, M. O., Cohn, J. F., & Guerrero, S. E. (2016) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2016.2515606 · Cited by 527 · see citing works
Survey on Emotional Body Gesture Recognition
Noroozi, F., Corneanu, C. A., Kamińska, D., Sapiński, T., Escalera, S., & Anbarjafari, G. (2018) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2018.2874986 · Cited by 491 · see citing works
Human Computing and Machine Understanding of Human Behavior: A Survey
Pantić, M., Pentland, A., Nijholt, A., & Huang, T. S. (2007) · Springer eBooks · doi:10.1007/978-3-540-72348-6_3 · Cited by 318 · see citing works
Gunes, H., & Piccardi, M. (2006) · Proceedings - International Conference on Pattern Recognition/Proceedings/International Conference on Pattern Recognition · doi:10.1109/icpr.2006.39 · Cited by 206 · see citing works
Self-citationSchuller, B. W., Müller, R., Eyben, F., Gast, J., Hörnler, B., Wöllmer, M., Rigoll, G., Hoethker, A., & Konosu, H. (2009) · Image and Vision Computing · doi:10.1016/j.imavis.2009.02.013 · Cited by 194 · see citing works
Selective Transfer Machine for Personalized Facial Expression Analysis
Chu, W.-S., la Torre, F. D., & Cohn, J. F. (2016) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2016.2547397 · Cited by 190 · see citing works
Human computing and machine understanding of human behavior
Pantić, M., Pentland, A., Nijholt, A., & Huang, T. S. (2006) · INTERNATIONAL CONFERENCE ON MULTIMODAL INTERACTION · doi:10.1145/1180995.1181044 · Cited by 153 · see citing works
A survey of affect recognition methods: audio, visual and spontaneous expressions
Zeng, Z., Pantic, M., Roisman, G. I., & Huang, T. S. (2007) · Proceedings of the 9th international conference on Multimodal interfaces · doi:10.1145/1322192.1322216 · Cited by 144 · see citing works
Multimodal Emotion Recognition
Haq, S., & Jackson, P. J. B. (2010) · IGI Global eBooks · doi:10.4018/978-1-61520-919-4.ch017 · Cited by 143 · see citing works
Depeursinge, A., Racoceanu, D., Iavindrasana, J., Cohen, G., Platon, A., Poletti, P., & Müller, H. (2010) · Artificial Intelligence in Medicine · doi:10.1016/j.artmed.2010.04.006 · Cited by 124 · see citing works
Salama, E. S., El-Khoribi, R. A., Shoman, M. E., & Shalaby, M. A. W. (2020) · Egyptian Informatics Journal · doi:10.1016/j.eij.2020.07.005 · Cited by 117 · see citing works
Using physiological signals for emotion recognition
Szwoch, W. (2013) · Conference on Human System Interactions · doi:10.1109/hsi.2013.6577880 · Cited by 109 · see citing works
Automatic Emotion Recognition Using Temporal Multimodal Deep Learning
Nakisa, B., Rastgoo, M. N., Rakotonirainy, A., Maire, F., & Chandran, V. (2020) · IEEE Access · doi:10.1109/access.2020.3027026 · Cited by 101 · see citing works
Wang, W. (2010) · IGI Global eBooks · doi:10.4018/978-1-61520-919-4 · Cited by 135 · see citing works
ARBEE: Towards Automated Recognition of Bodily Expression of Emotion in the Wild
Luo, Y., Ye, J., Adams, R. B., Li, J., Newman, M. G., & Wang, J. Z. (2019) · International Journal of Computer Vision · doi:10.1007/s11263-019-01215-y · Cited by 101 · see citing works
Multimodal Learning with Incomplete Modalities by Knowledge Distillation
Wang, Q., Zhan, L., Thompson, P. M., & Zhou, J. (2020) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/3394486.3403234 · Cited by 102 · see citing works
Emotion Recognition from Skeletal Movements
Sapiński, T., Kamińska, D., Pelikant, A., & Anbarjafari, G. (2019) · Entropy · doi:10.3390/e21070646 · Cited by 94 · see citing works
Emotion recognition and its application in software engineering
Kołakowska, A., Landowska, A., Szwoch, M., Szwoch, W., & Wróbel, M. R. (2013) · Conference on Human System Interactions · doi:10.1109/hsi.2013.6577877 · Cited by 87 · see citing works
Zhang, X., Shen, J. G., Din, Z. U., Liu, J., Wang, G., & Hu, B. (2019) · IEEE Journal of Biomedical and Health Informatics · doi:10.1109/jbhi.2019.2938247 · Cited by 90 · see citing works
Khan, M. A., Akram, T., Sharif, M. I., & Saba, T. (2020) · Multimedia Tools and Applications · doi:10.1007/s11042-020-09244-3 · Cited by 87 · see citing works
Wöllmer, M., Al-Hames, M., Eyben, F., Schuller, B. W., & Rigoll, G. (2009) · Neurocomputing · doi:10.1016/j.neucom.2009.08.005 · Cited by 86 · see citing works
Leveraging Intra and Inter Modality Relationship for Multimodal Fake News Detection
Singhal, S., Pandey, T., Mrig, S., Shah, R. R., & Kumaraguru, P. (2022) · International World Wide Web Conference (WWW) · doi:10.1145/3487553.3524650 · Cited by 86 · see citing works
Personalized Emotion Recognition by Personality-Aware High-Order Learning of Physiological Signals
Zhao, S., Gholaminejad, A., Ding, G., Gao, Y., Han, J., & Keutzer, K. (2019) · ACM Transactions on Multimedia Computing Communications and Applications · doi:10.1145/3233184 · Cited by 81 · see citing works
A survey on epistemic (model) uncertainty in supervised learning: Recent advances and applications
Zhou, X., Liu, H., Pourpanah, F., Zeng, T., & Wang, X. (2021) · Neurocomputing · doi:10.1016/j.neucom.2021.10.119 · Cited by 81 · see citing works
Modeling Emotion in Complex Stories: The Stanford Emotional Narratives Dataset
Ong, D. C., Wu, Z., Zhi‐Xuan, T., Reddan, M. C., Kahhalé, I., Mattek, A. M., & Zaki, J. (2019) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2019.2955949 · Cited by 77 · see citing works
Using Synthetic Data to Improve Facial Expression Analysis with 3D Convolutional Networks
Abbasnejad, I., Sridharan, S., Nguyen, D., Denman, S., Fookes, C., & Lucey, S. (2017) · IEEE International Conference on Computer Vision Workshops · doi:10.1109/iccvw.2017.189 · Cited by 73 · see citing works
A Survey on Human Emotion Recognition Approaches, Databases and Applications
Vinola, C., & Vimaladevi, K. (2015) · ELCVIA Electronic Letters on Computer Vision and Image Analysis · doi:10.5565/rev/elcvia.795 · Cited by 72 · see citing works
Multimodal Representation Learning by Alternating Unimodal Adaptation
Zhang, X., Yoon, J., Bansal, M., & Yao, H. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.02592 · Cited by 71 · see citing works
FPS-Net: A convolutional fusion network for large-scale LiDAR point cloud segmentation
Xiao, A., Yang, X., Lu, S., Guan, D., & Huang, J. (2021) · ISPRS Journal of Photogrammetry and Remote Sensing · doi:10.1016/j.isprsjprs.2021.04.011 · Cited by 65 · see citing works
Fusing Visual and Clinical Information for Lung Tissue Classification in HRCT Data
Depeursinge, A., Racoceanu, D., Iavindrasana, J., Cohen, G., Platon, A., Poletti, P., & Müller, H. (2009) · Cited by 59 · see citing works
Fusion Techniques for Combining Textual and Visual Information Retrieval
Depeursinge, A., & Müller, H. (2010) · The Kluwer international series on information retrieval · doi:10.1007/978-3-642-15181-1_6 · Cited by 53 · see citing works
Gaze-X: Adaptive, Affective, Multimodal Interface for Single-User Office Scenarios
Maat, L., & Pantic, M. (2007) · Lecture Notes in Computer Science · doi:10.1007/978-3-540-72348-6_13 · Cited by 44 · see citing works
Li, B., & Sano, A. (2020) · Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC) · doi:10.1109/embc44109.2020.9175463 · Cited by 43 · see citing works
Emotion Recognition Based on Multi-View Body Gestures
Shen, Z., Cheng, J., Hu, X., & Dong, Q. (2019) · International Conference on Information Photonics · doi:10.1109/icip.2019.8803460 · Cited by 35 · see citing works
Facial emotion recognition using depth data
Szwoch, M., & Pieniążek, P. (2015) · Conference on Human System Interactions · doi:10.1109/hsi.2015.7170679 · Cited by 42 · see citing works
Integrating Facial Expression and Body Gesture in Videos for Emotion Recognition
Yan, J., Zheng, W., Minhai, X. I. N., & Yan, J. (2014) · IEICE Transactions on Information and Systems · doi:10.1587/transinf.e97.d.610 · Cited by 42 · see citing works
Multimodal information fusion for selected multimedia applications
Guan, L., Wang, Y., Zhang, R., Tie, Y., Bulzacki, A., & Ibrahim, M. (2010) · International Journal of Multimedia Intelligence and Security · doi:10.1504/ijmis.2010.035969 · Cited by 34 · see citing works
Dogan, G., & Akbulut, F. P. (2023) · Neural Computing and Applications · doi:10.1007/s00521-023-09036-4 · Cited by 38 · see citing works
Emotion Recognition for Affect Aware Video Games
Szwoch, M., & Szwoch, W. (2014) · Advances in intelligent systems and computing · doi:10.1007/978-3-319-10662-5_28 · Cited by 36 · see citing works
Emotion Recognition Based on Multimodal Information
Zeng, Z., Pantić, M., & Huang, T. S. (2008) · Springer eBooks · doi:10.1007/978-1-84800-306-4_14 · Cited by 33 · see citing works
Liu, H., & Zhang, L. (2019) · Expert Systems with Applications · doi:10.1016/j.eswa.2019.04.051 · Cited by 32 · see citing works
Human Body Pose Interpretation and Classification for Social Human-Robot Interaction
McColl, D., Zhang, Z., & Nejat, G. (2011) · International Journal of Social Robotics · doi:10.1007/s12369-011-0099-6 · Cited by 32 · see citing works
Hierarchical Context Embedding for Region-Based Object Detection
Chen, Z.-M., Jin, X., Zhao, B., Wei, X.-S., & Guo, Y. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-58589-1_38 · Cited by 31 · see citing works
Maat, L., & Pantić, M. (2006) · INTERNATIONAL CONFERENCE ON MULTIMODAL INTERACTION · doi:10.1145/1180995.1181032 · Cited by 32 · see citing works
HCE: Hierarchical Context Embedding for Region-Based Object Detection
Chen, Z.-M., Jin, X., Zhao, B.-R., Zhang, X., & Guo, Y. (2021) · IEEE Transactions on Image Processing · doi:10.1109/tip.2021.3099733 · Cited by 30 · see citing works
FEEDB: A multimodal database of facial expressions and emotions
Szwoch, M. (2013) · Conference on Human System Interactions · doi:10.1109/hsi.2013.6577876 · Cited by 29 · see citing works
Towards New Mappings between Emotion Representation Models
Landowska, A. (2018) · Applied Sciences · doi:10.3390/app8020274 · Cited by 29 · see citing works
Emotion Monitoring – Verification of Physiological Characteristics Measurement Procedures
Landowska, A. (2014) · Metrology and Measurement Systems · doi:10.2478/mms-2014-0049 · Cited by 27 · see citing works
Comparison of selected off-the-shelf solutions for emotion recognition based on facial expressions
Brodny, G., Kołakowska, A., Landowska, A., Szwoch, M., Szwoch, W., & Wróbel, M. R. (2016) · Conference on Human System Interactions · doi:10.1109/hsi.2016.7529664 · Cited by 28 · see citing works
Multiscale 3D-Shift Graph Convolution Network for Emotion Recognition From Human Actions
Shi, H., Peng, W., Chen, H., Liu, X., & Zhao, G. (2022) · IEEE Intelligent Systems · doi:10.1109/mis.2022.3147585 · Cited by 26 · see citing works
Using body movement and posture for emotion detection in non-acted scenarios
Garber-Barron, M., & Si, M. (2012) · IEEE International Conference on Fuzzy Systems (FUZZ-IEEE) · doi:10.1109/fuzz-ieee.2012.6250780 · Cited by 26 · see citing works
A novel approach for emotion classification based on fusion of text and speech
Houjeij, A., Hamieh, L., Mehdi, N., & Hajj, H. M. (2012) · doi:10.1109/ictel.2012.6221211 · Cited by 26 · see citing works
Emotion Recognition in Naturalistic Speech and Language—A Survey
Weninger, F. J., Wöllmer, M., & Schuller, B. W. (2015) · Wiley eBooks · doi:10.1002/9781118910566.ch10 · Cited by 26 · see citing works
Emotion Recognition from Physiological Channels Using Graph Neural Network
Wierciński, T., Rock, M., Zwierzycki, R., Zawadzka, T., & Zawadzki, M. (2022) · Sensors · doi:10.3390/s22082980 · Cited by 26 · see citing works
Deep Learning for Scene Classification: A SurveyPreprint
Zeng, D., Liao, M., Tavakolian, M., Guo, Y., Zhou, B., Hu, D., Pietikäinen, M., & Liu, L. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2101.10531 · Cited by 25 · see citing works
DF-DM: A foundational process model for multimodal data fusion in the artificial intelligence eraPreprint
Restrepo, D., Wu, C.-W., Vásquez-Venegas, C., Nakayama, L., Celi, L., & Lopez, D. M. (2024) · Research Square · doi:10.21203/rs.3.rs-4277992/v1 · Cited by 25 · see citing works
Cross-modal dynamic convolution for multi-modal emotion recognition
Wen, H., You, S., & Fu, Y. (2021) · Journal of Visual Communication and Image Representation · doi:10.1016/j.jvcir.2021.103178 · Cited by 23 · see citing works
Sequence neural network for recommendation with multi-feature fusion
Gu, X. Q., Hai-ping, Z., & Jian, L. (2022) · Expert Systems with Applications · doi:10.1016/j.eswa.2022.118459 · Cited by 23 · see citing works
WCG-VMamba: A multi-modal classification model for corn disease
Wang, H., He, M.-F., Zhu, M., & Liu, G.-H. (2025) · Computers and Electronics in Agriculture · doi:10.1016/j.compag.2024.109835 · Cited by 24 · see citing works
Limitations of Emotion Recognition from Facial Expressions in e-Learning Context
Landowska, A., Brodny, G., & Wróbel, M. R. (2017) · International Conference on Computer Supported Education · doi:10.5220/0006357903830389 · Cited by 22 · see citing works
Multi-Modal Motion-Capture-Based Biometric Systems for Emergency Response and Patient Rehabilitation
Gavrilova, M. L., Ahmed, F., Bari, A. S. M. H., Liu, R., Liu, T., Maret, Y., Sieu, B. K., & Sudhakar, T. (2020) · IGI Global eBooks · doi:10.4018/978-1-7998-3432-8.ch032 · Cited by 22 · see citing works
Applicability of Emotion Recognition and Induction Methods to Study the Behavior of Programmers
Wróbel, M. R. (2018) · Applied Sciences · doi:10.3390/app8030323 · Cited by 21 · see citing works
Deep knowledge integration of heterogeneous features for domain adaptive SAR target recognition
Zhang, Y., Guo, X., Li, L., & Ansari, N. (2022) · Pattern Recognition · doi:10.1016/j.patcog.2022.108590 · Cited by 21 · see citing works
Limitations of Emotion Recognition in Software User Experience Evaluation Context
Landowska, A., & Miler, J. (2016) · Annals of Computer Science and Information Systems · doi:10.15439/2016f535 · Cited by 21 · see citing works
Uncertainty in emotion recognition
Landowska, A. (2019) · Journal of Information Communication and Ethics in Society · doi:10.1108/jices-03-2019-0034 · Cited by 20 · see citing works
Deep Multimodal Sequence Fusion by Regularized Expressive Representation Distillation
Guo, X., Kong, A. W., & Kot, A. C. (2022) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2022.3142448 · Cited by 20 · see citing works
Data fusion methods in multimodal human computer dialog
Yang, M., & Tao, J. (2019) · Virtual Reality & Intelligent Hardware · doi:10.3724/sp.j.2096-5796.2018.0010 · Cited by 20 · see citing works
Recognizing emotions on the basis of keystroke dynamics
Kolakowska, A. (2015) · 2015 8th International Conference on Human System Interaction (HSI) · doi:10.1109/hsi.2015.7170682 · Cited by 30 · see citing works
Automated recognition of construction worker activities using multimodal decision-level fusion
Gong, Y., Seo, J., 강경수Kang, K.-S., & Shi, M. (2025) · Automation in Construction · doi:10.1016/j.autcon.2025.106032 · Cited by 19 · see citing works
MIFI: MultI-Camera Feature Integration for Robust 3D Distracted Driver Activity Recognition
Kuang, J., Li, W., Li, F., Zhang, J., & Wu, Z. (2023) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2023.3304317 · Cited by 19 · see citing works
Bridging expert knowledge with deep learning techniques for just-in-time defect prediction
Zhou, X., Han, D., & Lo, D. (2024) · Empirical Software Engineering · doi:10.1007/s10664-024-10591-0 · Cited by 23 · see citing works
Machine Understanding of Human Behavior
Pantić, M., Pentland, A., Nijholt, A., & Huang, T. S. (2007) · University of Twente Research Information · Cited by 19 · see citing works
Attending to Emotional Narratives
Wu, Z., Zhang, X., Zhi‐Xuan, T., Zaki, J., & Ong, D. C. (2019) · International Conference on Affective Computing and Intelligent Interaction and workshops · doi:10.1109/acii.2019.8925497 · Cited by 20 · see citing works
Multimodal Pain Recognition in Postoperative Patients: Machine Learning Approach
Subramanian, A., Cao, R., Naeini, E. K., Aqajari, S. A. H., Hughes, T. D., Calderon, M.-D., Zheng, K., Dutt, N. D., Liljeberg, P., Salanterä, S., et al. (2024) · JMIR Formative Research · doi:10.2196/67969 · Cited by 18 · see citing works
Clapés, A., Pardo, À., Pujol, O., & Escalera, S. (2018) · Machine Vision and Applications · doi:10.1007/s00138-018-0931-1 · Cited by 16 · see citing works
Emotion Recognition Using Physiological Signals
Szwoch, W. (2015) · doi:10.1145/2814464.2814479 · Cited by 16 · see citing works
Creating and Annotating Affect Databases from Face and Body Display: A Contemporary Survey
Güneş, H., & Piccardi, M. (2006) · Conference proceedings/Conference proceedings - IEEE International Conference on Systems, Man, and Cybernetics · doi:10.1109/icsmc.2006.385227 · Cited by 16 · see citing works
Self-citationSubject-adaptive Integration of Multiple SICE Brain Networks with Different Sparsity
Zhang, J., Zhou, L., & Wang, L. (2016) · Pattern Recognition · doi:10.1016/j.patcog.2016.09.024 · Cited by 15 · see citing works
Zhou, X., Han, D., & Lo, D. (2022) · IEEE International Conference on Program Comprehension (ICPC) · doi:10.1145/3524610.3527910 · Cited by 14 · see citing works
Emotion Monitor - Concept, Construction and Lessons Learned
Landowska, A. (2015) · Annals of Computer Science and Information Systems · doi:10.15439/2015f264 · Cited by 14 · see citing works
On Facial Expressions and Emotions RGB-D Database
Szwoch, M. (2014) · Communications in computer and information science · doi:10.1007/978-3-319-06932-6_37 · Cited by 14 · see citing works
Guo, Y., Yong-ping, Z., Xiong, X., Jiang, X., Tian, H., & Zhang, Q. (2023) · IEEE Access · doi:10.1109/access.2023.3299822 · Cited by 13 · see citing works
Emotion Recognition From Full-Body Motion Using Multiscale Spatio-Temporal Network
Wang, T., Liu, S., He, F., Dai, W., Du, M., Ke, Y., & Ming, D. (2023) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2023.3305197 · Cited by 13 · see citing works
Survey of Automated Methods for Nonverbal Behavior Analysis in Parent-Child Interactions
Karaca, B., Salah, A. A., Denissen, J. J. A., Poppe, R. W., & de Zwarte, S. M. C. (2024) · International Conference on Automatic Face and Gesture Recognition · doi:10.1109/fg59268.2024.10582009 · Cited by 14 · see citing works
Methodology of Affective Intervention Design for Intelligent Systems
Landowska, A., Szwoch, M., & Szwoch, W. (2016) · Interacting with Computers · doi:10.1093/iwc/iwv047 · Cited by 13 · see citing works
Unimodal late fusion for NIST i ‐vector challenge on speaker detection
Ali, H., Garcez, A. D., Tran, S. N., Zhou, X., & Iqbal, K. (2014) · Electronics Letters · doi:10.1049/el.2014.1207 · Cited by 12 · see citing works
Recognition of emotions by the emotional feedback through behavioral human poses
Rázuri, J. G., Larsson, A., Sundgren, D., Bonet, I., & Morán, A. (2015) · DiVA (Stockholm University) · Cited by 12 · see citing works
Multi-Modal Motion-Capture-Based Biometric Systems for Emergency Response and Patient Rehabilitation
Gavrilova, M. L., Ahmed, F., Bari, A. S. M. H., Liu, R., Liu, T., Maret, Y., Sieu, B. K., & Sudhakar, T. (2018) · Advances in medical technologies and clinical practice book series · doi:10.4018/978-1-5225-7525-2.ch007 · Cited by 12 · see citing works
Robust Multi-Feature Learning for Skeleton-Based Action Recognition
Wang, Y., Xu, Z., Li, L., & Yao, J. (2019) · IEEE Access · doi:10.1109/access.2019.2945632 · Cited by 12 · see citing works
Khanmohammadi, S., Golafshani, E. M., Bai, Y., Li, H., Bazli, M., & Arashpour, M. (2023) · International Journal of Disaster Risk Reduction · doi:10.1016/j.ijdrr.2023.103972 · Cited by 12 · see citing works
Mood Swings: design and evaluation of affective interactive art
Bialoskorski, L. S. S., Westerink, J. H. D. M., & van den Broek, E. L. (2009) · New Review of Hypermedia and Multimedia · doi:10.1080/13614560903131898 · Cited by 12 · see citing works
Verbal Behaviors and Persuasiveness in Online Multimedia Content
Chatterjee, M., Park, S., Shim, H. S., Sagae, K., & Morency, L. (2014) · doi:10.3115/v1/w14-5908 · Cited by 12 · see citing works
Soft Voting Strategy for Multi-Modal Emotion Recognition Using Deep-learning- Facial Images and EEG
Chinta, U., Kalita, J. K., & Atyabi, A. (2023) · IEEE Annual Computing and Communication Workshop and Conference (CCWC) · doi:10.1109/ccwc57344.2023.10099070 · Cited by 11 · see citing works
Liu, I., Liu, F., Zhong, Q., Ma, F., & Ni, S. (2024) · PeerJ Computer Science · doi:10.7717/peerj-cs.1912 · Cited by 12 · see citing works
Intelligence methods of multi-modal information fusion in human-computer interaction
Yang, M., & Tao, J. (2018) · Scientia Sinica Informationis · doi:10.1360/n112017-00211 · Cited by 10 · see citing works
Integration of multiple network views in Wikipedia
Wu, G., & Cunningham, P. (2014) · Knowledge and Information Systems · doi:10.1007/s10115-014-0802-7 · Cited by 9 · see citing works
Towards Emotion Acquisition in IT Usability Evaluation Context
Landowska, A. (2015) · doi:10.1145/2814464.2814470 · Cited by 9 · see citing works
- Authors
- Gunes, H., & Piccardi, M.
- Affiliations
- University of Technology Sydney (AU)
- ORCID
- Piccardi, M.
- Venue
- 2005 IEEE International Conference on Systems, Man and Cybernetics
- Publisher
- IEEE
- Published
- 2006-01-18
- Volume
- 4
- Pages
- 3437-3443
- ISSN
- 1062-922X (ISSN-L)
- OpenAlex
- W1587447546
- Cited by
- 191
- References
- 29
- Citations by year
- 2026: 8 · 2025: 9 · 2024: 17 · 2023: 13 · 2022: 14 · 2021: 17 · 2020: 14 · 2019: 21 · 2018: 13 · 2017: 5 · 2016: 6 · 2015: 14 · 2014: 7 · 2013: 6 · 2012: 3
- FWCI
- 11.27
- Type
- conference paper
- Language
- en
- Open access
- green
- hdl.handle.net
- Locations
- Conference proceedings/Conference proceedings - IEEE International Conference on Systems, Man, and Cybernetics (publishedVersion)
- UTS ePRESS (University of Technology Sydney) (submittedVersion)
- Topics
- Social Sciences › Psychology › Experimental and Cognitive Psychology › Emotion and Mood Recognition
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Face and Expression Recognition
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Face recognition and analysis
- Keywords
- affect recognition, visual emotion recognition, facial expressions, body gestures, multimodal fusion, feature-level fusion, decision fusion
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review