The DOI matches its Crossref record; 14 of the 14 references it cites were found in a registry or an index.
Multimodal affect recognition in learning environments
Abstract
We propose a multi-sensor affect recognition system and evaluate it on the challenging task of classifying interest (or disinterest) in children trying to solve an educational puzzle on the computer. The multimodal sensory information from facial expressions and postural shifts of the learner is combined with information about the learner's activity on the computer. We propose a unified approach, based on a mixture of Gaussian Processes, for achieving sensor fusion under the problematic conditions of missing channels and noisy labels. This approach generates separate class labels corresponding to each individual modality. The final classification is based upon a hidden random variable, which probabilistically combines the sensors. The multimodal Gaussian Process approach achieves accuracy of over 86%, significantly outperforming classification using the individual modalities, and several other combination schemes.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Adaptive Mixtures of Local Experts
Jacobs, R. A., Jordan, M. I., Nowlan, S. J., & Hinton, G. E. (1991) · Neural Computation · doi:10.1162/neco.1991.3.1.79 · Cited by 5,384
Kittler, J., Hatef, M., Duin, R. P. W., & Matas, J. (1998) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/34.667881 · Cited by 5,290
Toward machine emotional intelligence: analysis of affective physiological state
Picard, R. W., Vyzas, E., & Healey, J. (2001) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/34.954607 · Cited by 2,334 · cited 1×
“Exceptions include the work of Picard et al.[ 15 ], which achieved 81% classication accuracy of eight emotional states of an individual over many days of data, based on four physiological signals, and several eorts that have combined audio of the voice with video of the face, e.g.”
Self-citationExpectation Propagation for approximate Bayesian inference
Minka, T. P. (2013) · arXiv (Cornell University) · doi:10.48550/arxiv.1301.2294 · Cited by 1,392
Toward an affect-sensitive multimodal human-computer interaction
Pantic, M., & Rothkrantz, L. J. M. (2003) · Proceedings of the IEEE · doi:10.1109/jproc.2003.817122 · Cited by 791
Probabilistic assessment of user's emotions in educational games
Conati, C. (2002) · Applied Artificial Intelligence · doi:10.1080/08839510290030390 · Cited by 498
Layered representations for learning and inferring office activity from multiple sensory channels
Oliver, N., Garg, A., & Horvitz, E. (2004) · Computer Vision and Image Understanding · doi:10.1016/j.cviu.2004.02.004 · Cited by 316
Automated Posture Analysis for Detecting Learner's Interest Level
Mota, S., & Picard, R. W. (2003) · 2003 Conference on Computer Vision and Pattern Recognition Workshop · doi:10.1109/cvprw.2003.10047 · Cited by 314
Self-citationIntelligent Tutoring Systems: At the Crossroads of Artificial Intelligence and Education
Frasson, C., & Gauthier, G. (1990) · Greenwood Publishing Group Inc. eBooks · Cited by 172
Towards a Learning Companion that Recognizes Affect
Kapoor, A., Mota, S., & Picard, R. W. (2001) · National Conference on Artificial Intelligence · Cited by 127
Self-citationProbabilistic combination of multiple modalities to detect interest
Kapoor, A., Picard, R. W., & Ivanov, Y. A. (2004) · Proceedings - International Conference on Pattern Recognition/Proceedings/International Conference on Pattern Recognition · doi:10.1109/icpr.2004.1334690 · Cited by 100
Self-citationReal-time, fully automatic upper facial feature tracking
Kapoor, A., & Picard, R. W. (2003) · International Conference on Automatic Face and Gesture Recognition · doi:10.1109/afgr.2002.1004123 · Cited by 83
Self-citationCritic-driven ensemble classification
Miller, D. J., & Yan, L. (1999) · IEEE Transactions on Signal Processing · doi:10.1109/78.790663 · Cited by 50
Gomes, E. R., Silveira, R. A., & Vicari, R. M. (2006) · Lecture notes in computer science · doi:10.1007/11774303 · Cited by 35
Cited by
A Survey of Affect Recognition Methods: Audio, Visual, and Spontaneous Expressions
Zeng, Z., Pantic, M., Roisman, G. I., & Huang, T. S. (2009) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2008.52 · Cited by 2,797
Affect Detection: An Interdisciplinary Review of Models, Methods, and Their Applications
Calvo, R. A., & D'Mello, S. (2010) · IEEE Transactions on Affective Computing · doi:10.1109/t-affc.2010.1 · Cited by 1,893
A review of affective computing: From unimodal analysis to multimodal fusion
Poria, S., Cambria, E., Bajpai, R., & Hussain, A. (2017) · Information Fusion · doi:10.1016/j.inffus.2017.02.003 · Cited by 1,644
The Faces of Engagement: Automatic Recognition of Student Engagementfrom Facial Expressions
Whitehill, J., Serpell, Z., Lin, Y.-C., Foster, A., & Movellan, J. R. (2014) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2014.2316163 · Cited by 677
A Review and Meta-Analysis of Multimodal Affect Detection Systems
D'mello, S. K., & Kory, J. (2015) · ACM Computing Surveys · doi:10.1145/2682899 · Cited by 628
Metadata
- Authors
- Kapoor, A., & Picard, R. W.
- Venue
- Proceedings of the 13th annual ACM international conference on Multimedia
- Publisher
- ACM
- Published
- 2005-11-06
- Pages
- 677-682
- OpenAlex
- W2109626108
- Cited by
- 339
- References
- 14
- FWCI
- 28.24
- Type
- conference paper
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.