Towards Emotion Detection in Educational Scenarios from Facial Expressions and Body Movements through Multimodal Approaches
Open accessThe DOI matches its Crossref record; 9 of the 9 references it cites were found in a registry or an index.
Abstract
We report current findings when considering video recordings of facial expressions and body movements to provide affective personalized support in an educational context from an enriched multimodal emotion detection approach. In particular, we describe an annotation methodology to tag facial expression and body movements that conform to changes in the affective states of learners while dealing with cognitive tasks in a learning process. The ultimate goal is to combine these annotations with additional affective information collected during experimental learning sessions from different sources such as qualitative, self-reported, physiological, and behavioral information. These data altogether are to train data mining algorithms that serve to automatically identify changes in the learners’ affective states when dealing with cognitive tasks which help to provide emotional personalized support.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Measuring emotion: The self-assessment manikin and the semantic differential
Bradley, M. M., & Lang, P. J. (1994) · Journal of Behavior Therapy and Experimental Psychiatry · doi:10.1016/0005-7916(94)90063-9 · Cited by 9,538 · cited 1×
“…learning activity were complemented with emotional feedback collected directly from learners in terms of the Self-Assessment Manikin (SAM) scale [56] to measure emotions in a dimensional space [57], and emotional reports filled by the participant when the tasks were finished so they could…”
MethodA Multimodal Database for Affect Recognition and Implicit Tagging
Soleymani, M., Lichtenauer, J., Pun, T., & Pantic, M. (2012) · IEEE Transactions on Affective Computing · doi:10.1109/t-affc.2011.25 · Cited by 1,664
Mehrabian, A. (1996) · Current Psychology · doi:10.1007/bf02686918 · Cited by 1,493
Cardiovascular, electrodermal, and respiratory response patterns to fear‐ and sadness‐inducing films
Kreibig, S. D., Wilhelm, F. H., Roth, W. T., & Gross, J. J. (2007) · Psychophysiology · doi:10.1111/j.1469-8986.2007.00550.x · Cited by 495
Black, M. J., & Yacoob, Y. (2002) · Proceedings of IEEE International Conference on Computer Vision · doi:10.1109/iccv.1995.466915 · Cited by 457 · cited 1×
“…of optical flow [18], face identification, pose recovery and facial expression [19], models of image motion to recover nonrigid facial motion [20], active contours [21], robust appearance filter [22], probabilistic tracking [23], active [24] and adaptive [25] appearance model, image-based…”
BackgroundExpert system for automatic analysis of facial expressions
Pantic, M., & Rothkrantz, L. J. M. (2000) · Image and Vision Computing · doi:10.1016/s0262-8856(00)00034-2 · Cited by 415 · cited 1×
“Facial expressions, body movements, and physiological reactions are considered as elements of the nonverbal communication forms [5].”
BackgroundCohen, I., Sebe, N., Gozman, F. G., Cirelo, M. C., & Huang, T. S. (2003) · 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2003. Proceedings. · doi:10.1109/cvpr.2003.1211408 · Cited by 155
Facial expression recognition – A real time approach
Geetha, A., Ramalingam, V., Palanivel, S., & Palaniappan, B. (2009) · Expert Systems with Applications · doi:10.1016/j.eswa.2007.09.002 · Cited by 57 · cited 1×
“…research work, different algorithms have been applied to facial expression recognition, such as Hidden Markov Models [28], Support Vector Machines [29], Neural Networks [30], Bayesian Network Classifiers [31], using other algorithms as Principal Component Analysis [32], and Linear Discriminant…”
MethodInvolving Users to Improve the Collaborative Logical Framework
Santos, O. C., & Boticario, J. G. (2014) · The Scientific World Journal · doi:10.1155/2014/893525 · Cited by 25 · cited 2×
“The other one followed a collaborative approach [48].”
BackgroundSelf-citation
Research works printed but not linked: 32
- 2012, Bulletin of the IEEE Technical Committee on Learning Technology, 14, 35
- 10.1007/978-3-642-39112-5_102 · DOI
- 2014, An approach for an affective educational recommendation model
- 2009, Educational Technology & Society, 12, 176
- 2006, Journal of Mutimedia, 1
- 2005, Measuring facial action by manual coding, facial EMG, and automatic facial image analysis, 9
- 10.1016/j.neunet.2005.03.004 · DOI · Paper page
- 1991, IEICE Transactions on Information and Systems, 74, 3474
- 10.1109/TIP.2003.821445 · DOI · Paper page
- 10.1109/TPAMI.2004.45 · DOI · Paper page
- 10.1109/TIP.2005.854473 · DOI
- 10.1007/978-3-642-38628-2_90 · DOI
- 10.1109/TSMCB.2004.825930 · DOI · Paper page
- 10.1016/S0923-5965(02)00076-0 · DOI
- 10.1016/S0167-8655(02)00371-9 · DOI
- 10.1016/j.intcom.2006.04.003 · DOI · Paper page
- 10.1073/pnas.0507650102 · DOI · Paper page
- 10.1016/j.neunet.2008.05.003 · DOI · Paper page
- 10.1007/978-3-540-74889-2_5 · DOI
- 1982, Measurement of body motion and orientation, 199
- 10.1007/s10919-012-0130-0 · DOI · Paper page
- 10.1109/T-AFFC.2011.7 · DOI · Paper page
- 2005, Proxemics, kinesics, and gaze, 137
- 2001, Infancy, 2, 419, 10.1207/S15327078IN0204_02 · DOI · Paper page
- 2011, Educational Technology & Society, 14, 21
- 10.1007/978-3-642-39188-0_61 · DOI
- 2007, Why using dotLRN? UNED use cases, 195
- 1988, Journal of Personality and Social Psychology, 54, 1063, 10.1037/0022-3514.54.6.1063 · DOI · Paper page
- 10.1007/978-3-642-13388-6_33 · DOI
- 10.1007/s10919-012-0130-0 · DOI · Paper page
- 10.1016/j.knosys.2013.04.017 · DOI
- 10.1111/exsy.12041 · DOI
Cited by
Human Emotion Recognition: Review of Sensors and Methods
Dzedzickis, A., Kaklauskas, A., & Bucinskas, V. (2020) · Sensors · doi:10.3390/s20030592 · Cited by 582
Engagement detection in online learning: a review
Dewan, M. A. A., Murshed, M., & Lin, F. (2019) · Smart Learning Environments · doi:10.1186/s40561-018-0080-z · Cited by 350
Shoumy, N. J., Ang, L.-M., Seng, K. P., Rahaman, D. M. M., & Zia, T. (2020) · Journal of Network and Computer Applications · doi:10.1016/j.jnca.2019.102447 · Cited by 171
Multimodal Emotion Recognition Using Deep Neural Networks
Tang, H., Liu, W., Zheng, W.-L., & Lu, B.-L. (2017) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-70093-9_86 · Cited by 194
IEEE 7010: A New Standard for Assessing the Well-being Implications of Artificial Intelligence
Schiff, D., Ayesh, A., Musikanski, L., & Havens, J. C. (2020) · 2020 IEEE International Conference on Systems, Man, and Cybernetics (SMC) · doi:10.1109/smc42975.2020.9283454 · Cited by 62
Metadata
- Authors
- Saneiro, M., Santos, O. C., Salmeron-Majadas, S., & Boticario, J. G.
- Venue
- The Scientific World Journal
- Publisher
- Wiley
- Published
- 2014
- Volume
- 2014
- Pages
- 1-14
- ISSN
- 1537-744X (ISSN-L)
- 2356-6140
- OpenAlex
- W1997400788
- Cited by
- 91
- References
- 9
- FWCI
- 9.31
- Type
- journal article
- Language
- en
- Open access
- gold
- Funders
- Spanish Ministry of Economy and Competitiveness (TIN2011-29221-C03-01)
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.