The DOI matches its Crossref record; 53 of the 53 references it cites were found in a registry or an index.
The Faces of Engagement: Automatic Recognition of Student Engagementfrom Facial Expressions
Abstract
Student engagement is a key concept in contemporary education, where it is valued as a goal in its own right. In this paper we explore approaches for automatic recognition of engagement from students' facial expressions. We studied whether human observers can reliably judge engagement from the face; analyzed the signals observers use to make these judgments; and automated the process using machine learning. We found that human observers reliably agree when discriminating low versus high degrees of engagement (Cohen's κ = 0.96). When fine discrimination is required (four distinct levels) the reliability decreases, but is still quite high ( κ = 0.56). Furthermore, we found that engagement labels of 10-second video clips can be reliably predicted from the average labels of their constituent frames (Pearson r=0.85), suggesting that static expressions contain the bulk of the information used by observers. We used machine learning to develop automatic engagement detectors and found that for binary classification (e.g., high engagement versus low engagement), automated engagement detectors perform with comparable accuracy to humans. Finally, we show that both human and automatic engagement judgments correlate with task performance. In our experiment, student post-test performance was predicted with comparable accuracy from engagement labels ( r=0.47) as from pre-test scores ( r=0.44).
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
A decision-theoretic generalization of on-line learning and an application to boosting
Freund, Y., & Schapire, R. E. (1997) · Journal of Computer and System Sciences · doi:10.1006/jcss.1997.1504 · Cited by 20,466
Robust Real-Time Face Detection
Viola, P., & Jones, M. J. (2004) · International Journal of Computer Vision · doi:10.1023/b:visi.0000013087.49260.fb · Cited by 14,097
School Engagement: Potential of the Concept, State of the Evidence
Fredricks, J. A., Blumenfeld, P. C., & Paris, A. H. (2004) · Review of Educational Research · doi:10.3102/00346543074001059 · Cited by 12,581
Friedman, J., Hastie, T., & Tibshirani, R. (2000) · The Annals of Statistics · doi:10.1214/aos/1016218223 · Cited by 6,884
Ekman, P., & Friesen, W. V. (2019) · PsycTESTS Dataset · doi:10.1037/t27734-000 · Cited by 6,019
Lucey, P., Cohn, J. F., Kanade, T., Saragih, J., Ambadar, Z., & Matthews, I. (2010) · 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition - Workshops · doi:10.1109/cvprw.2010.5543262 · Cited by 4,303
Acquisition of cognitive skill
Anderson, J. R. (1982) · Psychological Review · doi:10.1037/0033-295x.89.4.369 · Cited by 3,663
Dynamic Texture Recognition Using Local Binary Patterns with an Application to Facial Expressions
Zhao, G., & Pietikainen, M. (2007) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2007.1110 · Cited by 2,846
Comprehensive database for facial expression analysis
Kanade, T., Cohn, J. F., & Tian, Y. (2002) · Proceedings Fourth IEEE International Conference on Automatic Face and Gesture Recognition (Cat. No. PR00580) · doi:10.1109/afgr.2000.840611 · Cited by 2,591
A desicion-theoretic generalization of on-line learning and an application to boosting
Freund, Y., & Schapire, R. E. (1995) · Lecture Notes in Computer Science · doi:10.1007/3-540-59119-2_166 · Cited by 2,252
Cited by
Application and theory gaps during the rise of artificial intelligence in education
Chen, X., Xie, H., Zou, D., & Hwang, G.-J. (2020) · Computers and Education: Artificial Intelligence · doi:10.1016/j.caeai.2020.100002 · Cited by 1,003
Advanced, Analytic, Automated (AAA) Measurement of Engagement During Learning
D'Mello, S., Dieterle, E., & Duckworth, A. (2017) · Educational Psychologist · doi:10.1080/00461520.2017.1281747 · Cited by 350
Engagement detection in online learning: a review
Dewan, M. A. A., Murshed, M., & Lin, F. (2019) · Smart Learning Environments · doi:10.1186/s40561-018-0080-z · Cited by 350
Savchenko, A. V., Savchenko, L. V., & Makarov, I. (2022) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2022.3188390 · Cited by 346
Automated Detection of Engagement Using Video-Based Estimation of Facial Expressions and Heart Rate
Monkaresi, H., Bosch, N., Calvo, R. A., & D'Mello, S. K. (2017) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2016.2515084 · Cited by 329
Chen, X., Xie, H., & Hwang, G.-J. (2020) · Computers and Education: Artificial Intelligence · doi:10.1016/j.caeai.2020.100005 · Cited by 303
Microexpression Identification and Categorization Using a Facial Dynamics Map
Xu, F., Zhang, J., & Wang, J. Z. (2017) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2016.2518162 · Cited by 283
Goldberg, P., Sümer, Ö., Stürmer, K., Wagner, W., Göllner, R., Gerjets, P., Kasneci, E., & Trautwein, U. (2021) · Educational Psychology Review · doi:10.1007/s10648-019-09514-z · Cited by 280
Computation Offloading for Service Workflow in Mobile Cloud Computing
Deng, S., Huang, L., Taheri, J., & Zomaya, A. Y. (2015) · IEEE Transactions on Parallel and Distributed Systems · doi:10.1109/tpds.2014.2381640 · Cited by 276
Automatic Detection of Learning-Centered Affective States in the Wild
Bosch, N., D'Mello, S., Baker, R., Ocumpaugh, J., Shute, V., Ventura, M., Wang, L., & Zhao, W. (2015) · Proceedings of the 20th International Conference on Intelligent User Interfaces · doi:10.1145/2678025.2701397 · Cited by 178
Yang, J., Wang, K., Peng, X., & Qiao, Y. (2018) · Proceedings of the 20th ACM International Conference on Multimodal Interaction · doi:10.1145/3242969.3264981 · Cited by 79
On Incorporating Affective Support to an Intelligent Tutoring System: an Empirical Study
Cunha-Perez, C., Arevalillo-Herraez, M., Marco-Gimenez, L., & Arnau, D. (2018) · IEEE Revista Iberoamericana de Tecnologias del Aprendizaje · doi:10.1109/rita.2018.2831760 · Cited by 16
Gupta, S., Kumar, P., & Tekchandani, R. (2024) · Multimedia Tools and Applications · doi:10.1007/s11042-023-18021-x · Cited by 14
Modeling User’s Social Attitude in a Conversational System
Baur, T., Schiller, D., & André, E. (2016) · Human–Computer Interaction Series · doi:10.1007/978-3-319-31413-6_10 · Cited by 5
Han, B., Ma, Y., Coghlan, S., McKay, D., Buchanan, G., & Smith, W. (2026) · Proceedings of the 2026 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3772318.3790360 · Cited by 3
“This potentially offers rich situational information on student engagement [133] and real-time learning experiences, informing tools such as teacher-facing dashboards (e.g., EmoDash [42]).”
Relating Children’s Automatically Detected Facial Expressions to Their Behavior in RoboTutor
Saxena, M., Pillai, R., & Mostow, J. (2018) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v32i1.12190 · Cited by 2
Metadata
- Authors
- Whitehill, J., Serpell, Z., Lin, Y.-C., Foster, A., & Movellan, J. R.
- Venue
- IEEE Transactions on Affective Computing
- Published
- 2014-01-01
- Volume
- 5
- Issue
- 1
- Pages
- 86-98
- ISSN
- 1949-3045 (ISSN-L)
- OpenAlex
- W2081112272
- Cited by
- 677
- References
- 53
- FWCI
- 28.83
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.