Detecting and Addressing Frustration in a Serious Game for Military Training
Open accessThe DOI matches its Crossref record; 16 of the 16 references it cites were found in a registry or an index.
Abstract
Tutoring systems that are sensitive to affect show considerable promise for enhancing student learning experiences. Creating successful affective responses requires considerable effort both to detect student affect and to design appropriate responses to affect. Recent work has suggested that affect detection is more effective when both physical sensors and interaction logs are used, and that context-sensitive design of affective feedback is necessary to enhance engagement and improve learning. In this paper, we provide a comprehensive report on a multi-part study that integrates detection, validation, and intervention into a unified approach. This paper examines the creation of both sensor-based and interaction-based detectors of student affect, producing successful detectors of student affect. In addition, it reports results from an investigation of motivational feedback messages designed to address student frustration, and investigates whether linking these interventions to detectors improves outcomes. Our results are mixed, finding that self-efficacy enhancing interventions based on interaction-based affect detectors enhance outcomes in one of two experiments investigating affective interventions. This work is conducted in the context of the GIFT framework for intelligent tutoring, and the TC3Sim game-based simulation that provides training for first responder skills.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
The meaning and use of the area under a receiver operating characteristic (ROC) curve.
Hanley, J. A., & McNeil, B. J. (1982) · Radiology · doi:10.1148/radiology.143.1.7063747 · Cited by 21,856 · cited 2×
“Detector performance was evaluated using two metrics: Cohen’s Kappa (Cohen 1960) and A’ computed as the Wilcoxon statistic (Hanley and McNeil 1982).”
MethodMeasuring Presence in Virtual Environments: A Presence Questionnaire
Witmer, B. G., & Singer, M. J. (1998) · Presence: Teleoperators and Virtual Environments · doi:10.1162/105474698565686 · Cited by 6,371 · cited 3×
“In addition, participants completed two questionnaires about their experiences in TC3Sim: the Intrinsic Motivation Inventory (IMI) (Ryan 1982) and Presence Questionnaire (Witmer and Singer 1998; Witmer et al. 2005).”
MethodInfluentialDevelopment and validation of the short grit scale (grit-s)
Duckworth, A. L., & Quinn, P. D. (2009) · Journal of Personality Assessment · doi:10.1080/00223890802634290 · Cited by 3,376
Feedback and Self-Regulated Learning: A Theoretical Synthesis
Butler, D. L., & Winne, P. H. (1995) · Review of Educational Research · doi:10.3102/00346543065003245 · Cited by 3,334
A Survey of Affect Recognition Methods: Audio, Visual, and Spontaneous Expressions
Zeng, Z., Pantic, M., Roisman, G. I., & Huang, T. S. (2009) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2008.52 · Cited by 2,797
Control and information in the intrapersonal sphere: An extension of cognitive evaluation theory.
Ryan, R. M. (1982) · Journal of Personality and Social Psychology · doi:10.1037/0022-3514.43.3.450 · Cited by 2,721 · cited 3×
“In addition, participants completed two questionnaires about their experiences in TC3Sim: the Intrinsic Motivation Inventory (IMI) (Ryan 1982) and Presence Questionnaire (Witmer and Singer 1998; Witmer et al. 2005).”
BackgroundAffect Detection: An Interdisciplinary Review of Models, Methods, and Their Applications
Calvo, R. A., & D'Mello, S. (2010) · IEEE Transactions on Affective Computing · doi:10.1109/t-affc.2010.1 · Cited by 1,893 · cited 1×
“Research on affective learning environments has enabled the development of models that automatically detect learner affect using a wide variety of data modalities; an extensive review can be seen in Calvo and D'Mello (2010).”
BackgroundBaker, R. S. J. D., D'Mello, S. K., Rodrigo, M. M. T., & Graesser, A. C. (2010) · International Journal of Human-Computer Studies · doi:10.1016/j.ijhcs.2009.12.003 · Cited by 936 · cited 1×
“In conclusion, the results of this experiment support previous theories and empirical research that have recognized the need to identify and address affective states that lead to disengagement in learning (Baker et al. 2010; D'Mello and Graesser 2011; D’Mello et al. 2013).”
ResultSelf-citationAchievement goals and discrete achievement emotions: A theoretical model and prospective test.
Pekrun, R., Elliot, A. J., & Maier, M. A. (2006) · Journal of Educational Psychology · doi:10.1037/0022-0663.98.3.583 · Cited by 900
The Factor Structure of the Presence Questionnaire
Witmer, B. G., Jerome, C. J., & Singer, M. J. (2005) · Presence: Teleoperators and Virtual Environments · doi:10.1162/105474605323384654 · Cited by 770 · cited 1×
“In addition, participants completed two questionnaires about their experiences in TC3Sim: the Intrinsic Motivation Inventory (IMI) (Ryan 1982) and Presence Questionnaire (Witmer and Singer 1998; Witmer et al. 2005).”
Affect and learning: An exploratory look into the role of affect in learning with AutoTutor
Craig, S., Graesser, A., Sullins, J., & Gholson, B. (2004) · Journal of Educational Media · doi:10.1080/1358165042000283101 · Cited by 630 · cited 4×
“Some affective states have relatively uncomplicated relationships with student learning outcomes – for example, engaged concentration appears to be positively associated (Craig et al. 2004; Pardos et al. 2014) while boredom is negatively associated (Craig et al. 2004; Pardos et al. 2014).”
BackgroundInfluentialAttitude construction: evaluation in context
Schwarz, N. (2007) · Social Cognition · doi:10.1521/soco.2007.25.5.638 · Cited by 627 · cited 1×
“Social identity is aligned with the situated social cognition perspective that proposes cognition and action are not discrete entities but dynamically shaped by each other (Schwarz 2007, 2009; Smith and Semin 2004, 2007).”
BackgroundAutomatic prediction of frustration
Kapoor, A., Burleson, W., & Picard, R. W. (2007) · International Journal of Human-Computer Studies · doi:10.1016/j.ijhcs.2007.02.003 · Cited by 592 · cited 4×
“feedback, requires finding ways to help the learner recover, manage, and persist through frustration to persist in their learning tasks (Kapoor et al. 2007).”
BackgroundInfluentialIvon, A., Cooper David, G., Winslow, B., Park, W. B., Kasia, M., & Robert, C. (2009) · Frontiers in Artificial Intelligence and Applications · doi:10.3233/978-1-60750-028-5-17 · Cited by 382 · cited 5×
“Sensor-based detectors of affect have been developed using a range of physical indicators including facial expressions (Arroyo et al. 2009; Bosch et al. 2015), voice (Zeng et al. 2009), posture (D'Mello and Graesser 2010; Grafsgaard et al. 2012), physiological data (Nasoz et al. 2004), smart phone…”
BackgroundMethodInfluentialAutomatic analysis of affective postures and body motion to detect engagement with a game companion,
Sanghvi, J., Castellano, G., Leite, I., Pereira, A., McOwan, P. W., & Paiva, A. (2011) · Proceedings of the 6th international conference on Human-robot interaction · doi:10.1145/1957656.1957781 · Cited by 250 · cited 1×
“One direction for addressing these sources of variation in the future is to investigate alternate positioning for the Kinect sensors, including use of side angles (Sanghvi et al. 2011).”
BackgroundAutomatic Detection of Learning-Centered Affective States in the Wild
Bosch, N., D'Mello, S., Baker, R., Ocumpaugh, J., Shute, V., Ventura, M., Wang, L., & Zhao, W. (2015) · Proceedings of the 20th International Conference on Intelligent User Interfaces · doi:10.1145/2678025.2701397 · Cited by 178 · cited 5×
“Sensor-based detectors of affect have been developed using a range of physical indicators including facial expressions (Arroyo et al. 2009; Bosch et al. 2015), voice (Zeng et al. 2009), posture (D'Mello and Graesser 2010; Grafsgaard et al. 2012), physiological data (Nasoz et al. 2004), smart phone…”
BackgroundMethodInfluentialSelf-citation
Cited by
Mining Big Data in Education: Affordances and Challenges
Fischer, C., Pardos, Z. A., Baker, R. S., Williams, J. J., Smyth, P., Yu, R., Slater, S., Baker, R., & Warschauer, M. (2020) · Review of Research in Education · doi:10.3102/0091732x20903304 · Cited by 412
Self-citationOut of the laboratory and into the classroom: the future of artificial intelligence in education
Schiff, D. (2021) · AI & SOCIETY · doi:10.1007/s00146-020-01033-8 · Cited by 334
Proactive and reactive engagement of artificial intelligence methods for education: a review
Mallik, S., & Gangopadhyay, A. (2023) · Frontiers in Artificial Intelligence · doi:10.3389/frai.2023.1151391 · Cited by 139
“[140] compares the usage of sensor and interaction data in delivering motivational prompts in the course of military training.”
BackgroundHutt, S., Krasich, K., R. Brockmole, J., & K. D'Mello, S. (2021) · Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3411764.3445269 · Cited by 60
“Intervention design must acknowledge that MW detection is imperfect (see [3, 19, 34, 43]).”
Background
Metadata
- Authors
- DeFalco, J. A., Rowe, J. P., Paquette, L., Georgoulas-Sherry, V., Brawner, K., Mott, B. W., Baker, R. S., & Lester, J. C.
- Venue
- International Journal of Artificial Intelligence in Education
- Published
- 2017-09-12
- Volume
- 28
- Issue
- 2
- Pages
- 152-193
- ISSN
- 1560-4292 (ISSN-L)
- 1560-4306
- OpenAlex
- W2756344104
- Cited by
- 107
- References
- 16
- FWCI
- 5.25
- Type
- journal article
- Language
- en
- Open access
- hybrid
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.