Detecting and Addressing Frustration in a Serious Game for Military Training
Open accessThe DOI matches its Crossref record; 14 of the 14 references it cites were found in a registry or an index.
Abstract
Tutoring systems that are sensitive to affect show considerable promise for enhancing student learning experiences. Creating successful affective responses requires considerable effort both to detect student affect and to design appropriate responses to affect. Recent work has suggested that affect detection is more effective when both physical sensors and interaction logs are used, and that context-sensitive design of affective feedback is necessary to enhance engagement and improve learning. In this paper, we provide a comprehensive report on a multi-part study that integrates detection, validation, and intervention into a unified approach. This paper examines the creation of both sensor-based and interaction-based detectors of student affect, producing successful detectors of student affect. In addition, it reports results from an investigation of motivational feedback messages designed to address student frustration, and investigates whether linking these interventions to detectors improves outcomes. Our results are mixed, finding that self-efficacy enhancing interventions based on interaction-based affect detectors enhance outcomes in one of two experiments investigating affective interventions. This work is conducted in the context of the GIFT framework for intelligent tutoring, and the TC3Sim game-based simulation that provides training for first responder skills.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
The meaning and use of the area under a receiver operating characteristic (ROC) curve.
Hanley, J. A., & McNeil, B. J. (1982) · Radiology · doi:10.1148/radiology.143.1.7063747 · Cited by 21,856 · cited 2×
“Detector performance was evaluated using two metrics: Cohen’s Kappa (Cohen 1960) and A’ computed as the Wilcoxon statistic (Hanley and McNeil 1982).”
MethodMeasuring Presence in Virtual Environments: A Presence Questionnaire
Witmer, B. G., & Singer, M. J. (1998) · Presence: Teleoperators and Virtual Environments · doi:10.1162/105474698565686 · Cited by 6,371 · cited 3×
“In addition, participants completed two questionnaires about their experiences in TC3Sim: the Intrinsic Motivation Inventory (IMI) (Ryan 1982) and Presence Questionnaire (Witmer and Singer 1998; Witmer et al. 2005).”
MethodInfluentialDevelopment and validation of the short grit scale (grit-s)
Duckworth, A. L., & Quinn, P. D. (2009) · Journal of Personality Assessment · doi:10.1080/00223890802634290 · Cited by 3,376
A Survey of Affect Recognition Methods: Audio, Visual, and Spontaneous Expressions
Zeng, Z., Pantic, M., Roisman, G. I., & Huang, T. S. (2009) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2008.52 · Cited by 2,797
Control and information in the intrapersonal sphere: An extension of cognitive evaluation theory.
Ryan, R. M. (1982) · Journal of Personality and Social Psychology · doi:10.1037/0022-3514.43.3.450 · Cited by 2,721 · cited 3×
“In addition, participants completed two questionnaires about their experiences in TC3Sim: the Intrinsic Motivation Inventory (IMI) (Ryan 1982) and Presence Questionnaire (Witmer and Singer 1998; Witmer et al. 2005).”
BackgroundAffect Detection: An Interdisciplinary Review of Models, Methods, and Their Applications
Calvo, R. A., & D'Mello, S. (2010) · IEEE Transactions on Affective Computing · doi:10.1109/t-affc.2010.1 · Cited by 1,893 · cited 1×
“Research on affective learning environments has enabled the development of models that automatically detect learner affect using a wide variety of data modalities; an extensive review can be seen in Calvo and D'Mello (2010).”
BackgroundBaker, R. S. J. D., D'Mello, S. K., Rodrigo, M. M. T., & Graesser, A. C. (2010) · International Journal of Human-Computer Studies · doi:10.1016/j.ijhcs.2009.12.003 · Cited by 936
Self-citationThe Factor Structure of the Presence Questionnaire
Witmer, B. G., Jerome, C. J., & Singer, M. J. (2005) · Presence: Teleoperators and Virtual Environments · doi:10.1162/105474605323384654 · Cited by 770 · cited 1×
“In addition, participants completed two questionnaires about their experiences in TC3Sim: the Intrinsic Motivation Inventory (IMI) (Ryan 1982) and Presence Questionnaire (Witmer and Singer 1998; Witmer et al. 2005).”
Affect and learning: An exploratory look into the role of affect in learning with AutoTutor
Craig, S., Graesser, A., Sullins, J., & Gholson, B. (2004) · Journal of Educational Media · doi:10.1080/1358165042000283101 · Cited by 630 · cited 4×
“Some affective states have relatively uncomplicated relationships with student learning outcomes – for example, engaged concentration appears to be positively associated (Craig et al. 2004; Pardos et al. 2014) while boredom is negatively associated (Craig et al. 2004; Pardos et al. 2014).”
BackgroundInfluentialAttitude construction: evaluation in context
Schwarz, N. (2007) · Social Cognition · doi:10.1521/soco.2007.25.5.638 · Cited by 627 · cited 1×
“Social identity is aligned with the situated social cognition perspective that proposes cognition and action are not discrete entities but dynamically shaped by each other (Schwarz 2007, 2009; Smith and Semin 2004, 2007).”
Background
Cited by
Mining Big Data in Education: Affordances and Challenges
Fischer, C., Pardos, Z. A., Baker, R. S., Williams, J. J., Smyth, P., Yu, R., Slater, S., Baker, R., & Warschauer, M. (2020) · Review of Research in Education · doi:10.3102/0091732x20903304 · Cited by 412
Self-citationOut of the laboratory and into the classroom: the future of artificial intelligence in education
Schiff, D. (2021) · AI & SOCIETY · doi:10.1007/s00146-020-01033-8 · Cited by 334
Proactive and reactive engagement of artificial intelligence methods for education: a review
Mallik, S., & Gangopadhyay, A. (2023) · Frontiers in Artificial Intelligence · doi:10.3389/frai.2023.1151391 · Cited by 139
“[140] compares the usage of sensor and interaction data in delivering motivational prompts in the course of military training.”
BackgroundHutt, S., Krasich, K., R. Brockmole, J., & K. D'Mello, S. (2021) · Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3411764.3445269 · Cited by 60
“Intervention design must acknowledge that MW detection is imperfect (see [3, 19, 34, 43]).”
Background
Metadata
- Authors
- DeFalco, J. A., Rowe, J. P., Paquette, L., Georgoulas-Sherry, V., Brawner, K., Mott, B. W., Baker, R. S., & Lester, J. C.
- Venue
- International Journal of Artificial Intelligence in Education
- Published
- 2017-09-12
- Volume
- 28
- Issue
- 2
- Pages
- 152-193
- ISSN
- 1560-4292 (ISSN-L)
- 1560-4306
- OpenAlex
- W2756344104
- Cited by
- 107
- References
- 14
- FWCI
- 5.25
- Type
- journal article
- Language
- en
- Open access
- hybrid
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Detecting and Addressing Frustration in a Serious Game for Military Training. https://app.cochat.ai/papers/10.1007/s40593-017-0152-1
- The paper
- DeFalco, J. A., Rowe, J. P., Paquette, L., Georgoulas-Sherry, V., Brawner, K., Mott, B. W., Baker, R. S., & Lester, J. C. (2018). Detecting and Addressing Frustration in a Serious Game for Military Training. International Journal of Artificial Intelligence in Education. https://doi.org/10.1007/s40593-017-0152-1
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.