The DOI matches its Crossref record; 86 of the 86 references it cites were found in a registry or an index.
A selective meta-analysis on the relative incidence of discrete affective states during learning with technology.
Abstract
The last decade has witnessed considerable interest in the investigation of the affective dimensions of learning and in the development of advanced learning technologies that automatically detect and respond to student affect. Identifying the affective states that students experience in technology-enhanced learning contexts is a fundamental question in this area. This article provides an initial attempt to answer this question with a selective meta-analysis of 24 studies that utilized a mixture of methodologies (online self-reports, online observations, emote-aloud protocols, cued recall) and affect judges (students themselves, untrained peers, trained judges) for fine-grained monitoring of 14 discrete affective states of 1,740 middle school, high school, college, and adult students in 5 countries. Affective states occurred over the course of interactions with a range of learning technologies, including intelligent tutoring systems, serious games, simulation environments, and simple computer interfaces. Standardized effect sizes of relative frequency, computed by comparing the proportional occurrence of each affective state to the other states in each study, were modeled with random-effects models. Engagement/flow was consistently found to be relatively frequent (d+ = 2.5), and contempt, anger, disgust, sadness, anxiety, delight, fear, and surprise were consistently infrequent, with d+ ranging from −6.5 to −0.78. Effects for boredom (d+ = 0.19), confusion (d+ = 0.12), curiosity (d+ = −0.10), happiness (d+ = −0.13), and frustration (d+ = −2.5) varied substantially across studies. Mixed-effects models indicated that the source of the affect judgments (self vs. observers) and the authenticity of the learning contexts (classroom vs. laboratory) accounted for greater heterogeneity than the use of advanced learning technologies and training time. Theoretical and applied implications of the findings are discussed.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Conducting Meta-Analyses in R with the metafor Package
Viechtbauer, W. (2010) · Journal of Statistical Software · doi:10.18637/jss.v036.i03 · Cited by 20,464
Flow: the psychology of optimal experience
(1990) · Choice Reviews Online · doi:10.5860/choice.28-0597 · Cited by 12,637
The origins of intelligence in children.
Piaget, J. (1952) · doi:10.1037/11494-000 · Cited by 10,687
Borenstein, M., Hedges, L. V., Higgins, J. P. T., & Rothstein, H. R. (2021) · doi:10.1002/9781119558378 · Cited by 9,799
An argument for basic emotions
Ekman, P. (1992) · Cognition and Emotion · doi:10.1080/02699939208411068 · Cited by 9,646
Cohen, J. (1992) · Psychological Bulletin · doi:10.1037//0033-2909.112.1.155 · Cited by 8,771
Mindset: The New Psychology of Success
(2006) · Choice Reviews Online · doi:10.5860/choice.44-2397 · Cited by 7,804
Picard, R. W. (1997) · MIT Press · doi:10.7551/mitpress/1140.001.0001 · Cited by 7,382
Csíkszentmihályi, M. (1975) · Cited by 5,690
The combination of estimates from different experiments
Cochran, W. G. (1954) · Biometrics · doi:10.2307/3001666 · Cited by 5,380 · cited 1×
“The homogeneity statistic (Q) (Cochran, 1954) was used to test if the true effect sizes were consistent (homogeneity) or varied across studies (heterogeneity), with significant Q’s being indicative of heterogeneity.”
Method
Cited by
A Review and Meta-Analysis of Multimodal Affect Detection Systems
D'mello, S. K., & Kory, J. (2015) · ACM Computing Surveys · doi:10.1145/2682899 · Cited by 628
Self-citationTeacher emotions in the classroom and their implications for students
Frenzel, A. C., Daniels, L., & Burić, I. (2021) · Educational Psychologist · doi:10.1080/00461520.2021.1985501 · Cited by 470
Learner Engagement in Blended Learning Environments: A Conceptual Framework
Halverson, L. R., & Graham, C. R. (2019) · Online Learning · doi:10.24059/olj.v23i2.1481 · Cited by 437
Loderer, K., Pekrun, R., & Lester, J. C. (2020) · Learning and Instruction · doi:10.1016/j.learninstruc.2018.08.002 · Cited by 377
Measuring emotions during epistemic activities: the Epistemically-Related Emotion Scales
Pekrun, R., Vogl, E., Muis, K. R., & Sinatra, G. M. (2017) · Cognition and Emotion · doi:10.1080/02699931.2016.1204989 · Cited by 373
Technology‐mediated learning theory
Bower, M. (2019) · British Journal of Educational Technology · doi:10.1111/bjet.12771 · Cited by 321
Händel, M., Stephan, M., Gläser-Zikuda, M., Kopp, B., Bedenlier, S., & Ziegler, A. (2022) · Journal of Research on Technology in Education · doi:10.1080/15391523.2020.1846147 · Cited by 292
The Role of Epistemic Emotions in Personal Epistemology and Self-Regulated Learning
Muis, K. R., Chevrier, M., & Singh, C. A. (2018) · Educational Psychologist · doi:10.1080/00461520.2017.1421465 · Cited by 270
Liu, S., Liu, S., Liu, Z., Peng, X., & Yang, Z. (2022) · Computers & Education · doi:10.1016/j.compedu.2022.104461 · Cited by 269
A meta-analysis of how signaling affects learning with media
Schneider, S., Beege, M., Nebel, S., & Rey, G. D. (2018) · Educational Research Review · doi:10.1016/j.edurev.2017.11.001 · Cited by 268
Automatic Detection of Learning-Centered Affective States in the Wild
Bosch, N., D'Mello, S., Baker, R., Ocumpaugh, J., Shute, V., Ventura, M., Wang, L., & Zhao, W. (2015) · Proceedings of the 20th International Conference on Intelligent User Interfaces · doi:10.1145/2678025.2701397 · Cited by 178
Self-citationWhat multimodal data can tell us about the students’ regulation of their learning process?
Järvelä, S., Malmberg, J., Haataja, E., Sobocinski, M., & Kirschner, P. A. (2021) · Learning and Instruction · doi:10.1016/j.learninstruc.2019.04.004 · Cited by 163
Multimodal data to design visual learning analytics for understanding regulation of learning
Noroozi, O., Alikhani, I., Järvelä, S., Kirschner, P. A., Juuso, I., & Seppänen, T. (2019) · Computers in Human Behavior · doi:10.1016/j.chb.2018.12.019 · Cited by 140
“…physiological modalities such as eye-movement, facial gesture, and micro-gesture recognition data that are related to the learners’ cognitive demands indicating the extent to which they are bored, confused, stressed, and so on during the learning process (D’Mello, 2013; Fairclough et al., 2005).”
BackgroundMangaroska, K., Sharma, K., Gašević, D., & Giannakos, M. (2022) · Journal of Computer Assisted Learning · doi:10.1111/jcal.12590 · Cited by 100
Ruan, S., Willis, A., Xu, Q., Davis, G. M., Jiang, L., Brunskill, E., & Landay, J. A. (2019) · Proceedings of the Sixth (2019) ACM Conference on Learning @ Scale · doi:10.1145/3330430.3333643 · Cited by 81
Hagendorff, T. (2021) · Minds and Machines · doi:10.1007/s11023-021-09573-8 · Cited by 47
Schrader, C., & Kalyuga, S. (2020) · International Journal of Human-Computer Studies · doi:10.1016/j.ijhcs.2019.102374 · Cited by 31
Gupta, S., Kumar, P., & Tekchandani, R. (2024) · Multimedia Tools and Applications · doi:10.1007/s11042-023-18021-x · Cited by 14
Metadata
- Authors
- D'Mello, S.
- Venue
- Journal of Educational Psychology
- Published
- 2013-11
- Volume
- 105
- Issue
- 4
- Pages
- 1082-1099
- ISSN
- 0022-0663 (ISSN-L)
- 1939-2176
- OpenAlex
- W2171618137
- Cited by
- 300
- References
- 86
- FWCI
- 19.88
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.