Operationalizing and Detecting Disengagement Within Online Science Microworlds
The DOI matches its Crossref record; 23 of the 23 references it cites were found in a registry or an index.
Abstract
In recent years, there has been increased interest in engagement during learning. This is of particular interest in the science, technology, engineering, and mathematics domains, in which many students struggle and where the United States needs skilled workers. This article lays out some issues important for framing research on this topic and provides a review of some existing work with similar goals on engagement in science learning. Specifically, here we seek to help better concretize engagement, a fuzzy construct, by operationalizing and detecting (i.e., identifying using a computational method) disengaged behaviors that are antithetical to engagement. We, in turn, describe our real-time detector (i.e., machine learned model) of disengaged behavior and how it was developed. Last, we address our ongoing research on how our detector of disengaged behavior will be used to intervene in real time to better support students' science inquiry learning in Inq-ITS (Inquiry-Intelligent Tutoring System; Gobert, Sao Pedro, Baker, Toto, & Montalvo, 2012; Gobert, Sao Pedro, Raziuddin, & Baker, 2013).
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
School Engagement: Potential of the Concept, State of the Evidence
Fredricks, J. A., Blumenfeld, P. C., & Paris, A. H. (2004) · Review of Educational Research · doi:10.3102/00346543074001059 · Cited by 12,581 · cited 9×
“Three-dimensional models (see, e.g., Fredricks et al., 2004) add an emotional component to the cognitive and the behavioral.”
BackgroundMethodInfluentialGrit: perseverance and passion for long-term goals
Duckworth, A. L., Peterson, C., Matthews, M. D., & Kelly, D. R. (2007) · Journal of Personality and Social Psychology · doi:10.1037/0022-3514.92.6.1087 · Cited by 6,911
The relationship between Precision-Recall and ROC curves
Davis, J., & Goadrich, M. (2006) · Proceedings of the 23rd international conference on Machine learning - ICML '06 · doi:10.1145/1143844.1143874 · Cited by 6,559
The Skill Content of Recent Technological Change: An Empirical Exploration
Autor, D. H., Levy, F., & Murnane, R. J. (2003) · The Quarterly Journal of Economics · doi:10.1162/003355303322552801 · Cited by 6,205
Pekrun, R. (2006) · Educational Psychology Review · doi:10.1007/s10648-006-9029-9 · Cited by 5,566
Blackwell, L. S., Trzesniewski, K. H., & Dweck, C. S. (2007) · Child Development · doi:10.1111/j.1467-8624.2007.00995.x · Cited by 3,784
A leisurely look at the bootstrap, the jackknife, and cross-validation
Efron, B., & Gong, G. (1983) · The American Statistician · doi:10.1080/00031305.1983.10483087 · Cited by 3,229 · cited 1×
“These models were evaluated using a process of sixfold student-level cross-validation (Efron & Gong, 1983).”
MethodHandbook of Research on Student Engagement
(2012) · doi:10.1007/978-1-4614-2018-7 · Cited by 3,050 · cited 2×
“There has been a surge of interest and research on the topic of engagement in the last 20 years (Christenson, Reschly, & Wylie, 2012).”
BackgroundRelationships matter: linking teacher support to student engagement and achievement.
Klem, A. M., & Connell, J. P. (2004) · Journal of School Health · doi:10.1111/j.1746-1561.2004.tb08283.x · Cited by 2,082 · cited 1×
“In terms of research findings, engagement is associated with positive outcomes along academic, social, and emotional lines (Klem & Connell, 2004) and is a very good predictor of students’ learning, grades, achievement test scores, retention, and graduation (Appleton, Christenson, & Furlong, 2008).”
BackgroundEngagement and disaffection in the classroom: Part of a larger motivational dynamic?
Skinner, E., Furrer, C., Marchand, G., & Kindermann, T. (2008) · Journal of Educational Psychology · doi:10.1037/a0012840 · Cited by 2,037
Student engagement with school: Critical conceptual and methodological issues of the construct
Appleton, J. J., Christenson, S. L., & Furlong, M. J. (2008) · Psychology in the Schools · doi:10.1002/pits.20303 · Cited by 1,956 · cited 4×
“In terms of research findings, engagement is associated with positive outcomes along academic, social, and emotional lines (Klem & Connell, 2004) and is a very good predictor of students’ learning, grades, achievement test scores, retention, and graduation (Appleton, Christenson, & Furlong, 2008).”
BackgroundMethodInfluentialMeasuring cognitive and psychological engagement: Validation of the Student Engagement Instrument
Appleton, J. J., Christenson, S. L., Kim, D., & Reschly, A. L. (2006) · Journal of School Psychology · doi:10.1016/j.jsp.2006.04.002 · Cited by 1,858
Papastergiou, M. (2009) · Computers & Education · doi:10.1016/j.compedu.2008.06.004 · Cited by 1,689
Skinner, E. A., Kindermann, T. A., & Furrer, C. J. (2009) · Educational and Psychological Measurement · doi:10.1177/0013164408323233 · Cited by 1,587
Minner, D. D., Levy, A. J., & Century, J. (2010) · Journal of Research in Science Teaching · doi:10.1002/tea.20347 · Cited by 1,538
Academic success among students at risk for school failure
Finn, J. D., & Rock, D. A. (1997) · Journal of Applied Psychology · doi:10.1037/0021-9010.82.2.221 · Cited by 1,397
Student engagement in high school classrooms from the perspective of flow theory
Shernoff, D. J., Csikszentmihalyi, M., Shneider, B., & Shernoff, E. S. (2003) · School Psychology Quarterly · doi:10.1521/scpq.18.2.158.21860 · Cited by 1,237
Baker, R. S. J. D., D'Mello, S. K., Rodrigo, M. M. T., & Graesser, A. C. (2010) · International Journal of Human-Computer Studies · doi:10.1016/j.ijhcs.2009.12.003 · Cited by 936
Self-citationDetecting Student Misuse of Intelligent Tutoring Systems
Baker, R. S., Corbett, A. T., & Koedinger, K. R. (2004) · Lecture Notes in Computer Science · doi:10.1007/978-3-540-30139-4_50 · Cited by 353
Self-citationEnhancing student motivation and engagement: The effects of a multidimensional intervention
Martin, A. J. (2008) · Contemporary Educational Psychology · doi:10.1016/j.cedpsych.2006.11.003 · Cited by 293 · cited 1×
“For example, Martin (2008) proposed a two-dimensional model comprising mainly cognitive and behavioral dimensions.”
BackgroundHelp-Seeking Behavior in Learning
Gall, S. N.-L. (1985) · Review of Research in Education · doi:10.2307/1167146 · Cited by 237
Gobert, J. D., Sao Pedro, M., Raziuddin, J., & Baker, R. S. (2013) · Journal of the Learning Sciences · doi:10.1080/10508406.2013.837391 · Cited by 210
Self-citationGobert, J. D., Sao Pedro, M. A., Baker, R. S. J. D., Toto, E., & Montalvo, O. (2012) · Zenodo · doi:10.5281/zenodo.3554645 · Cited by 122 · cited 5×
“…2013; Wixon, Baker, Gobert, Ocumpaugh, & Bachmann, 2012) in our online science learning environment, Inquiry Intelligent Tutoring System (Inq-ITS; Gobert et al., 2012; Gobert et al., 2013) We, in turn, describe our real-time detector of one form of disengaged behavior, Disengaged from Task Goal…”
BackgroundMethodInfluentialSelf-citation
Cited by
The Challenges of Defining and Measuring Student Engagement in Science
Sinatra, G. M., Heddy, B. C., & Lombardi, D. (2015) · Educational Psychologist · doi:10.1080/00461520.2014.1002924 · Cited by 1,044
Measuring student engagement in technology-mediated learning: A review
Henrie, C. R., Halverson, L. R., & Graham, C. R. (2015) · Computers & Education · doi:10.1016/j.compedu.2015.09.005 · Cited by 1,010
Measuring Cognitive Engagement With Self-Report Scales: Reflections From Over 20 Years of Research
Greene, B. A. (2015) · Educational Psychologist · doi:10.1080/00461520.2014.989230 · Cited by 438
Learner Engagement in Blended Learning Environments: A Conceptual Framework
Halverson, L. R., & Graham, C. R. (2019) · Online Learning · doi:10.24059/olj.v23i2.1481 · Cited by 437
Azevedo, R. (2015) · Educational Psychologist · doi:10.1080/00461520.2015.1004069 · Cited by 418
Metadata
- Authors
- Gobert, J. D., Baker, R. S., & Wixon, M. B.
- Venue
- Educational Psychologist
- Publisher
- Informa UK Limited
- Published
- 2015-01-02
- Volume
- 50
- Issue
- 1
- Pages
- 43-57
- ISSN
- 0046-1520 (ISSN-L)
- 1532-6985
- OpenAlex
- W2055507946
- Cited by
- 114
- References
- 23
- FWCI
- 10.01
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.