Leveraging educational data mining for real-time performance assessment of scientific inquiry skills within microworlds.
Open accessThe DOI matches its DataCite record; 108 of the 108 references it cites were found in a registry or an index.
Abstract
We present Science Assistments, an interactive environment, which assesses students' inquiry skills as they engage in inquiry using science microworlds. We frame our variables, tasks, assessments, and methods of analyzing data in terms of evidence-centered design. Specifically, we focus on the student model, the task model, and the evidence model in the conceptual assessment framework. In order to support both assessment and the provision of scaffolding, the environment makes inferences about student inquiry skills using models developed through a combination of text replay tagging [cf. Sao Pedro et al. 2011], a method for rapid manual coding of student log files, and educational data mining. Models were developed for multiple inquiry skills, with particular focus on detecting if students are testing their articulated hypotheses, and if they are designing controlled experiments. Student-level cross-validation was applied to validate that this approach can automatically and accurately identify these inquiry skills for new students. The resulting detectors also can be applied at run-time to drive scaffolding intervention.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Citations
Cites
A Coefficient of Agreement for Nominal Scales
Cohen, J. (1960) · Educational and Psychological Measurement · doi:10.1177/001316446002000104 · Cited by 42,398 · cited 2×
“They were validated using six-fold student-level cross validation and had their predictive goodness assessed using two metrics, A’ 1 [Hanley and McNeil 1982] and Cohen’s Kappa (κ) 2 [Cohen 1960].”
MethodData Mining: Practical Machine Learning Tools and Techniques
(2011) · Elsevier eBooks · doi:10.1016/c2009-0-19715-5 · Cited by 25,437
Artificial intelligence: a modern approach
(1995) · Choice Reviews Online · doi:10.5860/choice.33-1577 · Cited by 21,974
The meaning and use of the area under a receiver operating characteristic (ROC) curve.
Hanley, J. A., & McNeil, B. J. (1982) · Radiology · doi:10.1148/radiology.143.1.7063747 · Cited by 21,856 · cited 2×
“They were validated using six-fold student-level cross validation and had their predictive goodness assessed using two metrics, A’ 1 [Hanley and McNeil 1982] and Cohen’s Kappa (κ) 2 [Cohen 1960].”
MethodFast algorithms for mining association rules
Agrawal, R., & Srikant, R. (1998) · Cited by 11,307
Witten, I. H. (2008) · Cited by 9,143
National Science Education Standards
Shea, J. H. (1995) · Journal of Geological Education · doi:10.5408/0022-1368-43.2.102 · Cited by 7,888
Kirschner, P. A., Sweller, J., & Clark, R. E. (2006) · Educational Psychologist · doi:10.1207/s15326985ep4102_1 · Cited by 7,776 · cited 3×
“This scaffolding design lead to a guided system; that is, we aimed to strike a balance between openended and guided inquiry [cf., Kirschner et al. 2006; Hmelo-Silver et al. 2007] such that students can be assessed on inquiry skills and so that they are provided with support so they do not flounder…”
BackgroundAxten, N., Newell, A., & Simon, H. A. (1973) · Contemporary Sociology · doi:10.2307/2063712 · Cited by 6,834
Newell, A. (1972) · Cited by 6,222
Cited by
Challenges and Future Directions of Big Data and Artificial Intelligence in Education
Luan, H., Geczy, P., Lai, H., Gobert, J., Yang, S. J. H., Ogata, H., Baltes, J., Guerra, R., Li, P., & Tsai, C.-C. (2020) · Frontiers in Psychology · doi:10.3389/fpsyg.2020.580820 · Cited by 667
“With the availability of large quantity and high-quality input training data, machine learning processes can achieve accurate results and facilitate informed decision making (Manyika et al., 2011; Gobert et al., 2012, 2013; Gobert and Sao Pedro, 2017).”
BackgroundSelf-citationEducational data mining: A survey and a data mining-based analysis of recent works
Peña-Ayala, A. (2014) · Expert Systems with Applications · doi:10.1016/j.eswa.2013.08.042 · Cited by 660
“119 Gobert et al. (2012) Machine learning,, probability Predictive Classification”
BackgroundInfluentialAldowah, H., Al-Samarraie, H., & Fauzy, W. M. (2019) · Telematics and Informatics · doi:10.1016/j.tele.2019.01.007 · Cited by 564
Mining Big Data in Education: Affordances and Challenges
Fischer, C., Pardos, Z. A., Baker, R. S., Williams, J. J., Smyth, P., Yu, R., Slater, S., Baker, R., & Warschauer, M. (2020) · Review of Research in Education · doi:10.3102/0091732x20903304 · Cited by 412
“…strategies, affective states, or self-regulated learning (SRL) behaviors, and they are sometimes validated based on real-time observations of student actions (Botelho et al., 2017; DeFalco et al., 2018; Pardos et al., 2014) or retrospective hand coding of data subsets (Gobert et al., 2012).”
BackgroundSelf-citationAcademic and Administrative Role of Artificial Intelligence in Education
Ahmad, S. F., Alam, M. M., Rahmat, M. K., Mubarik, M. S., & Hyder, S. I. (2022) · Sustainability · doi:10.3390/su14031101 · Cited by 309
“Such tools automatically set criteria for the desired candidates and information gathering which provides guidelines for interviews, etc. [11].”
Background
Metadata
- Authors
- Gobert, J. D., Sao Pedro, M. A., Baker, R. S. J. D., Toto, E., & Montalvo, O.
- Affiliations
- Worcester Polytechnic Institute (US)
- Worcester Polytechnic Institute
- Venue
- Zenodo
- Publisher
- Zenodo
- Published
- 2012-10-01
- Volume
- 4
- Issue
- 1
- Pages
- 111-143
- ISSN
- 2157-2100 (ISSN-L)
- OpenAlex
- W1485330246
- Cited by
- 122
- References
- 107
- Citations by year
- 2026: 2 · 2025: 2 · 2024: 5 · 2023: 9 · 2022: 11 · 2021: 7 · 2020: 9 · 2019: 7 · 2018: 11 · 2017: 8 · 2016: 12 · 2015: 11 · 2014: 10 · 2013: 16 · 2012: 2
- FWCI
- 12.28
- Type
- journal article
- Language
- en
- Open access
- Locations
- Journal of Educational Data Mining (publishedVersion)
- Zenodo (CERN European Organization for Nuclear Research) (submittedVersion, cc-by-nc-nd)
- Zenodo (CERN European Organization for Nuclear Research)
- Educational Data Mining
- Funders
- National Science Foundation (0742503, 0733286, 1008649)
- U.S. Department of Education
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Intelligent Tutoring Systems and Adaptive Learning
- Physical Sciences › Computer Science › Computer Science Applications › Online Learning and Analytics
- Social Sciences › Education › Science Education and Pedagogy
- Keywords
- scientific inquiry skills, evidence-centered design, student model, task modeling, evidence model, hypothesis testing, controlled experiments, scaffolding
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.