Student or educator? Sign up with your school email and get 6 months of Pro free.
CoChat

The DOI matches its Crossref record; 87 of the 87 references it cites were found in a registry or an index.

Applying machine learning in science assessment: a systematic review

Zhai, Xiaoming, Yin, Yue, Pellegrino, James W., Haudek, Kevin C., Shi, Lehong · 2020 · Studies in Science Education

Abstract

Machine learning (ML) is an emergent computerised technology that relies on algorithms built by ‘learning’ from training data rather than ‘instruction’, which holds great potential to revolutionise science assessment. This study systematically reviewed 49 articles regarding ML-based science assessment through a triangle framework with technical, validity, and pedagogical features on three vertices. We found that a majority of the studies focused on the validity vertex, as compared to the other two vertices. The existing studies primarily involve text recognition, classification, and scoring with an emphasis on constructing scientific explanations, with a vast range of human-machine agreement measures. To achieve the agreement measures, most of the studies employed a cross-validation method, rather than self- or split-validation. ML allows complex assessments to be used by teachers without the burden of human scoring, saving both time and cost. Most studies used supervised ML, which relies on extraction of attributes from student work that was first coded by humans to achieve automaticity, rather than semi- or unsupervised ML. We found that 24 studies were explicitly embedded in science learning activities, such as scientific inquiry and argumentation, to provide feedback or learning guidance. This study identifies existing research gaps and suggests that all three vertices of the ML triangle should be addressed in future assessment studies, with an emphasis on the pedagogy and technology features.

Overview

Integrity signals

Registry

Crossref · 10.1080/03057267.2020.1735757

Cites

87 of 87 references verified0 citation errors · 0 not checkable

Statistics

Recomputing p-values needs the paper’s full text.

Verification

Claims the paper makes

This paper’s full text isn’t openly available, so its claims and self-check weren’t run. The registry and citation checks still apply.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Citations

Cites

Show all 87 references

Research works printed but not linked: 13

  1. Assis Gomes C. M., 2017, Practical Assessment, Research & Evaluation, 22, 1
  2. Bennett, R. E., & Zhang, M, 2016, Technology and testing: Improving educational and psychological measurement
  3. Chen C.-K., 2010, IR Applications, 9, 1
  4. Ghali R., 2016, Journal of Education and Training Studies, 4, 192
  5. Ha, M. (2015). Assessing scientific practices using machine learning methods: Development of automated computer scoring models for written evolutionary explanations (Doctoral dissertation). The Ohio State University. Predicting the accuracy of computer scoring of text: Probabilistic, multi-model, and semantic similarity approaches (pp. 1112-1117). IEEE .
  6. Huang C. C., 2010, Journal of Educational Technology & Society, 13, 180
  7. Kim K. J., 2017, Journal of Educational Data Mining, 9, 45
  8. Lintean M., 2012, International Journal of Artificial Intelligence in Education, 21, 169
  9. Mayfield E., 2010, SIDE: The summarization IDE (User’s manual)
  10. Mayfield, E. & Rosé, C. (2013). LightSIDE: Open source machine learning for text handbook of automated essay evaluation (pp. 146–157). New York, NY: Routledge.
  11. Mitchell T., 1997, Machine learning
  12. Zhai, X. (2019, June) Parameters for machine learning (Internal Document: unpublished). DOI: 10.13140/RG.2.2.33563.31525 · doi:10.13140/rg.2.2.33563.31525
  13. Zhai X., Journal of Research in Science Teaching

Cited by

See all 257

Metadata

Authors
Zhai, X., Yin, Y., Pellegrino, J. W., Haudek, K. C., & Shi, L.
Affiliations
  • Department of Mathematics and Science Education, University of Georgia, Athens, GA, USA
  • University of Illinois Chicago (US)
  • Michigan State University (US)
  • University of Georgia (US)
Venue
Studies in Science Education
Published
2020-01-02
Volume
56
Issue
1
Pages
111-151
ISSN
  • 0305-7267 (ISSN-L)
  • 1940-8412
OpenAlex
W3012071850
Cited by
257
References
87
Citations by year
2026: 33 · 2025: 53 · 2024: 70 · 2023: 46 · 2022: 22 · 2021: 24 · 2020: 9
FWCI
8.36
Type
journal article
Language
en
Open access
closed
Locations
Studies in Science Education (publishedVersion)
Topics
  • Social Sciences › Education › Science Education and Pedagogy
  • Physical Sciences › Computer Science › Computer Science Applications › Online Learning and Analytics
  • Physical Sciences › Computer Science › Artificial Intelligence › Intelligent Tutoring Systems and Adaptive Learning
Keywords
machine learning, science assessment, automated grading, automatic text recognition, scientific inquiry, argumentation, self-validation, supervised learning

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: Applying machine learning in science assessment: a systematic review. https://app.cochat.ai/papers/10.1080/03057267.2020.1735757
The paper
Zhai, X., Yin, Y., Pellegrino, J. W., Haudek, K. C., & Shi, L. (2020). Applying machine learning in science assessment: a systematic review. Studies in Science Education. https://doi.org/10.1080/03057267.2020.1735757

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.