The DOI matches its Crossref record; 87 of the 87 references it cites were found in a registry or an index.
Applying machine learning in science assessment: a systematic review
Abstract
Machine learning (ML) is an emergent computerised technology that relies on algorithms built by ‘learning’ from training data rather than ‘instruction’, which holds great potential to revolutionise science assessment. This study systematically reviewed 49 articles regarding ML-based science assessment through a triangle framework with technical, validity, and pedagogical features on three vertices. We found that a majority of the studies focused on the validity vertex, as compared to the other two vertices. The existing studies primarily involve text recognition, classification, and scoring with an emphasis on constructing scientific explanations, with a vast range of human-machine agreement measures. To achieve the agreement measures, most of the studies employed a cross-validation method, rather than self- or split-validation. ML allows complex assessments to be used by teachers without the burden of human scoring, saving both time and cost. Most studies used supervised ML, which relies on extraction of attributes from student work that was first coded by humans to achieve automaticity, rather than semi- or unsupervised ML. We found that 24 studies were explicitly embedded in science learning activities, such as scientific inquiry and argumentation, to provide feedback or learning guidance. This study identifies existing research gaps and suggests that all three vertices of the ML triangle should be addressed in future assessment studies, with an emphasis on the pedagogy and technology features.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“The majority of contemporary AI systems in science classrooms concentrate on categorizing students' generated responses in scientific practices, particularly in areas like explanation and argumentation (X. Zhai et al., 2020).”
From Assessing student errors in experimentation using artificial intelligence and large language models: A comparative study with human raters (2023) · cites this paper as X. Zhai et al., 2020
In the citing paper
“X. Zhai et al. (2020) conclude that studies are needed which examine procedures in complex decision-making processes.”
From Assessing student errors in experimentation using artificial intelligence and large language models: A comparative study with human raters (2023) · cites this paper as X. Zhai et al. (2020)
Citations
Cites
Mastering the game of Go with deep neural networks and tree search
Silver, D., Huang, A., Maddison, C. J., Guez, A., Sifre, L., van den Driessche, G., Schrittwieser, J., Antonoglou, I., Panneershelvam, V., Lanctot, M., et al. (2016) · Nature · doi:10.1038/nature16961 · Cited by 15,612
Machine learning: Trends, perspectives, and prospects
Jordan, M. I., & Mitchell, T. M. (2015) · Science · doi:10.1126/science.aaa8415 · Cited by 9,955
Statistical Methods for Rates and Proportions.
Everitt, B. S., & Fleiss, J. L. (1981) · Biometrics · doi:10.2307/2530193 · Cited by 8,990
Ensemble methods in machine learning
Dietterich, T. G. (2000) · Lecture Notes in Computer Science · doi:10.1007/3-540-45014-9_1 · Cited by 8,408
Introduction to research in education
Ary, D. (1972) · Cited by 7,699
(2013) · Cited by 6,708
Some Studies in Machine Learning Using the Game of Checkers
Samuel, A. L. (1959) · IBM Journal of Research and Development · doi:10.1147/rd.33.0210 · Cited by 4,443
Statistical Modeling: The Two Cultures (with comments and a rejoinder by the author)
Breiman, L. (2001) · Statistical Science · doi:10.1214/ss/1009213726 · Cited by 4,340 · cited 2×
“Breiman (2001) suggested that the major di ff erence between the two fi elds is that statisticians and MLers have di ff erent assumptions and uses of data inference.”
BackgroundPractical Assessment, Research & Evaluation, 2000-2001.
Rudner, L. M., & Schafer, W. D. (2001) · Practical assessment, research & evaluation · Cited by 3,905
Mertens, D. M. (1997) · Cited by 3,703
Research works printed but not linked: 13
- Assis Gomes C. M., 2017, Practical Assessment, Research & Evaluation, 22, 1
- Bennett, R. E., & Zhang, M, 2016, Technology and testing: Improving educational and psychological measurement
- Chen C.-K., 2010, IR Applications, 9, 1
- Ghali R., 2016, Journal of Education and Training Studies, 4, 192
- Ha, M. (2015). Assessing scientific practices using machine learning methods: Development of automated computer scoring models for written evolutionary explanations (Doctoral dissertation). The Ohio State University. Predicting the accuracy of computer scoring of text: Probabilistic, multi-model, and semantic similarity approaches (pp. 1112-1117). IEEE .
- Huang C. C., 2010, Journal of Educational Technology & Society, 13, 180
- Kim K. J., 2017, Journal of Educational Data Mining, 9, 45
- Lintean M., 2012, International Journal of Artificial Intelligence in Education, 21, 169
- Mayfield E., 2010, SIDE: The summarization IDE (User’s manual)
- Mayfield, E. & Rosé, C. (2013). LightSIDE: Open source machine learning for text handbook of automated essay evaluation (pp. 146–157). New York, NY: Routledge.
- Mitchell T., 1997, Machine learning
- Zhai, X. (2019, June) Parameters for machine learning (Internal Document: unpublished). DOI: 10.13140/RG.2.2.33563.31525 · doi:10.13140/rg.2.2.33563.31525
- Zhai X., Journal of Research in Science Teaching
Cited by
Liang, J.-C., Hwang, G.-J., Chen, M.-R. A., & Darmawansah, D. (2023) · Interactive Learning Environments · doi:10.1080/10494820.2021.1958348 · Cited by 378
Almasri, F. (2024) · Research in Science Education · doi:10.1007/s11165-024-10176-3 · Cited by 361
Padhiary, M., Saha, D., Kumar, R., Sethi, L. N., & Kumar, A. (2024) · Smart Agricultural Technology · doi:10.1016/j.atech.2024.100483 · Cited by 226
Ding, A.-C. E., Shi, L., Yang, H., & Choi, I. (2024) · Computers and Education Open · doi:10.1016/j.caeo.2024.100178 · Cited by 228
Self-citationFine-tuning ChatGPT for automatic scoring
Latif, E., & Zhai, X. (2024) · Computers and Education: Artificial Intelligence · doi:10.1016/j.caeai.2024.100210 · Cited by 179
Metadata
- Authors
- Zhai, X., Yin, Y., Pellegrino, J. W., Haudek, K. C., & Shi, L.
- Affiliations
- Department of Mathematics and Science Education, University of Georgia, Athens, GA, USA
- University of Illinois Chicago (US)
- Michigan State University (US)
- University of Georgia (US)
- Venue
- Studies in Science Education
- Publisher
- Informa UK Limited
- Published
- 2020-01-02
- Volume
- 56
- Issue
- 1
- Pages
- 111-151
- ISSN
- 0305-7267 (ISSN-L)
- 1940-8412
- OpenAlex
- W3012071850
- Cited by
- 257
- References
- 87
- Citations by year
- 2026: 33 · 2025: 53 · 2024: 70 · 2023: 46 · 2022: 22 · 2021: 24 · 2020: 9
- FWCI
- 8.36
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Studies in Science Education (publishedVersion)
- Topics
- Social Sciences › Education › Science Education and Pedagogy
- Physical Sciences › Computer Science › Computer Science Applications › Online Learning and Analytics
- Physical Sciences › Computer Science › Artificial Intelligence › Intelligent Tutoring Systems and Adaptive Learning
- Keywords
- machine learning, science assessment, automated grading, automatic text recognition, scientific inquiry, argumentation, self-validation, supervised learning
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Applying machine learning in science assessment: a systematic review. https://app.cochat.ai/papers/10.1080/03057267.2020.1735757
- The paper
- Zhai, X., Yin, Y., Pellegrino, J. W., Haudek, K. C., & Shi, L. (2020). Applying machine learning in science assessment: a systematic review. Studies in Science Education. https://doi.org/10.1080/03057267.2020.1735757
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.