Can we and should we use artificial intelligence for formative assessment in science?
Open accessThe DOI matches its Crossref record; 3 of the 3 references it cites were found in a registry or an index.
Abstract
In this commentary, we respond to the recent article Applying machine learning to automatically assess scientific models by Zhai et al. (2022).The authors present automated assessment as a solution to the problem of limited time for assessment in middle school science classrooms.Drawing from our collective expertise in science assessment, machine learning (ML), artificial intelligence (AI), and culturally relevant and linguistically responsive pedagogy, we argue that there are significant limitations to the current applications of AI for formative assessment practices.Although we believe that these limitations extend to all students, we are particularly concerned about the implications for students from nondominant cultural and linguistic backgrounds.We first share our understanding of AI's role in formative assessment, with reference to the paper by Zhai and colleagues.Next, we ask whether AI can effectively assess students' emergent sensemaking and then consider whether we should use AI for purposes of formative assessment.Finally, we discuss how we can better use AI for formative assessment. | ARTIFICIAL INTELLIGENCE AND ASSESSMENT PRACTICESML is a form of AI that uses datasets to train algorithms to identify patterns and relationships, which can be used to simulate human decision-making.Zhai et al. (2021) argue that ML can All authors have made equal and significant contributions to this paper.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Toward a Theory of Culturally Relevant Pedagogy
Ladson-Billings, G. (1995) · American Educational Research Journal · doi:10.3102/00028312032003465 · Cited by 8,057
Developing the theory of formative assessment
Black, P., & Wiliam, D. (2009) · Educational Assessment, Evaluation and Accountability · doi:10.1007/s11092-008-9068-5 · Cited by 3,301
Improving Fairness in Machine Learning Systems: What Do Industry Practitioners Need?
Holstein, K., Wortman Vaughan, J., Daumé, H., Dudik, M., & Wallach, H. (2019) · Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3290605.3300830 · Cited by 777
Cited by
Meyer, J., Jansen, T., Schiller, R., Liebenow, L. W., Steinbach, M., Horbach, A., & Fleckenstein, J. (2024) · Computers and Education: Artificial Intelligence · doi:10.1016/j.caeai.2023.100199 · Cited by 311
Bewersdorff, A., Seßler, K., Baur, A., Kasneci, E., & Nerdel, C. (2023) · Computers and Education: Artificial Intelligence · doi:10.1016/j.caeai.2023.100177 · Cited by 76
“Some educational researchers in the field of science education raise concerns about the use of AI systems for formative assessment (Li et al., 2023).”
BackgroundShin, I., Hwang, S. B., Yoo, Y. J., Bae, S., & Kim, R. Y. (2025) · Proceedings of the 15th International Learning Analytics and Knowledge Conference · doi:10.1145/3706468.3706488 · Cited by 16
Metadata
- Authors
- Li, T., Reigh, E., He, P., & Adah Miller, E.
- Venue
- Journal of Research in Science Teaching
- Publisher
- Wiley
- Published
- 2023-05-10
- Volume
- 60
- Issue
- 6
- Pages
- 1385-1389
- ISSN
- 0022-4308 (ISSN-L)
- 1098-2736
- OpenAlex
- W4376141261
- Cited by
- 52
- References
- 3
- FWCI
- 38.55
- Type
- journal article
- Language
- en
- Open access
- bronze
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.