Validity arguments for diagnostic assessment using automated writing evaluation
The DOI matches its Crossref record; 4 of the 4 references it cites were found in a registry or an index.
Abstract
Two examples demonstrate an argument-based approach to validation of diagnostic assessment using automated writing evaluation (AWE). Criterion®, was developed by Educational Testing Service to analyze students’ papers grammatically, providing sentence-level error feedback. An interpretive argument was developed for its use as part of the diagnostic assessment process in undergraduate university English for academic purposes (EAP) classes. The Intelligent Academic Discourse Evaluator ( IADE) was developed for use in graduate EAP university classes, where the goal was to help students improve their discipline-specific writing. The validation for each was designed to support claims about the intended purposes of the assessments. We present the interpretive argument for each and show some of the data that have been gathered as backing for the respective validity arguments, which include the range of inferences that one would make in claiming validity of the interpretations, uses, and consequences of diagnostic AWE-based assessments.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
The Role of the Linguistic Environment in Second Language Acquisition
Long, M. H. (1996) · Handbook of Second Language Acquisition · doi:10.1016/b978-012589042-7/50015-3 · Cited by 3,927
The influence of teacher commentary on student revision
Ferris, D. R. (1997) · TESOL Quarterly · doi:10.2307/3588049 · Cited by 618 · cited 1×
“These coding schemes were developed based on Ferris (1997) but modified and refined to fit the data and context of the study, as shown in Table 2.”
MethodThe impact of teacher written feedback on individual writers
Hyland, F. (1998) · Journal of Second Language Writing · doi:10.1016/s1060-3743(98)90017-0 · Cited by 456 · cited 1×
“…for students” (Heift, 2003, p. 533), it embodies the following characteristics considered beneficial for language learning: individual ‐ specific (Hyland, 1998), explicit (Caroll & Swain, 1993; Lyster, 1998), metalinguistic (Rosa & Leow, 2004), negative cognitive (Mitchell & Myles, 1998),…”
Automated writing evaluation: Defining the classroom research agenda
Warschauer, M., & Ware, P. (2006) · Language Teaching Research · doi:10.1191/1362168806lr190oa · Cited by 337
Cited by
Koltovskaia, S. (2020) · Assessing Writing · doi:10.1016/j.asw.2020.100450 · Cited by 447
AI in the Foreign Language Classroom: A Pedagogical Overview of Automated Writing Assistance Tools
Alharbi, W. (2023) · Education Research International · doi:10.1155/2023/4253331 · Cited by 286
Partnering with AI: Intelligent writing assistance and instructed language learning
Godwin-Jones, R. (2022) · Language Learning & Technology · doi:10.64152/10125/73474 · Cited by 285
Link, S., Mehrzad, M., & Rahimi, M. (2022) · Computer Assisted Language Learning · doi:10.1080/09588221.2020.1743323 · Cited by 275
Automated written corrective feedback: how well can students make use of it?
Ranalli, J. (2018) · Computer Assisted Language Learning · doi:10.1080/09588221.2018.1428994 · Cited by 228
Metadata
- Authors
- Chapelle, C. A., Cotos, E., & Lee, J.
- Venue
- Language Testing
- Publisher
- SAGE Publications
- Published
- 2015-01-27
- Volume
- 32
- Issue
- 3
- Pages
- 385-405
- ISSN
- 0265-5322 (ISSN-L)
- 1477-0946
- OpenAlex
- W2051092771
- Cited by
- 156
- References
- 4
- FWCI
- 44.63
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.