The DOI matches its Crossref record; 2 of the 2 references it cites were found in a registry or an index.
Diagnostic assessment of writing: A comparison of two rating scales
Abstract
Alderson (2005) suggests that diagnostic tests should identify strengths and weaknesses in learners' use of language and focus on specific elements rather than global abilities. However, rating scales used in performance assessment have been repeatedly criticized for being imprecise and therefore often resulting in holistic marking by raters (Weigle, 2002). The aim of this study is to compare two rating scales for writing in an EAP context; one `a priori' developed scale with less specific descriptors of the kind commonly used in proficiency tests and one empirically developed scale with detailed level descriptors. The validation process involved 10 trained raters applying both sets of descriptors to the rating of 100 writing scripts yielded from a large-scale diagnostic assessment administered to both native and non-native speakers of English at a large university. A quantitative comparison of rater behaviour was undertaken using FACETS. Questionnaires and interviews were administered to elicit the raters' perceptions of the efficacy of the two types of scales. The results indicate that rater reliability was substantially higher and that raters were able to better distinguish between different aspects of writing when the more detailed descriptors were used. Rater feedback also showed a preference for the more detailed scale. The findings are discussed in terms of their implications for rater training and rating scale development.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
The application of electronic computers to factor analysis
Kaiser, H. F. (1960) · Educational and Psychological Measurement · doi:10.1177/001316446002000116 · Cited by 12,459
Weigle, S. C. (2002) · doi:10.1017/cbo9780511732997 · Cited by 779
Cited by
Eurasian Journal of Educational Research
(2021) · Eurasian Journal of Educational Research · doi:10.14689/ejer · Cited by 638
Reliability and validity of rubrics for assessment through writing
Rezaei, A. R., & Lovorn, M. (2010) · Assessing Writing · doi:10.1016/j.asw.2010.01.003 · Cited by 299
Li, J., & Huang, J. (2022) · Assessing Writing · doi:10.1016/j.asw.2021.100604 · Cited by 31
Constructing a Scale to Assess L2 Written Speech Act Performance: WDCT and E-mail Tasks
Chen, Y.-S., & Liu, J. (2016) · Language Assessment Quarterly · doi:10.1080/15434303.2016.1213844 · Cited by 20
“A reliable and valid rating scale is therefore necessary to objectively evaluate learner performance (Knoch, 2009).”
Background
Metadata
- Authors
- Knoch, U.
- Venue
- Language Testing
- Publisher
- SAGE Publications
- Published
- 2009-04
- Volume
- 26
- Issue
- 2
- Pages
- 275-304
- ISSN
- 0265-5322 (ISSN-L)
- 1477-0946
- OpenAlex
- W2104139582
- Cited by
- 215
- References
- 2
- FWCI
- 71.69
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.