Developing pedagogically-guided algorithms for intelligent writing feedback
The DOI matches its Crossref record; 5 of the 5 references it cites were found in a registry or an index.
Abstract
Various computer tools have been developed to support educators’ assessment of student writing, including automated essay scoring and automated writing evaluation systems. Research demonstrates that these systems exhibit relatively high scoring accuracy but uncertain instructional efficacy. Students’ writing proficiency does not necessarily improve as a result of interacting with the software. One question is whether these systems offer appropriate or sufficient formative feedback to students about their writing. To motivate further research in this area, we present a straightforward methodology for constructing automated feedback algorithms that are grounded in writing pedagogy and assessment. The resulting threshold algorithms are demonstrated to be meaningfully related to essay quality and informative regarding individualised, formative feedback for writers. Potential applications and extensions of this methodology are discussed.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Shute, V. J. (2008) · Review of Educational Research · doi:10.3102/0034654307313795 · Cited by 4,631 · cited 1×
“Formative feedback offers learners concrete guidance and methods for improvement (Shute, 2008), such as strategies for evaluating and presenting objective evidence.”
MethodToward an Understanding of the Distinct Nature of L2 Writing: The ESL Research and Its Implications
Silva, T. (1993) · TESOL Quarterly · doi:10.2307/3587400 · Cited by 860 · cited 2×
“However, it is plausible that criteria for ‘good’ and ‘poor’ writing and formative feedback will differ based on the writer’s academic grade level (Crossley et al., 2011), degree of proficiency with the written language (Silva, 1993), or learning disabilities (Graham and Harris, 1993).”
BackgroundAutomated writing evaluation: Defining the classroom research agenda
Warschauer, M., & Ware, P. (2006) · Language Teaching Research · doi:10.1191/1362168806lr190oa · Cited by 337
Automated Writing Assessment in the Classroom
Warschauer, M., & Grimes, D. (2008) · Pedagogies: An International Journal · doi:10.1080/15544800701771580 · Cited by 243 · cited 2×
“Warschauer and Grimes (2008) surveyed principals, teachers, and students in four schools (6th to 12th grade) about their experiences with Criterion or MyAccess.”
BackgroundMotivating revision of drafts through formative feedback
McGarrell, H., & Verbeem, J. (2007) · ELT Journal · doi:10.1093/elt/ccm030 · Cited by 102
Cited by
Comparing the quality of human and ChatGPT feedback of students’ writing
Steiss, J., Tate, T., Graham, S., Cruz, J., Hebert, M., Wang, J., Moon, Y., Tseng, W., Warschauer, M., & Olson, C. B. (2024) · Learning and Instruction · doi:10.1016/j.learninstruc.2024.101894 · Cited by 509
A hierarchical classification approach to automated essay scoring
McNamara, D. S., Crossley, S. A., Roscoe, R. D., Allen, L. K., & Dai, J. (2015) · Assessing Writing · doi:10.1016/j.asw.2014.09.002 · Cited by 241
Self-citationMaier, U., & Klotz, C. (2022) · Computers and Education: Artificial Intelligence · doi:10.1016/j.caeai.2022.100080 · Cited by 181
Roscoe, R. D., Wilson, J., Johnson, A. C., & Mayra, C. R. (2017) · Computers in Human Behavior · doi:10.1016/j.chb.2016.12.076 · Cited by 98
Self-citation(2016) · Grantee Submission · Cited by 79
Metadata
- Authors
- Roscoe, R. D., Varner, L. K., Crossley, S. A., & McNamara, D. S.
- Venue
- International Journal of Learning Technology
- Publisher
- Inderscience Publishers
- Published
- 2013
- Volume
- 8
- Issue
- 4
- Pages
- 362
- ISSN
- 1477-8386 (ISSN-L)
- 1741-8119
- OpenAlex
- W2122161412
- Cited by
- 40
- References
- 5
- FWCI
- 12.77
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Developing pedagogically-guided algorithms for intelligent writing feedback. https://app.cochat.ai/papers/10.1504/ijlt.2013.059131
- The paper
- Roscoe, R. D., Varner, L. K., Crossley, S. A., & McNamara, D. S. (2013). Developing pedagogically-guided algorithms for intelligent writing feedback. International Journal of Learning Technology. https://doi.org/10.1504/ijlt.2013.059131
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.