Automated writing evaluation: Defining the classroom research agenda
Abstract
With the advent of English as a global language, the ability to write well in English across diverse settings and for different audiences has become an imperative in second language education programmes throughout the world. Yet the teaching of second language writing is often hindered by the great amount of time and skill needed to evaluate repeated drafts of student writing. Online Automated Writing Evaluation programmes have been developed as a way to meet this challenge, and the scoring engines driving such programmes have been analysed in a considerable array of psychometric studies. However, relatively little research has been conducted on how AWE is used in the classroom and the results achieved with such use. In this article, we analyse recent developments in automated writing evaluation, explain the bases on which AWE systems operate, synthesize research with these systems, and propose a multifaceted process/product research programme on the instructional use of AWE. We explore this emerging area of inquiry by proposing a range of potential questions, methodologies and analytical tools that can define such a research agenda.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The opportunity to revise an essay multiple times at the learner's own pace is an advantage of AWE feedback.
“We have seen here that the opportunity to revise an essay multiple times at the learner's own pace is also an advantage of AWE feedback (cf Warschauer & Ware, 2006).”
From Student engagement with teacher and automated feedback on L2 writing (2018) · cites this paper as Warschauer & Ware, 2006
Cites
Researching technology in TESOL: Determinist, instrumental, and critical approaches
Warschauer, M. (1998) · TESOL Quarterly · doi:10.2307/3588010 · Cited by 123 · cited 2× · see citing works
“And technologies do not exert an ‘impact’ but instead help reshape the broader ecology the classroom and school (Warschauer, 1998).”
Background
Flagged first, then by citations
Cited by
Sorted by citations
Exploring the potential of using an AI language model for automated essay scoring
Mizumoto, A., & Eguchi, M. (2023) · Research Methods in Applied Linguistics · doi:10.1016/j.rmal.2023.100050 · Cited by 647 · see citing works
Student engagement with teacher and automated feedback on L2 writing
Zhang, Z. (., & Hyland, K. (2018) · Assessing Writing · doi:10.1016/j.asw.2018.02.004 · Cited by 546 · see citing works
Comparing the quality of human and ChatGPT feedback of students’ writing
Steiss, J., Tate, T., Graham, S., Cruz, J., Hebert, M., Wang, J., Moon, Y., Tseng, W., Warschauer, M., & Olson, C. B. (2024) · Learning and Instruction · doi:10.1016/j.learninstruc.2024.101894 · Cited by 509 · see citing works
Self-citationKoltovskaia, S. (2020) · Assessing Writing · doi:10.1016/j.asw.2020.100450 · Cited by 447 · see citing works
Chen, C.-F. E., & Cheng, W.-Y. E. C. (2008) · Language Learning & Technology · doi:10.64152/10125/44145 · Cited by 304 · see citing works
Link, S., Mehrzad, M., & Rahimi, M. (2022) · Computer Assisted Language Learning · doi:10.1080/09588221.2020.1743323 · Cited by 275 · see citing works
Barrot, J. S. (2023) · Computer Assisted Language Learning · doi:10.1080/09588221.2021.1936071 · Cited by 248 · see citing works
Automated Writing Evaluation and Feedback: Multiple Metrics of Efficacy
Wilson, J., & Roscoe, R. D. (2020) · Journal of Educational Computing Research · doi:10.1177/0735633119830764 · Cited by 176 · see citing works
Ferreira‐Mello, R., André, M., Pinheiro, A., Costa, E., & Romero, C. (2019) · WIREs Data Mining and Knowledge Discovery · doi:10.1002/widm.1332 · Cited by 193 · see citing works
Automated essay scoring: Psychometric guidelines and practices
Ramineni, C., & Williamson, D. M. (2013) · Assessing Writing · doi:10.1016/j.asw.2012.10.004 · Cited by 127 · see citing works
Shi, H., Chai, C. S., Zhou, S., & Aubrey, S. (2026) · Computer Assisted Language Learning · doi:10.1080/09588221.2025.2454541 · Cited by 109 · see citing works
Testing the viability of ChatGPT as a companion in L2 writing accuracy assessment
Mizumoto, A., Shintani, N., Sasaki, M., & Teng, M. F. (2024) · Research Methods in Applied Linguistics · doi:10.1016/j.rmal.2024.100116 · Cited by 104 · see citing works
The impact of computer-based feedback on students' written work?
El Ebyary, K., & Windeatt, S. (2010) · International Journal of English Studies · doi:10.6018/ijes/2010/2/119231 · Cited by 94 · see citing works
Which do students prefer to evaluate their essays: peers or computer program
Lai, Y. (2010) · British Journal of Educational Technology · doi:10.1111/j.1467-8535.2009.00959.x · Cited by 92 · see citing works
Wei, P., Wang, X., & Dong, H. (2023) · Frontiers in Psychology · doi:10.3389/fpsyg.2023.1249991 · Cited by 89 · see citing works
Checking and bootstrapping lexical norms by means of word similarity indexes
Bestgen, Y., & Vincze, N. (2012) · Behavior Research Methods · doi:10.3758/s13428-012-0195-z · Cited by 75 · see citing works
Can AI Provide Useful Holistic Essay Scoring?
Tate, T. P., Steiss, J., Bailey, D., Graham, S., Moon, Y., Ritchie, D., Tseng, W., & Warschauer, M. (2024) · Computers and Education: Artificial Intelligence · doi:10.1016/j.caeai.2024.100255 · Cited by 60 · see citing works
Self-citationShen, C., Shi, P., Guo, J., Xu, S., & Tian, J. (2023) · Frontiers in Psychology · doi:10.3389/fpsyg.2023.1258286 · Cited by 42 · see citing works
Automated Scoring of Chinese Engineering Students’ English Essays
Liu, M., Wang, Y., Xu, W., & Liu, L. (2017) · International Journal of Distance Education Technologies · doi:10.4018/ijdet.2017010104 · Cited by 17 · see citing works
Tate, T. P., Harnick-Shapiro, B., Ritchie, D. R., Tseng, W., Dennin, M., & Warschauer, M. (2025) · Discover Computing · doi:10.1007/s10791-025-09563-9 · Cited by 14 · see citing works
“Studies of the use of digital tools to support writing have evolved over decades, ranging from analyses of digital writing compared to writing by hand [2, 7, 37, 39], [66], to the use of automated writing evaluation to provide writing feedback and improve writing quality [22, 36, 57, 63].”
Self-citation
- Authors
- Warschauer, M., & Ware, P.
- Affiliations
- University of California, Irvine (US)
- Southern Methodist University (US)
- Corresponding author
- Warschauer, Mark
- Venue
- Language Teaching Research
- Publisher
- SAGE Publications
- Published
- 2006-04
- Volume
- 10
- Issue
- 2
- Pages
- 157-180
- ISSN
- 1362-1688 (ISSN-L)
- 1477-0954
- OpenAlex
- W2003577900
- Cited by
- 337
- References
- 1
- Citations by year
- 2026: 33 · 2025: 30 · 2024: 30 · 2023: 27 · 2022: 21 · 2021: 23 · 2020: 14 · 2019: 11 · 2018: 16 · 2017: 9 · 2016: 19 · 2015: 15 · 2014: 14 · 2013: 14 · 2012: 12
- FWCI
- 70.87
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Language Teaching Research (publishedVersion)
- Topics
- Social Sciences › Arts and Humanities › Language and Linguistics › EFL/ESL Teaching and Learning
- Physical Sciences › Computer Science › Information Systems › English Language Learning and Teaching
- Social Sciences › Psychology › Developmental and Educational Psychology › Second Language Acquisition and Learning
- Keywords
- automated writing evaluation, second language writing, writing instruction, research agenda, writing assessment
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
