Beyond the design of automated writing evaluation: Pedagogical practices and perceived learning effectiveness in EFL writing classes?
Open accessThe DOI matches its Crossref record; 17 of the 17 references it cites were found in a registry or an index.
Abstract
Automated writing evaluation (AWE) software is designed to provide instant computergenerated scores for a submitted essay along with diagnostic feedback.Most studies on AWE have been conducted on psychometric evaluations of its validity; however, studies on how effectively AWE is used in writing classes as a pedagogical tool are limited.This study employs a naturalistic classroom-based approach to explore the interaction between how an AWE program, MY Access!, was implemented in three different ways in three EFL college writing classes in Taiwan and how students perceived its effectiveness in improving writing.The findings show that, although the implementation of AWE was not in general perceived very positively by the three classes, it was perceived comparatively more favorably when the program was used to facilitate students' early drafting and revising process, followed by human feedback from both the teacher and peers during the later process.This study also reveals that the autonomous use of AWE as a surrogate writing coach with minimal human facilitation caused frustration to students and limited their learning of writing.In addition, teachers' attitudes toward AWE use and their technology-use skills, as well as students' learner characteristics and goals for learning to write, may also play vital roles in determining the effectiveness of AWE.With limitations inherent in the design of AWE technology, language teachers need to be more critically aware that the implementation of AWE requires well thought-out pedagogical designs and thorough considerations for its relevance to the objectives of the learning of writing.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“AWE feedback has discernible advantages over teacher feedback in terms of timeliness, convenience, multiple drafting, and even potential learner autonomy, which have also been shown in the previous studies (Chen & Cheng, 2008;Dikli, 2006).”
From Student engagement with teacher and automated feedback on L2 writing (2018) · cites this paper as Chen & Cheng, 2008
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
This paper’s reference list could not be split into single entries.
Assessment and Classroom Learning
Black, P., & Wiliam, D. (1998) · Assessment in Education: Principles, Policy & Practice · doi:10.1080/0969595980050102 · Cited by 8,152
Feedback on second language students' writing
Hyland, K., & Hyland, F. (2006) · Language Teaching · doi:10.1017/s0261444806003399 · Cited by 1,016
AUTOMATED ESSAY SCORING WITH E‐RATER® V.2.0
Attali, Y., & Burstein, J. (2004) · ETS Research Report Series · doi:10.1002/j.2333-8504.2004.tb01972.x · Cited by 622
Automated Essay Scoring: A Cross-disciplinary Perspective
(2003) · Routledge eBooks · doi:10.4324/9781410606860 · Cited by 501
Language Testing and Validation: An Evidence-Based Approach
Weir, C. J. (2004) · Cited by 463
(2006) · The Journal of Technology, Learning and Assessment (Boston College) · Cited by 424
Teaching & Researching: Computer-Assisted Language Learning
Beatty, K. (2013) · doi:10.4324/9781315833774 · Cited by 375
Automated writing evaluation: Defining the classroom research agenda
Warschauer, M., & Ware, P. (2006) · Language Teaching Research · doi:10.1191/1362168806lr190oa · Cited by 337
Stumping e-rater:challenging the validity of automated essay scoring
Powers, D. E., Burstein, J. C., Chodorow, M. S., Fowles, M. E., & Kukich, K. (2002) · Computers in Human Behavior · doi:10.1016/s0747-5632(01)00052-8 · Cited by 155
Page, E. B. (2003) · Cited by 148
Cited by
Items 301–316 of 316
From Human Raters to AI: A Critical Review of Research on ChatGPT in Writing Assessment
Wang, Y.-M., Harmer, P., & Xue, C. (2026) · Futurity Education · doi:10.57125/fed.2026.06.12
“…into educational settings is that students generally appreciate the immediate feedback it provides (Ariyanto et al., 2021; Correnti et al., 2022; Chen & Cheng, 2008; Gao, 2021), unlike the traditional teacher-rated approach, where students often wait days or even weeks to receive feedback…”
InfluentialCheng, T. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-32115-2_8
The role of psychological contract in sustaining learner engagement with AWE systems
Hu, X., Hu, J., Chen, S., & Li, Z. (2026) · Assessing Writing · doi:10.1016/j.asw.2026.101101
The Use of ChatGPT Chatbot in Enhancing Writing Skills of ESL Students
Yaprak, Z. (2026) · Journal of Language Research · doi:10.51726/jlr.1879060
Jitpaisarnwattana, N., & Saville, N. (2026) · LEARN Journal : Language Education and Acquisition Research Network · doi:10.70730/xywk3308 · Cited by 1
Toward More Effective Automated Writing Evaluation Through Relational Trust
Goldshtein, M., Chiou, E. K., & Roscoe, R. D. (2026) · Technology Knowledge and Learning · doi:10.1007/s10758-026-09999-y
Guo, J., Ren, J., Lu, Z., Zhang, Y., Zhang, H., Su, H., Ding, L., Zhang, S., Zhang, N., Pan, S., et al. (2026) · Systems · doi:10.3390/systems14091100
Chen, X., Liu, J., Ye, P., Li, Y., Sun, H., Li, R., Hou, Z., & Yan, J. (2026) · Journal of Computer Assisted Learning · doi:10.1002/jcal.70315
Evaluating AI feedback: teacher expertise and hidden labour in writing instruction
Li, W., Sulaiman, N. A., & Matore, M. E. E. M. (2026) · English in Education · doi:10.1080/04250494.2026.2723772
Pang, S., Mamoon, M., & Song, P. (2026) · SSRN Electronic Journal · doi:10.2139/ssrn.7496158
Le, T. T. P., & Bui, H. M. T. (2026) · Dalat University Journal of Science · doi:10.37569/dalatuniversity.16.4.1498(2026
THE ROLE OF ARTIFICIAL INTELLIGENCE IN THE DEVELOPMENT OF WRITING SKILLS
qizi, O. G. M. (2026) · JOURNAL OF INTERNATIONAL SCIENTIFIC RESEARCH
THE ROLE OF ENGLISH IN MODERN EDUCATION
Botirovna, S. Z., & Alisherovna, N. M. (2026) · JOURNAL OF INTERNATIONAL SCIENTIFIC RESEARCH
Is Gen AI an editor or a revision tool? A case study of EFL students in AI-assisted revision
Zhang, H., 艾姗姗, Zhang, L., & Lin, Y. (2026) · PLoS ONE · doi:10.1371/journal.pone.0359661
“Although AWE may facilitate early drafting and surface-level revision [22], learner uptake often remains limited.”
Sindia.T, Selvaraj, V., Sathikulameen, A., & Ojong, A. S. (2026) · Journal of Interdisciplinary Studies in Education · doi:10.32674/0nwtbv54
Metadata
- Authors
- Chen, C.-F. E., & Cheng, W.-Y. E. C.
- Affiliations
- National Kaohsiung First University of Science and Technology (TW)
- Venue
- Language Learning & Technology
- Published
- 2008-06-01
- Volume
- 12
- Issue
- 2
- Pages
- 94-112
- ISSN
- 1094-3501 (ISSN-L)
- OpenAlex
- W2122946606
- Cited by
- 304
- References
- 17
- Citations by year
- 2026: 22 · 2025: 17 · 2024: 33 · 2023: 32 · 2022: 28 · 2021: 29 · 2020: 15 · 2019: 16 · 2018: 19 · 2017: 11 · 2016: 17 · 2015: 17 · 2014: 13 · 2013: 13 · 2012: 9
- FWCI
- 43.13
- Type
- journal article
- Language
- en
- Open access
- diamond
- doi.org
- Licence
- Locations
- Language learning & technology (publishedVersion, cc-by-nc-nd)
- ScholarSpace (University of Hawaii at Manoa) (submittedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion)
- Language learning & technology (cc-by-nc-nd)
- Topics
- Social Sciences › Arts and Humanities › Language and Linguistics › EFL/ESL Teaching and Learning
- Social Sciences › Education › English Language Learning and Teaching
- Social Sciences › Education › Student Assessment and Feedback
- Keywords
- automated writing evaluation, EFL writing, writing instruction, student perceptions, human feedback, peer feedback, teacher attitudes, learner characteristics, technology-enhanced language learning
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.