The DOI matches its Crossref record; 50 of the 51 references it cites were found in a registry or an index.
L2 student engagement with automated feedback on writing: Potential for learning and issues of trust
Abstract
Abstract Recent case-study research on L2 student engagement with automated writing evaluation (AWE) feedback shows it is not simply a matter of understanding and implementing suggested changes but can instead be strongly influenced by individual and contextual factors. The present multiple-case study builds on this work by investigating the extent to which engagement supports learning from, and is influenced by trust in, AWE feedback. Six Mandarin L1 university students at different levels of English writing instruction participated, each providing a text they had written for a current course, which was submitted to the tool Grammarly for feedback. Screen-capture recordings, stimulated recalls, and interviews were used to collect information about how much of the feedback participants viewed, which of it they accepted, which they ignored or modified, and why they did so. Findings showed participants adopting a proofreading as opposed to a learning orientation toward the feedback, perhaps because most of it included specific corrections, thus requiring little work on the user’s part. Whereas previous studies attributed more successful engagement with AWE feedback to higher language proficiency, the present research showed different dimensions of trust in Grammarly’s feedback to be determining factors.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Results showed high engagement, increased agency in tool use, and flexible integration of AI-generated text, contributing to both linguistic creativity and learner autonomy.”
From AI-Integrated Scaffolding to Enhance Agency and Creativity in K-12 English Language Learners: A Systematic Review (2025) · cites this paper as [34]
In the citing paper
“Results showed high engagement, increased agency in tool use, and flexible integration of AI-generated text, contributing to both linguistic creativity and learner autonomy.”
From AI-Integrated Scaffolding to Enhance Agency and Creativity in K-12 English Language Learners: A Systematic Review (2025) · cites this paper as [34]
In the citing paper
“Results showed high engagement, increased agency in tool use, and flexible integration of AI-generated text, contributing to both linguistic creativity and learner autonomy.”
From AI-Integrated Scaffolding to Enhance Agency and Creativity in K-12 English Language Learners: A Systematic Review (2025) · cites this paper as [34]
In the citing paper
“Ranalli [34] provided important insights into the role of automated grammar support, finding that precise feedback on language use led to more sophisticated sentence structures without compromising student agency in the revision process.”
From AI-Integrated Scaffolding to Enhance Agency and Creativity in K-12 English Language Learners: A Systematic Review (2025) · cites this paper as [34]
In the citing paper
“This hands-on approach helped students develop both procedural and metacognitive skills, allowing them to make more informed decisions about integrating AI feedback into their writing.”
From AI-Integrated Scaffolding to Enhance Agency and Creativity in K-12 English Language Learners: A Systematic Review (2025) · cites this paper as [34]
Citations
Cites
The Measurement of Observer Agreement for Categorical Data
Landis, J. R., & Koch, G. G. (1977) · Biometrics · doi:10.2307/2529310 · Cited by 81,148
(2009) · Cited by 21,330
(2008) · Cited by 7,674
Trust in Automation: Designing for Appropriate Reliance
Lee, J. D., & See, K. A. (2004) · Human Factors: The Journal of the Human Factors and Ergonomics Society · doi:10.1518/hfes.46.1.50_30392 · Cited by 6,605
Teddlie, C. B., & Tashakkori, A. (2008) · Cited by 3,691
Trust, control strategies and allocation of function in human-machine systems
LEE, J., & MORAY, N. (1992) · Ergonomics · doi:10.1080/00140139208967392 · Cited by 1,459
(2017) · CRC Press eBooks · doi:10.4324/9781315155401 · Cited by 1,199
Feedback on second language students' writing
Hyland, K., & Hyland, F. (2006) · Language Teaching · doi:10.1017/s0261444806003399 · Cited by 1,048
(1995) · Choice Reviews Online · doi:10.5860/choice.32-2769 · Cited by 1,000
MUIR, B. M., & MORAY, N. (1996) · Ergonomics · doi:10.1080/00140139608964474 · Cited by 993
Performance Consequences of Automation-Induced 'Complacency'
Parasuraman, R., Molloy, R., & Singh, I. L. (1993) · The International Journal of Aviation Psychology · doi:10.1207/s15327108ijap0301_1 · Cited by 875
Feedback in Second Language Writing: Contexts and Issues
Hyland, K., & Hyland, F. (2006) · Cited by 579
Student engagement with teacher and automated feedback on L2 writing
Zhang, Z. (., & Hyland, K. (2018) · Assessing Writing · doi:10.1016/j.asw.2018.02.004 · Cited by 546
Exploring learner engagement with written corrective feedback in a Chinese tertiary EFL classroom
Han, Y., & Hyland, F. (2015) · Journal of Second Language Writing · doi:10.1016/j.jslw.2015.08.002 · Cited by 530
Ellis, R. (2010) · Studies in Second Language Acquisition · doi:10.1017/s0272263109990544 · Cited by 479
Koltovskaia, S. (2020) · Assessing Writing · doi:10.1016/j.asw.2020.100450 · Cited by 447
The Perceived Utility of Human and Automated Aids in a Visual Detection Task
Dzindolet, M. T., Pierce, L. G., Beck, H. P., & Dawe, L. A. (2002) · Human Factors: The Journal of the Human Factors and Ergonomics Society · doi:10.1518/0018720024494856 · Cited by 386
Learning-to-Write and Writing-to-Learn in an Additional Language
(2011) · Language learning and language teaching · doi:10.1075/lllt.31 · Cited by 385
ATTENTION, AWARENESS, AND INDIVIDUAL DIFFERENCES IN LANGUAGE LEARNING
Schmidt, R. (2010) · Cited by 359
Written corrective feedback for individual L2 writers
Ferris, D. R., Liu, H., Sinha, A., & Senna, M. (2012) · Journal of Second Language Writing · doi:10.1016/j.jslw.2012.09.009 · Cited by 348
LEARNERS' USES OF TWO TYPES OF WRITTEN FEEDBACK ON A L2 WRITING REVISION TASK
Sachs, R., & Polio, C. G. (2007) · Studies in Second Language Acquisition · doi:10.1017/s0272263107070039 · Cited by 326
Written Corrective Feedback for L2 Development
Bitchener, J., & Storch, N. (2016) · Multilingual Matters eBooks · doi:10.21832/9781783095056 · Cited by 286
Shintani, N., & Ellis, R. (2013) · Journal of Second Language Writing · doi:10.1016/j.jslw.2013.03.011 · Cited by 278
The effects of computer-generated feedback on the quality of writing
Stevenson, M., & Phakiti, A. (2014) · Assessing Writing · doi:10.1016/j.asw.2013.11.007 · Cited by 275
Zhang, Z. (. (2020) · Assessing Writing · doi:10.1016/j.asw.2019.100439 · Cited by 256
Eye Movements in Reading: Models and Data
Rayner, K. (2009) · Journal of Eye Movement Research · doi:10.16910/jemr.2.5.2 · Cited by 243
Sociocognitive Perspectives on Language Use and Language Learning
Hilton, H. (2011) · System · doi:10.1016/j.system.2011.06.006 · Cited by 240
Automated written corrective feedback: how well can students make use of it?
Ranalli, J. (2018) · Computer Assisted Language Learning · doi:10.1080/09588221.2018.1428994 · Cited by 228
Self-citationIn the face of fallible AWE feedback: how do students respond?
Bai, L., & Hu, G. (2016) · Educational Psychology · doi:10.1080/01443410.2016.1223275 · Cited by 209
Validity arguments for diagnostic assessment using automated writing evaluation
Chapelle, C. A., Cotos, E., & Lee, J. (2015) · Language Testing · doi:10.1177/0265532214565386 · Cited by 158
(2004) · Cited by 128
Errors and Intelligence in Computer-Assisted Language Learning: Parsers and Pedagogues
Heift, T., & Schulze, M. (2007) · Cited by 118
Gaze Path Stimulation in Retrospective Think-Aloud
Hyrskykari, A., Ovaska, S., Majaranta, P., Räihä, K., & Lehtinen, M. (2008) · Journal of Eye Movement Research · doi:10.16910/jemr.2.4.5 · Cited by 111
(2018) · doi:10.1201/9781315137957-2 · Cited by 108
Stefanou, C., & Révész, A. (2015) · Modern Language Journal · doi:10.1111/modl.12212 · Cited by 106
Lee, G., & Schallert, D. L. (2008) · Journal of Second Language Writing · doi:10.1016/j.jslw.2007.11.002 · Cited by 106
Does Automated Feedback Help Students Learn to Write?
Kellogg, R. T., Whiteford, A. P., & Quinlan, T. (2010) · Journal of Educational Computing Research · doi:10.2190/ec.42.2.c · Cited by 88
Kim, H. R., & Bowles, M. A. (2019) · TESOL Quarterly · doi:10.1002/tesq.522 · Cited by 83
Exploring the integration of automated feedback among lower-proficiency EFL learners
Huang, S. R., & Renandya, W. A. (2018) · Innovation in Language Learning and Teaching · doi:10.1080/17501229.2018.1471083 · Cited by 78
Automated Feedback and Second Language Writing
Stevenson, M., & Phakiti, A. (2019) · Cambridge University Press eBooks · doi:10.1017/9781108635547.009 · Cited by 77
Calhoun, C. S., Bobko, P., Gallimore, J. J., & Lyons, J. B. (2019) · Journal of Trust Research · doi:10.1080/21515581.2019.1579730 · Cited by 74
Li, Z., Feng, H.-H., & Sarıcaoğlu, A. (2017) · CALICO Journal · doi:10.1558/cj.26382 · Cited by 53
Automated Grammatical Error Detection for Language Learners, Second Edition
Leacock, C., Chodorow, M. S., Gamon, M., & Tetreault, J. (2014) · Synthesis lectures on human language technologies · doi:10.2200/s00562ed1v01y201401hlt025 · Cited by 46
Using Grammar Checkers in an ESL Context
John, P. S., & Woll, N. (2020) · CALICO Journal · doi:10.1558/cj.36523 · Cited by 44
Automated written corrective feedback: Error-correction performance and timing of delivery
Ranalli, J., & Yamashita, T. (2022) · Language learning & technology · doi:10.64152/10125/73465 · Cited by 38
Self-citationDoes Automated Feedback Improve Writing Quality
Wilson, J., Olinghouse, N. G., & Andrada, G. N. (2014) · Learning disabilities · Cited by 35
EFL business writing behaviors in differing feedback environments
Warden, C. A. (2000) · Language Learning · doi:10.1111/0023-8333.00141 · Cited by 34
Automated Grammatical Error Detection for Language Learners, Second Edition
Leacock, C., Chodorow, M. S., Gamon, M., & Tetreault, J. (2014) · Synthesis lectures on human language technologies · doi:10.1007/978-3-031-02153-4 · Cited by 33
Hyland, F. (2011) · The HKU Scholars Hub (University of Hong Kong) · Cited by 24
Students' engagement with feedback on writing: the role of learner agency/beliefs
Storch, N., & Wigglesworth, G. (2010) · Cited by 24
Cited by
Sorted by citations
AI in the Foreign Language Classroom: A Pedagogical Overview of Automated Writing Assistance Tools
Alharbi, W. (2023) · Education Research International · doi:10.1155/2023/4253331 · Cited by 286
“Some researchers, such as Ranalli [30], went further and strongly criticized the use of AWE tools in the language classroom and pointed out that these tools “ did not live up to expectations ” (p. 2).”
BackgroundMethodInfluentialA systematic review of AI-based automated written feedback research
Shi, H., & Aryadoust, V. (2024) · ReCALL · doi:10.1017/s0958344023000265 · Cited by 222
“In addition, the instructor ’ s willingness to apply AWF (e.g. Jiang et al ., 2020), perceived trust in AWF (e.g. Ranalli, 2021; Zhai & Ma, 2021), perceived ease of use (R. Li, 2021), and self-efficacy (e.g. Li et al ., 2019) are all found to play a role in users ’ interaction with AWF.”
Wang, C. (2025) · Technology Knowledge and Learning · doi:10.1007/s10758-024-09744-3 · Cited by 212
“Researchers also note challenges such as the varied level of technological readiness among teachers and students as well as concerns regarding accuracy, biases, accountability, transparency, and ethics (e.g., Kohnke et al., 2023; Memarian & Doleck, 2023; Ranalli, 2021).”
A review of AWE feedback: types, learning outcomes, and implications
Fu, Q.-K., Zou, D., Xie, H., & Cheng, G. (2022) · Computer Assisted Language Learning · doi:10.1080/09588221.2022.2033787 · Cited by 210
“Nevertheless, we reviewed the publications in these journals because they have been widely recognized as the most rigorous among research journals (Authors, 2020, 2021).”
BackgroundZhan, Y., & Yan, Z. (2026) · Assessment & Evaluation in Higher Education · doi:10.1080/02602938.2025.2471821 · Cited by 162
“In previous studies, researchers identified that students used similar cognitive strategies, such as noticing and understanding (e.g. Zhang 2017; Koltovskaia 2020; Ranalli 2021).”
Metadata
- Authors
- Ranalli, J.
- Affiliations
- Iowa State University (US)
- Corresponding author
- Ranalli, Jim
- ORCID
- Ranalli, Jim
- Venue
- Journal of Second Language Writing
- Publisher
- Elsevier BV
- Published
- 2021-06
- Volume
- 52
- Pages
- 100816
- ISSN
- 1060-3743 (ISSN-L)
- OpenAlex
- W3148248532
- Semantic Scholar
- bc20929cd12bf53ca312d3549b91fc4e442bebd3
- Cited by
- 184
- References
- 51
- Citations by year
- 2026: 68 · 2025: 46 · 2024: 30 · 2023: 18 · 2022: 19 · 2021: 3
- FWCI
- 69.13
- Type
- journal article
- Language
- en
- Open access
- closed
- Licence
- Locations
- Journal of Second Language Writing (publishedVersion)
- Topics
- Social Sciences › Arts and Humanities › Language and Linguistics › EFL/ESL Teaching and Learning
- Social Sciences › Education › Student Assessment and Feedback
- Social Sciences › Education › English Language Learning and Teaching
- Keywords
- automated writing evaluation, student engagement with feedback, Grammarly, second language writing, learning orientation, feedback acceptance, English as a foreign language
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
