CoChat
Sign Up
Cited by 2Artificial Intelligence in EducationStudent Assessment and FeedbackEFL/ESL Teaching and Learning

LLM feedback for academic writing: Effects on students’ performance and engagement

Open access
Glüsing, Robert, Fleckenstein, Johanna, Schmidt, Fabian T.C., Möller, Jens·2026·Contemporary Educational Psychology
DOI 10.1016/j.cedpsych.2026.102463
Open this paper in CoChat — free accountOpen PDF

Abstract

Writing and revising academic texts is a demanding task that benefits significantly from feedback provided by teachers or peers. However, providing elaborated formative feedback on students’ academic writing is time-intensive and therefore hard to implement in educational practice. As a supplementary resource, large language models (LLMs) offer the potential to support the writing process by generating automated feedback to help students enhance their texts. The present study examined the accuracy of LLM-generated feedback on student texts and its effectiveness in improving university students’ revision performance and engagement in academic writing. In a randomized controlled experiment, a sample of N = 144 university students wrote an abstract summarizing a research article. All participants were then instructed to revise their abstracts; half received individualized feedback generated by GPT-4 using a standardized prompting procedure. Controlling for the quality of the initial drafts, regression analyses revealed that LLM-generated feedback led to higher revision quality and increased behavioral engagement, as measured by revision time and edit distance. Furthermore, behavioral engagement partially mediated the effect of feedback on revision quality. These findings demonstrate that LLMs can provide high-accuracy, effective feedback on academic writing. The study discusses the potential applications and implications of this technology within higher education contexts.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1016/j.cedpsych.2026.102463

Cites

22 of 22 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Flagged first, then by citations

Research works printed but not linked: 63

  1. Ansarifar, A., Shahriari, H., & Pishghadam, R. (2018). Phrasal complexity in academic writing: A comparison of abstracts written by graduate students and expert writers in applied linguistics. Journal of English for Academic Purposes, 31, 58-71. https://doi. org/10.1016/j.jeap.2017.12.008 · DOI
  2. Attali, Y. (2004). Exploring the feedback and revision features of Criterion. Journal of Second Language Writing, 14(3), 191-205.
  3. Berg, B. C. (1999). The effects of trained peer response on ESL students' revision types and writing quality. Journal of Second Language Writing, 8(3), 215-241. https://doi. org/10.1016/S1060-3743(99)80115-5 · DOI
  4. Biber, D., Nekrasova, T., & Horn, B. (2011). The effectiveness of feedback for L1-English and L2-writing development: A meta-analysis. ETS Research Report Series, i-99. https://doi.org/10.1002/j.2333-8504.2011.tb02241.x · DOI · Paper page
  5. Black, P., & Wiliam, D. (1998). Assessment and Classroom Learning. Assessment in Education, 5, 7-74. https://doi.org/10.1080/0969595980050102 · DOI · Paper page
  6. Brookhart, S. M. (1997). A theoretical framework for the role of classroom assessment in motivating student effort and achievement. Applied Measurement in Education, 10, 161-180. https://doi.org/10.1207/s15324818ame1002_4 · DOI
  7. Carvalho, C., Santos, N. N., António, R., & Martins, D. S. M. (2021). Supporting students' engagement with teachers' feedback: The role of students' school identification. Educational Psychology, 41(7), 863-882. https://doi.org/10.1080/ 01443410.2020.1849564 · DOI
  8. Cauley, K. M., & McMillian, J. H. (2010). Formative assessment techniques to support student motivation and achievement. The Clearing House, 83(1), 1-6. https://doi. org/10.1080/00098650903267784 · DOI · Paper page
  9. Chan, S. T. S., Lo, N. P. K., & Wong, A. M. H. (2024). Enhancing university level English proficiency with generative AI: Empirical insights into automated feedback and learning outcomes. Contemporary Educational Technology, 16(4), Article 10.30935/ cedtech/15607. · DOI · Paper page
  10. Chen, S., & Cheung, A. C. (2025). Effect of generative artificial intelligence on university students learning outcomes: A systematic review and meta-analysis. Educational Research Review, 49, Article 100737. https://doi.org/10.1016/j.edurev.2025.100737 · DOI
  11. Cho, K., & Schunn, C. (2007). Scaffolded writing and rewriting in the discipline: A web- based reciprocal peer review system. Computers & Education, 48, 409-426. https:// doi.org/10.1016/j.compedu.2005.02.004 · DOI
  12. Choi, S.-Y., & Li, S. (2012). Corrective feedback and learner uptake in a child ESOL classroom. RELC Journal, 43(3), 331-351. https://doi.org/10.1177/ 0033688212463274 · DOI
  13. Drury, A., Pape, E., Dowling, M., Miguel, S., Fernández-Ortega, P., Papadopoulou, C., & Kotronoulas, G. (2023). How to write a comprehensive and informative research abstract. Seminars in oncology nursing, 39(2), Article 151395. https://doi.org/ 10.1016/j.soncn.2023.151395 · DOI
  14. Du, L., Wang, Y., Xing, X., Ya, Y., Li, X., Jiang, X., & Fang, X. (2023). Quantifying and attributing the hallucination of large language models via association analysis. arXiv preprint arXiv:2309.05217. 10.48550/arXiv.2309.05217. · DOI · Paper page
  15. Ellis, R. (2010). EPILOGUE: A framework for investigating oral and written corrective feedback. Studies in Second Language Acquisition, 32(2), 335-349. https://doi.org/ 10.1017/S0272263109990544 · DOI
  16. Fan, Y., van Der Graaf, J., Lim, L., Raković, M., Singh, S., Kilgour, J., Moore, J., Molenaar, I., Bannert, M., & Gašević, D. (2022). Towards investigating the validity of measurement of self-regulated learning based on trace data. Metacognition and Learning, 17(3), 949-987. https://doi.org/10.1007/s11409-022-09291-1 · DOI · Paper page
  17. Fleckenstein, J., Jansen, T., Meyer, J., Trüb, R., Raubach, E. E., & Keller, S. (2024). How am I going? Behavioral engagement mediates the effect of individual feedback on writing performance. Learning and Instruction, 93, Article 101977. https://doi.org/ 10.1016/j.learninstruc.2024.101977 · DOI
  18. Fong, C. J., & Schallert, D. L. (2023). "Feedback to the future": Advancing motivational and emotional perspectives in feedback research. Educational Psychologist, 58(3), 146-161. https://doi.org/10.1080/00461520.2022.2134135 · DOI
  19. Fu, Q. K., Zou, D., Xie, H., & Cheng, G. (2022). A review of AWE feedback: Types, learning outcomes, and implications. Computer Assisted Language Learning, 37(1-2), 179-221. https://doi.org/10.1080/09588221.2022.2033787 · DOI
  20. Goldstein, R. (1993). Conditioning diagnostics: Collinearity and weak data in regression. Technometrics, 35(1), 85-86. https://doi.org/10.1080/00401706.1993.10484997 · DOI
  21. Graham, S. (2018a). A revised writer(s)-within-community model of writing. Educational Psychologist, 53(4), 258-279. https://doi.org/10.1080/00461520.2018.1481406 · DOI
  22. Graham, S. (2018b). Instructional feedback in writing. In A. A. Lipnevich, & J. K. Smith (Eds.), The Cambridge Handbook of Instructional Feedback (pp. 145-168). chapter, Cambridge: Cambridge University Press. https://doi.org/10.1017/ 9781316832134.009. · DOI
  23. Graham, S. (2023). Writer(s)-Within-Community model of writing as a lens for studying the teaching of writing. In The Routledge International Handbook of Research on Writing (Second Edition, pp. 337-350). Taylor and Francis. https://doi.org/10.4324/ 9780429437991-27. · DOI
  24. Graham, S., & Harris, K. R. (2017). Evidence-based writing practices: A meta-analysis of existing meta-analyses. In Design principles for teaching effective writing (pp. 13-37). Brill. https://doi.org/10.1163/9789004270480_003. · DOI
  25. Graham, S., Hebert, M., & Harris, K. R. (2015). Formative assessment and writing: A meta-analysis. The Elementary School Journal, 115(4), 523-547. https://doi.org/ 10.1086/681947 · DOI · Paper page
  26. Graham, S., Kim, Y.-S., Cao, Y., Lee, J. W., Tate, T., Collins, P., Cho, M., Moon, Y., Chung, H. Q., & Olson, C. B. (2023). A meta-analysis of writing treatments for students in grades 6-12. Journal of Educational Psychology, 115(7), 1004-1027. https://doi.org/10.1037/edu0000819 · DOI
  27. Hahn, M. G., Navarro, S. M. B., De-La-Fuente-Valentin, L., & Burgos, D. (2021). A systematic review of the effects of automatic scoring and automatic feedback in educational settings. IEEE Access, 9, 108190-108198. https://doi.org/10.1109/ ACCESS.2021.3100890 · DOI · Paper page
  28. Hattie, J. (2009). Visible Learning: A Synthesis of over 800 Meta-analyses Relating to Achievement. Routledge.
  29. Hirst, J. M., & DiGennaro Reed, F. D. (2015). An examination of the effects of feedback accuracy on academic task acquisition in analogue settings. The Psychological Record, 65(1), 49-65. https://doi.org/10.1007/s40732-014-0087-y · DOI
  30. Horbach, A., Laarmann-Quante, R., Liebenow, L., Jansen, T., Keller, S., Meyer, J., Zesch, T., & Fleckenstein, J. (2022). Bringing automatic scoring into the classroom - Measuring the impact of automated analytic feedback on student writing performance. Proceedings of the 11th Workshop on Natural Language Processing for Computer Assisted Language Learning (NLP4CALL 2022), 190, 72-83. 10.5281/ zenodo.12779118. · DOI
  31. Hughes, J. N., Luo, W., Kwok, O.-M., & Loyd, L. K. (2008). Teacher-student support, effortful engagement, and achievement: A 3-year longitudinal study. Journal of Educational Psychology, 100(1), 1-14. https://doi.org/10.1037/0022-0663.100.1.1 · DOI
  32. Ihme, T. A., Möller, J., & Pohlmann, B. (2009). Effekte von Kooperation auf die Qualität von Lehrmaterial [Effects of cooperation on the quality of teaching material]. Zeitschrift für Pädagogische Psychologie / German Journal of Educational Psychology, 23 (3-4), 259-263. https://doi.org/10.1024/1010-0652.23.34.259 · DOI
  33. Järvelä, S., Veermans, M., & Leinonen, P. (2008). Investigating student engagement in computer-supported inquiry: A process-oriented analysis. Social Psychology of Education, 11(3), 299-322. https://doi.org/10.1007/s11218-007-9047-6 · DOI
  34. Jia, Q., Cui, J., Xi, R., Liu, C., Rashid, P., Li, R., & Gehringer, E. (2024). On assessing the faithfulness of LLM-generated feedback on student assignments. In Proceedings of the 17th International Conference on Educational Data Mining (pp. 491-499).
  35. Kaliisa, R., Misiejuk, K., López-Pernas, S., & Saqr, M. (2026). How does artificial intelligence compare to human feedback? A meta-analysis of performance, feedback perception, and learning dispositions. Educational Psychology, 46(1), 80-111. https://doi.org/10.1080/01443410.2025.2553639 · DOI
  36. Kellogg, R. T., & Whiteford, A. P. (2009). Training advanced writing skills: The case for deliberate practice. Educational Psychologist, 44, 250-266. https://doi.org/10.1080/ 00461520903213600 · DOI
  37. Lv, X., Ren, W., & Xie, Y. (2021). The effects of online feedback on ESL/EFL writing: A meta-analysis. The Asia-Pacific Education Researcher, 30, 643-653. https://doi.org/ 10.1007/s40299-021-00594-6 · DOI
  38. Meyer, J., Jansen, T., & Fleckenstein, J. (2025). Non-engagement and unsuccessful engagement with feedback in lower secondary education: The role of student characteristics. Contemporary Educational Psychology, 81, Article 102363. https:// doi.org/10.1016/j.cedpsych.2025.102363 · DOI
  39. Meyer, J., Jansen, T., Schiller, R., Liebenow, L. W., Steinbach, M., Horbach, A., & Fleckenstein, J. (2024). Using LLMs to bring evidence-based feedback into the classroom: AI-generated feedback increases secondary students' text revision, motivation, and positive emotions. Computers and Education: Artificial Intelligence, 6, Article 100199. https://doi.org/10.1016/j.caeai.2023.100199 · DOI · Paper page
  40. Min, H. T. (2005). Training students to become successful peer reviewers. System, 33(2), 293-308. https://doi.org/10.1016/j.system.2004.11.003 · DOI
  41. Mulliner, E., & Tucker, M. (2017). Feedback on feedback practice: Perceptions of students and academics. Assessment & Evaluation in Higher Education, 42(2), 266-288. https://doi.org/10.1080/02602938.2015.1103365 · DOI · Paper page
  42. Murray, D. M. (1978). Internal revision: A process of discovery. In C. Cooper, & L. Odell (Eds.), Research on composing: Points of departure (pp. 85-103). Urbana, IL: National Council of Teachers of English.
  43. Nguyen, T. T. D. T., Garncarz, T., Ng, F., Dabbish, L. A., & Dow, S. P. (2017). Fruitful feedback: Positive affective language and source anonymity improve critique reception and work outcomes. Proceedings of the 2017 ACM Conference on Computer Supported Cooperative Work and Social Computing, 1024-1034. 10.1145/ 2998181.2998319. · DOI
  44. Nunes, A., Cordeiro, C., Limpo, T., & Castro, S. L. (2022). Effectiveness of automated writing evaluation systems in school settings: A systematic review of studies from 2000 to 2020. Journal of Computer Assisted Learning, 38(2), 599-620. https://doi. org/10.1111/jcal.12635 · DOI
  45. OpenAI. (2023). GPT-4 technical report. arXiv:2303.08774. 10.48550/ arXiv.2303.08774. · DOI · Paper page
  46. Ramesh, D., & Sanampudi, S. K. (2022). An automated essay scoring systems: A systematic literature review. Artificial Intelligence Review, 55, 2495-2527. https:// doi.org/10.1007/s10462-021-10068-2 · DOI · Paper page
  47. Roberts, D. (1982). Teaching abstracts in technical writing: Early and often. Technical Writing Teacher, 10(1), 12-16.
  48. Rosseel, Y. (2012). lavaan: An R Package for Structural Equation Modeling. Journal of Statistical Software, 48(2), Article 10.18637/jss.v048.i02. · DOI · Paper page
  49. Rupp, A. A., Casabianca, J. M., Krüger, M., Keller, S., & Köller, O. (2019). Automated essay scoring at scale: A case study in Switzerland and Germany. ETS Research Report Series, 1-23. https://doi.org/10.1002/ets2.12249 · DOI
  50. Schiller, R., Meyer, J., Mertens, U., & Fleckenstein, J. (under review). Validating process measures of task-level behavioral learning engagement during text revision. Manuscript submitted for publication.
  51. Soto, X., Olszewski, A., & Goldstein, H. (2019). A systematic review of phonological awareness interventions for latino children in early and primary grades. Journal of Early Intervention, 41(4), 340-365. https://doi.org/10.1177/1053815119856067 · DOI
  52. Tärning, B., Lee, Y. J., Andersson, R., Månsson, K., Gulz, A., & Haake, M. (2020). Assessing the black box of feedback neglect in a digital educational game for elementary school. Journal of the Learning Sciences, 29(4-5), 511-549. https://doi. org/10.1080/10508406.2020.1770092 · DOI
  53. Tingley, D., Yamamoto, T., Hirose, K., Keele, L., & Imai, K. (2014). mediation: R Package for Causal Mediation Analysis. Journal of Statistical Software, 59(5), Article 10.18637/jss.v059.i05. · DOI
  54. van der Loo, M. (2014). "The stringdist package for approximate string matching." the. R Journal, 6, 111-122. https://doi.org/10.32614/RJ-2014-011 · DOI
  55. van Niekerk, J., Delport, P. M., & Sutherland, I. (2025). Addressing the use of generative AI in academic writing. Computers and Education: Artificial Intelligence, 100342. https://doi.org/10.1016/j.caeai.2024.100342 · DOI · Paper page
  56. Wainer, H., & Thissen, D. (1993). Combining multiple-choice and constructed-response test scores: Toward a marxist theory of test construction. Applied Measurement in Education, 6(2), 103-118. https://doi.org/10.1207/s15324818ame0602_1 · DOI · Paper page
  57. Wang, E. L., Matsumura, L. C., Correnti, R., Litman, D., Zhang, H., Howe, E., Magooda, A., & Quintana, R. (2020). eRevis(ing): Students' revision of text evidence use in an automated writing evaluation system. Assessing Writing, 44, Article 100449. https://doi.org/10.1016/j.asw.2020.100449 · DOI
  58. Weber-Fève, S. (2009). A writing design: Using abstracts in the writing process. The French Review, 82(6), 1255-1267. http://www.jstor.org/stable/25613826.
  59. Wiboolyasarin, W., Wiboolyasarin, K., Suwanwihok, K., Jinowat, N., & Muenjanchoey, R. (2024). Synergizing collaborative writing and AI feedback: An investigation into enhancing L2 writing proficiency in wiki-based environments. Computers and Education: Artificial Intelligence, 6, Article 100228. https://doi.org/10.1016/j. caeai.2024.100228 · DOI · Paper page
  60. Wischgoll, A. (2017). Improving undergraduates' and postgraduates' academic writing skills with strategy training and feedback. Frontiers in education, 2. https://doi.org/ 10.3389/feduc.2017.00033 · DOI
  61. Wong, Z. Y., Liem, G. A. D., Chan, M., & Datu, J. A. D. (2024). Student engagement and its association with academic achievement and subjective well-being: A systematic review and meta-analysis. Journal of Educational Psychology, 116(1), 48-75. https:// doi.org/10.1037/edu0000833 · DOI
  62. Zhang, Z. V., & Hyland, K. (2022). Fostering student engagement with feedback: An integrated approach. Assessing Writing, 51, Article 100586. https://doi.org/10.1016/ j.asw.2021.100586 · DOI
  63. Zheng, R. (2025). AWE and LLMs in L2 writing feedback: An exploration of EFL learners' experiences. Academic Journal of Humanities & Social Sciences, 8(7), 60-69. 10.25236/AJHSS.2025.080708. · DOI

Cited by

Sorted by citations

Authors
Glüsing, R., Fleckenstein, J., Schmidt, F. T. C., & Möller, J.
Affiliations
University of Hildesheim (DE)
Corresponding author
Glüsing, Robert
ORCID
Venue
Contemporary Educational Psychology
Publisher
Elsevier BV
Published
2026-06
Volume
85
Pages
102463
ISSN
0361-476X (ISSN-L)
DOI
10.1016/j.cedpsych.2026.102463
OpenAlex
W4414060277
Cited by
2
References
22
Citations by year
2026: 2
FWCI
9.09
Type
journal article
Language
en
Open access
Licence
Locations
  • Contemporary Educational Psychology (publishedVersion, cc-by)
  • SSRN Electronic Journal (acceptedVersion)
Topics
  • Physical Sciences › Computer Science › Information Systems › Artificial Intelligence in Education
  • Social Sciences › Education › Student Assessment and Feedback
  • Social Sciences › Arts and Humanities › Language and Linguistics › EFL/ESL Teaching and Learning
Keywords
large language models, LLM-generated feedback, academic writing, writing revision, GPT-4, behavioral engagement, automated feedback, higher education, student writing, abstract writing

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review