CoChat
Sign Up
Cited by 90EFL/ESL Teaching and LearningEnglish Language Learning and TeachingSecond Language Acquisition and Learning

The impact of automated writing evaluation on second language writing skills of Chinese EFL learners: a randomized controlled trial

Open access
Wei, Ping, Wang, Xiaosai, Dong, Hui·2023·Frontiers in Psychology
DOI 10.3389/fpsyg.2023.1249991PMC PMC10570814
Open this paper in CoChat — free accountOpen PDF

Abstract

Introduction: In the context of the burgeoning field of second language (L2) education, where proficient writing plays an integral role in effective language acquisition and communication, the ever-increasing technology development has influenced the trajectory of L2 writing development. Methods: To address the need for enhanced writing skills among English as a Foreign Language (EFL) learners, this study investigates the efficacy of Automated Writing Evaluation (AWE) training. A randomized controlled trial employing repeated measures was conducted, involving a participant pool of 190 Chinese EFL students. The study comprehensively assessed the effects of AWE training, utilizing the Grammarly platform-an AI-driven program-on various dimensions of writing skills, encompassing task achievement, coherence and cohesion, lexicon, and grammatical accuracy. Control variables included writing self-efficacy and global English proficiency. Writing skills were evaluated through the administration of an International English Language Testing System (IELTS) writing sample test. Results: The results unequivocally demonstrate that the experimental group consistently exhibited superior performance across all facets of writing skills compared to the control group. Furthermore, the predictive influence of pre-test scores was pronounced in task achievement, coherence and cohesion, and lexicon, highlighting the pivotal role of learners' initial proficiency levels in shaping subsequent writing outcomes. Notably, the emergence of writing self-efficacy as a significant predictor of task achievement and coherence and cohesion underscores the role of learners' beliefs and confidence in shaping their writing abilities. Discussion: These findings conclusively suggest that Artificial Intelligence-based instructional programs, specifically AWE, hold the potential to effectively enhance second language writing skills, especially among learners with lower proficiency levels. This study carries crucial implications for EFL educators and researchers, advocating for the seamless integration of AWE into pedagogical strategies to foster a marked improvement in writing competence.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.3389/fpsyg.2023.1249991

Cites

93 of 93 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Some studies found stronger effects among lower-proficiency learners.

    “Some studies found stronger effects among lower-proficiency learners (Wei et al., 2023; Shen et al., 2023)”

    From The Use of Artificial Intelligence Tools in English Academic Writing among University Students: A Scoping Review (2025) · cites this paper as Wei et al., 2023

    The Results report a stronger effect of AWE-based instruction on lexicon for learners with lower pre-test scores, supporting the claim for that specific outcome. The Discussion says the effect did not vary significantly by pre-test score for task achievement or coherence and cohesion, so the finding should not be generalized to writing skills overall or to global English proficiency. (LLM, conf=0.88)

    Experimental — accuracy not yet measured

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Flagged first, then by citations

3 of the 96 references couldn’t be matched to a known paper.

Research works printed but not linked: 2

  1. Allan D. (2004). Oxford placement test. Oxford: Oxford University Press.
  2. Enders C. K. (2010). Applied missing data analysis. Methodology in the social sciences. New York, NY: Guilford Press.

Cited by

Sorted by citations

Related

Authors
Wei, P., Wang, X., & Dong, H.
Affiliations
Tangshan Normal University (CN)
Corresponding author
Wei, Ping
ORCID
Dong, Hui
Venue
Frontiers in Psychology
Publisher
Frontiers Media SA
Published
2023-09-29
Volume
14
Pages
1249991-1249991
ISSN
1664-1078 (ISSN-L)
DOI
10.3389/fpsyg.2023.1249991
PMC
PMC10570814
OpenAlex
W4387213334
Cited by
89
References
96
Citations by year
2026: 40 · 2025: 33 · 2024: 15 · 2023: 1
FWCI
99.83
Type
journal article
Language
en
Open access
Licence
Locations
  • Frontiers in Psychology (publishedVersion, cc-by)
  • PubMed Central (publishedVersion, cc-by)
  • PubMed (publishedVersion)
  • DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
Topics
  • Social Sciences › Arts and Humanities › Language and Linguistics › EFL/ESL Teaching and Learning
  • Physical Sciences › Computer Science › Information Systems › English Language Learning and Teaching
  • Social Sciences › Psychology › Developmental and Educational Psychology › Second Language Acquisition and Learning
Keywords
automated writing evaluation, second language writing, EFL learners, Chinese learners of English, Grammarly, writing self-efficacy, task completion, cohesion and coherence, vocabulary, grammatical accuracy, IELTS writing, writing skills

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review