Generative AI, visual scaffolds, and metacognition in EFL writing
Open accessThe DOI matches its Crossref record; 50 of the 51 references it cites were found in a registry or an index; 16 of the 57 sources its claims cite were checked for support.
Abstract
Fostering metacognitive awareness and reflective thinking remains a challenge in English as a Foreign Language (EFL) writing instruction. These cognitive processes are essential for enabling learners to monitor, evaluate, and engage in meaningful self-reflection throughout the writing cycle. Using Flower and Hayes’s cognitive process model and Jasper’s Experience-Reflection-Action framework, this study proposes a Generative AI-assisted Visually Guided Progressive (GAI-VGP) writing approach. This pedagogical design integrates step-by-step visual scaffolds with interactive AI feedback to foreground learner agency and promote reflective engagement. To evaluate this approach, a semester-long quasi-experimental study was conducted with 58 EFL university students. Data sources included pre- and post-writing tests, metacognitive awareness questionnaires, reflective thinking scales, AI chat logs, and reflection notes. The results showed that the GAI-VGP group significantly outperformed the conventional VGP group in writing performance, planning and evaluation strategies, and reflective awareness. Qualitative analyses indicated increased engagement in understanding and reflectively evaluating writing strategies, although critical reflection remained limited across both groups. Although AI significantly enhanced planning and evaluation, monitoring showed no significant gains, with prompt-log evidence revealing a displacement pattern in which learners outsourced real-time regulation to AI validation.
Overview
CoChat Summary
TL;DR
This quasi-experimental study tested a Generative AI-assisted Visually Guided Progressive (GAI-VGP) writing approach with 58 EFL university students over a semester. It found significant gains in writing performance, planning/evaluation metacognition, and reflective understanding/reflection, but no significant monitoring or critical reflection gains, with prompt logs suggesting monitoring was outsourced to AI validation.
Key findings
- Writing performance: the GAI-VGP group had a higher adjusted post-test mean (67.94) than the cVGP group (60.44), F = 4.46, p < .05, η² = .08.
- Metacognitive awareness: overall adjusted post-test mean was 3.97 for GAI-VGP vs. 3.64 for cVGP, F(1, 55) = 10.47, p < .05, η² = .16. Planning was significant (Adj.M = 3.97 vs. 3.59, F = 5.53, η² = .09, p < .05) and evaluation was significant (Adj.M = 3.98 vs. 3.01, F = 20.94, η² = .28, p < .001), but monitoring was not (F = 3.67, p > .05).
- Prompt logs showed planning prompts were most frequent (35), followed by evaluation (26) and monitoring (14); monitoring prompts often took the form of validation-seeking queries, which the authors interpret as a “monitoring displacement” pattern in which learners outsourced real-time regulation to AI validation.
- Reflective thinking: understanding was significant (Adj.M = 3.93 vs. 3.31, F = 4.49, p < .05, η² = .08) and reflection was significant (Adj.M = 3.93 vs. 3.16, F = 9.96, p < .01, η² = .15); habitual action and critical reflection were not significant (F = 1.54 and F = 2.20, respectively, p > .05). Reflection-note counts were 14 vs. 5 for understanding, 15 vs. 10 for reflection, and 3 vs. 1 for critical reflection.
Methods
- Quasi-experimental convergent parallel mixed-methods design with 58 second-year English majors (45 females, 13 males; aged 19–21) in an East Asian EFL context; two intact classes were assigned as GAI-VGP (n = 30) and cVGP (n = 28) over an 18-week semester.
- Data sources included pre/post argumentative writing tests scored with a TOEFL-based analytic rubric (100 points; Cohen’s κ = .75), an 18-item Metacognitive Writing Strategies Questionnaire (Cronbach’s α = .96 pre, .89 post), a Reflective Thinking Questionnaire, AI chat logs, and reflection notes from 20 volunteers.
- Quantitative analysis used one-way ANCOVA controlling for pre-test scores, with partial η² and Bonferroni adjustments; qualitative data were analyzed through directed content analysis/thematic coding in NVivo 14, with prompt coding κ = .74, reflection-note coding κ = .66, and qualitative coding κ = .82.
Limitations
- Stated: the EFL context is often teacher-centered, which may predispose students to accept AI suggestions without critical evaluation; reflection-note participation was voluntary, introducing self-selection bias; and “proxy dilution” occurred when a minority of students dominated AI interactions, potentially benefiting high-agency learners and leading to “surface consultation.”
- Evident from the text: the sample was small (58) and used intact classes, limiting generalizability; recorded AI prompts were relatively limited and used selectively, so gains cannot be attributed solely to AI use; critical reflection remained limited across both groups, and the monitoring displacement pattern suggests the design may not have scaffolded real-time self-regulation during drafting.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Figures
Verification
Claims the paper makes
13 Valid · 1 Not valid · 21 Not checked · 88 Own results and interpretations
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“Research suggests that AI-mediated scaffolding can externalize metacognition by prompting learners to justify thesis statements, reflect on evidence quality, and evaluate rhetorical effectiveness (Kim et al., 2025).”
§ Generative AI, visual scaffolds, and metacognition in EFL writing > Generative AI as a Scaffolding Mediator for VGP Writing
What the sources say
- Partly supportedcites Kim et al., 2025 matched to “Exploring students' perspectives on generative AI-assisted academic writing” (2025) sourceExperimental — accuracy not yet measured
The Conclusion proposes that GenAI could nudge students toward critical thinking, and the Findings and discussion section (4.2) reports that students perceived GenAI feedback as helping them reflect on and revise their writing. Our copy does not describe prompts asking learners to justify thesis statements, assess evidence quality, or evaluate rhetorical effectiveness specifically. No page numbers are available in our copy. (LLM, conf=0.85)
hedgedstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Partly supported
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“Despite strong empirical evidence that explicit metacognitive instruction improves writing performance (Teng et al., 2022;Wang et al., 2023), novice EFL learners often struggle to apply these strategies independently.”
§ Generative AI, visual scaffolds, and metacognition in EFL writing > Metacognitive Regulation and Reflective Agency in L2 Writing
What the sources say
- Partly supportedcites Teng et al., 2022 matched to “Metacognitive writing strategies, critical thinking skills, and academic writing performance: A structural equation modeling approach” (2022) sourceExperimental — accuracy not yet measured
The 'Metacognition in EFL writing' section reports that a metacognitive-training intervention improved writing accuracy, but not complexity or fluency. The paper's own academic writing course did not teach metacognitive strategies, so its findings do not establish that explicit instruction improves writing performance generally. (LLM, conf=0.88)
- Partly supportedcites Wang et al., 2023 matched to “Developing the Metacognitive Awareness of Grit Scale for a better understanding of learners of English as a foreign language” (2023) sourceExperimental — accuracy not yet measured
The Introduction (page not provided) says metacognitive strategies are associated with improved EFL academic performance. The Literature review discusses metacognitive strategies in writing and an instructional intervention that improved monitoring accuracy, but it does not report that explicit metacognitive instruction improves writing performance. (LLM, conf=0.78)
boostedstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Partly supported
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“Without external scaffolding, they frequently focus on surface-level language errors rather than deeper conceptual weaknesses (Muamaroh et al., 2020).”
§ Generative AI, visual scaffolds, and metacognition in EFL writing > Metacognitive Regulation and Reflective Agency in L2 Writing
What the sources say
- Partly supportedcites Muamaroh et al., 2020 matched to “The Process and Problems of EFL Learners in English Writing” (2020) sourceExperimental — accuracy not yet measured
The Factors Inhibiting and Improving the Students’ Writing Skills (page not provided) reports students’ concern about grammar and vocabulary, while Content (page not provided) documents difficulty developing ideas in depth. The paper does not establish that learners focused on language errors rather than conceptual weaknesses, or that a lack of external scaffolding caused that focus. (LLM, conf=0.77)
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Partly supported
Every source the paper cites for this claim was checked and supports it.
In the paper
“The process-oriented interpretation of these findings aligns with, yet also extends, classroom-based qualitative evidence reported by Aydın Yıldız (2025), who showed that GAI-supported multimodal writing pedagogy can foster more strategic and reflective AI use across writing stages.”
§ Generative AI, visual scaffolds, and metacognition in EFL writing > Discussion and Conclusions
What the sources say
- Supportedcites Aydın Yıldız (2025) matched to “Sustainability Education in L2 Writing: AI-Based Multimodal Awareness and Engagement” (2025) source
The Post-Intervention Findings (Section 3.2) report a shift toward strategic, reflective AI use, including planning, organizing, and revising rather than only grammar-checking. The Conclusions (Section 5) identify this as an outcome of the AI-supported multimodal writing tasks, while noting that some students still relied heavily on automated suggestions. (LLM, conf=0.93)
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“Consequently, EFL students often struggle to move beyond surface-level mechanics and engage in the deep planning, monitoring, and evaluation essential for effective composition (Kim et al., 2025;Taye & Mengesha, 2024).”
§ Generative AI, visual scaffolds, and metacognition in EFL writing > Introduction
What the sources say
- Partly supportedcites Kim et al., 2025 matched to “Exploring students' perspectives on generative AI-assisted academic writing” (2025) sourceExperimental — accuracy not yet measured
The Introduction says academic writing requires goal setting and other cognitive skills and can be especially challenging for ESL students. Findings §4.3 reports difficulty critically evaluating AI-generated content and developing arguments, but the paper does not establish that EFL students generally remain focused on surface-level mechanics or struggle with planning and monitoring; §4.2 instead describes students valuing GenAI's planning support. No page numbers are available in our copy. (LLM, conf=0.84)
- Partly supportedcites Taye & Mengesha, 2024 matched to “Identifying and analyzing common English writing challenges among regular undergraduate students” (2024) sourceExperimental — accuracy not yet measured
The Results (Analysis of students’ questionnaire; page unavailable) report that 22% of students did not follow writing stages such as prewriting, drafting, revising, and editing, supporting a narrower claim about difficulty with the writing process. But 58.82% reported following those stages, and the paper does not establish that EFL students often struggle specifically with deep monitoring and evaluation or with moving beyond surface-level mechanics. (LLM, conf=0.85)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Partly supported
52 more claims · 2 with their cited sources checked
Create a free account to see all 123 claims and to verify the 4 references still open.
- It may state two separate claims as one. 39
- Probably not a substantial claim. 34
- The quote only partly states this claim. 5
Statistics recomputed
Every reported test result, recomputed from its test statistic and degrees of freedom. A result shows as not adding up only once an independent check has confirmed it.
- ConsistentF(1, 55) = 10.47reported p < .05 · recomputed p = .002
“F(1, 55) = 10.47,p < .05”
- ConsistentF(1, 55) = 4.49reported p < .05 · recomputed p = .039
“F(1, 55) = 4.49, p < .05”
- ConsistentF(1, 55) = 9.96reported p < .01 · recomputed p = .003
“F(1, 55) = 9.96, p < .01”
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
Each contradiction between two passages listed here was confirmed by an independent check.
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ Generative AI, visual scaffolds, and metacognition in EFL writing > Reliability of Instruments Used to Measure Writing Learning Outcomes
“Reflection 10 5-point Likert scale 5 Cronbach's α Fu & Hali (2025); Kember et al. (2000)”
§ Generative AI, visual scaffolds, and metacognition in EFL writing > Reliability of Instruments Used to Measure Writing Learning Outcomes
“Understanding 4 5-point Likert scale 5 .86.99Habitual reflection 4 5-point Likert scale 5 .92.95Reflection 4 5-point Likert scale 5 .90.96Critical reflection 4 5-point Likert scale 5 .88.97”
Citations
Cites
A cognitive process theory of writing
Flower, L., & Hayes, J. R. (1981) · College Composition & Communication · doi:10.58680/ccc198115885 · Cited by 1,025 · cited 3×
“Writing is a complex problem-solving activity that demands the orchestration of multiple cognitive processes (Flower & Hayes, 1981; Xue, 2024; Yao & Fan, 2025).”
Development of a Questionnaire to Measure the Level of Reflective Thinking
Kember, D., Leung, D. Y. P., Jones, A., Loke, A. Y., McKay, J., Sinclair, K., Tse, H., Webb, C., Yuet Wong, F. K., Wong, M., et al. (2000) · Assessment & Evaluation in Higher Education · doi:10.1080/713611442 · Cited by 812 · cited 1×
“The Reflective Thinking Questionnaire, modified from the RTQ (Fu & Hali, 2025; Kember et al., 2000), assessed learners’ reflective thought processes across four validated dimensions: (1) habitual action, referring to automatic, routine writing behaviors without conscious deliberation; (2)…”
Dergaa, I., Chamari, K., Zmijewski, P., & Ben Saad, H. (2023) · Biology of Sport · doi:10.5114/biolsport.2023.125623 · Cited by 796 · cited 1×
“Unlike static writing tools, GAI systems function as interactive partners capable of simulating rhetorical genres and providing immediate adaptive feedback (Dergaa et al., 2023; Liu et al., 2024; Zhang et al., 2025c).”
The Psychology of Written Composition
(2013) · doi:10.4324/9780203812310 · Cited by 505
Exploring students' perspectives on generative AI-assisted academic writing
Kim, J., Yu, S., Detrick, R., & Li, N. (2025) · Education and Information Technologies · doi:10.1007/s10639-024-12878-7 · Cited by 438 · cited 3×
“Consequently, EFL students often struggle to move beyond surface-level mechanics and engage in the deep planning, monitoring, and evaluation essential for effective composition (Kim et al., 2025; Taye & Mengesha, 2024).”
A context-aware ubiquitous learning environment for conducting complex science experiments
Hwang, G.-J., Yang, T.-C., Tsai, C.-C., & Yang, S. J. H. (2009) · Computers & Education · doi:10.1016/j.compedu.2009.02.016 · Cited by 432 · cited 1×
“Additional details are provided in Appendix C. Reflection notes were collected using a semi-structured protocol with seven open-ended questions designed to explore students’ perspectives on the two VGP approaches, based on Hwang et al.’s (2009) interview outline.”
Barrett, A., & Pack, A. (2023) · International Journal of Educational Technology in Higher Education · doi:10.1186/s41239-023-00427-0 · Cited by 391 · cited 1×
“However, unguided AI use may lead to superficial engagement or overreliance on generated output (Barrett & Pack, 2023).”
Ghanizadeh, A. (2017) · Higher Education · doi:10.1007/s10734-016-0031-y · Cited by 334 · cited 1×
“Reflective thinking plays a key role in bridging this gap, as it operates in close interplay with critical thinking and self-monitoring to support academic achievement in higher education (Ghanizadeh, 2017).”
Liu, M., Zhang, L. J., & Biebricher, C. (2024) · Computers & Education · doi:10.1016/j.compedu.2023.104977 · Cited by 282 · cited 1×
“Unlike static writing tools, GAI systems function as interactive partners capable of simulating rhetorical genres and providing immediate adaptive feedback (Dergaa et al., 2023; Liu et al., 2024; Zhang et al., 2025c).”
Lee, H.-Y., Chen, P.-H., Wang, W.-S., Huang, Y.-M., & Wu, T.-T. (2024) · International Journal of Educational Technology in Higher Education · doi:10.1186/s41239-024-00447-4 · Cited by 273
Research works printed but not linked: 2
- Jasper, M. (2013). Beginning reflective practice (2nd ed.). Nelson Thornes.
- Scardamalia, M., & Bereiter, C. (1987). Knowledge telling and knowledge transforming in written composition. In S. Rosenberg (Ed.), Advances in applied psycholinguistics: Reading, writing, and language learning (Vol. 2, pp. 142-175). Cambridge University Press.
Cited by
Shang, X. (2026) · Frontiers in Psychology · doi:10.3389/fpsyg.2026.1974520 · Cited by 1
Metadata
- Authors
- Chen, M.-R. A.
- Affiliations
- Soochow University (TW)
- Corresponding author
- Chen, Mei-Rong Alice
- ORCID
- Chen, Mei-Rong Alice
- Venue
- Language Learning & Technology
- Published
- 2026-06-29
- Volume
- 30
- Issue
- 1
- Pages
- 1-26
- ISSN
- 1094-3501 (ISSN-L)
- OpenAlex
- W7166587502
- Cited by
- 1
- References
- 49
- Citations by year
- 2026: 1
- FWCI
- 20.73
- Type
- journal article
- Language
- en
- Open access
- diamond
- doi.org
- Licence
- cc-by-nc-nd
- Locations
- Language learning & technology (publishedVersion, cc-by-nc-nd)
- ScholarSpace (University of Hawaii at Manoa) (submittedVersion, cc-by-nc-nd)
- Funders
- Ministry of Education, India
- National Science and Technology Council
- Topics
- Social Sciences › Arts and Humanities › Language and Linguistics › EFL/ESL Teaching and Learning
- Social Sciences › Education › English Language Learning and Teaching
- Social Sciences › Psychology › Developmental and Educational Psychology › Innovative Teaching and Learning Methods
- Keywords
- generative AI, metacognition, EFL writing, reflective thinking, AI-assisted writing, learner agency, displacement effects, AI-generated feedback, critical reflection
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
The claim verdicts this report counts are experimental: their accuracy is not yet measured.
A language model drew the claims from the paper on 2026-10-10, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-10-10 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Cited sources are checked with a language model’s help (gpt-6-sol).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.



