Personalized language learning with an LLM chatbot: effects of immediate vs. delayed corrective feedback
Open accessThe DOI matches its Crossref record; 115 of the 116 references it cites were found in a registry or an index; none of the 97 sources its claims cite is checked for support yet.
Abstract
The emergence of Large Language Models (LLMs) has opened new possibilities for language learning through conversational interaction with chatbots. Yet, little empirical evidence exists on how students experience such interactions and how corrective feedback should be provided. Research suggests that immediate corrective feedback is generally more effective than delayed feedback. Nevertheless, learners' perception of this effectiveness and their preferences for feedback timing, particularly in the domain of Computer-Assisted Language Learning (CALL), remain underexplored. This study investigates the feasibility of providing immediate feedback and examines the impact of feedback timing on user experience and grammar learning gains in English. An in-the-wild experiment was conducted with 66 L2 English learners, who integrated chatbot sessions into their English course as an extracurricular activity over one semester. Participants were randomly assigned to two groups receiving feedback either during or after the conversation. Findings reveal no significant difference in learning gains, but immediate feedback enhanced user experience, leading to overall positive perceptions of the chatbot. Additionally, we explore users' perceptions of the chatbot's social role and personality, offering a roadmap for future enhancements. These results provide valuable insights into the potential of LLMs and chatbots for language learning.
Overview
CoChat Summary
TL;DR
In an in-the-wild experiment with 66 L2 English learners (54 analyzed), an LLM-powered chatbot provided either immediate or delayed corrective feedback over 12 sessions. Immediate feedback improved perceived effectiveness and user experience, but the study found no significant difference in grammar learning gains between feedback timing conditions.
Key findings
- No significant difference in grammar learning outcomes between immediate corrective feedback (ICF) and delayed corrective feedback (DCF): language assessment scores were ICF x = 3.14 vs. DCF x = 2.18, t(47.99) = 1.35, p = 0.092, Cohen’s d = 0.37; overall participants corrected x = 2.62 of 15 mistakes. A GLMM found no significant relationship between error counts and condition, session number, or error type.
- Immediate feedback improved perceived effectiveness: post-questionnaire ICF x = 3.65 vs. DCF x = 3.08, t(47.96) = 2.01, p = 0.0495, Cohen’s d = 0.56; ICF showed a significant paired increase (x = 1.48, t(22) = 2.63, p = 0.015, d = 0.68), while DCF did not (x = 0.43, t(29) = 1.15, p = 0.259, d = 0.23). Pre-test effectiveness did not differ between groups (t(44.28) = 0.25, p = 0.8, d = 0.07).
- Overall pre-post effectiveness ratings improved: sign test found 22 of 31 non-zero changes were positive, p = 0.0294; estimated probability of positive change was 70.97%, 95% CI [51.96%, 85.78%]. 72.22% (39/54) reported learning something in English, χ²(1) = 9.79, p = 0.0017, 95% CI [58.14%, 83.14%].
- The chatbot was perceived anthropomorphically: 56% considered it male, 31% assigned no gender; mean assigned age was 30.94 (s = 12.59); social role choices were 40% “a random person,” 27.3% “friend,” 10.9% “teacher,” and 21.8% “a mobile application.” Experience was moderately positive (x = 3.69, s = 0.72) and politeness high (x = 4.52, s = 0.75); determiners were the most common error type.
Methods
- In-the-wild between-subject experiment: 66 bachelor students enrolled, 54 completed and were analyzed (24 ICF, 30 DCF); ages 18–23 (x = 20.59, s = 0.98), 43 males and 11 females, mostly native Czech speakers, with self-assessed English levels from A2 to C2. Participants completed 12 chatbot sessions over 4 weeks (3 per week), each until 1,000 characters, outside school.
- The chatbot was a web application using OpenAI GPT-3 text-davinci-003 API, Python/Flask/JavaScript, and Gramformer for grammatical error detection/correction. Feedback types were explicit recast, negative evidence, and combined feedback; ICF corrected the first mistake immediately during conversation, while DCF accumulated errors and showed a summary report after the conversation.
- Measures included pre/post questionnaires on effectiveness, ranking, chatbot personality/social role, experience, and politeness; a language assessment of 25 sentences (about 15 containing the participant’s own errors) scored 1 point per corrected mistake; and logged error counts analyzed with a GLMM. Analyses used R 4.3.2 with sign test, independent/paired t-tests, one-sample proportions test, and p < 0.05.
Limitations
- Paper-stated: no comprehensive pre/post language test; learning was measured via participants’ own mistakes, self-reported proficiency may bias results; varying session topics and external factors such as tiredness or mood may affect error counts or attention; no control group; large dropout (12 of 66); small sample limits generalization; one-month duration may be insufficient; asking participants to correct their own errors may have minimal priming; time was not controlled, though the paper argues this was an alternative design choice rather than necessarily a limitation.
- Paper-stated future needs: more controlled designs, larger samples, a control group, within-subject designs, focus on specific structures such as determiners, diverse learner backgrounds, long-term engagement, and more advanced LLMs.
- Evident from the text: attrition produced imbalanced groups (24 ICF vs. 30 DCF); the paper notes N = 54 may have been underpowered to detect subtle grammar acquisition differences; open-domain conversation lacked structured output practice, which the paper suggests may explain comparable gains despite differing perceived salience.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Figures
Verification
Claims the paper makes
0 Valid · 7 Not valid · 73 Not checked · 110 Own results and interpretations
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“while the DCF group showed no significant change (x = 0.43), (t(29) = 1.15, p = 0.259, Cohen's d = 0.23) in effectiveness scores.”
§ Personalized language learning with an LLM chatbot: effects of immediate vs. delayed corrective feedback > 5.1.3 Between-condition comparisons
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Using an independent samples t-test, we compared the responses between the two groups from the pre-questionnaire, which showed no significant difference (t(44.28)= 0.25, p = 0.8, Cohen's d = 0.07).”
§ Personalized language learning with an LLM chatbot: effects of immediate vs. delayed corrective feedback > 5.1.3 Between-condition comparisons
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The exact binomial test yielded p value = 0.0294”
§ Personalized language learning with an LLM chatbot: effects of immediate vs. delayed corrective feedback > 5.1.2 Pre-post change
plainn = 54 ·- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper states — it cites no source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Findings reveal no significant difference in learning gains, but immediate feedback enhanced user experience, leading to overall positive perceptions of the chatbot.”
§ Personalized language learning with an LLM chatbot: effects of immediate vs. delayed corrective feedback > Abstract
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Out of the 31 non-zero changes, 22 displayed an increase in effectiveness ratings.”
§ Personalized language learning with an LLM chatbot: effects of immediate vs. delayed corrective feedback > 5.1.2 Pre-post change
plainn = 54 ·- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
50 more claims
Create a free account to see all 190 claims and to verify the 82 references still open.
- Probably not a substantial claim. 111
- It may state two separate claims as one. 62
Statistics that don’t add up
Each reported p-value recomputed from its test statistic and degrees of freedom.
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
“t(47.99)= 1.35, p = 0.092”
§ Personalized language learning with an LLM chatbot: effects of immediate vs. delayed corrective feedback > 5.3.2 Objective measure 5.3.2.1 Language assessment
Reported p = .092 · recomputed p = .183
Statistics recomputed
Every reported test result, recomputed from its test statistic and degrees of freedom. A result shows as not adding up only once an independent check has confirmed it.
- Doesn’t add upt(47.99) = 1.35reported p = .092 · recomputed p = .183
“t(47.99)= 1.35, p = .092”
- Consistentt(44.28) = 0.25reported p = .8 · recomputed p = .804
“t(44.28)= 0.25, p = .8”
- Consistentt(47.96) = 2.01reported p = .0495 · recomputed p = .050
“t(47.96)= 2.01, p = .0495”
- Consistentt(22) = 2.63reported p = .015 · recomputed p = .015
“t(22) = 2.63, p = .015”
- Consistentt(29) = 1.15reported p = .259 · recomputed p = .260
“t(29) = 1.15, p = .259”
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
Each contradiction between two passages listed here was confirmed by an independent check.
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ Personalized language learning with an LLM chatbot: effects of immediate vs. delayed corrective feedback > Abstract
“An in-thewild experiment was conducted with 66 L2 English learners, who integrated chatbot sessions into their English course as an extracurricular activity over one semester.”
§ Personalized language learning with an LLM chatbot: effects of immediate vs. delayed corrective feedback > 1.1 Research questions
“In this study, we present an in-the-wild experiment where 66 students interacted with an LLM-powered chatbot over 12 sessions distributed over a one-month period.”
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ Personalized language learning with an LLM chatbot: effects of immediate vs. delayed corrective feedback > 2.4.2 Recent developments in generative AI for language learning
“Since our data collection in early 2022, the release of ChatGPT has accelerated research on generative AI in language education.”
§ Personalized language learning with an LLM chatbot: effects of immediate vs. delayed corrective feedback > 3.2.2 Grammatical error detection and correction
“At the time the study was conducted in late 2022, this was the most advanced model available to us.”
Citations
Cites
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2018) · arXiv · doi:10.48550/arxiv.1810.04805 · Cited by 73,173
Distributed Representations of Words and Phrases and their Compositionality
Mikolov, T., Sutskever, I., Chen, K., Corrado, G., & Dean, J. (2013) · arXiv · doi:10.48550/arxiv.1310.4546 · Cited by 24,533
Language Models are Few-Shot Learners
Brown, T. B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., et al. (2020) · arXiv · doi:10.48550/arxiv.2005.14165 · Cited by 23,372
RoBERTa: A Robustly Optimized BERT Pretraining Approach
Liu, Y., Ott, M., Goyal, N., Du, J., Joshi, M., Chen, D., Levy, O., Lewis, M., Zettlemoyer, L., & Stoyanov, V. (2019) · arXiv · doi:10.48550/arxiv.1907.11692 · Cited by 16,515
Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer
Raffel, C., Shazeer, N., Roberts, A., Lee, K., Narang, S., Matena, M., Zhou, Y., Li, W., & Liu, P. J. (2019) · arXiv · doi:10.48550/arxiv.1910.10683 · Cited by 10,929
The episodic buffer: a new component of working memory?
Baddeley, A. (2000) · Trends in Cognitive Sciences · doi:10.1016/s1364-6613(00)01538-2 · Cited by 7,359 · cited 1×
“The serial position effect suggests that words or phrases at the beginning (primacy effect) and end (recency effect) of a sequence are better recalled, impacting sentence structure and retention in language processing (Atkinson and Shiffrin, 1968; Baddeley, 2000; Baddeley et al., 2009).”
Human Memory: A Proposed System and its Control Processes
Atkinson, R. C., & Shiffrin, R. M. (1968) · Psychology of Learning and Motivation · doi:10.1016/s0079-7421(08)60422-3 · Cited by 7,216
Foreign Language Classroom Anxiety
HORWITZ, E. K., HORWITZ, M. B., & COPE, J. (1986) · The Modern Language Journal · doi:10.1111/j.1540-4781.1986.tb05256.x · Cited by 5,697
The Role of Consciousness in Second Language Learning1
SCHMIDT, R. W. (1990) · Applied Linguistics · doi:10.1093/applin/11.2.129 · Cited by 5,435 · cited 7×
“On top of all, the “Noticing” hypothesis also supports the value of interaction by positing that conscious awareness of linguistic features is a prerequisite for their acquisition (Schmidt, 1990, 2001).”
InfluentialELIZA—a computer program for the study of natural language communication between man and machine
Weizenbaum, J. (1966) · Communications of the ACM · doi:10.1145/365153.365168 · Cited by 4,452 · cited 1×
“One of the earliest chatbots, ELIZA, was introduced as a virtual psychoanalyst (Weizenbaum, 1966).”
Research works printed but not linked: 5
- Haines, 1995, For and against: pairwork, Modern Engl. Teach, 4, 55
- AI, expert or peer? – examining the impact of perceived feedback source on pre-service teachers feedback Perception and uptake Jacobsen L. J. Mertens U. Jansen T. Weber K. E. arXiv 2025
- Kim, 2019, Future English learning: chatbots and artificial intelligence, Multimedia-Assis. Lang. Learn, 22, 32, 10.15702/mall.2019.22.3.32 · doi:10.15702/mall.2019.22.3.32
- 2023, R: A Language and Environment for Statistical Computing
- Radford, 2019, Language models are unsupervised multitask learners, OpenAI blog, 1, 9
Cited by
Pitychoutis, K. M., & Spathopoulou, F. (2026) · Feedback Research in Second Language · doi:10.32038/frsl.2026.04.02 · Cited by 8
Kafabih, A., Margana, M., Herda, R. K., & Majid, S. (2026) · Research and Innovation in Applied Linguistics-Electronic Journal · doi:10.31963/rial.v4i2.6456 · Cited by 1
Early Language Development in the Digital Age (e-LADDA)Preprint
Kamelabad, A. M., Tüchler, A. F., Diprossimo, L., de Eccher, M., Sivridag, F., Zavala, K., Kalchhauser, T., Aldemir, H., Zantonello, G., Raggioli, L., et al. (2023) · doi:10.17605/osf.io/a4dhq
Wu, J. (2026) · Transactions on Social Science Education and Humanities Research · doi:10.62051/bdwbk164
Shijie, H., Zhao, Y., Yuhan, Z., & Jiani, L. (2026) · American Journal of Artificial Intelligence · doi:10.11648/j.ajai.20261002.14
Metadata
- Authors
- M. Kamelabad, A., Turano, B., Lundin, M., & Skantze, G.
- Affiliations
- Division of Speech, Music, and Hearing, KTH Royal Institute of Technology
- University of Trento (IT)
- KTH Royal Institute of Technology (SE)
- ORCID
- Skantze, Gabriel
- Venue
- Frontiers in Education
- Publisher
- Frontiers Media SA
- Published
- 2026-02-24
- Volume
- 11
- ISSN
- 2504-284X (ISSN-L)
- OpenAlex
- W7131254148
- Cited by
- 13
- References
- 113
- Citations by year
- 2026: 12
- FWCI
- 82.84
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- Locations
- Frontiers in Education (publishedVersion, cc-by)
- DiVA (Digitala Vetenskapliga Arkivet) (submittedVersion, other-oa)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- OSF Preprints (OSF Preprints) (submittedVersion)
- Open Science Framework
- Funders
- H2020 Marie Skłodowska-Curie Actions (857897)
- HORIZON EUROPE Marie Sklodowska-Curie Actions (857897)
- Topics
- Social Sciences › Arts and Humanities › Language and Linguistics › EFL/ESL Teaching and Learning
- Physical Sciences › Computer Science › Artificial Intelligence › Intelligent Tutoring Systems and Adaptive Learning
- Social Sciences › Education › English Language Learning and Teaching
- Keywords
- large language models, chatbots, corrective feedback, feedback timing, grammar learning, English as a second language, computer-assisted language learning, user experience, learner perceptions, chatbot personality
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
A language model drew the claims from the paper on 2026-11-20, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-11-20 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Personalized language learning with an LLM chatbot: effects of immediate vs. delayed corrective feedback. https://app.cochat.ai/papers/10.3389/feduc.2026.1703664
- The paper
- M. Kamelabad, A., Turano, B., Lundin, M., & Skantze, G. (2026). Personalized language learning with an LLM chatbot: effects of immediate vs. delayed corrective feedback. Frontiers in Education. https://doi.org/10.3389/feduc.2026.1703664
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.




