CoChat
Sign Up
Cited by 298Artificial Intelligence in EducationIntelligent Tutoring Systems and Adaptive LearningAI in Service Interactions

A Meta-Analysis and Systematic Review of the Effect of Chatbot Technology Use in Sustainable Education

Open access
Deng, Xinjie, Yu, Zhonggen·2023·Sustainability
DOI 10.3390/su15042940
Open this paper in CoChat — free accountOpen PDF

Abstract

The development of artificial intelligence in recent years has promoted the use of chatbot technology in sustainable education. Many studies examined the effect of chatbots on learning outcomes. However, scant studies summarized the effectiveness of chatbots in education. The aim of the study is to investigate the effect of chatbot-assisted learning on various components and how different moderator variables influenced the effectiveness. This study, through a meta-analysis, reviewed 32 empirical studies with 2201 participants published between 2010 and 2022. The results showed that chatbot technology exerted a medium-to-high effect on overall learning outcomes regardless of moderator variables, i.e., intervention duration, chatbot roles, and learning content. In terms of learning components, chatbots could significantly improve explicit reasoning, learning achievement, knowledge retention, and learning interest despite negative findings in critical thinking, learning engagement, and motivation. Future research could expand chatbot research by including different learning components.

AI snapshot

Written from the paper’s full text.

TL;DR

This meta-analysis and systematic review examined 32 empirical studies with 2201 participants and found that chatbot technology had a medium-to-large positive overall effect on learning outcomes (d = 0.789). Effects were significant for explicit reasoning, learning achievement, knowledge retention, and learning interest, but not for critical thinking, learning engagement, or motivation.

Key findings

  • The overall effect size of chatbot use in education was d = 0.789 (95% CI [0.593, 0.985], p < 0.001), described as medium-to-large.
  • Chatbots significantly improved explicit reasoning (d = 1.190, p < 0.001), learning achievement (d = 1.033, p < 0.001), knowledge retention (d = 0.691, p = 0.022), and learning interest (d = 0.842, p = 0.041).
  • Chatbots did not significantly improve critical thinking (d = 0.690, p = 0.144), learning engagement (d = 0.147, p = 0.114), or learning motivation (d = 0.409, p = 0.114).
  • Moderator analyses found no statistically significant differences by intervention duration, chatbot roles, or learning content (all p > 0.05), though subgroup effect sizes varied.

Methods

  • The authors conducted a PRISMA-guided meta-analysis and systematic review, searching major databases including Web of Science, Wiley Online Library, Springer Link, Taylor & Francis Online, Elsevier ScienceDirect, and Google Scholar on 3 September 2022.
  • They included 32 empirical studies published between 2010 and 2022 with 2201 participants, calculating 76 effect sizes using Cohen’s d; studies needed experimental/control comparisons and sufficient statistics.
  • Analyses used Stata MP/14.0, with publication-bias checks (funnel plot, Begg’s test, Egger’s test), heterogeneity tests (I²), sensitivity analysis, and meta-regression for intervention duration, chatbot roles, and learning content.

Limitations

  • The paper states that only studies written in English were included, that only three moderator variables were examined, and that unpublished studies with insignificant results may still influence the findings despite no detected publication bias.
  • Evident from the text: several outcome domains had high heterogeneity (e.g., critical thinking I² = 92.8%, motivation I² = 93.2%, overall I² = 87.8%), and some categories had very few studies, limiting moderator conclusions.

Model-written summary. Check it against the paper before relying on it.

Integrity signals

Registry

Crossref · 10.3390/su15042940

Cites

63 of 63 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

0 of 35 cited sources checked · 0 supported · 0 flagged

  • Findingown data

    The meta-regression results revealed no statistically significant differences between effect sizes of different learning content categories.

    “The meta-regression results revealed no statistically significant differences between the abovementioned values.”
    Quote in the paperSubstantial claim
  • Findingown data

    No significant differences in learning engagement (d = 0.147, 95% CI [-0.068, 0.363], p = 0.114) were found between chatbot-assisted and traditional learning methods.

    “No significant differences in learning engagement (d = 0.147, 95% CI [-0.068, 0.363], p = 0.114) were found between chatbot-assisted and traditional learning methods.”
    Quote in the paperSubstantial claim
  • Findingown data

    There were no significant differences between effect sizes of the different intervention durations (p > 0.05).

    “However, there were no significant differences between effect sizes of the different intervention durations (p > 0.05).”
    Quote in the paperSubstantial claim
  • Findingown data

    There was a significant difference in explicit reasoning (d = 1.190, 95% CI [0.868, 1.512], p < 0.001) between the experimental and control groups.

    “Table 4 shows a significant difference in explicit reasoning (d = 1.190, 95% CI [0.868, 1.512], p < 0.001) between the experimental and control groups.”
    Quote in the paperSubstantial claim
  • Findingown data

    There was no significant difference in critical thinking (d = 0.690, 95% CI [-0.235, 1.615], p = 0.144) between chatbot-assisted and conventional learning methods.

    “The effect size did not show a significant difference in critical thinking (d = 0.690, 95% CI [-0.235, 1.615], p = 0.144) between the chatbot-assisted and conventional learning methods.”
    Quote in the paperSubstantial claim
  • May combine two claims. 23
  • Classified as probably not a substantial claim. 20

  • May combine two claims.

    Findingown data

    Begg's test showed a presence of publication bias in learning interest (z = 2.04, p = 0.042), which was different from the result of Egger's test (t = 15.30, p = 0.056).

    “However, Begg's test showed a presence of publication bias in learning interest (z = 2.04, p = 0.042), which was different from the result of Egger's test (t = 15.30,p = 0.056).”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data

    The study included 32 identified studies with a total sample size of 2201 and 76 effect sizes.

    “The researchers calculated 76 effect sizes with a total sample size of 2201 in 32 identified studies.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    None of the included studies could influence the pooled effect size when a specific study is omitted, confirming that the meta-analytical results are robust.

    “It indicates that none of the included studies could influence the pooled effect size when a specific study is omitted, confirming that the meta-analytical results are robust.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductcitation

    The researchers strictly followed the Preferred Reporting Items for Systematic reviews and Meta-Analyses (PRISMA) guidelines.

    “They strictly followed the Preferred Reporting Items for Systematic reviews and Meta-Analyses (PRISMA) guidelines [54].”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [54] Page, 2021, The PRISMA 2020 statement: An updated guideline for reporting systematic reviews, Br. Med. J. source

      Not checked in the free run.

  • May combine two claims.

    Interpretationown data

    The overall effect size was considered medium-to-large, indicating that the use of chatbot technology exerted a positive effect on learning outcomes.

    “The overall effect size in this study was thus considered as a mediumto-large level, indicating that the use of chatbot technology exerted a positive effect on learning outcomes.”
    Quote in the paperSubstantial claim

  • Classified as probably not a substantial claim.

    Premisecitation

    One display of explicit reasoning could associate with another one, which was the core of transactivity theory.

    “one display of explicit reasoning could associate with another one, which was the core of transactivity theory [70].”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [70] Pinkwart, N., and McLaren, B. (2012). Educational Technologies for Teaching Argumentation Skills, Bentham Science Publishers. Rosé, C.P. source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    Chatbots increased college students' interest in English vocabulary learning.

    “Chatbots also increased college students' interest in English vocabulary learning [47].”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [47] Kim, 2018, Chatbots and Korean EFL students’ English vocabulary learning, J. Digit. Converg.

      Not checked in the free run.

  • May combine two claims.

    Study design and conductown data

    This study included major databases and conducted a meta-analysis under the PRISMA guidelines.

    “Methodologically, this study included major databases and conducted a meta-analysis under the PRISMA guidelines to examine the effectiveness of chatbot technology on educational outcomes.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Premisecitation

    Chatbots could improve Chinese students' thinking ability and facilitate interactive learning.

    “Chatbots could improve Chinese students' thinking ability and facilitate interactive learning [13].”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [13] Li, 2022, Al intelligence chatbot to improve students learning in the higher education platform, J. Interconnect. Netw. source

      Not checked in the free run.

  • May combine two claims.

    Interpretationreasoning

    Chatbots also asked for clear and convincing statements, motivating students to find strong evidence.

    “Chatbots also asked for clear and convincing statements, motivating students to find strong evidence.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Premisecitation

    The LINE Bot used in military science could provide procedures for engine fan module decomposition, which improved trainee performance and reduced training costs.

    “The LINE Bot used in military science could provide procedures for engine fan module decomposition, which improved trainee performance and reduced training costs [30].”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [30] (2022, September 08). Development of Mobile Interactive Courses Based on an Artificial Intelligence Chatbot on the Communication Software LINE. Available online: https://doi.org/10.1080/10494820.2021.1937230. source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    The AsasaraBot, acting as a tutor, provided questions, encouragement, and interactions, aiming to support students' language learning.

    “The AsasaraBot, acting as a tutor, provided questions, encouragement, and interactions, aiming to support students' language learning [15].”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [15] Mageira, K., Pittou, D., Papasalouros, A., Kotis, K., Zangogianni, P., and Daradoumis, A. (2022). Educational AI chatbots for content and language integrated learning. Appl. Sci.-Basel, 12. source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    A narrative-based learning system equipped with chatbot feedback significantly improved users' engagement.

    “A narrative-based learning system equipped with chatbot feedback significantly improved users' engagement [37].”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [37] Ruan, S., He, J.Y., Ying, R., Burkle, J., Hakim, D., Wang, A., Yin, Y.F., Zhou, L., Xu, Q.Y., and AbuHashem, A. (2020, January 21–24). Supporting Children’s Math Learning with Feedback-Augmented Narrative Technology. Proceedings of the ACM Interaction Design and Children (IDC) Conference, London, UK. source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Premisecitation

    University students created conversations with the chatbot Elbot ranging from school life to movies, and after eight weeks, the experimental group better acquired vocabulary knowledge than the control group.

    “University students created conversations with the chatbot Elbot ranging from school life to movies.After eight weeks, the experimental group better acquired vocabulary knowledge than the control group [47].”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [47] Kim, 2018, Chatbots and Korean EFL students’ English vocabulary learning, J. Digit. Converg.

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    Chen et al. examined the effects of mobile devices on language learning across different intervention durations.

    “For example, Chen et al. [50] examined the effects of mobile devices on language learning across different intervention durations.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [50] Chen, 2020, The effects of using mobile devices on language learning: A meta-analysis, Educ. Technol. Res. Dev. source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    Chatbots improved Korean students' interest in foreign language learning.

    “Chatbots improved Korean students' interest in foreign language learning [43].”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [43] Han, 2020, The effects of voice-based AI chatbots on Korean EFL middle school students’ speaking competence and affective domains, Asia-Pac. J. Converg. Res. Interchange source

      Not checked in the free run.

  • Classified as probably not a substantial claim. May combine two claims.

    Premisenone given

    Many studies have measured the effectiveness of chatbots but have obtained inconsistent findings.

    “Given the benefits and concerns of educational chatbot use, many studies have measured the effectiveness of chatbots but have obtained inconsistent findings.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Premisecitation

    In nursing education, students used chatbot systems to learn courses about the physical examination, effectively enhancing students' academic performance.

    “In nursing education, students used chatbot systems to learn courses about the physical examination, effectively enhancing students' academic performance [53].”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [53] Chang, 2022, Chatbot-facilitated nursing education: Incorporating a knowledge-based chatbot system into a nursing training program, Educ. Technol. Soc.

      Not checked in the free run.

  • May combine two claims.

    Premisecitation

    The dynamic assessment of chatbots was effective in enhancing vocabulary retention and providing detailed information about personal learning.

    “The dynamic assessment of chatbots was effective in enhancing vocabulary retention and providing detailed information about personal learning [2].”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [2] (2022, September 08). Chatbot-Assisted Dynamic Assessment (CA-DA) for L2 Vocabulary Learning and Diagnosis. Available online: https://doi.org/10.1080/09588221.2021.1987272. source

      Not checked in the free run.

  • May combine two claims.

    Premisecitation

    Chatbot interventions could effectively stimulate group discussions and help students utter their thoughts.

    “Chatbot interventions could effectively stimulate group discussions and help students utter their thoughts [55].”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [55] Tegos, 2015, Promoting academically productive talk with conversational agent interventions in collaborative learning settings, Comput. Educ. source

      Not checked in the free run.

  • Classified as probably not a substantial claim. May combine two claims.

    Premisecitation

    Explicit reasoning could increase collaboration practices, improve learning outcomes, and promote conceptual knowledge acquisition.

    “Moreover, explicit reasoning could increase collaboration practices, improve learning outcomes, and promote conceptual knowledge acquisition [27].”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [27] Papadopoulos, 2013, ‘Make it explicit!’: Improving collaboration through increase of script coercion, J. Comput. Assist. Learn. source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    The meta-analysis of robot-assisted language learning examined the influence of robot roles on the effectiveness of social robots.

    “The meta-analysis of robot-assisted language learning [52] examined the influence of robot roles on the effectiveness of social robots.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [52] Lee, 2022, The effects of robot-assisted language learning: A meta-analysis, Educ. Res. Rev. source

      Not checked in the free run.

  • May combine two claims.

    Premisecitation

    The artificial intelligence-integrated chatbot was proved effective to enhance students' thinking ability and expectations.

    “The artificial intelligence-integrated chatbot was proved effective to enhance students' thinking ability and expectations [13].”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [13] Li, 2022, Al intelligence chatbot to improve students learning in the higher education platform, J. Interconnect. Netw. source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Premisecitation

    Fifteen instructional sessions over three weeks facilitated students' comprehension of English adjectival constructions but failed to help learners generate prepositional constructions.

    “Fifteen instructional sessions carried out over three weeks facilitated students' comprehension of English adjectival constructions but failed to help learners generate prepositional constructions [48].”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [48] Kim, R. (2022). Effects of AI Chatbot-Based Instruction on the Learning of English Adjectival Transitive Resultative Construction by Korean High School Students. [Ph.D. Thesis, Seoul National University]. Available online: https://hdl.handle.net/10371/181242.

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    Nursing college students using chatbots experienced a higher level of learning interest than their peers in the control group.

    “Similarly, nursing college students using chatbots experienced a higher level of learning interest than their peers in the control group [46].”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [46] Han, J.W., Park, J., and Lee, H.N. (2022). Analysis of the effect of an artificial intelligence chatbot educational program on non-face-to-face classes: A quasi-experimental study. BMC Med. Educ., 22. source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Premisecitation

    The novelty effect may appear when students are introduced to new technology, and improvement of learning outcomes might result from students' newness to chatbot technology.

    “The novelty effect may appear when students are introduced to new technology.The improvement of learning outcomes might result from students' newness to chatbot technology [8].”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [8] Fryer, 2017, Stimulating and sustaining interest in a language course: An experimental comparison of Chatbot and Human task partners, Comput. Hum. Behav. source

      Not checked in the free run.

  • May combine two claims.

    Interpretationreasoning

    The conversational behaviors in the chatbot group were considered more transactive, with more explicit arguments, than those in the control group.

    “Thus, the conversational behaviors in the chatbot group were considered more transactive, i.e., with more explicit arguments, than those in the control group.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Premisecitation

    Review activities before presenting new information activate students' prior knowledge, facilitating the integration of the new and old knowledge.

    “Review activities before presenting new information activate students' prior knowledge, facilitating the integration of the new and old knowledge [57].”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [57] Lee, 2022, Impacts of an AI-based chatbot on college students’ after-class review, academic performance, self-efficacy, learning attitude, and motivation, Educ. Technol. Res. Dev. source

      Not checked in the free run.

  • May combine two claims.

    Study design and conductnone given

    The control group in the included studies did not obtain access to chatbots, making it difficult to compare users' attitudes between the control and intervention groups.

    “Since the control group in the included studies did not obtain access to chatbots, it was difficult to compare users' attitudes between the control and intervention groups.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Premisecitation

    Pressure was a significant predictor of motivation in the chatbot-assisted learning context, whereas perceived competency was an influencing factor in the traditional context.

    “Specifically, pressure was a significant predictor of motivation in the chatbot-assisted learning context, whereas perceived competency was an influencing factor in the traditional context [18].”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [18] Yin, 2021, Conversation technology with micro-learning: The impact of chatbot-based learning on students’ learning motivation and performance, J. Educ. Comput. Res. source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductnone given

    The researchers only included studies written in English.

    “First, the researchers only included studies written in English.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Study design and conductnone given

    This study only included three moderator variables that did not significantly influence the effectiveness of chatbots.

    “Second, this study only included three moderator variables that did not significantly influence the effectiveness of chatbots.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

    Study design and conductnone given

    The results may still be influenced by unpublished studies with insignificant results despite the absence of publication bias.

    “Third, the results may still be influenced by unpublished studies with insignificant results despite the absence of publication bias.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Premisecitation

    Computer science and language were the most targeted fields in chatbot-based education, while engineering and mathematics received less attention.

    “Computer science and language were the most targeted fields in chatbot-based education, while engineering and mathematics received less attention [12].”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [12] (2022, September 08). Interacting with Educational Chatbots: A Systematic Review. Available online: https://doi.org/10.1007/s10639-022-11177-3. source

      Not checked in the free run.

34 of 34

  • The paper relies on this without support.

    The included studies had sufficient statistical power to detect effects on learning engagement and motivation.

    “However, significant differences in learning engagement and motivation were not found between the chatbot-based condition and the control condition.”
1 of 1

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • The study highlights the constraints of ChatGPT, particularly its potential to limit creativity and critical thinking, aligning with Deng and Yu (2023) but contrasting with Abramson's (2023) view that AI can enhance creativity by offloading routine tasks.

    “However, the study also highlights the constraints of ChatGPT, particularly its potential to limit creativity and critical thinking, which aligns with the findings of Deng and Yu (2023) but contrasts with Abramson's (2023) view that AI can enhance creativity by offloading routine tasks.”

    From Motivations and Affordances of ChatGPT Usage for College Students’ Learning (2025) · cites this paper as Deng and Yu (2023)

    Split into 2 atoms — 0 confirmed, 2 partially supported, 0 disputed, 0 misattributed, 0 unverifiable. Expand the claim to see per-atom verdicts.

    Experimental — accuracy not yet measured

  • The notable increase in knowledge may be associated with interactive chatbot engagement.

    “This notable increase may be associated with interactive chatbot engagement; according to Deng and Yu (2023), one-way and two-way text message exchanges retain approximately 25% more knowledge.”

    From SiKurang: Development and Field Evaluation of an Offline-First mHealth System for Community-Based Stunting Risk Detection and Counselling in Low-Connectivity Primary Care Settings (2026) · cites this paper as Deng and Yu (2023)

    The Results (section 4.3) report significantly better knowledge retention with chatbots than in the control group, and the Discussion (section 5) links chatbot interactions to learning and recall. The paper does not report that one-way and two-way text-message exchanges retain approximately 25% more knowledge; its retention result is a standardized effect size, not that percentage. (LLM, conf=0.88)

    Experimental — accuracy not yet measured

  • One-way and two-way text message exchanges retain approximately 25% more knowledge.

    “according to Deng and Yu (2023), one-way and two-way text message exchanges retain approximately 25% more knowledge.”

    From SiKurang: Development and Field Evaluation of an Offline-First mHealth System for Community-Based Stunting Risk Detection and Counselling in Low-Connectivity Primary Care Settings (2026) · cites this paper as Deng and Yu (2023)

    Section 4.3 reports significantly better knowledge retention with chatbot-assisted learning than in the control group (d = 0.691). It does not report an approximately 25% gain or distinguish one-way from two-way text-message exchanges; searches for those terms found no supporting passage. Page numbers were not available in our copy. (LLM, conf=0.88)

    Experimental — accuracy not yet measured

  • A meta-analysis by Deng and Yu indicated that chatbot programs can serve three roles in education-teaching assistant, personal tutor, and learning partner.

    “This aligns with a meta-analysis conducted by Deng and Yu [31], which indicated that chatbot programs can serve three roles in education—teaching assistant, personal tutor, and learning partner (Fig. 2).”

    From Development and effects of a chatbot education program for self-directed learning in nursing students (2025) · cites this paper as [31]

  • AI tools such as chatbots have been shown to enhance student interaction and improve learning outcomes.

    “AI tools, such as chatbots, have been shown to enhance student interaction and improve learning outcomes (Deng & Yu, 2023;Rasul et al., 2023).”

    From Ethical use of artificial intelligence based tools in higher education: Are future business leaders ready? (2025) · cites this paper as Deng & Yu, 2023

Statistics recomputed

Every reported test result, recomputed from its test statistic and degrees of freedom. A result shows as not adding up only once an independent check has confirmed it.

  • Consistentz = 1.47reported p = .142 · recomputed p = .142

    “z = 1.47, p = .142”

  • Consistentz = 1.32reported p = .188 · recomputed p = .187

    “z = 1.32, p = .188”

  • Consistentz = 0.53reported p = .598 · recomputed p = .596

    “z = 0.53, p = .598”

  • Consistentz = 1.35reported p = .176 · recomputed p = .177

    “z = 1.35, p = .176”

  • Consistentz = 1.23reported p = .217 · recomputed p = .219

    “z = 1.23, p = .217”

  • Consistentz = 2.04reported p = .042 · recomputed p = .041

    “(z = 2.04, p = .042”

6 of 6

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

Each contradiction between two passages listed here was confirmed by an independent check.

  • One study's group sizes do not match its reported total

    Minor

    For H. L. Chen et al. (2020), the experimental and control groups contain 19 and 29 participants, totaling 48 rather than the reported 58.

    Reported 58 · recomputed 48

  • Learning-engagement p-value differs between text and table

    Minor

    The results text reports p = 0.114 for learning engagement, while Table 4 reports p = 0.180 for the same effect. Both are nonsignificant, so the conclusion is unchanged.

2 of 2

Cites

Flagged first, then by citations

1 of the 64 references couldn’t be matched to a known paper.

Research works printed but not linked: 6

  1. Lee, 2022, A meta-analysis of the effects of using AI chatbot in Korean EFL education, Stud. Engl. Lang. Lit., 48, 213
  2. Sternberg, R.J., and Preiss, D.D. (2010). Innovations in Educational Psychology: Perspectives on Learning, Teaching and Human Development, Springer.
  3. Cole, P.G., and Chan, L.K.S. (1994). Teaching Principles and Practice, Prentice Hall.
  4. Kim, R. (2022). Effects of AI Chatbot-Based Instruction on the Learning of English Adjectival Transitive Resultative Construction by Korean High School Students. [Ph.D. Thesis, Seoul National University]. Available online: https://hdl.handle.net/10371/181242.
  5. Abbasi, 2019, Effect of chatbot systems on student’s learning outcomes, Sylwan, 163, 49
  6. Cohen, J. (1988). Statistical Power Analysis for the Behavioral Sciences, Routledge. [2nd ed.].

Cited by

Sorted by citations

Related

Authors
Deng, X., & Yu, Z.
Affiliations
Beijing Language and Culture University (CN)
Corresponding author
Yu, Zhonggen
ORCID
Venue
Sustainability
Publisher
MDPI AG
Published
2023-02-06
Volume
15
Issue
4
Pages
2940
ISSN
2071-1050 (ISSN-L)
DOI
10.3390/su15042940
OpenAlex
W4319440682
Semantic Scholar
27f4d355b27a6878332a87845a6ff13940fe9158
Cited by
298
References
64
Citations by year
2026: 76 · 2025: 115 · 2024: 72 · 2023: 35
FWCI
12.28
Type
journal article
Language
en
Open access
Licence
Locations
  • Sustainability (publishedVersion, cc-by)
  • RePEc: Research Papers in Economics (submittedVersion)
Funders
  • 2019 MOOC of Beijing Language and Culture University (MOOC201902) (important) “Introduction to Linguistics” (202010032003, SJTS202108, 22YCX017)
  • “Introduction to Linguistics” of online and offline mixed courses in Beijing Language and Culture University in 2020 (202010032003, SJTS202108, 22YCX017)
  • Beijing Co-construction Project Research and reform of the “Undergraduate Teaching Reform and Innovation Project” of Beijing higher education in 2020-innovative “multilingual +” excellent talent training system (202010032003, SJTS202108, 22YCX017)
  • The research project of Graduate Students of Beijing Language and Culture University “Xi Jinping: The Governance of China” (202010032003, SJTS202108, 22YCX017)
  • Beijing Language and Culture University (202010032003, SJTS202108, 22YCX017)
Topics
  • Physical Sciences › Computer Science › Information Systems › Artificial Intelligence in Education
  • Physical Sciences › Computer Science › Artificial Intelligence › Intelligent Tutoring Systems and Adaptive Learning
  • Physical Sciences › Computer Science › Artificial Intelligence › AI in Service Interactions
Keywords
chatbots, chatbot-assisted learning, sustainable education, learning outcomes, moderators, explicit reasoning, knowledge retention, academic performance, student learning interest, critical thinking, student engagement, student motivation, artificial intelligence in education

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Claims read by deepseek-v4.1-flash under definition version 3 (extract-v3) on 2026-10-06.

Classified by jev-1.13-20260917 (gold-guideline-v0.2) on 2026-10-06.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review