The DOI matches its Crossref record; 63 of the 64 references it cites were found in a registry or an index; none of the 35 sources its claims cite is checked for support yet.
A Meta-Analysis and Systematic Review of the Effect of Chatbot Technology Use in Sustainable Education
Open accessAbstract
The development of artificial intelligence in recent years has promoted the use of chatbot technology in sustainable education. Many studies examined the effect of chatbots on learning outcomes. However, scant studies summarized the effectiveness of chatbots in education. The aim of the study is to investigate the effect of chatbot-assisted learning on various components and how different moderator variables influenced the effectiveness. This study, through a meta-analysis, reviewed 32 empirical studies with 2201 participants published between 2010 and 2022. The results showed that chatbot technology exerted a medium-to-high effect on overall learning outcomes regardless of moderator variables, i.e., intervention duration, chatbot roles, and learning content. In terms of learning components, chatbots could significantly improve explicit reasoning, learning achievement, knowledge retention, and learning interest despite negative findings in critical thinking, learning engagement, and motivation. Future research could expand chatbot research by including different learning components.
Overview
AI snapshot
Written from the paper’s full text.
TL;DR
This meta-analysis and systematic review examined 32 empirical studies with 2201 participants and found that chatbot technology had a medium-to-large positive overall effect on learning outcomes (d = 0.789). Effects were significant for explicit reasoning, learning achievement, knowledge retention, and learning interest, but not for critical thinking, learning engagement, or motivation.
Key findings
- The overall effect size of chatbot use in education was d = 0.789 (95% CI [0.593, 0.985], p < 0.001), described as medium-to-large.
- Chatbots significantly improved explicit reasoning (d = 1.190, p < 0.001), learning achievement (d = 1.033, p < 0.001), knowledge retention (d = 0.691, p = 0.022), and learning interest (d = 0.842, p = 0.041).
- Chatbots did not significantly improve critical thinking (d = 0.690, p = 0.144), learning engagement (d = 0.147, p = 0.114), or learning motivation (d = 0.409, p = 0.114).
- Moderator analyses found no statistically significant differences by intervention duration, chatbot roles, or learning content (all p > 0.05), though subgroup effect sizes varied.
Methods
- The authors conducted a PRISMA-guided meta-analysis and systematic review, searching major databases including Web of Science, Wiley Online Library, Springer Link, Taylor & Francis Online, Elsevier ScienceDirect, and Google Scholar on 3 September 2022.
- They included 32 empirical studies published between 2010 and 2022 with 2201 participants, calculating 76 effect sizes using Cohen’s d; studies needed experimental/control comparisons and sufficient statistics.
- Analyses used Stata MP/14.0, with publication-bias checks (funnel plot, Begg’s test, Egger’s test), heterogeneity tests (I²), sensitivity analysis, and meta-regression for intervention duration, chatbot roles, and learning content.
Limitations
- The paper states that only studies written in English were included, that only three moderator variables were examined, and that unpublished studies with insignificant results may still influence the findings despite no detected publication bias.
- Evident from the text: several outcome domains had high heterogeneity (e.g., critical thinking I² = 92.8%, motivation I² = 93.2%, overall I² = 87.8%), and some categories had very few studies, limiting moderator conclusions.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
0 of 35 cited sources checked · 0 supported · 0 flagged
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The meta-regression results revealed no statistically significant differences between the abovementioned values.”
§ A Meta-Analysis and Systematic Review of the Effect of Chatbot Technology Use in Sustainable Education > 4.3. Results of Moderator Analyses
Load-bearingStrength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesA result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“No significant differences in learning engagement (d = 0.147, 95% CI [-0.068, 0.363], p = 0.114) were found between chatbot-assisted and traditional learning methods.”
§ A Meta-Analysis and Systematic Review of the Effect of Chatbot Technology Use in Sustainable Education > 4.3. Results of Moderator Analyses
Load-bearingStrength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesA result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“However, there were no significant differences between effect sizes of the different intervention durations (p > 0.05).”
§ A Meta-Analysis and Systematic Review of the Effect of Chatbot Technology Use in Sustainable Education > 4.3. Results of Moderator Analyses
Load-bearingStrength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesA result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Table 4 shows a significant difference in explicit reasoning (d = 1.190, 95% CI [0.868, 1.512], p < 0.001) between the experimental and control groups.”
§ A Meta-Analysis and Systematic Review of the Effect of Chatbot Technology Use in Sustainable Education > 4.3. Results of Moderator Analyses
Load-bearingStrength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesA result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The effect size did not show a significant difference in critical thinking (d = 0.690, 95% CI [-0.235, 1.615], p = 0.144) between the chatbot-assisted and conventional learning methods.”
§ A Meta-Analysis and Systematic Review of the Effect of Chatbot Technology Use in Sustainable Education > 4.3. Results of Moderator Analyses
Load-bearingStrength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesA result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“and retention (d = 0.691, 95% CI [0.101, 1.281], p = 0.022) in the experimental group significantly improved compared with the control group.”
§ A Meta-Analysis and Systematic Review of the Effect of Chatbot Technology Use in Sustainable Education > 4.3. Results of Moderator Analyses
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesA result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“the experimental group maintained significantly higher levels of learning interest (d = 0.842, 95% CI [0.034, 1.650], p = 0.041).”
§ A Meta-Analysis and Systematic Review of the Effect of Chatbot Technology Use in Sustainable Education > 4.3. Results of Moderator Analyses
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesA result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The overall effect size of using chatbots in education was 0.789 (p < 0.001), with a 95% confidence interval between 0.593 and 0.985 (Table 4).”
§ A Meta-Analysis and Systematic Review of the Effect of Chatbot Technology Use in Sustainable Education > 4.3. Results of Moderator Analyses
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesA result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“However, learning motivation in both groups was not significantly different (d = 0.409, 95% CI [-0.099, 0.916], p = 0.114).”
§ A Meta-Analysis and Systematic Review of the Effect of Chatbot Technology Use in Sustainable Education > 4.3. Results of Moderator Analyses
Load-bearingStrength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesA result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Additionally, the heterogeneity test showed that the overall effect size (Q = 582.09,I 2 = 87.8%,p < 0.001) was of considerable heterogeneity”
§ A Meta-Analysis and Systematic Review of the Effect of Chatbot Technology Use in Sustainable Education > 4.3. Results of Moderator Analyses
Load-bearingStrength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passes
- It may state two separate claims as one. 23
- Probably not a substantial claim. 20
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“However, Begg's test showed a presence of publication bias in learning interest (z = 2.04, p = 0.042), which was different from the result of Egger's test (t = 15.30,p = 0.056).”
§ A Meta-Analysis and Systematic Review of the Effect of Chatbot Technology Use in Sustainable Education > 4.1. Analyses of Publication Bias
Strength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes? Load-bearing: unclear✓ Based: passesIt may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The researchers calculated 76 effect sizes with a total sample size of 2201 in 32 identified studies.”
§ A Meta-Analysis and Systematic Review of the Effect of Chatbot Technology Use in Sustainable Education > 3.4. Data Analysis
Load-bearingStrength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesIt may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“It indicates that none of the included studies could influence the pooled effect size when a specific study is omitted, confirming that the meta-analytical results are robust.”
§ A Meta-Analysis and Systematic Review of the Effect of Chatbot Technology Use in Sustainable Education > 4.2. Results of Sensitivity Analysis
Strength:hedged· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes? Load-bearing: unclear✓ Based: passesIt may state two separate claims as one.
In the paper
“They strictly followed the Preferred Reporting Items for Systematic reviews and Meta-Analyses (PRISMA) guidelines [54].”
§ A Meta-Analysis and Systematic Review of the Effect of Chatbot Technology Use in Sustainable Education > 3. Materials and Methods
What the sources say
- Not checked yetcites [54] Page, 2021, The PRISMA 2020 statement: An updated guideline for reporting systematic reviews, Br. Med. J. source
Strength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The overall effect size in this study was thus considered as a mediumto-large level, indicating that the use of chatbot technology exerted a positive effect on learning outcomes.”
§ A Meta-Analysis and Systematic Review of the Effect of Chatbot Technology Use in Sustainable Education > 4.3. Results of Moderator Analyses
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesIt may state two separate claims as one.
In the paper
“one display of explicit reasoning could associate with another one, which was the core of transactivity theory [70].”
§ A Meta-Analysis and Systematic Review of the Effect of Chatbot Technology Use in Sustainable Education > 5. Discussion
What the sources say
- Not checked yetcites [70] Pinkwart, N., and McLaren, B. (2012). Educational Technologies for Teaching Argumentation Skills, Bentham Science Publishers. Rosé, C.P. source
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesProbably not a substantial claim.
In the paper
“Chatbots also increased college students' interest in English vocabulary learning [47].”
§ A Meta-Analysis and Systematic Review of the Effect of Chatbot Technology Use in Sustainable Education > H6.
What the sources say
- Not checked yetcites [47] Kim, 2018, Chatbots and Korean EFL students’ English vocabulary learning, J. Digit. Converg.
Strength:plain· not yet calibratedCausal:direct· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“Methodologically, this study included major databases and conducted a meta-analysis under the PRISMA guidelines to examine the effectiveness of chatbot technology on educational outcomes.”
§ A Meta-Analysis and Systematic Review of the Effect of Chatbot Technology Use in Sustainable Education > 6.1. Major Contributions
Load-bearingStrength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesIt may state two separate claims as one.
In the paper
“Chatbots could improve Chinese students' thinking ability and facilitate interactive learning [13].”
§ A Meta-Analysis and Systematic Review of the Effect of Chatbot Technology Use in Sustainable Education > 1. Introduction
What the sources say
- Not checked yetcites [13] Li, 2022, Al intelligence chatbot to improve students learning in the higher education platform, J. Interconnect. Netw. source
Strength:hedged· not yet calibratedCausal:conditional· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Chatbots also asked for clear and convincing statements, motivating students to find strong evidence.”
§ A Meta-Analysis and Systematic Review of the Effect of Chatbot Technology Use in Sustainable Education > 5. Discussion
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesIt may state two separate claims as one.
In the paper
“The LINE Bot used in military science could provide procedures for engine fan module decomposition, which improved trainee performance and reduced training costs [30].”
§ A Meta-Analysis and Systematic Review of the Effect of Chatbot Technology Use in Sustainable Education > 2.7. Learning Content
What the sources say
- Not checked yetcites [30] (2022, September 08). Development of Mobile Interactive Courses Based on an Artificial Intelligence Chatbot on the Communication Software LINE. Available online: https://doi.org/10.1080/10494820.2021.1937230. source
Strength:hedged· not yet calibratedCausal:direct· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.
In the paper
“The AsasaraBot, acting as a tutor, provided questions, encouragement, and interactions, aiming to support students' language learning [15].”
§ A Meta-Analysis and Systematic Review of the Effect of Chatbot Technology Use in Sustainable Education > 2.6. Chatbot Roles
What the sources say
- Not checked yetcites [15] Mageira, K., Pittou, D., Papasalouros, A., Kotis, K., Zangogianni, P., and Daradoumis, A. (2022). Educational AI chatbots for content and language integrated learning. Appl. Sci.-Basel, 12. source
Strength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it.
In the paper
“A narrative-based learning system equipped with chatbot feedback significantly improved users' engagement [37].”
§ A Meta-Analysis and Systematic Review of the Effect of Chatbot Technology Use in Sustainable Education > 2.4. Learners' Engagement, Motivation, and Interest
What the sources say
- Not checked yetcites [37] Ruan, S., He, J.Y., Ying, R., Burkle, J., Hakim, D., Wang, A., Yin, Y.F., Zhou, L., Xu, Q.Y., and AbuHashem, A. (2020, January 21–24). Supporting Children’s Math Learning with Feedback-Augmented Narrative Technology. Proceedings of the ACM Interaction Design and Children (IDC) Conference, London, UK. source
Strength:plain· not yet calibratedCausal:direct· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it.
In the paper
“University students created conversations with the chatbot Elbot ranging from school life to movies.After eight weeks, the experimental group better acquired vocabulary knowledge than the control group [47].”
§ A Meta-Analysis and Systematic Review of the Effect of Chatbot Technology Use in Sustainable Education > 2.6. Chatbot Roles
What the sources say
- Not checked yetcites [47] Kim, 2018, Chatbots and Korean EFL students’ English vocabulary learning, J. Digit. Converg.
Strength:plain· not yet calibratedCausal:direct· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.
In the paper
“For example, Chen et al. [50] examined the effects of mobile devices on language learning across different intervention durations.”
§ A Meta-Analysis and Systematic Review of the Effect of Chatbot Technology Use in Sustainable Education > 2.5. Intervention Duration
What the sources say
- Not checked yetcites [50] Chen, 2020, The effects of using mobile devices on language learning: A meta-analysis, Educ. Technol. Res. Dev. source
Strength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it.
In the paper
“Chatbots improved Korean students' interest in foreign language learning [43].”
§ A Meta-Analysis and Systematic Review of the Effect of Chatbot Technology Use in Sustainable Education > H6.
What the sources say
- Not checked yetcites [43] Han, 2020, The effects of voice-based AI chatbots on Korean EFL middle school students’ speaking competence and affective domains, Asia-Pac. J. Converg. Res. Interchange source
Strength:plain· not yet calibratedCausal:direct· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“Given the benefits and concerns of educational chatbot use, many studies have measured the effectiveness of chatbots but have obtained inconsistent findings.”
§ A Meta-Analysis and Systematic Review of the Effect of Chatbot Technology Use in Sustainable Education > 1. Introduction
Load-bearingStrength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesProbably not a substantial claim. It may state two separate claims as one.
In the paper
“In nursing education, students used chatbot systems to learn courses about the physical examination, effectively enhancing students' academic performance [53].”
§ A Meta-Analysis and Systematic Review of the Effect of Chatbot Technology Use in Sustainable Education > 2.7. Learning Content
What the sources say
- Not checked yetcites [53] Chang, 2022, Chatbot-facilitated nursing education: Incorporating a knowledge-based chatbot system into a nursing training program, Educ. Technol. Soc.
Strength:plain· not yet calibratedCausal:direct· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes? Load-bearing: unclear✓ Based: passesIt may state two separate claims as one.
In the paper
“The dynamic assessment of chatbots was effective in enhancing vocabulary retention and providing detailed information about personal learning [2].”
§ A Meta-Analysis and Systematic Review of the Effect of Chatbot Technology Use in Sustainable Education > 2.3. Learning Achievement and Knowledge Retention
What the sources say
- Not checked yetcites [2] (2022, September 08). Chatbot-Assisted Dynamic Assessment (CA-DA) for L2 Vocabulary Learning and Diagnosis. Available online: https://doi.org/10.1080/09588221.2021.1987272. source
Strength:plain· not yet calibratedCausal:direct· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes? Load-bearing: unclear✓ Based: passesIt may state two separate claims as one.
In the paper
“Chatbot interventions could effectively stimulate group discussions and help students utter their thoughts [55].”
§ A Meta-Analysis and Systematic Review of the Effect of Chatbot Technology Use in Sustainable Education > 5. Discussion
What the sources say
- Not checked yetcites [55] Tegos, 2015, Promoting academically productive talk with conversational agent interventions in collaborative learning settings, Comput. Educ. source
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesIt may state two separate claims as one.
In the paper
“Moreover, explicit reasoning could increase collaboration practices, improve learning outcomes, and promote conceptual knowledge acquisition [27].”
§ A Meta-Analysis and Systematic Review of the Effect of Chatbot Technology Use in Sustainable Education > 2.2. Explicit Reasoning
What the sources say
- Not checked yetcites [27] Papadopoulos, 2013, ‘Make it explicit!’: Improving collaboration through increase of script coercion, J. Comput. Assist. Learn. source
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesProbably not a substantial claim. It may state two separate claims as one.
In the paper
“The meta-analysis of robot-assisted language learning [52] examined the influence of robot roles on the effectiveness of social robots.”
§ A Meta-Analysis and Systematic Review of the Effect of Chatbot Technology Use in Sustainable Education > 2.6. Chatbot Roles
What the sources say
- Not checked yetcites [52] Lee, 2022, The effects of robot-assisted language learning: A meta-analysis, Educ. Res. Rev. source
Strength:plain· not yet calibratedCausal:correlational· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it.
In the paper
“The artificial intelligence-integrated chatbot was proved effective to enhance students' thinking ability and expectations [13].”
§ A Meta-Analysis and Systematic Review of the Effect of Chatbot Technology Use in Sustainable Education > 2.1. Critical Thinking
What the sources say
- Not checked yetcites [13] Li, 2022, Al intelligence chatbot to improve students learning in the higher education platform, J. Interconnect. Netw. source
Strength:boosted· not yet calibratedCausal:direct· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes? Load-bearing: unclear✓ Based: passesIt may state two separate claims as one.
In the paper
“Fifteen instructional sessions carried out over three weeks facilitated students' comprehension of English adjectival constructions but failed to help learners generate prepositional constructions [48].”
§ A Meta-Analysis and Systematic Review of the Effect of Chatbot Technology Use in Sustainable Education > 2.5. Intervention Duration
What the sources say
- Not checked yetcites [48] Kim, R. (2022). Effects of AI Chatbot-Based Instruction on the Learning of English Adjectival Transitive Resultative Construction by Korean High School Students. [Ph.D. Thesis, Seoul National University]. Available online: https://hdl.handle.net/10371/181242.
Strength:plain· not yet calibratedCausal:direct· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.
In the paper
“Similarly, nursing college students using chatbots experienced a higher level of learning interest than their peers in the control group [46].”
§ A Meta-Analysis and Systematic Review of the Effect of Chatbot Technology Use in Sustainable Education > H6.
What the sources say
- Not checked yetcites [46] Han, J.W., Park, J., and Lee, H.N. (2022). Analysis of the effect of an artificial intelligence chatbot educational program on non-face-to-face classes: A quasi-experimental study. BMC Med. Educ., 22. source
Strength:plain· not yet calibratedCausal:direct· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it.
In the paper
“The novelty effect may appear when students are introduced to new technology.The improvement of learning outcomes might result from students' newness to chatbot technology [8].”
§ A Meta-Analysis and Systematic Review of the Effect of Chatbot Technology Use in Sustainable Education > 1. Introduction
What the sources say
- Not checked yetcites [8] Fryer, 2017, Stimulating and sustaining interest in a language course: An experimental comparison of Chatbot and Human task partners, Comput. Hum. Behav. source
Strength:hedged· not yet calibratedCausal:conditional· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Thus, the conversational behaviors in the chatbot group were considered more transactive, i.e., with more explicit arguments, than those in the control group.”
§ A Meta-Analysis and Systematic Review of the Effect of Chatbot Technology Use in Sustainable Education > 5. Discussion
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesIt may state two separate claims as one.
In the paper
“Review activities before presenting new information activate students' prior knowledge, facilitating the integration of the new and old knowledge [57].”
§ A Meta-Analysis and Systematic Review of the Effect of Chatbot Technology Use in Sustainable Education > 5. Discussion
What the sources say
- Not checked yetcites [57] Lee, 2022, Impacts of an AI-based chatbot on college students’ after-class review, academic performance, self-efficacy, learning attitude, and motivation, Educ. Technol. Res. Dev. source
Strength:plain· not yet calibratedCausal:direct· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“Since the control group in the included studies did not obtain access to chatbots, it was difficult to compare users' attitudes between the control and intervention groups.”
§ A Meta-Analysis and Systematic Review of the Effect of Chatbot Technology Use in Sustainable Education > 6.3. Implications for Future Research
Strength:plain· not yet calibratedCausal:direct· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes? Load-bearing: unclear✓ Based: passesIt may state two separate claims as one.
In the paper
“Specifically, pressure was a significant predictor of motivation in the chatbot-assisted learning context, whereas perceived competency was an influencing factor in the traditional context [18].”
§ A Meta-Analysis and Systematic Review of the Effect of Chatbot Technology Use in Sustainable Education > 5. Discussion
What the sources say
- Not checked yetcites [18] Yin, 2021, Conversation technology with micro-learning: The impact of chatbot-based learning on students’ learning motivation and performance, J. Educ. Comput. Res. source
Strength:plain· not yet calibratedCausal:direct· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes? Load-bearing: unclear✓ Based: passesIt may state two separate claims as one.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“First, the researchers only included studies written in English.”
§ A Meta-Analysis and Systematic Review of the Effect of Chatbot Technology Use in Sustainable Education > 6.2. Limitations
Strength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“Second, this study only included three moderator variables that did not significantly influence the effectiveness of chatbots.”
§ A Meta-Analysis and Systematic Review of the Effect of Chatbot Technology Use in Sustainable Education > 6.2. Limitations
Strength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“Third, the results may still be influenced by unpublished studies with insignificant results despite the absence of publication bias.”
§ A Meta-Analysis and Systematic Review of the Effect of Chatbot Technology Use in Sustainable Education > 6.2. Limitations
Strength:hedged· not yet calibratedCausal:conditional· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✗ Based: failsProbably not a substantial claim. The paper’s conclusions may not depend on it. The paper may give no reason why it holds.
In the paper
“Computer science and language were the most targeted fields in chatbot-based education, while engineering and mathematics received less attention [12].”
§ A Meta-Analysis and Systematic Review of the Effect of Chatbot Technology Use in Sustainable Education > 5. Discussion
What the sources say
- Not checked yetcites [12] (2022, September 08). Interacting with Educational Chatbots: A Systematic Review. Available online: https://doi.org/10.1007/s10639-022-11177-3. source
Strength:plain· not yet calibratedCausal:correlational· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes? Load-bearing: unclear✓ Based: passesIt may state two separate claims as one.
The paper relies on this without support.
The included studies had sufficient statistical power to detect effects on learning engagement and motivation.
“However, significant differences in learning engagement and motivation were not found between the chatbot-based condition and the control condition.”
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“However, the study also highlights the constraints of ChatGPT, particularly its potential to limit creativity and critical thinking, which aligns with the findings of Deng and Yu (2023) but contrasts with Abramson's (2023) view that AI can enhance creativity by offloading routine tasks.”
From Motivations and Affordances of ChatGPT Usage for College Students’ Learning (2025) · cites this paper as Deng and Yu (2023)
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“This notable increase may be associated with interactive chatbot engagement; according to Deng and Yu (2023), one-way and two-way text message exchanges retain approximately 25% more knowledge.”
From SiKurang: Development and Field Evaluation of an Offline-First mHealth System for Community-Based Stunting Risk Detection and Counselling in Low-Connectivity Primary Care Settings (2026) · cites this paper as Deng and Yu (2023)
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“according to Deng and Yu (2023), one-way and two-way text message exchanges retain approximately 25% more knowledge.”
From SiKurang: Development and Field Evaluation of an Offline-First mHealth System for Community-Based Stunting Risk Detection and Counselling in Low-Connectivity Primary Care Settings (2026) · cites this paper as Deng and Yu (2023)
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“This aligns with a meta-analysis conducted by Deng and Yu [31], which indicated that chatbot programs can serve three roles in education—teaching assistant, personal tutor, and learning partner (Fig. 2).”
From Development and effects of a chatbot education program for self-directed learning in nursing students (2025) · cites this paper as [31]
In the citing paper
“AI tools, such as chatbots, have been shown to enhance student interaction and improve learning outcomes (Deng & Yu, 2023;Rasul et al., 2023).”
From Ethical use of artificial intelligence based tools in higher education: Are future business leaders ready? (2025) · cites this paper as Deng & Yu, 2023
In the citing paper
“chatbots, including ChatGPT, have demonstrated the potential to reshape and revolutionize education, creating new, novel opportunities for enhancing the learning process and outcomes [83], facilitating different learning approaches, and offering a range of pedagogical benefits [19, 43, 72]”
From A systematic literature review of empirical research on ChatGPT in education (2024) · cites this paper as [19]
In the citing paper
“chatbots, including ChatGPT, have the potential to significantly impact students' thinking abilities, including retention, reasoning, analysis skills [19, 45], and foster innovation and creativity capabilities [83]”
From A systematic literature review of empirical research on ChatGPT in education (2024) · cites this paper as [19]
Statistics recomputed
Every reported test result, recomputed from its test statistic and degrees of freedom. A result shows as not adding up only once an independent check has confirmed it.
- Consistentz = 1.47reported p = .142 · recomputed p = .142
“z = 1.47, p = .142”
- Consistentz = 1.32reported p = .188 · recomputed p = .187
“z = 1.32, p = .188”
- Consistentz = 0.53reported p = .598 · recomputed p = .596
“z = 0.53, p = .598”
- Consistentz = 1.35reported p = .176 · recomputed p = .177
“z = 1.35, p = .176”
- Consistentz = 1.23reported p = .217 · recomputed p = .219
“z = 1.23, p = .217”
- Consistentz = 2.04reported p = .042 · recomputed p = .041
“(z = 2.04, p = .042”
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
Each contradiction between two passages listed here was confirmed by an independent check.
One study's group sizes do not match its reported total
MinorFor H. L. Chen et al. (2020), the experimental and control groups contain 19 and 29 participants, totaling 48 rather than the reported 58.
Reported 58 · recomputed 48
“| 8 | H. L. Chen et al. (2020) [17] | 58 (19/29) | 4 weeks | Tutor | Language | Learning achievement and retention |”
Learning-engagement p-value differs between text and table
MinorThe results text reports p = 0.114 for learning engagement, while Table 4 reports p = 0.180 for the same effect. Both are nonsignificant, so the conclusion is unchanged.
“No significant differences in learning engagement (d = 0.147, 95% CI [-0.068, 0.363], p = 0.114) were found between chatbot-assisted and traditional learning methods.”
“| 5 | Learning engagement | 0.147 | -0.068, 0.363 | 1.34 | 0.180 | 3.22 | 6 | 0.780 | 0.0% |”
Citations
Cites
The PRISMA 2020 statement: An updated guideline for reporting systematic reviews
Page, M. J., McKenzie, J. E., Bossuyt, P. M., Boutron, I., Hoffmann, T. C., Mulrow, C. D., Shamseer, L., Tetzlaff, J. M., Akl, E. A., Brennan, S. E., et al. (2021) · BMJ · doi:10.1136/bmj.n71 · Cited by 108,283
Measuring inconsistency in meta-analyses
Higgins, J. P. T., Thompson, S. G., Deeks, J. J., & Altman, D. G. (2003) · BMJ · doi:10.1136/bmj.327.7414.557 · Cited by 63,139 · cited 1×
“The heterogeneity will be considered low if I2 is less than 25%, moderate if I2 falls between 25% and 75%, and substantial if I2 is greater than 75% [68].”
BackgroundIntrinsic and Extrinsic Motivations: Classic Definitions and New Directions
Ryan, R. M., & Deci, E. L. (2000) · Contemporary Educational Psychology · doi:10.1006/ceps.1999.1020 · Cited by 18,666 · cited 1×
“Intrinsic motivation focuses on learners’ inner satisfaction, whereas extrinsic motivation is defined as the behavior for external and separable results [42].”
BackgroundToward a hierarchical model of intrinsic and extrinsic motivation
Vallerand, R. J. (1997) · Advances in experimental social psychology · doi:10.1016/s0065-2601(08)60019-2 · Cited by 3,198 · cited 1×
“Motivation can be categorized into intrinsic and extrinsic motivations [41].”
BackgroundChatbots applications in education: A systematic review
Okonkwo, C. W., & Ade-Ibijola, A. (2021) · Computers and Education: Artificial Intelligence · doi:10.1016/j.caeai.2021.100033 · Cited by 879 · cited 3×
“[1], Okonkwo, and Ade-Ibijola [6], Perez et al.”
BackgroundInteracting with educational chatbots: A systematic review
Kuhail, M. A., Alturki, N., Alramlawi, S., & Alhejori, K. (2023) · Education and Information Technologies · doi:10.1007/s10639-022-11177-3 · Cited by 841 · cited 3×
“Computer science and language were the most targeted fields in chatbot-based education, while engineering and mathematics received less attention [12].”
BackgroundHuang, W., Hew, K. F., & Fryer, L. K. (2022) · Journal of Computer Assisted Learning · doi:10.1111/jcal.12610 · Cited by 830
Interest, motivation and learning: An educational-psychological perspective
Krapp, A. (1999) · European Journal of Psychology of Education · doi:10.1007/bf03173109 · Cited by 595 · cited 1×
“Interest indicates the individual readiness or predisposition to engage in a given learning task with effort [45].”
BackgroundA review of opportunities and challenges of chatbots in education
Hwang, G.-J., & Chang, C.-Y. (2023) · Interactive Learning Environments · doi:10.1080/10494820.2021.1952615 · Cited by 530
Rediscovering the use of chatbots in education: A systematic literature review
Pérez, J. Q., Daradoumis, T., & Puig, J. M. M. (2020) · Computer Applications in Engineering Education · doi:10.1002/cae.22326 · Cited by 483 · cited 1×
“Problems have also included technological limitations and training side effects [7].”
Background
Flagged first, then by citations
1 of the 64 references couldn’t be matched to a known paper.
Research works printed but not linked: 6
- Lee, 2022, A meta-analysis of the effects of using AI chatbot in Korean EFL education, Stud. Engl. Lang. Lit., 48, 213
- Sternberg, R.J., and Preiss, D.D. (2010). Innovations in Educational Psychology: Perspectives on Learning, Teaching and Human Development, Springer.
- Cole, P.G., and Chan, L.K.S. (1994). Teaching Principles and Practice, Prentice Hall.
- Kim, R. (2022). Effects of AI Chatbot-Based Instruction on the Learning of English Adjectival Transitive Resultative Construction by Korean High School Students. [Ph.D. Thesis, Seoul National University]. Available online: https://hdl.handle.net/10371/181242.
- Abbasi, 2019, Effect of chatbot systems on student’s learning outcomes, Sylwan, 163, 49
- Cohen, J. (1988). Statistical Power Analysis for the Behavioral Sciences, Routledge. [2nd ed.].
Cited by
Sorted by citations
Items 201–298 of 298
E-learning & Artificial Intelligence
(2023) · E-learning · doi:10.34916/el.2023.15 · Cited by 1
Barriers to AI Course Assistant Adoption: Understanding Student Non-Utilization at LAPUPreprint
Hanshaw, G. (2024) · Research Square · doi:10.21203/rs.3.rs-5378875/v1 · Cited by 1
Artificial Intelligence (AI) in Education
Anisah, T. N., & Andika, A. (2025) · Wiley eBooks · doi:10.1002/9781394411283.ch2 · Cited by 1
Istambul, M. R., Parlindungan, P., Henry Wijaya, J., Zezarina, R., & Fachrizsal, D. (2026) · Glosains: Jurnal Sains Global Indonesia · doi:10.59784/glosains.v7i2.670
“This notable increase may be associated with interactive chatbot engagement; according to Deng and Yu (2023), one-way and two-way text message exchanges retain approximately 25% more knowledge.”
Language sense classification model based on neural network
Gu, L., Wang, Y., & Wu, Y. (2023) · Applied and Computational Engineering · doi:10.54254/2755-2721/27/20230297
Pengembangan Buku Ajar Berbasis QR Code Materi Akhlak Terpuji dalam Meningkatkan Sikap Sosial Siswa
Fatwa, H. M., Alnashr, M. S., & Munif, M. (2023) · BASICA Journal of Arts and Science in Primary Education · doi:10.37680/basica.v3i1.2970
SAMANTHA: A chatbot to assist users in training tasks to prevent workplace hazards
Aguilar, D. C., Medina, F., Oyanedel, M., Salamó, M., & Sànchez–Marrè, M. (2024) · International Conference on Human Computer Interaction (Interacción) · doi:10.1145/3657242.3658587 · Cited by 1
“…to obtain timely and efficient assistance, as denoted in [5]; and (iii) It increases user engagement and has a positive impact on the experience and results of online training for employees, as denoted in [9, 22] and in the numerous educational chatbots that are available these days such as [13].”
BackgroundEmotional Intelligence Training Tools in Entrepreneurship Education
Fedorova, Y. V., Pilková, A., Kovalenko, D., Mikuš, J., & Бондаренко, Т. (2024) · Lecture notes in networks and systems · doi:10.1007/978-3-031-72430-5_17
Research Trends In Educational Sciences
Eren, E., Dökme, Ï., Yıldız, G. Ş. G., Çetin, D., Şener, F., Aydoğdu, C., KOÇAK, E., ÇELİK, A. Y., Uluyol, Ç., Eren, E., et al. (2025) · Akademisyen Kitabevi eBooks · doi:10.37609/akya.3415
Peixer, C. G., Nass, F. A., Hirt, F. N., de Moura, G. S., Trainoti, J. D. A. S., Schroeder, K., Wagner, L. A., Becker, L. C., Rodrigues, N. C., & Busnello, V. E. I. (2025) · doi:10.29327/5474955
Raghuwanshi, S., Hasan, A., Sushma, R., Agrawal, R., Singh, A. S., Dubey, N. K., & Kumar, P. (2025) · Data & Metadata · doi:10.56294/dm2025676
Gestaltung eines Chatbot-Playground zur Kompetenzförderung von Hochschullehrenden
Beyer, S., Wrede, S. E., Hoffmann, C., & Schulmann, C. (2025) · Zeitschrift für Hochschulentwicklung · doi:10.21240/zfhe/sh-ki-1/08
Gürbüz, F., & Sarıoğlan, A. B. (2025) · Journal of Education and Future · doi:10.30786/jef.1609883
Case Study on Providing AI Chabot-Based Support Services in E-Learning Platforms
Baneș, V., & Ravariu, C. (2025) · Lecture notes in networks and systems · doi:10.1007/978-3-031-83520-9_56
ПСИХОЛОГИЧЕСКОЕ ВОЗДЕЙСТВИЕ ДИАЛОГОВЫХ ИИ-ТЕХНОЛОГИЙ НА УЧЕБНУЮ МОТИВАЦИЮ СТУДЕНТОВ.Preprint
Лукинова, П. (2025) · doi:10.24108/preprints-3113469
Valente, A. S. O., Warley, M. V., & de Holanda, M. T. (2025) · doi:10.5753/educomp.2025.4904
Cyberpsychology: Validity of the AI Chatbots Usage Scale for University StudentsPreprint
Ibrahim, A., & Nemt-allah, M. A. (2025) · Research Square · doi:10.21203/rs.3.rs-5953281/v1
Karaca, H. (2025) · e-Kafkas Eğitim Araştırmaları Dergisi · doi:10.30900/kafkasegt.1534014
A Sentence‐Level Encoder–Decoder Architecture for Designing an Administrative Roman Urdu Chatbot
Maqbool, M. N., Saleem, R. M., Sarwar, N., Ibrahim, M., & Hashmi, M. S. A. (2025) · Applied Computational Intelligence and Soft Computing · doi:10.1155/acis/4728280
“Te transformative meta-analysis of 32 empirical studies with 2201 participants [18] highlights the signifcant impact of chatbot-assisted learning in sustainable education.”
Alé-Silva, J., & Rodríguez-Donoso, M. (2025) · Research Square · doi:10.21203/rs.3.rs-5954038/v1
Hashmi, N., Intaratat, K., Lomchavakarn, P., & Bansal, A. (2025) · Communications in computer and information science · doi:10.1007/978-3-031-89760-3_6
Anju, M., & Srishti, S. (2025) · INTERNATIONAL JOURNAL OF CREATIVE RESEARCH THOUGHTS · doi:10.56975/ijcrt.v13i5.285539
“It is claimed by Deng and Yu (2023) that while chatbots have a positive and large impact on many aspects of learning, they have no appreciable impact on students' motivation.”
Exploring Prospects of Artificial Intelligence-Based ChatGPT for Higher Education Context
Makgato-Khunou, P., & Tlale, L. D. N. (2025) · Journal of Education and Learning Technology · doi:10.38159/jelt.2025652
The Impact of Chatbot-Driven Artificial Intelligence in Improving Students' Vocabulary
Hutauruk, B. S., Ambarita, P. D. E., & Dhillon, B. P. S. (2025) · Akademika · doi:10.34005/ak.v14i01.4305
Whole-class, high-quality peer tutoring is achievable with minimal effort or expense for teachersPreprint
Reshetov, D. (2025) · OSF Preprints (OSF Preprints) · doi:10.31219/osf.io/me9ku_v1
A Meta-analysis of Pedagogical Agents promoting Self-Regulated Learning
Yin, X., Ye, J., Yu, S., Luo, S., Nan, M., Zhao, G., Xie, Y., & Zheng, L. (2025) · International Conference on Computer Science and Technologies in Education (CSTE) · doi:10.1109/cste64638.2025.11091971
“Deng et al. [20] also found that subject content is not the main factor affecting the effectiveness of PAs.”
Language Learning in the XR Era: Possibilities and Challenges of Immersive Technology
Yasuda, A., & Maruyama, Y. (2025) · Lecture notes in computer science · doi:10.1007/978-3-031-97769-5_18
Mutesasira, G., Makena, B., Kasumba, H., Masha, A. K., Kumanda, N., & Baidoo, J. (2025) · DOAJ (DOAJ: Directory of Open Access Journals) · doi:10.38159/ehass.20256825
Kihwele, J. E., Nicodemus, M. W., & Jember, S. T. (2025) · Future education and learning spaces · doi:10.1007/978-981-96-7937-9_8
Desty, D. S. N., Putra, F. G., & Hidayah, N. (2025) · Journal of Literacy Education · doi:10.64780/jole.v1i2.73
“These results are consistent with Deng & Yu.”
An Analysis of Factors Influencing Chatbots on College Students’ Programming Self-learning Ability
Ran, X., Li, Z., & Yang, Y. (2025) · SAGE Open · doi:10.1177/21582440251375649 · Cited by 2
“Chatbots can easily detect learners’ gaps and respond accordingly, creating meaningful interactions that significantly benefit learners’ performance and memory (Deng & Yu, 2023).”
Kao, L.-C., Cheng, S., Chang, W., Lai, P., & Hsieh, M.-L. (2025) · Clinical Simulation in Nursing · doi:10.1016/j.ecns.2025.101827
Teachers’ Perspectives on Employing Diffit for AI-Assisted EFL Reading Tasks
Nguyen, T. L., & Nguyen, H. L. (2025) · Studies in computational intelligence · doi:10.1007/978-3-032-01348-4_9
AI-Based Chatbot for Personalized Student Assistance
Harini, K., Ankitha, N., Siddhi, A., & Reddy, P. V. (2025) · International Conference on Innovative Mechanisms for Industry Applications · doi:10.1109/icimia67127.2025.11200802 · Cited by 1
“The findings [14] shows that chatbots have a medium-high positive impact on overall learning outcomes.”
Yuen, K. H. K., Yeung, T. H., & Chu, B. (2025) · Innovative Teaching and Learning · doi:10.4208/itl.20250202
“…and are noted for their capacity to provide immediate assistance and access to information, which can significantly enhance learning experiences (Okonkwo & Ade-Ibijola, 2021) and support the development of competence through timely clarification and enhanced knowledge retention (Deng & Yu, 2023).”
An Exploration and Scale Validation on Critical Thinking Engagement in Context of Generative AI
Cornet, A. (2025) · Springer Proceedings in Behavioral & Health Sciences. · doi:10.1007/978-981-95-1936-1_10
Kadam, A. J., Durge, T., Kale, C. A., Kute, P. R., Shelke, P. M., & Deshpande, L. (2025) · Cureus Journal of Computer Science. · doi:10.7759/s44389-024-02275-z
“The SWOC application overcomes these limitations by using data-driven, automated analysis to deliver objective, continuous, and unbiased evaluations based on the most recent performance data [18-19].”
Effects of a Customized Chatbot Intervention on Social Work Students’ Counseling Self-Efficacy
Mo, K. Y., Chow, L., Mok, L. W.-K., & Wang, C. J.-Y. (2025) · Research on Social Work Practice · doi:10.1177/10497315251394310
AIming High: Do Goal Structures Matter in Learning With ChatGPT ?
Schmidt, L., Obergassel, N., & Roelle, J. (2025) · Applied Cognitive Psychology · doi:10.1002/acp.70148
Shen, Z. (2025) · Interactive Learning Environments · doi:10.1080/10494820.2025.2596061
“For example, Pérez et al. (2020) find that chatbot integration in education positively impacts student engagement, learning outcomes, and stress reduction, while Deng and Yu (2023) observe nonsignificant effects of chatbots on critical thinking, learning engagement, and motivation.”
The Impact of Generative AI on Student Identity Formation
Thibaud, E. (2025) · Springer eBooks · doi:10.1007/978-3-031-99804-1_19 · Cited by 1
Designing a scenario-based AI chatbot for concept learning in middle school mathematics
Jung, E. Y., & Ko, H. K. (2025) · The Mathematical Education · doi:10.63311/mathedu.25.6448
Razvi, A. F., & Bashir, E. (2025) · IEEE International Smart Cities Conference (ISC2) · doi:10.1109/isc266238.2025.11293322
“Deng & Yu (2023) [9] Chatbots improve surface-level comprehension but harm critical thinking if used passively.”
Young Adults' Attitudes Toward Generative Artificial Intelligence and Its Role in STEM Education
Çolakoğlu, M., Öncü, S., & Çolak, H. (2025) · Advances in computational intelligence and robotics book series · doi:10.4018/979-8-3373-8805-2.ch007
Use of a Vocational Test Using a Chatbot in the Selection of a University Career
Martínez, M. Á. Q., Rodriguez-Huancayo, J.-D., Castro-Arias, S., & Ríos, M. D. G. (2026) · Lecture notes in networks and systems · doi:10.1007/978-3-031-98890-5_19
Chatbots como mediación didáctica para el fortalecimiento del pensamiento crítico universitario
Arcos, D. A. O., & Obando, A. A. V. (2026) · Arandu-UTIC. · doi:10.69639/arandu.v12i4.1846
Chatbots como mediación didáctica para el fortalecimiento del pensamiento crítico universitario
Arcos, D. A. O., & Obando, A. A. V. (2026) · Arandu-UTIC. · doi:10.69639/arandu.v12i4.1847
Development of Learning Agent for Data Structures on Smart Education Platform
Gong, X. (2025) · International Conference on Artificial Intelligence and Future Education (AIFE) · doi:10.1145/3785987.3786065
“Zhang and Aslan [11] identified personalized learning and intelligent tutoring as research priorities, while Deng and Yu [12] found that chatbot technology demonstrates significant effects in education.”
AI Adoption in Bahraini Higher Education: Challenges and Opportunities for Sustainable Learning
Naser, H., & Saeed, S. (2026) · Lecture notes in networks and systems · doi:10.1007/978-3-032-16389-9_39
Raimbault, T. (2025) · International Conference on Education Technology and Computer · doi:10.1109/icetc66579.2025.11387679
Kaba, A., Shimray, S. R., Zaitoun, E. A., & Refae, G. A. E. (2026) · Open Information Science · doi:10.1515/opis-2025-0038
“Deng and Yu (2023) described that chatbots de fi nitely a ff ect various learning-related aspects; however, they do not considerably improve student motivation.”
Does AI-assisted Learning Improve Students' Learning Outcomes? Evidence From A Meta-analysis
Liu, C., Meng, H., Zhu, H., & Rong, C. (2026) · The Asia-Pacific Education Researcher · doi:10.1007/s40299-026-01104-2 · Cited by 1
Intelligent Whatsapp Chatbot for E Commerce and Learning Support
Sa, A., Avinarasi, S., L, S. K., Primya, T., & Kanagaraj, G. (2026) · International Conference on Communication, Computing and Electronics Systems (ICCCES) · doi:10.1109/iccces62661.2026.11437130
Bosco, M., Kán, P., Wieser, A., Shariattalab, N., Kovačić, I., & Kaufmann, H. (2026) · IEEE International Conference on Artificial Intelligence and eXtended and Virtual Reality (AIxVR) · doi:10.1109/aixvr67263.2026.00025
“Conversational Tutoring Systems (CTSs) have been shown to enhance educational outcomes, improving reasoning, knowledge retention, academic achievement, and learning interest [21], although they may sometimes negatively affect critical thinking, engagement, or motivation.”
Examining the role of chatbots in online learning and student engagement
Saini, S. K., Kaur, R., Saini, R., & Singh, H. (2026) · AIP conference proceedings · doi:10.1063/5.0330769
Vinothkumar, T. S., Basheer, S. N., Murugesan, S., Algadhi, A. A., Albar, N. H. M., Abdulrab, S., Al-Ak’hali, M. S., & Halboub, E. S. (2026) · BMC Oral Health · doi:10.1186/s12903-026-08276-9
“AI is rapidly transforming healthcare, medical, and dental education by enhancing diagnostic accuracy, optimizing treatment plans, and improving student learning outcomes [6, 18, 25].”
InfluentialHuman-AI Collaboration in Entrepreneurship
Pareek, K., Farooqi, A., & Goyal, N. K. (2026) · Advances in computational intelligence and robotics book series · doi:10.4018/979-8-2600-1353-3.ch008
The Role of AI Chatbots in the Evolutionary Culturology Educational Ecosystem
Velikovsky, J. T. (2026) · Advances in computational intelligence and robotics book series · doi:10.4018/408154
AI-Powered Creative Generation and Intelligent Assessment System for Visual Design Education
Jin, L., & Liu, Y. W. (2026) · International Conference on Informatics Education and Computer Technology Applications (IECA) · doi:10.1145/3802133.3802236
Kellam, H., Cortés, L. E. P., Dange, A., & Sterling, K. (2026) · doi:10.2196/preprints.100513
Artificial Intelligence in Preschool Education and Primary Education
Λαμπρόπουλος, Γ., & Papadakis, S. (2026) · Springer eBooks · doi:10.1007/978-3-032-17190-0_18
Salazar, A. P., & Raga, R. C. (2025) · International Conference on Education and E-Learning (ICEEL) · doi:10.1145/3789859.3789928
“Deng and Yu (2023) however, executed a meta-analysis that looked into findings from multiple chatbot technology studies and here is how they assessed the overall utility accruing to student learning across different subject domains.”
Improving Chatbot Usability Through Structured Prompt-Based Interaction Design
Guacho, G. K. B., Martínez, E. P. A., Martínez, M. A. G., & Vallejo, P. X. M. (2026) · Multimodal Technologies and Interaction · doi:10.3390/mti10060060
“Deng and Yu [10] conducted a meta-analysis of the effect of chatbot technology in sustainable education, synthesizing results from multiple quantitative studies.”
Hamza-Jamann, A., & Lemoine, V. (2026) · Recherches en didactiques · doi:10.3917/rdid1.041.0113
AI-Powered tools for Learning Content Discovery and Student Reflective Scaffolding in MOOCs
Martín, S., Rodríguez‐Artacho, M., Castro, M. A., Sancristobal, E., & Márquez-Barja, J. M. (2026) · IEEE Global Engineering Education Conference (EDUCON) · doi:10.1109/educon67543.2026.11574062
“…the use of Retrieval-augmented generation (RAG) to foster authoring and gathering of learning content in MOOCS along with an AI-based companion called Socratic Chatbot to guide students to reinforce knowledge acquisition, as long as to provide a constructivist scenario for the learning path [4].”
Perspectives on AI-Enabled Education for Sustainable Development
Chand, A., Sarthi, S., Aanchal, & R., G. C. (2026) · Springer eBooks · doi:10.1007/978-3-032-28217-0_11
Fan, L., & Li, Z. (2026) · SAGE Open · doi:10.1177/21582440261420472
“Positive learning experiences and emotions encourage proactive student engagement and efficient resource allocation, improving academic ability, while negative emotions may distract students and undermine performance (Deng & Yu, 2023).”
Tzagkaraki, E., Papadakis, S., & Kalogiannakis, M. (2026) · Discover Education · doi:10.1007/s44217-026-01558-y · Cited by 1
“In the field of STEM subjects, and in particular in Mathematics and Natural Sciences, Chatbots have been used to facilitate the understanding of concepts, guide problem solving and enhance learning engagement [16, 32].”
Mahyiyuddin, A. A., & Erlita (2026) · Research Square · doi:10.21203/rs.3.rs-10221208/v1
The Speed Factor: Designing a Training Chatbot for Spoken Technical Texts
Chatzipanagiotidis, S., Valavani, C., & Alexandris, C. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-29596-5_27
Laachach, A., Farissi, N., Laaraj, N., & Mumtaz, S. (2026) · Journal of Tourism Futures · doi:10.1108/jtf-11-2024-0246
Chatbot-Supported Implementation on Pre-Service English Language Teachers' Academic Speaking Skills
BADAK, E. F., & Ersanlı, C. Y. (2026) · Elt research journal · doi:10.71362/eltrj.1865412
Azzubaidi, K., & Alhamidani, M. (2026) · Frontiers in Psychology · doi:10.3389/fpsyg.2026.1861868 · Cited by 1
Zhorabekova, A., Kurmasheva, M., Yesnazar, A., & Nuralieva, A. (2026) · Frontiers in Education · doi:10.3389/feduc.2026.1902394
Castro, R. A. G., Asté, W. M. B., Aratia, G. P., Cruz, K. M. L.-D. L., Huanacuni, L. E. A., Ancoa, N. Z., & Romero, M. A. (2026) · doi:10.59899/dis-met
Cao, L., Chau, K. T., & Yahaya, W. A. J. W. (2026) · Journal of Educational Computing Research · doi:10.1177/07356331261469058
Enhancing learning from video lectures with proactive AI chatbots
Pi, Z., Zhou, D., Wei, Y., Zheng, S., & Li, X. (2026) · Education and Information Technologies · doi:10.1007/s10639-026-14081-2 · Cited by 1
“A large body of literature has focused on the roles, challenges, and opportunities of AI chatbots (Deng & Yu, 2023 ; Kuhail et al., 2023 ).”
InfluentialLee, J., & Kim, B. (2026) · Applied Nursing Research · doi:10.1016/j.apnr.2026.152133
Win, K. Z. H., Khadka, A., & Win, T. Z. H. (2026) · BMC Medical Education · doi:10.1186/s12909-026-10017-z
“This high rate of intended continued use is consistent with surveys of undergraduate and postgraduate students in other contexts, where ease of access and perceived utility are consistently identified as primary drivers of AI adoption [16, 19–21].”
AhBot - A GenAI Solution to support Flipped Learning
Chia, T. C., Foong, R., Gu, W., & Tan, W. Y. (2026) · International Conference on Information Technology and Education Technology (ITET) · doi:10.1109/itet70052.2026.11650898
“However, there are also negative findings in critical thinking, learning engagement, and motivation [12].”
InfluentialExamining student perceptions of custom GPT Chatbots to support introductory physics instruction
Young, N. T., Freeman, J., Balasubramaniam, S., & Weliweriya, N. (2026) · ASEE Annual Conference & Exposition · doi:10.18260/1-2--59642
Weinhandl, R., & Große, C. S. (2026) · Computers in Human Behavior Reports · doi:10.1016/j.chbr.2026.101275
Lam, A. L.-K. (2026) · TESOL Quarterly · doi:10.1002/tesq.70216
Visualizing the Knowledge Mapping of Chatbots in Education: A Bibliometric Analysis in CiteSpace
Ling, K.-H., & Zhang, S. (2026) · International Conference on Education and Training Technologies (ICETT) · doi:10.1109/icett70124.2026.11666085
“Deng et al. [25] further pointed out that chatbots have a moderate to high impact on overall learning outcomes, particularly in improving explicit reasoning, learning achievement, knowledge retention, and learning interest.”
Wie belastbar ist die KI-Bildungsforschung?
Friederichs, H. (2026) · doi:10.59350/twbre-sbr89
Wie belastbar ist die KI-Bildungsforschung?
Friederichs, H. (2026) · doi:10.59350/p0n7d-1qe84
Jamirsah, N., Yusof, I. J., Awang, A., Idris, N. H., & Azmy, S. N. (2026) · ALAM CIPTA International Journal Of Sustainable Tropical Design & Practice · doi:10.47836/ac.19.2.paper08
Agentic Generative AI in Higher Education: Perceived Benefits, Risks, and Implications for Learning
Ilieva, G., Yankova, T., Ruseva, M., Klisarova-Belcheva, S., Georgiev, P., & Totkov, G. (2026) · Trends in Higher Education · doi:10.3390/higheredu5040108
“Systematic reviews reported advantages in availability, response speed, and learner support, but also identified narrow knowledge domains, limited conversational flexibility, inconsistent pedagogical grounding, and a strong reliance on satisfaction-based evaluation measures [4,11,12].”
Kajan, K., Shi, W. Y., & Wanatowski, D. (2026) · UK-China Teaching, Learning, and Educational Technology (TILE-TEC) · doi:10.1109/tile-tec70450.2026.11682164
“Nevertheless, they highlight recurring risks: inaccurate outputs, overreliance, and reduced independent thinking [5], [6] – [9].”
Kannan, S., Khambari, M. N. M., Syed-Abdullah, S. I. S., & Mahat, J. (2026) · Pertanika Journal of Vocational Science and Technology Education · doi:10.47836/pjvste.2.1.02
Digital irresponsibility in the gig economy: insights from a mixed-methods study
Prentice, C., Gautam, N., & Zeidan, S. (2026) · Journal of Enterprise Information Management · doi:10.1108/jeim-02-2026-0271
Öksüz, H. İ., Afyoncu, T., Gök, S., Yıldırım, T., & Yıldırım, K. (2026) · Interactive Learning Environments · doi:10.1080/10494820.2026.2734314
Ayberkin, D., & Yirik, E. (2026) · Discover Education · doi:10.1007/s44217-026-02226-x
“Chatbots can sustain natural-language conversations, generate contextual explanations, provide immediate feedback, and adapt response complexity to user needs [6, 11].”
Xiaotian, R., Li, X., Gao, L., Wei, Q., Liu, H., & Yu, X. (2026) · Educational Psychology Review · doi:10.1007/s10648-026-10217-5
“In addition, the reference lists of all included studies and relevant systematic reviews of AI interventions were manually screened (Deng & Yu, 2023; Dong et al., 2025; Zhai et al., 2024 ).”
Marsico, E., Barbieri, U., Filippone, A., Fuccio, R. D., & Carruba, M. C. (2026) · Communications in computer and information science · doi:10.1007/978-3-032-31850-3_10
Torab-Miandoab, A., Ghafourifard, M., Habibi-Chenaran, S., & Maghalian, M. (2026) · Journal of Medical Education and Curricular Development · doi:10.1177/23821205261491865
Yujra, E., Vargas, C. A. A., Mendiguri, J. A. N., & Choque, L. D. S. (2026) · Horizontes Revista de Investigación en Ciencias de la Educación · doi:10.33996/revistahorizontes.v10i44.1328
AI Chatbot-Assisted Teaching and Student Learning Outcomes
Yang, G. (2026) · International Journal of Web-Based Learning and Teaching Technologies · doi:10.4018/ijwltt.426375
Metadata
- Authors
- Deng, X., & Yu, Z.
- Affiliations
- Beijing Language and Culture University (CN)
- Corresponding author
- Yu, Zhonggen
- Venue
- Sustainability
- Publisher
- MDPI AG
- Published
- 2023-02-06
- Volume
- 15
- Issue
- 4
- Pages
- 2940
- ISSN
- 2071-1050 (ISSN-L)
- OpenAlex
- W4319440682
- Semantic Scholar
- 27f4d355b27a6878332a87845a6ff13940fe9158
- Cited by
- 298
- References
- 64
- Citations by year
- 2026: 76 · 2025: 115 · 2024: 72 · 2023: 35
- FWCI
- 12.28
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- Locations
- Sustainability (publishedVersion, cc-by)
- RePEc: Research Papers in Economics (submittedVersion)
- Funders
- 2019 MOOC of Beijing Language and Culture University (MOOC201902) (important) “Introduction to Linguistics” (202010032003, SJTS202108, 22YCX017)
- “Introduction to Linguistics” of online and offline mixed courses in Beijing Language and Culture University in 2020 (202010032003, SJTS202108, 22YCX017)
- Beijing Co-construction Project Research and reform of the “Undergraduate Teaching Reform and Innovation Project” of Beijing higher education in 2020-innovative “multilingual +” excellent talent training system (202010032003, SJTS202108, 22YCX017)
- The research project of Graduate Students of Beijing Language and Culture University “Xi Jinping: The Governance of China” (202010032003, SJTS202108, 22YCX017)
- Beijing Language and Culture University (202010032003, SJTS202108, 22YCX017)
- Topics
- Social Sciences › Education › Artificial Intelligence in Education
- Physical Sciences › Computer Science › Artificial Intelligence › Intelligent Tutoring Systems and Adaptive Learning
- Social Sciences › Business, Management and Accounting › Marketing › AI in Service Interactions
- Keywords
- chatbots, chatbot-assisted learning, sustainable education, learning outcomes, moderators, explicit reasoning, knowledge retention, academic performance, student learning interest, critical thinking, student engagement, student motivation, artificial intelligence in education
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
A language model drew the claims from the paper on 2026-10-06, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-10-06 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
