The DOI matches its Crossref record; 63 of the 64 references it cites were found in a registry or an index; none of the 35 sources its claims cite is checked for support yet.
A Meta-Analysis and Systematic Review of the Effect of Chatbot Technology Use in Sustainable Education
Open accessAbstract
The development of artificial intelligence in recent years has promoted the use of chatbot technology in sustainable education. Many studies examined the effect of chatbots on learning outcomes. However, scant studies summarized the effectiveness of chatbots in education. The aim of the study is to investigate the effect of chatbot-assisted learning on various components and how different moderator variables influenced the effectiveness. This study, through a meta-analysis, reviewed 32 empirical studies with 2201 participants published between 2010 and 2022. The results showed that chatbot technology exerted a medium-to-high effect on overall learning outcomes regardless of moderator variables, i.e., intervention duration, chatbot roles, and learning content. In terms of learning components, chatbots could significantly improve explicit reasoning, learning achievement, knowledge retention, and learning interest despite negative findings in critical thinking, learning engagement, and motivation. Future research could expand chatbot research by including different learning components.
Overview
AI snapshot
Written from the paper’s full text.
TL;DR
This meta-analysis and systematic review examined 32 empirical studies with 2201 participants and found that chatbot technology had a medium-to-large positive overall effect on learning outcomes (d = 0.789). Effects were significant for explicit reasoning, learning achievement, knowledge retention, and learning interest, but not for critical thinking, learning engagement, or motivation.
Key findings
- The overall effect size of chatbot use in education was d = 0.789 (95% CI [0.593, 0.985], p < 0.001), described as medium-to-large.
- Chatbots significantly improved explicit reasoning (d = 1.190, p < 0.001), learning achievement (d = 1.033, p < 0.001), knowledge retention (d = 0.691, p = 0.022), and learning interest (d = 0.842, p = 0.041).
- Chatbots did not significantly improve critical thinking (d = 0.690, p = 0.144), learning engagement (d = 0.147, p = 0.114), or learning motivation (d = 0.409, p = 0.114).
- Moderator analyses found no statistically significant differences by intervention duration, chatbot roles, or learning content (all p > 0.05), though subgroup effect sizes varied.
Methods
- The authors conducted a PRISMA-guided meta-analysis and systematic review, searching major databases including Web of Science, Wiley Online Library, Springer Link, Taylor & Francis Online, Elsevier ScienceDirect, and Google Scholar on 3 September 2022.
- They included 32 empirical studies published between 2010 and 2022 with 2201 participants, calculating 76 effect sizes using Cohen’s d; studies needed experimental/control comparisons and sufficient statistics.
- Analyses used Stata MP/14.0, with publication-bias checks (funnel plot, Begg’s test, Egger’s test), heterogeneity tests (I²), sensitivity analysis, and meta-regression for intervention duration, chatbot roles, and learning content.
Limitations
- The paper states that only studies written in English were included, that only three moderator variables were examined, and that unpublished studies with insignificant results may still influence the findings despite no detected publication bias.
- Evident from the text: several outcome domains had high heterogeneity (e.g., critical thinking I² = 92.8%, motivation I² = 93.2%, overall I² = 87.8%), and some categories had very few studies, limiting moderator conclusions.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
34 Not checked yet · 18 Own result · 10 No basis given · 8 Own interpretation · 5 Own method
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The meta-regression results revealed no statistically significant differences between the abovementioned values.”
§ A Meta-Analysis and Systematic Review of the Effect of Chatbot Technology Use in Sustainable Education > 4.3. Results of Moderator Analyses
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“No significant differences in learning engagement (d = 0.147, 95% CI [-0.068, 0.363], p = 0.114) were found between chatbot-assisted and traditional learning methods.”
§ A Meta-Analysis and Systematic Review of the Effect of Chatbot Technology Use in Sustainable Education > 4.3. Results of Moderator Analyses
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“However, there were no significant differences between effect sizes of the different intervention durations (p > 0.05).”
§ A Meta-Analysis and Systematic Review of the Effect of Chatbot Technology Use in Sustainable Education > 4.3. Results of Moderator Analyses
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Table 4 shows a significant difference in explicit reasoning (d = 1.190, 95% CI [0.868, 1.512], p < 0.001) between the experimental and control groups.”
§ A Meta-Analysis and Systematic Review of the Effect of Chatbot Technology Use in Sustainable Education > 4.3. Results of Moderator Analyses
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The effect size did not show a significant difference in critical thinking (d = 0.690, 95% CI [-0.235, 1.615], p = 0.144) between the chatbot-assisted and conventional learning methods.”
§ A Meta-Analysis and Systematic Review of the Effect of Chatbot Technology Use in Sustainable Education > 4.3. Results of Moderator Analyses
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“and retention (d = 0.691, 95% CI [0.101, 1.281], p = 0.022) in the experimental group significantly improved compared with the control group.”
§ A Meta-Analysis and Systematic Review of the Effect of Chatbot Technology Use in Sustainable Education > 4.3. Results of Moderator Analyses
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“the experimental group maintained significantly higher levels of learning interest (d = 0.842, 95% CI [0.034, 1.650], p = 0.041).”
§ A Meta-Analysis and Systematic Review of the Effect of Chatbot Technology Use in Sustainable Education > 4.3. Results of Moderator Analyses
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The overall effect size of using chatbots in education was 0.789 (p < 0.001), with a 95% confidence interval between 0.593 and 0.985 (Table 4).”
§ A Meta-Analysis and Systematic Review of the Effect of Chatbot Technology Use in Sustainable Education > 4.3. Results of Moderator Analyses
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“However, learning motivation in both groups was not significantly different (d = 0.409, 95% CI [-0.099, 0.916], p = 0.114).”
§ A Meta-Analysis and Systematic Review of the Effect of Chatbot Technology Use in Sustainable Education > 4.3. Results of Moderator Analyses
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Additionally, the heterogeneity test showed that the overall effect size (Q = 582.09,I 2 = 87.8%,p < 0.001) was of considerable heterogeneity”
§ A Meta-Analysis and Systematic Review of the Effect of Chatbot Technology Use in Sustainable Education > 4.3. Results of Moderator Analyses
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- It may state two separate claims as one. 23
- Probably not a substantial claim. 20
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“However, Begg's test showed a presence of publication bias in learning interest (z = 2.04, p = 0.042), which was different from the result of Egger's test (t = 15.30,p = 0.056).”
§ A Meta-Analysis and Systematic Review of the Effect of Chatbot Technology Use in Sustainable Education > 4.1. Analyses of Publication Bias
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The researchers calculated 76 effect sizes with a total sample size of 2201 in 32 identified studies.”
§ A Meta-Analysis and Systematic Review of the Effect of Chatbot Technology Use in Sustainable Education > 3.4. Data Analysis
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“It indicates that none of the included studies could influence the pooled effect size when a specific study is omitted, confirming that the meta-analytical results are robust.”
§ A Meta-Analysis and Systematic Review of the Effect of Chatbot Technology Use in Sustainable Education > 4.2. Results of Sensitivity Analysis
hedged- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
In the paper
“They strictly followed the Preferred Reporting Items for Systematic reviews and Meta-Analyses (PRISMA) guidelines [54].”
§ A Meta-Analysis and Systematic Review of the Effect of Chatbot Technology Use in Sustainable Education > 3. Materials and Methods
What the sources say
- Not checked yetcites [54] Page, 2021, The PRISMA 2020 statement: An updated guideline for reporting systematic reviews, Br. Med. J. source
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The overall effect size in this study was thus considered as a mediumto-large level, indicating that the use of chatbot technology exerted a positive effect on learning outcomes.”
§ A Meta-Analysis and Systematic Review of the Effect of Chatbot Technology Use in Sustainable Education > 4.3. Results of Moderator Analyses
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
In the paper
“one display of explicit reasoning could associate with another one, which was the core of transactivity theory [70].”
§ A Meta-Analysis and Systematic Review of the Effect of Chatbot Technology Use in Sustainable Education > 5. Discussion
What the sources say
- Not checked yetcites [70] Pinkwart, N., and McLaren, B. (2012). Educational Technologies for Teaching Argumentation Skills, Bentham Science Publishers. Rosé, C.P. source
hedgedcorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim.
In the paper
“Chatbots also increased college students' interest in English vocabulary learning [47].”
§ A Meta-Analysis and Systematic Review of the Effect of Chatbot Technology Use in Sustainable Education > H6.
What the sources say
- Not checked yetcites [47] Kim, 2018, Chatbots and Korean EFL students’ English vocabulary learning, J. Digit. Converg.
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“Methodologically, this study included major databases and conducted a meta-analysis under the PRISMA guidelines to examine the effectiveness of chatbot technology on educational outcomes.”
§ A Meta-Analysis and Systematic Review of the Effect of Chatbot Technology Use in Sustainable Education > 6.1. Major Contributions
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
In the paper
“Chatbots could improve Chinese students' thinking ability and facilitate interactive learning [13].”
§ A Meta-Analysis and Systematic Review of the Effect of Chatbot Technology Use in Sustainable Education > 1. Introduction
What the sources say
- Not checked yetcites [13] Li, 2022, Al intelligence chatbot to improve students learning in the higher education platform, J. Interconnect. Netw. source
hedgedconditional- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Chatbots also asked for clear and convincing statements, motivating students to find strong evidence.”
§ A Meta-Analysis and Systematic Review of the Effect of Chatbot Technology Use in Sustainable Education > 5. Discussion
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
In the paper
“The LINE Bot used in military science could provide procedures for engine fan module decomposition, which improved trainee performance and reduced training costs [30].”
§ A Meta-Analysis and Systematic Review of the Effect of Chatbot Technology Use in Sustainable Education > 2.7. Learning Content
What the sources say
- Not checked yetcites [30] (2022, September 08). Development of Mobile Interactive Courses Based on an Artificial Intelligence Chatbot on the Communication Software LINE. Available online: https://doi.org/10.1080/10494820.2021.1937230. source
hedgeddirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.
In the paper
“The AsasaraBot, acting as a tutor, provided questions, encouragement, and interactions, aiming to support students' language learning [15].”
§ A Meta-Analysis and Systematic Review of the Effect of Chatbot Technology Use in Sustainable Education > 2.6. Chatbot Roles
What the sources say
- Not checked yetcites [15] Mageira, K., Pittou, D., Papasalouros, A., Kotis, K., Zangogianni, P., and Daradoumis, A. (2022). Educational AI chatbots for content and language integrated learning. Appl. Sci.-Basel, 12. source
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
In the paper
“A narrative-based learning system equipped with chatbot feedback significantly improved users' engagement [37].”
§ A Meta-Analysis and Systematic Review of the Effect of Chatbot Technology Use in Sustainable Education > 2.4. Learners' Engagement, Motivation, and Interest
What the sources say
- Not checked yetcites [37] Ruan, S., He, J.Y., Ying, R., Burkle, J., Hakim, D., Wang, A., Yin, Y.F., Zhou, L., Xu, Q.Y., and AbuHashem, A. (2020, January 21–24). Supporting Children’s Math Learning with Feedback-Augmented Narrative Technology. Proceedings of the ACM Interaction Design and Children (IDC) Conference, London, UK. source
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
In the paper
“University students created conversations with the chatbot Elbot ranging from school life to movies.After eight weeks, the experimental group better acquired vocabulary knowledge than the control group [47].”
§ A Meta-Analysis and Systematic Review of the Effect of Chatbot Technology Use in Sustainable Education > 2.6. Chatbot Roles
What the sources say
- Not checked yetcites [47] Kim, 2018, Chatbots and Korean EFL students’ English vocabulary learning, J. Digit. Converg.
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.
In the paper
“For example, Chen et al. [50] examined the effects of mobile devices on language learning across different intervention durations.”
§ A Meta-Analysis and Systematic Review of the Effect of Chatbot Technology Use in Sustainable Education > 2.5. Intervention Duration
What the sources say
- Not checked yetcites [50] Chen, 2020, The effects of using mobile devices on language learning: A meta-analysis, Educ. Technol. Res. Dev. source
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
In the paper
“Chatbots improved Korean students' interest in foreign language learning [43].”
§ A Meta-Analysis and Systematic Review of the Effect of Chatbot Technology Use in Sustainable Education > H6.
What the sources say
- Not checked yetcites [43] Han, 2020, The effects of voice-based AI chatbots on Korean EFL middle school students’ speaking competence and affective domains, Asia-Pac. J. Converg. Res. Interchange source
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“Given the benefits and concerns of educational chatbot use, many studies have measured the effectiveness of chatbots but have obtained inconsistent findings.”
§ A Meta-Analysis and Systematic Review of the Effect of Chatbot Technology Use in Sustainable Education > 1. Introduction
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. It may state two separate claims as one.
In the paper
“In nursing education, students used chatbot systems to learn courses about the physical examination, effectively enhancing students' academic performance [53].”
§ A Meta-Analysis and Systematic Review of the Effect of Chatbot Technology Use in Sustainable Education > 2.7. Learning Content
What the sources say
- Not checked yetcites [53] Chang, 2022, Chatbot-facilitated nursing education: Incorporating a knowledge-based chatbot system into a nursing training program, Educ. Technol. Soc.
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
In the paper
“The dynamic assessment of chatbots was effective in enhancing vocabulary retention and providing detailed information about personal learning [2].”
§ A Meta-Analysis and Systematic Review of the Effect of Chatbot Technology Use in Sustainable Education > 2.3. Learning Achievement and Knowledge Retention
What the sources say
- Not checked yetcites [2] (2022, September 08). Chatbot-Assisted Dynamic Assessment (CA-DA) for L2 Vocabulary Learning and Diagnosis. Available online: https://doi.org/10.1080/09588221.2021.1987272. source
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
In the paper
“Chatbot interventions could effectively stimulate group discussions and help students utter their thoughts [55].”
§ A Meta-Analysis and Systematic Review of the Effect of Chatbot Technology Use in Sustainable Education > 5. Discussion
What the sources say
- Not checked yetcites [55] Tegos, 2015, Promoting academically productive talk with conversational agent interventions in collaborative learning settings, Comput. Educ. source
hedgedconditional- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
In the paper
“Moreover, explicit reasoning could increase collaboration practices, improve learning outcomes, and promote conceptual knowledge acquisition [27].”
§ A Meta-Analysis and Systematic Review of the Effect of Chatbot Technology Use in Sustainable Education > 2.2. Explicit Reasoning
What the sources say
- Not checked yetcites [27] Papadopoulos, 2013, ‘Make it explicit!’: Improving collaboration through increase of script coercion, J. Comput. Assist. Learn. source
hedgedconditional- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. It may state two separate claims as one.
In the paper
“The meta-analysis of robot-assisted language learning [52] examined the influence of robot roles on the effectiveness of social robots.”
§ A Meta-Analysis and Systematic Review of the Effect of Chatbot Technology Use in Sustainable Education > 2.6. Chatbot Roles
What the sources say
- Not checked yetcites [52] Lee, 2022, The effects of robot-assisted language learning: A meta-analysis, Educ. Res. Rev. source
plaincorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
In the paper
“The artificial intelligence-integrated chatbot was proved effective to enhance students' thinking ability and expectations [13].”
§ A Meta-Analysis and Systematic Review of the Effect of Chatbot Technology Use in Sustainable Education > 2.1. Critical Thinking
What the sources say
- Not checked yetcites [13] Li, 2022, Al intelligence chatbot to improve students learning in the higher education platform, J. Interconnect. Netw. source
boosteddirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
In the paper
“Fifteen instructional sessions carried out over three weeks facilitated students' comprehension of English adjectival constructions but failed to help learners generate prepositional constructions [48].”
§ A Meta-Analysis and Systematic Review of the Effect of Chatbot Technology Use in Sustainable Education > 2.5. Intervention Duration
What the sources say
- Not checked yetcites [48] Kim, R. (2022). Effects of AI Chatbot-Based Instruction on the Learning of English Adjectival Transitive Resultative Construction by Korean High School Students. [Ph.D. Thesis, Seoul National University]. Available online: https://hdl.handle.net/10371/181242.
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.
In the paper
“Similarly, nursing college students using chatbots experienced a higher level of learning interest than their peers in the control group [46].”
§ A Meta-Analysis and Systematic Review of the Effect of Chatbot Technology Use in Sustainable Education > H6.
What the sources say
- Not checked yetcites [46] Han, J.W., Park, J., and Lee, H.N. (2022). Analysis of the effect of an artificial intelligence chatbot educational program on non-face-to-face classes: A quasi-experimental study. BMC Med. Educ., 22. source
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
In the paper
“The novelty effect may appear when students are introduced to new technology.The improvement of learning outcomes might result from students' newness to chatbot technology [8].”
§ A Meta-Analysis and Systematic Review of the Effect of Chatbot Technology Use in Sustainable Education > 1. Introduction
What the sources say
- Not checked yetcites [8] Fryer, 2017, Stimulating and sustaining interest in a language course: An experimental comparison of Chatbot and Human task partners, Comput. Hum. Behav. source
hedgedconditional- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Thus, the conversational behaviors in the chatbot group were considered more transactive, i.e., with more explicit arguments, than those in the control group.”
§ A Meta-Analysis and Systematic Review of the Effect of Chatbot Technology Use in Sustainable Education > 5. Discussion
plaincorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
In the paper
“Review activities before presenting new information activate students' prior knowledge, facilitating the integration of the new and old knowledge [57].”
§ A Meta-Analysis and Systematic Review of the Effect of Chatbot Technology Use in Sustainable Education > 5. Discussion
What the sources say
- Not checked yetcites [57] Lee, 2022, Impacts of an AI-based chatbot on college students’ after-class review, academic performance, self-efficacy, learning attitude, and motivation, Educ. Technol. Res. Dev. source
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“Since the control group in the included studies did not obtain access to chatbots, it was difficult to compare users' attitudes between the control and intervention groups.”
§ A Meta-Analysis and Systematic Review of the Effect of Chatbot Technology Use in Sustainable Education > 6.3. Implications for Future Research
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
In the paper
“Specifically, pressure was a significant predictor of motivation in the chatbot-assisted learning context, whereas perceived competency was an influencing factor in the traditional context [18].”
§ A Meta-Analysis and Systematic Review of the Effect of Chatbot Technology Use in Sustainable Education > 5. Discussion
What the sources say
- Not checked yetcites [18] Yin, 2021, Conversation technology with micro-learning: The impact of chatbot-based learning on students’ learning motivation and performance, J. Educ. Comput. Res. source
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“First, the researchers only included studies written in English.”
§ A Meta-Analysis and Systematic Review of the Effect of Chatbot Technology Use in Sustainable Education > 6.2. Limitations
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“Second, this study only included three moderator variables that did not significantly influence the effectiveness of chatbots.”
§ A Meta-Analysis and Systematic Review of the Effect of Chatbot Technology Use in Sustainable Education > 6.2. Limitations
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“Third, the results may still be influenced by unpublished studies with insignificant results despite the absence of publication bias.”
§ A Meta-Analysis and Systematic Review of the Effect of Chatbot Technology Use in Sustainable Education > 6.2. Limitations
hedgedconditional- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. The paper may give no reason why it holds.
In the paper
“Computer science and language were the most targeted fields in chatbot-based education, while engineering and mathematics received less attention [12].”
§ A Meta-Analysis and Systematic Review of the Effect of Chatbot Technology Use in Sustainable Education > 5. Discussion
What the sources say
- Not checked yetcites [12] (2022, September 08). Interacting with Educational Chatbots: A Systematic Review. Available online: https://doi.org/10.1007/s10639-022-11177-3. source
plaincorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
The paper relies on this without support.
The included studies had sufficient statistical power to detect effects on learning engagement and motivation.
“However, significant differences in learning engagement and motivation were not found between the chatbot-based condition and the control condition.”
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“However, the study also highlights the constraints of ChatGPT, particularly its potential to limit creativity and critical thinking, which aligns with the findings of Deng and Yu (2023) but contrasts with Abramson's (2023) view that AI can enhance creativity by offloading routine tasks.”
From Motivations and Affordances of ChatGPT Usage for College Students’ Learning (2025) · cites this paper as Deng and Yu (2023)
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“This notable increase may be associated with interactive chatbot engagement; according to Deng and Yu (2023), one-way and two-way text message exchanges retain approximately 25% more knowledge.”
From SiKurang: Development and Field Evaluation of an Offline-First mHealth System for Community-Based Stunting Risk Detection and Counselling in Low-Connectivity Primary Care Settings (2026) · cites this paper as Deng and Yu (2023)
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“according to Deng and Yu (2023), one-way and two-way text message exchanges retain approximately 25% more knowledge.”
From SiKurang: Development and Field Evaluation of an Offline-First mHealth System for Community-Based Stunting Risk Detection and Counselling in Low-Connectivity Primary Care Settings (2026) · cites this paper as Deng and Yu (2023)
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“This aligns with a meta-analysis conducted by Deng and Yu [31], which indicated that chatbot programs can serve three roles in education—teaching assistant, personal tutor, and learning partner (Fig. 2).”
From Development and effects of a chatbot education program for self-directed learning in nursing students (2025) · cites this paper as [31]
In the citing paper
“AI tools, such as chatbots, have been shown to enhance student interaction and improve learning outcomes (Deng & Yu, 2023;Rasul et al., 2023).”
From Ethical use of artificial intelligence based tools in higher education: Are future business leaders ready? (2025) · cites this paper as Deng & Yu, 2023
In the citing paper
“chatbots, including ChatGPT, have demonstrated the potential to reshape and revolutionize education, creating new, novel opportunities for enhancing the learning process and outcomes [83], facilitating different learning approaches, and offering a range of pedagogical benefits [19, 43, 72]”
From A systematic literature review of empirical research on ChatGPT in education (2024) · cites this paper as [19]
In the citing paper
“chatbots, including ChatGPT, have the potential to significantly impact students' thinking abilities, including retention, reasoning, analysis skills [19, 45], and foster innovation and creativity capabilities [83]”
From A systematic literature review of empirical research on ChatGPT in education (2024) · cites this paper as [19]
Statistics recomputed
Every reported test result, recomputed from its test statistic and degrees of freedom. A result shows as not adding up only once an independent check has confirmed it.
- Consistentz = 1.47reported p = .142 · recomputed p = .142
“z = 1.47, p = .142”
- Consistentz = 1.32reported p = .188 · recomputed p = .187
“z = 1.32, p = .188”
- Consistentz = 0.53reported p = .598 · recomputed p = .596
“z = 0.53, p = .598”
- Consistentz = 1.35reported p = .176 · recomputed p = .177
“z = 1.35, p = .176”
- Consistentz = 1.23reported p = .217 · recomputed p = .219
“z = 1.23, p = .217”
- Consistentz = 2.04reported p = .042 · recomputed p = .041
“(z = 2.04, p = .042”
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
Each contradiction between two passages listed here was confirmed by an independent check.
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
“| 8 | H. L. Chen et al. (2020) [17] | 58 (19/29) | 4 weeks | Tutor | Language | Learning achievement and retention |”
§ A Meta-Analysis and Systematic Review of the Effect of Chatbot Technology Use in Sustainable Education > 6.3. Implications for Future Research
Reported 58 · recomputed 48
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ A Meta-Analysis and Systematic Review of the Effect of Chatbot Technology Use in Sustainable Education > 4.3. Results of Moderator Analyses
“No significant differences in learning engagement (d = 0.147, 95% CI [-0.068, 0.363], p = 0.114) were found between chatbot-assisted and traditional learning methods.”
§ A Meta-Analysis and Systematic Review of the Effect of Chatbot Technology Use in Sustainable Education > 6.3. Implications for Future Research
“| 5 | Learning engagement | 0.147 | -0.068, 0.363 | 1.34 | 0.180 | 3.22 | 6 | 0.780 | 0.0% |”
Citations
Cites
The PRISMA 2020 statement: An updated guideline for reporting systematic reviews
Page, M. J., McKenzie, J. E., Bossuyt, P. M., Boutron, I., Hoffmann, T. C., Mulrow, C. D., Shamseer, L., Tetzlaff, J. M., Akl, E. A., Brennan, S. E., et al. (2021) · BMJ · doi:10.1136/bmj.n71 · Cited by 108,283
Measuring inconsistency in meta-analyses
Higgins, J. P. T., Thompson, S. G., Deeks, J. J., & Altman, D. G. (2003) · BMJ · doi:10.1136/bmj.327.7414.557 · Cited by 63,139 · cited 1×
“The heterogeneity will be considered low if I2 is less than 25%, moderate if I2 falls between 25% and 75%, and substantial if I2 is greater than 75% [68].”
BackgroundIntrinsic and Extrinsic Motivations: Classic Definitions and New Directions
Ryan, R. M., & Deci, E. L. (2000) · Contemporary Educational Psychology · doi:10.1006/ceps.1999.1020 · Cited by 18,666 · cited 1×
“Intrinsic motivation focuses on learners’ inner satisfaction, whereas extrinsic motivation is defined as the behavior for external and separable results [42].”
BackgroundToward a hierarchical model of intrinsic and extrinsic motivation
Vallerand, R. J. (1997) · Advances in experimental social psychology · doi:10.1016/s0065-2601(08)60019-2 · Cited by 3,198 · cited 1×
“Motivation can be categorized into intrinsic and extrinsic motivations [41].”
BackgroundChatbots applications in education: A systematic review
Okonkwo, C. W., & Ade-Ibijola, A. (2021) · Computers and Education: Artificial Intelligence · doi:10.1016/j.caeai.2021.100033 · Cited by 879 · cited 3×
“[1], Okonkwo, and Ade-Ibijola [6], Perez et al.”
BackgroundInteracting with educational chatbots: A systematic review
Kuhail, M. A., Alturki, N., Alramlawi, S., & Alhejori, K. (2023) · Education and Information Technologies · doi:10.1007/s10639-022-11177-3 · Cited by 841 · cited 3×
“Computer science and language were the most targeted fields in chatbot-based education, while engineering and mathematics received less attention [12].”
BackgroundHuang, W., Hew, K. F., & Fryer, L. K. (2022) · Journal of Computer Assisted Learning · doi:10.1111/jcal.12610 · Cited by 830
Interest, motivation and learning: An educational-psychological perspective
Krapp, A. (1999) · European Journal of Psychology of Education · doi:10.1007/bf03173109 · Cited by 595 · cited 1×
“Interest indicates the individual readiness or predisposition to engage in a given learning task with effort [45].”
BackgroundA review of opportunities and challenges of chatbots in education
Hwang, G.-J., & Chang, C.-Y. (2023) · Interactive Learning Environments · doi:10.1080/10494820.2021.1952615 · Cited by 530
Rediscovering the use of chatbots in education: A systematic literature review
Pérez, J. Q., Daradoumis, T., & Puig, J. M. M. (2020) · Computer Applications in Engineering Education · doi:10.1002/cae.22326 · Cited by 483 · cited 1×
“Problems have also included technological limitations and training side effects [7].”
Background
Research works printed but not linked: 6
- Lee, 2022, A meta-analysis of the effects of using AI chatbot in Korean EFL education, Stud. Engl. Lang. Lit., 48, 213
- Sternberg, R.J., and Preiss, D.D. (2010). Innovations in Educational Psychology: Perspectives on Learning, Teaching and Human Development, Springer.
- Cole, P.G., and Chan, L.K.S. (1994). Teaching Principles and Practice, Prentice Hall.
- Kim, R. (2022). Effects of AI Chatbot-Based Instruction on the Learning of English Adjectival Transitive Resultative Construction by Korean High School Students. [Ph.D. Thesis, Seoul National University]. Available online: https://hdl.handle.net/10371/181242.
- Abbasi, 2019, Effect of chatbot systems on student’s learning outcomes, Sylwan, 163, 49
- Cohen, J. (1988). Statistical Power Analysis for the Behavioral Sciences, Routledge. [2nd ed.].
Cited by
Sorted by citations
Items 101–200 of 298
Lampropoulos, G., López-Belmonte, J., Pozo-Sánchez, S., & Marín-Marín, J.-A. (2026) · Advances in Computational Intelligence and Robotics · doi:10.4018/979-8-2600-0101-1.ch001 · Cited by 9
Wang, X., Zainuddin, Z., Leng, C. H., Dong, W., & Li, X. (2025) · On the Horizon The International Journal of Learning Futures · doi:10.1108/oth-11-2024-0079 · Cited by 15
Inteligencia Artificial en la Educación: Estado del Arte
Sambola, D.-M. (2023) · Wani · doi:10.5377/wani.v39i79.16806 · Cited by 27
Motivations and Affordances of ChatGPT Usage for College Students’ Learning
Lee, S. K., Ryu, J., Jie, Y., & Ma, D. H. (2025) · Media and Communication · doi:10.17645/mac.9508 · Cited by 7
“On the other hand, Deng and Yu (2023) examined the use of chatbot technology in learning and showed that it could not significantly improve critical thinking, learning engagement, and learning motivation.”
Enhancing Student Motivation Through LLM-Powered Learning Environments
Seßler, K., Kepir, O., & Kasneci, E. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-72312-4_21 · Cited by 7
Perdana, R. S., & Adikara, P. P. (2025) · Journal of Information Systems Engineering and Business Intelligence · doi:10.20473/jisebi.11.1.1-16 · Cited by 10
“Significant development on close domain chatbots (CDC) has been proven by the emergence of various applications in several domains, such as customer support [1][2][3], healthcare [4][5][6], education [7][8][9], and finance [10][11][12], where domain specific conversation assistance is needed.”
Effects of Artificial Intelligence on Educational Functioning: A Review and Meta-Analysis
Yeo, G., & Lansford, J. E. (2025) · Educational Psychology Review · doi:10.1007/s10648-025-10085-5 · Cited by 16
“…reviews and meta-analyses, focus on specific or composite levels of learn - ing and/or types of AI, such as Intelligent Tutoring Systems (ITS; Ma et al., 2014), chatbots (Deng & Yu, 2023), and generative AI (Sun & Zhou, 2024), with a particular emphasis on cognition and psychological functioning.”
InfluentialSomabut, A., Tuamsuk, K., Lowatcharin, G., Traiyarach, S., & Kwangmuang, P. (2025) · Social Sciences & Humanities Open · doi:10.1016/j.ssaho.2025.102259 · Cited by 13
Niraula, S. (2024) · Advances in Mobile Learning Educational Research · doi:10.25082/amler.2024.01.009 · Cited by 11
“As an example, Deng and Yu (2023) see ChatGPT as an opportunity to enhance student motivation, engagement, and learning results.”
BackgroundWu, S., & Luo, M. (2025) · IEEE Access · doi:10.1109/access.2025.3530413 · Cited by 9
Sain, Z. H., & Kurniati, V. S. D. (2023) · TAMANSISWA INTERNATIONAL JOURNAL IN EDUCATION AND SCIENCE · doi:10.30738/tijes.v5i1.16274 · Cited by 11
Artificial Intelligence and Creativity: Anthropological, Sociological and Pedagogical Reflections
Siri, A., Olcese, G., & Spulber, D. (2024) · Geopolitical, Social Security and Freedom Journal · doi:10.2478/gssfj-2024-0001 · Cited by 6
İlhan, Ü. D., & Duran, C. (2025) · Sustainability · doi:10.3390/su17125637 · Cited by 7
EdgeAIGC: Model caching and resource allocation for edge artificial intelligence generated content
Wen, W., Huang, Y., Zhao, X., Zhang, P., Liu, K., & Shi, G. (2025) · Digital Communications and Networks · doi:10.1016/j.dcan.2025.07.003 · Cited by 6
Zhang, R., Zou, D., Xie, H., & Wang, F. L. (2025) · Computers and Education Artificial Intelligence · doi:10.1016/j.caeai.2025.100489 · Cited by 7
Okyar, H. (2023) · RumeliDE Dil ve Edebiyat Araştırmaları Dergisi :/RumeliDe Dil ve Edebiyat Araştırmaları Dergisi · doi:10.29000/rumelide.1369170 · Cited by 7
“This finding agrees with previous review and meta-analysis studies such as Lee and Hwang (2022), Deng and Yu (2023), and Haristiani (2019), which also underlined that chatbots had positive effects on learning success. Additionally, different chatbots used in the studies offered various pedagogical affordances to the s…”
BackgroundMethodResultInfluentialA Bibliometric Analysis of Artificial Intelligence Chatbots in Language Education
Liu, W., Yu-peng, L., & Yu, Z. (2024) · Social Education Research · doi:10.37256/ser.5120243372 · Cited by 7
“Meta-analytical approaches could only reveal the effectiveness of chatbots (Deng & Yu, 2023; Wu & Yu, 2023; Zhang et al., 2023), causing the issue that technological and theoretical developments were seldom examined in these reviews.”
BackgroundSelf-citationChatbots in Academic Literacy Exploring AI in Education
Ali, M. H., Kainat, M., Maqsood, M., Fernandez, S. F., Ali, S., & Ahmad, M. (2024) · Advances in higher education and professional development book series · doi:10.4018/979-8-3693-1666-5.ch002 · Cited by 6
Postdigital Citizenship Education and Argumentation: Can AI Support Students’ Critical Literacy?
Rapanta, C. (2025) · Globalisation, comparative education and policy research · doi:10.1007/978-3-031-86145-1_5 · Cited by 6
Wıbowo, F. C., Hasbey, H., Aziz, T. A., Darman, D. R., Kusuma, A. A. K., Ihsan, I., & Bunyamin, M. A. H. (2025) · Multidisciplinary Science Journal · doi:10.31893/multiscience.2025583 · Cited by 10
Boolzen, C., Kühn, J., Flegr, S., Rott, E.-M., Stausberg, N., & Küchemann, S. (2026) · Artificial Intelligence Review · doi:10.1007/s10462-026-11665-9 · Cited by 13
Opinion Mining on Chat GPT based on Twitter Users
Nashrulloh, M. R., Julianto, I. T., & Muzaky, R. K. (2023) · Journal of Applied Intelligent System · doi:10.33633/jais.v8i2.8399 · Cited by 6
“A Chatbot is a program within the field of Artificial Intelligence that can process inputs and provide information through interactive chat interactions[1].”
BackgroundIlma, A., & Sampurna, R. A. (2024) · Peradaban Journal of Interdisciplinary Educational Research · doi:10.59001/pjier.v2i1.152 · Cited by 8
“Consistent with the findings of Deng and Yu (2023), these tools provide recommendations for alternative sentence structures, the rephrasing of ideas, and enhancements to improve overall clarity.”
Mejia, Y. Z. O. A., Navarro, C. A. D., Casildo-Bedón, N., & Ramírez-Pezo, Y. E. (2024) · Frontiers in Education · doi:10.3389/feduc.2024.1459303 · Cited by 5
Chatbot de IA para prevenir el abandono de la educación superior: una revisión de la literatura
Segovia-García, N., & García, L. S. (2024) · Education in the Knowledge Society (EKS) · doi:10.14201/eks.31416 · Cited by 13
Castro-López, A., Cervero, A., & Blanco, L. Á. (2025) · Revista Tecnología Ciencia y Educación · doi:10.51302/tce.2025.22219 · Cited by 12
Çer, E. (2025) · STEM Education · doi:10.3934/steme.2025018 · Cited by 8
Enhancing educational Q&A systems using a Chaotic Fuzzy Logic-Augmented large language model
Chen, H., Shi, N., Chen, L., & Lee, R. (2024) · Frontiers in Artificial Intelligence · doi:10.3389/frai.2024.1404940 · Cited by 5
Uso del ChatGPT y el rendimiento académico en estudiantes de una Universidad Privada
Cruz, S. M. C., Arredondo-Zela, S. O., & Grández-Ventura, L. M. (2024) · REVISTA EDUSER · doi:10.18050/eduser.v11n1a3 · Cited by 6
Jogezai, N., Baloch, F. A., Jaffar, M., & Khilji, G. (2025) · Quality Assurance in Education · doi:10.1108/qae-12-2024-0255 · Cited by 5
A Systematic Review of the Role of AI-Enabled Chatbots in Modern Education
Olugbade, D. (2025) · Advances in computational intelligence and robotics book series · doi:10.4018/979-8-3373-0122-8.ch018 · Cited by 5
Shu, J., Kou, H., Zhang, J., Zhang, T., Zhang, H., Xu, T., & Zhou, Y. (2025) · International Conference on Computer Science and Technologies in Education (CSTE) · doi:10.1109/cste64638.2025.11091997 · Cited by 4
“As an innovative solution [4][5], Artificial Intelligence Generated Content (AIGC) has emerged as a focal point of research [6][7][8].”
Thompson, K. N., Chandler, K. L., Morgan, C., Khashabi, D., Delinski, E. A., & Durme, B. V. (2025) · Journal of Advanced Academics · doi:10.1177/1932202x251356323 · Cited by 5
“We refer to the LLM chatbot as a co-tutor as it overlapped with the teaching assistant and personal tutor chatbot roles outlined by Deng and Yu (2023), with its primary tasks of mentoring and assessment and feedback (Alemdag, 2023).”
Castro, R. A. G., Asté, W. M. B., Quezada, J. L. M., & Huanacuni, L. E. A. (2025) · Online Journal of Communication and Media Technologies · doi:10.30935/ojcmt/17303 · Cited by 4
STEM Undergraduates’ Perceptions of AI Chatbots: A Cross-Sectional Descriptive Survey
Kajan, K., Shi, W. Y., & Wanatowski, D. (2025) · AI in Education · doi:10.3390/aieduc1010004 · Cited by 6
“Deng and Yu (2023) report significant effects on reasoning (d = 1.190) and achievement (d = 1.033) across 32 studies, while Abbas et al. (2024) warn of declines in engagement when reliance increases.”
InfluentialColbran, S., Jha, M., & Schiavone, C. (2026) · International Journal of Educational Technology in Higher Education · doi:10.1186/s41239-026-00605-w · Cited by 6
“While chatbot design (Kuhail et al., 2023) and technical performance (Deng & Yu, 2023) are clearly relevant, our study examined the diverse motivations, barriers, and perceptions that shape student engagement.”
Dang, T. P. (2025) · Proceedings of the AsiaCALL International Conference · doi:10.54855/paic.24623 · Cited by 4
“A considerable number of other papers also centered on the topic of conversation al AI’s impact on education (Chen et al., 2020 ; Crompton & Burke, 2023; Deng & Yu, 2023; Gökçearslan et al., 2024; Krstić et al., 2022; Pérez et al., 2020; Tahiru, 2021).”
Miranda, C. H., Pérez, J. D. R., & Torres, G. S. (2024) · REVISTA COLOMBIANA DE TECNOLOGIAS DE AVANZADA · doi:10.24054/rcta.v1i43.2803 · Cited by 4
“Furthermore, a meta-analysis in [31] aggregating results from 32 studies showed chatbots have a medium-to-high positive effect on learning outcomes like reasoning, achievement, and interest.”
ResultBibliometric Analyses of the Impact of Pedagogical Agents on Learning
Liu, L., Yan, S., Deng, X., & Yu, Z. (2024) · International Journal of Mobile and Blended Learning · doi:10.4018/ijmbl.346976 · Cited by 4
“Pedagogical agents might exert influence on some dimensions of critical thinking, such as inquiring mindset (Deng & Yu, 2023).”
Self-citationTeachers’ Perspectives on AI-Driven Quillionz for Generating EFL Reading Comprehension Quizzes
Nguyen, T. L., Nguyen, H. L., & Le, D. H. (2024) · Proceedings of the AsiaCALL International Conference · doi:10.54855/paic.2462 · Cited by 4
“These technologies respond actively to the learner's input, thereby ensuring optimal levels of challenge and scaffolding (Deng & Yu, 2023).”
BackgroundKaryotaki, M., Drigas, A. S., & Skianis, C. A. (2024) · Sustainability · doi:10.3390/su16072913 · Cited by 4
Pesce, M. A., & Blanco, D. F. (2024) · Research Square · doi:10.21203/rs.3.rs-4542918/v1 · Cited by 4
Exploring Student Perspectives on ChatGPT in Higher Education
Shard, Vaidya, P., Koul, S., Arora, A., Sonia, & Jain, P. (2024) · 2024 3rd Edition of IEEE Delhi Section Flagship Conference (DELCON) · doi:10.1109/delcon64804.2024.10866660 · Cited by 3
Cinq enjeux d’évaluation face à l’émergence des IA génératives en éducation
Audran, J. (2024) · Mesure et évaluation en éducation · doi:10.7202/1114564ar · Cited by 4
Korchak, A., Patarakin, E., & Costley, J. (2025) · RUDN Journal of Informatization in Education · doi:10.22363/2312-8631-2025-22-1-37-57 · Cited by 5
Can ChatGPT as a generative AI tool enhance university student academic achievement?
Yu, Q., Yu, K., Yang, R., & Li, B. (2025) · Innovations in Education and Teaching International · doi:10.1080/14703297.2025.2605651 · Cited by 5
Identifying and Navigating the Current Trends in Business Librarianship and Data Librarianship
Pistone, R. A. (2023) · Computer and Information Science · doi:10.5539/cis.v16n3p1 · Cited by 3
“Deng and Yu (2023) advise that, “chatbots have caught the attention of stakeholders in the educational context” (p. 1).”
BackgroundInfluentialLANGUAGE LEARNING TOOLS AND IMPROVEMENTS IN SPEAKING AND WRITING SKILLS
Alkhawaldeh, M. A., & Mohammad Abedrabbu Alkhawaldeh, M. A. S. K. (2023) · Journal of Southwest Jiaotong University · doi:10.35741/issn.0258-2724.58.5.33 · Cited by 3
Qiu, W., Chit, L. S., Jamil, N. B., Thway, M., Hwee, S. N. S., Zhang, L., Lim, F. S., & Lai, J. W. (2024) · doi:10.35542/osf.io/7yga3 · Cited by 3
Amorin, R. B., & Gallaron, G. M. (2025) · MIER Journal of Educational Studies Trends and Practices · doi:10.52634/mier/2025/v15/i1/2827 · Cited by 3
“Deng and Yu (2023) demonstrated increased motivation and participation when using ChatGPT as an instructional tool, espe - cially in virtual learning environments.”
Widianto, R., Rieger, M., Murshed, M., & Adrison, V. (2024) · Bulletin of Indonesian Economic Studies · doi:10.1080/00074918.2024.2437824 · Cited by 5
“Chatbots are increasingly being used in various fields, such as health (Singh et al. 2023) and education (Deng and Yu 2023).”
Helgert, A., Weis, L., & Straßmann, C. (2025) · CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA) · doi:10.1145/3706599.3720278 · Cited by 2
“…chatbots in video-based collaborative learning environments to understand what role participants assign to multi-party AI-driven chatbots in collaborative settings and whether there are differences based on the categorization from Deng and Yu (teaching assistant, learning partner or tutor) [7].”
The Impact of Chatbots on Students' Reflective Thinking in Introductory Programming Course
Rachmat, A., Watterson, C. A., & Lundqvist, K. Ø. (2025) · IEEE Global Engineering Education Conference (EDUCON) · doi:10.1109/educon62633.2025.11016417 · Cited by 7
“However, further research is necessary to understand how chatbots can better support the development of higher-order thinking skills in learners [5], [7].”
AI Chatbots for Educators: Transforming Online Learning in Higher Education
Engeness, I., Nohr, M., & Fossland, T. (2025) · Lecture notes in computer science · doi:10.1007/978-3-032-00056-9_18 · Cited by 6
Zhang, R., Zou, D., & Cheng, G. (2025) · Education and Information Technologies · doi:10.1007/s10639-025-13776-2 · Cited by 5
“…cacy in EFL argumentative writing and to facilitate non-academic learner socialisa - tion, its core interaction features and teaching methods—such as telling jokes and providing exercises—are common and characteristic of ChatGPT-assisted learning tools (see Deng & Yu, 2023 ; Kohnke et al., 2023 ).”
Masalimova, A. R., Kosheleva, Y. P., Burov, А. I., Payushina, O. V., Sokolova, N. L., & Khvatova, M. A. (2025) · Contemporary Educational Technology · doi:10.30935/cedtech/17482 · Cited by 2
Leitgeb, T., & Leitgeb, M. (2025) · Artificial Intelligence in Education · doi:10.1108/aiie-01-2025-0013 · Cited by 4
ARIKAN, A. (2025) · Anadolu Kültürel Araştırmalar Dergisi · doi:10.63556/ankad.v9i4.269 · Cited by 4
“In a meta - analysis study, Deng and Yu (2023) found that AI technologies have a significant impact on learning outcomes, with a moderate to large effect size, regardless of factors such as intervention duration, chatbot function, or learning content.”
Sakka, F. (2026) · Public Organization Review · doi:10.1007/s11115-025-00934-w · Cited by 2
ChatGPT ‐Generated Versus Teacher‐Prepared Read‐Aloud Plans: A Comparative Effectiveness Study
SÖNMEZ, M., Öksüz, H. İ., Gök, S., & Yıldırım, K. (2026) · European Journal of Education · doi:10.1111/ejed.70458 · Cited by 3
Chernbumroong, S., Intawong, K., Asawimalkit, U., Puritat, K., & Julrode, P. (2026) · Informatics · doi:10.3390/informatics13010016 · Cited by 7
“Meta-analyses indicate that chatbots can promote learner support, motivation, and reflective thinking, although their impact on achievement varies depending on interaction design and instructional alignment [27].”
Artificial Intelligence in Gifted STEAM Education
Milakis, E. D., Argyrakou, C. C., & Samala, A. D. (2026) · Advances in computational intelligence and robotics book series · doi:10.4018/979-8-2600-0101-1.ch008 · Cited by 2
Kajan, K., Shi, W. Y., Wanatowski, D., & Ryan, M. (2026) · Education Sciences · doi:10.3390/educsci16040554 · Cited by 2
Anirvinna, C., & Pokhriyal, D. (2026) · Journal of Innovation and Entrepreneurship · doi:10.1186/s13731-026-00653-6 · Cited by 7
“Chatbots can even influence users to adopt green habits by showing them green choices, hence becoming tools of sustainability and service learning (Majid et al. 2025b; Deng & Yu, 2023).”
Conversational AI and Personal Growth: Insights from a Critical Integrative Review
Sharma, S., Vohra, P., & Vowels, L. M. (2026) · Behavioral Sciences · doi:10.3390/bs16050756 · Cited by 3
“Educational chatbots demonstrate meaningful gains in reasoning and knowledge retention (Deng & Yu, 2023), building the domain competence that humanistic theory (Maurer & Daukantait˙e, 2020) identifies as foundational to self-actualisation.”
Alubthane, F. (2026) · Frontiers in Education · doi:10.3389/feduc.2026.1838188 · Cited by 3
Pesce, M., & Blanco, D. F. (2024) · European Journal of Education Studies · doi:10.46827/ejes.v11i12.5687 · Cited by 4
“…from which the following central themes were identified: students' knowledge about chatbots, previous experiences with these tools, and the types of uses students make of ChatGPT during their university training (Deng and Yu, 2023; Kelly et al. , 2023; Limna et al. , 2023; Shoufan, 2023).”
Chaotic Fuzzy Logic for Enhanced Educational Q&A System
Chen, H., Shi, N., Chen, L., & Lee, R. (2024) · IEEE International Conference on Power, Intelligent Computing and Systems (ICPICS) · doi:10.1109/icpics62053.2024.10795881 · Cited by 2
“LLMs effects on educational domains have transformed to interactive and personalized learning environments, which highlighted the profound implications of chat-based Q&A systems on educational engagement and efficacy [9].”
Chatbots as Learning Companions Exploring Ethical Dimensions of AI in Education
Maqsood, M., Butt, H. H., Ali, M. A., Chinedu, C. C., & Ahmad, M. (2025) · Advances in web technologies and engineering book series · doi:10.4018/979-8-3693-2973-3.ch012 · Cited by 2
Chatbots: history, technology and application in higher education
Stoyanova, T., & Stoyanov, S. (2023) · Annual of Konstantin Preslavsky University of Shumen Faculty of mathematics and informatics · doi:10.46687/knbu2020 · Cited by 2
Setyaningrum, V., Erlina, E., & Thongsan, N. C. (2023) · THABIEA JOURNAL OF NATURAL SCIENCE TEACHING · doi:10.21043/thabiea.v6i2.23487 · Cited by 1
“Deng & Yu (2023), also states that according to the meta-analysis, chatbot technology had positive effects on various learning outcomes.”
ResultAdvantages, Problems and Challenges in the Application of AI Technologies in School Education
Glushkova, T. A., & Malinova, A. (2023) · E-learning · doi:10.34916/el.2023.15.04 · Cited by 1
Business and Optimization Applications Using AI Chatbots
Özbek, H. E., & Demı̇rcı̇oğlu, M. (2024) · Lecture notes in networks and systems · doi:10.1007/978-3-031-55911-2_5 · Cited by 1
Commercial chatbot monitoring: Approaches focused on automated conversation analysis
Kuligowska, K., & Stanusch, M. (2024) · Humanities & Social Sciences Reviews · doi:10.18510/hssr.2024.1227 · Cited by 2
“…capabilities, as well as with best business-derived processes optimization approaches (Nuruzzaman, Hussain, 2018) (Xu, Sanders, et al., 2021) (Deng, Yu, 2023), it is possible to significantly enhance the performance and reliability of commercial chatbots, ensuring they meet both user…”
BackgroundAbdullahu, D., & Ahmetaj, E. (2024) · Advances in electronic government, digital divide, and regional development book series · doi:10.4018/979-8-3693-7205-0.ch007 · Cited by 1
AI-Powered GPT-Based University-Specific Chat Assistant: UniROBO
Budak, S., & Aslan, S. (2024) · NATURENGS MTU Journal of Engineering and Natural Sciences Malatya Turgut Ozal University · doi:10.46572/naturengs.1587879 · Cited by 1
Almousa, M. M., & AbuSa’aleek, A. O. (2025) · Arab World English Journal · doi:10.24093/awej/ai.9 · Cited by 1
Ömür, G. A., Öztırak, M., Yazıcı, A. M., & Çiçeklioğlu, H. (2025) · Studia Humana · doi:10.2478/sh-2025-0007 · Cited by 1
Abunaser, F. M., Hamd, M. M., Bani-Oraba, A. M. N., Hamed, O., Alshiyab, M. Q. M., & Shebani, Z. (2025) · Sustainability · doi:10.3390/su17157092 · Cited by 4
Jogezai, N. A., Jaffar, M., Baloch, F. A., & Khilji, G. K. (2025) · International Journal of Information and Learning Technology · doi:10.1108/ijilt-10-2024-0229 · Cited by 3
Yılmaz, T., & Dökme, Ï. (2025) · El-Cezeri Fen ve Mühendislik Dergisi · doi:10.31202/ecjse.1726783 · Cited by 1
El-Sobkey, S. B., Mohamed, D. A.-A., ElKholy, M., Abdeldayem, T. S., Fawzy, A., Ahmed, Y. F., Khatib, A. E., Khalid, H., Alharbi, M. D., Fathy, K., et al. (2025) · Journal of Taibah University Medical Sciences · doi:10.1016/j.jtumed.2025.08.005 · Cited by 1
“7,27,31 — 33 Additionally, AICs promote self-learning by helping students set goals, access resources, and self-assess progress.”
Mulhim, E. N. A. (2025) · Frontiers in Education · doi:10.3389/feduc.2025.1711832 · Cited by 4
Cobelli, N., Grimaldi, M., Ugolini, M. M., Maione, G., & De Cristofaro, C. (2026) · Springer Proceedings in Complexity · doi:10.1007/978-3-032-09062-1_1 · Cited by 1
Alé-Silva, J., & Rodríguez-Donoso, M. (2026) · Discover Education · doi:10.1007/s44217-026-01131-7 · Cited by 3
Jiang, X. (2026) · Interactive Technology and Smart Education · doi:10.1108/itse-01-2026-0004 · Cited by 1
Shehata, B., Huang, R., Kanwar, A. S., Liu, D., Zheng, L., & Liu, Z. (2026) · Smart Learning Environments · doi:10.1186/s40561-026-00449-x · Cited by 2
Zhang, Q., Zhang, N., & Lu, C. (2026) · Educational Research Review · doi:10.1016/j.edurev.2026.100783 · Cited by 3
Sindhvananda, K., Ruengwattanachot, K., Leksuwankun, S., Pongpitakmetha, T., Hiransuthikul, A., Surawattanawong, T., Anukoolwittaya, P., & Panjasriprakarn, P. (2026) · BMC Medical Education · doi:10.1186/s12909-026-09511-1 · Cited by 3
“Prior meta-analytic evidence on educational chatbots reported an overall medium-to-large positive effects on educational outcomes, with a pooled effect size of 0.789 across 32 studies [36].”
InfluentialEnhancing Management Education With AI: Chatbot-Assisted Case Studies
Bensalem, A., & Bésanger, S. (2026) · Organizational Behavior Teaching Review · doi:10.1177/10525629261452651 · Cited by 1
“While existing studies consider engagement and knowledge retention (Crompton & Burke, 2023; Deng & Yu, 2023), Mendoza et al. (2022, p. 17) refer to a ‘certain epistemic immaturity’ within the field of educational chatbots.”
Tan, J. W. D., Oh, H. X., Krishnan, V., Neo, A. W. M., Mohamed, I. N., Mohamed, R. M. P., Chowdhury, M. E. H., & Kumar, J. (2026) · Frontiers in Education · doi:10.3389/feduc.2026.1750324 · Cited by 1
Hasan, M. A., Kashef, A., Mohsen, W. A., Khan, A. G. Y., AlOmari, M., & Aishan, H. (2026) · Frontiers in Education · doi:10.3389/feduc.2026.1895511 · Cited by 1
Zhong, S., & Simons, J. D. (2026) · Frontiers in Psychology · doi:10.3389/fpsyg.2026.1831006 · Cited by 1
Analyzing and Responding to Google Maps Reviews with a Chatbot in Healthcare
Akkol, E., & Doğan, O. (2023) · Lecture notes in networks and systems · doi:10.1007/978-3-031-39777-6_14 · Cited by 1
A Study to Measure the Potential Impact of Generative Artificial Intelligence on Academic Integrity
Duane, A. (2024) · Resilience Through Digital Innovation: Enabling the Twin Transition · doi:10.18690/um.fov.4.2024.1 · Cited by 2
Funcionalidades Emergentes para Tutorias em Ambientes Virtuais de Aprendizagem
Pereira, A. J., Gomes, A. S., & Primo, T. T. (2024) · Revista Iberoamericana de Tecnología en Educación y Educación en Tecnología · doi:10.24215/18509959.38.e6 · Cited by 1
SMat-Bot: An Interactive Chatbot for Facilitating Students’ Comprehension of Mathematics
Berida, O. T., Oguntimilehin, A., Badeji–Ajisafe, B., Tope-Oke, A. M., Babalola, G. O., Akinduyite, C. O., Okebule, T., Obamiyi, S. E., & Sanya, O. A. (2024) · International Conference on Science, Engineering and Business for Driving Sustainable Development Goals (SEB4SDG) · doi:10.1109/seb4sdg60871.2024.10630005 · Cited by 1
AskMe – A Customer Service Chatbot
D, K. M., Dsouza, B. J., & Nagesh, K. N. (2025) · International Conference on Electronics and Renewable Systems (ICEARS) · doi:10.1109/icears64219.2025.10941047 · Cited by 1
Karaköse, T., & Polat, H. (2025) · Studies in computational intelligence · doi:10.1007/978-3-031-82915-4_18 · Cited by 1
Ríos, A. G. N., López, A. R. J., & Contreras, D. F. Q. (2025) · ASCE MAGAZINE · doi:10.70577/asce/2431.2453/2025 · Cited by 1
Metadata
- Authors
- Deng, X., & Yu, Z.
- Affiliations
- Beijing Language and Culture University (CN)
- Corresponding author
- Yu, Zhonggen
- Venue
- Sustainability
- Publisher
- MDPI AG
- Published
- 2023-02-06
- Volume
- 15
- Issue
- 4
- Pages
- 2940
- ISSN
- 2071-1050 (ISSN-L)
- OpenAlex
- W4319440682
- Semantic Scholar
- 27f4d355b27a6878332a87845a6ff13940fe9158
- Cited by
- 298
- References
- 64
- Citations by year
- 2026: 76 · 2025: 115 · 2024: 72 · 2023: 35
- FWCI
- 12.28
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- Locations
- Sustainability (publishedVersion, cc-by)
- RePEc: Research Papers in Economics (submittedVersion)
- Funders
- 2019 MOOC of Beijing Language and Culture University (MOOC201902) (important) “Introduction to Linguistics” (202010032003, SJTS202108, 22YCX017)
- “Introduction to Linguistics” of online and offline mixed courses in Beijing Language and Culture University in 2020 (202010032003, SJTS202108, 22YCX017)
- Beijing Co-construction Project Research and reform of the “Undergraduate Teaching Reform and Innovation Project” of Beijing higher education in 2020-innovative “multilingual +” excellent talent training system (202010032003, SJTS202108, 22YCX017)
- The research project of Graduate Students of Beijing Language and Culture University “Xi Jinping: The Governance of China” (202010032003, SJTS202108, 22YCX017)
- Beijing Language and Culture University (202010032003, SJTS202108, 22YCX017)
- Topics
- Social Sciences › Education › Artificial Intelligence in Education
- Physical Sciences › Computer Science › Artificial Intelligence › Intelligent Tutoring Systems and Adaptive Learning
- Social Sciences › Business, Management and Accounting › Marketing › AI in Service Interactions
- Keywords
- chatbots, chatbot-assisted learning, sustainable education, learning outcomes, moderators, explicit reasoning, knowledge retention, academic performance, student learning interest, critical thinking, student engagement, student motivation, artificial intelligence in education
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
A language model drew the claims from the paper on 2026-10-06, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-10-06 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
