Interacting with educational chatbots: A systematic review
Open accessThe DOI matches its Crossref record; 117 of the 117 references it cites were found in a registry or an index; 14 of the 67 sources its claims cite were checked for support.
Abstract
Abstract Chatbots hold the promise of revolutionizing education by engaging learners, personalizing learning activities, supporting educators, and developing deep insight into learners’ behavior. However, there is a lack of studies that analyze the recent evidence-based chatbot-learner interaction design techniques applied in education. This study presents a systematic review of 36 papers to understand, compare, and reflect on recent attempts to utilize chatbots in education using seven dimensions: educational field, platform, design principles, the role of chatbots, interaction styles, evidence, and limitations. The results show that the chatbots were mainly designed on a web platform to teach computer science, language, general education, and a few other fields such as engineering and mathematics. Further, more than half of the chatbots were used as teaching agents, while more than a third were peer agents. Most of the chatbots used a predetermined conversational path, and more than a quarter utilized a personalized learning approach that catered to students’ learning needs, while other chatbots used experiential and collaborative learning besides other design principles. Moreover, more than a third of the chatbots were evaluated with experiments, and the results primarily point to improved learning and subjective satisfaction. Challenges and limitations include inadequate or insufficient dataset training and a lack of reliance on usability heuristics. Future studies should explore the effect of chatbot personality and localization on subjective satisfaction and learning effectiveness.
Overview
CoChat Summary
TL;DR
This systematic review analyzes 36 papers on educational chatbots across seven dimensions: field, platform, role, interaction style, design principles, empirical evidence, and limitations. It finds most chatbots are web-based teaching agents with chatbot-driven flow-based interaction, evaluated mainly by experiments showing improved learning and satisfaction, while dataset training and usability/personality gaps remain.
Key findings
- Roles: 20 (55.55%) chatbots were teaching agents, 13 (36.11%) peer agents, 2 teachable agents, and 2 motivational agents.
- Platforms: 25 (69.44%) operated on the web, 6 (16.66%) on mobile, and 5 (13.88%) on desktop.
- Interaction styles: 88.88% used chatbot-driven conversation; 52.77% were flow-based, 36.11% intent-based, and 4 (11.11%) were user-driven.
- Evaluation and outcomes: 13 (36.11%) used experiments, 10 (27.77%) evaluation studies, 10 (27.77%) questionnaires, and 3 (8.22%) focus groups; 8 experiments produced statistically significant improved learning, with other studies reporting improved engagement, interest, and subjective satisfaction.
Methods
- Systematic review following Keele et al. (2007), with seven research questions covering educational field, platform, role, interaction style, design principles, empirical evidence, and challenges/limitations.
- Searched ACM Digital Library, Scopus, IEEE Xplore, and SpringerLink for January 2011–April 2021 using Boolean terms for chatbot/conversational agent/pedagogical agent and education/learning/learner/teaching/teacher/student.
- From 1,208 initial articles, applied inclusion/exclusion criteria to select 36 papers; four authors worked in pairs to cross-check selection and classification.
Limitations
- Stated: the review was restricted to January 2011–April 2021 and four digital libraries (ACM, Scopus, IEEE Xplore, SpringerLink), so relevant articles in other libraries such as Web of Science may have been missed.
- Stated: exclusion criteria applied to 1,208 initial articles may have introduced bias (e.g., excluding short papers with original ideas or papers without sufficient evidence), and classification may have been somewhat inaccurate due to researchers’ diverse experience, mitigated by cross-checking.
- Evident from the text: many evaluations rely on questionnaires or small focus groups (e.g., focus groups with 10 participants), and the review notes that none of the surveyed articles explicitly relied on usability heuristics or assessed a distinct chatbot personality.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Figures









Verification
Claims the paper makes
13 Valid · 5 Not valid · 53 Not checked · 119 Own results and interpretations
Every source the paper cites for this claim was checked and supports it.
In the paper
“only one study (Wollny et al., 2021) analyzed the educational roles the chatbots are playing”
§ Interacting with educational chatbots: A systematic review > 3 Related work
What the sources say
- Supportedcites Wollny et al., 2021 source
The Methods (Analysis) identify pedagogical roles as a question used to code the reviewed publications. The Results (Pedagogical Roles) and Discussion (Answer to Research Questions) analyze those roles as Learning, Assisting, and Mentoring. The citation identifies Wollny et al. as the study doing that analysis; “only one” describes the citing authors’ selection of related studies. (LLM, conf=0.91)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“Only four studies (Hwang & Chang, 2021;Wollny et al., 2021;Smutny & Schreiberova, 2020;Winkler & Söllner, 2018) examined the field of application.”
§ Interacting with educational chatbots: A systematic review > 3 Related work
What the sources say
- Not matchedcites Hwang & Chang, 2021
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
- Supportedcites Wollny et al., 2021 source
The Related Work section discusses application clusters for educational chatbots, and the Results section examines their domains. Wollny et al. is therefore a study of the field of application; the count of four refers to the citing authors’ list of studies, not a finding attributed to Wollny et al. (LLM, conf=0.92)
- Supportedcites Smutny & Schreiberova, 2020 source
The Discussion states that this study reviewed educational chatbots by subject matter, and Table 1 lists the subject-matter categories. That supports citing it as a study that examined the field of application; the count of four studies is the citing authors’ description of their selected literature. (LLM, conf=0.90)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Not matched
The sources with a verdict support this claim; the rest weren’t checked yet, have no full text we could read, or their verdicts aren’t rated yet.
In the paper
“some teaching agents recommended tutorials to students based upon learning styles (Redondo-Hernández & Pérez-Marín, 2011), students' historical learning (Coronado et al., 2018), and pattern matching (Ondáš et al., 2019).”
§ Interacting with educational chatbots: A systematic review > Teaching agents
What the sources say
- Not matchedcites Redondo-Hernández & Pérez-Marín, 2011
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
- Supportedcites Coronado et al., 2018 source
Section 4 (Design of a teaching agent) states that 'The agent behaviours and decision making are based on the information provided by i) the properties and relations given by the domain model, ii) the stages in the learning path, and iii) the historical student queries stored in the KB', and describes the teaching agent's proactive behaviours 'Suggest related content to learn about' (analysing the topics of the student's last queries) and 'Suggest follow up topics to learn' (using the learning path refined by students' behaviour). Section 5 adds that 'Since it stores the students progress, in successive conversations, the same concepts will not be suggested as relevant', i.e. recommendations driven by the student's learning history, which is what the citing sentence attributes here (rendering the suggested learning content as 'tutorials'). (LLM, conf=0.82)
- Not checked — no full text of the sourcecites Ondáš et al., 2019 source
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Not matched
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“With this approach, the students received 12% more accurate answers than those given chatbots without personalized learning materials.”
§ Interacting with educational chatbots: A systematic review > 5.5 RQ5 -What are the principles used to guide the design of the educational chatbots?
What the sources say
- Partly supportedcites Latham et al., 2011 sourceExperimental — accuracy not yet measured
The Experimental section reports that students given learning materials matched to their learning styles averaged 73% correct answers, versus 61% for students given mismatched materials; the Conclusions describe this as 12% more correct answers. The study measured students’ answers, not answers students received from chatbots, and its comparator was mismatched material within Oscar rather than chatbots without personalized materials. (LLM, conf=0.90)
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Partly supported
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“Our results are similar to those reported in (Smutny & Schreiberova, 2020) which classified most chatbots as teaching agents that recommend content, conducted formative assessments, and set learning goals.”
§ Interacting with educational chatbots: A systematic review > 6 Discussion and future research directions
What the sources say
- Partly supportedcites Smutny & Schreiberova, 2020 sourceExperimental — accuracy not yet measured
Table 2 (section 5, Conclusions; page unavailable) lists recommending learning content, quiz feedback, and setting goals as teaching-related quality attributes. The Discussion reports that all ten top-ranked chatbots used quizzes or a question-and-answer module. The paper does not say it classified most chatbots as teaching agents that performed all three functions. (LLM, conf=0.86)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Partly supported
86 more claims
Create a free account to see all 190 claims and to verify the 6 references still open.
- Probably not a substantial claim. 78
- It may state two separate claims as one. 31
- Borderline: one of the tests for a substantial claim is unclear. 2
- The quote only partly states this claim. 1
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“chatbots, including ChatGPT, have demonstrated the potential to reshape and revolutionize education, creating new, novel opportunities for enhancing the learning process and outcomes [83], facilitating different learning approaches, and offering a range of pedagogical benefits [19, 43, 72]”
From A systematic literature review of empirical research on ChatGPT in education (2024) · cites this paper as [43]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Chatbots allow learners to engage in conversations anytime and anywhere, free from time and space constraints. Learners can interact with chatbots, lower tension and stress levels, and engage in repetitive learning [36].”
From Development and effects of a chatbot education program for self-directed learning in nursing students (2025) · cites this paper as [36]
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“This contrasts with earlier studies such as Kuhail et al. (2023) and Huang et al. (2022), which emphasized motivation and engagement as key affordances of chatbots.”
From Motivations and Affordances of ChatGPT Usage for College Students’ Learning (2025) · cites this paper as Kuhail et al. (2023)
In the citing paper
“Erosion of human relations: The emphasis on AI-driven verbal communication tools may inadvertently contribute to the erosion of nonverbal communication competencies and social skills among learners and educators2,85,86.”
From On opportunities and challenges of large multimodal foundation models in education (2025) · cites this paper as [85]
In the citing paper
“chatbots have been shown to be effective in various roles when engaging with students, including teaching agents, peer agents, teachable agents, and motivational agents [3].”
From Interacting with a Chatbot-Based Advising System: Understanding the Effect of Chatbot Personality and User Gender on Behavior (2022) · cites this paper as [3]
3 more claims about this paper
Create a free account to see every claim other papers make about this one.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
Each contradiction between two passages listed here was confirmed by an independent check.
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ Interacting with educational chatbots: A systematic review > 6 Discussion and future research directions
“More than a third of the chatbots were developed to teach computer science topics, including programming languages and networks.”
§ Interacting with educational chatbots: A systematic review > 5 Results
“12 articles; 33.33%) developed chatbots used to teach computer science”
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ Interacting with educational chatbots: A systematic review > 5.3 RQ3 -What role do the educational chatbots play when interacting with students?
“four roles (teaching agents, peer agents, teachable agents, and peer agents)”
§ Interacting with educational chatbots: A systematic review > 5.3 RQ3 -What role do the educational chatbots play when interacting with students?
“Only two studies used chatbots as teachable agents, and two studies used them as motivational agents.”
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ Interacting with educational chatbots: A systematic review > 5 Results
“The vast majority of selected articles were written or co-written by researchers from American universities.”
§ Interacting with educational chatbots: A systematic review > 5 Results
“the research that emerged from all European universities combined was the highest in the number of articles (19 articles).Asian universities have contributed 10 articles, while American universities contributed 9 articles.”
Citations
Cites
(2002) · Cited by 13,211
Hattie, J., & Timperley, H. (2007) · Review of Educational Research · doi:10.3102/003465430298487 · Cited by 12,905 · cited 1×
“• Lack of Feedback Feedback is a crucial element that affects learning in various environments (Hattie and Timperley, 2007).”
BackgroundELIZA—a computer program for the study of natural language communication between man and machine
Weizenbaum, J. (1966) · Communications of the ACM · doi:10.1145/365153.365168 · Cited by 4,452
International Journal of Scientific and Research Publications
(2022) · International Journal of Scientific and Research Publications · doi:10.29322/ijsrp · Cited by 2,691
Morgan, D. L. (1996) · Annual Review of Sociology · doi:10.1146/annurev.soc.22.1.129 · Cited by 2,353
Nass, C., Steuer, J., & Tauber, E. R. (1994) · Proceedings of the SIGCHI Conference on Human Factors in Computing Systems · doi:10.1145/191666.191703 · Cited by 2,148 · cited 1×
“…words such as “family” or “together” (Hirsh et al., 2009), words that are regarded as positive emotionally such as “like” or “nice” (Hirsh et al., 2009), words indicating assurance such as “sure” (Nass et al., 1994), as well as certain emojis (Völkel et al., 2019), as suggested by the literature.”
Encyclopedia of the Sciences of Learning
(2012) · doi:10.1007/978-1-4419-1428-6 · Cited by 1,091
Key Concepts in Social Research
Payne, G., & Payne, J. (2004) · doi:10.4135/9781849209397 · Cited by 975
Effectiveness of Intelligent Tutoring Systems: A Meta-Analytic Review
Kulik, J. A., & Fletcher, J. D. (2016) · Review of Educational Research · doi:10.3102/0034654315581420 · Cited by 930 · cited 1×
“Teaching agents play the role of human teachers and can present instructions, illustrate examples, ask questions (Wambs-ganss et al., 2020), and provide immediate feedback (Kulik & Fletcher, 2016).”
BackgroundChatbots applications in education: A systematic review
Okonkwo, C. W., & Ade-Ibijola, A. (2021) · Computers and Education: Artificial Intelligence · doi:10.1016/j.caeai.2021.100033 · Cited by 879
Research works printed but not linked: 9
- Benotti, L., Martnez, M.C., & Schapachnik, F. (2018). Atool for introducing computer science with automatic formative assessment. IEEE Transactions on Learning Technologies, 11(2), 179–192. https://doi.org/10.1109/TLT.2017.2682084. · DOI · Paper page
- Daud, S.H.M., Teo, N.H.I., & Zain, N.H.M. (2020). Ejava chatbot for learning programming language: Apost-pandemic alternative virtual tutor. International Journal, 8(7), 3290–3298.
- Haake, M., & Gulz, A. (2009). A look atthe roles of look & roles in embodied pedagogical agents–a user preference perspective. International Journal of Artificial Intelligence in Education, 19(1), 39–71.
- Ruttkay, Z., & Pelachaud, C. (2006). From browstotrust: Evaluating embodied conversational agents Vol. 7. Berlin: Springer Science & Business Media.
- Shneiderman, B., Plaisant, C., Cohen, M.S, Jacobs, S., Elmqvist, N., & Diakopoulos, N. (2016). Designingthe user interface: strategies for effective human-computer interaction. London: Pearson.
- Bradeško, L., & Mladenić, D. (2012). A survey of chatbot systemsthrough a loebner prize competition. In Proceedings of Slovenian language technologies society eighth conference of language technologies (pp. 34–37).
- Hobert, S., & Meyer von Wolff, R. (2019). Say helloto your new automatedtutor–a structured literature review on pedagogical conversational agents.
- Laurillard, D. (2013). Rethinking university teaching: A conversational framework for the effective use of learning technologies. Routledge.
- Qin, C., Huang, W., & Hew, K.F. (2020). Usingthe community of inquiry framework to develop an educational chatbot: lesson learned from a mobile instant messaging learning environment. In Proceedings ofthe 28th international conference on computers in education.
Cited by
What if the devil is my guardian angel: ChatGPT as a case study of using chatbots in education
Tlili, A., Shehata, B., Adarkwah, M. A., Bozkurt, A., Hickey, D. T., Huang, R., & Agyemang, B. (2023) · Smart Learning Environments · doi:10.1186/s40561-023-00237-x · Cited by 2,003
ChatGPT for Language Teaching and Learning
Kohnke, L., Moorhouse, B. L., & Zou, D. (2023) · RELC Journal · doi:10.1177/00336882231162868 · Cited by 1,371
Role of AI chatbots in education: systematic literature review
Labadze, L., Grigolia, M., & Machaidze, L. (2023) · International Journal of Educational Technology in Higher Education · doi:10.1186/s41239-023-00426-1 · Cited by 1,043
Large Language Models in Medical Education: Opportunities, Challenges, and Future Directions
Abd-alrazaq, A., AlSaad, R., Alhuwail, D., Ahmed, A., Healy, P. M., Latifi, S., Aziz, S., Damseh, R., Alabed Alrazak, S., & Sheikh, J. (2023) · JMIR Medical Education · doi:10.2196/48291 · Cited by 722
Jeon, J., & Lee, S. (2023) · Education and Information Technologies · doi:10.1007/s10639-023-11834-1 · Cited by 696
“…to predefined dialogue paths or feature simplistic question-and-answer dialogue structures (for a review on existing educational chatbots, see Kuhail et al., 2022), this exponentially advanced chatbot can generate answers according to the context of a given prompt, and thus, can engage in a…”
BackgroundInfluential
Metadata
- Authors
- Kuhail, M. A., Alturki, N., Alramlawi, S., & Alhejori, K.
- Affiliations
- Zayed University (AE)
- Princess Nourah bint Abdulrahman University (SA)
- Corresponding author
- Alturki, Nazik
- Venue
- Education and Information Technologies
- Published
- 2022-07-09
- Volume
- 28
- Issue
- 1
- Pages
- 973-1018
- ISSN
- 1360-2357 (ISSN-L)
- 1573-7608
- OpenAlex
- W4284991798
- Cited by
- 841
- References
- 110
- Citations by year
- 2026: 201 · 2025: 253 · 2024: 248 · 2023: 133 · 2022: 4
- FWCI
- 11.12
- Type
- journal article
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- cc-by
- creativecommons.org/licenses/by/4.0 (tdm)
- creativecommons.org/licenses/by/4.0 (vor)
- Locations
- Education and Information Technologies (publishedVersion, cc-by)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Intelligent Tutoring Systems and Adaptive Learning
- Social Sciences › Education › Artificial Intelligence in Education
- Social Sciences › Business, Management and Accounting › Marketing › AI in Service Interactions
- Keywords
- educational chatbots, personalized learning, peer agents, chatbot personality, student satisfaction, chatbot evaluation, chatbot usability
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
The claim verdicts this report counts are experimental: their accuracy is not yet measured.
A language model drew the claims from the paper on 2026-10-10, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-10-10 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Cited sources are checked with a language model’s help (gpt-6-sol and qwen3.8-omni-flash).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Interacting with educational chatbots: A systematic review. https://app.cochat.ai/papers/10.1007/s10639-022-11177-3
- The paper
- Kuhail, M. A., Alturki, N., Alramlawi, S., & Alhejori, K. (2023). Interacting with educational chatbots: A systematic review. Education and Information Technologies. https://doi.org/10.1007/s10639-022-11177-3
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.