CoChat

Interacting with educational chatbots: A systematic review

Open access
Kuhail, Mohammad Amin, Alturki, Nazik, Alramlawi, Salwa, Alhejori, Kholood · 2023 · Education and Information Technologies

The DOI matches its Crossref record; 117 of the 117 references it cites were found in a registry or an index; 14 of the 67 sources its claims cite were checked for support.

Abstract

Abstract Chatbots hold the promise of revolutionizing education by engaging learners, personalizing learning activities, supporting educators, and developing deep insight into learners’ behavior. However, there is a lack of studies that analyze the recent evidence-based chatbot-learner interaction design techniques applied in education. This study presents a systematic review of 36 papers to understand, compare, and reflect on recent attempts to utilize chatbots in education using seven dimensions: educational field, platform, design principles, the role of chatbots, interaction styles, evidence, and limitations. The results show that the chatbots were mainly designed on a web platform to teach computer science, language, general education, and a few other fields such as engineering and mathematics. Further, more than half of the chatbots were used as teaching agents, while more than a third were peer agents. Most of the chatbots used a predetermined conversational path, and more than a quarter utilized a personalized learning approach that catered to students’ learning needs, while other chatbots used experiential and collaborative learning besides other design principles. Moreover, more than a third of the chatbots were evaluated with experiments, and the results primarily point to improved learning and subjective satisfaction. Challenges and limitations include inadequate or insufficient dataset training and a lack of reliance on usability heuristics. Future studies should explore the effect of chatbot personality and localization on subjective satisfaction and learning effectiveness.

Overview

CoChat Summary

TL;DR

This systematic review analyzes 36 papers on educational chatbots across seven dimensions: field, platform, role, interaction style, design principles, empirical evidence, and limitations. It finds most chatbots are web-based teaching agents with chatbot-driven flow-based interaction, evaluated mainly by experiments showing improved learning and satisfaction, while dataset training and usability/personality gaps remain.

Key findings

  • Roles: 20 (55.55%) chatbots were teaching agents, 13 (36.11%) peer agents, 2 teachable agents, and 2 motivational agents.
  • Platforms: 25 (69.44%) operated on the web, 6 (16.66%) on mobile, and 5 (13.88%) on desktop.
  • Interaction styles: 88.88% used chatbot-driven conversation; 52.77% were flow-based, 36.11% intent-based, and 4 (11.11%) were user-driven.
  • Evaluation and outcomes: 13 (36.11%) used experiments, 10 (27.77%) evaluation studies, 10 (27.77%) questionnaires, and 3 (8.22%) focus groups; 8 experiments produced statistically significant improved learning, with other studies reporting improved engagement, interest, and subjective satisfaction.

Methods

  • Systematic review following Keele et al. (2007), with seven research questions covering educational field, platform, role, interaction style, design principles, empirical evidence, and challenges/limitations.
  • Searched ACM Digital Library, Scopus, IEEE Xplore, and SpringerLink for January 2011–April 2021 using Boolean terms for chatbot/conversational agent/pedagogical agent and education/learning/learner/teaching/teacher/student.
  • From 1,208 initial articles, applied inclusion/exclusion criteria to select 36 papers; four authors worked in pairs to cross-check selection and classification.

Limitations

  • Stated: the review was restricted to January 2011–April 2021 and four digital libraries (ACM, Scopus, IEEE Xplore, SpringerLink), so relevant articles in other libraries such as Web of Science may have been missed.
  • Stated: exclusion criteria applied to 1,208 initial articles may have introduced bias (e.g., excluding short papers with original ideas or papers without sufficient evidence), and classification may have been somewhat inaccurate due to researchers’ diverse experience, mitigated by cross-checking.
  • Evident from the text: many evaluations rely on questionnaires or small focus groups (e.g., focus groups with 10 participants), and the review notes that none of the surveyed articles explicitly relied on usability heuristics or assessed a distinct chatbot personality.

Model-written summary. Check it against the paper before relying on it.

Integrity signals

Registry

Crossref · 10.1007/s10639-022-11177-3

Cites

117 of 117 references verified0 citation errors · 0 not checkable

Statistics

Nothing to recompute.

Figures

Fig. 1 Flowchart of the process of the selection of the studies
Fig. 1 Flowchart of the process of the selection of the studiesCC-BY · From the PDF
Fig. 2 A timeline of the selected studies
Fig. 2 A timeline of the selected studiesCC-BY · From the PDF
Fig. 3 A geographical mapping of the selected articles
Fig. 3 A geographical mapping of the selected articlesCC-BY · From the PDF
Fig. 4 The fields of the chatbots in the selected articles
Fig. 4 The fields of the chatbots in the selected articlesCC-BY · From the PDF
Fig. 5 The platforms of the chatbots in the selected articles
Fig. 5 The platforms of the chatbots in the selected articlesCC-BY · From the PDF
Fig. 6 The roles of the chatbots in the selected articles
Fig. 6 The roles of the chatbots in the selected articlesCC-BY · From the PDF
Fig. 7 The interaction styles of the chatbots in the selected articles
Fig. 7 The interaction styles of the chatbots in the selected articlesCC-BY · From the PDF
Fig. 8 The principles used to design the chatbots
Fig. 8 The principles used to design the chatbotsCC-BY · From the PDF
Fig. 9 Empirical evaluation methods applied in the selected studies
Fig. 9 Empirical evaluation methods applied in the selected studiesCC-BY · From the PDF

Verification

Claims the paper makes

13 Valid · 5 Not valid · 53 Not checked · 119 Own results and interpretations

86 more claims

Create a free account to see all 190 claims and to verify the 6 references still open.

See all claims — free account
  • Probably not a substantial claim. 78
  • It may state two separate claims as one. 31
  • Borderline: one of the tests for a substantial claim is unclear. 2
  • The quote only partly states this claim. 1

Create a free account to see these claims.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

3 more claims about this paper

Create a free account to see every claim other papers make about this one.

See all claims — free account

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

Each contradiction between two passages listed here was confirmed by an independent check.

3 of 3

Citations

Cites

Show all 117 references

Research works printed but not linked: 9

  1. Benotti, L., Martnez, M.C., & Schapachnik, F. (2018). Atool for introducing computer science with automatic formative assessment. IEEE Transactions on Learning Technologies, 11(2), 179–192. https://doi.org/10.1109/TLT.2017.2682084. · DOI · Paper page
  2. Daud, S.H.M., Teo, N.H.I., & Zain, N.H.M. (2020). Ejava chatbot for learning programming language: Apost-pandemic alternative virtual tutor. International Journal, 8(7), 3290–3298.
  3. Haake, M., & Gulz, A. (2009). A look atthe roles of look & roles in embodied pedagogical agents–a user preference perspective. International Journal of Artificial Intelligence in Education, 19(1), 39–71.
  4. Ruttkay, Z., & Pelachaud, C. (2006). From browstotrust: Evaluating embodied conversational agents Vol. 7. Berlin: Springer Science & Business Media.
  5. Shneiderman, B., Plaisant, C., Cohen, M.S, Jacobs, S., Elmqvist, N., & Diakopoulos, N. (2016). Designingthe user interface: strategies for effective human-computer interaction. London: Pearson.
  6. Bradeško, L., & Mladenić, D. (2012). A survey of chatbot systemsthrough a loebner prize competition. In Proceedings of Slovenian language technologies society eighth conference of language technologies (pp. 34–37).
  7. Hobert, S., & Meyer von Wolff, R. (2019). Say helloto your new automatedtutor–a structured literature review on pedagogical conversational agents.
  8. Laurillard, D. (2013). Rethinking university teaching: A conversational framework for the effective use of learning technologies. Routledge.
  9. Qin, C., Huang, W., & Hew, K.F. (2020). Usingthe community of inquiry framework to develop an educational chatbot: lesson learned from a mobile instant messaging learning environment. In Proceedings ofthe 28th international conference on computers in education.

Cited by

See all 841

Metadata

Authors
Kuhail, M. A., Alturki, N., Alramlawi, S., & Alhejori, K.
Affiliations
  • Zayed University (AE)
  • Princess Nourah bint Abdulrahman University (SA)
Corresponding author
Alturki, Nazik
Venue
Education and Information Technologies
Published
2022-07-09
Volume
28
Issue
1
Pages
973-1018
ISSN
  • 1360-2357 (ISSN-L)
  • 1573-7608
OpenAlex
W4284991798
Cited by
841
References
110
Citations by year
2026: 201 · 2025: 253 · 2024: 248 · 2023: 133 · 2022: 4
FWCI
11.12
Type
journal article
Language
en
Open access
Licence
Locations
Education and Information Technologies (publishedVersion, cc-by)
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › Intelligent Tutoring Systems and Adaptive Learning
  • Social Sciences › Education › Artificial Intelligence in Education
  • Social Sciences › Business, Management and Accounting › Marketing › AI in Service Interactions
Keywords
educational chatbots, personalized learning, peer agents, chatbot personality, student satisfaction, chatbot evaluation, chatbot usability

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

The claim verdicts this report counts are experimental: their accuracy is not yet measured.

A language model drew the claims from the paper on 2026-10-10, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).

A second model labelled each claim on 2026-10-10 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).

Cited sources are checked with a language model’s help (gpt-6-sol and qwen3.8-omni-flash).

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: Interacting with educational chatbots: A systematic review. https://app.cochat.ai/papers/10.1007/s10639-022-11177-3
The paper
Kuhail, M. A., Alturki, N., Alramlawi, S., & Alhejori, K. (2023). Interacting with educational chatbots: A systematic review. Education and Information Technologies. https://doi.org/10.1007/s10639-022-11177-3

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.