A Systematic Literature Review of Intelligent Tutoring Systems With Dialogue in Natural Language
Open accessThe DOI matches its Crossref record; 21 of the 21 references it cites were found in a registry or an index.
Abstract
Intelligent tutoring systems (ITSs) are computer programs that provide instruction adapted to the needs of individual students. Dialog systems are computer programs that communicate with human users by using natural language. This paper presents a systematic literature review to address ITSs that incorporate dialog systems and have been implemented in the last twenty years. The review found 33 ITSs and focused on answering the following five research questions. a) What ITSs with natural language dialogue have been developed? b) What is the main purpose of the tutoring dialogue in each system? c) What are the pedagogical features of the teaching process performed by the ITSs with natural language dialogue? d) What natural language understanding approach does each system employ to understand students’ utterances? e) What evidence exists related to the evaluation of ITSs with natural language dialogue? The results of this review reveal that most ITSs are directed toward science, technology, engineering, and mathematics (STEM) domains at the university level, and the majority of the selected ITSs implement the expectations and misconceptions tailored approach. Furthermore, most ITSs use dialog to help students learn how to solve a problem by applying rules, laws, etc. (the apply level in Bloom’s taxonomy). With regard to the instructional approach, the selected ITSs help students write correct explanations or answers for deep questions; assist students in problem solving; or support a reflective dialogue motivated by either previously provided content or the result of a simulation. Additionally, we found empirical evaluations for 90.91% of the selected ITSs that measure the learning gains and/or assess the impacts of different tutoring strategies.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Does Active Learning Work? A Review of the Research
Prince, M. (2004) · Journal of Engineering Education · doi:10.1002/j.2168-9830.2004.tb00809.x · Cited by 7,334 · cited 1×
“Natural language tutoring systems implement several wellvalidated instructional strategies, such as active participation in learning [8], adequate feedback [9], collaborative interaction [10], or the presence of impasse or cognitive imbalance [11].”
BackgroundShute, V. J. (2008) · Review of Educational Research · doi:10.3102/0034654307313795 · Cited by 4,631
Experimentation in Software Engineering
Wohlin, C., Runeson, P., Höst, M., Ohlsson, M. C., Regnell, B., & Wesslén, A. (2012) · doi:10.1007/978-3-642-29044-2 · Cited by 4,460
The 2 Sigma Problem: The Search for Methods of Group Instruction as Effective as One-to-One Tutoring
BLOOM, B. S. (1984) · Educational Researcher · doi:10.3102/0013189x013006004 · Cited by 2,734 · cited 1×
“For reference, students working one-on-one with expert human tutors often score 2.0 higher than students working on the same topic in classrooms [106].”
BackgroundInfluentialVanLEHN, K. (2011) · Educational Psychologist · doi:10.1080/00461520.2011.611369 · Cited by 2,323
Advances in natural language processing
Hirschberg, J., & Manning, C. D. (2015) · Science · doi:10.1126/science.aaa8685 · Cited by 1,712
AI in CAI: An Artificial-Intelligence Approach to Computer-Assisted Instruction
Carbonell, J. (1970) · IEEE Transactions on Man Machine Systems · doi:10.1109/tmms.1970.299942 · Cited by 955 · cited 1×
“The term ITS has its origin in the publication of Carbonell [16], in which a system called Scholar is detailed.”
BackgroundEffectiveness of Intelligent Tutoring Systems: A Meta-Analytic Review
Kulik, J. A., & Fletcher, J. D. (2016) · Review of Educational Research · doi:10.3102/0034654315581420 · Cited by 930
Question Asking During Tutoring
Graesser, A. C., & Person, N. K. (1994) · American Educational Research Journal · doi:10.3102/00028312031001104 · Cited by 900 · cited 1×
“…each feature that we present in Table 5 were defined: for the instructional approach from the considered studies; for the support resource we were inspired by the section on complementary media described in [12]; and for the input text format, we based it on the classification established in [81].”
MethodChi, M. T. H., Siler, S. A., Jeong, H., Yamauchi, T., & Hausmann, R. G. (2001) · Cognitive Science · doi:10.1207/s15516709cog2504_1 · Cited by 855
Intelligent tutoring systems and learning outcomes: A meta-analysis
Ma, W., Adesope, O. O., Nesbit, J. C., & Liu, Q. (2014) · Journal of Educational Psychology · doi:10.1037/a0037123 · Cited by 840 · cited 1×
“As a result, we have considered the ITS types mentioned in [4]: model-tracing tutors (MTTs), constraint-based modeling (CBM), Bayesian network modeling (BNM), and expectation and misconceptions tailoring (EMT).”
Eliciting Self‐Explanations Improves Understanding
Chi, M. T. H., De Leeuw, N., Chiu, M., & Lavancher, C. (1994) · Cognitive Science · doi:10.1207/s15516709cog1803_3 · Cited by 757
Mousavinasab, E., Zarifsanaiey, N., R. Niakan Kalhori, S., Rakhshan, M., Keikha, L., & Ghazi Saeedi, M. (2021) · Interactive Learning Environments · doi:10.1080/10494820.2018.1558257 · Cited by 605
Using Bayesian Networks to Manage Uncertainty in Student Modeling
Conati, C., Gertner, A., & VanLehn, K. (2002) · User Modeling and User-Adapted Interaction · doi:10.1023/a:1021258506583 · Cited by 493 · cited 1×
“Additionally, every piece of target domain knowledge has an associated probability that represents the system’s best assumption that the student knows that piece of knowledge [29].”
BackgroundWhen Are Tutorial Dialogues More Effective Than Reading?
VanLehn, K., Graesser, A. C., Jackson, G. T., Jordan, P., Olney, A., & Rosé, C. P. (2007) · Cognitive Science · doi:10.1080/03640210709336984 · Cited by 378
Steenbergen-Hu, S., & Cooper, H. (2014) · Journal of Educational Psychology · doi:10.1037/a0034752 · Cited by 355
Nye, B. D. (2015) · International Journal of Artificial Intelligence in Education · doi:10.1007/s40593-014-0028-6 · Cited by 259
An adaptation algorithm for an intelligent natural language tutoring system
Latham, A., Crockett, K., & McLean, D. (2014) · Computers & Education · doi:10.1016/j.compedu.2013.09.014 · Cited by 113 · cited 1×
“…[54], Lana [55], Beetle II [56], CIRCSIM-Tutor [57], Dialog [58], Geometry Explanation Tutor [59], ITSpoke [60], My Science Tutor [61], Oscar [62], ProPL [63], ReportTutor [64], Rimac [65], RMT [66] and Why2-Atlas [67], after analyzing the way in which they perform the tutoring, we concluded…”
Background“Guru: a computer tutor that models expert human tutors,”
Olney, A. M., D’Mello, S., Person, N., Cade, W., Hays, P., Williams, C., Lehman, B., & Graesser, A. (2012) · Lecture notes in computer science · doi:10.1007/978-3-642-30950-2_32 · Cited by 102
El Saadawi, G. M., Tseytlin, E., Legowski, E., Jukic, D., Castine, M., Fine, J., Gormley, R., & Crowley, R. S. (2008) · Advances in Health Sciences Education · doi:10.1007/s10459-007-9081-3 · Cited by 48
Barriers to ITS Adoption: A Systematic Mapping Study
Nye, B. D. (2014) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-07221-0_74 · Cited by 20
Cited by
Conversational Agents: Goals, Technologies, Vision and Challenges
Allouch, M., Azaria, A., & Azoulay, R. (2021) · Sensors · doi:10.3390/s21248448 · Cited by 168
“There have been multiple attempts to enhance online learning by using intelligent tutoring systems (ITS) [136], which are customized, computer-based instruction and feed-back methods without human intervention.”
BackgroundAI in Education: Enhancing Learning Experiences and Student Outcomes
Xu, Z. (2024) · Applied and Computational Engineering · doi:10.54254/2755-2721/51/20241187 · Cited by 151
“Intelligent tutors use NLP to comprehend students' questions, offer explanations, and assess their understanding [16].”
BackgroundGuo, S., Zheng, Y., & Zhai, X. (2024) · Education and Information Technologies · doi:10.1007/s10639-024-12491-8 · Cited by 117
“In 2020, Paladines and Ramírez conducted a systematic review of 49 studies using ITS with dialogue systems and found that most ITSs aimed at STEM learners at the university level (Paladines & Ramírez, 2020).”
Oliveira, E., Galvao de Barba, P., & Corrin, L. (2021) · Australasian Journal of Educational Technology · doi:10.14742/ajet.6792 · Cited by 50
“Well-known examples of CITS include AutoTutor (Graesser et al., 2004), and Oscar (Latham et al., 2014) (for a more detailed review of CITS see Paladines & Ramírez, 2020).”
MethodLee, H.-Y., Chen, P.-H., Lin, C.-J., Huang, Y.-M., & Wu, T.-T. (2025) · Education and Information Technologies · doi:10.1007/s10639-025-13733-z · Cited by 21
“Recent studies have explored learning analytics and intelligent tutoring systems to support reflective learning in programming education (López-Pernas et al., 2021 ; Paladines & Ramirez, 2020 ).”
Metadata
- Authors
- Paladines, J., & Ramirez, J.
- Venue
- IEEE Access
- Published
- 2020
- Volume
- 8
- Pages
- 164246-164267
- ISSN
- 2169-3536 (ISSN-L)
- OpenAlex
- W3083063794
- Cited by
- 121
- References
- 21
- FWCI
- 1.06
- Type
- journal article
- Language
- en
- Open access
- gold
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.