Measuring Effectiveness of Learning Chatbot Systems on Student's Learning Outcome and Memory Retention
Abstract
Creating a learning environment in which students learn more effectively remains the great challenge from decades; different approaches are proposed, for example, Intelligent Tutoring Systems, Question Answering System and chatbot. All these approaches used natural language to achieve that goal. A comparison of these systems viz-a-viz student learning outcome and behavior is of eminent importance. To achieve this goal a chatbot system with knowledge base for Object-Oriented Programming Languages is developed and deployed. Case study was made to assess and evaluate the chatbot system for student learning methodology. Learning outcomes and Memory retention have been measured for the developed system. Comparisons were made between the results obtained using Google search engine and our chatbot system. The results indicate that learning through Chabot have a significant impact on memory retention and Learning outcomes of the students.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The choice of a relatively-large effect size of 0.8 in this study was based on previous research examining the effectiveness of chatbots targeting college students, all of which reported effect sizes greater than 0.8.
“The choice of a relatively-large effect size of 0.8 in this study was based on previous research examining the effectiveness of chatbots targeting college students [26, 27], all of whom reported effect sizes greater than 0.8.”
From Development and effects of a chatbot education program for self-directed learning in nursing students (2025) · cites this paper as [26]
Prior studies have found that even previous rule-based, non-LLM chatbots improve learning.
“Prior studies have found that even previous rule-based, non-LLM chatbots improve learning (Abbasi and Kazi 2014, Chen et al. 2020, Yin et al. 2021, Chang et al. 2022, Essel et al. 2022, Ait Baha et al. 2023).”
From AI Meets the Classroom: When Do Large Language Models Harm Learning? (2024) · cites this paper as Abbasi and Kazi 2014
Cites
Mayer, R. (2020) · doi:10.1017/9781316941355 · Cited by 7,777 · see citing works
Dunlosky, J., Rawson, K. A., Marsh, E. J., Nathan, M. J., & Willingham, D. T. (2013) · Psychological Science in the Public Interest · doi:10.1177/1529100612453266 · Cited by 3,138 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
Are We There Yet? - A Systematic Literature Review on Chatbots in Education
Wollny, S., Schneider, J., Di Mitri, D., Weidlich, J., Rittberger, M., & Drachsler, H. (2021) · Frontiers in Artificial Intelligence · doi:10.3389/frai.2021.654924 · Cited by 557 · see citing works
Chang, D. H., Lin, M. P.-C., Hajian, S., & Wang, Q. Q. (2023) · Sustainability · doi:10.3390/su151712921 · Cited by 370 · see citing works
“To our best knowledge, the design of chatbots has focused greatly on the backend design [43], user interface [44], and improving learning [36,45,46].”
Deng, X., & Yu, Z. (2023) · Sustainability · doi:10.3390/su15042940 · Cited by 298 · see citing works
“(2019) [63] 110 (55/55) Not available Teaching assistant Computer science Learning achievement 28 Abbasi and Kazi (2014) [64] 72 (36/36) Not available Teaching assistant Computer science Learning achievement 29 Lin and Chang (2020) [65] 357 (167/190) 2 weeks Teaching assistant Language Learning achievement”
BackgroundHan, J.-W., Park, J., & Lee, H. (2022) · BMC Medical Education · doi:10.1186/s12909-022-03898-3 · Cited by 187 · see citing works
Al-Abdullatif, A. M., Al-Dokhny, A. A., & Drwish, A. M. (2023) · Frontiers in Psychology · doi:10.3389/fpsyg.2023.1129070 · Cited by 80 · see citing works
“This positive result could be attributed to the instant, flexible retrieval of course information (Chuah and Kabilan, 2021), which might positively influence students’ memory retention (Abbasi and Kazi, 2014), skill acquisition, and overall learning performance (Lin and Mubarok, 2021; Okonkwo and Ade-Ibijola, 2021; Pé…”
BackgroundChatbots in Healthcare Curricula: The Case of a Conversational Virtual Patient
Dolianiti, F., Tsoupouroglou, I., Antoniou, P., Konstantinidis, S., Anastasiades, S., & Bamidis, P. (2020) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-60735-7_15 · Cited by 37 · see citing works
Han, J.-W., Park, J., & Lee, H. (2025) · BMC Medical Education · doi:10.1186/s12909-025-07316-2 · Cited by 28 · see citing works
“The choice of a relatively-large effect size of 0.8 in this study was based on previous research examining the effectiveness of chatbots targeting college students [26, 27], all of whom reported effect sizes greater than 0.8.”
InfluentialAI Meets the Classroom: When Do Large Language Models Harm Learning?
Lehmann, M., Cornelius, P. B., & Sting, F. J. (2024) · arXiv · doi:10.48550/arxiv.2409.09047 · Cited by 11 · see citing works
- Authors
- Abbasi, S., & Kazi, H.
- Affiliations
- Sindh Agriculture University (PK)
- Isra University (PK)
- Venue
- Asian Journal of Applied Science and Engineering
- Publisher
- ABC Journals
- Published
- 2014-08-31
- Volume
- 3
- Issue
- 7
- Pages
- 57
- ISSN
- 2305-915X (ISSN-L)
- 2307-9584
- OpenAlex
- W2045214467
- Cited by
- 67
- References
- 2
- Citations by year
- 2026: 2 · 2025: 7 · 2024: 12 · 2023: 11 · 2022: 12 · 2021: 4 · 2020: 12 · 2019: 5 · 2018: 2
- FWCI
- 0.00
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Asian Journal of Applied Science and Engineering (publishedVersion)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Intelligent Tutoring Systems and Adaptive Learning
- Physical Sciences › Computer Science › Information Systems › Artificial Intelligence in Education
- Physical Sciences › Computer Science › Artificial Intelligence › Edcuational Technology Systems
- Keywords
- object-oriented programming, memory retention, learning outcomes, knowledge base, intelligent tutoring systems, question answering systems, student learning
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
