A ChatBot for Learning Chinese: Learning Achievement and Technology Acceptance
Abstract
ChatBot has potential as a language learning tool, especially for learning Chinese vocabulary. This study aimed to investigate the impact of using a newly developed ChatBot to learn Chinese vocabulary by comparing how it works in different learning environments and to explore the ChatBot with reference to the Technology Acceptance Model (TAM). This study was conducted with 58 students divided into two independent groups. The control group used ChatBot in a one-on-many classroom. The experimental group applied the ChatBot in one-on-one tutor sessions. A pretest and a posttest were used to measure the effect of the ChatBot, while TAM was explored through questionnaire and interview. Data analysis includes a paired-sample t test, analysis of covariance, and levels of effect. The results indicated that the ChatBot significantly improved the students’ learning achievement and that having a one-on-one environment may lead to better outcome than what could be achieved in a classroom. The TAM model was tested using partial least square. The result showed that perceived usefulness was the predictor of behavioral intention, whereas perceived ease of use was not. The students agreed that the ChatBot benefited their learning of Chinese vocabulary, with several adjustments need to be made for further progress.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Extensive efforts have been made to enhance nursing education through programs based on artificial intelligence (AI) that provide personalized interactions.
“Extensive efforts have been made to enhance nursing education through programs based on artificial intelligence (AI) that provide personalized interactions [15].”
From Development and effects of a chatbot education program for self-directed learning in nursing students (2025) · cites this paper as [15]
Prior studies have found that even previous rule-based, non-LLM chatbots improve learning.
“Prior studies have found that even previous rule-based, non-LLM chatbots improve learning (Abbasi and Kazi 2014, Chen et al. 2020, Yin et al. 2021, Chang et al. 2022, Essel et al. 2022, Ait Baha et al. 2023).”
From AI Meets the Classroom: When Do Large Language Models Harm Learning? (2024) · cites this paper as Chen et al. 2020
The vast majority of chatbots have been employed as teaching agents and assessed using the experimental approach, and the findings largely point to improved learning performance and satisfaction.
“The vast majority of chatbots have been employed as teaching agents and assessed using the experimental approach, and the findings largely point to improved learning performance and satisfaction (e.g., Pereira et al., 2019; Chen et al., 2020; Yin et al., 2021; Khalil and Rambech, 2022).”
From Implementing the bashayer chatbot in Saudi higher education: measuring the influence on students' motivation and learning strategies (2023) · cites this paper as Chen et al., 2020
Similar results on self-efficacy have been found in previous studies such as Chen et al. (2020), Pérez et al. (2020), and Chang et al. (2022).
“Similar results have been found in previous studies such as Chen et al. (2020), Pérez et al. (2020), and Chang et al. (2022).”
From Implementing the bashayer chatbot in Saudi higher education: measuring the influence on students' motivation and learning strategies (2023) · cites this paper as Chen et al. (2020)
The study's results on learning motivation align with previous research on chatbot systems' potential to support learning motivation.
“These results align with previous research studies on chatbot systems' potential to support learning motivation (Nghi et al., 2019; Chen et al., 2020; Yin et al., 2021; Fidan and Gencel, 2022; Khalil and Rambech, 2022).”
From Implementing the bashayer chatbot in Saudi higher education: measuring the influence on students' motivation and learning strategies (2023) · cites this paper as Chen et al., 2020
Cites
Evaluating structural equation models with unobservable variables and measurement error
Fornell, C., & Larcker, D. F. (1981) · Journal of Marketing Research · doi:10.1177/002224378101800104 · Cited by 109,190 · see citing works
Perceived usefulness, perceived ease of use, and user acceptance of information technology
Davis, F. D. (1989) · MIS Quarterly · doi:10.2307/249008 · Cited by 68,483 · see citing works
On the Evaluation of Structural Equation Models
Bagozzi, R. P., & Yi, Y. (1988) · Journal of the Academy of Marketing Science · doi:10.1007/bf02723327 · Cited by 26,163 · see citing works
User Acceptance of Computer Technology: A Comparison of Two Theoretical Models
Davis, F. D., Bagozzi, R. P., & Warshaw, P. R. (1989) · Management Science · doi:10.1287/mnsc.35.8.982 · Cited by 25,709 · see citing works
Venkatesh, V. (2000) · Information Systems Research · doi:10.1287/isre.11.4.342.11872 · Cited by 6,664 · see citing works
Modern Methods for Business Research
(1998) · doi:10.4324/9781410604385 · Cited by 6,534 · see citing works
Extrinsic and Intrinsic Motivation to Use Computers in the Workplace <sup>1</sup>
Davis, F. D., Bagozzi, R. P., & Warshaw, P. R. (1992) · Journal of Applied Social Psychology · doi:10.1111/j.1559-1816.1992.tb00945.x · Cited by 6,231 · cited 2× · see citing works
“How people expect a technology to improve their job performance defines their PU of the technology (Davis et al., 1992).”
BackgroundThe 2 Sigma Problem: The Search for Methods of Group Instruction as Effective as One-to-One Tutoring
BLOOM, B. S. (1984) · Educational Researcher · doi:10.3102/0013189x013006004 · Cited by 2,734 · cited 10× · see citing works
“ChatBot, a version of virtual assistant, is a program that is able to imitate human conversation (Tatai et al., 2003). Several studies have investigated the possibility of using ChatBot in language teaching and learning (Fryer et al., 2019; Jia, 2004; Lu et al., 2006) and found it to be an effective means of stimulati…”
BackgroundMethodInfluentialYi, M. Y., & Hwang, Y. (2003) · International Journal of Human-Computer Studies · doi:10.1016/s1071-5819(03)00114-9 · Cited by 1,306 · see citing works
Disentangling behavioral intention and behavioral expectation
Warshaw, P. R., & Davis, F. D. (1985) · Journal of Experimental Social Psychology · doi:10.1016/0022-1031(85)90017-4 · Cited by 963 · see citing works
Meta-Analysis of Research on Class Size and Achievement
Glass, G., & Smith, M. (1979) · Educational Evaluation and Policy Analysis · doi:10.3102/01623737001001002 · Cited by 599 · see citing works
Impact of media richness and flow on e-learning technology acceptance
Liu, S.-H., Liao, H.-L., & Pratt, J. A. (2009) · Computers & Education · doi:10.1016/j.compedu.2008.11.002 · Cited by 594 · see citing works
Chatbot learning partners: connecting learning experiences, interest and competence
Fryer, L. K., Nakao, K., & Thompson, A. (2019) · Computers in Human Behavior · doi:10.1016/j.chb.2018.12.023 · Cited by 470 · see citing works
Second language reading and vocabulary learning
Huckin, T. N. (1995) · Medical Entomology and Zoology · Cited by 438 · see citing works
Nikou, S. A., & Economides, A. A. (2016) · Computers in Human Behavior · doi:10.1016/j.chb.2016.11.020 · Cited by 416 · see citing works
What Makes Literacy Tutoring Effective?
Juel, C. L. (1996) · Reading Research Quarterly · doi:10.1598/rrq.31.3.3 · Cited by 263 · see citing works
Immersive Virtual Reality as an Effective Tool for Second Language Vocabulary Learning
Legault, J., Zhao, J., Chi, Y.-A., Weitao, C., Klippel, A., & Li, P. (2019) · Languages · doi:10.3390/languages4010013 · Cited by 228 · see citing works
Applying the Technology Acceptance Model to Online Learning in the Egyptian Universities
فرحات, ط. ع. ا. (2012) · Procedia - Social and Behavioral Sciences · doi:10.1016/j.sbspro.2012.11.012 · Cited by 220 · see citing works
Linguistics in Language Teaching
Runnalls, G. A., & Wilkins, D. A. (1973) · The Modern Language Review · doi:10.2307/3725001 · Cited by 210 · see citing works
Does ease-of-use contributes to the perception of enjoyment? A case of gamification in e-banking
Rodrigues, L. F., Oliveira, A., & Costa, C. J. D. C. (2016) · Computers in Human Behavior · doi:10.1016/j.chb.2016.03.015 · Cited by 202 · see citing works
Effectiveness of Mobile Devices on Vocabulary Learning: A Meta-Analysis
Mahdi, H. S. (2017) · Journal of Educational Computing Research · doi:10.1177/0735633117698826 · Cited by 160 · see citing works
Edge, D., Searle, E., Chiu, K., Zhao, J. H., & Landay, J. A. (2011) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/1978942.1979413 · Cited by 143 · see citing works
Classes That Click: Fast, Rich Feedback to Enhance Student Learning and Satisfaction
Chen, J. C., Whittinghill, D. C., & Kadlowec, J. A. (2010) · Journal of Engineering Education · doi:10.1002/j.2168-9830.2010.tb01052.x · Cited by 111 · see citing works
Kozlov, M. D., & Johansen, M. K. (2010) · Cyberpsychology Behavior and Social Networking · doi:10.1089/cyber.2009.0310 · Cited by 108 · see citing works
Chan, S. C. H., Wan, C. L. J., & Ko, S. J. (2019) · The International Journal of Management Education · doi:10.1016/j.ijme.2018.12.004 · Cited by 100 · see citing works
Mostow, J., Aist, G., Burkhead, P., Corbett, A. T., Cuneo, A., Eitelman, S. M., Huang, C., Junker, B. W., Sklar, M. B., & Tobin, B. (2003) · Journal of Educational Computing Research · doi:10.2190/06ax-qw99-eq5g-rdcf · Cited by 94 · see citing works
Understanding technology acceptance features in learning through a serious game
Malaquias, R. F., Malaquias, F. F. D. O., & Hwang, Y. (2018) · Computers in Human Behavior · doi:10.1016/j.chb.2018.06.008 · Cited by 75 · see citing works
Liao, C., Tsou, C., & Shu, Y.-C. (2008) · International journal of business and information/International journal of business & information · doi:10.6702/ijbi.2008.3.1.2 · Cited by 70 · see citing works
The Study of the Application of a Web-Based Chatbot System on the Teaching of Foreign Languages
Jia, J. (2004) · Society for Information Technology & Teacher Education International Conference · Cited by 65 · see citing works
Rote Memorization of Vocabulary and Vocabulary Development
Yang, W., & Dai, W. (2011) · English Language Teaching · doi:10.5539/elt.v4n4p61 · Cited by 60 · see citing works
Tsai, Y.-R. (2015) · CALICO Journal · doi:10.1558/calico.v32i1.25961 · Cited by 60 · see citing works
Hussain, A., Mkpojiogu, E. O. C., & Yusof, M. (2016) · AIP conference proceedings · doi:10.1063/1.4960891 · Cited by 59 · see citing works
Chiva-Bartoll, Ò., Miravet, L. M., & Salvador-García, C. (2020) · Children and Youth Services Review · doi:10.1016/j.childyouth.2020.104841 · Cited by 50 · see citing works
Chen, B., Sivo, S. A., Seilhamer, R. M., Sugar, A. B., & Mao, J. J. (2013) · Journal of Educational Computing Research · doi:10.2190/ec.49.3.c · Cited by 48 · see citing works
Mastery versus nonmastery instruction with varying test item feedback treatments.
Wentling, T. L. (1973) · Journal of Educational Psychology · doi:10.1037/h0034820 · Cited by 47 · see citing works
Languages learning strategies and the vocabulary size
Nacera, A. (2010) · Procedia - Social and Behavioral Sciences · doi:10.1016/j.sbspro.2010.03.634 · Cited by 43 · see citing works
Using Instant Messaging to Provide an Intelligent Learning Environment
Lu, C.-H., Chiou, G., Day, M.-Y., Ong, C., & Hsu, W. (2006) · Lecture notes in computer science · doi:10.1007/11774303_57 · Cited by 34 · see citing works
Preference for Anonymous Classroom Participation
Latham, A., & Hill, N. S. (2013) · Organizational Behavior Teaching Review · doi:10.1177/1052562913488109 · Cited by 33 · see citing works
Tatai, G., Csordás, A., Kiss, Á., Szaló, A., & Laufer, L. (2003) · Lecture notes in computer science · doi:10.1007/978-3-540-39396-2_2 · Cited by 27 · see citing works
Vocabulary Learning Strategies among Malaysian TEVT Students in German-Malaysian Institute (GMI)
Mutalib, A. H. B. A., Kadir, R. B. A., Robani, R. B., et al. (2014) · Procedia - Social and Behavioral Sciences · doi:10.1016/j.sbspro.2014.01.1434 · Cited by 25 · see citing works
Fokides, E., & Atsikpasi, P. (2018) · Education and Information Technologies · doi:10.1007/s10639-018-9719-1 · Cited by 19 · see citing works
Moody, W. B., Bausell, R. B., & Jenkins, J. R. (1973) · Journal for Research in Mathematics Education · doi:10.2307/748512 · Cited by 18 · see citing works
Evaluating user experience in joint activities between schools and museums in virtual worlds
Barneche-Naya, V., & Hernández-Ibáñez, L. A. (2014) · Universal Access in the Information Society · doi:10.1007/s10209-014-0367-y · Cited by 16 · see citing works
Jiménez, M. Á. F., Rodríguez, E. M., & Vidal, L. I. E. (2017) · Procedia - Social and Behavioral Sciences · doi:10.1016/j.sbspro.2017.02.063 · Cited by 16 · see citing works
(2016) · Revista de la Facultad de Ingeniería · doi:10.21311/002.31.5.04 · Cited by 15 · see citing works
Effects of Learner Ability and Instructional Modifications on Time Needed for Learning and Retention
Gettinger, M. (1983) · The Journal of Educational Research · doi:10.1080/00220671.1983.10885484 · Cited by 10 · see citing works
Park, R. E. (1937) · American Journal of Sociology · doi:10.1086/217650 · Cited by 7 · see citing works
Sequential Exercises and Personal Response System in Project Management Courses
Gharaie, E. (2016) · Procedia - Social and Behavioral Sciences · doi:10.1016/j.sbspro.2016.07.016 · Cited by 4 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
Are We There Yet? - A Systematic Literature Review on Chatbots in Education
Wollny, S., Schneider, J., Di Mitri, D., Weidlich, J., Rittberger, M., & Drachsler, H. (2021) · Frontiers in Artificial Intelligence · doi:10.3389/frai.2021.654924 · Cited by 557 · see citing works
Chocarro, R., Cortiñas, M., & Marcos-Matás, G. (2023) · Educational Studies · doi:10.1080/03055698.2020.1850426 · Cited by 343 · see citing works
Wang, X., Li, L., Tan, S. C., Yang, L., & Lei, J. (2023) · Computers in Human Behavior · doi:10.1016/j.chb.2023.107798 · Cited by 336 · see citing works
Lee, Y.-F., Hwang, G.-J., & Chen, P.-Y. (2022) · Educational Technology Research and Development · doi:10.1007/s11423-022-10142-8 · Cited by 323 · see citing works
Deng, X., & Yu, Z. (2023) · Sustainability · doi:10.3390/su15042940 · Cited by 298 · see citing works
“The tutorial role required chatbots to offer questions and answers, guided students to start their learning [17], and give quizzes [18].”
BackgroundKumar, J. A. (2021) · International Journal of Educational Technology in Higher Education · doi:10.1186/s41239-021-00302-w · Cited by 183 · see citing works
The effect of artificial intelligence tools on EFL learners' engagement, enjoyment, and motivation
Yuan, L., & Liu, X. (2025) · Computers in Human Behavior · doi:10.1016/j.chb.2024.108474 · Cited by 188 · see citing works
Chatbot application in a 5th grade science course
Deveci Topal, A., Dilek Eren, C., & Kolburan Geçer, A. (2021) · Education and Information Technologies · doi:10.1007/s10639-021-10627-8 · Cited by 182 · see citing works
The impact of chatbots based on large language models on second language vocabulary acquisition
Zhang, Z., & Huang, X. (2024) · Heliyon · doi:10.1016/j.heliyon.2024.e25370 · Cited by 107 · see citing works
Al-Abdullatif, A. M. (2023) · Education Sciences · doi:10.3390/educsci13111151 · Cited by 102 · see citing works
A bibliometric analysis of artificial intelligence chatbots in educational contexts
Lin, Y., & Yu, Z. (2024) · Interactive Technology and Smart Education · doi:10.1108/itse-12-2022-0165 · Cited by 99 · see citing works
Al-Abdullatif, A. M., Al-Dokhny, A. A., & Drwish, A. M. (2023) · Frontiers in Psychology · doi:10.3389/fpsyg.2023.1129070 · Cited by 80 · see citing works
“Chen et al. (2020) Examining the influence of employing a designed chatbot for acquiring Chinese language.”
BackgroundResultInfluentialWang, F., Cheung, A. C. K., Neitzel, A. J., & Chai, C. S. (2025) · Review of Educational Research · doi:10.3102/00346543241255621 · Cited by 64 · see citing works
Using ChatGPT for medical education: the technical perspective
Chan, K. Y., Yuen, T. H., & Co, M. (2025) · BMC Medical Education · doi:10.1186/s12909-025-06785-9 · Cited by 34 · see citing works
Han, J.-W., Park, J., & Lee, H. (2025) · BMC Medical Education · doi:10.1186/s12909-025-07316-2 · Cited by 28 · see citing works
AI Meets the Classroom: When Do Large Language Models Harm Learning?
Lehmann, M., Cornelius, P. B., & Sting, F. J. (2024) · arXiv · doi:10.48550/arxiv.2409.09047 · Cited by 11 · see citing works
- Authors
- Chen, H.-L., Vicki Widarso, G., & Sutrisno, H.
- Affiliations
- National Taiwan University of Science and Technology (TW)
- Graduate Institute of Digital Learning and Education, National Taiwan University of Science and Technology
- Corresponding author
- Chen, Hsiu-Ling
- Venue
- Journal of Educational Computing Research
- Publisher
- SAGE Publications
- Published
- 2020-06-10
- Volume
- 58
- Issue
- 6
- Pages
- 1161-1189
- ISSN
- 0735-6331 (ISSN-L)
- 1541-4140
- OpenAlex
- W3034267466
- Cited by
- 220
- References
- 48
- Citations by year
- 2026: 22 · 2025: 61 · 2024: 60 · 2023: 43 · 2022: 24 · 2021: 7 · 2020: 3
- FWCI
- 7.56
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Journal of Educational Computing Research (publishedVersion)
- Funders
- Ministry of Science and Technology, Taiwan (MOST 108-2511-H-011-007-MY2)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Intelligent Tutoring Systems and Adaptive Learning
- Physical Sciences › Computer Science › Information Systems › Mobile Learning in Education
- Physical Sciences › Computer Science › Information Systems › Artificial Intelligence in Education
- Keywords
- chatbots, Chinese as a foreign language, vocabulary acquisition, technology acceptance model, perceived usefulness, perceived ease of use, behavioral intention, one-on-one tutoring, academic performance
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
