Effect of an AI-based chatbot on students’ learning performance in alternate reality game-based museum learning
Abstract
Abstract Recently, alternate reality games (ARGs) have been applied in museum learning to encourage learners' active engagement through playful problem‐solving activities. However, most learners have insufficient prior knowledge and metacognitive skills to complete the learning tasks in such games. To support learning with ARGs, there is a need to provide proper feedback so that the learners are capable of self‐regulated learning and solving the problems encountered during the learning process. Considering the influences of individual differences, this research developed a chatbot as a learning partner in ARG‐based learning to support students’ learning with adaptive feedback in a museum. A quasi‐experimental study was conducted to evaluate the effectiveness with and without the proposed approach. The results showed that the AI‐based chatbot approach could significantly improve learners' metacognition awareness, emotional engagement and behavioural engagement. Besides, it is helpful to facilitate students' double‐loop learning. Practitioner notes What is already known about this topic Alternate reality games (ARGs) enable students to interact with real‐world contexts. The complexities of real contexts and ambiguities of the ARGs could frustrate students and demotivate their engagement in problem‐solving activities. Conventional corrected feedback may fail to match individual students' requirements in solving encountered problems. What this paper adds An AI‐based chatbot was designed as a smart learning partner to support the students in solving encountered problems. The experimental results showed that the proposed approach improved students' metacognitive awareness and engagement. A drawing analysis and a behaviour sequential analysis were applied to further examine the students' perceptions and behavioural patterns. Implications for practice and/or policy It is potentially worth integrating AI‐based chatbots into game‐based learning to promote students' metacognitive awareness and engagement. AI‐based chatbots provide adaptive feedback to effectively facilitate students' self‐regulated learning and double‐loop learning.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Cites
Power failure: why small sample size undermines the reliability of neuroscience
Button, K. S., Ioannidis, J. P. A., Mokrysz, C., Nosek, B. A., Flint, J., Robinson, E. S. J., & Munafò, M. R. (2013) · Nature Reviews Neuroscience · doi:10.1038/nrn3475 · Cited by 7,641 · see citing works
Assessing Metacognitive Awareness
Schraw, G., & Dennison, R. S. (1994) · Contemporary Educational Psychology · doi:10.1006/ceps.1994.1033 · Cited by 3,199 · see citing works
Interaction between learning and development
(1980) · Mind in Society · doi:10.2307/j.ctvjf9vz4.11 · Cited by 2,150 · see citing works
What Is the Impact of ChatGPT on Education? A Rapid Review of the Literature
Lo, C. K. (2023) · Education Sciences · doi:10.3390/educsci13040410 · Cited by 2,024 · see citing works
What if the devil is my guardian angel: ChatGPT as a case study of using chatbots in education
Tlili, A., Shehata, B., Adarkwah, M. A., Bozkurt, A., Hickey, D. T., Huang, R., & Agyemang, B. (2023) · Smart Learning Environments · doi:10.1186/s40561-023-00237-x · Cited by 2,003 · see citing works
Grassini, S. (2023) · Education Sciences · doi:10.3390/educsci13070692 · Cited by 1,048 · see citing works
Huang, W., Hew, K. F., & Fryer, L. K. (2022) · Journal of Computer Assisted Learning · doi:10.1111/jcal.12610 · Cited by 830 · see citing works
Chatbots for learning: A review of educational chatbots for the Facebook Messenger
Smutny, P., & Schreiberova, P. (2020) · Computers & Education · doi:10.1016/j.compedu.2020.103862 · Cited by 770 · see citing works
Berge, Z. L., & Muilenburg, L. Y. (2013) · Routledge eBooks · doi:10.4324/9780203118764 · Cited by 489 · see citing works
Rediscovering the use of chatbots in education: A systematic literature review
Pérez, J. Q., Daradoumis, T., & Puig, J. M. M. (2020) · Computer Applications in Engineering Education · doi:10.1002/cae.22326 · Cited by 483 · see citing works
Chatbot learning partners: connecting learning experiences, interest and competence
Fryer, L. K., Nakao, K., & Thompson, A. (2019) · Computers in Human Behavior · doi:10.1016/j.chb.2018.12.023 · Cited by 470 · see citing works
Meta-design: A Framework for the Future of End-User Development
Fischer, G., & Giaccardi, E. (2006) · Springer eBooks · doi:10.1007/1-4020-5386-x_19 · Cited by 417 · see citing works
Huang, B., Hew, K. F., & Lo, C. K. (2019) · Interactive Learning Environments · doi:10.1080/10494820.2018.1495653 · Cited by 406 · see citing works
ChatGPT for Education and Research: A Review of Benefits and Risks
Sok, S., & Heng, K. (2023) · doi:10.2139/ssrn.4378735 · Cited by 386 · see citing works
Yin, J., Goh, T.-T., Yang, B., & Xiaobin, Y. (2021) · Journal of Educational Computing Research · doi:10.1177/0735633120952067 · Cited by 349 · see citing works
Feedback loops and the longer-term: towards feedback spirals
Carless, D. (2019) · Assessment & Evaluation in Higher Education · doi:10.1080/02602938.2018.1531108 · Cited by 328 · see citing works
Foundation for problem‐based gaming
Kiili, K. J. M. (2007) · British Journal of Educational Technology · doi:10.1111/j.1467-8535.2007.00704.x · Cited by 295 · see citing works
Chang, C., Hwang, G., & Gau, M. (2022) · British Journal of Educational Technology · doi:10.1111/bjet.13158 · Cited by 294 · see citing works
Self-citationDeeva, G., Bogdanova, D., Serral, E., Snoeck, M., & Weerdt, J. D. (2020) · Computers & Education · doi:10.1016/j.compedu.2020.104094 · Cited by 292 · see citing works
Hew, K. F., Huang, W., Du, J., & Jia, C. (2023) · Journal of Computing in Higher Education · doi:10.1007/s12528-022-09338-x · Cited by 245 · see citing works
Self-Regulated Learning: The Interactive Influence of Metacognitive Awareness and Goal-Setting
Ridley, D. S., Schutz, P. A., Glanz, R. S., & Weinstein, C. E. (1992) · The Journal of Experimental Education · doi:10.1080/00220973.1992.9943867 · Cited by 222 · see citing works
An alternate reality game for language learning: ARGuing for multilingual motivation
Connolly, T. M., Stansfield, M., & Hainey, T. (2011) · Computers & Education · doi:10.1016/j.compedu.2011.01.009 · Cited by 193 · see citing works
Ninaus, M., Greipl, S., Kiili, K. J. M., Lindstedt, A., Huber, S., Klein, E., Karnath, H., & Moeller, K. (2019) · Computers & Education · doi:10.1016/j.compedu.2019.103641 · Cited by 189 · see citing works
Bibauw, S., François, T., & Desmet, P. (2019) · Computer Assisted Language Learning · doi:10.1080/09588221.2018.1535508 · Cited by 177 · see citing works
The formation of visitor behavioral intention to creative tourism: The role of push–Pull motivation
Dean, D., & Suhartanto, D. (2019) · Asia Pacific Journal of Tourism Research · doi:10.1080/10941665.2019.1572631 · Cited by 177 · see citing works
Enhancing problem-solving skills with smart personal assistant technology
Winkler, R., Söllner, M., & Leimeister, J. M. (2021) · Computers & Education · doi:10.1016/j.compedu.2021.104148 · Cited by 134 · see citing works
(2021) · DOAJ (DOAJ: Directory of Open Access Journals) · Cited by 127 · see citing works
What affects learner engagement in flipped learning and what predicts its outcomes?
Lee, J., Park, T., & Davis, R. O. (2018) · British Journal of Educational Technology · doi:10.1111/bjet.12717 · Cited by 121 · see citing works
Engaging Museum Visitors with AI: The Case of Chatbots
Gaia, G., Boiano, S., & Borda, A. (2019) · Springer series on cultural computing · doi:10.1007/978-3-319-97457-6_15 · Cited by 93 · see citing works
A cultural heritage framework using a Deep Learning based Chatbot for supporting tourist journey
Sperlí, G. (2021) · Expert Systems with Applications · doi:10.1016/j.eswa.2021.115277 · Cited by 92 · see citing works
Yılmaz, F. G. K., & Yılmaz, R. (2019) · Computers & Education · doi:10.1016/j.compedu.2019.02.001 · Cited by 87 · see citing works
Gilliam, M. L., Jagoda, P., Fabiyi, C. A., Lyman, P., Wilson, M. C., Hill, B. J., & Bouris, A. M. (2017) · Journal of Science Education and Technology · doi:10.1007/s10956-016-9679-4 · Cited by 57 · see citing works
Yang, K.-H. (2017) · Interactive Learning Environments · doi:10.1080/10494820.2017.1286099 · Cited by 45 · see citing works
Siew, N. M., & Saidi, S. S. (2019) · International Electronic Journal of Mathematics Education · doi:10.29333/iejme/5755 · Cited by 37 · see citing works
A qualitative exploration of student perspectives of chatbot use during emergency remote teaching
Kohnke, L. (2022) · International Journal of Mobile Learning and Organisation · doi:10.1504/ijmlo.2022.125966 · Cited by 36 · see citing works
Educational games and students’ game engagement in elementary school classrooms
Kim, S., Chang, M., Deater-Deckard, K. D., Evans, M. A., Norton, A., & Samur, Y. (2017) · Journal of Computers in Education · doi:10.1007/s40692-017-0095-4 · Cited by 34 · see citing works
Flow Experience as a Quality Measure in Evaluating Physically Activating Collaborative Serious Games
Kiili, K. J. M., Perttula, A., Lindstedt, A., et al. (2014) · International Journal of Serious Games · doi:10.17083/ijsg.v1i3.23 · Cited by 32 · see citing works
Alternate reality games as learning environments for student induction
Whitton, N., Jones, R., Wilson, S. J., et al. (2012) · Interactive Learning Environments · doi:10.1080/10494820.2011.641683 · Cited by 25 · see citing works
Hsu, T., & Liang, H. (2021) · Library Hi Tech · doi:10.1108/lht-08-2020-0198 · Cited by 20 · see citing works
Self-citationPhenomenon Based Learning Implemented in Abu Dhabi School Model
Valanne, E., Dhaheri, R. M. A., Kylmalahti, R., et al. (2017) · International Journal of Humanities and Social Sciences · Cited by 19 · see citing works
Elsom, S., Westacott, M., Stieler‐Hunt, C., Glencross, S., & Rutter, K. (2021) · Interactive Learning Environments · doi:10.1080/10494820.2021.1894181 · Cited by 16 · see citing works
Alternate reality games (ARG) as innovative digital information sources
Beer, D., & Bothma, T. (2016) · Library Hi Tech · doi:10.1108/lht-02-2016-0018 · Cited by 14 · see citing works
Motivated to lose? Evaluating challenge and player motivations in games
Tornqvist, D., & Tichon, J. (2019) · Behaviour and Information Technology · doi:10.1080/0144929x.2019.1672789 · Cited by 12 · see citing works
Exercising library and information literacies through alternate reality gaming
Jerrett, A., Bothma, T., & Beer, D. (2017) · Aslib Journal of Information Management · doi:10.1108/ajim-11-2016-0185 · Cited by 11 · see citing works
Meekaew, N., & Ketpichainarong, W. (2021) · International Journal of Mobile Learning and Organisation · doi:10.1504/ijmlo.2021.116509 · Cited by 10 · see citing works
Tulloch, R., Wolfenden, H., & Sercombe, H. (2021) · Media Practice and Education · doi:10.1080/25741136.2020.1864179 · Cited by 5 · see citing works
Tao, X., Goh, W. P., Zhang, J., et al. (2021) · International Journal of Mobile Learning and Organisation · doi:10.1504/ijmlo.2021.118436 · Cited by 4 · see citing works
Koepfler, J. A., Sneeringer, K. K., & Goodlander, G. (2017) · RIT Scholar Works (Rochester Institute of Technology) · Cited by 2 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
Retrieval-augmented generation for educational application: A systematic survey
Li, Z., Wang, Z., Wang, W., Hung, K., Xie, H., & Wang, F. L. (2025) · Computers and Education: Artificial Intelligence · doi:10.1016/j.caeai.2025.100417 · Cited by 151 · see citing works
Wei, S., Wei, W., & Zou, G. (2026) · Frontiers in Medicine · doi:10.3389/fmed.2026.1832598 · Cited by 2 · see citing works
“In one study, an AI chatbot named Eva was integrated into an online peer review system, significantly improving the quality of student feedback (41).”
- Authors
- Liang, H., Hwang, G., Hsu, T., & Yeh, J.
- Affiliations
- I-Shou University (TW)
- National Taiwan University of Science and Technology (TW)
- National Museum of Natural Science (TW)
- Corresponding author
- Hwang, Gwo‐Jen
- Venue
- British Journal of Educational Technology
- Publisher
- Wiley
- Published
- 2024-02-27
- Volume
- 55
- Issue
- 5
- Pages
- 2315-2338
- ISSN
- 0007-1013 (ISSN-L)
- 1467-8535
- OpenAlex
- W4392192640
- Cited by
- 78
- References
- 48
- Citations by year
- 2026: 41 · 2025: 32 · 2024: 5
- FWCI
- 11.49
- Type
- journal article
- Language
- en
- Open access
- closed
- Licence
- Locations
- British Journal of Educational Technology (publishedVersion)
- Funders
- National Science and Technology Council
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Intelligent Tutoring Systems and Adaptive Learning
- Social Sciences › Psychology › Developmental and Educational Psychology › Educational Games and Gamification
- Physical Sciences › Computer Science › Information Systems › Artificial Intelligence in Education
- Keywords
- alternate reality games, museum learning, AI chatbots, adaptive feedback, metacognitive awareness, student self-regulation, double-loop learning, emotional engagement, behavioral engagement, game-based learning, individual differences, learning partners
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review