Scientific and Fantastical: Creating Immersive, Culturally Relevant Learning Experiences with Augmented Reality and Large Language Models
Abstract
Motivating children to learn is a major challenge in education. One way to inspire motivation to learn is through immersion. We combine the immersive potential of augmented reality (AR), narrative, and large language models (LLMs) to bridge fantasy with reality in a mobile application, Moon Story, that teaches elementary schoolers astronomy and environmental science. Our system also builds upon learning theories such as culturally-relevant pedagogy. Using our application, a child embarks on a journey inspired by Chinese mythology, engages in real-world AR activities, and converses with a fictional character powered by an LLM. We conducted a controlled experiment (N = 50) with two conditions: one using an LLM and one that was hard-coded. Both conditions resulted in learning gains, high engagement levels, and increased science learning motivation. Participants in the LLM condition also wrote more relevant answers. Finally, participants of both Chinese and non-Chinese heritage found the culturally-based narrative compelling.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Studies addressed persistent challenges in traditional curricula, such as difficulties in understanding poetry, classical literature, or cultural symbolism, using AI to help students engage with texts more deeply through visual and multimodal support.
“These studies addressed persistent challenges in traditional curricula, such as difficulties in understanding poetry, classical literature, or cultural symbolism. AI was used to help students engage with texts more deeply through visual and multimodal support [70-74].”
From A Systematic Review of Generative AI in K–12: Mapping Goals, Activities, Roles, and Outcomes via the 3P Model (2025) · cites this paper as [74]
Numerous studies report higher enthusiasm, participation, curiosity, and satisfaction after AI use, alongside a general preference for AI tools as efficient and convenient.
“Numerous studies report higher enthusiasm, participation, curiosity, and satisfaction after AI use [6,70,74,81,87,91,92,101,104,106,113,114,116,119,131,141,142], alongside a general preference for AI tools as efficient and convenient [99,111,124,127,135,140].”
From A Systematic Review of Generative AI in K–12: Mapping Goals, Activities, Roles, and Outcomes via the 3P Model (2025) · cites this paper as [74]
Cites
Situated Learning: Legitimate Peripheral Participation.
Bloch, M. E. F., Lave, J., & Wenger, É. (1994) · Man · doi:10.2307/2804509 · Cited by 39,323 · see citing works
ChatGPT for Good? On Opportunities and Challenges of Large Language Models for Education
Kasneci, E., Seßler, K., Küchemann, S., Bannert, M., Dementieva, D., Fischer, F., Gasser, U., Groh, G., Günnemann, S., Hüllermeier, E., et al. (2023) · doi:10.35542/osf.io/5er8f · Cited by 7,552 · see citing works
ChatGPT for good? On opportunities and challenges of large language models for education
Kasneci, E., Sessler, K., Küchemann, S., Bannert, M., Dementieva, D., Fischer, F., Gasser, U., Groh, G., Günnemann, S., Hüllermeier, E., et al. (2023) · Learning and Individual Differences · doi:10.1016/j.lindif.2023.102274 · Cited by 7,326 · see citing works
On the Opportunities and Risks of Foundation Models
Bommasani, R., Hudson, D. A., Adeli, E., Altman, R., Arora, S., von Arx, S., Bernstein, M. S., Bohg, J., Bosselut, A., Brunskill, E., et al. (2021) · arXiv · doi:10.48550/arxiv.2108.07258 · Cited by 2,676 · see citing works
Generative Agents: Interactive Simulacra of Human Behavior
Park, J. S., O'Brien, J., Cai, C. J., Morris, M. R., Liang, P., & Bernstein, M. S. (2023) · Proceedings of the 36th Annual ACM Symposium on User Interface Software and Technology · doi:10.1145/3586183.3606763 · Cited by 2,187 · cited 1× · see citing works
“This could be achieved using a method similar to the one Park et al. [52] employed to create generative agents to simulate human behavior in a game-like environment.”
MethodCordova, D. I., & Lepper, M. R. (1996) · Journal of Educational Psychology · doi:10.1037/0022-0663.88.4.715 · Cited by 1,568 · see citing works
Dunleavy, M., Dede, C., & Mitchell, R. (2008) · Journal of Science Education and Technology · doi:10.1007/s10956-008-9119-1 · Cited by 1,526 · see citing works
Next Generation Science Standards
National Research Council (2013) · National Academies Press eBooks · doi:10.17226/18290 · Cited by 1,326 · see citing works
Hamilton, D., McKechnie, J., Edgerton, E., & Wilson, C. (2020) · Journal of Computers in Education · doi:10.1007/s40692-020-00169-2 · Cited by 1,276 · see citing works
Carifio, J., & Perla, R. J. (2007) · Journal of Social Sciences · doi:10.3844/jssp.2007.106.116 · Cited by 1,061 · see citing works
Lindgren, R., & Johnson‐Glenberg, M. C. (2013) · Educational Researcher · doi:10.3102/0013189x13511661 · Cited by 527 · see citing works
Ji, H., Han, I., & Ko, Y. (2023) · Journal of Research on Technology in Education · doi:10.1080/15391523.2022.2142873 · Cited by 394 · see citing works
Chittaro, L., & Buttussi, F. (2015) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2015.2391853 · Cited by 373 · see citing works
Pinkard, N., Erete, S. L., Martin, C. K., & de Royston, M. M. (2017) · Journal of the Learning Sciences · doi:10.1080/10508406.2017.1307199 · Cited by 274 · see citing works
Huang, W., Roscoe, R. D., Johnson‐Glenberg, M. C., & Craig, S. D. (2020) · Journal of Computer Assisted Learning · doi:10.1111/jcal.12520 · Cited by 240 · see citing works
Why should my students use AR? A comparative review of the educational impacts of augmented-reality
Radu, I. W. (2012) · Proceedings - International Symposium on Mixed and Augmented Reality, ISMAR · doi:10.1109/ismar.2012.6402590 · Cited by 236 · see citing works
GPTeach: Interactive TA Training with GPT-based Students
Markel, J. M., Opferman, S. G., Landay, J., & Piech, C. (2023) · ACM Conference on Learning @ Scale · doi:10.1145/3573051.3593393 · Cited by 230 · see citing works
Self-citationSupportive teacher talk: the importance of the F-move
Cullen, R. (2002) · ELT Journal · doi:10.1093/elt/56.2.117 · Cited by 215 · see citing works
Effects of fantasy contexts on children's learning and motivation: Making learning more fun.
Parker, L. E., & Lepper, M. R. (1992) · Journal of Personality and Social Psychology · doi:10.1037/0022-3514.62.4.625 · Cited by 215 · see citing works
Chryst, B., Marlon, J. R., van der Linden, S., Leiserowitz, A., Maibach, E. W., & Roser‐Renouf, C. (2018) · Environmental Communication · doi:10.1080/17524032.2018.1508047 · Cited by 205 · see citing works
Georgiou, Y., & Kyza, E. Α. (2018) · Computers in Human Behavior · doi:10.1016/j.chb.2018.08.011 · Cited by 186 · see citing works
Story-Based Learning: The Impact of Narrative on Learning Experiences and Outcomes
McQuiggan, S. W., Rowe, J., Lee, S., & Lester, J. C. (2008) · Lecture notes in computer science · doi:10.1007/978-3-540-69132-7_56 · Cited by 173 · see citing works
Flora, J. A., Saphir, M. N., Lappé, M., Roser‐Renouf, C., Maibach, E. W., & Leiserowitz, A. (2014) · Climatic Change · doi:10.1007/s10584-014-1274-1 · Cited by 164 · see citing works
Johnson, B. P., & Manoli, C. C. (2010) · The Journal of Environmental Education · doi:10.1080/00958964.2010.503716 · Cited by 157 · see citing works
Ruan, S., Jiang, L., Xu, J., Tham, B. J.-K., Qiu, Z., Zhu, Y., Murnane, E. L., Brunskill, E., & Landay, J. A. (2019) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3290605.3300587 · Cited by 156 · see citing works
Self-citationThe Attitudes of Elementary School Children toward School and Subject Matters
Haladyna, T., & Thomas, G. (1979) · The Journal of Experimental Education · doi:10.1080/00220973.1979.11011707 · Cited by 133 · see citing works
Proceedings of the Interaction Design and Children Conference
(2020) · doi:10.1145/3392063 · Cited by 121 · see citing works
Enhancing students’ botanical learning by using augmented reality
Chien, Y.-C., Su, Y.-N., Wu, T., & Huang, Y. (2017) · Universal Access in the Information Society · doi:10.1007/s10209-017-0590-4 · Cited by 103 · see citing works
Tarng, W., Ou, K.-L., Yu, C.-S., Liou, F.-L., & Liou, H.-H. (2015) · Virtual Reality · doi:10.1007/s10055-015-0265-5 · Cited by 102 · see citing works
Narrative-based learning: Possible benefits and problems
Glaser, M., Garsoffky, B., & Schwan, S. (2009) · Communications · doi:10.1515/comm.2009.026 · Cited by 84 · see citing works
Using nature elements in mobile AR for education with children
Alakärppä, I., Jaakkola, E., Väyrynen, J., & Häkkilä, J. (2017) · International Conference on Human-Computer Interaction with Mobile Devices and Services (MobileHCI) · doi:10.1145/3098279.3098547 · Cited by 81 · see citing works
Pedagogical Agents: Back to the Future
Johnson, W., & Lester, J. C. (2018) · The AI Magazine · doi:10.1609/aimag.v39i2.2793 · Cited by 69 · see citing works
ARMath: Augmenting Everyday Life with Math Learning
Kang, S., Shokeen, E., Byrne, V. L., Norooz, L., Bonsignore, E., Williams, C. C., & Froehlich, J. E. (2020) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3313831.3376252 · Cited by 67 · see citing works
Supporting children's math learning with feedback-augmented narrative technology
Ruan, S., He, J., Ying, R., Burkle, J., Hakim, D., Wang, A., Yin, Y., Zhou, L., Xu, Q., AbuHashem, A. A., et al. (2020) · International Conference on Interaction Design and Children · doi:10.1145/3392063.3394400 · Cited by 59 · see citing works
Self-citationGeorgiou, Y., & Kyza, E. Α. (2020) · International Journal of Human-Computer Studies · doi:10.1016/j.ijhcs.2020.102546 · Cited by 45 · see citing works
Malinverni, L., Maya, J., Schaper, M.-M., & Parés, N. (2017) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3025453.3025955 · Cited by 34 · see citing works
Experiencing narrative worlds: On the psychological activities of reading
Miall, D. S. (2000) · Journal of Pragmatics · doi:10.1016/s0378-2166(99)00017-x · Cited by 19 · see citing works
Designing Immersive, Narrative-Based Interfaces to Guide Outdoor Learning
Cheng, A. Y., Ritchie, J., Agrawal, N., Childs, E., DeVeaux, C., Jee, Y., Leon, T., Maples, B., Cuadra, A., & Landay, J. A. (2023) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3544548.3581365 · Cited by 14 · see citing works
Self-citationPillat, R., Nagendran, A., & Lindgren, R. (2012) · IEEE International Symposium on Mixed and Augmented Reality - Arts, Media, and Humanities (ISMAR-AMH) · doi:10.1109/ismar-amh.2012.6484003 · Cited by 11 · see citing works
Fact via Fiction: Stories that Communicate Science
Negrete, A. (2005) · doi:10.13140/rg.2.1.5110.1207 · Cited by 7 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
Pang, R. Y., Schroeder, H., Smith, K. S., Barocas, S., Xiao, Z., Tseng, E., & Bragg, D. (2025) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3706598.3713726 · Cited by 68 · see citing works
Tang, Y., Situ, J., Cui, A. Y., Wu, M., & Huang, Y. (2025) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3706598.3714224 · Cited by 44 · see citing works
Harvey, E., Koenecke, A., & Kizilcec, R. F. (2025) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3706598.3713210 · Cited by 37 · see citing works
Zhang, Z., Sun, B., & An, P. (2025) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3706598.3713853 · Cited by 32 · see citing works
Inie, N., Falk, J., & Selvan, R. (2025) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3706598.3714227 · Cited by 26 · see citing works
GPT for Games: An Updated Scoping Review (2020–2024)
Yang, D., Kleinman, E., & Harteveld, C. (2025) · IEEE Transactions on Games · doi:10.1109/tg.2025.3563780 · Cited by 23 · see citing works
Dango: A Mixed-Initiative Data Wrangling System using Large Language Model
Chen, W., Tong, W., Case, A., & Zhang, T. (2025) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3706598.3714135 · Cited by 20 · see citing works
Chen, J.-J., Tang, M.-L., Lu, Y., Yao, B., Fan, E., Ma, X., Xu, Y., Wang, D.-K., Sun, Y.-L., & He, L. (2025) · International Conference on Human Factors in Computing Systems · doi:10.1145/3706598.3713275 · Cited by 44 · see citing works
Lin, X., & Tan, H. (2025) · Systems · doi:10.3390/systems13100840 · Cited by 22 · see citing works
agentAR: Creating Augmented Reality Applications with Tool-Augmented LLM-based Autonomous Agents
Zhu, C., Hsia, S.-K., Hu, X., Liu, Z., Shi, J., & Karthik Ramani, A. (2025) · ACM Symposium on User Interface Software and Technology (UIST) · doi:10.1145/3746059.3747676 · Cited by 13 · see citing works
Niu, R., Hu, J., Peng, S., Cao, C. C., Liu, C., Han, S., & Guo, Y. (2025) · CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA) · doi:10.1145/3706599.3719762 · Cited by 11 · see citing works
Lee, H., Stinar, F., Zong, R., Valdiviejas, H., Wang, D., & Bosch, N. (2025) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3706598.3713960 · Cited by 10 · see citing works
AI, Jobs, and the Automation Trap: Where Is HCI?
Constantinides, M., & Quercia, D. (2025) · Annual Symposium on Human-Computer Interaction for Work (CHIWORK) · doi:10.1145/3729176.3729191 · Cited by 12 · see citing works
Bendarkawi, J., Ponce, A., Mata, S. C., Aliu, A., Liu, Y., Zhang, L., Liaqat, A., Rao, V. N., & Monroy‐Hernández, A. (2025) · CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA) · doi:10.1145/3706599.3720162 · Cited by 16 · see citing works
Systemization of Knowledge (SoK): Goals, Coverage, and Evaluation in Cybersecurity and Privacy Games
Huang, Y., Grobler, M. M., Ferro, L. S., Psaroulis, G., Das, S., Wei, J., & Janicke, H. (2025) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3706598.3713798 · Cited by 8 · see citing works
Kolil, V. K., Prasanna, P. S., & Achuthan, K. (2025) · Education and Information Technologies · doi:10.1007/s10639-025-13799-9 · Cited by 8 · see citing works
Museum in the Classroom: Engaging Students with Augmented Reality Museum Artifacts and Generative AI
Ly, C., Peng, E., Liu, K., Qin, A., Howe, G., Cheng, A. Y., & Cuadra, A. (2025) · CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA) · doi:10.1145/3706599.3719787 · Cited by 7 · see citing works
Self-citationPark, S., Lee, W., Lee, S., & Lee, W. (2025) · Designing Interactive Systems Conference · doi:10.1145/3715336.3735716 · Cited by 5 · see citing works
Wang, X. E., Sin, Z. P. T., Jia, Y., Archer, D., Fong, W. H. Y., Li, Q., & Li, C. (2025) · ACM Symposium on User Interface Software and Technology (UIST) · doi:10.1145/3746059.3747673 · Cited by 5 · see citing works
ZooWear: Animal-Inspired Head-Mounted Haptic Interfaces to Augment the Zoo Experience
Chen, P., Yu, B., Sun, X., Xia, J., Liu, X., & Ren, X. (2025) · International Journal of Human-Computer Interaction · doi:10.1080/10447318.2025.2583471 · Cited by 4 · see citing works
Purposeful XR: Affordances, Challenges, and Speculations for an Ethical Future
Childs, E., Ghosh, S., Cmentowski, S., Cuadra, A., & Ratan, R. (2025) · CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA) · doi:10.1145/3706599.3706739 · Cited by 3 · see citing works
Self-citationLi, W., Pea, R. D., Haber, N., & Subramonyam, H. (2025) · Lecture notes in computer science · doi:10.1007/978-3-031-98462-4_2 · Cited by 3 · see citing works
Self-citation"Ask Sir Oliver Ingham": LLM-based Social Simulations for History Education
Park, K., Song, H., Seo, S., Kim, J., & Suh, B. (2025) · CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA) · doi:10.1145/3706599.3719728 · Cited by 2 · see citing works
Zhao, Y., Li, J., & Shu, S. (2024) · International Symposium on Computational Intelligence and Design · doi:10.1109/iscid63852.2024.00059 · Cited by 2 · see citing works
Oak Story: Improving Learner Outcomes with LLM-Mediated Interactive Narratives
Cheng, A. Y., Zou, C., Xie, A., Hsu, M., Yan, F., Huang, F., Zhang, D. K., Sharma, A., Poole, R., Rosli, D. W., et al. (2025) · ACM Symposium on User Interface Software and Technology (UIST) · doi:10.1145/3746059.3747698 · Cited by 2 · see citing works
Self-citationSpatiaLearn: Exploring XR Learning Environments for Reflective Writing
Li, J., Neshaei, S. P., Müller, L., Rietsche, R., Davis, R. L., & Wambsganß, T. (2025) · CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA) · doi:10.1145/3706599.3719742 · Cited by 1 · see citing works
Bridging Theory and Practice: A Multiphase Study of GenAI-Assisted Visualization Learning
Ahmad, M., & Ma, K. (2025) · IEEE Computer Graphics and Applications · doi:10.1109/mcg.2025.3553396 · Cited by 1 · see citing works
Michel, S., Taylor, B., Díaz, S. P., Wiggins, J. B., Finn, E., & Nourani, M. (2026) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3772318.3790489 · Cited by 2 · see citing works
Transforming Human-AI Collaboration using “Large Whatever Models” (LWMs)
Elagroudy, P., Väänänen, K., Li, J., Oulasvirta, A., Barbareschi, G., Gruenerbl, A., Churchill, E. F., Mackay, W. E., Schmidt, A., & Lukowicz, P. (2025) · CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA) · doi:10.1145/3706599.3716293
Generative AI in Rural High Schools: Challenges and Opportunities
Michel, S. (2025) · ACM Symposium on Applied Computing · doi:10.1145/3672608.3707992 · Cited by 1 · see citing works
Liu, Q., Zuo, Y., Ye, R., Chang, Y., Zhang, X., & Wu, L. (2025) · Communications in computer and information science · doi:10.1007/978-981-95-1525-7_2
Sayeed, S., Pial, S. M. R., & Iqbal, A. (2025) · Biannual Conference of the Italian SIGCHI Chapter (CHItaly) · doi:10.1145/3750069.3750094
Empowered XR through Generative AI: Balancing Superpowers and Risks
Tang, Y., Wu, M., Situ, J., Liu, J., Cui, A. Y., & Huang, Y. (2026) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3772318.3791307
Interaction-Aware Shared Scene Synthesis for VR Telepresence
Tan, Z., Ma, Q., Fan, R., Im, S. K., & Wang, L. (2026) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2026.3679086
Cheon, S., Stanbury, A. J., Arjunamahanthi, P., Kottur, H. R., & Kang, H. J. (2026) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3772318.3793705
Oju, J. U., Vincent, O. R., & Onwodi, G. O. (2026) · International Conference on Artificial Intelligence, Computer, Data Sciences and Applications (ACDSA) · doi:10.1109/acdsa67686.2026.11467826
Radensky, M., Shahid, S., Fok, R., Siangliulue, P., Hope, T., & Weld, D. S. (2026) · ACM Conference on AI and Agentic Systems (CAIS) · doi:10.1145/3786335.3813161
- Authors
- Cheng, A. Y., Guo, M., Ran, M., Ranasaria, A., Sharma, A., Xie, A., Le, K. N., Vinaithirthan, B., Luan, S. (., Wright, D. T. H., Cuadra, A., Pea, R., & Landay, J. A.
- Affiliations
- Stanford University (US)
- University of California San Diego (US)
- Nagoya University (JP)
- ORCID
- Venue
- Proceedings of the CHI Conference on Human Factors in Computing Systems
- Publisher
- ACM
- Published
- 2024-05-11
- Pages
- 1-23
- OpenAlex
- W4396832803
- Cited by
- 37
- References
- 38
- Citations by year
- 2026: 6 · 2025: 30 · 2024: 1
- FWCI
- 7.66
- Type
- conference paper
- Language
- en
- Open access
- closed
- Locations
- CHI Conference on Human Factors in Computing Systems (publishedVersion)
- Funders
- Stanford Institute for Human-Centered Artificial Intelligence, Stanford University (1247957-50-WZACW)
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Augmented Reality Applications
- Physical Sciences › Computer Science › Information Systems › Multimedia Learning Systems
- Physical Sciences › Computer Science › Artificial Intelligence › Intelligent Tutoring Systems and Adaptive Learning
- Keywords
- augmented reality, large language models, culturally relevant pedagogy, science motivation, elementary school students, Chinese mythology, narrative-based learning, fictional characters, mobile learning applications, astronomy education, environmental science education, cultural heritage, student engagement
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
