Stage Wizard: Enhancing Tangible Storytelling with Multimodal LLMs
The DOI matches its Crossref record; 16 of the 16 references it cites were found in a registry or an index.
Abstract
This paper introduces a pipeline that integrates multimodal large language models (LLMs) for tangible storytelling, featuring flexible materials generation, intuitive hands-on performance, and easy finalization. The design system enables teachers, parents, and children to create stage elements through natural language interactions and generate paper-cut style images. These elements can be easily fabricated using standard printing paper and assembled into a reconfigurable cardstock stage, allowing children to craft various plotlines through manipulation. The storytelling process can be directly recorded as a short film or transformed into an elaborate storybook using styled image filters and refining LLMs. By introducing the role of the stage in both the design and manipulation processes, this pipeline offers intuitive guidance and affordance for free but organized creation. The flexibility introduced by LLMs supports educators in diverse course design and children in self-expression. Without the requirement for specific hardware, the system also has the potential to be applied more broadly in less developed areas.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Citations
Cites
A meta-analysis of writing instruction for adolescent students
Graham, S., & Perin, D. (2007) · Journal of Educational Psychology · doi:10.1037/0022-0663.99.3.445 · Cited by 1,609 · cited 1×
“…more suitable for older children, who are becoming more aware of their individual identities, leading to a greater emphasis on personal expression and the quality of their creative outputs[22], with opportunities to refine and enhance their designs and narratives to produce a sophisticated outcome.”
Tangible User Interfaces: Past, Present, and Future Directions
Shaer, O. (2009) · now publishers, Inc. eBooks · doi:10.1561/9781601983299 · Cited by 414
From interaction to trajectories: designing coherent journeys through user experiences
Benford, S., Giannachi, G., Koleva, B., & Rodden, T. (2009) · Proceedings of the SIGCHI Conference on Human Factors in Computing Systems · doi:10.1145/1518701.1518812 · Cited by 314 · cited 1×
“With the natural affordance and constraints of the stage, it also facilitates intuitive management of the visual elements like characters and props[20], and potentially contributes to a more structured story[21].”
Morrow, L. M. (1985) · The Elementary School Journal · doi:10.1086/461427 · Cited by 249 · cited 2×
“This allows children to improve their literacy by comparing their writing with the refined plot texts[21].”
(2012) · Cited by 139
Design implications of generative AI systems for visual storytelling for young learners
Han, A., & Cai, Z. (2023) · Proceedings of the 22nd Annual ACM Interaction Design and Children Conference · doi:10.1145/3585088.3593867 · Cited by 127 · cited 1×
“LLMs can significantly reduce barriers to entry in visual storytelling by enabling users to generate high-quality visual content without advanced drawing skills[12].”
Tangible viewpoints: a physical approach to multimedia stories
Mazalek, A., Davenport, G., & Ishii, H. (2002) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/641007.641037 · Cited by 111 · cited 1×
“By physically manipulating story elements, users, especially children, can explore narratives in a more immersive and hands-on manner, shaping them into digital media such as image[6], audio[7], video[8], or animation[9].”
Towards a Framework for Tangible Narratives
Harley, D., Chu, J. H., Kwan, J., & Mazalek, A. (2016) · International Conference on Tangible, Embedded, and Embodied Interaction (TEI) · doi:10.1145/2839462.2839471 · Cited by 80 · cited 1×
“In addition, children often encounter challenges when creating longer and structured stories, a problem that is often neglected by existing systems[10].”
Interactive augmented reality storytelling guided by scene semantics
Li, C., Li, W., Huang, H., & Yu, L.-F. (2022) · ACM Transactions on Graphics · doi:10.1145/3528223.3530061 · Cited by 59 · cited 1×
“Recent advancements in storytelling techniques, as extensively documented in several noteworthy papers, highlight the growing interest in integrating technology like AR[1], VR[2] and robotics[3].”
Towards a Tangible Storytelling Kit for Exploring Emotions with Children
Wallbaum, T., Ananthanarayan, S., Borojeni, S. S., Heuten, W., & Boll, S. (2017) · Thematic Workshops of ACM Multimedia · doi:10.1145/3126686.3126702 · Cited by 46 · cited 1×
“A popular way of implementing a drama-like storytelling experience in tangible interaction is to provide a physical stage[19].”
Research works printed but not linked: 6
- Soenke Kirchhof. 2018. Storytelling for volumetric VR. In ACM SIGGRAPH 2018 Educator S Forum (SIGGRAPH '18). ACM, New York, NY, USA, Article 6, 1-2.
- Sophia C. Steinhaeusser and Birgit Lugrin. 2022. Effects of Colored LEDs in Robotic Storytelling on Storytelling Experience and Robot Perception. In Proceedings of the 2022 ACM/IEEE International Conference on Human-Robot Interaction (HRI '22). IEEE Press, 1053-1058.
- Huang, C. C., et.al. (2015, January). Olegoru: a soundscape composition tool to enhance imaginative storytelling with tangible objects. In Proceedings of the ninth international conference on tangible, embedded, and embodied interaction (pp. 709-714).
- Zhang, L., & Agrawala, M. (2023). Adding conditional control to text-to-image diffusion models. arXivpreprint arXiv:2302.05543. · arXiv
- Ai, H., & Sheng, L. (2023). Stable Diffusion Reference Only: Image Prompt and Blueprint Jointly Guided Multi-Condition Diffusion Model for Secondary Painting. arXiv preprint arX-iv:2311.02343.
- Milord M. (2007). The effects of using storytelling on reading comprehension through drama and theater (Doctoral dissertation SUNY College at Brockport).
Cited by
Where Do I 'Add the Egg'?: Exploring Agency and Ownership in AI Creative Co-Writing Systems
Carrera, D., Thomas-Mitchell, J., & Wigdor, D. (2026) · Proceedings of the 2026 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3772318.3791952 · Cited by 3
“We observed the application of magic metaphors in multiple VR and visual art co-creative AI systems [23, 30, 79, 81].”
Object-Driven Narrative in AR: A Scenario-Metaphor Framework with VLM IntegrationPreprint
Sun, Y., Guan, H., Chan, L. K. Y., & Kuo, Y. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2504.13119 · Cited by 3
“While some recent work[9] explores multimodal generation using VLMs, it has yet to deeply integrate spatial understanding with narrative reasoning.”
Kang, S., Zoh, J., Kim, H., Park, H., Lim, H., & Lee, J. (2026) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3772318.3790370 · Cited by 2
Ribeiro, P., Musumeci, M., Schimmelpfennig, H., Kannen, K., & Ressel, C. (2025) · ACM International Conference on Interaction Design and Children (IDC) · doi:10.1145/3713043.3731546 · Cited by 1
““Stage Wizard” [8] addresses this opportunity by presenting a storytelling system that combines tangible interaction with multimodal LLMs.”
Into the Makerverse: Communal Tangible Making and Place-Based Futuring with AI
Pait, S., Janicki, S., Hou, Y., Farmer, C. B., Kowalski, L. L., Long, J., Alvarenga, T., Li, X., Namani, V., Lew, L., et al. (2026) · International Conference on Tangible, Embedded, and Embodied Interaction (TEI) · doi:10.1145/3731459.3774487 · Cited by 1
“Tangible engagement in AI integrated storytelling experiences with children has shown potential for AI learning [23, 43, 54].”
Influential
Metadata
- Authors
- Han, K., Tang, K., & Wang, M.
- Affiliations
- Tsinghua University (CN)
- Beijing Institute for General Artificial Intelligence (CN)
- Venue
- Proceedings of the Nineteenth International Conference on Tangible, Embedded, and Embodied Interaction
- Publisher
- ACM
- Published
- 2025-03-04
- Pages
- 1-13
- OpenAlex
- W4407941976
- Cited by
- 8
- References
- 16
- Citations by year
- 2026: 6 · 2025: 2
- FWCI
- 63.88
- Type
- conference paper
- Language
- en
- Open access
- closed
- Locations
- International Conference on Tangible, Embedded, and Embodied Interaction (TEI) (publishedVersion)
- Topics
- Social Sciences › Education › Digital Storytelling and Education
- Physical Sciences › Computer Science › Computer Graphics and Computer-Aided Design › Digital Media and Visual Art
- Physical Sciences › Computer Science › Human-Computer Interaction › Interactive and Immersive Displays
- Keywords
- multimodal large language models, natural language interaction, children's creative expression, low-cost educational technology
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Stage Wizard: Enhancing Tangible Storytelling with Multimodal LLMs. https://app.cochat.ai/papers/10.1145/3689050.3704429
- The paper
- Han, K., Tang, K., & Wang, M. (2025). Stage Wizard: Enhancing Tangible Storytelling with Multimodal LLMs. Proceedings of the Nineteenth International Conference on Tangible, Embedded, and Embodied Interaction. https://doi.org/10.1145/3689050.3704429
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.