Vellum of Lies: Designing and Evaluating a Multimodal Tangible Interaction System for Embodied Serious Historical Narratives
Open accessThe DOI matches its Crossref record; 49 of the 49 references it cites were found in a registry or an index.
Abstract
Many existing historical narrative experiences still rely on screen-based viewing and limited multimodal interaction, which tends to keep users in a passive receiving role and limits their cognitive understanding, reflective thinking, user engagement, and emotional response. To address this, we designed Vellum of Lies, an immersive tangible user interface for anti-feudal historical narratives. The work integrates physical props, sensors, motion-based input, and projection mapping, organizing the story into continuous interaction nodes that allow users to participate in narrative progression through embodied interaction, including spatial movement, object manipulation, and final choice-making. The final choice mechanism operationalizes Tragic Agency by allowing users to make meaningful choices while constraining their ability to alter the final tragic outcome, with the aim of evoking tension between perceived choice and narrative inevitability. A mixed-methods user experience evaluation (N = 32) compared Vellum of Lies with a video-watching narrative experience. In this small-scale exploratory study, the experimental condition showed significant advantages in the HUQ Overall score, Reflection, the UES-SF Overall score, Aesthetic Appeal, Interest/Enjoyment, Pleasure/Valence, and Arousal, while also showing lower NASA-TLX Overall scores, Mental Demand, and Effort. Other measured subdimensions did not reach statistical significance. These preliminary findings suggest that multimodal tangible interaction may function not only as a presentation technique, but also as a narrative mechanism that coordinates physical input, bodily movement, projected feedback, and constrained choice-making to support selected aspects of understanding, engagement, and emotional reflection in serious historical narratives.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Citations
Cites
The weirdest people in the world?
Henrich, J., Heine, S. J., & Norenzayan, A. (2010) · Behavioral and Brain Sciences · doi:10.1017/s0140525x0999152x · Cited by 12,641
Measuring emotion: The self-assessment manikin and the semantic differential
Bradley, M. M., & Lang, P. J. (1994) · Journal of Behavior Therapy and Experimental Psychiatry · doi:10.1016/0005-7916(94)90063-9 · Cited by 9,538
NASA-Task Load Index (NASA-TLX); 20 Years Later
Hart, S. G. (2006) · Proceedings of the Human Factors and Ergonomics Society Annual Meeting · doi:10.1177/154193120605000909 · Cited by 4,297
Control and information in the intrapersonal sphere: An extension of cognitive evaluation theory.
Ryan, R. M. (1982) · Journal of Personality and Social Psychology · doi:10.1037/0022-3514.43.3.450 · Cited by 2,721
O’Brien, H. L., Cairns, P., & Hall, M. (2018) · International Journal of Human-Computer Studies · doi:10.1016/j.ijhcs.2018.01.004 · Cited by 1,014
User experience over time: an initial framework
Karapanos, E., Zimmerman, J., Forlizzi, J., & Martens, J.-B. (2009) · Proceedings of the SIGCHI Conference on Human Factors in Computing Systems · doi:10.1145/1518701.1518814 · Cited by 561
Why Story Matters: A Review of Narrative in Serious Games
Naul, E., & Liu, M. (2020) · Journal of Educational Computing Research · doi:10.1177/0735633119859904 · Cited by 204
Mixed Reality and Gamification for Cultural Heritage
(2017) · Springer eBooks · doi:10.1007/978-3-319-49607-8 · Cited by 191
The narrative arc: Revealing core narrative structures through text analysis
Boyd, R. L., Blackburn, K. G., & Pennebaker, J. W. (2020) · Science Advances · doi:10.1126/sciadv.aba2196 · Cited by 185
Škola, F., Rizvić, S., Cozza, M., Barbieri, L., Bruno, F., Skarlatos, D., & Liarokapis, F. (2020) · Sensors · doi:10.3390/s20205851 · Cited by 180
Revealing and Reframing Apocalyptic Tragedy in Global Warming Discourse
Foust, C. R., & O'Shannon Murphy, W. (2009) · Environmental Communication · doi:10.1080/17524030902916624 · Cited by 177
Gamifying cultural heritage: Exploring the potential of immersive virtual exhibitions
Wang, H., Gao, Z., Zhang, X., Du, J., Xu, Y., & Wang, Z. (2024) · Telematics and Informatics Reports · doi:10.1016/j.teler.2024.100150 · Cited by 172
Interaction in eXtended Reality Applications for Cultural Heritage
Okanović, V., Ivkovic-Kihic, I., Bošković, D., Mijatović, B., Prazina, I., Skaljo, E., & Rizvić, S. (2022) · Applied Sciences · doi:10.3390/app12031241 · Cited by 168
Cultural cognition in usability evaluation
Clemmensen, T., Hertzum, M., Hornbæk, K., Shi, Q., & Yammiyavar, P. (2009) · Interacting with Computers · doi:10.1016/j.intcom.2009.05.003 · Cited by 154
On the role of interaction mode and story structure in virtual reality serious games
Ferguson, C., van den Broek, E. L., & van Oostendorp, H. (2020) · Computers & Education · doi:10.1016/j.compedu.2019.103671 · Cited by 147
Embodied Pathways and Ethical Trails: Studying Learning in and through Relational Histories
Vossoughi, S., Jackson, A., Chen, S., Roldan, W., & Escudé, M. (2020) · Journal of the Learning Sciences · doi:10.1080/10508406.2019.1693380 · Cited by 142
Ambient Information Visualisation and Visitors’ Technology Acceptance of Mixed Reality in Museums
Hammady, R., Ma, M., & Strathearn, C. (2020) · Journal on Computing and Cultural Heritage · doi:10.1145/3359590 · Cited by 128
Chang, S., & Suh, J. (2025) · Sensors · doi:10.3390/s25092914 · Cited by 105
Nofal, E., Panagiotidou, G., Reffat, R. M., Hameeuw, H., Boschloos, V., & Vande Moere, A. (2020) · Journal on Computing and Cultural Heritage · doi:10.1145/3350427 · Cited by 101
A Systematic Literature Review of Gamification in/for Cultural Heritage: Leveling up, Going Beyond
Marques, C. G., Pedro, J. P., & Araújo, I. (2023) · Heritage · doi:10.3390/heritage6080312 · Cited by 98
Multisensory museum experience: an integrative view and future research directions
Luo, D., Doucé, L., & Nys, K. (2024) · Museum Management and Curatorship · doi:10.1080/09647775.2024.2357071 · Cited by 97
Nikolakopoulou, V., Printezis, P., Maniatis, V., Kontizas, D., Vosinakis, S., Chatzigrigoriou, P., & Koutsabasis, P. (2022) · Heritage · doi:10.3390/heritage5020056 · Cited by 95
Embodied Engagement with Narrative: A Design Framework for Presenting Cultural Heritage Artifacts
Chu, J. H., & Mazalek, A. (2019) · Multimodal Technologies and Interaction · doi:10.3390/mti3010001 · Cited by 90
Petousi, D., Katifori, A., Servi, K., Roussou, M., & Ioannidis, Y. (2022) · Frontiers in Education · doi:10.3389/feduc.2022.942834 · Cited by 90
Teaching Cultural Heritage through a Narrative-based Game
Malegiannaki, I., Daradoumis, Τ., & Retalis, S. (2020) · Journal on Computing and Cultural Heritage · doi:10.1145/3414833 · Cited by 67
Digital Game-Based Heritage Education: Analyzing the Potential of Heritage-Based Video Games
García, D. C., Reche, M. P. C., Hernández, M. D. L. E. C., & Lorenzo-Martín, M. E. (2024) · Education Sciences · doi:10.3390/educsci14040396 · Cited by 64
Srdanović, P., Skala, T., & Maričević, M. (2024) · Applied Sciences · doi:10.3390/app15010257 · Cited by 64
The interactive museum: Video games as history lessons through lore and affective design
Anderson, S. L. (2019) · E-Learning and Digital Media · doi:10.1177/2042753019834957 · Cited by 51
Gamifying History: Designing and Implementing a Game-Based Learning Course Design Framework
Scholz, K. W., Komornicka, J. N., & Moore, A. (2021) · Teaching & Learning Inquiry The ISSOTL Journal · doi:10.20343/teachlearninqu.9.1.9 · Cited by 36
Duranti, D., Spallazzo, D., & Petrelli, D. (2024) · Journal on Computing and Cultural Heritage · doi:10.1145/3631132 · Cited by 35
Li, X., Zheng, C. Y., Pan, Z., Huang, Z., Niu, Y., Wang, P., & Geng, W. (2023) · International Journal of Human-Computer Interaction · doi:10.1080/10447318.2023.2276526 · Cited by 26
Champion, E. M. (2020) · Ubiquity Press eBooks · doi:10.5334/bch · Cited by 23
The Impact of Short Videos on the Creation and Dissemination of Intangible Cultural Heritage
Zhang, Y. (2024) · Communications in Humanities Research · doi:10.54254/2753-7064/32/20240034 · Cited by 18
Self-citationJi, Y., Zhong, S., Zhang, B., & Sean, C. (2021) · International Symposium of Chinese CHI (Chinese CHI) · doi:10.1145/3490355.3490373 · Cited by 17
The Remediation of the Epic in Digital Games: The Elder Scrolls V: Skyrim
Melnic, V. (2018) · American, British and Canadian Studies · doi:10.2478/abcsj-2018-0009 · Cited by 10
Interactive Artifacts and Stories: Design Considerations for an Object-Based Learning History Game
Urban, A. C. (2023) · Technology Knowledge and Learning · doi:10.1007/s10758-023-09653-x · Cited by 9
Story Inspiration Station: Deeper Engagement with Museum Objects via Participatory Interpretation
Bowen, S. J., Rowling, H. C., & Kirk, D. S. (2025) · Journal on Computing and Cultural Heritage · doi:10.1145/3715158 · Cited by 7
At the nexus of ludology and narratology: Advances in reality-based story-driven games
Chen, L., Dowling, D. R., & Goetz, C. G. (2023) · F1000Research · doi:10.12688/f1000research.129113.1 · Cited by 6
Shashwat, S., Padmagirwar, A., Arora, S., & Srivastava, A. (2024) · International Conference on Tangible, Embedded, and Embodied Interaction (TEI) · doi:10.1145/3623509.3635251 · Cited by 6
Sail with Columbus: Navigation through Tangible and Interactive Storytelling
Ciotoli, L., Alinam, M., & Torre, I. (2021) · Biannual Conference of the Italian SIGCHI Chapter (CHItaly) · doi:10.1145/3464385.3464741 · Cited by 5
Zhuiniu Water Buffalo Ritual of the Miao: Cultural Narrative Performed
Riccio, T. (2022) · Religions · doi:10.3390/rel13040303 · Cited by 5
Mediatization of Communicative Memory and Crisis of Critical Thinking in Digital Age
Igaeva, K. V., & Николаи, Ф. В. (2024) · Nauchnyi Dialog · doi:10.24224/2227-1295-2024-13-9-189-205 · Cited by 3
Museums and narrative. from ‘Grand Narratives’ and objects biographies to exhibition storytelling
Chourmouziadi, A. (2025) · Museum Management and Curatorship · doi:10.1080/09647775.2025.2467711 · Cited by 3
Oedipus at the Crossroads: Tragic Agency in Contemporary Videogame
Zhang, X. (2025) · doi:10.26503/dl.v2025i3.2508 · Cited by 2
Designing an Immersive Game Environment for the Raja Raja Chola -I Conquest
Alfred Daniel, J., & Santhosh, S. (2024) · International Conference on Emerging Research in Computational Science (ICERCS) · doi:10.1109/icercs63125.2024.10895168 · Cited by 1
Re-examining feudal ethics and individual awakening in Ba Jin's Family
Wang, Q. (2025) · Advances in Humanities Research · doi:10.54254/2753-7080/2025.24236 · Cited by 1
Chen, Z. (2025) · Communications in Humanities Research · doi:10.54254/2753-7064/2025.ht27170 · Cited by 1
Zhou, T., Xu, C., & Yao, X. (2025) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3746027.3756137 · Cited by 1
Zheng, Y., Wu, R., Xiang, W., Chen, Z., & Kuang, L. (2025) · International Conference on Computer Science and Management Technology (ICCSMT) · doi:10.1145/3795154.3795345 · Cited by 1
Research works printed but not linked: 5
- Peters, 2022, News as They Know It: Young Adults’ Information Repertoires in the Digital Media Landscape, Digit. Journal., 10, 62
- Koenitz, H. (2015). Interactive Digital Narrative: History, Theory and Practice, Taylor & Francis Group. [1st ed.]. Routledge Studies in European Communication Research and Education Ser.
- Vassos, 2017, Museum Experience Design: A Modern Storytelling Methodology, Des. J., 20, S3975
- Rizvic, S., Okanovic, V., Prazina, I., and Sadzak, A. (2016). 4D Virtual Reconstruction of White Bastion Fortress. Eurographics Workshop on Graphics and Cultural Heritage, Eurographics Association.
- Kim, 2020, Shang Dynasty Ritual Activities for Rain as Seen in Oracle Bone Inscriptions, J. Chin. Stud., 70, 1
Cited by
Still being checked.
Metadata
- Authors
- Hou, Y., Wang, S., Su, Z., Bi, L., Ye, X., & Zhang, Y.
- Affiliations
- University of Edinburgh (GB)
- Corresponding author
- Hou, Yuli
- Venue
- Multimodal Technologies and Interaction
- Publisher
- MDPI AG
- Published
- 2026-09-20
- Volume
- 10
- Issue
- 9
- Pages
- 96
- ISSN
- 2414-4088 (ISSN-L)
- OpenAlex
- W7213999781
- Cited by
- 0
- References
- 49
- FWCI
- 0.00
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- Locations
- Multimodal Technologies and Interaction (publishedVersion, cc-by)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion)
- Topics
- Physical Sciences › Computer Science › Human-Computer Interaction › Interactive and Immersive Displays
- Physical Sciences › Computer Science › Human-Computer Interaction › Innovative Human-Technology Interaction
- Physical Sciences › Computer Science › Human-Computer Interaction › Virtual Reality Applications and Impacts
- Keywords
- tangible user interfaces, multimodal interaction, embodied interaction, constrained choice, projection mapping, user engagement, emotional reflection, user experience evaluation
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.