CoChat

Vellum of Lies: Designing and Evaluating a Multimodal Tangible Interaction System for Embodied Serious Historical Narratives

Open access
Hou, Yuli, Wang, Shuting, Su, Zhoutong, Bi, Leying, Ye, Xiaopei, Zhang, Yifan · 2026 · Multimodal Technologies and Interaction

The DOI matches its Crossref record; 49 of the 49 references it cites were found in a registry or an index.

Abstract

Many existing historical narrative experiences still rely on screen-based viewing and limited multimodal interaction, which tends to keep users in a passive receiving role and limits their cognitive understanding, reflective thinking, user engagement, and emotional response. To address this, we designed Vellum of Lies, an immersive tangible user interface for anti-feudal historical narratives. The work integrates physical props, sensors, motion-based input, and projection mapping, organizing the story into continuous interaction nodes that allow users to participate in narrative progression through embodied interaction, including spatial movement, object manipulation, and final choice-making. The final choice mechanism operationalizes Tragic Agency by allowing users to make meaningful choices while constraining their ability to alter the final tragic outcome, with the aim of evoking tension between perceived choice and narrative inevitability. A mixed-methods user experience evaluation (N = 32) compared Vellum of Lies with a video-watching narrative experience. In this small-scale exploratory study, the experimental condition showed significant advantages in the HUQ Overall score, Reflection, the UES-SF Overall score, Aesthetic Appeal, Interest/Enjoyment, Pleasure/Valence, and Arousal, while also showing lower NASA-TLX Overall scores, Mental Demand, and Effort. Other measured subdimensions did not reach statistical significance. These preliminary findings suggest that multimodal tangible interaction may function not only as a presentation technique, but also as a narrative mechanism that coordinates physical input, bodily movement, projected feedback, and constrained choice-making to support selected aspects of understanding, engagement, and emotional reflection in serious historical narratives.

Overview

Integrity signals

Registry

Crossref · 10.3390/mti10090096

Cites

49 of 49 references verified0 citation errors · 0 not checkable

Statistics

Recomputing p-values needs the paper’s full text.

Verification

Claims the paper makes

We couldn’t retrieve this paper’s full text, so its claims and self-check weren’t run. The registry and citation checks still apply.

Citations

Cites

Research works printed but not linked: 5

  1. Peters, 2022, News as They Know It: Young Adults’ Information Repertoires in the Digital Media Landscape, Digit. Journal., 10, 62
  2. Koenitz, H. (2015). Interactive Digital Narrative: History, Theory and Practice, Taylor & Francis Group. [1st ed.]. Routledge Studies in European Communication Research and Education Ser.
  3. Vassos, 2017, Museum Experience Design: A Modern Storytelling Methodology, Des. J., 20, S3975
  4. Rizvic, S., Okanovic, V., Prazina, I., and Sadzak, A. (2016). 4D Virtual Reconstruction of White Bastion Fortress. Eurographics Workshop on Graphics and Cultural Heritage, Eurographics Association.
  5. Kim, 2020, Shang Dynasty Ritual Activities for Rain as Seen in Oracle Bone Inscriptions, J. Chin. Stud., 70, 1

Cited by

Still being checked.

Metadata

Authors
Hou, Y., Wang, S., Su, Z., Bi, L., Ye, X., & Zhang, Y.
Affiliations
University of Edinburgh (GB)
Corresponding author
Hou, Yuli
Venue
Multimodal Technologies and Interaction
Publisher
MDPI AG
Published
2026-09-20
Volume
10
Issue
9
Pages
96
ISSN
2414-4088 (ISSN-L)
OpenAlex
W7213999781
Cited by
0
References
49
FWCI
0.00
Type
journal article
Language
en
Open access
Locations
  • Multimodal Technologies and Interaction (publishedVersion, cc-by)
  • DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion)
Topics
  • Physical Sciences › Computer Science › Human-Computer Interaction › Interactive and Immersive Displays
  • Physical Sciences › Computer Science › Human-Computer Interaction › Innovative Human-Technology Interaction
  • Physical Sciences › Computer Science › Human-Computer Interaction › Virtual Reality Applications and Impacts
Keywords
tangible user interfaces, multimodal interaction, embodied interaction, constrained choice, projection mapping, user engagement, emotional reflection, user experience evaluation

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.