Student or educator? Sign up with your school email and get 6 months of Pro free.

The DOI matches its Crossref record; 23 of the 23 references it cites were found in a registry or an index; 5 of the 12 sources its claims cite were checked for support.

Game Plot Design With an LLM-Powered Assistant: An Empirical Study With Game Designers

Open access

Abstract

We introduce GamePlot, an LLM-powered assistant that supports game designers in crafting immersive narratives for turn-based games, and allows them to test these games through a collaborative game play and refine the plot throughout the process. Our user study with 14 game designers shows high levels of both satisfaction with the generated game plots and sense of ownership over the narratives, but also reconfirms that LLM are limited in their ability to generate complex and truly innovative content. We also show that diverse user populations have different expectations from AI assistants, and encourage researchers to study how tailoring assistants to diverse user groups could potentially lead to increased job satisfaction and greater creativity and innovation over time.

Overview

CoChat Summary

TL;DR

GamePlot is an LLM-powered assistant for turn-based game narrative design that supports plot generation, collaborative playtesting, live plot refinement, and Wizard-of-Oz NPC control. In a user study with 14 game designers, participants reported high satisfaction and ownership, while also confirming LLM limitations in creativity, consistency, and narrative complexity.

Key findings

  • Participants rated the AI assistant’s help in improving their game story at 4/5 on average, satisfaction with the generated game plot at 4/5 on average, and perceived control/ownership over the story at 4.5/5 on average; all ratings exceeded 3.
  • The game room’s usefulness for testing with players averaged 4.21, with 13 out of 14 participants giving ratings of 4 or 5.
  • The most-liked feature was the ability to change the game plot during the test session, favored by 12 out of 14 participants; half also favored the NPC hidden-states dropdown and the multiplayer setup.
  • Designers with different backgrounds had different expectations: game developers often preferred to offload narrative writing to the AI, while narrative writers wanted creative control but valued AI for exploring narrative paths. LLM outputs were described as lacking creativity, consistency, tension balance, and organic dialogue.

Methods

  • The authors built GamePlot, a web application with a design room and a game room, using GPT-3.5-Turbo-16k with default hyperparameters, a 2,000-token limit for summarization/plot generation, a 1,000-token limit for next-turn generation, and summarization of earlier turns after 40,000 characters.
  • They recruited 14 participants through Upwork and social media; 64.3% had at least one year of professional gaming-industry experience, and 8 identified as narrative-writing experts. Sessions lasted 1–2 hours, with compensation at $25/hour.
  • Participants received a tutorial and template story, designed a game in the design room for 20–25 minutes, tested it in a game room with one author as a player for 15–20 minutes, then completed a post-study questionnaire; free-text responses were manually classified by the authors.

Limitations

  • Stated: the study used only GPT-3.5, so results may not extend to other LLMs without empirical testing; the sample of 14 narrative designers is small and may not capture diverse designer backgrounds; and there is likely sampling bias because participants were already inclined to embrace AI in game narrative design.
  • Evident: the study has no human-only or alternative-tool control condition, so it cannot empirically establish whether GamePlot improves final narrative quality over unaided design; the paper itself notes the final product may not surpass a human-written story in emotional depth or complexity.
  • Evident: testing involved one of the authors joining as a player, which may have shaped the collaborative playtest experience, though the paper does not discuss this as a limitation.

Model-written summary. Check it against the paper before relying on it.

Integrity signals

Registry

Crossref · 10.1109/tg.2026.3663566

Cites

0 citation errors · 0 not checkable

Statistics

Nothing to recompute.

Verification

Claims the paper makes

4 Valid · 1 Not valid · 5 Not checked · 71 Own results and interpretations

35 more claims

Create a free account to see all 81 claims and every source verdict.

See all claims — free account
  • Probably not a substantial claim. 34
  • It may state two separate claims as one. 16

Create a free account to see these claims.

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

Each contradiction between two passages listed here was confirmed by an independent check.

1 of 1

Citations

Cites

The publisher hasn’t deposited this paper’s reference list.

Research works printed but not linked: 4

  1. Nader Akoury, Ronan Salz, and Mohit Iyyer. 2023. Towards grounded dialogue generation in video game environments. In Creative AI Across Modalities Workshop, AAAI .
  2. Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, Ilya Sutskever, et al. 2019. Language models are unsupervised multitask learners. OpenAI blog , 1(8):9.
  3. Judith van Stegeren and Jakub My´ sliwiec. 2021. Finetuning gpt-2 on annotated rpg quests for npc dialogue generation. In Proceedings of the 16th International Conference on the Foundations of Digital Games , FDG '21, New York, NY, USA. Association for Computing Machinery.
  4. 3rd Wordplay: When Language Meets Games Workshop (Wordplay 2022) , pages 25-43, Seattle, United States. Association for Computational Linguistics.

Metadata

Authors
Alavi, S. H., Xu, W., Jojic, N., Kennett, D., Ng, R. T., Rao, S., Zhang, H., Dolan, B., & Shwartz, V.
Affiliations
  • University of British Columbia (CA)
  • University of British Columbia
Venue
IEEE Transactions on Games
Published
2026
Pages
1-10
ISSN
  • 2475-1502 (ISSN-L)
  • 2475-1510
OpenAlex
W7128620905
Cited by
0
References
0
FWCI
0.00
Type
journal article
Language
en
Open access
Locations
  • IEEE Transactions on Games (publishedVersion, cc-by)
  • arXiv (Cornell University) (submittedVersion)
  • arXiv (Cornell University)
Funders
  • Canadian Institute for Advanced Research
  • Natural Sciences and Engineering Research Council of Canada
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › Artificial Intelligence in Games
  • Social Sciences › Sociology and Political Science › Digital Games and Media
  • Social Sciences › Psychology › Developmental and Educational Psychology › Educational Games and Gamification
Keywords
large language models, turn-based games, game narrative, collaborative gameplay, user satisfaction, sense of ownership, user heterogeneity, creativity and innovation

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

The claim verdicts this report counts are experimental: their accuracy is not yet measured.

A language model drew the claims from the paper on 2026-10-27, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).

A second model labelled each claim on 2026-10-27 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).

Cited sources are checked with a language model’s help (gpt-6-sol).

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: Game Plot Design With an LLM-Powered Assistant: An Empirical Study With Game Designers. https://app.cochat.ai/papers/10.1109/tg.2026.3663566
The paper
Alavi, S. H., Xu, W., Jojic, N., Kennett, D., Ng, R. T., Rao, S., Zhang, H., Dolan, B., & Shwartz, V. (2026). Game Plot Design With an LLM-Powered Assistant: An Empirical Study With Game Designers. IEEE Transactions on Games. https://doi.org/10.1109/tg.2026.3663566

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.