Student or educator? Sign up with your school email and get 6 months of Pro free.
CoChat

The DOI matches its Crossref record; 50 of the 50 references it cites were found in a registry or an index; 14 of the 17 sources its claims cite were checked for support.

Game Knowledge Management System: Schema-Governed LLM Pipeline for Executable Narrative Generation in RPGs

Open access

Abstract

Procedural approaches have long been used in game development to reduce authoring costs and increase content diversity; however, traditional rule-based systems struggle to scale narrative complexity, whereas recent large language model (LLM)-based methods often produce outputs that are structurally invalid or incompatible with real-time game engines. This gap reflects a fundamental limitation in current practice: generative models lack systematic mechanisms for managing executable game knowledge rather than merely producing free-form narrative texts. To address this issue, we propose a Game Knowledge Management System (G-KMS) that reformulates LLM-based narrative generation as a structured knowledge management process. The proposed framework integrates knowledge grounding, schema-governed generation, normalization-based repair, engine-aligned knowledge admission, and application within a unified pipeline. The system was evaluated on a compact 2D Unity-based RPG benchmark using automated structural and semantic analyses, engine-level playability probes, and a controlled human player study. The experimental results demonstrated high reliability in knowledge admission, stable procedural structures, controlled expressive diversity, and a strong alignment between system-level metrics and player-perceived narrative quality, indicating that LLMs can function as dependable knowledge-construction components when embedded within a governed management pipeline. Beyond the evaluated RPG setting, this study suggests a practical and reproducible approach that may be extended to other executable systems, such as interactive simulations and training environments.

Overview

CoChat Summary

TL;DR

The paper proposes G-KMS, a three-stage schema-governed LLM pipeline that grounds narrative entities, generates JSON quest artifacts under constraints, normalizes/validates them, and executes them in Unity. Evaluated on a compact 2D Unity RPG benchmark, it reports high structural admissibility, stable quest graphs, controlled expressive diversity, and positive player ratings.

Key findings

  • Structural validity: final generator smoke-test pass rate was 100%; validation pass rates were 78% at T0.3, 85% at T0.7, and 77% at T1.0. Removing normalization produced 0% validation despite 100% smoke pass, while VLST3 reached 100% smoke and 100% validation.
  • Task-graph metrics: average path length was fixed at 3.0; branching ratios were 1.341 (T0.3), 1.361 (T0.7), and 1.362 (T1.0); dead ends were 0; reachable endings were about 5.16, 5.38, and 5.15. The handcrafted golden sample had branching ratio 1.455 and 8 endings.
  • Textual diversity: token entropy rose from 1.71 (T0.3) to 2.45 (T0.7) to 2.65 (T1.0); mean semantic similarity fell from 0.612 to 0.607 to 0.598; near-duplicate rate (>0.9) fell from 4.8% to 2.3% to 1.1%; average token count rose from 82.3 to 92.7 to 100.8.
  • Evaluation: LLM self-evaluation average overall scores were 4.20 (T0.3), 4.35 (T0.7), and 4.25 (T1.0). In the human study (N = 15), playability/clarity scored 4.17, narrative logic 4.23, world consistency 4.40, character voice/dialogue quality 4.30, interestingness 4.07, and overall experience 4.57; dialogue naturalness was the highest individual score at 4.60.

Methods

  • Design science research paradigm; the G-KMS artifact is a three-stage pipeline: data/world modeling, schema-constrained LLM generation, and evaluation/execution in Unity.
  • Data/sample: a compact 2D Unity-based RPG benchmark; narrative source text is converted into a world bible and aligned with Unity assets, prefabs, walkable coordinates, and a unified task schema. GPT-4o was the generator, GPT-4o-mini the separate evaluator; 60 validated quests were used for LLM self-evaluation, and 15 participants each completed at least three randomly assigned quests in a Unity WebGL demo with a thirteen-item Likert questionnaire.
  • Analysis: structural smoke/validation tests, textual diversity and redundancy metrics, task-graph metrics, engine-level playability proxies (spatial heatmaps and prefab usage), LLM-based self-evaluation, and a human player study.

Limitations

  • Stated: task schemas and world knowledge remain largely static, constraining long-term narrative evolution and cross-task dependencies; all validations were conducted in a compact 2D Unity RPG benchmark; LLM self-evaluation is inherently biased and treated as auxiliary rather than ground truth; the human study is a pilot with N = 15 and participants ranging from general players to technically informed users, not restricted to expert developers; source code and model checkpoints are not publicly available due to institutional and commercial constraints.
  • Evident: the single compact benchmark and small human sample limit generalizability; the paper explicitly says it does not cover the full organizational KM scope of standards such as ISO 30401, focusing instead on the technical/operational governance layer.

Model-written summary. Check it against the paper before relying on it.

Integrity signals

Registry

Crossref · 10.3390/systems14020175

Cites

50 of 50 references verified0 citation errors · 0 not checkable

Statistics

Nothing to recompute.

Verification

Claims the paper makes

4 Valid · 3 Not valid · 2 Not checked · 116 Own results and interpretations

52 more claims

Create a free account to see all 125 claims and every source verdict.

See all claims — free account
  • It may state two separate claims as one. 29
  • Probably not a substantial claim. 38
  • The quote only partly states this claim. 7

Create a free account to see these claims.

Citations

Cites

Research works printed but not linked: 5

  1. Kang, T., and Lin, M.C. (2025). Action2Dialogue: Generating Character-Centric Narratives from Scene-Level Prompts. arXiv.
  2. Koppen, L. (2024). Integrating a Human Feedback Loop in PCG for Level Design Using LLMs. [Bachelor’s Thesis, University of Twente].
  3. Takayama, J., Ohagi, M., Mizumoto, T., and Yoshikawa, K. (2025). Persona-Consistent Dialogue Generation via Pseudo Preference Tuning. Proceedings of the 31st International Conference on Computational Linguistics, Abu Dhabi, UAE, Association for Computational Linguistics.
  4. Hinton, 2008, Visualizing Data Using t-SNE, J. Mach. Learn. Res., 9, 2579
  5. McIntyre, N., and Lapata, M. (2010). Plot Induction and Evolutionary Search for Story Generation. Proceedings of the 48th Annual Meeting of the Association for Computational Linguistics, Uppsala, Sweden, Association for Computational Linguistics.

Cited by

See all 2

Metadata

Authors
Rahman, A., Yu, A., & Cho, K.
Affiliations
Dongguk University (KR)
Corresponding author
Cho, Kyungeun
Venue
Systems
Publisher
MDPI AG
Published
2026-02-05
Volume
14
Issue
2
Pages
175
ISSN
2079-8954 (ISSN-L)
OpenAlex
W7127883269
Cited by
2
References
23
Citations by year
2026: 2
FWCI
6.93
Type
journal article
Language
en
Open access
Locations
  • Systems (publishedVersion, cc-by)
  • DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion)
Funders
  • Institute of Information and Communications Technology Planning and Evaluation (IITP-2026-RS-2023-00254592)
  • Joycity (2024)
  • IITP (Institute of Information & Communications Technology Planning & Evaluation)-ICAN (IITP-2026-RS-2023-00260248)
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › Artificial Intelligence in Games
  • Physical Sciences › Computer Science › Artificial Intelligence › AI-based Problem Solving and Planning
  • Social Sciences › Sociology and Political Science › Digital Games and Media
Keywords
large language models, role-playing games, knowledge grounding, procedural content generation, unity

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

The claim verdicts this report counts are experimental: their accuracy is not yet measured.

A language model drew the claims from the paper on 2026-10-02, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).

A second model labelled each claim on 2026-10-02 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.