The DOI matches its Crossref record; 50 of the 50 references it cites were found in a registry or an index; 14 of the 17 sources its claims cite were checked for support.
Game Knowledge Management System: Schema-Governed LLM Pipeline for Executable Narrative Generation in RPGs
Open accessAbstract
Procedural approaches have long been used in game development to reduce authoring costs and increase content diversity; however, traditional rule-based systems struggle to scale narrative complexity, whereas recent large language model (LLM)-based methods often produce outputs that are structurally invalid or incompatible with real-time game engines. This gap reflects a fundamental limitation in current practice: generative models lack systematic mechanisms for managing executable game knowledge rather than merely producing free-form narrative texts. To address this issue, we propose a Game Knowledge Management System (G-KMS) that reformulates LLM-based narrative generation as a structured knowledge management process. The proposed framework integrates knowledge grounding, schema-governed generation, normalization-based repair, engine-aligned knowledge admission, and application within a unified pipeline. The system was evaluated on a compact 2D Unity-based RPG benchmark using automated structural and semantic analyses, engine-level playability probes, and a controlled human player study. The experimental results demonstrated high reliability in knowledge admission, stable procedural structures, controlled expressive diversity, and a strong alignment between system-level metrics and player-perceived narrative quality, indicating that LLMs can function as dependable knowledge-construction components when embedded within a governed management pipeline. Beyond the evaluated RPG setting, this study suggests a practical and reproducible approach that may be extended to other executable systems, such as interactive simulations and training environments.
Overview
CoChat Summary
TL;DR
The paper proposes G-KMS, a three-stage schema-governed LLM pipeline that grounds narrative entities, generates JSON quest artifacts under constraints, normalizes/validates them, and executes them in Unity. Evaluated on a compact 2D Unity RPG benchmark, it reports high structural admissibility, stable quest graphs, controlled expressive diversity, and positive player ratings.
Key findings
- Structural validity: final generator smoke-test pass rate was 100%; validation pass rates were 78% at T0.3, 85% at T0.7, and 77% at T1.0. Removing normalization produced 0% validation despite 100% smoke pass, while VLST3 reached 100% smoke and 100% validation.
- Task-graph metrics: average path length was fixed at 3.0; branching ratios were 1.341 (T0.3), 1.361 (T0.7), and 1.362 (T1.0); dead ends were 0; reachable endings were about 5.16, 5.38, and 5.15. The handcrafted golden sample had branching ratio 1.455 and 8 endings.
- Textual diversity: token entropy rose from 1.71 (T0.3) to 2.45 (T0.7) to 2.65 (T1.0); mean semantic similarity fell from 0.612 to 0.607 to 0.598; near-duplicate rate (>0.9) fell from 4.8% to 2.3% to 1.1%; average token count rose from 82.3 to 92.7 to 100.8.
- Evaluation: LLM self-evaluation average overall scores were 4.20 (T0.3), 4.35 (T0.7), and 4.25 (T1.0). In the human study (N = 15), playability/clarity scored 4.17, narrative logic 4.23, world consistency 4.40, character voice/dialogue quality 4.30, interestingness 4.07, and overall experience 4.57; dialogue naturalness was the highest individual score at 4.60.
Methods
- Design science research paradigm; the G-KMS artifact is a three-stage pipeline: data/world modeling, schema-constrained LLM generation, and evaluation/execution in Unity.
- Data/sample: a compact 2D Unity-based RPG benchmark; narrative source text is converted into a world bible and aligned with Unity assets, prefabs, walkable coordinates, and a unified task schema. GPT-4o was the generator, GPT-4o-mini the separate evaluator; 60 validated quests were used for LLM self-evaluation, and 15 participants each completed at least three randomly assigned quests in a Unity WebGL demo with a thirteen-item Likert questionnaire.
- Analysis: structural smoke/validation tests, textual diversity and redundancy metrics, task-graph metrics, engine-level playability proxies (spatial heatmaps and prefab usage), LLM-based self-evaluation, and a human player study.
Limitations
- Stated: task schemas and world knowledge remain largely static, constraining long-term narrative evolution and cross-task dependencies; all validations were conducted in a compact 2D Unity RPG benchmark; LLM self-evaluation is inherently biased and treated as auxiliary rather than ground truth; the human study is a pilot with N = 15 and participants ranging from general players to technically informed users, not restricted to expert developers; source code and model checkpoints are not publicly available due to institutional and commercial constraints.
- Evident: the single compact benchmark and small human sample limit generalizability; the paper explicitly says it does not cover the full organizational KM scope of standards such as ISO 30401, focusing instead on the technical/operational governance layer.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
4 Valid · 3 Not valid · 2 Not checked · 116 Own results and interpretations
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Although this number is lower than that of the handcrafted sample”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Across all configurations of the Final Generator, the smoke test pass rate remained at 100%”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“| Final Generator (T = 0.3) | 100 | 100% | 78% |”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Although handcrafted golden samples exhibited higher branching ratios, the generated tasks consistently achieved nontrivial branching within schema-defined limits”
plainstates an association- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“semantic- and prefab-related errors decrease progressively as normalization and prompt constraints are refined, reaching near-elimination in the final system”
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
52 more claims
Create a free account to see all 125 claims and every source verdict.
- It may state two separate claims as one. 29
- Probably not a substantial claim. 38
- The quote only partly states this claim. 7
Citations
Cites
Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks
Reimers, N., & Gurevych, I. (2019) · Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP) · doi:10.18653/v1/d19-1410 · Cited by 12,778
Generative Agents: Interactive Simulacra of Human Behavior
Park, J. S., O'Brien, J., Cai, C. J., Morris, M. R., Liang, P., & Bernstein, M. S. (2023) · Proceedings of the 36th Annual ACM Symposium on User Interface Software and Technology · doi:10.1145/3586183.3606763 · Cited by 2,187
A Diversity-Promoting Objective Function for Neural Conversation Models
Li, J., Galley, M., Brockett, C., Gao, J., & Dolan, B. (2016) · Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies · doi:10.18653/v1/n16-1014 · Cited by 1,926
Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena
Zheng, L., Chiang, W.-L., Sheng, Y., Zhuang, S., Wu, Z., Zhuang, Y., Lin, Z., Li, Z., Li, D., Xing, E. P., et al. (2023) · arXiv · doi:10.48550/arxiv.2306.05685 · Cited by 1,888
The k-means Algorithm: A Comprehensive Survey and Performance Evaluation
Ahmed, M., Seraj, R., & Islam, S. M. S. (2020) · Electronics · doi:10.3390/electronics9081295 · Cited by 1,717
The Curious Case of Neural Text Degeneration
Holtzman, A., Buys, J., Du, L., Forbes, M., & Choi, Y. (2019) · arXiv · doi:10.48550/arxiv.1904.09751 · Cited by 1,304
Search-Based Procedural Content Generation: A Taxonomy and Survey
Togelius, J., Yannakakis, G. N., Stanley, K. O., & Browne, C. (2011) · IEEE Transactions on Computational Intelligence and AI in Games · doi:10.1109/tciaig.2011.2148116 · Cited by 706
Narrative Planning: Balancing Plot and Character
Riedl, M. O., & Young, R. M. (2010) · Journal of Artificial Intelligence Research · doi:10.1613/jair.2989 · Cited by 522
Procedural Content Generation in Games
Shaker, N., Togelius, J., & Nelson, M. J. (2016) · Computational Synthesis and Creative Systems · doi:10.1007/978-3-319-42716-4 · Cited by 519
Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedback
Bai, Y., Jones, A., Ndousse, K., Askell, A., Chen, A., DasSarma, N., Drain, D., Fort, S., Ganguli, D., Henighan, T., et al. (2022) · arXiv · doi:10.48550/arxiv.2204.05862 · Cited by 442
Research works printed but not linked: 5
- Kang, T., and Lin, M.C. (2025). Action2Dialogue: Generating Character-Centric Narratives from Scene-Level Prompts. arXiv.
- Koppen, L. (2024). Integrating a Human Feedback Loop in PCG for Level Design Using LLMs. [Bachelor’s Thesis, University of Twente].
- Takayama, J., Ohagi, M., Mizumoto, T., and Yoshikawa, K. (2025). Persona-Consistent Dialogue Generation via Pseudo Preference Tuning. Proceedings of the 31st International Conference on Computational Linguistics, Abu Dhabi, UAE, Association for Computational Linguistics.
- Hinton, 2008, Visualizing Data Using t-SNE, J. Mach. Learn. Res., 9, 2579
- McIntyre, N., and Lapata, M. (2010). Plot Induction and Evolutionary Search for Story Generation. Proceedings of the 48th Annual Meeting of the Association for Computational Linguistics, Uppsala, Sweden, Association for Computational Linguistics.
Cited by
Knowledge-Driven Generative Design of Role-Playing Game Scenarios
Owczarek, W., Wróbel, J., & Pęszor, D. (2026) · Applied Sciences · doi:10.3390/app16062966 · Cited by 2
Metadata
- Authors
- Rahman, A., Yu, A., & Cho, K.
- Affiliations
- Dongguk University (KR)
- Corresponding author
- Cho, Kyungeun
- Venue
- Systems
- Publisher
- MDPI AG
- Published
- 2026-02-05
- Volume
- 14
- Issue
- 2
- Pages
- 175
- ISSN
- 2079-8954 (ISSN-L)
- OpenAlex
- W7127883269
- Cited by
- 2
- References
- 23
- Citations by year
- 2026: 2
- FWCI
- 6.93
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- Locations
- Systems (publishedVersion, cc-by)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion)
- Funders
- Institute of Information and Communications Technology Planning and Evaluation (IITP-2026-RS-2023-00254592)
- Joycity (2024)
- IITP (Institute of Information & Communications Technology Planning & Evaluation)-ICAN (IITP-2026-RS-2023-00260248)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Artificial Intelligence in Games
- Physical Sciences › Computer Science › Artificial Intelligence › AI-based Problem Solving and Planning
- Social Sciences › Sociology and Political Science › Digital Games and Media
- Keywords
- large language models, role-playing games, knowledge grounding, procedural content generation, unity
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
The claim verdicts this report counts are experimental: their accuracy is not yet measured.
A language model drew the claims from the paper on 2026-10-02, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-10-02 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Game Knowledge Management System: Schema-Governed LLM Pipeline for Executable Narrative Generation in RPGs. https://app.cochat.ai/papers/10.3390/systems14020175
- The paper
- Rahman, A., Yu, A., & Cho, K. (2026). Game Knowledge Management System: Schema-Governed LLM Pipeline for Executable Narrative Generation in RPGs. Systems. https://doi.org/10.3390/systems14020175
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.