The DOI matches its Crossref record; 50 of the 50 references it cites were found in a registry or an index; 14 of the 17 sources its claims cite were checked for support.
Game Knowledge Management System: Schema-Governed LLM Pipeline for Executable Narrative Generation in RPGs
Open accessAbstract
Procedural approaches have long been used in game development to reduce authoring costs and increase content diversity; however, traditional rule-based systems struggle to scale narrative complexity, whereas recent large language model (LLM)-based methods often produce outputs that are structurally invalid or incompatible with real-time game engines. This gap reflects a fundamental limitation in current practice: generative models lack systematic mechanisms for managing executable game knowledge rather than merely producing free-form narrative texts. To address this issue, we propose a Game Knowledge Management System (G-KMS) that reformulates LLM-based narrative generation as a structured knowledge management process. The proposed framework integrates knowledge grounding, schema-governed generation, normalization-based repair, engine-aligned knowledge admission, and application within a unified pipeline. The system was evaluated on a compact 2D Unity-based RPG benchmark using automated structural and semantic analyses, engine-level playability probes, and a controlled human player study. The experimental results demonstrated high reliability in knowledge admission, stable procedural structures, controlled expressive diversity, and a strong alignment between system-level metrics and player-perceived narrative quality, indicating that LLMs can function as dependable knowledge-construction components when embedded within a governed management pipeline. Beyond the evaluated RPG setting, this study suggests a practical and reproducible approach that may be extended to other executable systems, such as interactive simulations and training environments.
Overview
CoChat Summary
TL;DR
The paper proposes G-KMS, a three-stage schema-governed LLM pipeline that grounds narrative entities, generates JSON quest artifacts under constraints, normalizes/validates them, and executes them in Unity. Evaluated on a compact 2D Unity RPG benchmark, it reports high structural admissibility, stable quest graphs, controlled expressive diversity, and positive player ratings.
Key findings
- Structural validity: final generator smoke-test pass rate was 100%; validation pass rates were 78% at T0.3, 85% at T0.7, and 77% at T1.0. Removing normalization produced 0% validation despite 100% smoke pass, while VLST3 reached 100% smoke and 100% validation.
- Task-graph metrics: average path length was fixed at 3.0; branching ratios were 1.341 (T0.3), 1.361 (T0.7), and 1.362 (T1.0); dead ends were 0; reachable endings were about 5.16, 5.38, and 5.15. The handcrafted golden sample had branching ratio 1.455 and 8 endings.
- Textual diversity: token entropy rose from 1.71 (T0.3) to 2.45 (T0.7) to 2.65 (T1.0); mean semantic similarity fell from 0.612 to 0.607 to 0.598; near-duplicate rate (>0.9) fell from 4.8% to 2.3% to 1.1%; average token count rose from 82.3 to 92.7 to 100.8.
- Evaluation: LLM self-evaluation average overall scores were 4.20 (T0.3), 4.35 (T0.7), and 4.25 (T1.0). In the human study (N = 15), playability/clarity scored 4.17, narrative logic 4.23, world consistency 4.40, character voice/dialogue quality 4.30, interestingness 4.07, and overall experience 4.57; dialogue naturalness was the highest individual score at 4.60.
Methods
- Design science research paradigm; the G-KMS artifact is a three-stage pipeline: data/world modeling, schema-constrained LLM generation, and evaluation/execution in Unity.
- Data/sample: a compact 2D Unity-based RPG benchmark; narrative source text is converted into a world bible and aligned with Unity assets, prefabs, walkable coordinates, and a unified task schema. GPT-4o was the generator, GPT-4o-mini the separate evaluator; 60 validated quests were used for LLM self-evaluation, and 15 participants each completed at least three randomly assigned quests in a Unity WebGL demo with a thirteen-item Likert questionnaire.
- Analysis: structural smoke/validation tests, textual diversity and redundancy metrics, task-graph metrics, engine-level playability proxies (spatial heatmaps and prefab usage), LLM-based self-evaluation, and a human player study.
Limitations
- Stated: task schemas and world knowledge remain largely static, constraining long-term narrative evolution and cross-task dependencies; all validations were conducted in a compact 2D Unity RPG benchmark; LLM self-evaluation is inherently biased and treated as auxiliary rather than ground truth; the human study is a pilot with N = 15 and participants ranging from general players to technically informed users, not restricted to expert developers; source code and model checkpoints are not publicly available due to institutional and commercial constraints.
- Evident: the single compact benchmark and small human sample limit generalizability; the paper explicitly says it does not cover the full organizational KM scope of standards such as ISO 30401, focusing instead on the technical/operational governance layer.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
4 Valid · 3 Not valid · 2 Not checked · 116 Own results and interpretations
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Although this number is lower than that of the handcrafted sample”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Across all configurations of the Final Generator, the smoke test pass rate remained at 100%”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“| Final Generator (T = 0.3) | 100 | 100% | 78% |”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Although handcrafted golden samples exhibited higher branching ratios, the generated tasks consistently achieved nontrivial branching within schema-defined limits”
plainstates an association- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“semantic- and prefab-related errors decrease progressively as normalization and prompt constraints are refined, reaching near-elimination in the final system”
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
52 more claims
Create a free account to see all 125 claims and every source verdict.
- It may state two separate claims as one. 29
- Probably not a substantial claim. 38
- The quote only partly states this claim. 7
Citations
Cites
Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks
Reimers, N., & Gurevych, I. (2019) · Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP) · doi:10.18653/v1/d19-1410 · Cited by 12,778
Generative Agents: Interactive Simulacra of Human Behavior
Park, J. S., O'Brien, J., Cai, C. J., Morris, M. R., Liang, P., & Bernstein, M. S. (2023) · Proceedings of the 36th Annual ACM Symposium on User Interface Software and Technology · doi:10.1145/3586183.3606763 · Cited by 2,187
A Diversity-Promoting Objective Function for Neural Conversation Models
Li, J., Galley, M., Brockett, C., Gao, J., & Dolan, B. (2016) · Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies · doi:10.18653/v1/n16-1014 · Cited by 1,926
Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena
Zheng, L., Chiang, W.-L., Sheng, Y., Zhuang, S., Wu, Z., Zhuang, Y., Lin, Z., Li, Z., Li, D., Xing, E. P., et al. (2023) · arXiv · doi:10.48550/arxiv.2306.05685 · Cited by 1,888
The k-means Algorithm: A Comprehensive Survey and Performance Evaluation
Ahmed, M., Seraj, R., & Islam, S. M. S. (2020) · Electronics · doi:10.3390/electronics9081295 · Cited by 1,717
The Curious Case of Neural Text Degeneration
Holtzman, A., Buys, J., Du, L., Forbes, M., & Choi, Y. (2019) · arXiv · doi:10.48550/arxiv.1904.09751 · Cited by 1,304
Search-Based Procedural Content Generation: A Taxonomy and Survey
Togelius, J., Yannakakis, G. N., Stanley, K. O., & Browne, C. (2011) · IEEE Transactions on Computational Intelligence and AI in Games · doi:10.1109/tciaig.2011.2148116 · Cited by 706
Narrative Planning: Balancing Plot and Character
Riedl, M. O., & Young, R. M. (2010) · Journal of Artificial Intelligence Research · doi:10.1613/jair.2989 · Cited by 522
Procedural Content Generation in Games
Shaker, N., Togelius, J., & Nelson, M. J. (2016) · Computational Synthesis and Creative Systems · doi:10.1007/978-3-319-42716-4 · Cited by 519
Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedback
Bai, Y., Jones, A., Ndousse, K., Askell, A., Chen, A., DasSarma, N., Drain, D., Fort, S., Ganguli, D., Henighan, T., et al. (2022) · arXiv · doi:10.48550/arxiv.2204.05862 · Cited by 442
Procedural Content Generation via Machine Learning (PCGML)
Summerville, A., Snodgrass, S., Guzdial, M., Holmgard, C., Hoover, A. K., Isaksen, A., Nealen, A., & Togelius, J. (2018) · IEEE Transactions on Games · doi:10.1109/tg.2018.2846639 · Cited by 425
A survey of real-time strategy game AI research and competition in StarCraft
Ontanon, S., Synnaeve, G., Uriarte, A., Richoux, F., Churchill, D., & Preuss, M. (2013) · IEEE Transactions on Computational Intelligence and AI in Games · doi:10.1109/tciaig.2013.2286295 · Cited by 423
Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks
Reimers, N., & Gurevych, I. (2019) · arXiv · doi:10.48550/arxiv.1908.10084 · Cited by 343
Neural Text Generation with Unlikelihood Training
Welleck, S., Kulikov, I., Roller, S., Dinan, E., Cho, K., & Weston, J. (2019) · arXiv · doi:10.48550/arxiv.1908.04319 · Cited by 254
Self-citationHuman-level play in the game of Diplomacy by combining language models with strategic reasoning
Meta Fundamental AI Research Diplomacy Team (FAIR)†, Bakhtin, A., Brown, N., Dinan, E., Farina, G., Flaherty, C., Fried, D., Goff, A., Gray, J., Hu, H., et al. (2022) · Science · doi:10.1126/science.ade9097 · Cited by 241
Voyager: An Open-Ended Embodied Agent with Large Language Models
Wang, G., Xie, Y., Jiang, Y., Mandlekar, A., Xiao, C., Zhu, Y., Fan, L., & Anandkumar, A. (2023) · arXiv · doi:10.48550/arxiv.2305.16291 · Cited by 222
Increasing generality in machine learning through procedural content generation
Risi, S., & Togelius, J. (2020) · Nature Machine Intelligence · doi:10.1038/s42256-020-0208-z · Cited by 201
Tanagra: Reactive Planning and Constraint Solving for Mixed-Initiative Level Design
Smith, G., Whitehead, J., & Mateas, M. (2011) · IEEE Transactions on Computational Intelligence and AI in Games · doi:10.1109/tciaig.2011.2159716 · Cited by 156
Character-LLM: A Trainable Agent for Role-Playing
Shao, Y., Li, L., Dai, J., & Qiu, X. (2023) · Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/2023.emnlp-main.814 · Cited by 140
Character-LLM: A Trainable Agent for Role-Playing
Shao, Y., Li, L., Dai, J., & Qiu, X. (2023) · arXiv · doi:10.48550/arxiv.2310.10158 · Cited by 140
Controlling Narrative Generation with Planning Trajectories: The Role of Constraints
Porteous, J., & Cavazza, M. (2009) · Lecture notes in computer science · doi:10.1007/978-3-642-10643-9_28 · Cited by 118
Song, H., Wang, Y., Zhang, W., Liu, X., & Liu, T. (2020) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2020.acl-main.516 · Cited by 107
MarioGPT: Open-Ended Text2Level Generation through Large Language Models
Sudhakaran, S., González-Duque, M., Freiberger, M., Glanois, C., Najarro, E., & Risi, S. (2023) · neural information processing systems · doi:10.52202/075280-2361 · Cited by 90
From Role-Play to Drama-Interaction: An LLM Solution
Wu, W., Wu, H., Jiang, L., Liu, X.-C., Hong, J.-L., Zhao, H., & Zhang, M. (2024) · Annual Meeting of the Association for Computational Linguistics · doi:10.48550/arxiv.2405.14231 · Cited by 66
Language as Reality: A Co-Creative Storytelling Game Experience in 1001 Nights Using Generative AI
Sun, Y., Li, Z., Fang, K., Lee, C. H., & Asadipour, A. (2023) · Proceedings of the AAAI Conference on Artificial Intelligence and Interactive Digital Entertainment · doi:10.1609/aiide.v19i1.27539 · Cited by 63
Word2World: Generating Stories and Worlds through Large Language ModelsPreprint
Nasir, M. U., James, S. D., & Togelius, J. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2405.06686 · Cited by 48
Smith, G., Othenin-Girard, A., Whitehead, J., & Wardrip–Fruin, N. (2012) · International Conference on the Foundations of Digital Games (FDG) · doi:10.1145/2282338.2282375 · Cited by 45
PANGeA: Procedural Artificial Narrative Using Generative AI for Turn-Based, Role-Playing Video Games
Buongiorno, S., Klinkert, L., Zhuang, Z., Chawla, T., & Clark, C. (2024) · Proceedings of the AAAI Conference on Artificial Intelligence and Interactive Digital Entertainment · doi:10.1609/aiide.v20i1.31876 · Cited by 37
Enhancing Persona Consistency for LLMs’ Role-Playing using Persona-Aware Contrastive Learning
Ji, K., Lian, Y., Li, L., Gao, J., Li, W., & Dai, B. (2025) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2025.findings-acl.1344 · Cited by 34
Toward Automated Quest Generation in Text-Adventure GamesPreprint
Ammanabrolu, P., Broniec, W., Mueller, A., Paul, J., & Riedl, M. O. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1909.06283 · Cited by 33
Enhancing Persona Consistency for LLMs' Role-Playing using Persona-Aware Contrastive LearningPreprint
Ji, K., Lian, Y., Li, L., Gao, J., Li, W., & Dai, B. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2503.17662 · Cited by 26
Deriving quests from open world mechanics
Alexander, R., & Martens, C. (2017) · Proceedings of the 12th International Conference on the Foundations of Digital Games · doi:10.1145/3102071.3102098 · Cited by 24
Game Generation via Large Language Models
Hu, C., Zhao, Y., & Liu, J. (2024) · 2024 IEEE Conference on Games (CoG) · doi:10.1109/cog60054.2024.10645597 · Cited by 23
Integrating plan-based behavior generation with game environments
Young, R. M., & Riedl, M. O. (2005) · Proceedings of the 2005 ACM SIGCHI International Conference on Advances in computer entertainment technology · doi:10.1145/1178477.1178553 · Cited by 21
What's the Game, then? Opportunities and Challenges for Runtime Behavior Generation
Jennings, N., Wang, H., Li, I., Smith, J., & Hartmann, B. (2024) · Proceedings of the 37th Annual ACM Symposium on User Interface Software and Technology · doi:10.1145/3654777.3676358 · Cited by 20
From Role-Play to Drama-Interaction: An LLM Solution
Wu, W., Wu, H., Jiang, L., Liu, X., Zhao, H., & Zhang, M. (2024) · Findings of the Association for Computational Linguistics ACL 2024 · doi:10.18653/v1/2024.findings-acl.196 · Cited by 20
Am I Me or You? State-of-the-Art Dialogue Models Cannot Maintain an Identity
Shuster, K., Urbanek, J., Szlam, A., & Weston, J. (2022) · Findings of the Association for Computational Linguistics: NAACL 2022 · doi:10.18653/v1/2022.findings-naacl.182 · Cited by 20
Zheng, S., He, K., Yang, L., & Xiong, J. (2024) · IEEE Access · doi:10.1109/access.2024.3393485 · Cited by 18
(2009) · Lecture Notes in Computer Science · doi:10.1007/978-3-642-10643-9 · Cited by 11
Dr.Spider: A Diagnostic Evaluation Benchmark towards Text-to-SQL Robustness
Chang, S., Wang, J., Dong, M., Pan, L., Zhu, H., Li, A. H., Lan, W., Zhang, S., Jiang, J., Lilien, J., et al. (2023) · arXiv · doi:10.48550/arxiv.2301.08881 · Cited by 11
End-to-End Procedural Level Generation in Educational Games with Natural Language Instruction
Kumaran, V., Carpenter, D., Rowe, J., Mott, B., & Lester, J. C. (2023) · IEEE Conference on Games (CoG) · doi:10.1109/cog57401.2023.10333195 · Cited by 10
Procedural Level Generation in Educational Games From Natural Language Instruction
Kumaran, V., Carpenter, D., Rowe, J., Mott, B., & Lester, J. C. (2024) · IEEE Transactions on Games · doi:10.1109/tg.2024.3392670 · Cited by 9
Deriving Quests from Open World MechanicsPreprint
Alexander, R., & Martens, C. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1705.00341 · Cited by 9
Managing and Controlling Digital Role-Playing Game Elements: A Current State of Affairs
da Rocha Franco, A. D. O., de Carvalho, W. V., da Silva, J. W. F., Maia, J. G. R., & de Castro, M. F. (2024) · Entertainment Computing · doi:10.1016/j.entcom.2024.100708 · Cited by 5
Dialogue Shaping: Empowering Agents through NPC Interaction
Zhou, W., Peng, X., & Riedl, M. (2023) · arXiv · doi:10.48550/arxiv.2307.15833 · Cited by 4
Personalized Non-Player Characters: A Framework for Character-Consistent Dialogue Generation
Liu, X., Xie, Z., & Jiang, S. (2025) · AI · doi:10.3390/ai6050093 · Cited by 3
Am I Me or You? State-of-the-Art Dialogue Models Cannot Maintain an IdentityPreprint
Shuster, K., Urbanek, J., Szlam, A. D., et al. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2112.05843 · Cited by 3
Unbounded: A Generative Infinite Game of Character Life Simulation
Li, J., Li, Y., Wadhwa, N., Pritch, Y., Jacobs, D. E., Rubinstein, M., Bansal, M., & Ruiz, N. (2024) · arXiv · doi:10.48550/arxiv.2410.18975 · Cited by 2
Immersive Text Game and Personality ClassificationPreprint
Li, W., Bai, Y., Lu, J., et al. (2022) · arXiv (Cornell University) · doi:10.48550/arxiv.2203.10621
Character-Centered Dialogue Generation from Scene-Level Prompts
Kang, T., & Lin, M. C. (2025) · arXiv · doi:10.48550/arxiv.2505.16819
Research works printed but not linked: 5
- Kang, T., and Lin, M.C. (2025). Action2Dialogue: Generating Character-Centric Narratives from Scene-Level Prompts. arXiv.
- Koppen, L. (2024). Integrating a Human Feedback Loop in PCG for Level Design Using LLMs. [Bachelor’s Thesis, University of Twente].
- Takayama, J., Ohagi, M., Mizumoto, T., and Yoshikawa, K. (2025). Persona-Consistent Dialogue Generation via Pseudo Preference Tuning. Proceedings of the 31st International Conference on Computational Linguistics, Abu Dhabi, UAE, Association for Computational Linguistics.
- Hinton, 2008, Visualizing Data Using t-SNE, J. Mach. Learn. Res., 9, 2579
- McIntyre, N., and Lapata, M. (2010). Plot Induction and Evolutionary Search for Story Generation. Proceedings of the 48th Annual Meeting of the Association for Computational Linguistics, Uppsala, Sweden, Association for Computational Linguistics.
Cited by
Knowledge-Driven Generative Design of Role-Playing Game Scenarios
Owczarek, W., Wróbel, J., & Pęszor, D. (2026) · Applied Sciences · doi:10.3390/app16062966 · Cited by 2
Metadata
- Authors
- Rahman, A., Yu, A., & Cho, K.
- Affiliations
- Dongguk University (KR)
- Corresponding author
- Cho, Kyungeun
- Venue
- Systems
- Publisher
- MDPI AG
- Published
- 2026-02-05
- Volume
- 14
- Issue
- 2
- Pages
- 175
- ISSN
- 2079-8954 (ISSN-L)
- OpenAlex
- W7127883269
- Cited by
- 2
- References
- 23
- Citations by year
- 2026: 2
- FWCI
- 6.93
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- Locations
- Systems (publishedVersion, cc-by)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion)
- Funders
- Institute of Information and Communications Technology Planning and Evaluation (IITP-2026-RS-2023-00254592)
- Joycity (2024)
- IITP (Institute of Information & Communications Technology Planning & Evaluation)-ICAN (IITP-2026-RS-2023-00260248)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Artificial Intelligence in Games
- Physical Sciences › Computer Science › Artificial Intelligence › AI-based Problem Solving and Planning
- Social Sciences › Sociology and Political Science › Digital Games and Media
- Keywords
- large language models, role-playing games, knowledge grounding, procedural content generation, unity
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
The claim verdicts this report counts are experimental: their accuracy is not yet measured.
A language model drew the claims from the paper on 2026-10-02, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-10-02 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.