Level Generation Through Large Language Models
Open accessThe DOI matches its Crossref record; 6 of the 6 references it cites were found in a registry or an index.
Abstract
Large Language Models (LLMs) are powerful tools, capable of leveraging their training on natural language to write stories, generate code, and answer questions. But can they generate functional video game levels? Game levels, with their complex functional constraints and spatial relationships in more than one dimension, are very different from the kinds of data an LLM typically sees during training. Datasets of game levels are also hard to come by, potentially taxing the abilities of these data-hungry models. We investigate the use of LLMs to generate levels for the game Sokoban, finding that LLMs are indeed capable of doing so, and that their performance scales dramatically with dataset size. We also perform preliminary experiments on controlling LLM level generators and discuss promising areas for future work.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Language Models are Few-Shot Learners
Brown, T. B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., et al. (2020) · arXiv · doi:10.48550/arxiv.2005.14165 · Cited by 23,372
Scaling Laws for Neural Language Models
Kaplan, J., McCandlish, S., Henighan, T., Brown, T. B., Chess, B., Child, R., Gray, S., Radford, A., Wu, J., & Amodei, D. (2020) · arXiv · doi:10.48550/arxiv.2001.08361 · Cited by 9,550 · cited 1×
“Another well-known property of LLMs is that their performance on a variety of NLP tasks tends to scalewith the amount of training data [7], but does this trend hold for the specialized task of generating game levels? This question is particularly important because in many situations it is difficult or impossible to co…”
BackgroundCodeXGLUE: A Machine Learning Benchmark Dataset for Code Understanding and Generation
Lu, S., Guo, D., Ren, S., Huang, J., Svyatkovskiy, A., Blanco, A., Clement, C., Drain, D., Jiang, D., Tang, D., et al. (2021) · arXiv · doi:10.48550/arxiv.2102.04664 · Cited by 536 · cited 1×
“…WebText dataset, consisting of the content of 45 million links [18], the java-adapted model was pretrained on the CodeSearchNet dataset of Java code, consisting of 1.6 million Java methods [14], and the untrained model, un-surprisingly, received no pretraining (weights are randomly initialized).”
MethodInfluentialPCGRL: Procedural Content Generation via Reinforcement Learning
Khalifa, A., Bontrager, P., Earle, S., & Togelius, J. (2020) · Proceedings of the AAAI Conference on Artificial Intelligence and Interactive Digital Entertainment · doi:10.1609/aiide.v16i1.7416 · Cited by 156
Self-citationReed, S., Zolna, K., Parisotto, E., Colmenarejo, S. G., Novikov, A., Barth-Maron, G., Gimenez, M., Sulsky, Y., Kay, J., Springenberg, J. T., et al. (2022) · arXiv · doi:10.48550/arxiv.2205.06175 · Cited by 63
Schrum, J., Gutierrez, J., Volz, V., Liu, J., Lucas, S., & Risi, S. (2020) · Proceedings of the 2020 Genetic and Evolutionary Computation Conference · doi:10.1145/3377930.3389821 · Cited by 63
Cited by
Huber, S. E., Kiili, K., Nebel, S., Ryan, R. M., Sailer, M., & Ninaus, M. (2024) · Educational Psychology Review · doi:10.1007/s10648-024-09868-z · Cited by 85
Kumaran, V., Rowe, J., Mott, B., & Lester, J. (2023) · Proceedings of the AAAI Conference on Artificial Intelligence and Interactive Digital Entertainment · doi:10.1609/aiide.v19i1.27504 · Cited by 78
“These capabilities of LLMs have been used for procedural content generation (PCG) to create game levels by fine-tuning pretrained GPT-2 models on the ASCII representation of a puzzle game, Sokoban (Todd et al. 2023).”
MethodLLMatic: Neural Architecture Search Via Large Language Models And Quality Diversity Optimization
Nasir, M. U., Earle, S., Togelius, J., James, S., & Cleghorn, C. (2024) · Proceedings of the Genetic and Evolutionary Computation Conference · doi:10.1145/3638529.3654017 · Cited by 52
“2022; Nasir and Mchechesi 2022), they led to creating models for a wide variety of other tasks, such as generating video game levels and code (Chen et al. 2021; Todd et al. 2023; Nasir and Togelius 2023).”
Self-citationPlayer-Driven Emergence in LLM-Driven Game Narrative
Peng, X., Quaye, J., Rao, S., Xu, W., Botchway, P., Brockett, C., Jojic, N., DesGarennes, G., Lobb, K., Xu, M., et al. (2024) · 2024 IEEE Conference on Games (CoG) · doi:10.1109/cog60054.2024.10645607 · Cited by 27
“LLMs have also been used to generate offline game data such as NPC scripts [3], narrative scene [4], affective dialogue [25], 2D-game rooms [26], game levels [27] and game quest descriptions [2], [28].”
MethodGame Generation via Large Language Models
Hu, C., Zhao, Y., & Liu, J. (2024) · 2024 IEEE Conference on Games (CoG) · doi:10.1109/cog60054.2024.10645597 · Cited by 23
“However, they require extra training time and are limited to the given games such as Super Mario Bros. [6] and Sokoban [7].”
Background
Metadata
- Authors
- Todd, G., Earle, S., Nasir, M. U., Green, M. C., & Togelius, J.
- Venue
- Proceedings of the 18th International Conference on the Foundations of Digital Games
- Publisher
- ACM
- Published
- 2023-04-12
- Pages
- 1-8
- OpenAlex
- W4363651325
- Cited by
- 90
- References
- 6
- FWCI
- 17.01
- Type
- conference paper
- Language
- en
- Open access
- bronze
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.