Increasing generality in machine learning through procedural content generation
Open accessThe DOI matches its Crossref record; 25 of the 25 references it cites were found in a registry or an index.
Abstract
Procedural content generation (PCG) refers to the practice of generating game content, such as levels, quests or characters, algorithmically. Motivated by the need to make games replayable, as well as to reduce authoring burden and enable particular aesthetics, many PCG methods have been devised. At the same time that researchers are adapting methods from machine learning (ML) to PCG problems, the ML community has become more interested in PCG-inspired methods. One reason for this development is that ML algorithms often only work for a particular version of a particular task with particular initial parameters. In response, researchers have begun exploring randomization of problem parameters to counteract such overfitting and to allow trained policies to more easily transfer from one environment to another, such as from a simulated robot to a robot in the real world. Here we review existing work on PCG, its overlap with current efforts in ML, and promising new research directions such as procedurally generated learning environments. Although originating in games, we believe PCG algorithms are critical to creating more general machine intelligence. This review covers the history of procedural content generation (PCG) approaches for video games, and how these approaches are now used to generate training data and environments for machine learning models. The authors then discuss how PCG may be crucial for training agents which generalise well.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
LeCun, Y., Bengio, Y., & Hinton, G. (2015) · Nature · doi:10.1038/nature14539 · Cited by 84,126
Storn, R., & Price, K. (1997) · Journal of Global Optimization · doi:10.1023/a:1008202821328 · Cited by 29,569
Completely Derandomized Self-Adaptation in Evolution Strategies
Hansen, N., & Ostermeier, A. (2001) · Evolutionary Computation · doi:10.1162/106365601750190398 · Cited by 4,397
Domain randomization for transferring deep neural networks from simulation to the real world
Tobin, J., Fong, R., Ray, A., Schneider, J., Zaremba, W., & Abbeel, P. (2017) · 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros.2017.8202133 · Cited by 3,124 · cited 5×
“In addition to trying to encourage machine learning systems to be more robust and general, another use case of domain randomization is to facilitate the transfer of policies trained in a simulator to the real world [10, 40, 41, 42].”
BackgroundMethodInfluentialContinual lifelong learning with neural networks: A review
Parisi, G. I., Kemker, R., Part, J. L., Kanan, C., & Wermter, S. (2019) · Neural Networks · doi:10.1016/j.neunet.2019.01.012 · Cited by 3,117
Equality of Opportunity in Supervised Learning
Hardt, M., Price, E., & Srebro, N. (2016) · arXiv · doi:10.48550/arxiv.1610.02413 · Cited by 2,536 · cited 1×
“Training neural networks with many free parameters and over long training times has lead to state-of-the-art performance in many domains, but these solutions are typically overfitted to the particular training examples, achieving high accuracy on a training set but performing poorly on data not used for training [2, 3…”
BackgroundThe Effectiveness of Data Augmentation in Image Classification using Deep Learning
Perez, L., & Wang, J. (2017) · arXiv · doi:10.48550/arxiv.1712.04621 · Cited by 2,499
The Arcade Learning Environment: An Evaluation Platform for General Agents
Bellemare, M. G., Naddaf, Y., Veness, J., & Bowling, M. (2013) · Journal of Artificial Intelligence Research · doi:10.1613/jair.3912 · Cited by 2,161
Neural Architecture Search: A Survey
Elsken, T., Metzen, J. H., & Hutter, F. (2018) · arXiv · doi:10.48550/arxiv.1808.05377 · Cited by 1,763
Geirhos, R., Rubisch, P., Michaelis, C., Bethge, M., Wichmann, F. A., & Brendel, W. (2018) · arXiv · doi:10.48550/arxiv.1811.12231 · Cited by 1,490 · cited 1×
“A different form of data augmentation was introduced by Geirhos et al. [37], in which the authors showed that training the same network architecture but with a styl-ized version of ImageNet images (e.g. a cat with the texture of an elephant) can significantly increase the model’s accuracy and…”
Background
Cited by
A Survey of Zero-shot Generalisation in Deep Reinforcement Learning
Kirk, R., Zhang, A., Grefenstette, E., & Rocktäschel, T. (2023) · Journal of Artificial Intelligence Research · doi:10.1613/jair.1.14174 · Cited by 193
“Previous literature has defined PCG as any process by which an algorithm produces MDPs given some input (Risi & Togelius, 2020), which applies to both kinds of context sets we have described.”
MethodThe NetHack Learning Environment
Küttler, H., Nardelli, N., Miller, A. H., Raileanu, R., Selvatici, M., Grefenstette, E., & Rocktäschel, T. (2020) · arXiv · doi:10.48550/arxiv.2006.13760 · Cited by 44
Jiang, M., Grefenstette, E., & Rocktäschel, T. (2020) · arXiv · doi:10.48550/arxiv.2010.03934 · Cited by 33
Replay-Guided Adversarial Environment Design
Jiang, M., Dennis, M., Parker-Holder, J., Foerster, J., Grefenstette, E., & Rocktäschel, T. (2021) · arXiv · doi:10.48550/arxiv.2110.02439 · Cited by 21
“DR itself is subsumed by procedural content generation [30], for which UED and DCD may be seen as providing a formal, decision-theoretic framework, enabling development of provably optimal algorithms.”
BackgroundHuman-Timescale Adaptation in an Open-Ended Task Space
Adaptive Agent Team, Bauer, J., Baumli, K., Baveja, S., Behbahani, F., Bhoopchand, A., Bradley-Schmieg, N., Chang, M., Clay, N., Collister, A., et al. (2023) · arXiv · doi:10.48550/arxiv.2301.07608 · Cited by 19
“PCG has been studied for many years in the games community (Risi and Togelius, 2020; Togelius and Schmidhuber, 2008) and more recently has been used to create testbeds for RL agents (Cobbe et al.”
Method
Metadata
- Authors
- Risi, S., & Togelius, J.
- Venue
- Nature Machine Intelligence
- Published
- 2020-08-03
- Volume
- 2
- Issue
- 8
- Pages
- 428-436
- ISSN
- 2522-5839 (ISSN-L)
- OpenAlex
- W3011540878
- Cited by
- 122
- References
- 25
- FWCI
- 7.01
- Type
- journal article
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Increasing generality in machine learning through procedural content generation. https://app.cochat.ai/papers/10.1038/s42256-020-0208-z
- The paper
- Risi, S., & Togelius, J. (2020). Increasing generality in machine learning through procedural content generation. Nature Machine Intelligence. https://doi.org/10.1038/s42256-020-0208-z
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.