Automated Playtesting With Procedural Personas Through MCTS With Evolved Heuristics
Open accessThe DOI matches its Crossref record; 26 of the 29 references it cites were found in a registry or an index.
Abstract
This paper describes a method for generative player modeling and its application to the automatic testing of game content using archetypal player models called procedural personas. Theoretically grounded in psychological decision theory, procedural personas are implemented using a variation of Monte Carlo tree search (MCTS) where the node selection criteria are developed using evolutionary computation, replacing the standard UCB1 criterion of MCTS. Using these personas, we demonstrate how generative player models can be applied to a varied corpus of game levels and demonstrate how different playstyles can be enacted in each level. In short, we use artificially intelligent personas to construct synthetic playtesters. The proposed approach could be used as a tool for automatic play testing when human feedback is not readily available or when quick visualization of potential interactions is necessary. Possible applications include interactive tools during game development or procedural content generation systems where many evaluations must be conducted within a short time span.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source says what the claim attributes to it.
In the citing paper
“In contrast to prior studies that rely on reward coefficient diversification or utility function adjustments [16], [28], [31], [32], the UBCL framework leverages actual game metrics in the reward function, enabling approximate reproduction of desired human-like behaviors by using game-play data, with accuracy varying across behavioral dimensions.”
From UBCL: A Reinforcement Learning Framework for Controllable and Diverse Player Behaviors (2026) · cites this paper as [16]
The cited source says what the claim attributes to it.
In the citing paper
“Procedural personas [4] [5] accomplish this by rewiring the feedback mechanism by a utility function.”
From Playtesting: What is Beyond Personas (2023) · cites this paper as [4]
The cited source says what the claim attributes to it.
In the citing paper
“Second, depending on the level layout, personas can execute a similar sequence [4].”
From Playtesting: What is Beyond Personas (2023) · cites this paper as [4]
The cited source says what the claim attributes to it.
In the citing paper
“As a result, personas enabled distinct playstyles and helped to playtest a game like distinct players.”
From Playtesting: What is Beyond Personas (2023) · cites this paper as [4]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
A Survey of Monte Carlo Tree Search Methods
Browne, C. B., Powley, E., Whitehouse, D., Lucas, S. M., Cowling, P. I., Rohlfshagen, P., Tavener, S., Perez, D., Samothrakis, S., & Colton, S. (2012) · IEEE Transactions on Computational Intelligence and AI in Games · doi:10.1109/tciaig.2012.2186810 · Cited by 2,970 · cited 4×
“1) Selection: MCTS chooses the next node to expand via the tree policy, starting at the root node and recursively picking the highest scoring child “until the most urgent expandable node is reached” [26] or a terminal state (i.”
BackgroundMethodInfluentialCooper, A. (1999) · Cited by 1,157
The Island Model Genetic Algorithm: On Separability, Population Size and Convergence
Whitley, D., Rana, S., & Heckendorn, R. B. (2015) · Journal of Computing and Information Technology · Cited by 380
Towards automatic personalised content creation for racing games
Togelius, J., De Nardi, R., & Lucas, S. M. (2007) · 2007 IEEE Symposium on Computational Intelligence and Games · doi:10.1109/cig.2007.368106 · Cited by 283 · cited 1×
“Perhaps the most obvious approach is to use some form of supervised learning to derive a model from play traces [18], [19].”
BackgroundSelf-citationGame Analytics : Maximizing the Value of Player Data
El‐Nasr, M. S., Drachen, A., & Canossa, A. (2013) · Cited by 260
Game design workshop : designing, prototyping, and playtesting games
Fullerton, T., Swain, C., & Hoffman, S. J. (2004) · 기초학문자료센터(KRM) (Korea Research Foundation) · Cited by 258
Game Analytics: Maximizing the Value of Player Data
(2013) · Springer eBooks · doi:10.1007/978-1-4471-4769-5 · Cited by 228 · cited 1×
“ase of continuously updated “live” games. Each approach has different strengths, weaknesses and costs, making it relevant for different game makers or different stages of the game making process [1], [2]. In this paper we suggest and demonstrate a method taking a new position on this spectrum (illustrated in Figure 1)…”
MethodSelf-citationDefining personas in games using metrics
Tychsen, A., & Canossa, A. (2008) · Proceedings of the 2008 Conference on Future Play: Research, Play, Share · doi:10.1145/1496984.1496997 · Cited by 169 · cited 1×
“They call this new conceptualization play personas and operationalized their definition through data mining, suggesting how the persona design process could be supported by analyzing quantitative game data gathered via telemetrics [13].”
BackgroundYannakakis, G. N., Spronck, P. H. M., Loiacono, D., & André, E. (2013) · DROPS (Schloss Dagstuhl – Leibniz Center for Informatics) · doi:10.4230/dfu.vol6.12191.45 · Cited by 108 · cited 1×
“Player modeling is the learning and use of computational models of player preference, experience, and/or behavior [17].”
BackgroundEvolving personas for player decision modeling
Holmgard, C., Liapis, A., Togelius, J., & Yannakakis, G. N. (2014) · 2014 IEEE Conference on Computational Intelligence and Games · doi:10.1109/cig.2014.6932911 · Cited by 106 · cited 1×
“featured in previous attempts at modeling personas via MCTS [3] or in the simpler MiniDungeons game [6], [7].”
MethodSelf-citationImitating human playing styles in Super Mario Bros
Ortega, J. G., Shaker, N., Togelius, J., & Yannakakis, G. N. (2012) · Entertainment Computing · doi:10.1016/j.entcom.2012.10.001 · Cited by 104 · cited 1×
“Perhaps the most obvious approach is to use some form of supervised learning to derive a model from play traces [18], [19].”
BackgroundSelf-citationCowley, B. U., & Charles, D. K. (2016) · User Modeling and User-Adapted Interaction · doi:10.1007/s11257-016-9170-1 · Cited by 61 · cited 1×
“Cowley and Charles developed the concept of behavlets, features of play derived from observed action sequences, structured through a top–down application of psychological temperament theory combined with machine learning [20].”
BackgroundPatterns of Play: Play-Personas in User-Centred Game Development
Canossa, A., & Drachen, A. (2009) · doi:10.26503/dl.v2009i1.399 · Cited by 58 · cited 1×
“Canossa and Drachen transported this approach into the realm of game design [12], defining personas less in terms of general life concerns and more in terms of player interaction preferences within the space of the game.”
BackgroundProcedural Personas as Critics for Dungeon Generation
Liapis, A., Holmgård, C., Yannakakis, G. N., & Togelius, J. (2015) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-16549-3_27 · Cited by 50 · cited 3×
“It also suggests that personas could successfully be integrated as critics in a procedural level generation system; this was previously done for MiniDungeons, a simpler game with simpler persona implementations [9].”
BackgroundSelf-citationModifying MCTS for human-like general video game playing
Khalifa, A., Isaksen, A., Togelius, J., & Nealen, A. (2016) · OAR@UM (University of Malta) · Cited by 48
Self-citationLearning Policies for First Person Shooter Games Using Inverse Reinforcement Learning
Tastan, B., & Sukthankar, G. (2011) · Proceedings of the AAAI Conference on Artificial Intelligence and Interactive Digital Entertainment · doi:10.1609/aiide.v7i1.12430 · Cited by 44 · cited 1×
“In contrast, procedural personas are driven by utility functions that can be either specified in a top–down manner by game or persona designers or formulated from play data through methods like inverse reinforcement learning [21] or evolution [14].”
BackgroundGame Metrics Without Players: Strategies for Understanding Game Artifacts
Nelson, M. (2011) · Proceedings of the AAAI Conference on Artificial Intelligence and Interactive Digital Entertainment · doi:10.1609/aiide.v7i3.12479 · Cited by 41
Generative agents for player decision modeling in games
Holmgård, C., Liapis, A., Togelius, J., & Yannakakis, G. N. (2014) · OAR@UM (University of Malta) · Cited by 36
Self-citationPersonas versus Clones for Player Decision Modeling
Holmgård, C., Liapis, A., Togelius, J., & Yannakakis, G. N. (2014) · Lecture notes in computer science · doi:10.1007/978-3-662-45212-7_20 · Cited by 28 · cited 2×
“In contrast, procedural personas are driven by utility functions that can be either specified in a top–down manner by game or persona designers or formulated from play data through methods like inverse reinforcement learning [21] or evolution [14].”
BackgroundSelf-citationMonte-Carlo Tree Search for Simulation-based Play Strategy Analysis.
Zook, A., Harrison, B. E., & Riedl, M. O. (2015) · Foundations of Digital Games · Cited by 28
Combining Gameplay Data with Monte Carlo Tree Search to Emulate Human Play
Devlin, S., Anspoka, A., Sephton, N., Cowling, P., & Rollason, J. (2016) · Proceedings of the AAAI Conference on Artificial Intelligence and Interactive Digital Entertainment · doi:10.1609/aiide.v12i1.12858 · Cited by 28 · cited 2×
“showed how observations of human play data can be used to bias MCTS to play the card game Spades [22].”
BackgroundInvestigating vanilla MCTS scaling on the GVG-AI game corpus
Nelson, M. J. (2016) · IEEE Conference on Computational Intelligence and Games (CIG) · doi:10.1109/cig.2016.7860443 · Cited by 23 · cited 5×
“[24] M. Nelson, “Investigating vanilla MCTS scaling on the GVG-AI game corpus,” in Proc.”
BackgroundResultInfluentialEvolving models of player decision making: Personas versus clones
Holmgård, C., Liapis, A., Togelius, J., & Yannakakis, G. N. (2015) · Entertainment Computing · doi:10.1016/j.entcom.2015.09.002 · Cited by 22 · cited 4×
“This could allow players to be mapped to one of the four personas based on the similarity of persona and player gameplay traces either on the action-by action-level or on a more macrostrategy level, as done in [8].”
BackgroundInfluentialSelf-citationMonte-Carlo Tree Search for Persona Based Player Modeling
Holmgård, C., Liapis, A., Togelius, J., & Yannakakis, G. N. (2015) · Proceedings of the AAAI Conference on Artificial Intelligence and Interactive Digital Entertainment · doi:10.1609/aiide.v11i5.12849 · Cited by 18 · cited 2×
“Prior work has shown that evolutionary methods and MCTS have potential for defining personas for turn-based roguelike games [5],”
BackgroundSelf-citationEvolving Game-Specific UCB Alternatives for General Video Game Playing
Bravi, I., Khalifa, A., Holmgård, C., & Togelius, J. (2017) · Lecture notes in computer science · doi:10.1007/978-3-319-55849-3_26 · Cited by 15 · cited 1×
“explored the possibility of evolving UCB1 replacements that were not specialized for one particular game [28].”
BackgroundSelf-citationMiniDungeons 2: An Experimental Game for Capturing and Modeling Player Decisions.
Holmgård, C., Liapis, A., Togelius, J., & Yannakakis, G. N. (2015) · OAR@UM (University of Malta) · Cited by 7
Self-citation
Research works printed but not linked: 1
- hunicke, 0, MDA: A formal approach to game design and game research, Proc AAAI Workshop Challenges Game AI, 4, 1
Cited by
Deep learning for procedural content generation
Liu, J., Snodgrass, S., Khalifa, A., Risi, S., Yannakakis, G. N., & Togelius, J. (2021) · Neural Computing and Applications · doi:10.1007/s00521-020-05383-8 · Cited by 143
Self-citationBootstrapping Conditional GANs for Video Game Level Generation
Torrado, R. R., Khalifa, A., Green, M. C., Justesen, N., Risi, S., & Togelius, J. (2020) · IEEE Conference on Games (CoG) · doi:10.1109/cog47356.2020.9231576 · Cited by 134
Self-citationA Survey of 15 Years of Data-Driven Persona Development
Salminen, J. O., Guan, K., Jung, S., & Jansen, B. J. J. (2021) · International Journal of Human-Computer Interaction · doi:10.1080/10447318.2021.1908670 · Cited by 142
“These procedural personas simulate real-time decision making under various environmental stimuli, especially in videogame context (Holmgård et al., 2018), and help understand how different player types react to in-game events.”
BackgroundHuman-Like Playtesting with Deep Learning
Gudmundsson, S. F., Eisen, P., Poromaa, E., Nodet, A., Purmonen, S., Kozakowski, B., Meurling, R., & Cao, L. (2018) · 2018 IEEE Conference on Computational Intelligence and Games (CIG) · doi:10.1109/cig.2018.8490442 · Cited by 90
“With player modeling or "personas" [15]–[17] we could learn policies for different clusters of players and build agents for each cluster that better predict the different policies.”
BackgroundImproving Playtesting Coverage via Curiosity Driven Reinforcement Learning Agents
Gordillo, C., Bergdahl, J., Tollmar, K., & Gisslen, L. (2021) · 2021 IEEE Conference on Games (CoG) · doi:10.1109/cog52621.2021.9619048 · Cited by 50
Metadata
- Authors
- Holmgard, C., Green, M. C., Liapis, A., & Togelius, J.
- Affiliations
- New York University (US)
- University of Malta (MT)
- Venue
- IEEE Transactions on Games
- Published
- 2019-12
- Volume
- 11
- Issue
- 4
- Pages
- 352-362
- ISSN
- 2475-1502 (ISSN-L)
- 2475-1510
- OpenAlex
- W2789119104
- Cited by
- 125
- References
- 29
- Citations by year
- 2026: 4 · 2025: 10 · 2024: 7 · 2023: 14 · 2022: 25 · 2021: 17 · 2020: 24 · 2019: 16
- FWCI
- 9.15
- Type
- journal article
- Language
- en
- Open access
- green
- arxiv.org
- Licence
- Locations
- IEEE Transactions on Games (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Funders
- GAANN Program
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Artificial Intelligence in Games
- Social Sciences › Sociology and Political Science › Digital Games and Media
- Physical Sciences › Computer Science › Artificial Intelligence › AI-based Problem Solving and Planning
- Keywords
- Monte Carlo tree search, UCB1, game development tools, procedural content generation
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.