Automated Playtesting With Procedural Personas Through MCTS With Evolved Heuristics
Open accessThe DOI matches its Crossref record; 26 of the 29 references it cites were found in a registry or an index.
Abstract
This paper describes a method for generative player modeling and its application to the automatic testing of game content using archetypal player models called procedural personas. Theoretically grounded in psychological decision theory, procedural personas are implemented using a variation of Monte Carlo tree search (MCTS) where the node selection criteria are developed using evolutionary computation, replacing the standard UCB1 criterion of MCTS. Using these personas, we demonstrate how generative player models can be applied to a varied corpus of game levels and demonstrate how different playstyles can be enacted in each level. In short, we use artificially intelligent personas to construct synthetic playtesters. The proposed approach could be used as a tool for automatic play testing when human feedback is not readily available or when quick visualization of potential interactions is necessary. Possible applications include interactive tools during game development or procedural content generation systems where many evaluations must be conducted within a short time span.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source says what the claim attributes to it.
In the citing paper
“In contrast to prior studies that rely on reward coefficient diversification or utility function adjustments [16], [28], [31], [32], the UBCL framework leverages actual game metrics in the reward function, enabling approximate reproduction of desired human-like behaviors by using game-play data, with accuracy varying across behavioral dimensions.”
From UBCL: A Reinforcement Learning Framework for Controllable and Diverse Player Behaviors (2026) · cites this paper as [16]
The cited source says what the claim attributes to it.
In the citing paper
“Procedural personas [4] [5] accomplish this by rewiring the feedback mechanism by a utility function.”
From Playtesting: What is Beyond Personas (2023) · cites this paper as [4]
The cited source says what the claim attributes to it.
In the citing paper
“Second, depending on the level layout, personas can execute a similar sequence [4].”
From Playtesting: What is Beyond Personas (2023) · cites this paper as [4]
The cited source says what the claim attributes to it.
In the citing paper
“As a result, personas enabled distinct playstyles and helped to playtest a game like distinct players.”
From Playtesting: What is Beyond Personas (2023) · cites this paper as [4]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
A Survey of Monte Carlo Tree Search Methods
Browne, C. B., Powley, E., Whitehouse, D., Lucas, S. M., Cowling, P. I., Rohlfshagen, P., Tavener, S., Perez, D., Samothrakis, S., & Colton, S. (2012) · IEEE Transactions on Computational Intelligence and AI in Games · doi:10.1109/tciaig.2012.2186810 · Cited by 2,970 · cited 4×
“1) Selection: MCTS chooses the next node to expand via the tree policy, starting at the root node and recursively picking the highest scoring child “until the most urgent expandable node is reached” [26] or a terminal state (i.”
BackgroundMethodInfluentialCooper, A. (1999) · Cited by 1,157
The Island Model Genetic Algorithm: On Separability, Population Size and Convergence
Whitley, D., Rana, S., & Heckendorn, R. B. (2015) · Journal of Computing and Information Technology · Cited by 380
Towards automatic personalised content creation for racing games
Togelius, J., De Nardi, R., & Lucas, S. M. (2007) · 2007 IEEE Symposium on Computational Intelligence and Games · doi:10.1109/cig.2007.368106 · Cited by 283 · cited 1×
“Perhaps the most obvious approach is to use some form of supervised learning to derive a model from play traces [18], [19].”
BackgroundSelf-citationGame Analytics : Maximizing the Value of Player Data
El‐Nasr, M. S., Drachen, A., & Canossa, A. (2013) · Cited by 260
Game design workshop : designing, prototyping, and playtesting games
Fullerton, T., Swain, C., & Hoffman, S. J. (2004) · 기초학문자료센터(KRM) (Korea Research Foundation) · Cited by 258
Game Analytics: Maximizing the Value of Player Data
(2013) · Springer eBooks · doi:10.1007/978-1-4471-4769-5 · Cited by 228 · cited 1×
“ase of continuously updated “live” games. Each approach has different strengths, weaknesses and costs, making it relevant for different game makers or different stages of the game making process [1], [2]. In this paper we suggest and demonstrate a method taking a new position on this spectrum (illustrated in Figure 1)…”
MethodSelf-citationDefining personas in games using metrics
Tychsen, A., & Canossa, A. (2008) · Proceedings of the 2008 Conference on Future Play: Research, Play, Share · doi:10.1145/1496984.1496997 · Cited by 169 · cited 1×
“They call this new conceptualization play personas and operationalized their definition through data mining, suggesting how the persona design process could be supported by analyzing quantitative game data gathered via telemetrics [13].”
BackgroundImitating human playing styles in Super Mario Bros
Ortega, J. G., Shaker, N., Togelius, J., & Yannakakis, G. N. (2012) · Entertainment Computing · doi:10.1016/j.entcom.2012.10.001 · Cited by 121 · cited 1×
“Perhaps the most obvious approach is to use some form of supervised learning to derive a model from play traces [18], [19].”
BackgroundSelf-citationYannakakis, G. N., Spronck, P. H. M., Loiacono, D., & André, E. (2013) · DROPS (Schloss Dagstuhl – Leibniz Center for Informatics) · doi:10.4230/dfu.vol6.12191.45 · Cited by 108 · cited 1×
“Player modeling is the learning and use of computational models of player preference, experience, and/or behavior [17].”
Background
Research works printed but not linked: 4
- [12] A. Canossa and A. Drachen, 'Patterns of Play: Play-Personas in UserCentred Game Development,' in Proceedings of the DiGRA International Conference , 2009. -
- [4] R. Hunicke, M. LeBlanc, and R. Zubek, 'MDA: A formal approach to game design and game research,' in Proceedings of the AAAI Workshop on Challenges in Game AI , vol. 4, no. 1, 2004. -
- [5] C. Holmg˚ ard, A. Liapis, J. Togelius, and G. N. Yannakakis, 'Montecarlo tree search for persona based player modeling,' in Proceedings of the AIIDE workshop on Player Modeling , 2015. -
- [3] C. Holmg˚ ard, A. Liapis, J. Togelius, and G. N. Yannakakis, 'MiniDungeons 2: An Experimental Game for Capturing and Modeling Player Decisions,' in Proceedings of the Foundations of Digital Games Conference , 2015. -
Cited by
Deep learning for procedural content generation
Liu, J., Snodgrass, S., Khalifa, A., Risi, S., Yannakakis, G. N., & Togelius, J. (2021) · Neural Computing and Applications · doi:10.1007/s00521-020-05383-8 · Cited by 143
Self-citationBootstrapping Conditional GANs for Video Game Level Generation
Torrado, R. R., Khalifa, A., Green, M. C., Justesen, N., Risi, S., & Togelius, J. (2020) · IEEE Conference on Games (CoG) · doi:10.1109/cog47356.2020.9231576 · Cited by 134
Self-citationA Survey of 15 Years of Data-Driven Persona Development
Salminen, J. O., Guan, K., Jung, S., & Jansen, B. J. J. (2021) · International Journal of Human-Computer Interaction · doi:10.1080/10447318.2021.1908670 · Cited by 142
“These procedural personas simulate real-time decision making under various environmental stimuli, especially in videogame context (Holmgård et al., 2018), and help understand how different player types react to in-game events.”
BackgroundHuman-Like Playtesting with Deep Learning
Gudmundsson, S. F., Eisen, P., Poromaa, E., Nodet, A., Purmonen, S., Kozakowski, B., Meurling, R., & Cao, L. (2018) · 2018 IEEE Conference on Computational Intelligence and Games (CIG) · doi:10.1109/cig.2018.8490442 · Cited by 90
“With player modeling or "personas" [15]–[17] we could learn policies for different clusters of players and build agents for each cluster that better predict the different policies.”
BackgroundImproving Playtesting Coverage via Curiosity Driven Reinforcement Learning Agents
Gordillo, C., Bergdahl, J., Tollmar, K., & Gisslen, L. (2021) · 2021 IEEE Conference on Games (CoG) · doi:10.1109/cog52621.2021.9619048 · Cited by 50
Two-step constructive approaches for dungeon generation
Green, M. C., Khalifa, A., Alsoughayer, A., Surana, D., Liapis, A., & Togelius, J. (2019) · International Conference on the Foundations of Digital Games (FDG) · doi:10.1145/3337722.3341847 · Cited by 52
Self-citationDungeons & Replicants: Automated Game Balancing via Deep Player Behavior Modeling
Pfau, J., Liapis, A., Volkmar, G., Yannakakis, G. N., & Malaka, R. (2020) · 2020 IEEE Conference on Games (CoG) · doi:10.1109/cog47356.2020.9231958 · Cited by 48
Self-citationWinning Is Not Everything: Enhancing Game Development With Intelligent Agents
Zhao, Y., Borovikov, I., Silva, F. D. M., Beirami, A., Rupert, J., Somers, C., Harder, J., Kolen, J. F., Pinto, J., Pourabolghasem, R., et al. (2020) · IEEE Transactions on Games · doi:10.1109/tg.2020.2990865 · Cited by 83
Video Game Automated Testing Approaches: An Assessment Framework
Albaghajati, A., & Ahmed, M. (2020) · IEEE Transactions on Games · doi:10.1109/tg.2020.3032796 · Cited by 68
Multifunctional Radar Cognitive Jamming Decision Based on Dueling Double Deep Q-Network
Feng, L.-W., Liu, S.-T., & Xu, H.-Z. (2022) · IEEE Access · doi:10.1109/access.2022.3214842 · Cited by 38
“Currently, it has been widely used in game competitions [9], [10], decision optimization [11], scheduling control [12], [13], and many other areas.”
BackgroundThe Future of Data-driven Personas: A Marriage of Online Analytics Numbers and Human Attributes
Salminen, J. O., Jung, S., & Jansen, B. J. J. (2019) · International Conference on Enterprise Information Systems · doi:10.5220/0007744706080615 · Cited by 48
Predicting Game Difficulty and Churn Without Players
Roohi, S., Relas, A., Takatalo, J., Heiskanen, H., & Hämäläinen, P. (2020) · Proceedings of the Annual Symposium on Computer-Human Interaction in Play · doi:10.1145/3410404.3414235 · Cited by 34
Using a Team of General AI Algorithms to Assist Game Design and Testing
Guerrero-Romero, C., Lucas, S. M., & Pérez-Liébana, D. (2018) · IEEE Conference on Computational Intelligence and Games (CIG) · doi:10.1109/cig.2018.8490417 · Cited by 34
The Many AI Challenges of Hearthstone
Hoover, A. K., Togelius, J., Lee, S., & Silva, F. D. M. (2019) · Künstliche Intell. · doi:10.1007/s13218-019-00615-z · Cited by 38
Self-citationAutomated Playtesting of Matching Tile Games
Mugrai, L., Silva, F., Holmgard, C., & Togelius, J. (2019) · 2019 IEEE Conference on Games (CoG) · doi:10.1109/cig.2019.8848057 · Cited by 28
“rd Upper Confidence Bound 1 (UCB1) formula, we followed a strategy similar to that of the work of Christoff Holmg˚ard (et. al.): to use genetic programming to evolve persona-specific evolution formulas [1]. We replaced the standard node selection criteria in MCTS for genetically evolved player persona utility functions.…”
BackgroundMethodInfluentialSelf-citationPlaytesting in Match 3 Game Using Strategic Plays via Reinforcement Learning
Shin, Y., Kim, J.-W., Jin, K., & Kim, Y. (2020) · IEEE Access · doi:10.1109/access.2020.2980380 · Cited by 31
“Numerous methods have been developed to address the issues related to level-designing via automation, including traditional ones such as heuristic methods [2], in which”
MethodInfluentialGenerative Personas That Behave and Experience Like Humans
Barthet, M., Khalifa, A., Liapis, A., & Yannakakis, G. N. (2022) · International Conference on the Foundations of Digital Games (FDG) · doi:10.1145/3555858.3555879 · Cited by 37
“Modeling players’ experience in games is an active research topic that is also tied heavily to automatic play testing [17, 18, 35] and EDPCG [14, 24].”
BackgroundSelf-citationAutomated Gameplay Testing and Validation With Curiosity-Conditioned Proximal Trajectories
Sestini, A., Gisslén, L., Bergdahl, J., Tollmar, K., & Bagdanov, A. D. (2022) · IEEE Transactions on Games · doi:10.1109/tg.2022.3226910 · Cited by 47
Reveal-More: Amplifying Human Effort in Quality Assurance Testing Using Automated Exploration
Chang, K., Aytemiz, B., & Smith, A. M. (2019) · IEEE Conference on Games (CoG) · doi:10.1109/cig.2019.8848091 · Cited by 39
“In further applications to playtesting [24], personas can be developed to figure out how players eventually learn how the game is played.”
BackgroundPfau, J., Liapis, A., Yannakakis, G. N., & Malaka, R. (2022) · IEEE Transactions on Games · doi:10.1109/tg.2022.3167728 · Cited by 42
“The work of Holmgård et al. [15] explored the definition of multiple procedural personas which could test the game as archetypal players, although the definition of such personas still relied on designer input rather than direct human traces.”
BackgroundMethodSelf-citationMonster Carlo: An MCTS-based Framework for Machine Playtesting Unity Games
Keehl, O. G., & Smith, A. M. (2018) · IEEE Conference on Computational Intelligence and Games (CIG) · doi:10.1109/cig.2018.8490363 · Cited by 22
Action Guidance with MCTS for Deep Reinforcement Learning
Kartal, B., Hernández-Leal, P., & Taylor, M. E. (2019) · Proceedings of the AAAI Conference on Artificial Intelligence and Interactive Digital Entertainment · doi:10.1609/aiide.v15i1.5238 · Cited by 28
Search-Based Automated Play Testing of Computer Games: A Model-Based Approach
Ferdous, R., Kifetew, F. M., Prandi, D., Prasetya, I. S. W. B., Shirzadehhajimahmood, S., & Susi, A. (2021) · Lecture notes in computer science · doi:10.1007/978-3-030-88106-1_5 · Cited by 22
“For example Holmg˚ard et al [6] uses Monte Carlo Search Tree (MCTS) based agents to replace human play testers.”
MethodSalminen, J. O., Chhirang, K., Jung, S., Thirumuruganathan, S., Guan, K., & Jansen, B. J. J. (2022) · Big Data · doi:10.1089/big.2021.0177 · Cited by 22
Predicting Game Difficulty and Engagement Using AI Players
Roohi, S., Guckelsberger, C., Relas, A., Heiskanen, H., Takatalo, J., & Hämäläinen, P. (2021) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3474658 · Cited by 35
“[25] have showed how each playing style results in different behavior and interaction with game objects in various dungeon maps.”
BackgroundToward Designer Modeling Through Design Style Clustering
Alvarez, A., Font, J., & Togelius, J. (2022) · IEEE Transactions on Games · doi:10.1109/tg.2022.3143800 · Cited by 27
“and ML approaches to create such models from user data or user-generated gameplay data [4], [11]–[13].”
BackgroundMethodSelf-citationModeling Temporal Target Selection: A Perspective from Its Spatial Correspondence
Yu, D., Syiem, B. V., Irlitti, A., Dingler, T., Velloso, E., & Gonçalves, J. (2023) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3544548.3581011 · Cited by 20
“Previous research suggested that predicting game difficulty is an uneasy task for game designers [16].”
BackgroundJacob, M., Devlin, S., & Hofmann, K. (2020) · Proceedings of the AAAI Conference on Artificial Intelligence and Interactive Digital Entertainment · doi:10.1609/aiide.v16i1.7415 · Cited by 28
Estimating Player Completion Rate in Mobile Puzzle Games Using Reinforcement Learning
Kristensen, J. T., Valdivia, A., & Burelli, P. (2020) · IEEE Conference on Games (CoG) · doi:10.1109/cog47356.2020.9231581 · Cited by 21
Automated Playtesting with a Cognitive Model of Sensorimotor Coordination
Lee, I., Kim, H., & Lee, B. (2021) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3474085.3475429 · Cited by 16
“Predicting the game design’s difficulty is not easy for game designers [18].”
BackgroundANALYSIS OF ARTIFICIAL INTELLIGENCE APPLICATIONS FOR AUTOMATED TESTING OF VIDEO GAMES
Zarembo, I. (2019) · Environment Technology Resources Proceedings of the International Scientific and Practical Conference · doi:10.17770/etr2019vol2.4158 · Cited by 20
“Archetypal player models called procedural personas can be used for generative player modelling and automatic testing of game content [19].”
BackgroundExploratory Automated Analysis of Structural Features of Interactive Narrative
Partlan, N., Carstensdóttir, E., Snodgrass, S., Kleinman, E., Smith, G., Harteveld, C., & El‐Nasr, M. S. (2018) · Proceedings of the AAAI Conference on Artificial Intelligence and Interactive Digital Entertainment · doi:10.1609/aiide.v14i1.13019 · Cited by 21
“Using our playthrough approach, we can develop automated playtesting tools for interactive narrative, perhaps by adding procedural personas (Holmgård et al. 2018).”
MethodGuzdial, M., & Riedl, M. O. (2021) · IEEE Transactions on Games · doi:10.1109/tg.2021.3060005 · Cited by 23
Advances in AI-assisted Game Testing
Roohi, S. (2022) · Aaltodoc (Aalto University) · Cited by 16
Technical Challenges of Deploying Reinforcement Learning Agents for Game Testing in AAA Games
Gillberg, J., Bergdahl, J., Sestini, A., Eakins, A., & Gisslén, L. (2023) · IEEE Conference on Games (CoG) · doi:10.1109/cog57401.2023.10333194 · Cited by 31
“These works have primarily relied on classical hand-scripted AI [11, 12, 13] or proposed model-based testing, such as Iftikhar et al. [14] for platform games, and Lovreto et al. [15] for mobile games [16, 17, 18, 19].”
MethodMonster Carlo 2: Integrating Learning and Tree Search for Machine Playtesting
Keehl, O. G., & Smith, A. M. (2019) · IEEE Conference on Games (CoG) · doi:10.1109/cig.2019.8847989 · Cited by 13
“When the design questions to be asked are not specific to the hand-crafted personas, evolved utility functions could be used instead [5].”
BackgroundInspector: Pixel-Based Automated Game Testing via Exploration, Detection, and Investigation
Liu, G., Cai, M., Zhao, L., Qin, T., Brown, A., Bischoff, J., & Liu, T. (2022) · IEEE Conference on Games (CoG) · doi:10.1109/cog51982.2022.9893630 · Cited by 25
“[9] and [12] built MCTS [41] based agents to exercise synthetic and human-like test goals.”
BackgroundGamers' Behaviour and Communication Analysis in Massively Multiplayer Online Games: A Survey
Rezvani, M. H., & Diana, K. (2018) · National and International Digital Games Research Conference: Trends, Technologies, and Applications (DGRC) · doi:10.1109/dgrc.2018.8712055 · Cited by 15
Configurable Agent With Reward As Input: A Play-Style Continuum Generation
Le Pelletier de Woillemont, P., Labory, R., & Corruble, V. (2021) · 2021 IEEE Conference on Games (CoG) · doi:10.1109/cog52621.2021.9619127 · Cited by 12
“To that effect Holmgård et al. use RL [7], evolutionary [8] and MCTS [9] agents.”
BackgroundMethodInfluentialEvaluation of an automatically-constructed graph-based representation for interactive narrative
Partlan, N., Carstensdóttir, E., Kleinman, E., Snodgrass, S., Harteveld, C., Smith, G., Matuk, C. F., Sutherland, S. C., & El‐Nasr, M. S. (2019) · International Conference on the Foundations of Digital Games (FDG) · doi:10.1145/3337722.3341858 · Cited by 16
Winning Isn't Everything: Training Human-Like Agents for Playtesting and Game AIPreprint
Zhao, Y.-Q., Borovikov, I., Beirami, A., Rupert, J., Somers, C., Harder, J., Silva, F., Kolen, J., Pinto, J., Pourabolghasem, R., et al. (2019) · arXiv.org · doi:10.48550/arxiv.1903.10545 · Cited by 10
Toward Automated Assessment of User Experience in Extended Reality
Ansari, S. G. (2020) · IEEE International Conference on Software Testing, Verification and Validation (ICST) · doi:10.1109/icst46399.2020.00056 · Cited by 9
Designing Prototype Player Personas from a Game Preference Survey
Salminen, J. O., Vahlo, J., Koponen, A., Jung, S., Chowdhury, S. A., & Jansen, B. J. J. (2020) · CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA) · doi:10.1145/3334480.3382785 · Cited by 12
Visualizing AI Playtesting Data of 2D Side-scrolling Games
Agarwal, S., Herrmann, C., Wallner, G., & Beck, F. (2020) · IEEE Conference on Games (CoG) · doi:10.1109/cog47356.2020.9231915 · Cited by 13
“Although some of the above mentioned works discuss the potential of visualization (e.g., [7]) or make use of data visualizations (e.g., [4], [8], [12]), the main focus is on the AI technology rather than on dedicated solutions for visualization of AI behavior.”
BackgroundAn online agent-based search approach in automated computer game testing with model construction
Shirzadehhajimahmood, S., Prasetya, I. S. W. B., Dignum, F. P. M., & Dastani, M. M. (2022) · International Workshop on Automating Test Case Design, Selection and Evaluation (A-TEST) · doi:10.1145/3548659.3561309 · Cited by 10
“[9] described a method for generative player modeling through procedural personas and its application to the automatic game testing.”
MethodGenetic-WFC: Extending Wave Function Collapse With Genetic Search
Bailly, R., & Levieux, G. (2022) · IEEE Transactions on Games · doi:10.1109/tg.2022.3192930 · Cited by 11
Automated Play-Testing through RL Based Human-Like Play-Styles Generation
Le Pelletier de Woillemont, P., Labory, R., & Corruble, V. (2022) · Proceedings of the AAAI Conference on Artificial Intelligence and Interactive Digital Entertainment · doi:10.1609/aiide.v18i1.21958 · Cited by 7
Playtesting: What is Beyond Personas
Ariyurek, S., Surer, E., & Betin-Can, A. (2023) · IEEE Transactions on Games · doi:10.1109/tg.2022.3165882 · Cited by 7
New And Surprising Ways to Be Mean
Guckelsberger, C., Salge, C., & Togelius, J. (2018) · IEEE Conference on Computational Intelligence and Games (CIG) · doi:10.1109/cig.2018.8490453 · Cited by 23
Self-citationModel-based Player Experience Testing with Emotion Pattern Verification
Ansari, S. G., Prasetya, I. S. W. B., Prandi, D., Kifetew, F. M., Dastani, M. M., Dignum, F. P. M., & Keller, G. (2023) · Lecture notes in computer science · doi:10.1007/978-3-031-30826-0_9 · Cited by 9
Agent-based Testing of Extended Reality Systems
Prada, R., Prasetya, I. S. W. B., Kifetew, F. M., Dignum, F. P. M., Vos, T. E. J., Lander, J., Donnart, J.-Y., Kazmierowski, A., Davidson, J., & Fernandes, P. M. (2020) · IEEE International Conference on Software Testing, Verification and Validation (ICST) · doi:10.1109/icst46399.2020.00051 · Cited by 9
Radically Simplifying Game Engines: AI Emotions & Game Self-Evolution
Carbone, J. N., Crowder, J. A., & Carbone, R. A. (2020) · International Conference on Computational Science and Computational Intelligence (CSCI) · doi:10.1109/csci51800.2020.00085 · Cited by 11
“In addition, different players will interact with the game systems and characters in different ways, meaning that various strategies and complex interactions can emerge [26].”
BackgroundAdapting Procedural Content Generation to Player Personas Through Evolution
Fernandes, P. M., Jorgensen, J., & Poldervaart, N. N. T. G. (2021) · IEEE Symposium Series on Computational Intelligence (SSCI) · doi:10.1109/ssci50451.2021.9659880 · Cited by 10
Winning Isn't Everything: Training Human-Like Agents for Playtesting and Game AI.Preprint
Zhao, Y., Borovikov, I., Beirami, A., Rupert, J., Somers, C., Harder, J., Silva, F. D. M., Kolen, J. F., Pinto, J., Pourabolghasem, R., et al. (2019) · arXiv (Cornell University) · Cited by 11
Generating Personas for Games with Multimodal Adversarial Imitation Learning
Ahlberg, W., Sestini, A., Tollmar, K., & Gisslén, L. (2023) · 2023 IEEE Conference on Games (CoG) · doi:10.1109/cog57401.2023.10333167 · Cited by 5
“These methods were implemented in 2D discrete environments by engineering their fitness or reward function to condition the policies [10, 17].”
MethodGalván, E., Simpson, G., & Ameneyro, F. V. (2022) · IEEE Transactions on Games · doi:10.1109/tg.2022.3203232 · Cited by 5
“[13] used GP to evolve persona-specific evaluation formulae to be used in MCTS instead of the UCB1 in the deterministic game of MiniDungeons 2.”
MethodNew And Surprising Ways to Be Mean. Adversarial NPCs with Coupled Empowerment MinimisationPreprint
Christian Guckelsberger, B., Salge, C., & Togelius, J. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1806.01387 · Cited by 5
Self-citationSafer Deep RL with Shallow MCTS: A Case Study in PommermanPreprint
Kartal, B., Hernández-Leal, P., Gao, C., & Taylor, M. E. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1904.05759 · Cited by 7
“Other than for game playing agents [6, 47] and playtesting agents [4, 18, 54],”
BackgroundExploring the Dynamic Properties of Interaction in Mixed-Initiative Procedural Content Generation
Alvarez, A. (2020) · doi:10.24834/isbn.9789178771400 · Cited by 6
Game Mechanic Alignment Theory
Green, M. C., Khalifa, A., Canaan, R., Bontrager, P., & Togelius, J. (2021) · International Conference on the Foundations of Digital Games (FDG) · doi:10.1145/3472538.3472571 · Cited by 8
Self-citationCuriosity Driven Multi-agent Reinforcement Learning for 3D Game Testing
Ferdous, R., Kifetew, F. M., Prandi, D., & Susi, A. (2025) · IEEE International Conference on Software Testing, Verification and Validation Workshops (ICSTW) · doi:10.1109/icstw64639.2025.10962505 · Cited by 7
“A few research work is found using single agent RL for game play testing and coverage [29], [30], [31], [32], [33], [34], [35].”
Player Style Clustering without Game Variables
Ferguson, M., Devlin, S., Kudenko⋆, D., & Walker, J. A. (2020) · International Conference on the Foundations of Digital Games (FDG) · doi:10.1145/3402942.3402960 · Cited by 5
“Demonstrations of two play styles were generated based on previously defined personas [11].”
BackgroundInteractive Explanations: Diagnosis and Repair of Reinforcement Learning Based Agent Behaviors
Cruz, C. A., & Igarashi, T. (2021) · IEEE Conference on Games (CoG) · doi:10.1109/cog52621.2021.9618999 · Cited by 11
Reinforcement Learning Methods to Evaluate the Impact of AI Changes in Game Design
Gutiérrez-Sánchez, P., Gómez-Martín, M. A., Gonzalez-Calero, P. A., & Gómez-Martín, P. P. (2021) · Proceedings of the AAAI Conference on Artificial Intelligence and Interactive Digital Entertainment · doi:10.1609/aiide.v17i1.18885 · Cited by 8
“…mechanisms that allow replicating either flows/traces recorded by designers or testers (through imitation learning), or that emulate designer-devised player archetypes (through reward functions adjusted to different play styles, for example, along the lines of the work of (Holmgard et al. 2019)).”
BackgroundMethodAutomatic Generation of Level Maps with the Do What’s Possible Representation
Ashlock, D., & Salge, C. (2019) · IEEE Conference on Games (CoG) · doi:10.1109/cig.2019.8848032 · Cited by 4
Trace It Like You Believe It: Time-Continuous Believability Prediction
Pacheco, C., Melhárt, D., Liapis, A., Yannakakis, G. N., & Pérez-Liébana, D. (2021) · International Conference on Affective Computing and Intelligent Interaction and workshops · doi:10.1109/acii52823.2021.9597457 · Cited by 4
“Not only can believable agents improve the realism of the interaction, but they can be used for human-like automatic testing [1], compete in a game like humans would do [2], [3], collaborate for solving a task [4], or guide humans through a process that may include anything from visiting a…”
BackgroundSelf-citationPredicting Personas Using Mechanic Frequencies and Game State Traces
Green, M. C., Khalifa, A., Charity, M., Bhaumik, D., & Togelius, J. (2022) · IEEE Congress on Evolutionary Computation (CEC) · doi:10.1109/cec55065.2022.9870406 · Cited by 4
“Procedural agents developed via evolution [17], [18] and reinforcement learning [19] have shown to be accurate in emulating player behaviors in a game setting.”
BackgroundMethodSelf-citationButt, N. A., Awais, M. M., Shahzadi, S., Kim, T.-H., & Ashraf, I. (2024) · Journal of King Saud University - Computer and Information Sciences · doi:10.1016/j.jksuci.2024.102182 · Cited by 4
“These agents can assist with automated testing (Holmgård et al., 2018), offer a competitive and challenging game partner (Vinyals et al., 2019), collaborate with humans (Guerrero-Romero et al., 2018), and support the rehabilitation process (Lisetti, 2011), among other tasks.”
Interactive Design Exploration of Game StagesUsing Adjustable Synthetic Testers
Suetake, H., Fukusato, T., Cruz, C. A., Nealen, A., & Igarashi, T. (2020) · International Conference on the Foundations of Digital Games (FDG) · doi:10.1145/3402942.3402982 · Cited by 4
Imitating Playstyle with Dynamic Time Warping Imitation
Ferguson, M., Devlin, S., Kudenko⋆, D., & Walker, J. A. (2022) · International Conference on the Foundations of Digital Games (FDG) · doi:10.1145/3555858.3555878 · Cited by 5
“literature [19, 20] explored defining a set of personas and creating agents to explore these personas within the game.”
BackgroundTowards Understanding the Effects of Evolving the MCTS UCT Selection Policy
Ameneyro, F. V., & Galván, E. (2022) · IEEE Symposium Series on Computational Intelligence (SSCI) · doi:10.1109/ssci51031.2022.10022266 · Cited by 4
“, [8], [5], [20] have proposed evolving the MCTS UCT in hope to get a better performing MCTS.”
BackgroundMethodLambers, L., & Uchitel, S. (2023) · Lecture notes in computer science · doi:10.1007/978-3-031-30826-0 · Cited by 7
The Choice of a Persona: An Analysis of Why Stakeholders Choose a Given Persona for a Design Task
Salminen, J. O., Şengün, S., Santos, J. M., Jung, S., Nielsen, L., & Jansen, B. J. J. (2023) · Lecture notes in computer science · doi:10.1007/978-3-031-48038-6_18 · Cited by 4
Automatic Critical Mechanic Discovery Using Playtraces in Video Games
Green, M. C., Khalifa, A., Barros, G. A. B., Machado, T., & Togelius, J. (2020) · International Conference on the Foundations of Digital Games (FDG) · doi:10.1145/3402942.3402955 · Cited by 5
Self-citationBeyond Playing to Win: Creating a Team of Agents With Distinct Behaviors for Automated Gameplay
Guerrero-Romero, C., Lucas, S. M., & Pérez-Liébana, D. (2023) · IEEE Transactions on Games · doi:10.1109/tg.2023.3241864 · Cited by 3
“Our work is inspired by [13] but it aims to have a more general and portable approach, capable of being applied to several different games without having to design specific types, or utility functions, to fit the game under consideration.”
BackgroundMethodAutomated playtesting of platformer games using reinforcement learning
Sriram, V. (2019) · Digital Repository Service (Northeastern University) · doi:10.17760/d20335186 · Cited by 8
Contact Planning for Multilegged Robots Under Constraints Through Parallel MCTS
Xu, P., Ding, L., Ye, L., Pang, T., Tie, L., Yang, H., Gao, H., Deng, Z., & Pajarinen, J. (2025) · IEEE Transactions on Robotics · doi:10.1109/tro.2025.3619054 · Cited by 2
“Such characteristics have led to the widespread application of MCTS in extensive sequential decision problems, such as Go [26] and computer games [27].”
Automated Testing in Super Metroid with Abstraction-Guided Exploration
Mawhorter, R., & Smith, A. M. (2023) · International Conference on the Foundations of Digital Games (FDG) · doi:10.1145/3582437.3582447 · Cited by 2
“In order to produce this type of feedback, automated testing systems often rely internally on some kind of search to explore the game’s state graph [16, 31].”
BackgroundReinforcement Learning with Temporal Logic Specifications for Regression Testing NPCs in Video Games
Gutiérrez-Sánchez, P., Gómez-Martín, M. A., Gonzalez-Calero, P. A., & Gómez-Martín, P. P. (2023) · IEEE Conference on Games (CoG) · doi:10.1109/cog57401.2023.10333208 · Cited by 2
“In this regard, [18] proposes the use of procedural personas derived from utility functions defined around different play styles to perform automatic game testing, but still requires a manually specified reward function and is only applied to discrete 2D environments.”
BackgroundApplication of AI in Video Games
Bhutkuri, A., Azeez, A., Shetty, D., Saad, M., Vikram, & Y, M. R. H. (2024) · International Journal of Advanced Research in Science Communication and Technology · doi:10.48175/ijarsct-18675 · Cited by 2
A Progress-Based Algorithm for Interpretable Reinforcement Learning in Regression Testing
Gutiérrez-Sánchez, P., Gómez-Martín, M. A., Gonzalez-Calero, P. A., & Gómez-Martín, P. P. (2024) · IEEE Transactions on Games · doi:10.1109/tg.2024.3426601 · Cited by 4
Shyne, F., & Cooper, S. (2024) · Proceedings of the AAAI Conference on Artificial Intelligence and Interactive Digital Entertainment · doi:10.1609/aiide.v20i1.31883 · Cited by 4
Galván, E., & Ameneyro, F. V. (2025) · Information Sciences · doi:10.1016/j.ins.2025.122226 · Cited by 2
Categorical Clustering Applied to the Discovery of Character Builds in TCTD2: the BaT Approach
Renaudie, D., Lizatovic, R., Azadvar, A., Elmqvist, R., Hofmeister, K., & Kristmannsson, B. (2020) · IEEE Conference on Games (CoG) · doi:10.1109/cog47356.2020.9231916 · Cited by 2
“approaches to classify and further analyze player data for a myriad of purposes such as churn behavior and prediction [2], identification of social influencers [3], exploration of emerging game loops [4] and even AI agents with evolving personalities [5].”
BackgroundDiscrete versus Ordinal Time-Continuous Believability Assessment
Pacheco, C., Melhárt, D., Liapis, A., Yannakakis, G. N., & Pérez-Liébana, D. (2021) · International Conference on Affective Computing and Intelligent Interaction Workshops and Demos (ACIIW) · doi:10.1109/aciiw52867.2021.9666288 · Cited by 2
“Human-like behaviour is a crucial property of artificially intelligent agents in the field of Human-Computer Interaction, as these agents can aid automated testing [1], provide a challenging competitive partner [2], collaborate with users [3] or even support a therapy process [4].”
BackgroundSelf-citationSahibgareeva, G. F., & Kugurakova, V. V. (2022) · Program systems theory and applications · doi:10.25209/2079-3316-2022-13-3-255-273 · Cited by 2
Persona-driven Dominant/Submissive Map (PDSM) Generation for Tutorials
Green, M. C., Khalifa, A., Charity, & Togelius, J. (2022) · International Conference on the Foundations of Digital Games (FDG) · doi:10.1145/3555858.3555890 · Cited by 2
“[18, 21], three main personas are identified for MD2:”
BackgroundSelf-citationVideo Game's Mechanics Generation and Balancing
Sahibgareeva, G. F., Kugurakova, V. V., & Bolshakov, E. S. (2022) · Naučnyj servis v seti internet · doi:10.20948/abrau-2022-6 · Cited by 4
Automated structural analysis of interactive narratives
Carstensdóttir, E. (2020) · Digital Repository Service (Northeastern University) · doi:10.17760/d20356170 · Cited by 4
Game Mechanic Alignment Theory and DiscoveryPreprint
Green, M. C., Khalifa, A., Bontrager, P., Canaan, R., & Togelius, J. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2102.10247 · Cited by 2
Self-citationThe Randomizer Community does Procedural Content Generation Research
Mawhorter, R., Mawhorter, P. A., & Smith, A. M. (2022) · International Conference on the Foundations of Digital Games (FDG) · doi:10.1145/3555858.3563270 · Cited by 1
“This creates a unique opportunity for learning from human data (which is already a promising research area [4, 10]).”
BackgroundA case study in AI-assisted board game design
Goodman, J., Wallat, A., Pérez-Liébana, D., & Lucas, S. M. (2023) · IEEE Conference on Games (CoG) · doi:10.1109/cog57401.2023.10333138 · Cited by 6
“Different styles of agents have also been used to play-test a one-person dungeon exploration game, using evolved MCTS heuristics to emulate different anticipated human play-styles [8].”
MethodPrasetya, I. S. W. B., Dastani, M. M., Prada, R., Vos, T. E. J., Dignum, F. P. M., Kifetew, F. M., Mintjes, G., Shirzadehhajimahmood, S., & Ansari, S. G. (2026) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/3742473 · Cited by 2
“Another approach to automating playtesting is the use of machine learning to train a software agent to playtest [Ariyurek et al., 2019; Holmgård et al., 2018; Zheng et al., 2019].”
Comparative Analysis of GAT and BERT for Human-Like Playtesting
Fragkedaki, K., Panagiotakopoulos, T., Biasielli, M., & Wang, H. (2025) · IEEE Conference on Games (CoG) · doi:10.1109/cog64752.2025.11114390 · Cited by 1
“Other approaches search the action space using Monte Carlo Tree Search (MCTS) [10] or combine MCTS with player-based heuristics to emulate specific player styles [8].”
Adversarial Behaviour Debugging in a Two Button Fighting Game
John, N., & Gow, J. (2021) · IEEE Conference on Games (CoG) · doi:10.1109/cog52621.2021.9618893 · Cited by 1
“Various AI methods could be applied, whether that be a suite of MCTS agents with varying personas similar to Holmgård’s procedural personas [20], a population of agents evolved with neuroevolution (explored below), or — if the developer has a glut of compute resources available”
BackgroundThe Data-Driven Persona Revolution
Jansen, B. J. J., Salminen, J. O., Jung, S., & Guan, K. (2021) · Synthesis lectures on human-centered informatics · doi:10.1007/978-3-031-02231-9_1 · Cited by 1
Persona Agents from Playtraces and Emotion
Fernandes, P. M., Lopes, M. C., & Prada, R. (2023) · Proceedings of the AAAI Conference on Artificial Intelligence and Interactive Digital Entertainment · doi:10.1609/aiide.v19i1.27517 · Cited by 1
“Holmg˚ard et al. (Holmg˚ard et al. 2016) created 3 types of evolving models, using neural networks to control them in order to generate human-like behaviour.”
BackgroundData Driven Agents for User Experience Testing
Fernandes, P. M., Lopes, M. C., & Prada, R. (2024) · Communications in computer and information science · doi:10.1007/978-3-031-51452-4_1 · Cited by 1
Şengün, S., Salminen, J. O., Jung, S., Aldous, K. K., & Jansen, B. J. J. (2024) · Designing Interactive Systems Conference · doi:10.1145/3643834.3661512 · Cited by 3
Automated Game Testing With Online Search Agent and Model Construction, a Study
Shirzadehhajimahmood, S., Prasetya, I. S. W. B., Prandi, D., Dignum, F. P. M., Dastani, M. M., & Keller, G. (2025) · Software Testing Verification and Reliability · doi:10.1002/stvr.70002 · Cited by 4
“Its application for automated game testing was explored in, for example, the work of Mugrai et al. and Holmgård et al. [37, 38].”
Metadata
- Authors
- Holmgard, C., Green, M. C., Liapis, A., & Togelius, J.
- Affiliations
- New York University (US)
- University of Malta (MT)
- Venue
- IEEE Transactions on Games
- Published
- 2019-12
- Volume
- 11
- Issue
- 4
- Pages
- 352-362
- ISSN
- 2475-1502 (ISSN-L)
- 2475-1510
- OpenAlex
- W2789119104
- Cited by
- 125
- References
- 29
- Citations by year
- 2026: 4 · 2025: 10 · 2024: 7 · 2023: 14 · 2022: 25 · 2021: 17 · 2020: 24 · 2019: 16
- FWCI
- 9.15
- Type
- journal article
- Language
- en
- Open access
- green
- arxiv.org
- Licence
- Locations
- IEEE Transactions on Games (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Funders
- GAANN Program
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Artificial Intelligence in Games
- Social Sciences › Sociology and Political Science › Digital Games and Media
- Physical Sciences › Computer Science › Artificial Intelligence › AI-based Problem Solving and Planning
- Keywords
- Monte Carlo tree search, UCB1, game development tools, procedural content generation
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.