Human-Like Playtesting with Deep Learning
The DOI matches its Crossref record; 50 of the 53 references it cites were found in a registry or an index.
Abstract
We present an approach to learn and deploy human-like playtesting in computer games based on deep learning from player data. We are able to learn and predict the most "human" action in a given position through supervised learning on a convolutional neural network. Furthermore, we show how we can use the learned network to predict key metrics of new content - most notably the difficulty of levels. Our player data and empirical data come from Candy Crush Saga (CCS) and Candy Crush Soda Saga (CCSS). However, the method is general and well suited for many games, in particular where content creation is sequential. CCS and CCSS are non-deterministic match-3 puzzle games with multiple game modes spread over a few thousand levels, providing a diverse testbed for this technique. Compared to Monte Carlo Tree Search (MCTS) we show that this approach increases correlation with average level difficulty, giving more accurate predictions as well as requiring only a fraction of the computation time.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Researchers and developers have seen much success in bringing AI to bear on major problems in the industry, such as cheat detection in competitive games [26], as well as to improve common development processes such as playtesting, bug reporting, and other labor intensive aspects of game production [17,42].”
From Is Resistance Futile?: Early Career Game Developers, Generative AI, and Ethical Skepticism (2024) · cites this paper as [17]
In the citing paper
“Current narratives of these tools as productivity-enhancers, timesavers, and replacement labor [17,42,44] run counter to the values held by this generation of artists, designers, and (with some exception) programmers.”
From Is Resistance Futile?: Early Career Game Developers, Generative AI, and Ethical Skepticism (2024) · cites this paper as [17]
In the citing paper
“Therefore, researchers proposed methods to automate the playtesting process [1] [2] [3].”
From Playtesting: What is Beyond Personas (2023) · cites this paper as [2]
Citations
Cites
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474 · cited 1×
“In addition, we also experimented with batch normalization [44] and residual network [45] but neither managed to provide better generalization capability.”
MethodVery Deep Convolutional Networks for Large-Scale Image Recognition
Simonyan, K., & Zisserman, A. (2014) · arXiv · doi:10.48550/arxiv.1409.1556 · Cited by 114,148 · cited 2×
“It is also worth mentioning that adding the classic Fully-Connected (FC) layers with dropout regularization [38], [42] performed inferior to GAP.”
BackgroundMethodImageNet classification with deep convolutional neural networks
Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017) · Communications of the ACM · doi:10.1145/3065386 · Cited by 112,155
LeCun, Y., Bengio, Y., & Hinton, G. (2015) · Nature · doi:10.1038/nature14539 · Cited by 84,126
Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
Ioffe, S., & Szegedy, C. (2015) · arXiv · doi:10.48550/arxiv.1502.03167 · Cited by 27,818 · cited 1×
“In addition, we also experimented with batch normalization [44] and residual network [45] but neither managed to provide better generalization capability.”
MethodMastering the game of Go with deep neural networks and tree search
Silver, D., Huang, A., Maddison, C. J., Guez, A., Sifre, L., van den Driessche, G., Schrittwieser, J., Antonoglou, I., Panneershelvam, V., Lanctot, M., et al. (2016) · Nature · doi:10.1038/nature16961 · Cited by 15,612 · cited 4×
“[11] (AlphaGo) managed to improve the prediction accuracy to 55.”
BackgroundMethodInfluentialBatch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
Ioffe, S. (2024) · arXiv (Cornell University) · doi:10.57702/o9raffed · Cited by 14,214
Mastering the game of Go without human knowledge
Silver, D., Schrittwieser, J., Simonyan, K., Antonoglou, I., Huang, A., Guez, A., Hubert, T., Baker, L., Lai, M., Bolton, A., et al. (2017) · Nature · doi:10.1038/nature24270 · Cited by 9,084 · cited 1×
“[32] managed to create an agent that could outperform any previously best artificial and human Go player in the game of Go.”
BackgroundFast and Accurate Deep Network Learning by Exponential Linear Units (ELUs)
Clevert, D.-A., Unterthiner, T., & Hochreiter, S. (2015) · arXiv · doi:10.48550/arxiv.1511.07289 · Cited by 4,513 · cited 1×
“[10]–[12], [38], [42]), we opted to use the Exponential Linear Unit (ELU) function [43] instead, because it improved the validation accuracy by about 2.”
MethodA Survey of Monte Carlo Tree Search Methods
Browne, C. B., Powley, E., Whitehouse, D., Lucas, S. M., Cowling, P. I., Rohlfshagen, P., Tavener, S., Perez, D., Samothrakis, S., & Colton, S. (2012) · IEEE Transactions on Computational Intelligence and AI in Games · doi:10.1109/tciaig.2012.2186810 · Cited by 2,970 · cited 1×
“We benchmark this approach against an approach using MCTS [34].”
MethodConvergence Results for Neural Networks via Electrodynamics
Panigrahy, R., Rahimi, A., Sachdeva, S., & Zhang, Q. (2018) · Leibniz international proceedings in informatics · doi:10.4230/lipics.itcs.2018.22 · Cited by 2,761
The Arcade Learning Environment: An Evaluation Platform for General Agents
Bellemare, M. G., Naddaf, Y., Veness, J., & Bowling, M. (2013) · Journal of Artificial Intelligence Research · doi:10.1613/jair.3912 · Cited by 2,161 · cited 1×
“The same method was used in AlphaGo [11] and the success of agents based on reinforcement learning in Atari games [14] with a similar state-action setup suggests that we can do the same for many other types of games.”
MethodEfficient Selectivity and Backup Operators in Monte-Carlo Tree Search
Coulom, R. (2007) · Lecture Notes in Computer Science · doi:10.1007/978-3-540-75538-8_7 · Cited by 1,240 · cited 2×
“com process, until it reaches the end of the search time and chooses an action to perform [30].”
Background(2014) · arXiv (Cornell University) · Cited by 942
Overdispersion: Models and estimation
Hinde, J., & Demétrio, C. G. B. (1998) · Computational Statistics & Data Analysis · doi:10.1016/s0167-9473(98)00007-3 · Cited by 598 · cited 2×
“In statistical literature, such a situation is described as clustered measurements [46].”
BackgroundProcedural Content Generation in Games
Shaker, N., Togelius, J., & Nelson, M. J. (2016) · Computational Synthesis and Creative Systems · doi:10.1007/978-3-319-42716-4 · Cited by 519
Combining online and offline knowledge in UCT
Gelly, S., & Silver, D. (2007) · Proceedings of the 24th international conference on Machine learning · doi:10.1145/1273496.1273531 · Cited by 465
Monte-Carlo tree search and rapid action value estimation in computer Go
Gelly, S., & Silver, D. (2011) · Artificial Intelligence · doi:10.1016/j.artint.2011.03.007 · Cited by 367 · cited 1×
“[11], [32], [34], [36], [37]).”
Evolving mario levels in the latent space of a deep convolutional generative adversarial network
Volz, V., Schrum, J., Liu, J., Lucas, S. M., Smith, A., & Risi, S. (2018) · Proceedings of the Genetic and Evolutionary Computation Conference · doi:10.1145/3205455.3205517 · Cited by 239 · cited 1×
“For example, providing a fitness function for an evolutionary algorithm in a search-based approach [48].”
BackgroundGame Analytics: Maximizing the Value of Player Data
(2013) · Springer eBooks · doi:10.1007/978-1-4471-4769-5 · Cited by 228
Hamari, J., Hanner, N., & Koivisto, J. (2017) · International Journal of Information Management · doi:10.1016/j.ijinfomgt.2016.09.004 · Cited by 177
Multiagent Bidirectionally-Coordinated Nets for Learning to Play StarCraft Combat Games.Preprint
Peng, P., Yuan, Q., Wen, Y., Yang, Y., Tang, Z., Long, H., & Wang, Y.-X. (2017) · arXiv (Cornell University) · Cited by 162
Training Deep Convolutional Neural Networks to Play Go
Clark, C., & Storkey, A. (2015) · Cited by 131
Fast Heuristic Search for RTS Game Combat Scenarios
Churchill, D. G., Saffidine, A., & Buro, M. (2012) · Proceedings of the AAAI Conference on Artificial Intelligence and Interactive Digital Entertainment · doi:10.1609/aiide.v8i1.12527 · Cited by 125 · cited 1×
“[7], [21], [22]).”
Automated Playtesting With Procedural Personas Through MCTS With Evolved Heuristics
Holmgard, C., Green, M. C., Liapis, A., & Togelius, J. (2019) · IEEE Transactions on Games · doi:10.1109/tg.2018.2808198 · Cited by 125 · cited 1×
“With player modeling or "personas" [15]–[17] we could learn policies for different clusters of players and build agents for each cluster that better predict the different policies.”
BackgroundTowards gameplay analysis via gameplay metrics
Drachen, A., & Canossa, A. (2009) · Proceedings of the 13th International MindTrek Conference: Everyday Life in the Ubiquitous Era · doi:10.1145/1621841.1621878 · Cited by 109 · cited 1×
“Therefore, game designers usually let new content be playtested and tune the parameters in an iterative manner based on data obtained from those tests before releasing the new content to players [4], [5].”
BackgroundEvolving personas for player decision modeling
Holmgard, C., Liapis, A., Togelius, J., & Yannakakis, G. N. (2014) · 2014 IEEE Conference on Computational Intelligence and Games · doi:10.1109/cig.2014.6932911 · Cited by 106 · cited 1×
“With player modeling or "personas" [15]–[17] we could learn policies for different clusters of players and build agents for each cluster that better predict the different policies.”
BackgroundFree-to-Play Games: Professionals’ Perspectives
Alha, K., Koskinen, E., Paavilainen, J., Hamari, J., & Kinnunen, J. (2014) · DiGRA Digital Library · doi:10.26503/dl.v2014i2.702 · Cited by 103 · cited 1×
“[1], [2]).”
Fusing Visual and Behavioral Cues for Modeling User Experience in Games
Shaker, N., Asteriadis, S., Yannakakis, G. N., & Karpouzis, K. (2013) · IEEE Transactions on Cybernetics · doi:10.1109/tcyb.2013.2271738 · Cited by 91 · cited 1×
“Player experience can be measured with various metrics [3], [19], [20].”
BackgroundChallenge in Digital Games: Towards Developing a Measurement Tool
Denisova, A., Guckelsberger, C., & Zendle, D. (2017) · Proceedings of the 2017 CHI Conference Extended Abstracts on Human Factors in Computing Systems · doi:10.1145/3027063.3053209 · Cited by 84 · cited 2×
“[3] argue that challenge is the most important player experience.”
BackgroundKnowledge-based fast evolutionary MCTS for general video game playing
Perez, D., Samothrakis, S., & Lucas, S. (2014) · 2014 IEEE Conference on Computational Intelligence and Games · doi:10.1109/cig.2014.6932868 · Cited by 79 · cited 1×
“[7], [21], [22]).”
Exploring Game Space Using Survival Analysis.
Isaksen, A., Gopstein, D., & Nealen, A. (2015) · Foundations of Digital Games · Cited by 72
Matching Games and Algorithms for General Video Game Playing
Bontrager, P., Khalifa, A., Mendes, A., & Togelius, J. (2016) · Proceedings of the AAAI Conference on Artificial Intelligence and Interactive Digital Entertainment · doi:10.1609/aiide.v12i1.12884 · Cited by 55 · cited 1×
“One of the straightforward (but inefficient) solutions is the ensemble method, so authors of [28] investigated the relative performance of 7 algorithms to formulate their approach of general game evaluation and [29] show that there is no "one-fits-all" AI-algorithm available yet in General Video Game Playing.”
Recent Advances in General Game Playing
Świechowski, M., Park, H., Mańdziuk, J., & Kim, K.-J. (2015) · The Scientific World JOURNAL · doi:10.1155/2015/986262 · Cited by 54 · cited 1×
“For example, they are the state-of-the-art in General Game Playing (GGP) [35] and were a key component for the improvement of Go programs (e.”
BackgroundGeneral Video Game Evaluation Using Relative Algorithm Performance Profiles
Nielsen, T. S., Barros, G. A. B., Togelius, J., & Nelson, M. J. (2015) · Lecture notes in computer science · doi:10.1007/978-3-319-16549-3_30 · Cited by 53 · cited 1×
“One of the straightforward (but inefficient) solutions is the ensemble method, so authors of [28] investigated the relative performance of 7 algorithms to formulate their approach of general game evaluation and [29] show that there is no "one-fits-all" AI-algorithm available yet in General Video Game Playing.”
Modifying MCTS for human-like general video game playing
Khalifa, A., Isaksen, A., Togelius, J., & Nealen, A. (2016) · OAR@UM (University of Malta) · Cited by 48
Predicting Player Experience without the Player. An Exploratory Study
Guckelsberger, C., Salge, C., Gow, J., & Cairns, P. A. (2017) · Annual Symposium on Computer-Human Interaction in Play · doi:10.1145/3116595.3116631 · Cited by 43 · cited 1×
“Player experience can be measured with various metrics [3], [19], [20].”
BackgroundAutomatic Playtesting for Game Parameter Tuning via Active LearningPreprint
Zook, A., Fruchter, E., & Riedl, M. O. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1908.01417 · Cited by 41 · cited 1×
“In an attempt to tackle these disadvantages several approaches for automatic playtesting have been proposed [6]– [9].”
BackgroundHyper-heuristic general video game playing
Mendes, A., Togelius, J., & Nealen, A. (2016) · IEEE Conference on Computational Intelligence and Games (CIG) · doi:10.1109/cig.2016.7860398 · Cited by 38 · cited 2×
“[8], [25]–[27]) served as one of the mainstream approaches for simulating gameplay (as discussed in section II) and MCTS”
BackgroundMonte-Carlo Tree Search for Simulation-based Strategy AnalysisPreprint
Zook, A., Harrison, B. E., & Riedl, M. O. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1908.01423 · Cited by 36 · cited 1×
“Agents based on Monte-Carlo Tree Search, as have been proposed in [8], [23], [24], are generic and need little game-specific knowledge.”
BackgroundPersonas versus Clones for Player Decision Modeling
Holmgård, C., Liapis, A., Togelius, J., & Yannakakis, G. N. (2014) · Lecture notes in computer science · doi:10.1007/978-3-662-45212-7_20 · Cited by 28
Monte-Carlo Tree Search for Simulation-based Play Strategy Analysis.
Zook, A., Harrison, B. E., & Riedl, M. O. (2015) · Foundations of Digital Games · Cited by 28
Combining Gameplay Data with Monte Carlo Tree Search to Emulate Human Play
Devlin, S., Anspoka, A., Sephton, N., Cowling, P., & Rollason, J. (2016) · Proceedings of the AAAI Conference on Artificial Intelligence and Interactive Digital Entertainment · doi:10.1609/aiide.v12i1.12858 · Cited by 28 · cited 1×
“Agents based on Monte-Carlo Tree Search, as have been proposed in [8], [23], [24], are generic and need little game-specific knowledge.”
BackgroundPreference Learning for Move Prediction and Evaluation Function Approximation in Othello
Rúnarsson, T. P., & Lucas, S. M. (2014) · IEEE Transactions on Computational Intelligence and AI in Games · doi:10.1109/tciaig.2014.2307272 · Cited by 26 · cited 1×
“For instance, Runarsson [31] directly approximated a policy for Othello game using binary classification.”
MethodAnalysis of Motivational Elements of Social Games: A Puzzle Match 3-Games Study Case
Omori, M. T., & Felinto, A. S. (2012) · International Journal of Computer Games Technology · doi:10.1155/2012/640725 · Cited by 24 · cited 1×
“One of the biggest game types in the casual game genre is match-3 puzzle games with a few hundred million monthly active users [18].”
BackgroundMove prediction in Gomoku using deep learning
Shao, K., Zhao, D., Tang, Z., & Zhu, Y. (2016) · Youth Academic Annual Conference of Chinese Association of Automation (YAC) · doi:10.1109/yac.2016.7804906 · Cited by 20 · cited 3×
“[10]–[12], [38], [42]), we opted to use the Exponential Linear Unit (ELU) function [43] instead, because it improved the validation accuracy by about 2.”
BackgroundMethodEnhancements in Monte Carlo tree search algorithms for biased game trees
Imagawa, T., & Kaneko, T. (2015) · IEEE Conference on Computational Intelligence and Games (CIG) · doi:10.1109/cig.2015.7317924 · Cited by 9 · cited 2×
“[8], [25]–[27]) served as one of the mainstream approaches for simulating gameplay (as discussed in section II) and MCTS”
BackgroundCrushing Candy Crush : Predicting Human Success Rate in a Mobile Game using Monte-Carlo Tree Search
Poromaa, E. R. (2017) · KTH Publication Database DiVA (KTH Royal Institute of Technology) · Cited by 7
Self-citationSimulating Human Game Play for Level Difficulty Estimation with Convolutional Neural Networks
Eisen, P. (2017) · KTH Publication Database DiVA (KTH Royal Institute of Technology) · Cited by 6
Self-citationPredicting Game Level Difficulty Using Deep Neural Networks
Purmonen, S. (2017) · KTH Publication Database DiVA (KTH Royal Institute of Technology) · Cited by 4
Self-citation
Research works printed but not linked: 1
- silva, 2016, AI as Evaluator: Search Driven Playtesting in Game Design, Proceedings of AAAI
Cited by
Deep Learning for Video Game Playing
Justesen, N., Bontrager, P., Togelius, J., & Risi, S. (2020) · IEEE Transactions on Games · doi:10.1109/tg.2019.2896986 · Cited by 257
“Recently, the game company King published results using imitation learning to learn policies for playtesting of Candy Crush levels, showing a promising direction for new design tools [41].”
BackgroundDeep learning for procedural content generation
Liu, J., Snodgrass, S., Khalifa, A., Risi, S., Yannakakis, G. N., & Togelius, J. (2021) · Neural Computing and Applications · doi:10.1007/s00521-020-05383-8 · Cited by 143
“A CNN was trained on human player data, and then used to predict the action that human players most likely to select when playing levels that were unseen during training [35].”
MethodAutomated video game testing using synthetic and humanlike agents
Ariyurek, S., Betin-Can, A., & Surer, E. (2021) · IEEE Transactions on Games · doi:10.1109/tg.2019.2947597 · Cited by 66
“Humanlike agents are better suited to analyze the difficulty of the game [10], can become genuine opponents [11],”
BackgroundMethodDeep Learning Application Pros And Cons Over Algorithm
Moshayedi, A. J., Roy, A. S., Kolahdooz, A., & Shuxin, Y. (2022) · EAI Endorsed Transactions on AI and Robotics · doi:10.4108/airo.v1i.19 · Cited by 135
“(2018) [100] showed that it is possible to predict the next most “human” action, and from that, they can update future contents of the game.”
BackgroundAugmenting Automated Game Testing with Deep Reinforcement Learning
Bergdahl, J., Gordillo, C., Tollmar, K., & Gisslen, L. (2020) · 2020 IEEE Conference on Games (CoG) · doi:10.1109/cog47356.2020.9231552 · Cited by 91
“vised learning using human game play data has successfully been applied to testing in 2D mobile puzzle games by predicting human-like actions to evaluate the playability of game levels [6].”
Method
Metadata
- Authors
- Gudmundsson, S. F., Eisen, P., Poromaa, E., Nodet, A., Purmonen, S., Kozakowski, B., Meurling, R., & Cao, L.
- ORCID
- Cao, Lele
- Venue
- 2018 IEEE Conference on Computational Intelligence and Games (CIG)
- Publisher
- IEEE
- Published
- 2018-08
- Pages
- 1-8
- OpenAlex
- W2897406577
- Cited by
- 90
- References
- 53
- Citations by year
- 2025: 10 · 2024: 12 · 2023: 10 · 2022: 15 · 2021: 12 · 2020: 21 · 2019: 10
- FWCI
- 4.39
- Type
- conference paper
- Language
- en
- Open access
- closed
- Locations
- IEEE Conference on Computational Intelligence and Games (CIG) (publishedVersion)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Artificial Intelligence in Games
- Physical Sciences › Computer Science › Artificial Intelligence › Reinforcement Learning in Robotics
- Social Sciences › Sociology and Political Science › Digital Games and Media
- Keywords
- game testing, video games, player data, convolutional neural networks, game content creation
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.