CoChat
Sign Up
Cited by 28Artificial Intelligence in GamesReinforcement Learning in RoboticsAI-based Problem Solving and Planning

Combining Gameplay Data with Monte Carlo Tree Search to Emulate Human Play

Open access
Devlin, Sam, Anspoka, Anastasija, Sephton, Nick, Cowling, Peter, Rollason, Jeff·2016·Proceedings of the AAAI Conference on Artificial Intelligence and Interactive Digital Entertainment
DOI 10.1609/aiide.v12i1.12858
Open this paper in CoChat — free accountOpen PDF

Abstract

Monte Carlo Tree Search (MCTS) has become a popular solution for controlling non-player characters. Its use has repeatedly been shown to be capable of creating strong game playing opponents. However, the emergent playstyle of agents using MCTS is not necessarily human-like, believable or enjoyable. AI Factory Spades, currently the top rated Spades game in the Google Play store, uses a variant of MCTS to control non-player characters. In collaboration with the developers, we collected gameplay data from 27,592 games and showed in a previous study that the playstyle of human players significantly differed from that of the non-player characters. This paper presents a method of biasing MCTS using human gameplay data to create Spades playing agents that emulate human play whilst maintaining a strong, competitive performance. The methods of player modelling and biasing MCTS presented in this study are generally applicable to digital games with discrete actions.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1609/aiide.v12i1.12858

Cites

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • The widely used UCT selection formula often produces inhuman game-move choices.

    “the commonly used selection formula, Upper Confidence Bound for Trees (UCT), frequently leads to inhuman choices for game moves (Devlin et al. 2016).”

    From Expressive Response Curves: Testing Expressive Game Feel with A* (2023) · cites this paper as Devlin et al. 2016

    The Monte Carlo Tree Search section calls UCT one of the most commonly used tree policies. The Biasing Monte Carlo Tree Search section shows that move choices by the baseline ISMCTS agent differed from human choices and became more human-like when its UCT score was biased. It does not establish that the UCT formula itself frequently causes inhuman moves across games; the measured comparison is for Spades agents. (LLM, conf=0.85)

    Experimental — accuracy not yet measured

  • Automated playtesting research has mostly targeted generated levels, game balance, and imitation of human decision-making.

    “Automated playtesting for games has largely been focused on evaluating generated levels (Liapis et al. 2015;Powley et al. 2016;Holmgård et al. 2018) and game balance (Zook, Harrison, and Riedl 2015;Pfau et al. 2020Pfau et al. , 2022) ) and emulating human decision making (Devlin et al. 2016;Holmgård et al. 2016).”

    From Expressive Response Curves: Testing Expressive Game Feel with A* (2023) · cites this paper as Devlin et al. 2016

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

No references could be resolved yet.

Research works printed but not linked: 27

  1. Bauckhage, C.; Drachen, A.; and Sifa, R. 2015. Clustering game behavior data. IEEE Transactions on Computational Intelligence and AI in Games 7(3):266-278.
  2. Bohannon, J. 2010. Game-miners grapple with massive data. Science 330(6000):30-31.
  3. Breiman, L.; Friedman, J.; Stone, C. J.; and Olshen, R. A. 1984. Classification and regression trees. CRC press.
  4. Browne, C. B.; Powley, E.; Whitehouse, D.; Lucas, S. M.; Cowling, P.; Rohlfshagen, P.; Tavener, S.; Perez, D.; Samothrakis, S.; and Colton, S. 2012. A survey of Monte Carlo Tree Search Methods. IEEE Transactions on Compu- tational Intelligence and AI in Games 4(1):1-43.
  5. Chaslot, G. M. J.-B.; Saito, J.-T.; Bouzy, B.; Uiterwijk, J. W. H. M.; and van den Herik, H. J. 2006. Monte-Carlo Strategies for Computer Go. In Proceedings of the Benelux Conference on Artificial Intelligence, 83-91.
  6. Coulom, R. 2007. Efficient Selectivity and Backup Opera- tors in Monte-Carlo Tree Search. In Proceedings of the 5th International Conference on Computers and Games, LNCS 4630, 72-83.
  7. Cowling, P. I.; Devlin, S.; Powley, E. J.; Whitehouse, D.; and Rollason, J. 2015. Player preference and style in a leading mobile card game. IEEE Transactions on Computational Intelligence and AI in Games 7(3):233-242.
  8. Cowling, P.; Powley, E. J.; and Whitehouse, D. 2012. Infor- mation set Monte Carlo Tree Search. IEEE Transactions on Computational Intelligence and AI in Games 4(2):120-143.
  9. Dereszynski, E. W.; Hostetler, J.; Fern, A.; Dietterich, T. G.; Hoang, T.-T.; and Udarbe, M. 2011. Learning probabilis- tic behavior models in real-time strategy games. In AAAI Conference on Artificial Intelligence and Interactive Digital Entertainment.
  10. El-Nasr, M. S.; Drachen, A.; and Canossa, A. 2013. Game analytics: Maximizing the value of player data. Springer Science & Business Media.
  11. Fang, X.; Chan, S.; Brzezinski, J.; and Nair, C. 2010. Devel- opment of an instrument to measure enjoyment of computer game play. International Journal of Human-Computer In- teraction 26(9):868-886.
  12. Gelly, S.; Kocsis, L.; Schoenauer, M.; Sebag, M.; Silver, D.; Szepesvári, C.; and Teytaud, O. 2012. The grand challenge of computer Go: Monte Carlo Tree Search and extensions. Communications of the ACM 55(3):106-113.
  13. Ikeda, K., and Viennot, S. 2014. Efficiency of static knowl- edge bias in Monte-Carlo Tree Search. In Computers and Games. Springer. 26-38.
  14. Kocsis, L., and Szepesvári, C. 2006. Bandit based Monte-Carlo planning. In Machine Learning: ECML 2006. Springer. 282-293.
  15. Mekler, E. D.; Bopp, J. A.; Tuch, A. N.; and Opwis, K. 2014. A systematic review of quantitative studies on the enjoyment of digital entertainment games. In Proceedings of the 32nd annual ACM conference on Human factors in computing systems, 927-936. ACM.
  16. Moschitti, A. 2006. Efficient convolution kernels for depen- dency and constituent syntactic trees. In Machine Learning: ECML 2006. Springer. 318-329.
  17. Ortega, J.; Shaker, N.; Togelius, J.; and Yannakakis, G. N. 2013. Imitating human playing styles in Super Mario Bros. Entertainment Computing 4(2):93-104.
  18. Park, H.-S., and Jun, C.-H. 2009. A simple and fast algo- rithm for k-medoids clustering. Expert Systems with Appli- cations 36(2):3336-3341.
  19. Prechelt, L. 1998. Early stopping-but when? In Neural Networks: Tricks of the trade. Springer. 55-69.
  20. Robles-Kelly, A., and Hancock, E. R. 2005. Graph edit dis- tance from spectral seriation. IEEE Transactions on Pattern Analysis and Machine Intelligence 27(3):365-378.
  21. Silver, D.; Huang, A.; Maddison, C. J.; Guez, A.; Sifre, L.; Van Den Driessche, G.; Schrittwieser, J.; Antonoglou, I.; Panneershelvam, V.; Lanctot, M.; Dieleman, S.; Grewe, D.; Nham, J.; Kalchbrenner, N.; Sutskever, I.; Lillicrap, T.; Leach, M.; Kavukcuoglu, K.; Graepel, T.; and Hassabis, D. 2016. Mastering the game of go with deep neural networks and tree search. Nature 529(7587):484-489.
  22. Synnaeve, G., and Bessiere, P. 2011. A Bayesian model for opening prediction in RTS games with application to Star- Craft. In IEEE Conference on Computational Intelligence and Games, 281-288.
  23. Thurau, C.; Paczian, T.; Sagerer, G.; and Bauckhage, C. 2007. Bayesian imitation learning in game characters. In- ternational journal of intelligent systems technologies and applications 2(2-3):284-295.
  24. Togelius, J.; De Nardi, R.; and Lucas, S. M. 2007. Towards automatic personalised content creation for racing games. In 2007 IEEE Symposium on Computational Intelligence and Games, 252-259.
  25. Whitehouse, D.; Cowling, P. I.; Powley, E. J.; and Rolla- son, J. 2013. Integrating Monte Carlo Tree Search with knowledge-based methods to create engaging play in a com- mercial mobile game. In AAAI Conference on Artificial In- telligence and Interactive Digital Entertainment.
  26. Xiao, C., and Müller, M. 2016. Factorization ranking model for move prediction in the game of Go. In AAAI Conference on Artificial Intelligence.
  27. Zook, A.; Harrison, B.; and Riedl, M. O. 2015. Monte- carlo tree search for simulation-based strategy analysis. In Proceedings of the 10th Conference on the Foundations of Digital Games.

Cited by

Sorted by citations

Authors
Devlin, S., Anspoka, A., Sephton, N., Cowling, P., & Rollason, J.
Affiliations
  • University of York (GB)
  • AI Factory (United Kingdom) (GB)
ORCID
Venue
Proceedings of the AAAI Conference on Artificial Intelligence and Interactive Digital Entertainment
Publisher
Association for the Advancement of Artificial Intelligence (AAAI)
Published
2016-10-08
Volume
12
Issue
1
Pages
16-22
ISSN
  • 2326-909X (ISSN-L)
  • 2334-0924
DOI
10.1609/aiide.v12i1.12858
OpenAlex
W2733971344
Cited by
28
References
0
Citations by year
2025: 1 · 2024: 2 · 2023: 3 · 2021: 4 · 2020: 4 · 2019: 7 · 2018: 7
FWCI
2.51
Type
conference paper
Language
en
Open access
Locations
  • Proceedings of the AAAI Conference on Artificial Intelligence and Interactive Digital Entertainment (publishedVersion)
  • AAAI Publications (The Association for the Advancement of Artificial Intelligence (AAAI)) (submittedVersion)
Funders
  • Arts and Humanities Research Council (EP/M023265/1)
  • Engineering and Physical Sciences Research Council (EP/M023265/1)
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › Artificial Intelligence in Games
  • Physical Sciences › Computer Science › Artificial Intelligence › Reinforcement Learning in Robotics
  • Physical Sciences › Computer Science › Artificial Intelligence › AI-based Problem Solving and Planning
Keywords
Monte Carlo tree search, Spades, non-player characters, game AI

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review