CoChat
Sign Up
PreprintCited by 99Game Theory and ApplicationsEvolutionary Game Theory and CooperationExperimental Behavioral Economics Studies

Playing repeated games with Large Language Models

Open access
Akata, Elif, Schulz, Lion, Coda-Forno, Julian, Oh, Seong Joon, Bethge, Matthias, Schulz, Eric·2023·arXiv
DOI 10.48550/arxiv.2305.16867arXiv 2305.16867PMC PMC12283376
Open this paper in CoChat — free accountOpen PDF

Abstract

LLMs are increasingly used in applications where they interact with humans and other agents. We propose to use behavioural game theory to study LLM's cooperation and coordination behaviour. We let different LLMs play finitely repeated $2\times2$ games with each other, with human-like strategies, and actual human players. Our results show that LLMs perform particularly well at self-interested games like the iterated Prisoner's Dilemma family. However, they behave sub-optimally in games that require coordination, like the Battle of the Sexes. We verify that these behavioural signatures are stable across robustness checks. We additionally show how GPT-4's behaviour can be modulated by providing additional information about its opponent and by using a "social chain-of-thought" (SCoT) strategy. This also leads to better scores and more successful coordination when interacting with human players. These results enrich our understanding of LLM's social behaviour and pave the way for a behavioural game theory for machines.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

DataCite · 10.48550/arxiv.2305.16867

Cites

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • LLM-based agents can be leveraged for conducting simulation experiments and providing mental health support in psychology.

    “For the domain of psychology, LLM-based agents can be leveraged for conducting simulation experiments, providing mental health support and so on [102-105].”

    From A survey on large language model based autonomous agents (2024) · cites this paper as [103]

    The Discussion and Methods > Human–LLM interactions describe controlled game-theory experiments using LLM agents, supporting the simulation-experiments part of the claim. No page numbers are available in our copy. The paper does not describe LLM agents providing mental health support. (LLM, conf=0.88)

    Experimental — accuracy not yet measured

  • Akata et al. (2023) leverages behavioral game theory to examine LLMs' behavior in interactive social settings, particularly in games like iterated Prisoner's Dilemma and Battle of the Sexes.

    “[ Akata et al., 2023] leverages behavioral game theory to examine LLMs' behavior in interactive social settings, particularly their performance in games like the iterated Prisoner's Dilemma and Battle of the Sexes.”

    From Large Language Model based Multi-Agents: A Survey of Progress and Challenges (2024) · cites this paper as Akata et al., 2023

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

The citation lists haven’t been fetched yet

References 57 · Cited by 99

Research works printed but not linked: 53

  1. [1]Brants, T., Popat, A., Xu, P., Och, F. J. & Dean, J. Large language models in machine translation. In Proc. 2007 Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning (EMNLP-CoNLL) 858–867 (Association for Computational Linguistics, 2007).
  2. [2]Devlin, J., Chang, M. W., Lee, K., & Toutanova, K. Bert: Pre-training of deep bidirectional transformers for language understanding. In Proc. 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies Vol. 1 (Long and Short Papers) 4171–4186 (Association for Computational Linguistics, 2019).
  3. [4]Brown, T. et al. Language models are few-shot learners. Adv. Neural Inform. Process. Syst. 33, 1877–1901 (2020).
  4. [5]Wei, J. et al. Emergent abilities of large language models. Preprint at https://arxiv.org/abs/2206.07682 (2022). · arXiv
  5. [6]Webb, T., Holyoak, K. J. & Lu, H. Emergent analogical reasoning in large language models. Nat. Hum. Behav. 7, 1526–1541 (2023). 37524930
  6. [7]Chen, M. et al. Evaluating large language models trained on code. Preprint at https://arxiv.org/abs/2107.03374 (2021). · arXiv
  7. [8]Bubeck, S. et al. Sparks of artificial general intelligence: early experiments with GPT-4. Preprint at https://arxiv.org/abs/2303.12712 (2023). · arXiv
  8. [9]Coda-Forno, J. et al. Meta-in-context learning in large language models. Adv. Neural Inform. Process. Syst. 36, 65189–65201 (2023).
  9. [10]Bommasani, R. et al. On the opportunities and risks of foundation models. Preprint at https://arxiv.org/abs/2108.07258 (2021). · arXiv
  10. [13]Camerer, C. F. Behavioral Game Theory: Experiments in Strategic Interaction (Princeton University Press, 2011).
  11. [14]Fudenberg, D. & Tirole, J. Game Theory (MIT Press, 1991).
  12. [15]Von Neumann, J. & Morgenstern, O. Theory of Games and Economic Behavior (Princeton Univ. Press, 1944).
  13. [19]Johnson, T. & Obradovich, N. Measuring an artificial intelligence language model’s trust in humans using machine incentives. J. Phys. Complex. 5, 015003 (2024).
  14. [20]Achiam, J. et al. GPT-4 technical report. Preprint at https://arxiv.org/abs/2303.08774 (2023). · arXiv
  15. [21]Owen, G. Game Theory (Emerald Group Publishing, 2013).
  16. [22]Robinson, D. & Goforth, D. The Topology of the 2×2 Games: A New Periodic Table vol. 3 (Psychology Press, 2005).
  17. [24]Axelrod, R. & Hamilton, W. D. The evolution of cooperation. Science 211, 1390–1396 (1981). 7466396
  18. [25]Hawkins, R. X. & Goldstone, R. L. The formation of social conventions in real-time environments. PLoS ONE 11, e0151670 (2016). 27002729 PMC4803472
  19. [27]Andalman, A. & Kemp, C. Alternation in the Repeated Battle of the Sexes. 9.29, Spring 2004, MIT (MIT Press, 2004).
  20. [31]Swettenham, J. What’s inside someone’s head? Conceiving of the mind as a camera helps children with autism acquire an alternative to a theory of mind. Cogn. Neuropsychiatry 1, 73–88 (1996). 16571475
  21. [33]Begeer, S. et al. Theory of mind training in children with autism: a randomized controlled trial. J. Autism Dev. Disord. 41, 997–1006 (2011). 20976617 PMC3134713
  22. [34]Moghaddam, S. R. & Honey, C. J. Boosting theory-of-mind performance in large language models via prompting. Preprint at https://arxiv.org/abs/2304.11490 (2023). · arXiv
  23. [35]Ouyang, L. et al. Training language models to follow instructions with human feedback. Adv. Neural Inform. Process. Syst. 35, 27730–27744 (2022).
  24. [36]Wolf, Y., Wies, N., Levine, Y. & Shashua, A. Fundamental limitations of alignment in large language models. Preprint at https://arxiv.org/abs/2304.11082 (2023). · arXiv
  25. [37]Ullman, T. Large language models fail on trivial alterations to theory-of-mind tasks. Preprint at https://arxiv.org/abs/2302.08399 (2023). · arXiv
  26. [38]Le, M., Boureau, Y.-L. & Nickel, M. Revisiting the evaluation of theory of mind through question answering. In Proc. 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP) 5872–5877 (Association for Computational Linguistics, 2019).
  27. [39]Kosinski, M. Evaluating large language models in theory of mind tasks. Proc. Natl Acad. Sci. USA 121, e2405460121 (2024). 39471222 PMC11551352
  28. [41]Aher, G. V., Arriaga, R. I. & Kalai, A. T. Using large language models to simulate multiple humans and replicate human subject studies. In International Conference on Machine Learning 337–371 (PMLR, 2023).
  29. [43]Nowak, M. A. & Sigmund, K. Evolution of indirect reciprocity. Nature 437, 1291–1298 (2005). 16251955
  30. [45]Nowak, M. A. Five rules for the evolution of cooperation. Science 314, 1560–1563 (2006). 17158317 PMC3279745
  31. [46]Wei, J. et al. Chain-of-thought prompting elicits reasoning in large language models. Adv. Neural Inform. Process. Syst. 35, 24824–24837 (2022).
  32. [48]Rankin, D. J., Bargum, K. & Kokko, H. The tragedy of the commons in evolutionary biology. Trends Ecol. Evol. 22, 643–651 (2007). 17981363
  33. [49]Rahwan, I. et al. in Machine Learning and the City: Applications in Architecture and Urban Design 143–166 (John Wiley & Sons, 2022).
  34. [50]Schulz, E. & Dayan, P. Computational psychiatry for computers. iScience 23, 101772 (2020). 33294781 PMC7691174
  35. [51]Binz, M. & Schulz, E. Using cognitive psychology to understand GPT-3. Proc. Natl Acad. Sci. USA 120, e2218523120 (2023). 36730192 PMC9963545
  36. [52]Coda-Forno, J. et al. Inducing anxiety in large language models increases exploration and bias. Preprint at https://arxiv.org/abs/2304.11111 (2023). · arXiv
  37. [53]Kojima, T., Gu, S. S., Reid, M., Matsuo, Y. & Iwasawa, Y. Large language models are zero-shot reasoners. Adv. Neural Inform. Process. Syst. 35, 22199–22213 (2022).
  38. [54]Rabinowitz, N. et al. Machine theory of mind. In Proc. 35th International Conference on Machine Learning (eds Dy, J. & Krause, A.) 80:4218–4227 (PMLR, 2018).
  39. [55]Cuzzolin, F., Morelli, A., Cirstea, B. & Sahakian, B. J. Knowing me, knowing you: theory of mind in AI. Psychol. Med. 50, 1057–1061 (2020). 32375908 PMC7253617
  40. [56]Alon, N., Schulz, L., Dayan, P. & Rosenschein, J. A (dis-)information theory of revealed and unrevealed preferences. In NeurIPS 2022 Workshop on Information-Theoretic Principles in Cognitive Systems https://openreview.net/pdf?id=vcpQW_fGaj5 (2022).
  41. [57]Frith, C. & Frith, U. Theory of mind. Curr. Biol. 15, R644–R645 (2005). 16139190
  42. [58]Vélez, N. & Gweon, H. Learning from other minds: an optimistic critique of reinforcement learning models of social learning. Curr. Opin. Behav. Sci. 38, 110–115 (2021). 35321420 PMC8936759
  43. [59]Lissek, S. et al. Cooperation and deception recruit different subsets of the theory-of-mind network. PLoS ONE 3, e2023 (2008). 18431500 PMC2295259
  44. [60]Hula, A., Montague, P. R. & Dayan, P. Monte carlo planning method estimates planning horizons during interactive social exchange. PLoS Comput. Biol. 11, e1004254 (2015). 26053429 PMC4460182
  45. [61]Ho, M. K., Saxe, R. & Cushman, F. Planning with theory of mind. Trends Cogn. Sci. 26, 959–971 (2022). 36089494
  46. [63]Chan, A., Riché, M. & Clifton, J. Towards the scalable evaluation of cooperativeness in language models. Preprint at https://arxiv.org/abs/2303.13360 (2023). · arXiv
  47. [64]Lampinen, A. K. et al. Language models, like humans, show content effects on reasoning tasks. PNAS Nexus 3, pgae233 (2024). 39015546 PMC11250216
  48. [66]Goodfellow, I. et al. Generative adversarial networks. Commun. ACM 63, 139–144 (2020).
  49. [68]Guo, H. et al. Facilitating cooperation in human-agent hybrid populations through autonomous agents. iScience 26, 108179 (2023). 37920671 PMC10618689
  50. [69]Powers, S. T. et al. The stuff we swim in: regulation alone will not lead to justifiable trust in AI. IEEE Technol. Soc. Mag. 42, 95–106 (2023).
  51. [73]Wong, R. Y.-m & Hong, Y.-y Dynamic influences of culture on cooperation in the prisoner’s dilemma. Psychol. Sci. 16, 429–434 (2005). 15943667
  52. [74]Liu, P. et al. Pre-train, prompt, and predict: a systematic survey of prompting methods in natural language processing. ACM Comput. Surv. 55, 1–35 (2023).
  53. [75]Rouder, J. N., Speckman, P. L., Sun, D., Morey, R. D. & Iverson, G. Bayesian t tests for accepting and rejecting the null hypothesis. Psychon. Bull. Rev. 16, 225–237 (2009). 19293088

Authors
Akata, E., Schulz, L., Coda-Forno, J., Oh, S. J., Bethge, M., & Schulz, E.
Affiliations
  • Helmholtz Munich (DE)
  • Max Planck Institute for Biological Cybernetics (DE)
  • University of Tübingen (DE)
Corresponding author
Akata, Elif
ORCID
Venue
arXiv
Publisher
arXiv
Published
2023
Volume
9
Issue
7
Pages
1380-1390
ISSN
2397-3374 (ISSN-L)
DOI
10.48550/arxiv.2305.16867
arXiv
2305.16867
PMC
PMC12283376
OpenAlex
W4410251374
Cited by
99
References
57
Citations by year
2026: 39 · 2025: 31 · 2024: 20 · 2023: 9
FWCI
53.84
Type
journal article
Language
en
Open access
Licence
Locations
  • Nature Human Behaviour (publishedVersion, cc-by)
  • PubMed Central (publishedVersion, other-oa)
  • Europe PMC (PubMed Central) (publishedVersion, cc-by)
  • PubMed (publishedVersion)
  • arXiv (Cornell University) (submittedVersion)
  • MPG.PuRe (Max Planck Society) (submittedVersion)
  • arXiv (Cornell University)
Funders
  • International Max Planck Research School for Environmental, Cellular and Molecular Microbiology
  • Deutsche Forschungsgemeinschaft (206/4-1, EXC2064, EXC 2064_1, 390727645)
  • Volkswagen Foundation
  • Bundesministerium für Bildung und Forschung (01IS18039A)
  • Max-Planck-Gesellschaft
  • International Max Planck Research School for Advanced Methods in Process and Systems Engineering
  • Helmholtz Zentrum München
Topics
  • Social Sciences › Decision Sciences › Management Science and Operations Research › Game Theory and Applications
  • Social Sciences › Sociology and Political Science › Evolutionary Game Theory and Cooperation
  • Social Sciences › Safety Research › Experimental Behavioral Economics Studies
Keywords
large language models, repeated games, behavioural game theory, cooperation, coordination, iterated prisoner's dilemma, Battle of the Sexes, human-AI interaction, GPT-4
MeSH terms
Prisoner Dilemma, Large Language Models, Cooperative Behavior, Female, Game Theory, Humans, Language, Male, Social Behavior

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review