CoChat
Sign Up
Cited by 1Multi-Agent Systems and NegotiationLogic, Reasoning, and KnowledgeArtificial Intelligence in Games

Generative Agents for Multi-Agent Autoformalization of Interaction Scenarios

Open access
Mensfelt, Agnieszka, Stathis, Kostas, Trencsenyi, Vince·2025·Frontiers in Artificial Intelligence and Applications
DOI 10.3233/faia251256
Open this paper in CoChat — free account

Abstract

Multi-agent simulations are a versatile tool for exploring interactions among natural and artificial agents, but their development typically demands domain expertise and manual effort. This work introduces the Generative Agents for Multi-Agent Autoformalization (GAMA) framework, which automates the formalization of interaction scenarios in simulations using agents augmented with large language models (LLMs). To demonstrate the application of GAMA, we use natural language descriptions of game-theoretic scenarios representing social interactions, and we autoformalize them into executable logic programs defining game rules, with syntactic correctness enforced through a solver-based validation. To ensure runtime validity, an iterative, tournament-based procedure tests the generated rules and strategies, followed by exact semantic validation when ground truth outcomes are available. In experiments with 110 natural language descriptions across five 2 × 2 simultaneous-move games, GAMA achieves 100% syntactic and 76.5% semantic correctness with Claude 3.5 Sonnet, and 99.82% syntactic and 77% semantic correctness with GPT-4o. The framework also shows high semantic accuracy in autoformalizing agents’ strategies.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.3233/faia251256

Cites

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • [14] autoformalize game-theoretic scenarios into executable Prolog with an iterative solver-feedback correction loop.

    “[14] autoformalize game-theoretic scenarios into executable Prolog with an iterative solver-feedback correction loop-a close precedent for the translate-run-inspect-repair pattern we propose here.”

    From PrologMCP: A Standardized Prolog Tool Interface for LLM Agents (2026) · cites this paper as [14]

  • A growing line of work has explored combinations of LLMs and Prolog.

    “A growing line of work has explored combinations of LLMs and Prolog.”

    From PrologMCP: A Standardized Prolog Tool Interface for LLM Agents (2026) · cites this paper as [14]

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

No references could be resolved yet.

Research works printed but not linked: 35

  1. [1][1] J. Achiam, S. Adler, S. Agarwal, et al. GPT-4 technical report. arXiv preprint arXiv:2303.08774 , 2023. - · arXiv
  2. [2][2] E. Akata, L. Schulz, J. Coda-Forno, et al. Playing repeated games with large language models. arXiv preprint arXiv:2305.16867 , 2023. - · arXiv
  3. [4][4] R. Axelrod and W. D. Hamilton. The evolution of cooperation. science , 211(4489):1390-1396, 1981. -
  4. [5][5] Y. Chen, R. Gandhi, Y. Zhang, and C. Fan. Nl2tl: Transforming natural languages to temporal logics using large language models. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing , pages 15880-15903, 2023. -
  5. [6][6] M. Cosler, C. Hahn, D. Mendoza, F. Schmitt, and C. Trippel. nl2spec: interactively translating unstructured natural language to temporal logics with large language models. In International Conference on Computer Aided Verification , pages 383-396. Springer, 2023. -
  6. [7][7] C. Fan, J. Chen, Y. Jin, and H. He. Can large language models serve as rational players in game theory? a systematic analysis. arXiv preprint arXiv:2312.05488 , 2023. - · arXiv
  7. [8][8] J. Feng, R. Xu, J. Hao, H. Sharma, Y. Shen, D. Zhao, and W. Chen. Language models can be logical solvers. arXiv preprint arXiv:2311.06158 , 2023. - · arXiv
  8. [9][9] M. R. Genesereth, N. Love, and B. Pell. General game playing: Overview of the AAAI competition. AI Mag. , 26(2):62-72, 2005. -
  9. [10][10] G. D. Giacomo, Y. Lespérance, and A. R. Pearce. Situation calculus based programs for representing and reasoning about game structures. In F. Lin, U. Sattler, and M. Truszczynski, editors, Principles of Knowledge Representation and Reasoning: Proceedings of the Twelfth International Conference, KR 2010, Toronto, Ontario, Canada, May 9-13, 2010 . AAAI Press, 2010. -
  10. [11][11] R. Gibbons. A Primer in Game Theory . Pearson Education Limited, Harlow, Essex, United Kingdom, 1992. -
  11. [12][12] A. Gillioz, J. Casas, E. Mugellini, and O. A. Khaled. Overview of the transformer-based models for nlp tasks. In 2020 15th Conference on Computer Science and Information Systems (FedCSIS) , pages 179-183, 2020. doi: 10.15439/2020F20. - · DOI
  12. [13][13] Z. Gou, Z. Shao, Y. Gong, et al. Critic: Large language models can self-correct with tool-interactive critiquing. arXiv preprint arXiv:2305.11738 , 2024. - · arXiv
  13. [14][14] F. Guo. GPT agents in game theory experiments. arXiv preprint arXiv:2305.05516 , 2023. - · arXiv
  14. [15][15] J. He-Yueya, G. Poesia, R. E. Wang, and N. D. Goodman. Solving math word problems by combining language models with symbolic solvers. arXiv preprint arXiv:2304.09102 , 2023. - · arXiv
  15. [16][16] S. Imani, L. Du, and H. Shrivastava. Mathprompter: Mathematical reasoning using large language models. arXiv preprint arXiv:2303.05398 , 2023. - · arXiv
  16. [17][17] A. Q. Jiang, S. Welleck, J. P. Zhou, W. Li, J. Liu, M. Jamnik, T. Lacroix, Y. Wu, and G. Lample. Draft, sketch, and prove: Guiding formal theorem provers with informal proofs. arXiv preprint arXiv:2210.12283 , 2022. - · arXiv
  17. [18][18] Y. Lesperance, G. De Giacomo, M. Rostamigiv, and S. M. Khan. Abstraction of situation calculus concurrent game structures. Proceedings of the AAAI Conference on Artificial Intelligence , 38(9):10624-10634, Mar. 2024. doi: 10.1609/aaai.v38i9.28933. URL https://ojs.aaai.org/ index.php/AAAI/article/view/28933. - · DOI
  18. [19][19] N. Lorè and B. Heydari. Strategic behavior of large language models: Game structure vs. contextual framing. arXiv preprint arXiv:2309.05898 , 2023. - · arXiv
  19. [21][21] A. Madaan, N. Tandon, P. Gupta, et al. Self-refine: Iterative refinement with self-feedback. arXiv preprint arXiv:2303.17651 , 2023. - · arXiv
  20. [22][22] J. McCarthy and P. Hayes. Some philosophical problems from the standpoint of artificial intelligence. In B. L. Webber and N. J. Nils- 23. son, editors, Readings in Artificial Intelligence , pages 431-450. Morgan Kaufmann, 1981. -
  21. [23][23] A. Mensfelt, K. Stathis, and V. Tencsenyi. Autoformalization of Game Descriptions using Large Language Models. In 1st International Workshop on Next-Generation Language Models for Knowledge Representation and Reasoning , Hanoi, Vietnam, 2024. URL https://arxiv.org/abs/ 2409.12300. -
  22. [25][25] M. Osborne and A. Rubinstein. A Course in Game Theory . The MIT Press, 1994. -
  23. [26][26] L. Pan, A. Albalak, X. Wang, and W. Wang. Logic-lm: Empowering large language models with symbolic solvers for faithful logical reasoning. In Findings of the Association for Computational Linguistics: EMNLP 2023 , pages 3806-3824, 2023. -
  24. [27][27] X. Qiu, T. Sun, Y. Xu, Y. Shao, N. Dai, and X. Huang. Pre-trained models for natural language processing: A survey. Science China Technological Sciences , 63:1872-1897, 2020. doi: 10.1007/s11431-020-1647-3. - · DOI · Paper page
  25. [29][29] M. Renze and E. Guven. Self-reflection in llm agents: Effects on problem-solving performance. arXiv preprint arXiv:2405.06682 , 2024. - · arXiv
  26. [30][30] R. B. Scherl and H. J. Levesque. Knowledge, action, and the frame problem. Artif. Intell. , 144(1-2):1-39, 2003. doi: 10.1016/S0004-3702(02)00365-X. URL https://doi.org/10.1016/ S0004-3702(02)00365-X. - · DOI
  27. [31][31] S. Schiffel and M. Thielscher. Reasoning about general games described in gdl-ii. Proceedings of the AAAI Conference on Artificial Intelligence , 25(1):846-851, Aug. 2011. doi: 10.1609/aaai.v25i1.7944. - · DOI
  28. [32][32] N. Shinn, F. Cassano, E. Berman, et al. Reflexion: Language agents with verbal reinforcement learning. arXiv preprint arXiv:2303.11366 , 2023. - · arXiv
  29. [33][33] M. Thielscher. A general game description language for incomplete information games. In Proceedings of the Twenty-Fourth AAAI Conference on Artificial Intelligence , AAAI'10, page 994-999. AAAI Press, 2010. -
  30. [34][34] M. Thielscher. The general game playing description language is universal. In T. Walsh, editor, IJCAI 2011, Proceedings of the 22nd International Joint Conference on Artificial Intelligence, Barcelona, Catalonia, Spain, July 16-22, 2011 , pages 1107-1112. IJCAI/AAAI, 2011. -
  31. [35][35] Y. Wang and Y. Zhao. Rupbench: Benchmarking reasoning under perturbations for robustness evaluation in large language models. arXiv preprint arXiv:2406.11020 , 2024. - · arXiv
  32. [36][36] J. Wielemaker, T. Schrijvers, M. Triska, and T. Lager. Swi-prolog. Theory and Practice of Logic Programming , 12(1-2):67-96, 2012. -
  33. [37][37] Y. Wu, A. Q. Jiang, W. Li, M. Rabe, C. Staats, M. Jamnik, and C. Szegedy. Autoformalization with large language models. Advances in Neural Information Processing Systems , 35:32353-32368, 2022. -
  34. [38][38] Z. Yang, A. Ishay, and J. Lee. Coupling large language models with logic programming for robust and general reasoning from text. arXiv preprint arXiv:2307.07696 , 2023. - · arXiv
  35. [39][39] W. X. Zhao, K. Zhou, J. Li, et al. A survey of large language models. arXiv preprint arXiv:2303.18223 , 2023. · arXiv

Cited by

Sorted by citations

Authors
Mensfelt, A., Stathis, K., & Trencsenyi, V.
Affiliations
Royal Holloway University of London (GB)
ORCID
Venue
Frontiers in Artificial Intelligence and Applications
Publisher
IOS Press
Published
2025-10-21
ISSN
  • 0922-6389 (ISSN-L)
  • 1879-8314
DOI
10.3233/faia251256
OpenAlex
W4415428017
Cited by
0
References
0
FWCI
0.00
Type
book chapter
Language
en
Open access
Licence
Locations
  • Frontiers in artificial intelligence and applications (publishedVersion, cc-by-nc)
  • arXiv (Cornell University) (submittedVersion)
  • arXiv (Cornell University)
Funders
Leverhulme Trust (LIP-2022-001)
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › Multi-Agent Systems and Negotiation
  • Physical Sciences › Computer Science › Artificial Intelligence › Logic, Reasoning, and Knowledge
  • Physical Sciences › Computer Science › Artificial Intelligence › Artificial Intelligence in Games
Keywords
multi-agent simulation, autoformalization, large language models, social interaction modeling, rule validation, agent strategies, concurrent games

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review