The DOI matches its DataCite record; 77 of the 77 references it cites were found in a registry or an index; none of the 9 sources its claims cite is checked for support yet.
Game-theoretic LLM: Agent Workflow for Negotiation Games
Open accessAbstract
This paper investigates the rationality of large language models (LLMs) in strategic decision-making contexts, specifically within the framework of game theory. We evaluate several state-of-the-art LLMs across a spectrum of complete-information and incomplete-information games. Our findings reveal that LLMs frequently deviate from rational strategies, particularly as the complexity of the game increases with larger payoff matrices or deeper sequential trees. To address these limitations, we design multiple game-theoretic workflows that guide the reasoning and decision-making processes of LLMs. These workflows aim to enhance the models' ability to compute Nash Equilibria and make rational choices, even under conditions of uncertainty and incomplete information. Experimental results demonstrate that the adoption of these workflows significantly improves the rationality and robustness of LLMs in game-theoretic tasks. Specifically, with the workflow, LLMs exhibit marked improvements in identifying optimal strategies, achieving near-optimal allocations in negotiation scenarios, and reducing susceptibility to exploitation during negotiations. Furthermore, we explore the meta-strategic considerations of whether it is rational for agents to adopt such workflows, recognizing that the decision to use or forgo the workflow constitutes a game-theoretic issue in itself. Our research contributes to a deeper understanding of LLMs' decision-making capabilities in strategic contexts and provides insights into enhancing their rationality through structured workflows. The findings have implications for the development of more robust and strategically sound AI agents capable of navigating complex interactive environments. Code and data supporting this study are available at \url{https://github.com/Wenyueh/game_theory}.
Overview
CoChat Summary
TL;DR
This paper evaluates whether LLMs play game-theoretic games rationally and proposes game-theory-inspired workflows to improve their strategic decision-making. It finds that LLMs often deviate from Nash equilibrium, especially in complex games, but workflows substantially improve rationality and negotiation outcomes, while also creating a meta-strategic dilemma about whether to adopt the workflow.
Key findings
- Without workflows, LLMs frequently fail to reach Nash equilibria in complete-information games; average Nash-equilibrium rates were 0.45 (Sonnet), 0.32 (GPT-4o), 0.59 (Opus), and 0.78 (o1), with similar Pareto-optimal Nash rates. With workflows and no negotiation, averages rose to 0.76, 0.69, 0.71, and 0.81; with workflows plus 4 negotiation rounds, averages were 0.73, 0.78, 0.83, and 0.88.
- In the incomplete-information “Deal or No Deal” setting, raw LLM-vs-LLM negotiation produced a 95.45% agreement rate for Claude-3.5 Sonnet, average total reward 11.11 versus best possible 12.48 and human 6.64, with 70.45% Pareto-optimal and 70.45% envy-free outcomes. With both agents using the workflow, agreement was 100% across models; Opus achieved total reward 12.31, GPT-4o 12.18, and Sonnet 12.11, with near-perfect envy freeness (GPT-4o 100%, Sonnet 97.72%).
- When only one agent used the workflow, the non-workflow agent often obtained higher individual reward, indicating the workflow can be exploited. The decision to adopt the workflow itself becomes a game: for Claude-3.5 Sonnet, “not use” was dominant and the Nash equilibrium was both not using it (payoffs 5.55, 5.57) versus both using it (5.82, 6.16); for GPT-4o and Opus, using the workflow was dominant and Nash/Pareto optimal.
- LLM rationality was fragile: payoff perturbations that preserved Nash equilibria changed action distributions; assigned personalities affected rationality; negotiation sometimes undermined rational choices, including in Prisoner’s Dilemma; prompt engineering effects diminished with more negotiation rounds; and in a Golden Balls comparison, LLM cooperation was largely insensitive to jackpot size and generally lower than the human 53% cooperation rate.
Methods
- Evaluated Claude-3.5 Sonnet, Claude-3 Opus, GPT-4o, and o1 on 10 classic complete-information games (5 simultaneous, 5 sequential), with 10 trials per scenario and temperature 1.0 for Sonnet/Opus/GPT-4o and default for o1; measured Nash-equilibrium and Pareto-optimal Nash-equilibrium achievement.
- Designed workflows using dominant-strategy/best-response search for simultaneous games, backward induction for sequential games, and a Bayesian belief-updating negotiation algorithm for incomplete-information resource allocation with envy-freeness and Pareto-optimality criteria.
- For incomplete-information negotiation, used the “Deal or No Deal” dataset (over 5,800 human-human dialogues; 1,052 test dialogues) and selected the top-50 most difficult datapoints with an envy-free allocation, of which 44 were used; metrics included negotiation rounds, agreement, agent score, Pareto optimality, envy freeness, and total reward.
Limitations
- The paper states that o1 was excluded from workflow experiments due to prohibitively high computational cost, and its inaccessible intermediate chain-of-thought made performance drops hard to diagnose. It also states that the workflow does not eliminate all computational limitations, as the Escalation Game still suffered numerical computation errors.
- The paper states that the Bayesian update assumes opponent rationality and can be unrobust to deception; that negotiation can divert agents from workflow-derived Nash equilibria; that engineered prompts did not genuinely enhance rationality; and that the human-vs-LLM irrationality comparison is preliminary and not exhaustive.
- Evident from the text: the incomplete-information evaluation uses a selected difficult subset rather than the full test set, focuses on two-player resource allocation, and reports sensitivity to temperature, so generality across broader populations, games, and model settings is not established.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Figures
Verification
Claims the paper makes
0 Valid · 10 Not valid · 5 Not checked · 164 Own results and interpretations
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Among the LLM-based agents, Claude-3.5Sonnet demonstrates the best super-human performance.”
§ Game-theoretic LLM: Agent Workflow for Negotiation Games > 5.5.1 Both Agents without Workflow
boosted- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper states — it cites no source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“In coordination games such as the Stag Hunt, Battle of the Sexes, and Wait-Go Game, negotiation is a fundamental component of decision-making, making the improvement in performance through negotiation an intuitive result.”
§ Game-theoretic LLM: Agent Workflow for Negotiation Games > Games
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper states — it cites no source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“However, it is important to observe that the performance of model o1 declines considerably in this context.”
§ Game-theoretic LLM: Agent Workflow for Negotiation Games > 4.5 Experiments for Classic Game Theory with Workflow
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“As shown in Table 10, for these 44 datapoints, it is always possible to find pareto optimal and envy free allocations that maximize the total reward for both players.”
§ Game-theoretic LLM: Agent Workflow for Negotiation Games > 5.5.1 Both Agents without Workflow
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper states — it cites no source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“However, certain challenges persist, such as in the Escalation Game, where numerical computation errors hinder deriving the Nash Equilibrium following the workflow.”
§ Game-theoretic LLM: Agent Workflow for Negotiation Games > Games
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
107 more claims
Create a free account to see all 179 claims and every source verdict.
- It may state two separate claims as one. 51
- Probably not a substantial claim. 19
- The quote only partly states this claim. 2
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
Each contradiction between two passages listed here was confirmed by an independent check.
Rated major: it would touch a result — a value a result rests on, reported and recomputed p-values on opposite sides of the significance level, two passages that conflict with no sign which is right, or a cited work retracted before this paper was published.
In the paper
“If both agents were to use the workflow, they would receive higher individual payoffs -5.82 forAlice and 6.16 for Bob -and a higher combined total reward of 12.48”
§ Game-theoretic LLM: Agent Workflow for Negotiation Games > Models Sonnet GPT-4o Opus
Reported 12.48 · recomputed 11.98
Rated major: it would touch a result — a value a result rests on, reported and recomputed p-values on opposite sides of the significance level, two passages that conflict with no sign which is right, or a cited work retracted before this paper was published.
In the paper
§ Game-theoretic LLM: Agent Workflow for Negotiation Games > 5.5.1 Both Agents without Workflow
“Also notice that all LLMs can performance better than human baseline.”
§ Game-theoretic LLM: Agent Workflow for Negotiation Games > Development of Meta-Strategy:
“| Human | 2.86 | 0.6817 | 3.32 | 3.39 | 0.4317 0.4545 | 6.64 | | Sonnet | 7.07 | 0.9545 | 5.55 | 5.57 | 0.7045 0.7045 | 11.11 | | o1 | 3.86 | 0.7500 | 4.39 | 4.43 | 0.4545 0.4772 | 8.82 | | GPT-4o | 18.45 | 0.6363 | 2.80 | 4.38 | 0.4091 0.3864 | 7.14 | | Opus | 4.37 | 0.4772 | 2.68 | 3.02 | 0.3636 0.2727 | 5.70 |”
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
“| average | 0.76 | 0.69 | 0.71 0.81 | 0.76 | 0.69 | 0.71 | 0.81 | |”
§ Game-theoretic LLM: Agent Workflow for Negotiation Games > Development of Meta-Strategy:
Reported 0.69 · recomputed 0.75
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ Game-theoretic LLM: Agent Workflow for Negotiation Games > Performance of Valuation Estimation
“Notice that for all the three models, their estimations of the opponent's valuation are exactly the same”
§ Game-theoretic LLM: Agent Workflow for Negotiation Games > Development of Meta-Strategy:
“| Sonnet | 0.9545 | 0.3766 | 0.7033 | | GPT-4o | 0.9545 | 0.3515 | 0.6980 | | Opus | 0.7954 | 0.2737 | 0.6947 |”
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ Game-theoretic LLM: Agent Workflow for Negotiation Games > Models Sonnet GPT-4o Opus
“4.86,4.572.80,4.38Table 19”
§ Game-theoretic LLM: Agent Workflow for Negotiation Games > Development of Meta-Strategy:
“| Opus | 4.37 | 0.4772 | 2.68 | 3.02 | 0.3636 0.2727 | 5.70 |”
1 more finding
Create a free account to see every finding of this check.
Citations
Cites
Chain-Of-Thought Prompting Elicits Reasoning in Large Language Models
Wei, J., Wang, X., Schuurmans, D., Bosma, M., Ichter, B., Xia, F., Chi, E., Le, Q. V., & Zhou, D. (2022) · Advances in Neural Information Processing Systems 35 · doi:10.52202/068431-1800 · Cited by 4,505
Self-citationTree of Thoughts: Deliberate Problem Solving with Large Language Models
Yao, S., Yu, D., Zhao, J., Shafran, I., Griffiths, T. L., Cao, Y., & Narasimhan, K. (2023) · arXiv · doi:10.48550/arxiv.2305.10601 · Cited by 1,158
The Rise and Potential of Large Language Model Based Agents: A Survey
Xi, Z., Chen, W., Guo, X., He, W., Ding, Y., Hong, B., Zhang, M., Wang, J., Jin, S., Zhou, E., et al. (2023) · arXiv · doi:10.48550/arxiv.2309.07864 · Cited by 972
Self-citationThe rise and potential of large language model based agents: a survey
Xi, Z., Chen, W., Guo, X., He, W., Ding, Y., Hong, B., Zhang, M., Wang, J., Jin, S., Zhou, E., et al. (2025) · Science China Information Sciences · doi:10.1007/s11432-024-4222-0 · Cited by 970
Self-citationDemystifying LLM-Based Software Engineering Agents
Xia, C. S., Deng, Y., Dunn, S., & Zhang, L. (2025) · Proceedings of the ACM on Software Engineering · doi:10.1145/3715754 · Cited by 849
Fairness Versus Reason in the Ultimatum Game
Nowak, M. A., Page, K. M., & Sigmund, K. (2000) · Science · doi:10.1126/science.289.5485.1773 · Cited by 817
Theory of Games and Economic Behavior (60th Anniversary Commemorative Edition)
von Neumann, J., & Morgenstern, O. (2007) · Princeton University Press eBooks · doi:10.1515/9781400829460 · Cited by 700
Dawes, R. M., & Thaler, R. H. (1988) · Journal of Economic Perspectives · doi:10.1257/jep.2.3.187 · Cited by 621
Deal or No Deal? End-to-End Learning of Negotiation Dialogues
Lewis, M., Yarats, D., Dauphin, Y., Parikh, D., & Batra, D. (2017) · Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d17-1259 · Cited by 347
Playing repeated games with large language models
Akata, E., Schulz, L., Coda-Forno, J., Oh, S. J., Bethge, M., & Schulz, E. (2025) · Nature Human Behaviour · doi:10.1038/s41562-025-02172-y · Cited by 332
Research works printed but not linked: 5
- Nian Li, Chen Gao, Mingyu Li, Yong Li, and Qingmin Liao. Econagent: large language model-empowered agents for simulating macroeconomic activities. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 15523-15536, 2024.
- Drew Fudenberg and Jean Tirole. Game theory. MIT press, 1991.
- Yingqiang Ge, Wenyue Hua, Kai Mei, Jianchao Ji, Juntao Tan, Shuyuan Xu, Zelong Li, and Yongfeng Zhang. OpenAGI: When LLM meets domain experts. In Thirty-seventh Conference on Neural Information Processing Systems, 2023.
- Xinyi Li, Sai Wang, Siqi Zeng, Yu Wu, and Yi Yang. A survey on llm-based multi-agent systems: workflow, infrastructure, and challenges. Vicinagearth, 1(1):9, 2024.
- David Sally. Conversation and cooperation in social dilemmas: A meta-analysis of experiments from 1958 to 1992. Rationality and society, 7(1):58-92, 1995.
Cited by
Foundation models and intelligent decision-making: Progress, challenges, and perspectives
Huang, J., Xu, Y., Wang, Q., Wang, Q., Liang, X., Wang, F., Zhang, Z., Wei, W., Zhang, B., Huang, L., et al. (2025) · The Innovation · doi:10.1016/j.xinn.2025.100948 · Cited by 97
Emergent social conventions and collective bias in LLM populations
Ashery, A. F., Aiello, L. M., & Baronchelli, A. (2025) · Science Advances · doi:10.1126/sciadv.adu9368 · Cited by 81
TextArenaPreprint
Guertler, L., Cheng, B., Yu, S., Liu, B., Choshen, L., & Tan, C. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2504.11442 · Cited by 78
Virtual Agent EconomiesPreprint
Tomašev, N., Franklin, M., Leibo, J. Z., Jacobs, J., Cunningham, W. A., Gabriel, I., & Osindero, S. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2509.10147 · Cited by 51
TwinMarket: A Scalable Behavioral and Social Simulation for Financial Markets
Yang, Y., Zhang, Y., Wu, M., Zhang, K., Zhang, Y., Yu, H., Hu, Y., & Wang, B. (2025) · neural information processing systems · doi:10.52202/085713-2132 · Cited by 46
Self-citation
Metadata
- Authors
- Hua, W., Liu, O., Li, L., Amayuelas, A., Chen, J., Jiang, L., Jin, M., Fan, L., Sun, F., Wang, W., Wang, X., & Zhang, Y.
- Affiliations
- Rutgers, The State University of New Jersey (US)
- University of Southern California (US)
- University of South Florida (US)
- University of California, Santa Barbara (US)
- Harvard University (US)
- Institute of Computing Technology (CN)
- Venue
- arXiv
- Publisher
- arXiv
- Published
- 2024
- ISSN
- 2331-8422 (ISSN-L)
- arXiv
- 2411.05990
- OpenAlex
- W4404400455
- Cited by
- 101
- References
- 45
- Citations by year
- 2026: 50 · 2025: 48 · 2024: 3
- Type
- posted content
- Language
- en
- Open access
- green
- arxiv.org
- Licence
- Locations
- arXiv (Cornell University) (submittedVersion, cc-by)
- arXiv (Cornell University) (cc-by)
- Topics
- Social Sciences › Decision Sciences › Management Science and Operations Research › Game Theory and Applications
- Physical Sciences › Computer Science › Artificial Intelligence › Multi-Agent Systems and Negotiation
- Physical Sciences › Computer Science › Artificial Intelligence › Artificial Intelligence in Games
- Keywords
- large language models, game-theoretic reasoning, negotiation games, Nash equilibrium, strategic decision-making, incomplete information games, complete information games, meta-strategies
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
A language model drew the claims from the paper on 2026-11-14, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-11-14 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Game-theoretic LLM: Agent Workflow for Negotiation Games. https://app.cochat.ai/papers/10.48550/arxiv.2411.05990
- The paper
- Hua, W., Liu, O., Li, L., Amayuelas, A., Chen, J., Jiang, L., Jin, M., Fan, L., Sun, F., Wang, W., Wang, X., & Zhang, Y. (2024). Game-theoretic LLM: Agent Workflow for Negotiation Games. arXiv. https://doi.org/10.48550/arxiv.2411.05990
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.
