Human-level play in the game of Diplomacy by combining language models with strategic reasoning
The DOI matches its Crossref record; 73 of the 73 references it cites were found in a registry or an index.
Abstract
Despite much progress in training artificial intelligence (AI) systems to imitate human language, building agents that use language to communicate intentionally with humans in interactive environments remains a major challenge. We introduce Cicero, the first AI agent to achieve human-level performance in Diplomacy , a strategy game involving both cooperation and competition that emphasizes natural language negotiation and tactical coordination between seven players. Cicero integrates a language model with planning and reinforcement learning algorithms by inferring players' beliefs and intentions from its conversations and generating dialogue in pursuit of its plans. Across 40 games of an anonymous online Diplomacy league, Cicero achieved more than double the average score of the human players and ranked in the top 10% of participants who played more than one game.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Despite Meta’s efforts, CICERO turned out to be an expert liar.”
From AI deception: A survey of examples, risks, and potential solutions (2024) · cites this paper as [4]
In the citing paper
“CICERO engages in premeditated deception, breaks the deals to which it had agreed, and tells outright falsehoods.”
From AI deception: A survey of examples, risks, and potential solutions (2024) · cites this paper as [4]
In the citing paper
“We found numerous examples of deception that were not reported in the published paper.”
From AI deception: A survey of examples, risks, and potential solutions (2024) · cites this paper as [4]
In the citing paper
“It not only betrayed other players but also engaged in premeditated deception, planning in advance to build a fake alliance with a human player in order to trick that player into leaving themselves undefended for an attack.”
From AI deception: A survey of examples, risks, and potential solutions (2024) · cites this paper as [4]
In the citing paper
“The CICERO project picked Diplomacy in order to evaluate the abilities of AI systems to learn how to compete in games that involve human cooperation, where the AI cannot simply master the game through running simulations against itself.4”
From AI deception: A survey of examples, risks, and potential solutions (2024) · cites this paper as [4]
1 more claim about this paper
Create a free account to see every claim other papers make about this one.
Citations
Cites
Language Models are Few-Shot Learners
Brown, T. B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., et al. (2020) · arXiv · doi:10.48550/arxiv.2005.14165 · Cited by 23,372
Mastering the game of Go with deep neural networks and tree search
Silver, D., Huang, A., Maddison, C. J., Guez, A., Sifre, L., van den Driessche, G., Schrittwieser, J., Antonoglou, I., Panneershelvam, V., Lanctot, M., et al. (2016) · Nature · doi:10.1038/nature16961 · Cited by 15,612
Kaugeranna, R., Kaugeranna, E., & Aion, (. S. 4. 6. (2023) · DROPS (Schloss Dagstuhl – Leibniz Center for Informatics) · doi:10.4230/lipics.giscience.2023.43 · Cited by 13,690
Mastering the game of Go without human knowledge
Silver, D., Schrittwieser, J., Simonyan, K., Antonoglou, I., Huang, A., Guez, A., Hubert, T., Baker, L., Lai, M., Bolton, A., et al. (2017) · Nature · doi:10.1038/nature24270 · Cited by 9,084
Lewis, M., Liu, Y., Goyal, N., Ghazvininejad, M., Mohamed, A., Levy, O., Stoyanov, V., & Zettlemoyer, L. (2020) · Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/2020.acl-main.703 · Cited by 8,821
Self-citationLewis, M., Liu, Y., Goyal, N., Ghazvininejad, M., Mohamed, A., Levy, O., Stoyanov, V., & Zettlemoyer, L. (2019) · arXiv · doi:10.48550/arxiv.1910.13461 · Cited by 8,816 · cited 1×
“7 billion–parameter Transformer-based (23) encoder-decoder model pretrained on text from the internet by using a BART denoising objective (24).”
MethodSelf-citationNash, J. (1951) · The Annals of Mathematics · doi:10.2307/1969529 · Cited by 7,002
On the Dangers of Stochastic Parrots: Can Language Models Be Too Big? 🦜
Bender, E. M., Gebru, T., McMillan-Major, A., & Shmitchell, S. (2021) · Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency · doi:10.1145/3442188.3445922 · Cited by 6,988
Chain-Of-Thought Prompting Elicits Reasoning in Large Language Models
Wei, J., Wang, X., Schuurmans, D., Bosma, M., Ichter, B., Xia, F., Chi, E., Le, Q. V., & Zhou, D. (2022) · Advances in Neural Information Processing Systems 35 · doi:10.52202/068431-1800 · Cited by 4,505
A general reinforcement learning algorithm that masters chess, shogi, and Go through self-play
Silver, D., Hubert, T., Schrittwieser, J., Antonoglou, I., Lai, M., Guez, A., Lanctot, M., Sifre, L., Kumaran, D., Graepel, T., et al. (2018) · Science · doi:10.1126/science.aar6404 · Cited by 3,562
Research works printed but not linked: 16
- A. Bakhtin. Mastering the game of no-press Diplomacy via human-regularized reinforcement learning and planning. arXiv:2210.05492 [cs.GT] (2022). · arXiv
- A. Holtzman J. Buys L. Du M. Forbes Y. Choi The curious case of neural text degeneration 8th International Conference on Learning Representations ICLR 2020 Addis Ababa Ethiopia April 26-30 2020 (OpenReview.net 2020).
- A. Vaswani et al . Attention is all you need in Advances in Neural Information Processing Systems I. Guyon et al. eds. (Curran Associates Inc. 2017) vol. 30.
- A. P. Jacob et al . Modeling strong and human-like gameplay with KL-regularized search in International Conference on Machine Learning (PMLR 2022) pp. 9695–9728.
- Z. Ji. Survey of hallucination in natural language generation. arXiv:2202.03629 [cs.CL] (2022). · arXiv
- C. Berner. Dota 2 with large scale deep reinforcement learning. arXiv:1912.06680 (2019). · arXiv
- N. Carlini et al . Extracting training data from large language models in 30th USENIX Security Symposium (USENIX Security 21) (2021) pp. 2633–2650.
- R. Zellers et al . Defending against neural fake news in Advances in Neural Information Processing Systems 32: Annual Conference on Neural Information Processing Systems 2019 NeurIPS 2019 December 8–14 2019 Vancouver BC Canada H. M. Wallach et al. eds. (2019) pp. 9051–9062.
- J. Gray A. Lerer A. Bakhtin N. Brown Human-level performance in no-press Diplomacy via equilibrium search in International Conference on Learning Representations (2020).
- D. Yarats M. Lewis Hierarchical text generation and planning for strategic dialogue in International Conference on Machine Learning (PMLR 2018) pp. 5591–5599.
- S. Afantenos et al . Developing a corpus of strategic conversation in The Settlers of Catan Workshop on Games and NLP (GAMNLP-12) (2012).
- H. Cuayáhuitl S. Keizer O. Lemon Strategic dialogue management via deep reinforcement learning in NeurIPS Workshop on Deep Reinforcement Learning (2015).
- A. Radford, Language models are unsupervised multitask learners. OpenAI blog 1, 9 (2019).
- D. P. Kingma J. Ba Adam: A method for stochastic optimization in 3rd International Conference on Learning Representations ICLR 2015 San Diego CA USA May 7-9 2015 Conference Track Proceedings Y. Bengio Y. LeCun eds. (2015).
- E. M. Smith D. Gonzalez-Rico E. Dinan Y. Boureau Controlling style in generated dialogue. arXiv:2009.10855 [cs.CL] (2020). · arXiv
- J. Xu Recipes for safety in open-domain chatbots. arXiv:2010.07079 [cs.CL] (2020). · arXiv
Cited by
Items 401–404 of 404
Torres-Huitzil, C., García‐Bañuelos, L., de Jonge, D., Ahuactzin, J.-M., & Oliart, A. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-44179-9_23
Four Bridges: Measuring Deception Propensity in a Multi-Agent GamePreprint
Casey, R., Simeone, M. P., Tosato, A., Ernst, F., Moujahid, K. E., Tamplin, J., & Tosato, T. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2611.03600
Agents of Change: Self-Evolving LLM Agents for Strategic Planning
Belle, N., Barnes, D., Amayuelas, A., Bercovich, I., Wang, X. E., & Wang, W. (2025) · arXiv · doi:10.48550/arxiv.2506.04651
Algorithmic Collusion by Large Language Models
Fish, S., Gonczarowski, Y. A., & Shorrer, R. I. (2024) · arXiv · doi:10.48550/arxiv.2404.00806 · Cited by 10
Metadata
- Authors
- Meta Fundamental AI Research Diplomacy Team (FAIR)†, Bakhtin, A., Brown, N., Dinan, E., Farina, G., Flaherty, C., Fried, D., Goff, A., Gray, J., Hu, H., Jacob, A. P., Komeili, M., Konath, K., Kwon, M., Lerer, A., Lewis, M., Miller, A. H., Mitts, S., Renduchintala, A., Roller, S., Rowe, D., Shi, W., Spisak, J., Wei, A., Wu, D., Zhang, H., & Zijlstra, M.
- Affiliations
- Meta (United States) (US)
- Harvard University (US)
- Corresponding author
- Bakhtin, Anton
- Brown, Noam
- Dinan, Emily
- Flaherty, Colin
- Gray, Jonathan
- Hu, Hengyuan
- Jacob, Athul Paul
- Lerer, Adam
- Lewis, Mike
- Miller, Alexander H.
- Renduchintala, Adithya
- Shi, Weiyan
- Wu, David
- Zhang, Hugh
- ORCID
- Venue
- Science
- Published
- 2022-12-09
- Volume
- 378
- Issue
- 6624
- Pages
- 1067-1074
- ISSN
- 0036-8075 (ISSN-L)
- 1095-9203
- OpenAlex
- W4309663019
- Cited by
- 241
- References
- 65
- Citations by year
- 2026: 67 · 2025: 69 · 2024: 59 · 2023: 43 · 2022: 2
- FWCI
- 14.24
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Science (publishedVersion)
- PubMed (publishedVersion)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Artificial Intelligence in Games
- Physical Sciences › Computer Science › Artificial Intelligence › Speech and dialogue systems
- Physical Sciences › Computer Science › Artificial Intelligence › Multi-Agent Systems and Negotiation
- Keywords
- diplomacy, AI agents, human-AI interaction, strategic reasoning, language models, belief inference, dialogue generation, planning, reinforcement learning, Cicero
- MeSH terms
- Diplomacy, Artificial Intelligence, Humans, Language, Software, Video Games
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.