CoChat

Human-level play in the game of Diplomacy by combining language models with strategic reasoning

The DOI matches its Crossref record; 73 of the 73 references it cites were found in a registry or an index.

Abstract

Despite much progress in training artificial intelligence (AI) systems to imitate human language, building agents that use language to communicate intentionally with humans in interactive environments remains a major challenge. We introduce Cicero, the first AI agent to achieve human-level performance in Diplomacy , a strategy game involving both cooperation and competition that emphasizes natural language negotiation and tactical coordination between seven players. Cicero integrates a language model with planning and reinforcement learning algorithms by inferring players' beliefs and intentions from its conversations and generating dialogue in pursuit of its plans. Across 40 games of an anonymous online Diplomacy league, Cicero achieved more than double the average score of the human players and ranked in the top 10% of participants who played more than one game.

Overview

Integrity signals

Registry

Crossref · 10.1126/science.ade9097

Cites

73 of 73 references verified0 citation errors · 0 not checkable

Statistics

Recomputing p-values needs the paper’s full text.

Verification

Claims the paper makes

This paper’s full text isn’t openly available, so its claims and self-check weren’t run. The registry and citation checks still apply.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

1 more claim about this paper

Create a free account to see every claim other papers make about this one.

See all claims — free account

Citations

Cites

Show all 73 references

Research works printed but not linked: 16

  1. A. Bakhtin. Mastering the game of no-press Diplomacy via human-regularized reinforcement learning and planning. arXiv:2210.05492 [cs.GT] (2022). · arXiv
  2. A. Holtzman J. Buys L. Du M. Forbes Y. Choi The curious case of neural text degeneration 8th International Conference on Learning Representations ICLR 2020 Addis Ababa Ethiopia April 26-30 2020 (OpenReview.net 2020).
  3. A. Vaswani et al . Attention is all you need in Advances in Neural Information Processing Systems I. Guyon et al. eds. (Curran Associates Inc. 2017) vol. 30.
  4. A. P. Jacob et al . Modeling strong and human-like gameplay with KL-regularized search in International Conference on Machine Learning (PMLR 2022) pp. 9695–9728.
  5. Z. Ji. Survey of hallucination in natural language generation. arXiv:2202.03629 [cs.CL] (2022). · arXiv
  6. C. Berner. Dota 2 with large scale deep reinforcement learning. arXiv:1912.06680 (2019). · arXiv
  7. N. Carlini et al . Extracting training data from large language models in 30th USENIX Security Symposium (USENIX Security 21) (2021) pp. 2633–2650.
  8. R. Zellers et al . Defending against neural fake news in Advances in Neural Information Processing Systems 32: Annual Conference on Neural Information Processing Systems 2019 NeurIPS 2019 December 8–14 2019 Vancouver BC Canada H. M. Wallach et al. eds. (2019) pp. 9051–9062.
  9. J. Gray A. Lerer A. Bakhtin N. Brown Human-level performance in no-press Diplomacy via equilibrium search in International Conference on Learning Representations (2020).
  10. D. Yarats M. Lewis Hierarchical text generation and planning for strategic dialogue in International Conference on Machine Learning (PMLR 2018) pp. 5591–5599.
  11. S. Afantenos et al . Developing a corpus of strategic conversation in The Settlers of Catan Workshop on Games and NLP (GAMNLP-12) (2012).
  12. H. Cuayáhuitl S. Keizer O. Lemon Strategic dialogue management via deep reinforcement learning in NeurIPS Workshop on Deep Reinforcement Learning (2015).
  13. A. Radford, Language models are unsupervised multitask learners. OpenAI blog 1, 9 (2019).
  14. D. P. Kingma J. Ba Adam: A method for stochastic optimization in 3rd International Conference on Learning Representations ICLR 2015 San Diego CA USA May 7-9 2015 Conference Track Proceedings Y. Bengio Y. LeCun eds. (2015).
  15. E. M. Smith D. Gonzalez-Rico E. Dinan Y. Boureau Controlling style in generated dialogue. arXiv:2009.10855 [cs.CL] (2020). · arXiv
  16. J. Xu Recipes for safety in open-domain chatbots. arXiv:2010.07079 [cs.CL] (2020). · arXiv

Cited by

Items 201–300 of 404

Metadata

Authors
Meta Fundamental AI Research Diplomacy Team (FAIR)†, Bakhtin, A., Brown, N., Dinan, E., Farina, G., Flaherty, C., Fried, D., Goff, A., Gray, J., Hu, H., Jacob, A. P., Komeili, M., Konath, K., Kwon, M., Lerer, A., Lewis, M., Miller, A. H., Mitts, S., Renduchintala, A., Roller, S., Rowe, D., Shi, W., Spisak, J., Wei, A., Wu, D., Zhang, H., & Zijlstra, M.
Affiliations
  • Meta (United States) (US)
  • Harvard University (US)
Corresponding author
  • Bakhtin, Anton
  • Brown, Noam
  • Dinan, Emily
  • Flaherty, Colin
  • Gray, Jonathan
  • Hu, Hengyuan
  • Jacob, Athul Paul
  • Lerer, Adam
  • Lewis, Mike
  • Miller, Alexander H.
  • Renduchintala, Adithya
  • Shi, Weiyan
  • Wu, David
  • Zhang, Hugh
Venue
Science
Published
2022-12-09
Volume
378
Issue
6624
Pages
1067-1074
ISSN
  • 0036-8075 (ISSN-L)
  • 1095-9203
OpenAlex
W4309663019
Cited by
241
References
65
Citations by year
2026: 67 · 2025: 69 · 2024: 59 · 2023: 43 · 2022: 2
FWCI
14.24
Type
journal article
Language
en
Open access
closed
Locations
  • Science (publishedVersion)
  • PubMed (publishedVersion)
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › Artificial Intelligence in Games
  • Physical Sciences › Computer Science › Artificial Intelligence › Speech and dialogue systems
  • Physical Sciences › Computer Science › Artificial Intelligence › Multi-Agent Systems and Negotiation
Keywords
diplomacy, AI agents, human-AI interaction, strategic reasoning, language models, belief inference, dialogue generation, planning, reinforcement learning, Cicero
MeSH terms
Diplomacy, Artificial Intelligence, Humans, Language, Software, Video Games

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.