CoChat
Sign Up
PreprintCited by 231Language and cultural evolutionSpeech and dialogue systemsNatural Language Processing Techniques

Multi-Agent Cooperation and the Emergence of (Natural) Language

Open access
Lazaridou, Angeliki, Peysakhovich, Alexander, Baroni, Marco·2016·arXiv
DOI 10.48550/arxiv.1612.07182arXiv 1612.07182
Open this paper in CoChat — free accountOpen PDF

Abstract

The current mainstream approach to train natural language systems is to expose them to large amounts of text. This passive learning is problematic if we are interested in developing interactive machines, such as conversational agents. We propose a framework for language learning that relies on multi-agent communication. We study this learning in the context of referential games. In these games, a sender and a receiver see a pair of images. The sender is told one of them is the target and is allowed to send a message from a fixed, arbitrary vocabulary to the receiver. The receiver must rely on this message to identify the target. Thus, the agents develop their own language interactively out of the need to communicate. We show that two networks with simple configurations are able to learn to coordinate in the referential game. We further explore how to make changes to the game environment to cause the "word meanings" induced in the game to better reflect intuitive semantic properties of the images. In addition, we present a simple strategy for grounding the agents' code into natural language. Both of these are necessary steps towards developing machines that are able to communicate with humans productively.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

DataCite · 10.48550/arxiv.1612.07182

Cites

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Covert communication channels, including emergent steganographic protocols, can facilitate hidden data exfiltration or coordinated leakage across agents, further complicating detection and attribution.

    “Moreover, covert communication channels, including emergent steganographic protocols, can facilitate hidden data exfiltration or coordinated leakage across agents, further complicating detection and attribution (Lazaridou et al., 2016;Havrylov & Titov, 2017;Motwani et al., 2024d).”

    From Open Challenges in Multi-Agent Security: Towards Secure Systems of Interacting AI Agents (2025) · cites this paper as Lazaridou et al., 2016

    Section 4 (p. 4) reports that agents learned to communicate and coordinate through an emergent protocol in a referential game. The paper does not address covert or steganographic channels, data exfiltration, coordinated leakage, or difficulties with detection and attribution. (LLM, conf=0.86)

    Experimental — accuracy not yet measured

  • During several iterations of training, agents converge on a language-like system, i.e., a consistent mapping between targets and messages.

    “During several iterations of training, the agents converge on a language-like system, i.e., a consistent mapping between targets and messages [35,36].”

    From The role of pragmatic mechanisms in referential communication and categorization: An emergent communication model (2026) · cites this paper as [35]

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

No references could be resolved yet.

Research works printed but not linked: 37

  1. John Langshaw Austin. How to do things with words . Harvard University Press, Cambridge, MA, 1962.
  2. Andreas Blume, Douglas V DeJong, Yong-Gwan Kim, and Geoffrey B Sprinkle. Experimental evidence on the evolution of meaning of messages in sender-receiver games. The American Economic Review , 88(5):1323-1340, 1998.
  3. Ted Briscoe (ed.). Linguistic evolution through language acquisition . Cambridge University Press, Cambridge, UK, 2002.
  4. Angelo Cangelosi and Domenico Parisi (eds.). Simulating the evolution of language . Springer, New York, 2002.
  5. Alfonso Caramazza and Jennifer Shelton. Domain-specific knowledge systems in the brain the animate-inanimate distinction. Journal of Cognitive Neuroscience , 10(1):1-34, 1998.
  6. Herbert H Clark. Using language. 1996. Cambridge University Press: Cambridge) , 952:274-296, 1996.
  7. Vincent Crawford. A survey of experiments on communication via cheap talk. Journal of Economic theory , 78(2):286-298, 1998.
  8. Vincent P Crawford and Joel Sobel. Strategic information transmission. Econometrica: Journal of the Econometric Society , pp. 1431-1451, 1982.
  9. Jia Deng, Wei Dong, Richard Socher, Lia-Ji Li, and Li Fei-Fei. Imagenet: A large-scale hierarchical image database. In Proceedings of CVPR , pp. 248-255, Miami Beach, FL, 2009.
  10. Jakob N. Foerster, Yannis M. Assael, Nando de Freitas, and Shimon Whiteson. Learning to communicate to solve riddles with deep distributed recurrent q-networks. Technical Report arXiv:1602.02672, 2016. URL http://arxiv.org/pdf/1602.02672v1 . · arXiv
  11. Drew Fudenberg and Alexander Peysakhovich. Recency, records and recaps: learning and nonequilibrium behavior in a simple decision problem. In Proceedings of the fifteenth ACM conference on Economics and Computation , pp. 971-986. ACM, 2014.
  12. Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. Generative adversarial nets. In Advances in Neural Information Processing Systems , pp. 2672-2680, 2014.
  13. Emilio Jorge, Mikael K˚ ageb¨ ack, and Emil Gustavsson. Learning to play guess who? and inventing a grounded language as a consequence. https://arxiv.org/abs/1611.03218 , 2016. · arXiv
  14. Sahar Kazemzadeh, Vicente Ordonez, Mark Matten, and Tamara L Berg. Referitgame: Referring to objects in photographs of natural scenes. In EMNLP , pp. 787-798, 2014.
  15. David Lewis. Convention . Harvard University Press, Cambridge, MA, 1969.
  16. Ken McRae, George Cree, Mark Seidenberg, and Chris McNorgan. Semantic feature production norms for a large set of living and nonliving things. Behavior Research Methods , 37(4):547-559, 2005.
  17. Tomas Mikolov, Armand Joulin, and Marco Baroni. A roadmap towards machine intelligence. arXiv preprint arXiv:1511.08130 , 2015. · arXiv
  18. James Pustejovsky. The Generative Lexicon . MIT Press, Cambridge, MA, 1995.
  19. Alvin E Roth and Ido Erev. Learning in extensive-form games: Experimental data and simple dynamic models in the intermediate term. Games and economic behavior , 8(1):164-212, 1995.
  20. Ariel Rubinstein. The electronic mail game: Strategic behavior under 'almost common knowledge'. The American Economic Review , pp. 385-391, 1989.
  21. Karen Simonyan and Andrew Zisserman. Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556 , 2014. · arXiv
  22. Brian Skyrms. Signals: Evolution, learning, and information . Oxford University Press, 2010.
  23. Matthew Spike, Kevin Stadler, Simon Kirby, and Kenny Smith. Minimal requirements for the emergence of learned signaling. Cognitive Science , 2016. In press.
  24. Luc Steels and Martin Loetzsch. The grounded naming game. In Luc Steels (ed.), Experiments in Cultural Language Evolution , pp. 41-59. John Benjamins, Amsterdam, 2012.
  25. Peter Stone and Manuela Veloso. Towards collaborative and adversarial learning: A case study in robotic soccer. International Journal of Human-Computer Studies , 48(1):83-104, 1998.
  26. Sainbayar Sukhbaatar, Arthur Szlam, and Rob Fergus. Learning multiagent communication with backpropagation. arXiv preprint arXiv:1605.07736 , 2016. · arXiv
  27. Ilya Sutskever, Oriol Vinyals, and Quoc Le. Sequence to sequence learning with neural networks. In Proceedings of NIPS , pp. 3104-3112, Montreal, Canada, 2014.
  28. Richard Sutton and Andrew Barto. Reinforcement Learning: An Introduction . MIT Press, Cambridge, MA, 1998.
  29. Laurens Van der Maaten and Geoffrey Hinton. Visualizing data using t-SNE. Journal of Machine Learning Research , 9(2579-2605), 2008.
  30. Oriol Vinyals and Quoc Le. A neural conversational model. In Proceedings of the ICML Deep Learning Workshop , Lille, France, 2015. Published online: https://sites.google.com/ site/deeplearning2015/accepted-papers .
  31. Kyle Wagner, James A Reggia, Juan Uriagereka, and Gerald S Wilkinson. Progress in the simulation of emergent communication and language. Adaptive Behavior , 11(1):37-69, 2003.
  32. S. I. Wang, P. Liang, and C. Manning. Learning language games through interaction. In Association for Computational Linguistics (ACL) , 2016.
  33. Ronald J Williams. Simple statistical gradient-following algorithms for connectionist reinforcement learning. Machine learning , 8(3-4):229-256, 1992.
  34. Ludwig Wittgenstein. Philosophical Investigations . Blackwell, Oxford, UK, 1953. Translated by G.E.M. Anscombe.
  35. Michael Wooldridge. An introduction to multiagent systems . John Wiley & Sons, 2009.
  36. Kelvin Xu, Jimmy Ba, Ryan Kiros, Kyunghyun Cho, Aaron Courville, Ruslan Salakhudinov, Rich Zemel, and Yoshua Bengio. Show, attend and tell: Neural image caption generation with visual attention. In Proceedings of ICML , pp. 2048-2057, Lille, France, 2015.
  37. Matthew Zeiler and Rob Fergus. Visualizing and understanding convolutional networks. In Proceedings of ECCV (Part 1) , pp. 818-833, Zurich, Switzerland, 2014.

Cited by

Sorted by citations

Authors
Lazaridou, A., Peysakhovich, A., & Baroni, M.
Affiliations
  • University of Trento (IT)
  • Yale University (US)
ORCID
Venue
arXiv
Publisher
arXiv
Published
2016
ISSN
2331-8422 (ISSN-L)
DOI
10.48550/arxiv.1612.07182
arXiv
1612.07182
OpenAlex
W2564324149
Cited by
231
References
0
Citations by year
2026: 2 · 2025: 15 · 2024: 21 · 2023: 28 · 2022: 29 · 2021: 37 · 2020: 49 · 2019: 34 · 2018: 9 · 2017: 5 · 2016: 1
Type
posted content
Language
en
Open access
Licence
arxiv.org/licenses/nonexclusive-distrib/1.0/
Locations
  • arXiv (Cornell University) (submittedVersion)
  • arXiv (Cornell University)
Topics
  • Social Sciences › Cultural Studies › Language and cultural evolution
  • Physical Sciences › Computer Science › Artificial Intelligence › Speech and dialogue systems
  • Physical Sciences › Computer Science › Artificial Intelligence › Natural Language Processing Techniques
Keywords
agent cooperation, language emergence, referential games, interactive language learning, multi-agent communication, semantic grounding, chatbots, agent coordination, word meaning

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review