Mutual influence between language and perception in multi-agent communication games
Open accessThe DOI matches its Crossref record; 77 of the 77 references it cites were found in a registry or an index.
Abstract
Language interfaces with many other cognitive domains. This paper explores how interactions at these interfaces can be studied with deep learning methods, focusing on the relation between language emergence and visual perception. To model the emergence of language, a sender and a receiver agent are trained on a reference game. The agents are implemented as deep neural networks, with dedicated vision and language modules. Motivated by the mutual influence between language and perception in cognition, we apply systematic manipulations to the agents’ (i) visual representations, to analyze the effects on emergent communication, and (ii) communication protocols, to analyze the effects on visual representations. Our analyses show that perceptual biases shape semantic categorization and communicative content. Conversely, if the communication protocol partitions object space along certain attributes, agents learn to represent visual information about these attributes more accurately, and the representations of communication partners align. Finally, an evolutionary analysis suggests that visual representations may be shaped in part to facilitate the communication of environmentally relevant distinctions. Aside from accounting for co-adaptation effects between language and perception, our results point out ways to modulate and improve visual representation learning and emergent communication in artificial agents.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Figures










Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source says what the claim attributes to it.
In the citing paper
“Evidence for the efficient communication hypothesis has been found in the context of kinship [18] and color [22–25] systems.”
From The role of pragmatic mechanisms in referential communication and categorization: An emergent communication model (2026) · cites this paper as [24]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
(2008) · Journal of Machine Learning Research · Cited by 33,051
Learning Phrase Representations using RNN Encoder–Decoder for Statistical Machine Translation
Cho, K., van Merrienboer, B., Gulcehre, C., Bahdanau, D., Bougares, F., Schwenk, H., & Bengio, Y. (2014) · Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.3115/v1/d14-1179 · Cited by 25,545 · cited 1×
“The language module, l ( · ) , consists Figure 1: Schematic visualization of sender and receiver architecture and their interaction in one round of the reference game.”
Language Models are Few-Shot Learners
Brown, T. B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., et al. (2020) · arXiv · doi:10.48550/arxiv.2005.14165 · Cited by 23,372
Intriguing properties of neural networks
Szegedy, C., Zaremba, W., Sutskever, I., Bruna, J., Erhan, D., Goodfellow, I., & Fergus, R. (2013) · arXiv · doi:10.48550/arxiv.1312.6199 · Cited by 8,143 · cited 1×
“For example, they are susceptible to adversarial attacks [62] and lack useful learning mechanisms observed in children [63].”
Asynchronous Methods for Deep Reinforcement Learning
Mnih, V., Badia, A. P., Mirza, M., Graves, A., Lillicrap, T. P., Harley, T., Silver, D., & Kavukcuoglu, K. (2016) · arXiv · doi:10.48550/arxiv.1602.01783 · Cited by 7,132 · cited 1×
“We add an entropy regularization term [48] of 0.”
Basic objects in natural categories
Rosch, E., Mervis, C. B., Gray, W. D., Johnson, D. M., & Boyes-Braem, P. (1976) · Cognitive Psychology · doi:10.1016/0010-0285(76)90013-x · Cited by 5,643 · cited 1×
“For example, objects that are grouped under the same psychological concept often have similar shapes [61].”
Evolutionary Games and Population Dynamics
Hofbauer, J., & Sigmund, K. (1998) · doi:10.1017/cbo9781139173179 · Cited by 5,502 · cited 1×
“While the concept of an ESS has first been introduced in the context of biological evolution, it is useful also for analyzing the stable rest points of non-biological evolutionary optimization processes The latter is made possible by the fact that ESSs are the (locally) asymptotically stable rest points of the replica…”
Strategic information transmission
Crawford, V. P., & Sobel, J. (1982) · Econometrica · doi:10.2307/1913390 · Cited by 4,293 · cited 1×
“Communication games have been used to study the emergence and evolution of language theoretically [29], experimentally [30, 31], and computationally [32].”
Evolutionary stable strategies and game dynamics
Taylor, P. D., & Jonker, L. B. (1978) · Mathematical Biosciences · doi:10.1016/0025-5564(78)90077-9 · Cited by 3,450 · cited 2×
“…introduced in the context of biological evolution, it is useful also for analyzing the stable rest points of non-biological evolutionary optimization processes The latter is made possible by the fact that ESSs are the (locally) asymptotically stable rest points of the replicator dynamic [52, 53].”
Simple Statistical Gradient-Following Algorithms for Connectionist Reinforcement Learning
Williams, R. J. (1992) · Machine Learning · doi:10.1007/bf00992696 · Cited by 3,416 · cited 1×
“The agents minimize the negative expected reward, − E [ r ] , and their trainable weights are updated using REINFORCE [47], which is a basic policy gradient algorithm.”
Representational similarity analysis-connecting the branches of systems neuroscience
Kriegeskorte, N. (2008) · Frontiers in Systems Neuroscience · doi:10.3389/neuro.06.004.2008 · Cited by 3,355
The theory of games and the evolution of animal conflicts
Maynard Smith, J. (1974) · Journal of Theoretical Biology · doi:10.1016/0022-5193(74)90110-6 · Cited by 2,410 · cited 2×
“For this purpose, we use the static solution concept of evolutionary stability from evolutionary game theory [28].”
Conceptual spaces: the geometry of thought
(2000) · Choice Reviews Online · doi:10.5860/choice.38-1526 · Cited by 1,643
Hybrid computing using a neural network with dynamic external memory
Graves, A., Wayne, G., Reynolds, M., Harley, T., Danihelka, I., Grabska-Barwińska, A., Colmenarejo, S. G., Grefenstette, E., Ramalho, T., Agapiou, J., et al. (2016) · Nature · doi:10.1038/nature20101 · Cited by 1,412 · cited 1×
“On a basic level, also the role of attention or memory could be studied with our setup, for example by using neural network agents with attention mechanisms [90] or external memory [91].”
Population Games and Evolutionary DynamicsPreprint
Sandholm, W. H. (2005) · RePEc: Research Papers in Economics · Cited by 1,349
Deep supervised, but not unsupervised, models may explain IT cortical representation
Khaligh-Razavi, S.-M., & Kriegeskorte, N. (2014) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1003915 · Cited by 1,204 · cited 1×
“In terms of visual perception and representation learning, DNNs have been used to model brain activations in the visual cortex [12, 13, 14] and judgments of image similarity [15, 16].”
Kriegeskorte, N. (2015) · Annual Review of Vision Science · doi:10.1146/annurev-vision-082114-035447 · Cited by 1,093
Language as shaped by the brain
Christiansen, M. H., & Chater, N. (2008) · Behavioral and Brain Sciences · doi:10.1017/s0140525x08004998 · Cited by 1,063 · cited 1×
“However, it has been argued that—evolutionarily young and variable—language is rather shaped by the—evolutionarily old and stable—brain than vice versa [75].”
A deep learning framework for neuroscience
Richards, B. A., Lillicrap, T. P., Beaudoin, P., Bengio, Y., Bogacz, R., Christensen, A., Clopath, C., Costa, R. P., de Berker, A., Ganguli, S., et al. (2019) · Nature Neuroscience · doi:10.1038/s41593-019-0520-2 · Cited by 1,027 · cited 1×
“Still, there are many ideas on how correspondence between artificial and biological neural networks can be further improved by changing architectures, learning algorithms, input statistics, or training objectives [82, 83].”
Russian blues reveal effects of language on color discrimination
Winawer, J., Witthoft, N., Frank, M. C., Wu, L., Wade, A. R., & Boroditsky, L. (2007) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.0701644104 · Cited by 1,021 · cited 1×
“In particular, language affects perceptual processing by imposing categorical structure [20, 21].”
Mirativity: The grammatical marking of unexpected information
DELANCEY, S. (1997) · Linguistic Typology · doi:10.1515/lity.1997.1.1.33 · Cited by 745 · cited 1×
“These so-called mirative markers convey surprise [89], thereby telling the audience what we expected, but also what we pay attention to.”
Learning Through Reinforcement and Replicator Dynamics
Börgers, T., & Sarin, R. (1997) · Journal of Economic Theory · doi:10.1006/jeth.1997.2319 · Cited by 737 · cited 1×
“…rather encompassing high-level formalization of a wide variety of agent-internal optimization processes, be they cross-generational as in cultural evolution or (asexual) reproduction [54], or within-generational as in imitation-based dynamics [55, 54] or simple forms of reinforcement learning [56].”
Cichy, R. M., Khosla, A., Pantazis, D., Torralba, A., & Oliva, A. (2016) · Scientific Reports · doi:10.1038/srep27755 · Cited by 716 · cited 1×
“In terms of visual perception and representation learning, DNNs have been used to model brain activations in the visual cortex [12, 13, 14] and judgments of image similarity [15, 16].”
An Attentive Survey of Attention Models
Chaudhari, S., Mithal, V., Polatkan, G., & Ramanath, R. (2021) · ACM Transactions on Intelligent Systems and Technology · doi:10.1145/3465055 · Cited by 689 · cited 1×
“On a basic level, also the role of attention or memory could be studied with our setup, for example by using neural network agents with attention mechanisms [90] or external memory [91].”
Comparative mapping of higher visual areas in monkeys and humans
Orban, G. A., Van Essen, D., & Vanduffel, W. (2004) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2004.05.009 · Cited by 639 · cited 1×
“A mainstream view is that the two visual systems share many characteristics but are not identical [72, 73].”
Schuster, P., & Sigmund, K. (1983) · Journal of Theoretical Biology · doi:10.1016/0022-5193(83)90445-9 · Cited by 577
Convolutional Neural Networks as a Model of the Visual System: Past, Present, and Future
Lindsay, G. W. (2021) · Journal of Cognitive Neuroscience · doi:10.1162/jocn_a_01544 · Cited by 557 · cited 1×
“Thereby, they rely on the same principles—albeit being much simpler—as state of the art models of vision [80, 81].”
Evolutionary Dynamics and Extensive Form Games
Cressman, R. (2003) · The MIT Press eBooks · doi:10.7551/mitpress/2884.001.0001 · Cited by 551 · cited 1×
“This is also known as symmetrizing the game [57, Section 3.4].”
InfluentialA Survey of Experiments on Communication via Cheap Talk
Crawford, V. (1998) · Journal of Economic Theory · doi:10.1006/jeth.1997.2359 · Cited by 551 · cited 1×
“Communication games have been used to study the emergence and evolution of language theoretically [29], experimentally [30, 31], and computationally [32].”
Possible stages in the evolution of the language capacity
Jackendoff, R. (1999) · Trends in Cognitive Sciences · doi:10.1016/s1364-6613(99)01333-9 · Cited by 536
Coordinating perceptually grounded categories through language: A case study for colour
steels, L., & belpaeme, T. (2005) · Behavioral and Brain Sciences · doi:10.1017/s0140525x05000087 · Cited by 440 · cited 2×
“For example, in a case study with color stimuli, simulated agents established color categories and labels by playing a (perceptual) discrimination game, paired with a color reference game [36].”
Color naming reflects optimal partitions of color space
Regier, T., Kay, P., & Khetarpal, N. (2007) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.0610341104 · Cited by 409 · cited 1×
“Expressions for concrete concepts like colors depend on perception [17].”
Recurrence is required to capture the representational dynamics of the human visual system
Kietzmann, T. C., Spoerer, C. J., Sörensen, L. K. A., Cichy, R. M., Hauk, O., & Kriegeskorte, N. (2019) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1905544116 · Cited by 406 · cited 1×
“Including recurrence improves object recognition, especially under challenging conditions [85], and is required to model the representational dynamics of the visual system [86].”
Signals: Evolution, Learning, and Information
Skyrms, B. (2010) · DIAL (Catholic University of Leuven) · doi:10.1093/acprof:oso/9780199580828.001.0001 · Cited by 381
The mystery of language evolution
Hauser, M. D., Yang, C., Berwick, R. C., Tattersall, I., Ryan, M. J., Watumull, J., Chomsky, N., & Lewontin, R. C. (2014) · Frontiers in Psychology · doi:10.3389/fpsyg.2014.00401 · Cited by 329 · cited 1×
“Furthermore, it is uncertain when language emerged [74].”
Emergent Multi-Agent Communication in the Deep Learning EraPreprint
Lazaridou, A., & Baroni, M. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2006.02419 · Cited by 324
Wolff, P., & Holmes, K. J. (2011) · WIREs Cognitive Science · doi:10.1002/wcs.104 · Cited by 301 · cited 1×
“Such Whorfian effects are widespread; apart from visual perception they have, for example, been observed in motion, spatial relations, number, and false belief understanding [92].”
Bisk, Y., Holtzman, A., Thomason, J., Andreas, J., Bengio, Y., Chai, J., Lapata, M., Lazaridou, A., May, J., Nisnevich, A., et al. (2020) · Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2020.emnlp-main.703 · Cited by 274
Effects of Language on Visual Perception
Lupyan, G., Abdel Rahman, R., Boroditsky, L., & Clark, A. (2020) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2020.08.005 · Cited by 243
The role of similarity in the development of categorization
Sloutsky, V. M. (2003) · Trends in Cognitive Sciences · doi:10.1016/s1364-6613(03)00109-8 · Cited by 234 · cited 1×
“Given that concept formation in humans depends on perceptual similarity [26], our manipulations target the similarity relationships between object representations.”
Natural Language From Artificial Life
Kirby, S. (2002) · Artificial Life · doi:10.1162/106454602320184248 · Cited by 230 · cited 1×
“Communication games have been used to study the emergence and evolution of language theoretically [29], experimentally [30, 31], and computationally [32].”
Language games for autonomous robots
Steels, L. (2001) · IEEE Intelligent Systems · doi:10.1109/5254.956077 · Cited by 219 · cited 1×
“It has been shown that by playing communication games, artificial (robotic) agents can self-organize symbolic systems that are grounded in sensorimotor interactions with the world and other agents [34, 35, 36, 37].”
Recurrent Convolutional Neural Networks: A Better Model of Biological Object Recognition
Spoerer, C. J., McClure, P., & Kriegeskorte, N. (2017) · Frontiers in Psychology · doi:10.3389/fpsyg.2017.01551 · Cited by 211 · cited 1×
“Including recurrence improves object recognition, especially under challenging conditions [85], and is required to model the representational dynamics of the visual system [86].”
Özgen, E., & Davies, I. R. L. (2002) · Journal of Experimental Psychology: General · doi:10.1037//0096-3445.131.4.477 · Cited by 201 · cited 1×
“It has been shown that learning new color categories (through a perceptual task) induces categorical effects on color discrimination similar to those of natural color categories [64].”
Cangelosi, A., & Harnad, S. (2001) · Evolution of Communication · doi:10.1075/eoc.4.1.07can · Cited by 198 · cited 1×
“Later, Cangelosi and Harnad compared agents that learned categories from sensorimotor interaction with the world (“sensorimotor toil”) to agents that could additionally learn from communication signals (“symbolic theft”) [67].”
Compositionality and Generalization In Emergent Languages
Chaabouni, R., Kharitonov, E., Bouchacourt, D., Dupoux, E., & Baroni, M. (2020) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2020.acl-main.407 · Cited by 197
Deep neural networks in computational neuroscience
Kietzmann, T. C., McClure, P., & Kriegeskorte, N. (2019) · Oxford Research Encyclopedia of Neuroscience · doi:10.1093/acrefore/9780190264086.013.46 · Cited by 186
The conceptual grouping effect: Categories matter (and named categories matter more)
Lupyan, G. (2008) · Cognition · doi:10.1016/j.cognition.2008.03.009 · Cited by 163 · cited 1×
“Although there is a baseline effect of conceptual categories on visual processing, this effect increases if the target category is labeled [65].”
Referential communication and category acquisition.
Markman, A. B., & Makin, V. S. (1998) · Journal of Experimental Psychology: General · doi:10.1037//0096-3445.127.4.331 · Cited by 134 · cited 1×
“Category structure aligns between people who play a reference game [68], and more generally between people who assign novel labels to stimuli with the goal to coordinate [69].”
PERLOVSKY, L. I. (2009) · Neural Networks · doi:10.1016/j.neunet.2009.03.007 · Cited by 134 · cited 1×
“…that all interfaces between cognition and language are mutually adapted towards optimal interaction in the environments we face [93], such that language can guide the acquisition of cognitive representations from experience, and in turn, can be used to structure and exchange these experiences [94].”
Jozwik, K. M., Kriegeskorte, N., Storrs, K. R., & Mur, M. (2017) · Frontiers in Psychology · doi:10.3389/fpsyg.2017.01726 · Cited by 131 · cited 1×
“In terms of visual perception and representation learning, DNNs have been used to model brain activations in the visual cortex [12, 13, 14] and judgments of image similarity [15, 16].”
From global similarities to kinds of similarities: the construction of dimensions in development
Smith, L. B. (1989) · Similarity and Analogical Reasoning · doi:10.1017/cbo9780511529863.008 · Cited by 131 · cited 1×
“, color, shape, magnitude, texture) is a major achievement in child development (Smith, 1989).”
BackgroundEvaluating (and Improving) the Correspondence Between Deep Neural Networks and Human Representations
Peterson, J. C., Abbott, J. T., & Griffiths, T. L. (2018) · Cognitive Science · doi:10.1111/cogs.12670 · Cited by 130 · cited 1×
“In terms of visual perception and representation learning, DNNs have been used to model brain activations in the visual cortex [12, 13, 14] and judgments of image similarity [15, 16].”
Steels, L. (1998) · VUBIR (Vrije Universiteit Brussel) · Cited by 106
Experimental Evidence on the Evolution of Meaning of Messages in Sender-Receiver Games.
Blume, A. (1998) · RePEc: Research Papers in Economics · Cited by 105
The co-evolution of language and emotions
Jablonka, E., Ginsburg, S., & Dor, D. (2012) · Philosophical Transactions of the Royal Society B: Biological Sciences · doi:10.1098/rstb.2012.0117 · Cited by 96 · cited 1×
“…it seems likely that all interfaces between cognition and language are mutually adapted towards optimal interaction in the environments we face [93], such that language can guide the acquisition of cognitive representations from experience, and in turn, can be used to structure and exchange…”
(2021) · doi:10.1162/jocn_a_01755 · Cited by 85 · cited 1×
“Thereby, they rely on the same principles—albeit being much simpler—as state of the art models of vision [80, 81].”
Categorical facilitation with equally discriminable colors
Witzel, C., & Gegenfurtner, K. R. (2015) · Journal of Vision · doi:10.1167/15.8.22 · Cited by 81 · cited 1×
“For example, in color discrimination tasks, categorical effects are observed for naive but not trained observers [79], and sometimes only in the presence but not in the absence of verbal cues [21].”
How agents see things: On visual representations in an emergent language game
Bouchacourt, D., & Baroni, M. (2018) · Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d18-1119 · Cited by 78
Categories, Concepts, and Conceptual Development
Sloutsky, V. M., & Deng, W. (. (2019) · Language, Cognition and Neuroscience · doi:10.1080/23273798.2017.1391398 · Cited by 77 · cited 1×
“Categories can also be formed through interactions with the world, and nonhuman animals as well as preverbal human infants can learn categories [46].”
Constructing inferences and relations during text comprehension
Graesser, A. C., Hastings, P., & Wiemer-Hastings, K. (2001) · Human cognitive processing · doi:10.1075/hcp.8.14gra · Cited by 71 · cited 1×
“For instance, in understanding a written text, we are able to recruit from memory the right background assumptions to make the text coherent [88].”
The Evolution of Representation in Simple Cognitive Networks
Marstaller, L., Hintze, A., & Adami, C. (2013) · Neural Computation · doi:10.1162/neco_a_00475 · Cited by 70 · cited 1×
“Most likely, representational structure in humans is optimized to accommodate environmentally relevant conceptualizations as well [70, 71].”
(2017) · doi:10.3389/fncom.2016.00131 · Cited by 59
Lazaridou, A., Potapenko, A., & Tieleman, O. (2020) · Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/2020.acl-main.685 · Cited by 56
Forder, L., & Lupyan, G. (2019) · Journal of Experimental Psychology: General · doi:10.1037/xge0000560 · Cited by 51 · cited 1×
“For example, in color discrimination tasks, categorical effects are observed for naive but not trained observers [79], and sometimes only in the presence but not in the absence of verbal cues [21].”
Communicating artificial neural networks develop efficient color-naming systems
Chaabouni, R., Kharitonov, E., Dupoux, E., & Baroni, M. (2021) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2016569118 · Cited by 39 · cited 2×
“color-naming systems—low complexity is preferred [4].”
Emergent Linguistic Phenomena in Multi-Agent Communication Games
Harding Graesser, L., Cho, K., & Kiela, D. (2019) · Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP) · doi:10.18653/v1/d19-1384 · Cited by 39
Vagueness and Imprecise Imitation in Signalling Games
Franke, M., & Correia, J. P. (2017) · The British Journal for the Philosophy of Science · doi:10.1093/bjps/axx002 · Cited by 36 · cited 1×
“…rather encompassing high-level formalization of a wide variety of agent-internal optimization processes, be they cross-generational as in cultural evolution or (asexual) reproduction [54], or within-generational as in imitation-based dynamics [55, 54] or simple forms of reinforcement learning [56].”
Self-citationEmergent Language Generalization and Acquisition Speed are not tied to Compositionality
Kharitonov, E., & Baroni, M. (2020) · BlackboxNLP Workshop on Analyzing and Interpreting Neural Networks for NLP (BlackboxNLP) · doi:10.18653/v1/2020.blackboxnlp-1.2 · Cited by 34
Miss Tools and Mr Fruit: Emergent Communication in Agents Learning about Object Affordances
Bouchacourt, D., & Baroni, M. (2019) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/p19-1380 · Cited by 30 · cited 1×
“Some of these models skip any form of perceptual processing by using symbolic input data [38, 39, 40].”
Internal and external pressures on language emergence: least effort, object constancy and frequency
Luna, D. R., Ponti, E. M., Hupkes, D., & Bruni, E. (2020) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2020.findings-emnlp.397 · Cited by 30
Receptor architecture of macaque and human early visual areas: not equal, but comparable
Rapan, L., Niu, M., Zhao, L., Funck, T., Amunts, K., Zilles, K., & Palomero‐Gallagher, N. (2021) · Brain Structure and Function · doi:10.1007/s00429-021-02437-y · Cited by 29 · cited 1×
“A mainstream view is that the two visual systems share many characteristics but are not identical [72, 73].”
The Emergence of the Shape Bias Results from Communicative Efficiency
Portelance, E., Frank, M. C., Jurafsky, D., Sordoni, A., & Laroche, R. (2021) · Conference on Computational Natural Language Learning · doi:10.18653/v1/2021.conll-1.48 · Cited by 25
Learning sparse and meaningful representations through embodiment
Clay, V., König, P., Kühnberger, K., & Pipa, G. (2020) · Neural Networks · doi:10.1016/j.neunet.2020.11.004 · Cited by 22 · cited 1×
“As an example of a change in objective, an embodied DNN agent has been shown to learn sparse and interpretable representations through interactions with its simulated environment [87].”
A reinforcement-learning approach to efficient communication
Kågebäck, M., Carlsson, E., Dubhashi, D., & Sayeed, A. (2020) · PLOS ONE · doi:10.1371/journal.pone.0234894 · Cited by 13 · cited 1×
“In terms of language emergence and language evolution, simulations with neural network agents have been used to model, for example, the emergence of color naming systems [4, 5], contact linguistic phenomena [6], the emergence of word learning biases [7, 8], or the emergence of compositional structure [9, 10, 11].”
Ahn, S., Zelinsky, G. J., & Lupyan, G. (2021) · Journal of Vision · doi:10.1167/jov.21.13.13 · Cited by 12 · cited 1×
“As a relatively minor change, training on superordinate or both superordinate and basic labels, rather than on subordinate labels as is typically the case, makes visual representations more robust and more human-like [84].”
Novel Labels Increase Category Coherence, But Only When People Have the Goal to Coordinate
Suffill, E. M., Branigan, H. P., & Pickering, M. J. (2019) · Cognitive Science · doi:10.1111/cogs.12796 · Cited by 7 · cited 1×
“Category structure aligns between people who play a reference game [68], and more generally between people who assign novel labels to stimuli with the goal to coordinate [69].”
Background
Research works printed but not linked: 17
- D Lewis, 1969, Convention
- HH Clark, 1992, Arenas of language use
- Ohmer X, König P, Franke M. Reinforcement of semantic representations in pragmatic agents leads to the emergence of a mutual exclusivity bias. In: Proceedings of the 42nd Annual Meeting of the Cognitive Science Society (CogSci); 2020. p. 1779–1785.
- Choi E, Lazaridou A, de Freitas N. Compositional obverter communication learning from raw visual input. In: Proceedings of the 6th International Conference on Learning Representations (ICLR); 2018. p. 1–18.
- Li F, Bowling M. Ease-of-teaching and language structure from emergent communication. In: Proceedings of the 33rd International Conference on Neural Information Processing Systems (NeurIPS); 2019. p. 1–11.
- Ren Y, Guo S, Labeau M, Cohen SB, Kirby S. Compositional languages emerge in a neural iterated learning model. In: Proceedings of the 8th International Conference on Learning Representations (ICLR); 2020. p. 1–22.
- G Lakoff, 1980, Metaphors we live by
- Havrylov S, Titov I. Emergence of language with multi-agent games: Learning to communicate with sequences of symbols. In: Proceedings of the 31st International Conference on Neural Information Processing Systems (NeurIPS); 2017. p. 2149–2159.
- Mikolov T, Joulin A, Baroni M. A Roadmap towards machine intelligence. arXiv preprint. 2015;arXiv:1511.08130. · arXiv
- Bleys J, Loetzsch M, Spranger M, Steels L. The grounded colour naming game. In: Proceedings of the 18th IEEE International Symposium on Robot and Human Interactive Communication (Ro-Man); 2009. p. 1–7.
- Lazaridou A, Peysakhovich A, Baroni M. Multi-agent cooperation and the emergence of (natural) language. In: Proceedings of the 5th International Conference on Learning Representations (ICLR); 2017. p. 1–11.
- Hill F, Clark S, Hermann KM, Blunsom P. Simulating early word learning in situated connectionist agents. In: Proceedings of the 42nd Annual Meeting of the Cognitive Science Society (CogSci); 2020. p. 875–881.
- Devlin J, Chang MW, Lee K, Toutanova K. BERT: Pre-training of deep bidirectional transformers for language understanding. In: Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics (NAACL); 2019. p. 4171–4186.
- Gandhi K, Lake BM. Mutual exclusivity as a challenge for neural networks. In: Proceedings of the 34th International Conference on Neural Information Processing Systems (NeurIPS); 2020. p. 14182–14192.
- Harnad S, Hanson SJ, Lubin J. Categorical perception and the evolution of supervised learning in neural nets. In: Powers DW, Reeker L, editors. Working Papers of the AAAI Spring Symposium on Machine Learning of Natural Language and Ontology; 1991. p. 65–74.
- A Noë, 2004, Action in perception
- KR Storrs, 2021, Diverse deep neural networks all predict human inferior temporal cortex well, after training and fitting, Journal of Cognitive Neuroscience, 33, 2044
Cited by
Emergent language: a survey and taxonomy
Peters, J., de Puiseau, C. W., Tercan, H., Gopikrishnan, A., de Carvalho, G. A. L., Bitter, C., & Meisen, T. (2025) · Autonomous Agents and Multi-Agent Systems · doi:10.1007/s10458-025-09691-y · Cited by 37
Zubek, J., Korbak, T., & Rączaszek‐Leonardi, J. (2024) · Artificial Intelligence Review · doi:10.1007/s10462-024-11048-y · Cited by 10
“Similar models were used to study how aspects such as innate perceptual biases (Baronchelli et al. 2015; Ohmer et al. 2022) or the topology of a social network (Centola and Baronchelli 2015; Zubek et al. 2017 ) affect the formation of categories.”
Signalling and social learning in swarms of robots
Cazenille, L., Toquebiau, M., Lobato‐Dauzier, N., Loi, A., Macabre, L., Aubert-Kato, N., Genot, A. J., & Bredèche, N. (2025) · Philosophical Transactions of the Royal Society A Mathematical Physical and Engineering Sciences · doi:10.1098/rsta.2024.0148 · Cited by 14
Horst, R., Dörner, R., Sinha, S. N., & Kühn, P. J. (2026) · Computers & Graphics · doi:10.1016/j.cag.2026.104683 · Cited by 2
Communication games: Modeling language evolution through dyadic agent interactionsPreprint
Ohmer, X. I., & Cuskley, C. F. (2023) · doi:10.31234/osf.io/mqzgr · Cited by 1
Self-citation
Metadata
- Authors
- Ohmer, X., Marino, M., Franke, M., & König, P.
- Affiliations
- Osnabrück University (DE)
- Corresponding author
- Ohmer, Xenia
- Venue
- PLOS Computational Biology
- Publisher
- Public Library of Science (PLoS)
- Published
- 2022-10-31
- Volume
- 18
- Issue
- 10
- Pages
- e1010658
- ISSN
- 1553-734X (ISSN-L)
- 1553-7358
- PMC
- PMC9648844
- OpenAlex
- W4307698991
- Cited by
- 6
- References
- 75
- Citations by year
- 2026: 1 · 2025: 3 · 2024: 1 · 2023: 1
- FWCI
- 6.32
- Type
- journal article
- Language
- en
- Update notices
- new_version (2022-11-10)
- Open access
- gold
- doi.org
- Licence
- Locations
- PLoS Computational Biology (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion)
- arXiv (Cornell University) (cc-by)
- Funders
- Deutsche Forschungsgemeinschaft (GRK 2340)
- Topics
- Social Sciences › Cultural Studies › Language and cultural evolution
- Social Sciences › Psychology › Experimental and Cognitive Psychology › Categorization, perception, and language
- Social Sciences › Psychology › Developmental and Educational Psychology › Child and Animal Learning Development
- Keywords
- language emergence, visual perception, multi-agent communication, communication games, visual representation learning, communication protocols, semantic classification, perceptual bias, co-adaptation, evolutionary analysis, deep neural network
- MeSH terms
- Cognition, Communication, Language, Semantics, Visual Perception
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.