The DOI matches its Crossref record; 40 of the 42 references it cites were found in a registry or an index.
A reinforcement-learning approach to efficient communication
Open accessAbstract
We present a multi-agent computational approach to partitioning semantic spaces using reinforcement-learning (RL). Two agents communicate using a finite linguistic vocabulary in order to convey a concept. This is tested in the color domain, and a natural reinforcement learning mechanism is shown to converge to a scheme that achieves a near-optimal trade-off of simplicity versus communication efficiency. Results are presented both on the communication efficiency as well as on analyses of the resulting partitions of the color space. The effect of varying environmental factors such as noise is also studied. These results suggest that RL offers a powerful and flexible computational framework that can contribute to the development of communication schemes for color names that are near-optimal in an information-theoretic sense and may shape color-naming systems across languages. Our approach is not specific to color and can be used to explore cross-language variation in other semantic domains.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Figures
![Fig 1 Human semantic systems will tend to lie along this optimal frontier of achievable efficiency in communication systems. Reprinted with permission from [21].](https://figures.cochat.ai/paper-figures/2ae62c8844c34bbcb94d898cf076a0f9/a-reinforcement-learning-approach-to-efficient-communication-figure-1.jpg)
![Fig 2 Reproduced color mode maps [4] corresponding to four randomly selected languages of the WCS.](https://figures.cochat.ai/paper-figures/2ecd60a97e5c4be08a93787410860752/a-reinforcement-learning-approach-to-efficient-communication-figure-2.jpg)



![Fig 6 KL loss for varying number of color words used. The circles indicate the KL loss of individual WCS languages sorted based on the number of color words used. The shaded regions indicate a 95% confidence interval. Note, the WCS language data points is a reproduction from [4].](https://figures.cochat.ai/paper-figures/235efda5223e46938d5c6a2d43f1e5fc/a-reinforcement-learning-approach-to-efficient-communication-figure-6.jpg)








Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Utility-based approaches, including recent studies that evaluate the communicative efficiency of utility-maximizing systems (e.g., Carlsson et al., 2023;Chaabouni, Kharitonov, et al., 2021;Kågebäck et al., 2020) lack this type of explanatory power and instead turn to assumptions about environmental factors (e.g., noise or external rewards) and fixed vocabulary sizes to account for cross-language”
From Towards Human-Like Emergent Communication via Utility, Informativeness, and Complexity (2025) · cites this paper as Kågebäck et al., 2020
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Adam: A Method for Stochastic Optimization
Kingma, D. P., & Ba, J. (2014) · arXiv · doi:10.48550/arxiv.1412.6980 · Cited by 98,946 · cited 1×
“All parameters are initialized to random values and trained using stochastic gradient decent with ADAM [36].”
MethodReinforcement Learning: An Introduction
Sutton, R. S., & Barto, A. G. (1998) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.1998.712192 · Cited by 27,225 · cited 1×
“Introducing a baseline to the cost function is a standard procedure used to reduce the inherent high variance in the REINFORCE algorithm [34] and we add this baseline to cope with the difficulties induced by using discrete messages.”
Reinforcement Learning: An Introduction
(2005) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.2004.842673 · Cited by 24,496
Advances in Neural Information Processing Systems 29
Teymur, O., Zygalakis, K., & Calderhead, B. (2016) · Cited by 13,887
Objective Criteria for the Evaluation of Clustering Methods
Rand, W. M. (1971) · Journal of the American Statistical Association · doi:10.1080/01621459.1971.10482356 · Cited by 6,566
Simple Statistical Gradient-Following Algorithms for Connectionist Reinforcement Learning
Williams, R. J. (1992) · Machine Learning · doi:10.1007/bf00992696 · Cited by 3,416
Basic Color Terms: Their Universality and Evolution.
Morrill, W. T., Berlin, B., & Kay, P. (1971) · Man · doi:10.2307/2798490 · Cited by 2,173
Learning to Communicate with Deep Multi-Agent Reinforcement Learning
Foerster, J. N., Assael, Y. M., de Freitas, N., & Whiteson, S. (2016) · arXiv · doi:10.48550/arxiv.1605.06676 · Cited by 1,217 · cited 1×
“Thus, the environment will be non-stationary and it will be harder for the agents to agree on a certain color partition and converge [35].”
BackgroundHuman Behaviour and the Principle of Least Effort.
Florence, P. S., & Zipf, G. K. (1950) · The Economic Journal · doi:10.2307/2226729 · Cited by 1,194
Natural Scales in Geographical Patterns
Menezes, T., & Roth, C. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1704.01036 · Cited by 1,120
Bansal, N., Blum, A., & Chawla, S. (2004) · Machine Learning · doi:10.1023/b:mach.0000033116.57574.95 · Cited by 1,075 · cited 2×
“In fact this criterion is exactly the same as the maximizing agreements objective of the correlation clustering problem discussed extensively in the theoretical computer science literature [31, 32].”
BackgroundMethodSimple Statistical Gradient-Following Algorithms for Connectionist Reinforcement Learning
Williams, R. J. (1992) · Reinforcement Learning · doi:10.1007/978-1-4615-3618-5_2 · Cited by 903
Reinforcement learning in the brain
Niv, Y. (2009) · Journal of Mathematical Psychology · doi:10.1016/j.jmp.2008.12.005 · Cited by 717
Reinforcement learning: The Good, The Bad and The Ugly
Dayan, P., & Niv, Y. (2008) · Current Opinion in Neurobiology · doi:10.1016/j.conb.2008.08.003 · Cited by 704
Word lengths are optimized for efficient communication
Piantadosi, S. T., Tily, H., & Gibson, E. (2011) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1012551108 · Cited by 695
Reinforcement Learning: State-of-the-Art
(2012) · Adaptation, Learning, and Optimization · doi:10.1007/978-3-642-27645-3 · Cited by 691
Compression and communication in the cultural evolution of linguistic structure
Kirby, S., Tamariz, M., Cornish, H., & Smith, K. (2015) · Cognition · doi:10.1016/j.cognition.2015.03.016 · Cited by 557 · cited 1×
“One such increasingly influential proposal is that language is shaped by the need for efficient communication [3–7], which by its nature involves a trade-off [8, 9] between simplicity, which minimizes cognitive load, and informativeness which maximizes communication effectiveness.”
BackgroundCoordinating perceptually grounded categories through language: A case study for colour
steels, L., & belpaeme, T. (2005) · Behavioral and Brain Sciences · doi:10.1017/s0140525x05000087 · Cited by 440
Color naming reflects optimal partitions of color space
Regier, T., Kay, P., & Khetarpal, N. (2007) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.0610341104 · Cited by 409 · cited 5×
“We have left for future work another relevant baseline to which we could have compared our systems and which may set a higher bar for the comparison, as suggested by a reviewer: the rotational baseline [27], i.”
BackgroundMethodResultInfluentialColor naming across languages reflects color use
Gibson, E., Futrell, R., Jara-Ettinger, J., Mahowald, K., Bergen, L., Ratnasingam, S., Gibson, M., Piantadosi, S. T., & Conway, B. R. (2017) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1619666114 · Cited by 298 · cited 6×
“The information loss can be defined as the listener’s expected surprise [5], i.”
BackgroundMethodResultInfluentialCorrelation clustering in general weighted graphs
Demaine, E. D., Emanuel, D., Fiat, A., & Immorlica, N. (2006) · Theoretical Computer Science · doi:10.1016/j.tcs.2006.05.008 · Cited by 283 · cited 2×
“In fact this criterion is exactly the same as the maximizing agreements objective of the correlation clustering problem discussed extensively in the theoretical computer science literature [31, 32].”
BackgroundMethodMulti-Agent Cooperation and the Emergence of (Natural) Language
Lazaridou, A., Peysakhovich, A., & Baroni, M. (2016) · arXiv · doi:10.48550/arxiv.1612.07182 · Cited by 231
Lucy, J. A. (1997) · Color Categories in Thought and Language · doi:10.1017/cbo9780511519819.015 · Cited by 215
Word meanings across languages support efficient communication
Regier, T., Kemp, C., & Kay, P. (2015) · The Handbook of Language Emergence · doi:10.1002/9781118346136.ch11 · Cited by 196 · cited 10×
“An alternative approach, suggests the use of the KL divergence between the speaker distribution s and the listener distribution l [4], i.”
BackgroundMethodInfluentialEmergence of Language with Multi-agent Games: Learning to Communicate with Sequences of Symbols
Havrylov, S., & Titov, I. (2017) · arXiv · doi:10.48550/arxiv.1705.11192 · Cited by 195
Semantic typology and efficient communication
Kemp, C., Xu, Y., & Regier, T. (2018) · Annual Review of Linguistics · doi:10.1146/annurev-linguistics-011817-045406 · Cited by 180 · cited 4×
“Our approach could be used to explore various trade-offs at the heart of efficient communication [3].”
BackgroundMethodInfluentialEfficient compression in color naming and its evolution
Zaslavsky, N., Kemp, C., Regier, T., & Tishby, N. (2018) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1800521115 · Cited by 178
Modeling the emergence of universality in color naming patterns
Baronchelli, A., Gong, T., Puglisi, A., & Loreto, V. (2010) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.0908533107 · Cited by 175 · cited 1×
“While there has been previous work [23] on computational mechanisms involving naming games for the emergence of universality of color names, our work is the first to provide a mechanism based on a fundamental machine learning paradigm (reinforcement learning) that is also biologically plausible.”
BackgroundIt's not really red, green, yellow, blue: an inquiry into perceptual color space
Jameson, K., & D'Andrade, R. G. (1997) · Color Categories in Thought and Language · doi:10.1017/cbo9780511519819.014 · Cited by 159
Emergent Communication in a Multi-Modal, Multi-Step Referential GamePreprint
Evtimova, K., Drozdov, A., Kiela, D., & Cho, K. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1705.10369 · Cited by 116
Language structure: psychological and social constraints
Jäger, G., & van Rooij, R. (2006) · Synthese · doi:10.1007/s11229-006-9073-5 · Cited by 94
Simplicity and informativeness in semantic category systems
Carr, J. W., Smith, K., Culbertson, J., & Kirby, S. (2020) · Cognition · doi:10.1016/j.cognition.2020.104289 · Cited by 82 · cited 1×
“One such increasingly influential proposal is that language is shaped by the need for efficient communication [3–7], which by its nature involves a trade-off [8, 9] between simplicity, which minimizes cognitive load, and informativeness which maximizes communication effectiveness.”
BackgroundLearning to Play Guess Who? and Inventing a Grounded Language as a ConsequencePreprint
Jorge, E., Kågebäck, M., Johansson, F., & Gustavsson, E. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1611.03218 · Cited by 78
Self-citationReinforcement learning with Marr
Niv, Y., & Langdon, A. (2016) · Current Opinion in Behavioral Sciences · doi:10.1016/j.cobeha.2016.04.005 · Cited by 64
Explaining Universal Color Categories Through a Constrained Acquisition Process
Belpaeme, T., & Bleys, J. (2005) · Adaptive Behavior · doi:10.1177/105971230501300404 · Cited by 57
Baddeley, R., & Attewell, D. (2009) · Psychological Science · doi:10.1111/j.1467-9280.2009.02412.x · Cited by 46
Disinterring Basic Color Terms : a study in the mystique of cognitivism
Saunders, B. A. C. (1995) · History of the Human Sciences · doi:10.1177/095269519500800402 · Cited by 20 · cited 1×
“This work came under methodological criticism [25, 26], Fig 1.”
BackgroundCook, R. S. (2012) · Cited by 19
Simplicity and informativeness in semantic category systemsPreprint
Carr, J. W., Smith, K., Culbertson, J., et al. (2018) · doi:10.31234/osf.io/jkfyx · Cited by 6
(1949) · Max Planck Digital Library
Research works printed but not linked: 2
- TM Cover, 2006, Elements of Information Theory 2nd Edition (Wiley Series in Telecommunications and Signal Processing)
- S Havrylov, 2017, Advances in Neural Information Processing Systems 30, 2146
Cited by
Assessing the Strengths and Weaknesses of Large Language Models
Lappin, S. (2023) · Journal of Logic Language and Information · doi:10.1007/s10849-023-09409-x · Cited by 93
Communicating artificial neural networks develop efficient color-naming systems
Chaabouni, R., Kharitonov, E., Dupoux, E., & Baroni, M. (2021) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2016569118 · Cited by 39
“(21) do not explicitly discuss it, their results also point to a correlation between complexity reduction and discreteness, despite the fact that they control discreteness in a different way, that is, by injecting noise into a continuous channel.”
BackgroundThe evolution of color naming reflects pressure for efficiency: Evidence from the recent past
Zaslavsky, N., Garvin, K., Kemp, C., Tishby, N., & Regier, T. (2022) · Journal of Language Evolution · doi:10.1093/jole/lzac001 · Cited by 23
A review of reinforcement learning for natural language processing and applications in healthcare
Liu, Y., Wang, H., Zhou, H., Li, M., Hou, Y., Zhou, S., Wang, F., Hoetzlein, R., & Zhang, R. (2024) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocae215 · Cited by 33
Mutual influence between language and perception in multi-agent communication games
Ohmer, X., Marino, M., Franke, M., & König, P. (2022) · PLOS Computational Biology · doi:10.1371/journal.pcbi.1010658 · Cited by 6
“In terms of language emergence and language evolution, simulations with neural network agents have been used to model, for example, the emergence of color naming systems [4, 5], contact linguistic phenomena [6], the emergence of word learning biases [7, 8], or the emergence of compositional structure [9, 10, 11].”
Metadata
- Authors
- Kågebäck, M., Carlsson, E., Dubhashi, D., & Sayeed, A.
- Affiliations
- Chalmers University of Technology (SE)
- University of Gothenburg (SE)
- Corresponding author
- Kågebäck, Mikael
- Venue
- PLOS ONE
- Publisher
- Public Library of Science (PLoS)
- Published
- 2020-07-15
- Volume
- 15
- Issue
- 7
- Pages
- e0234894
- ISSN
- 1932-6203 (ISSN-L)
- PMC
- PMC7363069
- OpenAlex
- W3043710493
- Cited by
- 13
- References
- 42
- Citations by year
- 2026: 2 · 2025: 3 · 2024: 2 · 2023: 3 · 2022: 2 · 2021: 1
- FWCI
- 1.45
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- Locations
- PLoS ONE (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- RePEc: Research Papers in Economics (submittedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Chalmers Research (Chalmers University of Technology) (submittedVersion)
- Funders
- Vetenskapsrådet (2012-5738, 2015-2025)
- Topics
- Social Sciences › Psychology › Experimental and Cognitive Psychology › Categorization, perception, and language
- Social Sciences › Cultural Studies › Language and cultural evolution
- Physical Sciences › Computer Science › Artificial Intelligence › Reinforcement Learning in Robotics
- Keywords
- reinforcement learning, communication efficiency, multi-agent systems, color naming, color terms, information-theoretic optimality, noise
- MeSH terms
- Color, Color Perception, Communication, Humans, Language, Learning, Linguistics, Models, Theoretical, Names, Reinforcement, Psychology, Semantics, Vocabulary
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.