CoChat

A reinforcement-learning approach to efficient communication

Open access
Kågebäck, Mikael, Carlsson, Emil, Dubhashi, Devdatt, Sayeed, Asad · 2020 · PLOS ONE

The DOI matches its Crossref record; 40 of the 42 references it cites were found in a registry or an index.

Abstract

We present a multi-agent computational approach to partitioning semantic spaces using reinforcement-learning (RL). Two agents communicate using a finite linguistic vocabulary in order to convey a concept. This is tested in the color domain, and a natural reinforcement learning mechanism is shown to converge to a scheme that achieves a near-optimal trade-off of simplicity versus communication efficiency. Results are presented both on the communication efficiency as well as on analyses of the resulting partitions of the color space. The effect of varying environmental factors such as noise is also studied. These results suggest that RL offers a powerful and flexible computational framework that can contribute to the development of communication schemes for color names that are near-optimal in an information-theoretic sense and may shape color-naming systems across languages. Our approach is not specific to color and can be used to explore cross-language variation in other semantic domains.

Overview

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1371/journal.pone.0234894

Cites

40 of 40 references verified0 citation errors · 0 not checkable

Statistics

Figures

Fig 1 Human semantic systems will tend to lie along this optimal frontier of achievable efficiency in communication systems. Reprinted with permission from [21].
Fig 1 Human semantic systems will tend to lie along this optimal frontier of achievable efficiency in communication systems. Reprinted with permission from [21].CC BY · PubMed Central
Fig 2 Reproduced color mode maps [4] corresponding to four randomly selected languages of the WCS.
Fig 2 Reproduced color mode maps [4] corresponding to four randomly selected languages of the WCS.CC BY · PubMed Central
Fig 3 An overview showing each computation step in the model, while using continuous real valued messages during training, for one instance of the color game. Black solid arrows indicate a differentiable relation while red dashed arrows indicates a non-differentiable relation. The color of the ovals are used to highlight the different parts of the model where black is the model input, blue and yellow the agents, green the reward system, and red the reinforce cost function.
Fig 3 An overview showing each computation step in the model, while using continuous real valued messages during training, for one instance of the color game. Black solid arrows indicate a differentiable relation while red dashed arrows indicates a non-differentiable relation. The color of the ovals are used to highlight the different parts of the model where black is the model input, blue and yellow the agents, green the reward system, and red the reinforce cost function.CC BY · PubMed Central
Fig 4 An overview showing each computation step in the model, while using discrete messages during training, for one instance of the color game. Black solid arrows indicate a differentiable relation while red dashed arrows indicates a non-differentiable relation. The color of the ovals are used to highlight the different parts of the model where black is the model input, blue and yellow the agents, green the reward system, and red the reinforce cost function.
Fig 4 An overview showing each computation step in the model, while using discrete messages during training, for one instance of the color game. Black solid arrows indicate a differentiable relation while red dashed arrows indicates a non-differentiable relation. The color of the ovals are used to highlight the different parts of the model where black is the model input, blue and yellow the agents, green the reward system, and red the reinforce cost function.CC BY · PubMed Central
Fig 5 2D histogram showing the number of emerged communication systems that end up using a particular number of color terms when trained using a particular amount of environmental noise. 25 experiments are run for each level of noise σe²∈{1,2,4,8,16,32,64,128,256,512}. Hence, each bin on the x axis shows the distribution over number of words resulting from that level of noise.
Fig 5 2D histogram showing the number of emerged communication systems that end up using a particular number of color terms when trained using a particular amount of environmental noise. 25 experiments are run for each level of noise σe²∈{1,2,4,8,16,32,64,128,256,512}. Hence, each bin on the x axis shows the distribution over number of words resulting from that level of noise.CC BY · PubMed Central
Fig 6 KL loss for varying number of color words used. The circles indicate the KL loss of individual WCS languages sorted based on the number of color words used. The shaded regions indicate a 95% confidence interval. Note, the WCS language data points is a reproduction from [4].
Fig 6 KL loss for varying number of color words used. The circles indicate the KL loss of individual WCS languages sorted based on the number of color words used. The shaded regions indicate a 95% confidence interval. Note, the WCS language data points is a reproduction from [4].CC BY · PubMed Central
Fig 7 Expected surprise for varying number of color terms used. The shaded regions indicate a 95% confidence interval.
Fig 7 Expected surprise for varying number of color terms used. The shaded regions indicate a 95% confidence interval.CC BY · PubMed Central
Fig 8 Well–formedness for varying number of color words used. The circles indicate the well–formedness of individual WCS languages sorted based on the number of words. The shaded region indicates a 95% confidence interval.
Fig 8 Well–formedness for varying number of color words used. The circles indicate the well–formedness of individual WCS languages sorted based on the number of words. The shaded region indicates a 95% confidence interval.CC BY · PubMed Central
Fig 9 Color space partitions for different number of color terms from four different sources. The first column show human language consensus maps, i.e., consensus cross all languages in the WCS that uses the particular number of color term indicated to the left. The second column corresponds to the consensus maps over reinforcement learning partitionings using continuous real valued messages during training. The third column corresponds to the consensus maps over reinforcement learning partitionings using discrete messages during training. Finally, the fourth column of partitions is constructed using correlation clustering directly on the graph defined by the CIELAB distance between each color tile. Notice that, for 11 color terms, no partition was generated using discrete messages.
Fig 9 Color space partitions for different number of color terms from four different sources. The first column show human language consensus maps, i.e., consensus cross all languages in the WCS that uses the particular number of color term indicated to the left. The second column corresponds to the consensus maps over reinforcement learning partitionings using continuous real valued messages during training. The third column corresponds to the consensus maps over reinforcement learning partitionings using discrete messages during training. Finally, the fourth column of partitions is constructed using correlation clustering directly on the graph defined by the CIELAB distance between each color tile. Notice that, for 11 color terms, no partition was generated using discrete messages.CC BY · PubMed Central
Fig 10 Color maps captured during one training session as the emerging language progress towards an increasing number of terms.
Fig 10 Color maps captured during one training session as the emerging language progress towards an increasing number of terms.CC BY · PubMed Central
Fig 11 Change in the number of words used by the agents during training. The X-axis represents the number of episodes trained and the Y-axis the number of words used at that point.
Fig 11 Change in the number of words used by the agents during training. The X-axis represents the number of episodes trained and the Y-axis the number of words used at that point.CC BY · PubMed Central
Fig 12 KL loss change during training. The X-axis shows the number of episodes.
Fig 12 KL loss change during training. The X-axis shows the number of episodes.CC BY · PubMed Central
Fig 13 The number of color terms used by the agents when different amounts of noise are applied to their environments.
Fig 13 The number of color terms used by the agents when different amounts of noise are applied to their environments.CC BY · PubMed Central
Fig 14 The number of color terms used by the agents when different amounts of noise are applied to their communication.
Fig 14 The number of color terms used by the agents when different amounts of noise are applied to their communication.CC BY · PubMed Central

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Show all 40 references

Research works printed but not linked: 2

  1. TM Cover, 2006, Elements of Information Theory 2nd Edition (Wiley Series in Telecommunications and Signal Processing)
  2. S Havrylov, 2017, Advances in Neural Information Processing Systems 30, 2146

Cited by

See all 14

Metadata

Authors
Kågebäck, M., Carlsson, E., Dubhashi, D., & Sayeed, A.
Affiliations
  • Chalmers University of Technology (SE)
  • University of Gothenburg (SE)
Corresponding author
Kågebäck, Mikael
Venue
PLOS ONE
Published
2020-07-15
Volume
15
Issue
7
Pages
e0234894
ISSN
1932-6203 (ISSN-L)
OpenAlex
W3043710493
Cited by
13
References
42
Citations by year
2026: 2 · 2025: 3 · 2024: 2 · 2023: 3 · 2022: 2 · 2021: 1
FWCI
1.45
Type
journal article
Language
en
Open access
Locations
  • PLoS ONE (publishedVersion, cc-by)
  • PubMed Central (publishedVersion, cc-by)
  • PubMed (publishedVersion)
  • RePEc: Research Papers in Economics (submittedVersion)
  • DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
  • Chalmers Research (Chalmers University of Technology) (submittedVersion)
Funders
Vetenskapsrådet (2012-5738, 2015-2025)
Topics
  • Social Sciences › Psychology › Experimental and Cognitive Psychology › Categorization, perception, and language
  • Social Sciences › Cultural Studies › Language and cultural evolution
  • Physical Sciences › Computer Science › Artificial Intelligence › Reinforcement Learning in Robotics
Keywords
reinforcement learning, communication efficiency, multi-agent systems, color naming, color terms, information-theoretic optimality, noise
MeSH terms
Color, Color Perception, Communication, Humans, Language, Learning, Linguistics, Models, Theoretical, Names, Reinforcement, Psychology, Semantics, Vocabulary

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: A reinforcement-learning approach to efficient communication. https://app.cochat.ai/papers/10.1371/journal.pone.0234894
The paper
Kågebäck, M., Carlsson, E., Dubhashi, D., & Sayeed, A. (2020). A reinforcement-learning approach to efficient communication. PLOS ONE. https://doi.org/10.1371/journal.pone.0234894

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.