CoChat
Sign Up
Cited by 33Language and cultural evolutionCategorization, perception, and languageEmbodied and Extended Cognition

Emergent communication through Metropolis-Hastings naming game with deep generative models

Open access
Taniguchi, Tadahiro, Yoshida, Yuto, Matsui, Yuta, Le Hoang, Nguyen, Taniguchi, Akira, Hagiwara, Yoshinobu·2023·Advanced Robotics
DOI 10.1080/01691864.2023.2260856
Open this paper in CoChat — free accountOpen PDF

Abstract

Constructive studies on symbol emergence systems seek to investigate computational models that can better explain human language evolution, the creation of symbol systems, and the construction of internal representations. Specifically, emergent communication aims to formulate a computational model that enables agents to build efficient sign systems and internal representations. This study provides a new model for emergent communication, which is based on a probabilistic generative model (PGM) instead of a discriminative model based on deep reinforcement learning. We define the Metropolis-Hastings (MH) naming game by generalizing previously proposed models. It is not a referential game with explicit feedback, as assumed by many emergent communication studies. Instead, it is a game based on joint attention without explicit feedback. Mathematically, the MH naming game is proved to be a type of MH algorithm for an integrative PGM that combines two agents that play the naming game. From this viewpoint, symbol emergence is regarded as decentralized Bayesian inference, and semiotic communication is regarded as inter-personal cross-modal inference. This notion leads to the collective predictive coding hypothesis regarding language evolution and, in general, the emergence of symbols. We also propose the inter-Gaussian mixture model (GMM)+ variational autoencoder (VAE), a deep generative model for emergent communication based on the MH naming game. In this model, two agents create internal representations and categories and share signs (i.e. names of objects) from raw visual images observed from different viewpoints. The model has been validated on MNIST and Fruits 360 datasets. Experimental findings demonstrate that categories are formed from real images observed by agents, and signs are correctly shared across agents by successfully utilizing both of the observations of agents via the MH naming game. Furthermore, scholars verified that visual images were recalled from signs uttered by agents. Notably, emergent communication without supervision and reward feedback improved the performance of the unsupervised representation learning of agents.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1080/01691864.2023.2260856

Cites

4 of 4 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • The Metropolis-Hastings Naming Game (MHNG) has been proposed as a concrete algorithmic realization of CPC, and has been extended to N-agent populations through a recursive formulation.

    “The Metropolis-Hastings Naming Game (MHNG) has been proposed as its concrete algorithmic realization (Hagiwara et al., 2019; Taniguchi et al., 2023), which has been further extended to N -agent populations through a recursive formulation (Inukai et al., 2023).”

    From Social Reality Construction via Active Inference: Modeling the Dialectic of Conformity and Creativity (2026) · cites this paper as Taniguchi et al., 2023

  • MHNG is a method for sampling latent variables that can be shared between two agents who jointly attend to the same artifacts.

    “MHNG is a method for sampling latent variables that can be shared between two agents who jointly attend to the same artifacts.”

    From Social Reality Construction via Active Inference: Modeling the Dialectic of Conformity and Creativity (2026) · cites this paper as Taniguchi et al., 2023

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Flagged first, then by citations

Research works printed but not linked: 59

  1. Steels L. The synthetic modeling of language origins. Evol Commun J. 1997;10:1.
  2. Taniguchi T, Ugur E, Hoffmann M, et al. Symbol emer- gence in cognitive developmental systems: a survey. IEEE Trans Cogn Develop Syst. 2018;11(4):494-516. doi: 10.1109/TCDS · DOI
  3. Taniguchi T, Nagai T, Nakamura T, et al. Symbol emer- gence in robotics: a survey. Adv Robot. 2016;30(11-12): 706-728. doi: 10.1080/01691864.2016.1164622 · DOI · Paper page
  4. Taniguchi T, El Hafi L, Hagiwara Y, et al. Semiotically adaptive cognition: toward the realization of remotely- operated service robots for the new normal symbiotic society. Adv Robot. 2021;35:1-11.
  5. Barsalou LW. Perceptual symbol system. Behav Brain Sci. 1999;22:277-660.
  6. Steels L. The talking heads experiment: origins of words and meanings. Berlin: Language Science Press; 2015.
  7. Chandler D. Semiotics the basics. Oxfordshire: Routledge; 2002.
  8. Lewis D. Convention: a philosophical study. Toronto: John Wiley & Sons; 2008.
  9. Steels L, Kaplan F. AIBO's first words: the social learning of language and meaning. Evol Commun. 2000;4(1):3-32. doi: 10.1075/eoc.4.1.03ste. Available from: https://www. jbe-platform.com/content/journals/10.1075/eoc.4.1.03ste · DOI
  10. Steels L. A self-organizing spatial vocabulary. Artif Life. 1995;2(3):319-332. doi: 10.1162/artl.1995.2.319 · DOI
  11. Steels L, Belpaeme T. Coordinating perceptually grounded categories through language: a case study for colour. Behav Brain Sci. 2005;28(4):469-489. doi: 10.1017/S0140 525X05000087 · DOI · Paper page
  12. Steels L, Loetzsch M. Perspective alignment in spatial language. In: Coventry KR, Tenbrink T, Bateman JA, edi- tors. Spatial language and dialogue. Northamptonshire: Oxford University Press; 2008.
  13. Spranger M. The evolution of grounded spatial language [Ph.D. thesis]. Brussels: Vrije Universiteit Brussel; 2011.
  14. Spranger M, Loetzsch M, Steels L. A perceptual system for language game experiments. In: Steels L, Hild M, editors. Language grounding in robots. New York: Springer; 2012. p. 89-110.
  15. Spranger M. The evolution of grounded spatial language. Vol. 5, in Computational models of language evolution. Berlin: Language Science Press; 2015 .
  16. Vogt P. The physical symbol grounding problem. Cogn Syst Res. 2002;3(3):429-457. doi:10.1016/S1389-0417(02) 00051-7. Available from: https://www.sciencedirect.com/ science/article/pii/S1389041702000517 · DOI
  17. Vogt P. The emergence of compositional structures in perceptually grounded language games. Artif Intell. 2005;167(1):206-242. doi:10.1016/j.artint.2005.04.010. Available from: https://www.sciencedirect.com/science/ article/pii/S0004370205001001 · DOI
  18. Bleys J. Language strategies for the domain of colour, Computational models of language evolution, Berlin: Language Science Press, 2015.
  19. Foerster JN, Assael YM, de Freitas N, et al. Learning to communicate with deep multi-agent reinforcement learn- ing. In: International Conference on Neural Information Processing Systems (NIPS); 2016. p. 2145-2153 .
  20. Lazaridou A, Peysakhovich A, Baroni M. Multi-agent cooperation and the emergence of (natural) language. In: The International Conference on Learning Representa- tions (ICLR); 2017.
  21. Graesser L, Cho K, Kiela D. Emergent linguistic phenom- ena in multi-agent communication games. In: Conference on Empirical Methods in Natural Language Processing (EMNLP); 2019 .
  22. Chaabouni R, Strub F, Altché F, et al. Emergent communi- cation at scale. In: International Conference on Learning Representations (ICLR); 2021 .
  23. Lazaridou A, Baroni M. Emergent multi-agent commu- nication in the deep learning era; 2020. ArXiv preprint arXiv:200602419.
  24. Tucker M, Li H, Agrawal S, et al. Emergent discrete com- munication in semantic spaces. In: Advances in Neural Information Processing Systems (NeurIPS); 2021 .
  25. Mu J, Goodman N. Emergent communication of general- izations. In: Advances in Neural Information Processing Systems (NeurIPS); 2021 .
  26. Jiang J, Lu Z. Learning attentional communication for multi-agent cooperation. Adv Neural Inf Process Syst (NeurIPS). 2018;31.
  27. Kim W, Park J, Sung Y. Communication in multi-agent reinforcement learning: intention sharing. In: Interna- tional Conference on Learning Representations (ICLR); 2020 .
  28. Li F, Bowling M. Ease-of-teaching and language structure from emergent communication. Adv Neural Inf Process Syst (NeurIPS). 2019;32.
  29. Eccles T, Bachrach Y, Lever G, et al. Biases for emergent communication in multi-agent reinforcement learning. Adv Neural Inf Process Syst (NeurIPS). 2019;32.
  30. Cangelosi A, Schlesinger M. Developmental robotics: from babies to robots. Massachusetts: MIT Press; 2015.
  31. Kingma DP, Welling M. Auto-encoding variational bayes; 2013. ArXiv preprint arXiv:13126114.
  32. Suzuki M, Nakayama K, Matsuo Y. Joint multimodal learning with deep generative models. In: Interna- tional Conference on Learning Representations (ICLR); 2016.
  33. Taniguchi A, Hagiwara Y, Taniguchi T, et al. Improved and scalable online learning of spatial concepts and language models with mapping. Auton Robots. 2020;44:1-20.
  34. Nakamura T, Nagai T, Iwahashi N. Grounding of word meanings in multimodal concepts using lda. In: IEEE/RSJ International Conference on Intelligent Robots and Sys- tems (IROS); 2009. p. 3943-3948.
  35. Nakamura T, Nagai T, Funakoshi K, et al. Mutual learn- ing of an object concept and language model based on MLDA and NPYLM. In: IEEE/RSJ International Con- ference on Intelligent Robots and Systems (IROS); 2014. p. 600-607.
  36. Hohwy J. The predictive mind. Northamptonshire: Oxford University Press; 2013.
  37. Ciria A, Schillaci G, Pezzulo G, et al. Predictive pro- cessing in cognitive robotics: a review. Neural Comput. 2021;33(5):1402-1432. doi: 10.1162/neco_a_01383 · DOI · Paper page
  38. Friston K, Moran RJ, Nagai Y, et al. World model learn- ing and inference. Neural Netw. 2021;144:573-590. doi: 10.1016/j.neunet.2021.09.011. Available from: https:// www.sciencedirect.com/science/article/pii/S0893608021 003610 · DOI
  39. Okada M, Kosaka N, Taniguchi T. PlaNet of the Bayesians: reconsidering and improving deep planning network by incorporating bayesian inference. In: IEEE/RSJ Interna- tional Conference on Intelligent Robots and Systems (IROS); 2020. p. 5611-5618 .
  40. Hagiwara Y, Kobayashi H, Taniguchi A, et al. Sym- bol emergence as an interpersonal multimodal catego- rization. Frontiers in Robotics and AI. 2019;6:134. doi: 10.3389/frobt.2019.00134. Availale from: https://www. frontiersin.org/article/10.3389/frobt.2019.00134 · DOI · Paper page
  41. Hagiwara Y, Furukawa K, Taniguchi A, et al. Mul- tiagent multimodal categorization for symbol emer- gence: emergent communication via interpersonal cross- modal inference. Adv Robot. 2022;36(5-6):239-260. doi: 10.1080/01691864.2022.2029721 · DOI
  42. Taniguchi T, Nakamura T, Suzuki M, et al. Neuro-serket: development of integrative cognitive system through the composition of deep probabilistic generative models. New Gener Comput. 2020;38:1-26.
  43. Nakamura T, Nagai T, Taniguchi TS. An architecture for connecting stochastic models to realize a large-scale cognitive model. Front Neurorobot. 2018;12:25. doi: 10.3389/fnbot.2018.00025 · DOI
  44. Hastings WK. Monte Carlo sampling methods using Markov chains and their applications. Biometrika. 1970; 57:97-109. doi: 10.1093/biomet/57.1.97 · DOI · Paper page
  45. Galke L, Ram Y, Raviv L. Emergent communication for understanding human language evolution: what's miss- ing? In: Workshop on Emergent Communication at ICLR; 2022.
  46. Johnson MJ, Duvenaud DK, Wiltschko A, et al. Compos- ing graphical models with neural networks for structured representations and fast inference. Adv Neural Inf Process Syst (NIPS). 2016;29.
  47. Oord A, Vinyals O, Kavukcuoglu K. Neural discrete rep- resentation learning. In: Advances in Neural Information Processing Systems (NIPS); 2017 .
  48. Razavi A, Vinyals O. Generating diverse high-fidelity images with VQ-VAE-2. In: Advances in Neural Infor- mation Processing Systems (NeurIPS); 2019. p. 14866- 14876.
  49. Kuniyasu R, Nakamura T, Taniguchi T, et al. Robot con- cept acquisition based on interaction between probabilis- tic and deep generative models. Frontiers in Computer Science. 2021;3:3.
  50. Nakamura T, Nagai T, Taniguchi TS. An architecture for connecting stochastic models to realize a large- scale cognitive model. Front Neurorobot. 2018;12:25. doi: 10.3389/fnbot.2018.00025. Available from: https://www. frontiersin.org/article/10.3389/fnbot.2018.00025 · DOI
  51. Hubert L, Arabie P. Comparing partitions. J Classif. 1985;2(1):193-218. doi: 10.1007/BF01908075 · DOI · Paper page
  52. Cohen J. A coefficient of agreement for nominal scales. Educ Psychol Meas. 1960;20(1):37-46. doi: 10.1177/ 001316446002000104 · DOI · Paper page
  53. Van der Maaten L, Hinton G. Visualizing data using t-sne. J Mach Learn Res. 2008;9(11):2579-2605.
  54. Alcantarilla PF, Bartoli A, Davison AJ. KAZE features. In: European Conference on Computer Vision (ECCV); 2012. p. 214-227.
  55. Teh YW, Jordan MI, Beal MJ, et al. Hierarchical Dirich- let processes. J Am Stat Assoc. 2006;101(476):1566-1581. doi: 10.1198/016214506000000302 · DOI
  56. Blei DM, Jordan MI. Variational inference for Dirichlet process mixtures. Bayesian Analysis. 2006;1(1):121-143. doi: 10.1214/06-BA104 · DOI
  57. Nakamura T, Nagai T, Iwahashi N. Multimodal cate- gorization by hierarchical Dirichlet process. In: 2011 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS); 2011. p. 1520-1525 .
  58. Okumura R, Taniguchi T, Hagiwara Y, et al. Metropolis- Hastings algorithm in joint-attention naming game: experimental semiotics study; 2023. ArXiv preprint arXiv:230519936. .
  59. Landis JR, Koch GG. The measurement of observer agree- ment for categorical data. Biometrics. 1977;33:159-174. doi: 10.2307/2529310. · DOI · Paper page

Cited by

Sorted by citations

Authors
Taniguchi, T., Yoshida, Y., Matsui, Y., Le Hoang, N., Taniguchi, A., & Hagiwara, Y.
Affiliations
  • Ritsumeikan University (JP)
  • Graduate School of Information Science and Engineering, Ritsumeikan University, Kusatsu, Shiga, Japan
Corresponding author
Taniguchi, Tadahiro
ORCID
Venue
Advanced Robotics
Publisher
Informa UK Limited
Published
2023-09-29
Volume
37
Issue
19
Pages
1266-1282
ISSN
  • 0169-1864 (ISSN-L)
  • 1568-5535
DOI
10.1080/01691864.2023.2260856
OpenAlex
W4387164066
Cited by
33
References
4
Citations by year
2026: 6 · 2025: 9 · 2024: 10 · 2023: 7 · 2022: 1
FWCI
31.31
Type
journal article
Language
en
Open access
Licence
Locations
  • Advanced Robotics (publishedVersion, cc-by-nc-nd)
  • Institutional Repositories DataBase (IRDB) (publishedVersion)
  • arXiv (Cornell University) (submittedVersion)
  • arXiv (Cornell University)
Funders
  • Japan Society for the Promotion of Science (17H06383)
  • MEXT Grant-in-Aid for Scientific Research on Innovative Areas 4903 (JP21H04904, JP23H04835, and JP18K18134)
Topics
  • Social Sciences › Cultural Studies › Language and cultural evolution
  • Social Sciences › Psychology › Experimental and Cognitive Psychology › Categorization, perception, and language
  • Life Sciences › Neuroscience › Cognitive Neuroscience › Embodied and Extended Cognition
Keywords
emergent communication, naming game, Metropolis-Hastings algorithm, symbol emergence, language evolution, deep generative models, variational autoencoder, joint attention, predictive coding, unsupervised feature learning

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review