CoChat
Sign Up
Cited by 31Language and cultural evolutionCategorization, perception, and languageEmbodied and Extended Cognition

Symbol Emergence as an Interpersonal Multimodal Categorization

Open access
Hagiwara, Yoshinobu, Kobayashi, Hiroyoshi, Taniguchi, Akira, Taniguchi, Tadahiro·2019·Frontiers in Robotics and AI
DOI 10.3389/frobt.2019.00134PMC PMC7805687
Open this paper in CoChat — free accountOpen PDF

Abstract

This study focuses on category formation for individual agents and the dynamics of symbol emergence in a multi-agent system through semiotic communication. In this study, the semiotic communication refers to exchanging signs composed of the signifier (i.e., words) and the signified (i.e., categories). We define the generation and interpretation of signs associated with the categories formed through the agent's own sensory experience or by exchanging signs with other agents as basic functions of the semiotic communication. From the viewpoint of language evolution and symbol emergence, organization of a symbol system in a multi-agent system (i.e., agent society) is considered as a bottom-up and dynamic process, where individual agents share the meaning of signs and categorize sensory experience. A constructive computational model can explain the mutual dependency of the two processes and has mathematical support that guarantees a symbol system's emergence and sharing within the multi-agent system. In this paper, we describe a new computational model that represents symbol emergence in a two-agent system based on a probabilistic generative model for multimodal categorization. It models semiotic communication via a probabilistic rejection based on the receiver's own belief. We have found that the dynamics by which cognitively independent agents create a symbol system through their semiotic communication can be regarded as the inference process of a hidden variable in an interpersonal multimodal categorizer, i.e., the complete system can be regarded as a single agent performing multimodal categorization using the sensors of all agents, if we define the rejection probability based on the Metropolis-Hastings algorithm. The validity of the proposed model and algorithm for symbol emergence, i.e., forming and sharing signs and categories, is also verified in an experiment with two agents observing daily objects in the real-world environment. In the experiment, we compared three communication algorithms: no communication, no rejection, and the proposed algorithm. The experimental results demonstrate that our model reproduces the phenomena of symbol emergence, which does not require a teacher who would know a pre-existing symbol system. Instead, the multi-agent system can form and use a symbol system without having pre-existing categories.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.3389/frobt.2019.00134

Cites

1 of 1 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • The Metropolis-Hastings Naming Game (MHNG) has been proposed as a concrete algorithmic realization of CPC, and has been extended to N-agent populations through a recursive formulation.

    “The Metropolis-Hastings Naming Game (MHNG) has been proposed as its concrete algorithmic realization (Hagiwara et al., 2019; Taniguchi et al., 2023), which has been further extended to N -agent populations through a recursive formulation (Inukai et al., 2023).”

    From Social Reality Construction via Active Inference: Modeling the Dialectic of Conformity and Creativity (2026) · cites this paper as Hagiwara et al., 2019

    Section 3.2 describes a two-agent word-naming exchange governed by Metropolis–Hastings acceptance, and Section 3.4 presents the algorithm as a distributed inference procedure for symbol emergence. Our copy does not identify it as a realization of CPC. The recursive N-agent extension is attributed to Inukai et al. (2023) in the citing sentence, not to this source. (LLM, conf=0.84)

    Experimental — accuracy not yet measured

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Flagged first, then by citations

Research works printed but not linked: 43

  1. Ando Y. Nakamura T. Nagai T. (2013). “Formation of hierarchical object concept using hierarchical latent dirichlet allocation,” in IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) (Tokyo), 2272– 2279.
  2. Bleys J. (2015). Language Strategies for the Domain of Colour. Number 3 in Computational Models of Language Evolution. Berlin: Language Science Press.
  3. Cangelosi A. Schlesinger M. (2014). Developmental Robotics: From Babies to Robots. The MIT Press.
  4. De Beule J. De Vylder B. Belpaeme T. (2006). “A cross-situational learning algorithm for damping homonymy in the guessing game,” in Artificial Life X, eds L. M. Rocha, L. S. Yaeger, M. A. Bedau, D. Floreano, R. L. Goldstone, and A. Vespignani (MIT Press), 466– 472.
  5. Donahue J. Jia Y. Vinyals O. Hoffman J. Zhang N. Tzeng E. (2013). Decaf: a deep convolutional activation feature for generic visual recognition. CoRR abs/1310.1531. Available online at: http://arxiv.org/abs/1310.1531 · arXiv
  6. Graesser L. Cho K. Kiela D. (2019). Emergent linguistic phenomena in multi-agent communication games. CoRR abs/1901.08706. Available online at: http://arxiv.org/abs/1901.08706 · arXiv
  7. Hagiwara Y. Inoue M. Kobayashi H. Taniguchi T. (2018). Hierarchical spatial concept formation based on multimodal information for human support robots. Front. Neurorobot. 12: 11. 10.3389/fnbot.2018.00011 29593521 PMC5859180 · DOI
  8. Hagiwara Y. Masakazu I. Tadahiro T. (2016). Place concept learning by hmlda based on position and vision information. IFAC Papers Online 49, 216– 220. 10.1016/j.ifacol.2016.10.527 · DOI
  9. Isobe S. Taniguchi A. Hagiwara Y. Taniguchi T. (2017). “Learning relationships between objects and places by multimodal spatial concept with bag of objects,” in Social Robotics eds A. Kheddar, E, Yoshida, S. Ge, S, Sam, K. Suzuki, J. J. Cabibihan, F. Eyssel, and H. He (Cham: Springer International Publishing), 115– 125.
  10. Jia Y. Shelhamer E. Donahue J. Karayev S. Long J. Girshick R. (2014). “Caffe: convolutional architecture for fast feature embedding,” in Proceedings of the 22nd ACM international conference on Multimedia (Orlando, FL: ACM), 675– 678.
  11. Kirby S. (1998). “Learning, bottlenecks and the evolution of recursive syntax,” in Linguistic Evolution through Language Acquisition: Formal and Computational Models, ed T. Briscoe (Cambridge University Press).
  12. Krizhevsky A. Sutskever I. Hinton G. E. (2012). “Imagenet classification with deep convolutional neural networks,” in Advances in Neural Information Processing Systems (Lake Tahoe, NV), 1097– 1105.
  13. Landis J. R. Koch G. G. (1977). The measurement of observer agreement for categorical data. Biometrics 33, 159– 174. 843571
  14. Lazaridou A. Peysakhovich A. Baroni M. (2016). Multi-agent cooperation and the emergence of (natural) language. CoRR abs/1612.07182. Available online at: http://arxiv.org/abs/1612.07182 · arXiv
  15. Lee J. Cho K. Weston J. Kiela D. (2017). Emergent translation in multi-agent communication. CoRR abs/1710.06922. Available online at: http://arxiv.org/abs/1710.06922 · arXiv
  16. Marocco D. Cangelosi A. Fischer K. Belpaeme T. (2010). Grounding action words in the sensorimotor interaction with the world: experiments with a simulated icub humanoid robot. Front. Neurorobot. 4: 7. 10.3389/fnbot.2010.00007 20725503 PMC2901088 · DOI
  17. Matsukawa T. Suzuki E. (2016). “Person re-identification using cnn features learned from combination of attributes,” in 2016 23rd International Conference on Pattern Recognition (ICPR) (Cancun), 2428– 2433.
  18. Matuszek C. (2018). “Grounded language learning: where robotics and NLP meet,” in Proceedings of the Twenty-Seventh International Joint Conference on Artificial Intelligence, IJCAI-18, Stockholm: International Joint Conferences on Artificial Intelligence Organization, 5687– 5691.
  19. Morita J. Konno T. Hashimoto T. (2012). “The role of imitation in generating a shared communication system,” in Proceedings of the 34th Annual Meeting of the Cognitive Science Society (Sapporo), 779– 784.
  20. Nakamura T. Nagai T. Funakoshi K. Nagasaka S. Taniguchi T. Iwahashi N. (2014). “Mutual learning of an object concept and language model based on MLDA and NPYLM,” in 2014 IEEE/RSJ International Conference on Intelligent Robots and Systems (Chicago, IL), 600– 607.
  21. Nakamura T. Nagai T. Iwahashi N. (2009). “Grounding of word meanings in multimodal concepts using LDA,” in IEEE/RSJ International Conference on Intelligent Robots and Systems (St. Louis, MO), 3943– 3948.
  22. Nakamura T. Nagai T. Taniguchi T. (2018). Serket: an architecture for connecting stochastic models to realize a large-scale cognitive model. Front. Neurorobot. 12: 25. 10.3389/fnbot.2018.00025 29997493 PMC6028621 · DOI
  23. Okanoya K. Merker B. (2007). Neural Substrates for String-Context Mutual Segmentation: A Path to Human Language. London: Springer London, 421– 434.
  24. Oshikawa S. Nakamura T. Nagai T. Kaneko M. Funakoshi K. Iwahashi N. (2018). “Interaction modeling based on segmenting two persons motions using coupled GP-HSMM,” in 2018 27th IEEE International Symposium on Robot and Human Interactive Communication (RO-MAN) (Nanjing), 288– 293.
  25. Spranger M. (2011). The evolution of grounded spatial language (Ph.D. thesis), Vrije Universiteit Brussel, Brussels, Belgium.
  26. Spranger M. (2015). The Evolution of Grounded Spatial Language. Number 5 in Computational Models of Language Evolution. Berlin: Language Science Press.
  27. Spranger M. Loetzsch M. Steels L. (2012). “A perceptual system for language game experiments,” in Language Grounding in Robots, eds L. Steels, and M. Hild (Springer), 89– 110.
  28. Steels L. (1995). A self-organizing spatial vocabulary. Artif. Life 2, 319– 332. 8925502
  29. Steels L. (2015). The Talking Heads Experiment: Origins of Words and Meanings. Berlin: Language Science Press.
  30. Steels L. Belpaeme T. (2005). Coordinating perceptually grounded categories through language: a case study for colour. Behav. Brain Sci. 28, 469-489. 10.1017/S0140525X05000087 16209771 · DOI · Paper page
  31. Steels L. Kaplan F. (2000). Aibo's first words: the social learning of language and meaning. Evol. Commun. 4, 3– 32. 10.1075/eoc.4.1.03ste · DOI
  32. Steels L. Loetzsch M. (2008). “Perspective alignment in spatial language,” in Spatial Language and Dialogue, eds K. R. Coventry, T. Tenbrink, and J. A. Bateman (Oxford University Press).
  33. Stramandinoli F. Marocco D. Cangelosi A. (2017). Making sense of words: a robotic model for language abstraction. Auton. Robots 41, 367– 383. 10.1007/s10514-016-9587-8 · DOI
  34. Suzuki M. Nakayama K. Matsuo Y. (2016). Joint multimodal learning with deep generative models. arXiv 1611.01891. Available online at: https://arxiv.org/abs/1611.01891 · arXiv
  35. Tangiuchi T. Mochihashi D. Nagai T. Uchida S. Inoue N. Kobayashi I. (2019). Survey on frontiers of language and robotics. Adv. Robot. 33, 700– 730. 10.1080/01691864.2019.1632223 · DOI
  36. Taniguchi A. Taniguchi T. Cangelosi A. (2017). Cross-situational learning with Bayesian generative models for multimodal category and word learning in robots. Front. Neurorobot. 11: 66. 10.3389/fnbot.2017.00066 29311888 PMC5742219 · DOI
  37. Taniguchi A. Taniguchi T. Inamura T. (2016). Spatial concept acquisition for a mobile robot that integrates self-localization and unsupervised word discovery from spoken sentences. IEEE Trans. Cogn. Dev. Syst. 8, 285– 297. 10.1109/TCDS.2016.2565542 · DOI
  38. Taniguchi T. Nagai T. Nakamura T. Iwahashi N. Ogata T. Asoh H. (2016). Symbol emergence in robotics: a survey. Adv. Robot. 30, 706– 728. 10.1080/01691864.2016.1164622 · DOI · Paper page
  39. Vogt P. (2002). The physical symbol grounding problem. Cogn. Syst. Res. 3, 429– 457. 10.1016/S1389-0417(02)00051-7 · DOI
  40. Vogt P. (2005). The emergence of compositional structures in perceptually grounded language games. Artif. Intell. 167, 206– 242. 10.1016/j.artint.2005.04.010 · DOI
  41. Yamamoto T. Terada K. Ochiai A. Saito F. Asahara Y. Murase K. (2019). Development of human support robot as the research platform of a domestic mobile manipulator. ROBOMECH J. 6: 4 10.1186/s40648-019-0132-3 · DOI
  42. Zeiler M. D. Fergus R. (2014). “Visualizing and understanding convolutional networks,” in Computer Vision–ECCV 2014, eds D. Fleet, T. Pajdla, B. Schiele, and T. Tuytelaars (Cham: Springer International Publishing), 818– 833.
  43. Zhong J. Peniak M. Tani J. Ogata T. Cangelosi A. (2019). Sensorimotor input as a language generalisation tool: a neurorobotics model for generation and generalisation of noun-verb combinations with sensorimotor inputs. Auton. Robots 43, 1271– 1290. 10.1007/s10514-018-9793-7 · DOI

Cited by

Sorted by citations

Authors
Hagiwara, Y., Kobayashi, H., Taniguchi, A., & Taniguchi, T.
Affiliations
Ritsumeikan University (JP)
Corresponding author
Hagiwara, Yoshinobu
ORCID
Venue
Frontiers in Robotics and AI
Publisher
Frontiers Media SA
Published
2019-12-10
Volume
6
Pages
134-134
ISSN
2296-9144 (ISSN-L)
DOI
10.3389/frobt.2019.00134
PMC
PMC7805687
OpenAlex
W2996361813
Cited by
31
References
1
Citations by year
2026: 4 · 2025: 5 · 2024: 5 · 2023: 8 · 2022: 4 · 2021: 4 · 2020: 1
FWCI
16.41
Type
journal article
Language
en
Open access
Licence
Locations
  • Frontiers in Robotics and AI (publishedVersion, cc-by)
  • Europe PMC (PubMed Central) (publishedVersion, cc-by)
  • PubMed Central (publishedVersion, cc-by)
  • PubMed (publishedVersion)
  • arXiv (Cornell University) (submittedVersion)
  • DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion)
  • arXiv (Cornell University)
Funders
Japan Society for the Promotion of Science
Topics
  • Social Sciences › Cultural Studies › Language and cultural evolution
  • Social Sciences › Psychology › Experimental and Cognitive Psychology › Categorization, perception, and language
  • Life Sciences › Neuroscience › Cognitive Neuroscience › Embodied and Extended Cognition
Keywords
symbol emergence, semiotic communication, multi-agent systems, category formation, sign formation, probabilistic generative models, agent societies, language evolution

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review