Generative models, linguistic communication and active inference
Open accessThe DOI matches its Crossref record; 53 of the 53 references it cites were found in a registry or an index.
Abstract
This paper presents a biologically plausible generative model and inference scheme that is capable of simulating communication between synthetic subjects who talk to each other. Building on active inference formulations of dyadic interactions, we simulate linguistic exchange to explore generative models that support dialogues. These models employ high-order interactions among abstract (discrete) states in deep (hierarchical) models. The sequential nature of language processing mandates generative models with a particular factorial structure-necessary to accommodate the rich combinatorics of language. We illustrate linguistic communication by simulating a synthetic subject who can play the 'Twenty Questions' game. In this game, synthetic subjects take the role of the questioner or answerer, using the same generative model. This simulation setup is used to illustrate some key architectural points and demonstrate that many behavioural and neurophysiological correlates of linguistic communication emerge under variational (marginal) message passing, given the right kind of generative model. For example, we show that theta-gamma coupling is an emergent property of belief updating, when listening to another.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2018) · arXiv · doi:10.48550/arxiv.1810.04805 · Cited by 73,173
The free-energy principle: a unified brain theory?
Friston, K. (2010) · Nature Reviews Neuroscience · doi:10.1038/nrn2787 · Cited by 8,120 · cited 2×
“…ideas about how the brain works; from Kant through Helmholtz (Helmholtz, 1878 (1971)), from analysis by synthesis (Yuille and Kersten, 2006) to perception as hypothesis testing (Gregory, 1980), from the Helmholtz machine (Dayan et al., 1995) to the free energy principle (Friston, 2010).”
Self-citationRizzolatti, G., & Craighero, L. (2004) · Annual Review of Neuroscience · doi:10.1146/annurev.neuro.27.070203.144230 · Cited by 7,433 · cited 1×
“, visual search in active vision) to an interrogation of the world via semantics and semiotics, acquired by encultured learning (Constant et al., 2019; Creanza and Feldman, 2014; Penn et al., 2008; Rizzolatti and Craighero, 2004).”
Factor graphs and the sum-product algorithm
Kschischang, F. R., Frey, B. J., & Loeliger, H.-A. (2001) · IEEE Transactions on Information Theory · doi:10.1109/18.910572 · Cited by 6,486 · cited 1×
“The use of factors is known as a mean field approximation in the variational machine learning literature (Jaakkola and Jordan, 1998; Kschischang et al., 2001; Sallans and Hinton, 2004; Zhang et al., 2018) and is important for simplifying the form of the generative model and ensuing inference.”
The faculty of language: what is it, who has it, and how did it evolve?
Hauser, M. D., Chomsky, N., & Fitch, W. T. (2002) · Science · doi:10.1126/science.298.5598.1569 · Cited by 5,189
SCHEGLOFF, E. A., & SACKS, H. (1973) · Semiotica · doi:10.1515/semi.1973.8.4.289 · Cited by 4,589
Hoeting, J. A., Madigan, D., Raftery, A. E., & Volinsky, C. T. (1999) · Statistical Science · doi:10.1214/ss/1009212519 · Cited by 4,118 · cited 1×
“In Bayesian model selection, this phenomena is known as evidence dilution (Hoeting et al., 1999).”
Is the P300 component a manifestation of context updating?
Donchin, E., & Coles, M. G. H. (1988) · Behavioral and Brain Sciences · doi:10.1017/s0140525x00058027 · Cited by 3,585 · cited 7×
“Our simulations also predict electrophysiological violation responses, of the sorts observed in P300 and N400 studies.”
BackgroundInfluentialRecent Trends in Deep Learning Based Natural Language Processing [Review Article]
Young, T., Hazarika, D., Poria, S., & Cambria, E. (2018) · IEEE Computational Intelligence Magazine · doi:10.1109/mci.2018.2840738 · Cited by 2,921
Linguistic complexity: locality of syntactic dependencies
Gibson, E. (1998) · Cognition · doi:10.1016/s0010-0277(98)00034-1 · Cited by 2,537 · cited 1×
“There is a substantial literature on cognitive models of syntax processing (for a recent review, see Demberg and Keller, 2019) and how listeners deal with semantic ambiguity (Altmann and Steedman, 1988; Bever, 1970; Gibson, 1998, 2000).”
Cited by
The default mode network: where the idiosyncratic self meets the shared social world
Yeshurun, Y., Nguyen, M., & Hasson, U. (2021) · Nature Reviews Neuroscience · doi:10.1038/s41583-020-00420-w · Cited by 682
Laying down a forking path: tensions between enaction and the free energy principle
Di Paolo, E., Thompson, E., & Beer, R. (2022) · Philosophy and the Mind Sciences · doi:10.33735/phimisci.2022.9187 · Cited by 94
Generating meaning: active inference and the scope and limits of passive AI
Pezzulo, G., Parr, T., Cisek, P., Clark, A., & Friston, K. (2024) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2023.10.002 · Cited by 80
Self-citationFederated inference and belief sharing
Friston, K. J., Parr, T., Heins, C., Constant, A., Friedman, D., Isomura, T., Fields, C., Verbelen, T., Ramstead, M., Clippinger, J., et al. (2024) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2023.105500 · Cited by 30
Self-citationOn the minimal theory of consciousness implicit in active inference
Whyte, C. J., Corcoran, A. W., Robinson, J., Smith, R., Moran, R. J., Parr, T., Friston, K. J., Seth, A. K., & Hohwy, J. (2026) · Physics of Life Reviews · doi:10.1016/j.plrev.2025.11.002 · Cited by 15
Self-citation
Metadata
- Authors
- Friston, K. J., Parr, T., Yufik, Y., Sajid, N., Price, C. J., & Holmes, E.
- Venue
- Neuroscience & Biobehavioral Reviews
- Publisher
- Elsevier BV
- Published
- 2020-11
- Volume
- 118
- Pages
- 42-64
- ISSN
- 0149-7634 (ISSN-L)
- OpenAlex
- W3042228302
- Cited by
- 108
- References
- 53
- FWCI
- 7.39
- Type
- journal article
- Language
- en
- Open access
- hybrid
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Generative models, linguistic communication and active inference. https://app.cochat.ai/papers/10.1016/j.neubiorev.2020.07.005
- The paper
- Friston, K. J., Parr, T., Yufik, Y., Sajid, N., Price, C. J., & Holmes, E. (2020). Generative models, linguistic communication and active inference. Neuroscience & Biobehavioral Reviews. https://doi.org/10.1016/j.neubiorev.2020.07.005
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.