CoChat

The graphical brain: Belief propagation and active inference

Open access
Friston, Karl J., Parr, Thomas, de Vries, Bert · 2017 · Network Neuroscience

The DOI matches its Crossref record; 112 of the 112 references it cites were found in a registry or an index; none of the 73 sources its claims cite is checked for support yet.

Abstract

This paper considers functional integration in the brain from a computational perspective. We ask what sort of neuronal message passing is mandated by active inference—and what implications this has for context-sensitive connectivity at microscopic and macroscopic levels. In particular, we formulate neuronal processing as belief propagation under deep generative models. Crucially, these models can entertain both discrete and continuous states, leading to distinct schemes for belief updating that play out on the same (neuronal) architecture. Technically, we use Forney (normal) factor graphs to elucidate the requisite message passing in terms of its form and scheduling. To accommodate mixed generative models (of discrete and continuous states), one also has to consider link nodes or factors that enable discrete and continuous representations to talk to each other. When mapping the implicit computational architecture onto neuronal connectivity, several interesting features emerge. For example, Bayesian model averaging and comparison, which link discrete and continuous states, may be implemented in thalamocortical loops. These and other considerations speak to a computational connectome that is inherently state dependent and self-organizing in ways that yield to a principled (variational) account. We conclude with simulations of reading that illustrate the implicit neuronal message passing, with a special focus on how discrete (semantic) representations inform, and are informed by, continuous (visual) sampling of the sensorium.

Overview

CoChat Summary

TL;DR

This paper derives belief propagation under deep generative models as a computational architecture for functional integration in the brain, using Forney factor graphs to link discrete and continuous state representations. It maps the resulting message passing onto intrinsic/extrinsic cortical microcircuits and cortico–basal ganglia–thalamic loops, illustrating the scheme with simulations of pictographic reading.

Key findings

  • Belief propagation under deep generative models can be expressed with Forney factor graphs; discrete models use A/B/C/D matrices and policies, continuous models use generalized coordinates and predictive coding, and mixed models require link factors implementing Bayesian model averaging and comparison.
  • Discrete and continuous representations are linked by Bayesian model averaging (descending empirical priors) and Bayesian model comparison (ascending posterior over outcome models); thalamocortical loops may implement this, while expected free energy and policy expectations involve cortico–basal ganglia–thalamic loops.
  • In simulations of reading, the model has two discrete hierarchical levels with six sentences and four words, with letters/pictograms in quadrants; saccades are assumed to take about 256 ms, continuous integration uses 24/25 time steps of about 10 ms, and the subject resolves the sentence on the last saccade of the last word.
  • Minimizing variational free energy also minimizes Lyapunov exponents, so belief propagation organizes toward critical slowing; self-organized criticality is described as an epiphenomenon of self-evidencing.

Methods

  • Theoretical/computational derivation using Bayesian networks, Forney factor graphs, and neural network/circuit mappings; derives belief propagation for discrete, continuous, and mixed generative models.
  • Simulation of pictographic reading: two discrete hierarchical levels (six sentences, four words; letters/pictograms in quadrants), a continuous visuomotor model with 32 × 32 pixel images, equations integrated with 16 iterations per time point per discrete level, saccades about 256 ms, continuous integration 24/25 time steps of about 10 ms, and roughly 4.5 s of simulated time.
  • Main analysis is simulation-based: eye movements, posterior probabilities/evidence accumulation, simulated firing rates/depolarization, and band-pass filtered local field potentials between 4 Hz and 32 Hz; no empirical dataset is analyzed.

Limitations

  • Stated: the neural networks “should not be taken too seriously” and are offered as a starting point for refinement and deconstruction; many anatomical issues are ignored, including direct/indirect pathways and dopamine modulation.
  • Stated: the sensory attenuation model is “quite a crude way” of modeling sensory attenuation, and the reading simulation is “a bit abstract.”
  • Evident: the anatomical mappings are speculative and not empirically tested here, and the simulation results depend on the model assumptions and message-passing schedule.

Model-written summary. Check it against the paper before relying on it.

Integrity signals

Registry

Crossref · 10.1162/netn_a_00018

Cites

112 of 112 references verified0 citation errors · 0 not checkable

Statistics

Nothing to recompute.

Figures

Figure 1. Generative model for discrete states and outcomes.Upper left panel: These equations specify the generative model. A generative model is the joint probability of outcomes or consequences and their (latent or hidden) causes; see first equation. Usually, the model is expressed in terms of a likelihood (the probability of consequences given causes) and priors over causes. When a prior depends upon a random variable it is called an empirical prior. Here, the likelihood is specified by a matrix A whose elements are the probability of an outcome under every combination of hidden states. Cat denotes a categorical probability distribution. The empirical priors pertain to probabilistic transitions (in the B matrix) among hidden states that can depend upon actions, which are determined by policies (sequences of actions encoded by π). The key aspect of this generative model is that policies are more probable a priori if they minimize the (time integral of) expected free energy G,which depends upon prior preferences about outcomes or costs encoded in Cand the uncertainty or ambiguity about outcomes under each state, encoded by H. Finally, the vector D specifies the initial state. This completes the specification of the model in terms of parameters that constitute A, B, C, and D. Bayesian model inversion refers to the inverse mapping from consequences to causes; that is, estimating the hidden states and other variables that cause outcomes. In approximate Bayesian inference, one specifies the form of an approximate posterior distribution. This particular form in this paper uses a mean field approximation, in which posterior beliefs are approximated by the product of marginal distributions over time points. Subscripts index time (or policy). See the main text and Table 1a in Friston, Parr, et al. (2017) for a detailed explanation of the variables (italic variables represent hidden states, while bold variables indicate expectations about those states).Upper right panel: This Bayesian network represents the conditional dependencies among hidden states and how they cause outcomes. Open circles are random variables (hidden states and policies), while filled circles denote observable outcomes. Squares indicate fixed or known variables, such as the model parameters. We have used a slightly unusual convention where parameters have been placed on top of the edges (conditional dependencies) that may mediate.Lower left panel: These equalities are the belief updates mediating approximate Bayesian inference and action selection. The (Iverson) brackets in the action selection panel return one if the condition in square brackets is satisfied and zero otherwise.Lower right panel: This is an equivalent representation of the Bayesian network in terms of a Forney or normal style factor graph. Here the nodes (square boxes) correspond to factors and the edges are associated with unknown variables. Filled squares denote observable outcomes. The edges are labeled in terms of the sufficient statistics of their marginal posteriors (see approximate posterior). Factors have been labeled intuitively in terms of the parameters encoding the associated probability distributions (on the upper left). The circled numbers correspond to the messages that are passed from nodes to edges (the labels are placed on the edge that carries the message from each node). These correspond to the messages implicit in the belief updates (on the lower left).
Figure 1. Generative model for discrete states and outcomes.Upper left panel: These equations specify the generative model. A generative model is the joint probability of outcomes or consequences and their (latent or hidden) causes; see first equation. Usually, the model is expressed in terms of a likelihood (the probability of consequences given causes) and priors over causes. When a prior depends upon a random variable it is called an empirical prior. Here, the likelihood is specified by a matrix A whose elements are the probability of an outcome under every combination of hidden states. Cat denotes a categorical probability distribution. The empirical priors pertain to probabilistic transitions (in the B matrix) among hidden states that can depend upon actions, which are determined by policies (sequences of actions encoded by π). The key aspect of this generative model is that policies are more probable a priori if they minimize the (time integral of) expected free energy G,which depends upon prior preferences about outcomes or costs encoded in Cand the uncertainty or ambiguity about outcomes under each state, encoded by H. Finally, the vector D specifies the initial state. This completes the specification of the model in terms of parameters that constitute A, B, C, and D. Bayesian model inversion refers to the inverse mapping from consequences to causes; that is, estimating the hidden states and other variables that cause outcomes. In approximate Bayesian inference, one specifies the form of an approximate posterior distribution. This particular form in this paper uses a mean field approximation, in which posterior beliefs are approximated by the product of marginal distributions over time points. Subscripts index time (or policy). See the main text and Table 1a in Friston, Parr, et al. (2017) for a detailed explanation of the variables (italic variables represent hidden states, while bold variables indicate expectations about those states).Upper right panel: This Bayesian network represents the conditional dependencies among hidden states and how they cause outcomes. Open circles are random variables (hidden states and policies), while filled circles denote observable outcomes. Squares indicate fixed or known variables, such as the model parameters. We have used a slightly unusual convention where parameters have been placed on top of the edges (conditional dependencies) that may mediate.Lower left panel: These equalities are the belief updates mediating approximate Bayesian inference and action selection. The (Iverson) brackets in the action selection panel return one if the condition in square brackets is satisfied and zero otherwise.Lower right panel: This is an equivalent representation of the Bayesian network in terms of a Forney or normal style factor graph. Here the nodes (square boxes) correspond to factors and the edges are associated with unknown variables. Filled squares denote observable outcomes. The edges are labeled in terms of the sufficient statistics of their marginal posteriors (see approximate posterior). Factors have been labeled intuitively in terms of the parameters encoding the associated probability distributions (on the upper left). The circled numbers correspond to the messages that are passed from nodes to edges (the labels are placed on the edge that carries the message from each node). These correspond to the messages implicit in the belief updates (on the lower left).CC BY · PubMed Central
Figure 2. The generative process and model. This figure reproduces the Bayesian network and Forney factor graph of Figure 1. However, here the Bayesian network describes the process generating data, as opposed to the generative model of data. This means that we can link the two graphs to show how the policy half-edge of Figure 1 now couples back to the generative process (by generating an action that determines state transitions). The selected action corresponds to the most probable action under posterior beliefs about action sequences or policies. Here, the message labels have been replaced with little arrows to emphasize the circular causality implicit in active inference: The real world (red box) generates a sequence of outcomes that induce message passing and belief propagation to inform (approximate) posterior beliefs about policies (that also depend upon prior preferences and epistemic value). These policies then determine action, which generate new outcomes as time progresses, thereby closing the action perception cycle.
Figure 2. The generative process and model. This figure reproduces the Bayesian network and Forney factor graph of Figure 1. However, here the Bayesian network describes the process generating data, as opposed to the generative model of data. This means that we can link the two graphs to show how the policy half-edge of Figure 1 now couples back to the generative process (by generating an action that determines state transitions). The selected action corresponds to the most probable action under posterior beliefs about action sequences or policies. Here, the message labels have been replaced with little arrows to emphasize the circular causality implicit in active inference: The real world (red box) generates a sequence of outcomes that induce message passing and belief propagation to inform (approximate) posterior beliefs about policies (that also depend upon prior preferences and epistemic value). These policies then determine action, which generate new outcomes as time progresses, thereby closing the action perception cycle.CC BY · PubMed Central
Figure 3. Belief propagation and intrinsic connectivity. Right panel: The schematic on the right represents the message passing implicit in the differential (update) equations on the left. The expectations have been associated with neuronal populations (colored balls) that are arranged to reproduce known intrinsic (within cortical area) connections. Red connections are excitatory, blue connections are inhibitory, and green connections are modulatory (i.e., involve a multiplication or weighting). Here, edges correspond to intrinsic connections that mediate the message passing on the left. Cyan units correspond to expectations about hidden states and (future) outcomes under each policy, while red states indicate their Bayesian model averages. Pink units correspond to (state and outcome) prediction errors that are averaged to evaluate expected free energy and subsequent policy expectations (in the lower part of the network). This (neural) network formulation of belief updating means that connection strengths correspond to the parameters of the generative model in Figure 1. Only exemplar connections are shown to avoid visual clutter. Furthermore, we have just shown neuronal populations encoding of hidden states under two policies over three time points (i.e., two transitions). Left panel: The differential equations on the left share the same stationary solution as the belief updates in previous figures and can therefore be interpreted as a gradient descent on (marginal) free energy. The equations have been expressed in terms of prediction errors that come in two flavors. The first, state prediction error, scores the difference between the (logarithms of) expected states under each policy and time point—and the corresponding predictions based upon outcomes and the (preceding and subsequent) hidden states. These represent empirical prior and likelihood terms respectively; namely, messages 2, 3, and 4. The prediction error drives depolarization in state units, where the expectation per se is obtained via a softmax operator. The second, outcome prediction error, reports the difference between the (logarithms of) expected outcomes and those predicted under prior beliefs. This prediction error is weighted by the expected outcomes to evaluate the expected free energy G, via message 5. These policy-specific free energies are combined to give the policy expectations via a softmax function, via message 1.
Figure 3. Belief propagation and intrinsic connectivity. Right panel: The schematic on the right represents the message passing implicit in the differential (update) equations on the left. The expectations have been associated with neuronal populations (colored balls) that are arranged to reproduce known intrinsic (within cortical area) connections. Red connections are excitatory, blue connections are inhibitory, and green connections are modulatory (i.e., involve a multiplication or weighting). Here, edges correspond to intrinsic connections that mediate the message passing on the left. Cyan units correspond to expectations about hidden states and (future) outcomes under each policy, while red states indicate their Bayesian model averages. Pink units correspond to (state and outcome) prediction errors that are averaged to evaluate expected free energy and subsequent policy expectations (in the lower part of the network). This (neural) network formulation of belief updating means that connection strengths correspond to the parameters of the generative model in Figure 1. Only exemplar connections are shown to avoid visual clutter. Furthermore, we have just shown neuronal populations encoding of hidden states under two policies over three time points (i.e., two transitions). Left panel: The differential equations on the left share the same stationary solution as the belief updates in previous figures and can therefore be interpreted as a gradient descent on (marginal) free energy. The equations have been expressed in terms of prediction errors that come in two flavors. The first, state prediction error, scores the difference between the (logarithms of) expected states under each policy and time point—and the corresponding predictions based upon outcomes and the (preceding and subsequent) hidden states. These represent empirical prior and likelihood terms respectively; namely, messages 2, 3, and 4. The prediction error drives depolarization in state units, where the expectation per se is obtained via a softmax operator. The second, outcome prediction error, reports the difference between the (logarithms of) expected outcomes and those predicted under prior beliefs. This prediction error is weighted by the expected outcomes to evaluate the expected free energy G, via message 5. These policy-specific free energies are combined to give the policy expectations via a softmax function, via message 1.CC BY · PubMed Central
Figure 4. Deep generative models. This figure provides the Bayesian network and associated Forney factor graph for deep (temporal) generative models, described in terms of probability factors and belief updates on the left. The graphs adopt the same format as Figure 1; however, here the previous model has been extended hierarchically, where (bracketed) superscripts index the hierarchical level. The key aspect of this model is its hierarchical structure that represents sequences of hidden states over time or epochs. In this model, hidden states at higher levels generate the initial states for lower levels—that then unfold to generate a sequence of outcomes; cf. associative chaining (Page & Norris, 1998). Crucially, lower levels cycle over a sequence for each transition of the level above. This is indicated by the subgraphs enclosed in dashed boxes, which are “reused” as higher levels unfold. It is this scheduling that endows the model with deep temporal structure. In relation to the (shallow) models above, the probability distribution over initial states is now conditioned over the state (at the current time) of the level above. Practically, this means that D now becomes a matrix, as opposed to a vector. The messages passed from the corresponding factor node rest on Bayesian model averages that require the expected policies (message 1) and expected states under each policy. The resulting averages are then used to compose descending (message 2) and ascending (message 6) messages that mediate the exchange of empirical priors and posteriors between levels respectively.
Figure 4. Deep generative models. This figure provides the Bayesian network and associated Forney factor graph for deep (temporal) generative models, described in terms of probability factors and belief updates on the left. The graphs adopt the same format as Figure 1; however, here the previous model has been extended hierarchically, where (bracketed) superscripts index the hierarchical level. The key aspect of this model is its hierarchical structure that represents sequences of hidden states over time or epochs. In this model, hidden states at higher levels generate the initial states for lower levels—that then unfold to generate a sequence of outcomes; cf. associative chaining (Page & Norris, 1998). Crucially, lower levels cycle over a sequence for each transition of the level above. This is indicated by the subgraphs enclosed in dashed boxes, which are “reused” as higher levels unfold. It is this scheduling that endows the model with deep temporal structure. In relation to the (shallow) models above, the probability distribution over initial states is now conditioned over the state (at the current time) of the level above. Practically, this means that D now becomes a matrix, as opposed to a vector. The messages passed from the corresponding factor node rest on Bayesian model averages that require the expected policies (message 1) and expected states under each policy. The resulting averages are then used to compose descending (message 2) and ascending (message 6) messages that mediate the exchange of empirical priors and posteriors between levels respectively.CC BY · PubMed Central
Figure 5. A generative model for continuous states. This figure describes a generative model for a continuous state space model in terms of generalized motion, using the same format as Figure 1. Here, outcomes are generated (with a static nonlinear mapping g) from the current state, with some random fluctuations; similarly for higher orders of motion. The motion of hidden states is a function (equation of motion or flow f) of the current state, plus random fluctuations. Higher-order motion is derived in a similar way using generalized equations of motion (in the upper left inset). The corresponding Bayesian network (on the upper right) shows that discrete probability transition matrices are replaced with generalized equations of motion, while the discrete likelihood mapping becomes the static nonlinearity. The corresponding Forney factor graph is shown on the lower right.
Figure 5. A generative model for continuous states. This figure describes a generative model for a continuous state space model in terms of generalized motion, using the same format as Figure 1. Here, outcomes are generated (with a static nonlinear mapping g) from the current state, with some random fluctuations; similarly for higher orders of motion. The motion of hidden states is a function (equation of motion or flow f) of the current state, plus random fluctuations. Higher-order motion is derived in a similar way using generalized equations of motion (in the upper left inset). The corresponding Bayesian network (on the upper right) shows that discrete probability transition matrices are replaced with generalized equations of motion, while the discrete likelihood mapping becomes the static nonlinearity. The corresponding Forney factor graph is shown on the lower right.CC BY · PubMed Central
Figure 6. Active inference with continuous states (and time). This figure uses the same format as Figure 2 to illustrate how action couples to the generative process. As before, action supplements or replaces hidden causes in the generative model—to complete the generative process. In this instance, action minimizes variational free energy directly, as opposed to minimizing expected free energy via inference over policies. For simplicity, we have removed any dependency of the observation on causes. This dependency is reinstated in subsequent figures.
Figure 6. Active inference with continuous states (and time). This figure uses the same format as Figure 2 to illustrate how action couples to the generative process. As before, action supplements or replaces hidden causes in the generative model—to complete the generative process. In this instance, action minimizes variational free energy directly, as opposed to minimizing expected free energy via inference over policies. For simplicity, we have removed any dependency of the observation on causes. This dependency is reinstated in subsequent figures.CC BY · PubMed Central
Figure 7. Canonical microcircuits for predictive coding. This proposal is based upon the anatomy of intrinsic and extrinsic connections described in Bastos et al. (2012). This figure provides the update dynamics for a hierarchical generalization of the generative model in Figure 6, where the outputs of a higher level now become the hidden causes of the level below. In this hierarchical setting, the prediction errors include prediction errors on both hidden causes and states. As with the model for discrete states, the prediction errors have been assigned to granular layers that receive sensory afferents and ascending prediction errors from lower levels in the hierarchy.
Figure 7. Canonical microcircuits for predictive coding. This proposal is based upon the anatomy of intrinsic and extrinsic connections described in Bastos et al. (2012). This figure provides the update dynamics for a hierarchical generalization of the generative model in Figure 6, where the outputs of a higher level now become the hidden causes of the level below. In this hierarchical setting, the prediction errors include prediction errors on both hidden causes and states. As with the model for discrete states, the prediction errors have been assigned to granular layers that receive sensory afferents and ascending prediction errors from lower levels in the hierarchy.CC BY · PubMed Central
Figure 8. Linking discrete and continuous state models. This figure uses the same format as previous figures but combines the discrete Bayesian network in Figure 1 with the continuous Bayesian network from Figure 5. Here, the outcomes of the discrete model are now used to select a particular (noisy generalized) hidden cause that determines the (noisy generalized) motion or flow of hidden states generating (noisy generalized) observations. These generalized observations described a trajectory in continuous time.
Figure 8. Linking discrete and continuous state models. This figure uses the same format as previous figures but combines the discrete Bayesian network in Figure 1 with the continuous Bayesian network from Figure 5. Here, the outcomes of the discrete model are now used to select a particular (noisy generalized) hidden cause that determines the (noisy generalized) motion or flow of hidden states generating (noisy generalized) observations. These generalized observations described a trajectory in continuous time.CC BY · PubMed Central
Figure 9. Mixed message passing. This figure combines the Forney factor graphs from Figures 1 and 5 to create a message passing scheme that integrates discrete and continuous models. The key aspect of this integration is the role of a link node (E) associated with the distribution over discrete outcomes—that can also be considered as (outcome) models from the perspective of the (subordinate) continuous level. The consequent message passing means that descending signals from the discrete to continuous levels comprise empirical prior beliefs about the dynamics below, while ascending messages constitute the posterior beliefs over the same set of outcome models. The corresponding messages are shown with little arrows that are reproduced alongside the link functions in the left panels. These linking updates entail two sorts of (Bayesian model) averages. First, expected outcomes under each policy are averaged over policies. Second, prior expectations about hidden causes are averaged over outcome models. These constitute the descending empirical priors. Conversely, ascending messages correspond to the posterior over outcomes models based upon a post hoc Bayesian model comparison. This treats each discrete hidden cause as a different model. The expressions in the lower left inset define the log likelihood of the continuous observations under each outcome model, relative to the log likelihood under their Bayesian model average.
Figure 9. Mixed message passing. This figure combines the Forney factor graphs from Figures 1 and 5 to create a message passing scheme that integrates discrete and continuous models. The key aspect of this integration is the role of a link node (E) associated with the distribution over discrete outcomes—that can also be considered as (outcome) models from the perspective of the (subordinate) continuous level. The consequent message passing means that descending signals from the discrete to continuous levels comprise empirical prior beliefs about the dynamics below, while ascending messages constitute the posterior beliefs over the same set of outcome models. The corresponding messages are shown with little arrows that are reproduced alongside the link functions in the left panels. These linking updates entail two sorts of (Bayesian model) averages. First, expected outcomes under each policy are averaged over policies. Second, prior expectations about hidden causes are averaged over outcome models. These constitute the descending empirical priors. Conversely, ascending messages correspond to the posterior over outcomes models based upon a post hoc Bayesian model comparison. This treats each discrete hidden cause as a different model. The expressions in the lower left inset define the log likelihood of the continuous observations under each outcome model, relative to the log likelihood under their Bayesian model average.CC BY · PubMed Central
Figure 10. Deep architectures in the brain. This schematic illustrates a putative mapping of belief updating onto recurrent interactions within the cortico (basal ganglia) thalamic loops. This neural architecture is based upon the functional neuroanatomy described in Jahanshahi, Obeso, Rothwell, & Obeso (2015), which assigns motor updates to motor and premotor cortex projecting to the putamen, associative loops to prefrontal cortical projections to the caudate, and limbic loops to projections to the ventral striatum. The striatum (caudate and putamen) receive inputs from several cortical and subcortical areas. The internal segment of the globus pallidus constitutes the main output nucleus from the basal ganglia. The basal ganglia are connected to motor areas (motor cortex, supplementary motor cortex, premotor cortex, cingulate motor area, and frontal eye fields) and associative cortical areas. The basal ganglia nuclei have similar (topologically organized) motor, associative, and limbic territories; the posterior putamen is engaged in sensorimotor function, while the anterior putamen (or caudate) and the ventral striatum are involved in associative (cognitive) and limbic (motivation and emotion) functions (Jahanshahi et al., 2015). Here, ascending and descending messages among discrete levels are passed between (Bayesian model) averages of expected hidden states (that have been duplicated in this figure to account for the known laminar specificity of extrinsic cortico-cortical connections). We have placed the outcome prediction errors in deep layers (Layer 5 pyramidal cells) that project to the medium spiny cells of the striatum (Arikuni & Kubota, 1986). These outcome prediction errors are used to compute expected free energy and consequent expectations about policies. Policy expectations are then used to form (Bayesian model) averages of hidden states that are necessary for message passing between discrete hierarchical levels. Similarly, expected outcomes under each policy are passed via corticothalamic connections to the thalamus to evaluate the free energy of each dynamic model, in conjunction with posterior expectations from the continuous level. The resulting posterior expectations are conveyed by thalamocortical projections to inform discrete (hierarchical) message passing. The little arrows referred to the corresponding messages in Figure 9.
Figure 10. Deep architectures in the brain. This schematic illustrates a putative mapping of belief updating onto recurrent interactions within the cortico (basal ganglia) thalamic loops. This neural architecture is based upon the functional neuroanatomy described in Jahanshahi, Obeso, Rothwell, & Obeso (2015), which assigns motor updates to motor and premotor cortex projecting to the putamen, associative loops to prefrontal cortical projections to the caudate, and limbic loops to projections to the ventral striatum. The striatum (caudate and putamen) receive inputs from several cortical and subcortical areas. The internal segment of the globus pallidus constitutes the main output nucleus from the basal ganglia. The basal ganglia are connected to motor areas (motor cortex, supplementary motor cortex, premotor cortex, cingulate motor area, and frontal eye fields) and associative cortical areas. The basal ganglia nuclei have similar (topologically organized) motor, associative, and limbic territories; the posterior putamen is engaged in sensorimotor function, while the anterior putamen (or caudate) and the ventral striatum are involved in associative (cognitive) and limbic (motivation and emotion) functions (Jahanshahi et al., 2015). Here, ascending and descending messages among discrete levels are passed between (Bayesian model) averages of expected hidden states (that have been duplicated in this figure to account for the known laminar specificity of extrinsic cortico-cortical connections). We have placed the outcome prediction errors in deep layers (Layer 5 pyramidal cells) that project to the medium spiny cells of the striatum (Arikuni & Kubota, 1986). These outcome prediction errors are used to compute expected free energy and consequent expectations about policies. Policy expectations are then used to form (Bayesian model) averages of hidden states that are necessary for message passing between discrete hierarchical levels. Similarly, expected outcomes under each policy are passed via corticothalamic connections to the thalamus to evaluate the free energy of each dynamic model, in conjunction with posterior expectations from the continuous level. The resulting posterior expectations are conveyed by thalamocortical projections to inform discrete (hierarchical) message passing. The little arrows referred to the corresponding messages in Figure 9.CC BY · PubMed Central
Figure 11. A generative model of pictographic reading. In this model there are two discrete hierarchical levels with two sorts of hidden states at the second level and four at the first level. The hidden states at the higher level correspond to the sentence or narrative—generating sequences of words at the first level—and which wordthe agent is currently sampling (with six alternative sentences and four words respectively). These hidden states combine to specify the word at the first level (flee, feed, or wait). The hidden states at the first level comprise the current word and which quadrant the agent is looking at. These hidden states combine to generate outcomes in terms of letters or pictograms that would be seen at that location. In addition, two further hidden states flip the relative locations vertically or horizontally. The vertical flip can be thought of in terms of font substitution (uppercase versus lowercase), while the horizontal flip means a word is invariant under changes to the order of the letters (cf. palindromes). In this example, flee means that a bird is next to a cat, feed means a bird is next to some seeds, and wait means seeds are above (or below) the bird. Notice that there is a (proprioceptive) outcome signaling the word currently being sampled (e.g., head position), while at the lower level there are two discrete outcome modalities. The first (exteroceptive) outcome corresponds to the observed letter and the second (proprioceptive) outcome specifies a point of visual fixation(e.g., in a head-centered frame of reference). Similarly, there are policies at both levels. The high-level policy determines which word the agent is currently reading, while the lower level dictates eye movements among the quadrants containing letters. These discrete outcomes (the pictogram, what, and target location, where) generate continuous visuomotor signals as follows: the target location (specified by the discrete where outcome) is the center of the corresponding quadrant (denoted by L in the figure). This point of fixation attracts the current center of gaze (in the generative model) that is enacted by action (in the generative process), where action simply moves the eye horizontally or vertically. At every point in time, the visual outcome is sampled from an image (with 32 × 32 pixels), specified by the discrete what outcome. This sampling is eccentric, based upon the displacement between the target location and the current center of gaze (denoted by d in the figure). Finally, the image contrast is attenuated as a Gaussian function of displacement to emulate sensory attenuation. In short, the continuous state space model has two hidden causes target location and identity (denoted by vL and vI) and a single hidden state (x), corresponding to the current center gaze.
Figure 11. A generative model of pictographic reading. In this model there are two discrete hierarchical levels with two sorts of hidden states at the second level and four at the first level. The hidden states at the higher level correspond to the sentence or narrative—generating sequences of words at the first level—and which wordthe agent is currently sampling (with six alternative sentences and four words respectively). These hidden states combine to specify the word at the first level (flee, feed, or wait). The hidden states at the first level comprise the current word and which quadrant the agent is looking at. These hidden states combine to generate outcomes in terms of letters or pictograms that would be seen at that location. In addition, two further hidden states flip the relative locations vertically or horizontally. The vertical flip can be thought of in terms of font substitution (uppercase versus lowercase), while the horizontal flip means a word is invariant under changes to the order of the letters (cf. palindromes). In this example, flee means that a bird is next to a cat, feed means a bird is next to some seeds, and wait means seeds are above (or below) the bird. Notice that there is a (proprioceptive) outcome signaling the word currently being sampled (e.g., head position), while at the lower level there are two discrete outcome modalities. The first (exteroceptive) outcome corresponds to the observed letter and the second (proprioceptive) outcome specifies a point of visual fixation(e.g., in a head-centered frame of reference). Similarly, there are policies at both levels. The high-level policy determines which word the agent is currently reading, while the lower level dictates eye movements among the quadrants containing letters. These discrete outcomes (the pictogram, what, and target location, where) generate continuous visuomotor signals as follows: the target location (specified by the discrete where outcome) is the center of the corresponding quadrant (denoted by L in the figure). This point of fixation attracts the current center of gaze (in the generative model) that is enacted by action (in the generative process), where action simply moves the eye horizontally or vertically. At every point in time, the visual outcome is sampled from an image (with 32 × 32 pixels), specified by the discrete what outcome. This sampling is eccentric, based upon the displacement between the target location and the current center of gaze (denoted by d in the figure). Finally, the image contrast is attenuated as a Gaussian function of displacement to emulate sensory attenuation. In short, the continuous state space model has two hidden causes target location and identity (denoted by vL and vI) and a single hidden state (x), corresponding to the current center gaze.CC BY · PubMed Central
Figure 12. Simulated reading. This figure shows the trajectory of eye movements over four transitions at the second level that entail one or two saccadic eye movements at the first.
Figure 12. Simulated reading. This figure shows the trajectory of eye movements over four transitions at the second level that entail one or two saccadic eye movements at the first.CC BY · PubMed Central
Figure 13. Visual sampling. Upper left: Final location of a saccadic eye movement (from the center of the visual field) indicated by a red dot. The green dot is where the subject thought she was looking. The crosshairs demark the field of view. Middle left: This plot shows action that corresponds to the motion (in two dimensions) of the hidden state describing the current center of gaze. This trajectory is shown as a function of the time steps used to integrate the generalized belief updating (24 time steps, each corresponding to about 10 ms of real time). Upper right: These panels showed the visual samples every five time steps. Note that the luminance contrast increases as the center of gaze approaches the target location. Lower panels: These panels illustrate evidence accumulation during the (continuous) saccadic sampling in terms of the posterior probability over (discrete) visual (lower left) and proprioceptive (lower right) outcomes.
Figure 13. Visual sampling. Upper left: Final location of a saccadic eye movement (from the center of the visual field) indicated by a red dot. The green dot is where the subject thought she was looking. The crosshairs demark the field of view. Middle left: This plot shows action that corresponds to the motion (in two dimensions) of the hidden state describing the current center of gaze. This trajectory is shown as a function of the time steps used to integrate the generalized belief updating (24 time steps, each corresponding to about 10 ms of real time). Upper right: These panels showed the visual samples every five time steps. Note that the luminance contrast increases as the center of gaze approaches the target location. Lower panels: These panels illustrate evidence accumulation during the (continuous) saccadic sampling in terms of the posterior probability over (discrete) visual (lower left) and proprioceptive (lower right) outcomes.CC BY · PubMed Central
Figure 14. Simulated electrophysiological responses during reading. This figure illustrates belief propagation (producing the behavior shown in Figure 11), in terms of simulated firing rates and depolarization. Expectations about the initial hidden state (at the first time step) at the higher (upper panel) and lower (middle panel) discrete levels are presented in raster format. The horizontal axis is time over the entire simulation, corresponding roughly to 4.5 seconds. Firing rates are shown for populations encoding Bayesian model averages of the six sentences at the higher level and the three words at the lower level. A firing rate of one corresponds to black. The transients in the third panel are the firing rates in the upper panels filtered between 4 Hz and 32 Hz—and can be regarded as (band-pass filtered) fluctuations in depolarization. Saccade onsets are shown with the vertical (dashed) lines. The lower panel reproduces the eye movement trajectories of Figure 12.
Figure 14. Simulated electrophysiological responses during reading. This figure illustrates belief propagation (producing the behavior shown in Figure 11), in terms of simulated firing rates and depolarization. Expectations about the initial hidden state (at the first time step) at the higher (upper panel) and lower (middle panel) discrete levels are presented in raster format. The horizontal axis is time over the entire simulation, corresponding roughly to 4.5 seconds. Firing rates are shown for populations encoding Bayesian model averages of the six sentences at the higher level and the three words at the lower level. A firing rate of one corresponds to black. The transients in the third panel are the firing rates in the upper panels filtered between 4 Hz and 32 Hz—and can be regarded as (band-pass filtered) fluctuations in depolarization. Saccade onsets are shown with the vertical (dashed) lines. The lower panel reproduces the eye movement trajectories of Figure 12.CC BY · PubMed Central

Verification

Claims the paper makes

0 Valid · 1 Not valid · 45 Not checked · 146 Own results and interpretations

112 more claims

Create a free account to see all 192 claims and to verify the 73 references still open.

See all claims — free account
  • Probably not a substantial claim. 39
  • It may state two separate claims as one. 42

Create a free account to see these claims.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

Each contradiction between two passages listed here was confirmed by an independent check.

3 of 3

Citations

Cites

Research works printed but not linked: 4

  1. Beal M. J. (2003). Variational algorithms for approximate Bayesian inference (Doctoral thesis, University College London).
  2. Friston K. J., Parr T., & de Vries B. (2017). Appendices for “The graphical brain: Belief propagation and active inference.” Network Neuroscience. 10.1162/netn_a_00018 PMC5798592 29417960
  3. Loeliger H.-A. (2002). Least squares and Kalman filtering on Forney graphs. In Blahut R. E. & Koetter R. (Eds.), Codes, graphs, and systems: A celebration of the life and career of G. David Forney, Jr., on the occasion of his sixtieth birthday (pp. 113– 135). Boston, MA: Springer US.
  4. MacKay D. J. C. (2003). Information theory inference and learning algorithms. Cambridge: Cambridge University Press.

Cited by

See all 415

Metadata

Authors
Friston, K. J., Parr, T., & de Vries, B.
Affiliations
  • Wellcome Centre for Human Neuroimaging (GB)
  • Eindhoven University of Technology, Department of Electrical Engineering, Eindhoven, The Netherlands
Corresponding author
Parr, Thomas
Venue
Network Neuroscience
Publisher
MIT Press
Published
2017-12
Volume
1
Issue
4
Pages
381-414
ISSN
2472-1751 (ISSN-L)
OpenAlex
W2623066203
Cited by
415
References
111
Citations by year
2026: 49 · 2025: 39 · 2024: 41 · 2023: 61 · 2022: 51 · 2021: 56 · 2020: 55 · 2019: 32 · 2018: 25 · 2017: 6
FWCI
18.51
Type
journal article
Language
en
Open access
Licence
cc-by
Locations
  • Network Neuroscience (publishedVersion, cc-by)
  • PubMed Central (publishedVersion, cc-by)
  • PubMed (publishedVersion)
  • TU/e Research Portal (submittedVersion, other-oa)
  • TU/e Research Portal (submittedVersion)
  • DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion)
  • DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
  • UCL Discovery (University College London) (submittedVersion)
  • TU/e Research Portal (submittedVersion, cc-by)
Funders
  • Wellcome Trust (088130/Z/09/Z)
  • Rosetrees Trust (173346)
Topics
  • Life Sciences › Neuroscience › Cognitive Neuroscience › Embodied and Extended Cognition
  • Physical Sciences › Computer Science › Artificial Intelligence › Bayesian Modeling and Causal Inference
  • Life Sciences › Neuroscience › Cognitive Neuroscience › Neural dynamics and brain function
Keywords
active inference, belief propagation, deep generative models, factor graphs, Bayesian model averaging, thalamocortical loop, functional integration, reading, semantic representation, visual sampling

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

A language model drew the claims from the paper on 2026-10-07, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).

A second model labelled each claim on 2026-10-07 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.