Active Inference, Belief Propagation, and the Bethe Approximation
The DOI matches its Crossref record; 35 of the 35 references it cites were found in a registry or an index.
Abstract
When modeling goal-directed behavior in the presence of various sources of uncertainty, planning can be described as an inference process. A solution to the problem of planning as inference was previously proposed in the active inference framework in the form of an approximate inference scheme based on variational free energy. However, this approximate scheme was based on the mean-field approximation, which assumes statistical independence of hidden variables and is known to show overconfidence and may converge to local minima of the free energy. To better capture the spatiotemporal properties of an environment, we reformulated the approximate inference process using the so-called Bethe approximation. Importantly, the Bethe approximation allows for representation of pairwise statistical dependencies. Under these assumptions, the minimizer of the variational free energy corresponds to the belief propagation algorithm, commonly used in machine learning. To illustrate the differences between the mean-field approximation and the Bethe approximation, we have simulated agent behavior in a simple goal-reaching task with different types of uncertainties. Overall, the Bethe agent achieves higher success rates in reaching goal states. We relate the better performance of the Bethe agent to more accurate predictions about the consequences of its own actions. Consequently, active inference based on the Bethe approximation extends the application range of active inference to more complex behavioral tasks.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
The free-energy principle: a unified brain theory?
Friston, K. (2010) · Nature Reviews Neuroscience · doi:10.1038/nrn2787 · Cited by 8,120 · cited 3×
“As the variational free energy defines an upper bound on surprise (Bishop, 2006; Friston, 2010), minimizing the free energy minimizes an agent’s surprise about its sensations.”
BackgroundMethodZur Theorie der Metalle: I. Eigenwerte und Eigenfunktionen der linearen Atomkette
Bethe, H. (1931) · Zeitschrift f�r Physik · doi:10.1007/bf01341708 · Cited by 3,768 · cited 1×
“Impor-tantly, we will base the derivations on the so-called Bethe approximation (Bethe, 1931, 1935), which will allow us to establish a formal link between the free energy principle and the set of update equations known as belief propagation (Friston, Parr, & de Vries, 2017; Yedidia, Freeman, &…”
MethodInfluentialHuman-level concept learning through probabilistic program induction
Lake, B. M., Salakhutdinov, R., & Tenenbaum, J. B. (2015) · Science · doi:10.1126/science.aab3050 · Cited by 2,817
The Bayesian brain: The role of uncertainty in neural coding and computation
Knill, D. C., & Pouget, A. (2004) · Trends in Neurosciences · doi:10.1016/j.tins.2004.10.007 · Cited by 2,588 · cited 1×
“This suggests that humans actively use a representation of uncertainty when inferring the current and past states of the world and when making decisions (Friston & Kiebel, 2009; Lee & Mum-ford, 2003; Knill & Pouget, 2004; Dayan, Hinton, Neal, & Zemel, 1995).”
BackgroundDaw, N. D., Niv, Y., & Dayan, P. (2005) · Nature Neuroscience · doi:10.1038/nn1560 · Cited by 2,533 · cited 1×
“…Bayesian agent (FitzGerald, Hämmerer, Friston, Li, & Dolan, 2017; Meyniel, Schlunegger, & Dehaene, 2015; Lake, Salakhut-dinov, & Tenenbaum, 2015; Vossel et al., 2013; Payzan-LeNestour, Dunne, Bossaerts, & O’Doherty, 2013; Behrens, Hunt, Woolrich, & Rushworth, 2008; Daw, Niv, & Dayan, 2005).”
Dissociable Roles of Ventral and Dorsal Striatum in Instrumental Conditioning
O'Doherty, J., Dayan, P., Schultz, J., Deichmann, R., Friston, K., & Dolan, R. J. (2004) · Science · doi:10.1126/science.1094285 · Cited by 2,088 · cited 1×
“…for goal-directed behavior are typically based on the widely used computational framework of reinforcement learning (Sutton & Barto, 1998) with a large number of applications (Doll, Simon, & Daw, 2012; Rangel & Hare, 2010; Dayan & Niv, 2008; O’Doherty et al., 2004; Montague, Hyman, & Cohen, 2004).”
MethodLearning the value of information in an uncertain world
Behrens, T. E. J., Woolrich, M. W., Walton, M. E., & Rushworth, M. F. S. (2007) · Nature Neuroscience · doi:10.1038/nn1954 · Cited by 2,053
Uncertainty, Neuromodulation, and Attention
Yu, A. J., & Dayan, P. (2005) · Neuron · doi:10.1016/j.neuron.2005.04.026 · Cited by 1,774 · cited 1×
“However, a strong limitation of classical reinforcement learning models is that they do not take into account the influence of various sources of uncertainty on human behavior (Rushworth & Behrens, 2008; Doya, 2008; Behrens, Woolrich, Walton, & Rushworth, 2007; Yu & Dayan, 2005).”
BackgroundPredictive coding under the free-energy principle
Friston, K., & Kiebel, S. (2009) · Philosophical Transactions of the Royal Society B: Biological Sciences · doi:10.1098/rstb.2008.0300 · Cited by 1,579
Self-citationHierarchical Bayesian inference in the visual cortex
Lee, T. S., & Mumford, D. (2003) · Journal of the Optical Society of America A · doi:10.1364/josaa.20.001434 · Cited by 1,473 · cited 1×
“Interestingly, following this line of work, Lee and Mumford (2003) and Jardri and Denève (2013) discussed a possible link between belief propagation in cortical networks and optical illusions and hallucinations.”
BackgroundActive inference: a process theory
Friston, K., FitzGerald, T., Rigoli, F., Schwartenbeck, P., & Pezzulo, G. (2017) · Neural Computation · doi:10.1162/neco_a_00912 · Cited by 1,156 · cited 7×
“Additionally, learning principles have recently been introduced to active inference (Friston, FitzGerald, Rigoli, Schwartenbeck, & Pezzulo, 2016; Friston, FitzGerald, Rigoli, Schwartenbeck, O’Doherty et al., 2016), which could easily be combined with the Bethe approximation.”
BackgroundMethodInfluentialComputational roles for dopamine in behavioural control
Montague, P. R., Hyman, S. E., & Cohen, J. D. (2004) · Nature · doi:10.1038/nature03015 · Cited by 966
Associative learning of social value
Behrens, T. E. J., Hunt, L. T., Woolrich, M. W., & Rushworth, M. F. S. (2008) · Nature · doi:10.1038/nature07538 · Cited by 925 · cited 1×
“…Bayesian agent (FitzGerald, Hämmerer, Friston, Li, & Dolan, 2017; Meyniel, Schlunegger, & Dehaene, 2015; Lake, Salakhut-dinov, & Tenenbaum, 2015; Vossel et al., 2013; Payzan-LeNestour, Dunne, Bossaerts, & O’Doherty, 2013; Behrens, Hunt, Woolrich, & Rushworth, 2008; Daw, Niv, & Dayan, 2005).”
Action and behavior: a free-energy formulation
Friston, K. J., Daunizeau, J., Kilner, J., & Kiebel, S. J. (2010) · Biological Cybernetics · doi:10.1007/s00422-010-0364-z · Cited by 866 · cited 4×
“For this reason, the concept of describing planning as an inference process has found increasing interest within the cognitive neuroscience community (Botvinick & Toussaint, 2012; Solway & Botvinick, 2012; Friston, Daunizeau, Kilner, & Kiebel, 2010).”
BackgroundMethodInfluentialSelf-citationChoice, uncertainty and value in prefrontal and cingulate cortex
Rushworth, M. F. S., & Behrens, T. E. J. (2008) · Nature Neuroscience · doi:10.1038/nn2066 · Cited by 866
Optimal control of Markov processes with incomplete state information
Åström, K. J. (1965) · Journal of Mathematical Analysis and Applications · doi:10.1016/0022-247x(65)90154-x · Cited by 854 · cited 1×
“Here we model this decision process as a partially observable Markov decision process (Drake, 1962; Martin, 1967; Astrom, 1965; Monahan, 1982).”
MethodVision as Bayesian inference: analysis by synthesis?
Yuille, A., & Kersten, D. (2006) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2006.05.002 · Cited by 849
Active inference and epistemic value
Friston, K., Rigoli, F., Ognibene, D., Mathys, C., Fitzgerald, T., & Pezzulo, G. (2015) · Cognitive Neuroscience · doi:10.1080/17588928.2015.1020053 · Cited by 731
Friston, K., FitzGerald, T., Rigoli, F., Schwartenbeck, P., ODoherty, J., & Pezzulo, G. (2016) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2016.06.022 · Cited by 674
A Bayesian foundation for individual learning under uncertainty
Mathys, C. (2011) · Frontiers in Human Neuroscience · doi:10.3389/fnhum.2011.00039 · Cited by 655
Doya, K. (2008) · Nature Neuroscience · doi:10.1038/nn2077 · Cited by 642 · cited 1×
“However, a strong limitation of classical reinforcement learning models is that they do not take into account the influence of various sources of uncertainty on human behavior (Rushworth & Behrens, 2008; Doya, 2008; Behrens, Woolrich, Walton, & Rushworth, 2007; Yu & Dayan, 2005).”
BackgroundNeural computations associated with goal-directed choice
Rangel, A., & Hare, T. (2010) · Current Opinion in Neurobiology · doi:10.1016/j.conb.2010.03.001 · Cited by 532
The graphical brain: Belief propagation and active inference
Friston, K. J., Parr, T., & de Vries, B. (2017) · Network Neuroscience · doi:10.1162/netn_a_00018 · Cited by 415 · cited 2×
“…the derivations on the so-called Bethe approximation (Bethe, 1931, 1935), which will allow us to establish a formal link between the free energy principle and the set of update equations known as belief propagation (Friston, Parr, & de Vries, 2017; Yedidia, Freeman, & Weiss, 2005; Pearl, 1988).”
BackgroundTowards a mathematical theory of cortical micro-circuits
George, D., & Hawkins, J. (2009) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1000532 · Cited by 404 · cited 1×
“It might be necessary to have an extra neuronal pool to calculate the messages (George & Hawkins, 2009; Steimer, Maass, & Douglas, 2009).”
BackgroundThe ubiquity of model-based reinforcement learning
Doll, B. B., Simon, D. A., & Daw, N. D. (2012) · Current Opinion in Neurobiology · doi:10.1016/j.conb.2012.08.003 · Cited by 380
The anatomy of choice: dopamine and decision-making.
Friston, K., Schwartenbeck, P., FitzGerald, T., Moutoussis, M., Behrens, T., & Dolan, R. J. (2014) · Philosophical Transactions of the Royal Society B: Biological Sciences · doi:10.1098/rstb.2013.0481 · Cited by 289
The neural representation of unexpected uncertainty during value-based decision making
Payzan-LeNestour, E., Dunne, S., Bossaerts, P., & O’Doherty, J. P. (2013) · Neuron · doi:10.1016/j.neuron.2013.04.037 · Cited by 257
An efficient forward-backward algorithm for an explicit-duration hidden Markov model
Yu, S.-Z., & Kobayashi, H. (2003) · IEEE Signal Processing Letters · doi:10.1109/lsp.2002.806705 · Cited by 226 · cited 1×
“For this reason, the belief propagation algorithm is often applied in the machine learning field to sequential inference problems (Bishop, 2006; Yedidia et al., 2005; Yu & Kobayashi, 2003; Fan, 2001; Rabiner, 1989; Gelb, 1974; Kalman, 1960).”
MethodThe dopaminergic midbrain encodes the expected certainty about desired outcomes
Schwartenbeck, P., FitzGerald, T. H. B., Mathys, C., Dolan, R., & Friston, K. (2015) · Cerebral Cortex · doi:10.1093/cercor/bhu159 · Cited by 219
Circular inferences in schizophrenia
Jardri, R., & Denève, S. (2013) · Brain · doi:10.1093/brain/awt257 · Cited by 201 · cited 1×
“Interestingly, following this line of work, Lee and Mumford (2003) and Jardri and Denève (2013) discussed a possible link between belief propagation in cortical networks and optical illusions and hallucinations.”
BackgroundObserving the observer (I): meta-bayesian models of learning and decision-making
Daunizeau, J., den Ouden, H. E. M., Pessiglione, M., Kiebel, S. J., Stephan, K. E., & Friston, K. J. (2010) · PLoS ONE · doi:10.1371/journal.pone.0015554 · Cited by 167 · cited 2×
“…making, one typically postulates that the human brain uses an approximate inference scheme to update beliefs and generate plans (Mathys, Daunizeau, Friston, & Stephan, 2011; Nassar, Wilson, Heasly, & Gold, 2010; Daunizeau et al., 2010; Yuille & Kersten, 2006; Baker, Saxe, & Tenenbaum, 2005).”
BackgroundSelf-citationThe Sense of Confidence during Probabilistic Learning: A Normative Account
Meyniel, F., Schlunegger, D., & Dehaene, S. (2015) · PLOS Computational Biology · doi:10.1371/journal.pcbi.1004305 · Cited by 163 · cited 1×
“…evidence that belief updating in humans closely follows that of a rational Bayesian agent (FitzGerald, Hämmerer, Friston, Li, & Dolan, 2017; Meyniel, Schlunegger, & Dehaene, 2015; Lake, Salakhut-dinov, & Tenenbaum, 2015; Vossel et al., 2013; Payzan-LeNestour, Dunne, Bossaerts, & O’Doherty,…”
Spatial attention, precision, and Bayesian inference: a study of saccadic response speed
Vossel, S., Mathys, C., Daunizeau, J., Bauer, M., Driver, J., Friston, K. J., & Stephan, K. E. (2014) · Cerebral Cortex · doi:10.1093/cercor/bhs418 · Cited by 146 · cited 1×
“…Bayesian agent (FitzGerald, Hämmerer, Friston, Li, & Dolan, 2017; Meyniel, Schlunegger, & Dehaene, 2015; Lake, Salakhut-dinov, & Tenenbaum, 2015; Vossel et al., 2013; Payzan-LeNestour, Dunne, Bossaerts, & O’Doherty, 2013; Behrens, Hunt, Woolrich, & Rushworth, 2008; Daw, Niv, & Dayan, 2005).”
Model averaging, optimal inference, and habit formation
FitzGerald, T. H. B., Dolan, R. J., & Friston, K. J. (2014) · Frontiers in Human Neuroscience · doi:10.3389/fnhum.2014.00457 · Cited by 121
Evidence for surprise minimization over value maximization in choice behavior
Schwartenbeck, P., FitzGerald, T. H. B., Mathys, C., Dolan, R., Kronbichler, M., & Friston, K. (2015) · Scientific Reports · doi:10.1038/srep16575 · Cited by 96 · cited 3×
“…uncertainty condition, showing that agents do not intrinsically value informative states in our formulation using the predictive free energy, in contrast to previous formulations of active inference (Friston, FitzGerald, Rigoli, Schwartenbeck, & Pezzulo, 2016; Schwartenbeck et al., 2015).”
Method
Cited by
Active inference on discrete state-spaces: A synthesis
Da Costa, L., Parr, T., Sajid, N., Veselic, S., Neacsu, V., & Friston, K. (2020) · Journal of Mathematical Psychology · doi:10.1016/j.jmp.2020.102447 · Cited by 226
“…model is formulated, evidence for a plausible but distinct implementations of active inference would need to be compared, which come from various possible approximations to the free energy [84, 85, 117], each of which yields different belief updates and simulated electrophysiological responses.”
BackgroundMethodTheory Is All You Need: AI, Human Cognition, and Causal Reasoning
Felin, T., & Holweg, M. (2024) · doi:10.2139/ssrn.4737265 · Cited by 152
Theory Is All You Need: AI, Human Cognition, and Causal Reasoning
(2024) · doi:10.1287/stsc.2024.0189 · Cited by 151
Neuronal message passing using mean-field, bethe, and marginal approximations
Parr, T., Markovic, D., Kiebel, S. J., & Friston, K. J. (2019) · Scientific Reports · doi:10.1038/s41598-018-38246-3 · Cited by 128
Self-citationNeural dynamics under active inference: plausibility and efficiency of information processing
Da Costa, L., Parr, T., Sengupta, B., & Friston, K. (2021) · Entropy · doi:10.3390/e23040454 · Cited by 38
“Once the appropriate generative model is found, one would need to compare the evidence for a few possible practical implementations of active inference, which come from various possible approximations to the free energy [27,124,125], each of which yields different belief updates and simulated electrophysiological resp…”
Background
Metadata
- Authors
- Schwöbel, S., Kiebel, S., & Marković, D.
- Venue
- Neural Computation
- Publisher
- MIT Press
- Published
- 2018-09
- Volume
- 30
- Issue
- 9
- Pages
- 2530-2567
- ISSN
- 0899-7667 (ISSN-L)
- 1530-888X
- OpenAlex
- W2811289501
- Cited by
- 36
- References
- 35
- FWCI
- 2.77
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.