Free Energy, Value, and Attractors
Open accessThe DOI matches its Crossref record; 26 of the 26 references it cites were found in a registry or an index.
Abstract
It has been suggested recently that action and perception can be understood as minimising the free energy of sensory samples. This ensures that agents sample the environment to maximise the evidence for their model of the world, such that exchanges with the environment are predictable and adaptive. However, the free energy account does not invoke reward or cost-functions from reinforcement-learning and optimal control theory. We therefore ask whether reward is necessary to explain adaptive behaviour. The free energy formulation uses ideas from statistical physics to explain action in terms of minimising sensory surprise. Conversely, reinforcement-learning has its roots in behaviourism and engineering and assumes that agents optimise a policy to maximise future reward. This paper tries to connect the two formulations and concludes that optimal policies correspond to empirical priors on the trajectories of hidden environmental states, which compel agents to seek out the (valuable) states they expect to encounter.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
On Information and Sufficiency
Kullback, S., & Leibler, R. A. (1951) · The Annals of Mathematical Statistics · doi:10.1214/aoms/1177729694 · Cited by 20,193
A Stochastic Approximation Method
Robbins, H., & Monro, S. (1951) · The Annals of Mathematical Statistics · doi:10.1214/aoms/1177729586 · Cited by 9,643
Watkins, C. J. C. H., & Dayan, P. (1992) · Machine Learning · doi:10.1007/bf00992698 · Cited by 9,083
Discrete Choice Methods with Simulation
Train, K. E. (2001) · doi:10.1017/cbo9780511805271 · Cited by 7,784
Rao, R. P. N., & Ballard, D. H. (1999) · Nature Neuroscience · doi:10.1038/4580 · Cited by 5,483 · cited 1×
“This particular free energy minimisation scheme is known as generalised predictive coding and has become a useful metaphor for neuronal message passing in the brain: see also [12].”
BackgroundThe Bayesian brain: The role of uncertainty in neural coding and computation
Knill, D. C., & Pouget, A. (2004) · Trends in Neurosciences · doi:10.1016/j.tins.2004.10.007 · Cited by 2,588 · cited 1×
“This is the basis of perceptual inference and learning as articulated by the Bayesian brain hypothesis [10, 13, 48– 52].”
MethodHierarchical Bayesian inference in the visual cortex
Lee, T. S., & Mumford, D. (2003) · Journal of the Optical Society of America A · doi:10.1364/josaa.20.001434 · Cited by 1,473 · cited 1×
“This paper tries to connect the two formulations and concludes that optimal policies correspond to empirical priors on the trajectories of hidden environmental states, which compel agents to seek out the (valuable) states they expect to encounter.”
BackgroundToward a modern theory of adaptive networks: Expectation and prediction.
Sutton, R. S., & Barto, A. G. (1981) · Psychological Review · doi:10.1037/0033-295x.88.2.135 · Cited by 1,450 · cited 2×
“These policies are considered in reinforcement-learning and optimal control theory [2, 4, 14].”
BackgroundMethodA free energy principle for the brain
Friston, K., Kilner, J., & Harrison, L. (2006) · Journal of Physiology-Paris · doi:10.1016/j.jphysparis.2006.10.001 · Cited by 1,369
Self-citationObject perception as Bayesian inference
Kersten, D., Mamassian, P., & Yuille, A. (2004) · Annual Review of Psychology · doi:10.1146/annurev.psych.55.090902.142005 · Cited by 1,269
Cohen, J. D., McClure, S. M., & Yu, A. J. (2007) · Philosophical Transactions of the Royal Society B Biological Sciences · doi:10.1098/rstb.2007.2098 · Cited by 1,121
The dynamic brain: from spiking neurons to neural masses and cortical fields
Deco, G., Jirsa, V. K., Robinson, P. A., Breakspear, M., & Friston, K. (2008) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1000092 · Cited by 1,084 · cited 1×
“order parameters) are just the average of each state over oscillators; this particular phase-function is known as a mean field: see [27] for a discussion of mean field treatments of neuronal dynamics.”
BackgroundSelf-citationOn the computational architecture of the neocortex: II The role of cortico-cortical loops
Mumford, D. (1992) · Biological Cybernetics · doi:10.1007/bf00198477 · Cited by 1,074
On the theory of dynamic programming
Bellman, R. (1952) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.38.8.716 · Cited by 998
Keeping the neural networks simple by minimizing the description length of the weights
Hinton, G. E., & van Camp, D. (1993) · Proceedings of the sixth annual conference on Computational learning theory - COLT '93 · doi:10.1145/168304.168306 · Cited by 994
Hierarchical Models in the Brain
Friston, K. (2008) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1000211 · Cited by 952 · cited 1×
“Equation (4) assumes the fluctuations are temporally uncorrelated (Wiener) processes; however, because the fluctuations ω̃x(t) are in generalised coordinates of motion, the fluctuations on states per se can be smooth and analytic [34].”
BackgroundSelf-citationOn the computational architecture of the neocortex: I. The role of the thalamo-cortical loop
Mumford, D. (1991) · Biological Cybernetics · doi:10.1007/bf00202389 · Cited by 929 · cited 1×
“In neurobiological implementations of this scheme, the sources of prediction errors are usually thought to be superficial pyramidal cells while predictions are conveyed from deep pyramidal cells to superficial pyramidal cells encoding prediction error [54].”
BackgroundBirkhoff, G. D. (1931) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.17.2.656 · Cited by 867
Action and behavior: a free-energy formulation
Friston, K. J., Daunizeau, J., Kilner, J., & Kiebel, S. J. (2010) · Biological Cybernetics · doi:10.1007/s00422-010-0364-z · Cited by 866 · cited 3×
“We have shown recently that adaptive behaviour can be prescribed by prior expectations about sensory inputs, which action tries to fulfill [6].”
BackgroundInfluentialSelf-citationGregory, R. L. (1980) · Philosophical Transactions of the Royal Society of London. B, Biological Sciences · doi:10.1098/rstb.1980.0090 · Cited by 789
Decision theory, reinforcement learning, and the brain
Dayan, P., & Daw, N. D. (2008) · Cognitive, Affective, & Behavioral Neuroscience · doi:10.3758/cabn.8.4.429 · Cited by 580
Kass, R. E., & Steffey, D. (1989) · Journal of the American Statistical Association · doi:10.1080/01621459.1989.10478825 · Cited by 474 · cited 1×
“These predictions rest of on equations of motion that constitute (empirical) priors [55] on the flow of hidden states in the world.”
BackgroundReinforcement learning or active inference?
Friston, K. J., Daunizeau, J., & Kiebel, S. J. (2009) · PLoS ONE · doi:10.1371/journal.pone.0006421 · Cited by 443
Self-citationPerceptual illusions and brain models
Gregory, R. L. (1968) · Proceedings of the Royal Society of London. Series B. Biological Sciences · doi:10.1098/rspb.1968.0071 · Cited by 237
Friston, K., Stephan, K., Li, B., & Daunizeau, J. (2010) · Mathematical Problems in Engineering · doi:10.1155/2010/621670 · Cited by 171
Self-citationGlobal random attractors are uniquely determined by attracting deterministic compact sets
Crauel, H. (1999) · Annali di Matematica Pura ed Applicata · doi:10.1007/bf02505989 · Cited by 108 · cited 1×
“Technically speaking, if the base flow is ergodic and p(A(ω̃) ⊂ O) > 0 then A(ω̃) = ΩO(ω̃), almost surely [39].”
Cited by
Information-seeking, curiosity, and attention: computational and neural mechanisms
Gottlieb, J., Oudeyer, P.-Y., Lopes, M., & Baranes, A. (2013) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2013.09.001 · Cited by 784
“Converging evidence suggests that, when deployed in the service of a task, eye movements may be explained by the simple imperative to sample information in order to reduce uncertainty regarding future states (Tatler, Hayhoe et al. 2011; Friston and Ao 2012).”
Friston, K. (2013) · Journal of The Royal Society Interface · doi:10.1098/rsif.2013.0475 · Cited by 754
“…potential G ( x ) we will call Gibbs energy [32], Using this standard form [33], it is straightforward to show that p ( x j m ) ¼ exp( 2 G ( x )) is the equilibrium solution to the Fokker–Planck equation [12]: This means that we can express the flow in terms of the ergodic density f ¼ð G þ R Þ (cid”
Self-citationMoment-to-moment brain signal variability: A next frontier in human brain mapping?
Garrett, D. D., Samanez-Larkin, G. R., MacDonald, S. W. S., Lindenberger, U., McIntosh, A. R., & Grady, C. L. (2013) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2013.02.015 · Cited by 585
The free-energy self: a predictive coding account of self-recognition
Apps, M. A. J., & Tsakiris, M. (2014) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2013.01.029 · Cited by 514
“There is evidence in support of its claims (Brown and Friston, 2012; Friston, 2010a, b, 2012a; Friston et al., 2012a; Friston and Ao, 2012; Friston et al., 2012b; Friston et al., 2012c; Friston et al., 2010), although the theory is still in its infancy and therefore is neither largely supported or…”
Perceptions as Hypotheses: Saccades as Experiments
Friston, K., Adams, R. A., Perrinet, L., & Breakspear, M. (2012) · Frontiers in Psychology · doi:10.3389/fpsyg.2012.00151 · Cited by 462
Self-citation
Metadata
- Authors
- Friston, K., & Ao, P.
- Venue
- Computational and Mathematical Methods in Medicine
- Publisher
- Hindawi Limited
- Published
- 2012
- Volume
- 2012
- Pages
- 1-27
- ISSN
- 1748-670X (ISSN-L)
- 1748-6718
- OpenAlex
- W1971715144
- Cited by
- 139
- References
- 26
- FWCI
- 4.07
- Type
- journal article
- Language
- en
- Open access
- hybrid
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.