Free Energy, Value, and Attractors
Open accessThe DOI matches its Crossref record; 29 of the 29 references it cites were found in a registry or an index.
Abstract
It has been suggested recently that action and perception can be understood as minimising the free energy of sensory samples. This ensures that agents sample the environment to maximise the evidence for their model of the world, such that exchanges with the environment are predictable and adaptive. However, the free energy account does not invoke reward or cost-functions from reinforcement-learning and optimal control theory. We therefore ask whether reward is necessary to explain adaptive behaviour. The free energy formulation uses ideas from statistical physics to explain action in terms of minimising sensory surprise. Conversely, reinforcement-learning has its roots in behaviourism and engineering and assumes that agents optimise a policy to maximise future reward. This paper tries to connect the two formulations and concludes that optimal policies correspond to empirical priors on the trajectories of hidden environmental states, which compel agents to seek out the (valuable) states they expect to encounter.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
On Information and Sufficiency
Kullback, S., & Leibler, R. A. (1951) · The Annals of Mathematical Statistics · doi:10.1214/aoms/1177729694 · Cited by 20,193
A Stochastic Approximation Method
Robbins, H., & Monro, S. (1951) · The Annals of Mathematical Statistics · doi:10.1214/aoms/1177729586 · Cited by 9,643
Watkins, C. J. C. H., & Dayan, P. (1992) · Machine Learning · doi:10.1007/bf00992698 · Cited by 9,083
Discrete Choice Methods with Simulation
Train, K. E. (2001) · doi:10.1017/cbo9780511805271 · Cited by 7,784
Rao, R. P. N., & Ballard, D. H. (1999) · Nature Neuroscience · doi:10.1038/4580 · Cited by 5,483 · cited 1×
“This particular free energy minimisation scheme is known as generalised predictive coding and has become a useful metaphor for neuronal message passing in the brain: see also [12].”
BackgroundThe Bayesian brain: The role of uncertainty in neural coding and computation
Knill, D. C., & Pouget, A. (2004) · Trends in Neurosciences · doi:10.1016/j.tins.2004.10.007 · Cited by 2,588 · cited 1×
“This is the basis of perceptual inference and learning as articulated by the Bayesian brain hypothesis [10, 13, 48– 52].”
MethodHierarchical Bayesian inference in the visual cortex
Lee, T. S., & Mumford, D. (2003) · Journal of the Optical Society of America A · doi:10.1364/josaa.20.001434 · Cited by 1,473 · cited 1×
“This paper tries to connect the two formulations and concludes that optimal policies correspond to empirical priors on the trajectories of hidden environmental states, which compel agents to seek out the (valuable) states they expect to encounter.”
BackgroundToward a modern theory of adaptive networks: Expectation and prediction.
Sutton, R. S., & Barto, A. G. (1981) · Psychological Review · doi:10.1037/0033-295x.88.2.135 · Cited by 1,450 · cited 2×
“These policies are considered in reinforcement-learning and optimal control theory [2, 4, 14].”
BackgroundMethodA free energy principle for the brain
Friston, K., Kilner, J., & Harrison, L. (2006) · Journal of Physiology-Paris · doi:10.1016/j.jphysparis.2006.10.001 · Cited by 1,369
Self-citationObject perception as Bayesian inference
Kersten, D., Mamassian, P., & Yuille, A. (2004) · Annual Review of Psychology · doi:10.1146/annurev.psych.55.090902.142005 · Cited by 1,269
Cited by
Information-seeking, curiosity, and attention: computational and neural mechanisms
Gottlieb, J., Oudeyer, P.-Y., Lopes, M., & Baranes, A. (2013) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2013.09.001 · Cited by 784
“Converging evidence suggests that, when deployed in the service of a task, eye movements may be explained by the simple imperative to sample information in order to reduce uncertainty regarding future states (Tatler, Hayhoe et al. 2011; Friston and Ao 2012).”
Friston, K. (2013) · Journal of The Royal Society Interface · doi:10.1098/rsif.2013.0475 · Cited by 754
“…potential G ( x ) we will call Gibbs energy [32], Using this standard form [33], it is straightforward to show that p ( x j m ) ¼ exp( 2 G ( x )) is the equilibrium solution to the Fokker–Planck equation [12]: This means that we can express the flow in terms of the ergodic density f ¼ð G þ R Þ (cid”
Self-citationMoment-to-moment brain signal variability: A next frontier in human brain mapping?
Garrett, D. D., Samanez-Larkin, G. R., MacDonald, S. W. S., Lindenberger, U., McIntosh, A. R., & Grady, C. L. (2013) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2013.02.015 · Cited by 585
The free-energy self: a predictive coding account of self-recognition
Apps, M. A. J., & Tsakiris, M. (2014) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2013.01.029 · Cited by 514
“There is evidence in support of its claims (Brown and Friston, 2012; Friston, 2010a, b, 2012a; Friston et al., 2012a; Friston and Ao, 2012; Friston et al., 2012b; Friston et al., 2012c; Friston et al., 2010), although the theory is still in its infancy and therefore is neither largely supported or…”
Perceptions as Hypotheses: Saccades as Experiments
Friston, K., Adams, R. A., Perrinet, L., & Breakspear, M. (2012) · Frontiers in Psychology · doi:10.3389/fpsyg.2012.00151 · Cited by 462
Self-citationDopamine, affordance and active inference
Friston, K. J., Shiner, T., FitzGerald, T., Galea, J. M., Adams, R., Brown, H., Dolan, R. J., Moran, R., Stephan, K. E., & Bestmann, S. (2012) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1002327 · Cited by 381
Self-citationA free energy principle for biological systems
Karl, F. (2012) · Entropy · doi:10.3390/e14112100 · Cited by 377
“Domain Process or paradigm Perception • Perceptual categorisation (bird songs) [18] • Novelty and omission-related responses [18] • Perceptual inference (speech) [19] Sensory learning • Perceptual learning (mismatch negativity) [33] Attention • Attention and the Posner paradigm [20] • Attention and biased competition…”
BackgroundAnswering Schrödinger's question: A free-energy formulation
Ramstead, M. J. D., Badcock, P. B., & Friston, K. J. (2018) · Physics of Life Reviews · doi:10.1016/j.plrev.2017.09.001 · Cited by 354
“…also been applied to biological systems across spatial and temporal scales, ranging from phenomena at the micro-scale (e.g., dendritic self-organisation and morphogenesis; [27,28]), across intermediate scales (e.g., cultural ensembles; [29]), and at the macro-scale (e.g., natural selection; [30]).”
Self-citationWhat is optimal about motor control?
Friston, K. (2011) · Neuron · doi:10.1016/j.neuron.2011.10.018 · Cited by 332
Self-citationLevin, M. (2019) · Frontiers in Psychology · doi:10.3389/fpsyg.2019.02688 · Cited by 297
Knowing one's place: a free-energy approach to pattern regulation
Friston, K., Levin, M., Sengupta, B., & Pezzulo, G. (2015) · Journal of The Royal Society Interface · doi:10.1098/rsif.2014.1383 · Cited by 234
Self-citationActive inference and agency: optimal control without cost functions
Friston, K., Samothrakis, S., & Montague, R. (2012) · Biological Cybernetics · doi:10.1007/s00422-012-0512-8 · Cited by 241
“This formulation of behaviour is based on ergodic arguments about the nature of self organising systems (Ashby 1947)—for a fuller discussion please see Friston and Ao (2012) and Friston (2010) for their neurobiological implications .”
BackgroundMethodInfluentialSelf-citationA free energy principle for a particular physics
Friston, K. (2019) · arXiv · doi:10.48550/arxiv.1906.10184 · Cited by 222
Self-citationInformation and efficiency in the nervous system-a synthesis.
Sengupta, B., Stemmler, M. B., & Friston, K. J. (2013) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1003157 · Cited by 212
“An ergodic system has an invariant phase volume [11], which is a necessary condition for an organism to exist—in the sense that it would otherwise transgress phase boundaries and cease to exist [12].”
BackgroundSelf-citationA variational approach to niche construction
Constant, A., Ramstead, M. J. D., Veissière, S. P. L., Campbell, J. O., & Friston, K. J. (2018) · Journal of The Royal Society Interface · doi:10.1098/rsif.2017.0685 · Cited by 204
“Given the statistical conception of the phenotype described above, the upshot is that we can interpret active inference as the generalized homeostasis and allostasis of an organism [21,56].”
BackgroundSelf-citationLevin, M. (2022) · Frontiers in Systems Neuroscience · doi:10.3389/fnsys.2022.768201 · Cited by 195
Sentience and the Origins of Consciousness: From Cartesian Duality to Markovian Monism
Friston, K. J., Wiese, W., & Hobson, J. A. (2020) · Entropy · doi:10.3390/e22050516 · Cited by 186
Self-citationCorcoran, A. W., Pezzulo, G., & Hohwy, J. (2020) · Biology & Philosophy · doi:10.1007/s10539-020-09746-2 · Cited by 161
Markov blankets, information geometry and stochastic thermodynamics
Parr, T., Da Costa, L., & Friston, K. (2020) · Philosophical Transactions of the Royal Society A Mathematical Physical and Engineering Sciences · doi:10.1098/rsta.2019.0159 · Cited by 141
Self-citationDeep active inference as variational policy gradients
Millidge, B. (2020) · Journal of Mathematical Psychology · doi:10.1016/j.jmp.2020.102348 · Cited by 115
On markov blankets and hierarchical self-organisation
Palacios, E. R., Razi, A., Parr, T., Kirchhoff, M., & Friston, K. (2020) · Journal of Theoretical Biology · doi:10.1016/j.jtbi.2019.110089 · Cited by 115
Self-citationSmooth pursuit and visual occlusion: active inference and oculomotor control in schizophrenia
Adams, R. A., Perrinet, L. U., & Friston, K. (2012) · PLoS ONE · doi:10.1371/journal.pone.0047502 · Cited by 91
“Behaviour Heuristics and dynamical systems theory [92]”
Self-citationPerceptual awareness and active inference
Parr, T., Corcoran, A. W., Friston, K. J., & Hohwy, J. (2019) · Neuroscience of Consciousness · doi:10.1093/nc/niz012 · Cited by 91
“Term 3 may be thought of as encoding preferences about the sorts of outcomes that will be sought (Friston and Ao 2012).”
BackgroundSelf-citationA Multi-scale View of the Emergent Complexity of Life: A Free-Energy Proposal
Hesp, C., Ramstead, M., Constant, A., Badcock, P., Kirchhoff, M., & Friston, K. (2019) · Springer Proceedings in Complexity · doi:10.1007/978-3-030-00075-2_7 · Cited by 63
“Having said this, the FEP and thermodynamics are internally consistent in the sense that thermodynamics – particularly stochastic thermodynamics (Ao, 2008; Seifert, 2012) – can be regarded as a special case of the FEP when certain conditions are met (Friston & Ao, 2012).”
BackgroundSelf-citationPredictive coding networks for temporal prediction
Millidge, B., Tang, M., Osanlouy, M., Harper, N. S., & Bogacz, R. (2024) · PLOS Computational Biology · doi:10.1371/journal.pcbi.1011183 · Cited by 44
“The conceptual level of our model is grounded within the Bayesian Brain paradigm [5, 42–44].”
MethodA worked example of Fokker-Planck-based active inference
Koudahl, M. T., & de Vries, B. (2020) · Communications in Computer and Information Science · doi:10.1007/978-3-030-64919-7_4 · Cited by 7
“To apply the Fokker-Planck equation to active inference, we follow [5,6] and let J(x) denote a variational free energy functional.”
BackgroundMethodInfluential
Metadata
- Authors
- Friston, K., & Ao, P.
- Venue
- Computational and Mathematical Methods in Medicine
- Publisher
- Hindawi Limited
- Published
- 2012
- Volume
- 2012
- Pages
- 1-27
- ISSN
- 1748-670X (ISSN-L)
- 1748-6718
- OpenAlex
- W1971715144
- Cited by
- 139
- References
- 29
- FWCI
- 4.07
- Type
- journal article
- Language
- en
- Open access
- hybrid
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.