The DOI matches its Crossref record; 20 of the 20 references it cites were found in a registry or an index.
Exploration, novelty, surprise, and free energy minimization
Open accessAbstract
This paper reviews recent developments under the free energy principle that introduce a normative perspective on classical economic (utilitarian) decision-making based on (active) Bayesian inference. It has been suggested that the free energy principle precludes novelty and complexity, because it assumes that biological systems-like ourselves-try to minimize the long-term average of surprise to maintain their homeostasis. However, recent formulations show that minimizing surprise leads naturally to concepts such as exploration and novelty bonuses. In this approach, agents infer a policy that minimizes surprise by minimizing the difference (or relative entropy) between likely and desired outcomes, which involves both pursuing the goal-state that has the highest expected utility (often termed "exploitation") and visiting a number of different goal-states ("exploration"). Crucially, the opportunity to visit new states increases the value of the current state. Casting decision-making problems within a variational framework, therefore, predicts that our behavior is governed by both the entropy and expected utility of future states. This dissolves any dialectic between minimizing surprise and exploration or novelty seeking.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Reinforcement Learning: An Introduction
Sutton, R. S., & Barto, A. G. (1998) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.1998.712192 · Cited by 27,225
Information theory and statistical mechanics
Jaynes, E. T. (1957) · Physical Review · doi:10.1103/physrev.106.620 · Cited by 12,931
The free-energy principle: a unified brain theory?
Friston, K. (2010) · Nature Reviews Neuroscience · doi:10.1038/nrn2787 · Cited by 8,120 · cited 2×
“The free energy principle is a theoretical formulation of biological systems and their behavior (Friston et al., 2006; Friston, 2009, 2010) that has attracted much current research interest (Brown and Friston, 2012; Adams et al., 2013a; Apps and Tsakiris, 2013; Joffily and Coricelli, 2013; Moran et…”
BackgroundSelf-citationWhatever next? Predictive brains, situated agents, and the future of cognitive science
Clark, A. (2013) · Behavioral and Brain Sciences · doi:10.1017/s0140525x12000477 · Cited by 5,924
Canonical Microcircuits for Predictive Coding
Bastos, A. M., Usrey, W. M., Adams, R. A., Mangun, G. R., Fries, P., & Friston, K. J. (2012) · Neuron · doi:10.1016/j.neuron.2012.10.038 · Cited by 2,421 · cited 1×
“…a unified account of brain function and behavior and has proven useful for understanding neuroanatomy, neurophysiology (Feldman and Friston, 2010; Bastos et al., 2012; Brown and Friston, 2012; Adams et al., 2013b; Moran et al., 2013), and psychiatry (Edwards et al., 2012; Adams et al., 2013a).”
BackgroundSelf-citationThe free-energy principle: a rough guide to the brain?
Friston, K. (2009) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2009.04.005 · Cited by 1,831
Self-citationA free energy principle for the brain
Friston, K., Kilner, J., & Harrison, L. (2006) · Journal of Physiology-Paris · doi:10.1016/j.jphysparis.2006.10.001 · Cited by 1,369
Self-citationAttention, uncertainty, and free-energy
Feldman, H., & Friston, K. J. (2010) · Frontiers in Human Neuroscience · doi:10.3389/fnhum.2010.00215 · Cited by 1,365
Self-citationKeeping the neural networks simple by minimizing the description length of the weights
Hinton, G. E., & van Camp, D. (1993) · Proceedings of the sixth annual conference on Computational learning theory - COLT '93 · doi:10.1145/168304.168306 · Cited by 994
The Computational Anatomy of Psychosis
Adams, R. A., Stephan, K. E., Brown, H. R., Frith, C. D., & Friston, K. J. (2013) · Frontiers in Psychiatry · doi:10.3389/fpsyt.2013.00047 · Cited by 850 · cited 2×
“…principle is a theoretical formulation of biological systems and their behavior (Friston et al., 2006; Friston, 2009, 2010) that has attracted much current research interest (Brown and Friston, 2012; Adams et al., 2013a; Apps and Tsakiris, 2013; Joffily and Coricelli, 2013; Moran et al., 2013).”
BackgroundSelf-citation
Cited by
Carhart-Harris, R. L., Leech, R., Hellyer, P. J., Shanahan, M., Feilding, A., Tagliazucchi, E., Chialvo, D. R., & Nutt, D. (2014) · Frontiers in Human Neuroscience · doi:10.3389/fnhum.2014.00020 · Cited by 1,252
Active inference: a process theory
Friston, K., FitzGerald, T., Rigoli, F., Schwartenbeck, P., & Pezzulo, G. (2017) · Neural Computation · doi:10.1162/neco_a_00912 · Cited by 1,156
Self-citationDeYoung, C. G. (2015) · Journal of Research in Personality · doi:10.1016/j.jrp.2014.07.004 · Cited by 1,057
Active inference and epistemic value
Friston, K., Rigoli, F., Ognibene, D., Mathys, C., Fitzgerald, T., & Pezzulo, G. (2015) · Cognitive Neuroscience · doi:10.1080/17588928.2015.1020053 · Cited by 731
Self-citationFriston, K., FitzGerald, T., Rigoli, F., Schwartenbeck, P., ODoherty, J., & Pezzulo, G. (2016) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2016.06.022 · Cited by 674
Self-citationUncertainty in perception and the Hierarchical Gaussian Filter
Mathys, C. D., Lomakina, E. I., Daunizeau, J., Iglesias, S., Brodersen, K. H., Friston, K. J., & Stephan, K. E. (2014) · Frontiers in Human Neuroscience · doi:10.3389/fnhum.2014.00825 · Cited by 465
Self-citationResolving uncertainty in a social world
FeldmanHall, O., & Shenhav, A. (2019) · Nature Human Behaviour · doi:10.1038/s41562-019-0590-x · Cited by 274
Deep reinforcement learning and its neuroscientific implications
Botvinick, M., Wang, J. X., Dabney, W., Miller, K. J., & Kurth-Nelson, Z. (2020) · Neuron · doi:10.1016/j.neuron.2020.06.014 · Cited by 269
The Shifting Architecture of Cognition and Brain Function in Older Adulthood
Spreng, R. N., & Turner, G. R. (2019) · Perspectives on Psychological Science · doi:10.1177/1745691619827511 · Cited by 256
The dopaminergic midbrain encodes the expected certainty about desired outcomes
Schwartenbeck, P., FitzGerald, T. H. B., Mathys, C., Dolan, R., & Friston, K. (2015) · Cerebral Cortex · doi:10.1093/cercor/bhu159 · Cited by 219
Self-citationActive inference on discrete state-spaces: A synthesis
Da Costa, L., Parr, T., Sajid, N., Veselic, S., Neacsu, V., & Friston, K. (2020) · Journal of Mathematical Psychology · doi:10.1016/j.jmp.2020.102447 · Cited by 226
Self-citationCorcoran, A. W., Pezzulo, G., & Hohwy, J. (2020) · Biology & Philosophy · doi:10.1007/s10539-020-09746-2 · Cited by 161
A nice surprise? Predictive processing and the active pursuit of novelty
Clark, A. (2018) · Phenomenology and the Cognitive Sciences · doi:10.1007/s11097-017-9525-z · Cited by 156
The feeling of grip: novelty, error dynamics, and the predictive brain
Kiverstein, J., Miller, M., & Rietveld, E. (2019) · Synthese · doi:10.1007/s11229-017-1583-9 · Cited by 143
Play in predictive minds: A cognitive theory of play.
Andersen, M. M., Kiverstein, J., Miller, M., & Roepstorff, A. (2023) · Psychological Review · doi:10.1037/rev0000369 · Cited by 122
Dopamine, reward learning, and active inference
FitzGerald, T. H. B., Dolan, R. J., & Friston, K. (2015) · Frontiers in Computational Neuroscience · doi:10.3389/fncom.2015.00136 · Cited by 112
“…in Computational Neuroscience | www.frontiersin.org 12 November 2015 | Volume 9 | Article 136 to cognition (Tenenbaum et al., 2006; Friston et al., 2013; Pouget et al., 2013; Schwartenbeck et al., 2013), rather than to attribute a single, definitive, function to the dopaminergic system.”
BackgroundSelf-citationEvidence for surprise minimization over value maximization in choice behavior
Schwartenbeck, P., FitzGerald, T. H. B., Mathys, C., Dolan, R., Kronbichler, M., & Friston, K. (2015) · Scientific Reports · doi:10.1038/srep16575 · Cited by 96
Self-citationLosing Ourselves: Active Inference, Depersonalization, and Meditation
Deane, G., Miller, M., & Wilkinson, S. (2020) · Frontiers in Psychology · doi:10.3389/fpsyg.2020.539726 · Cited by 87
Curiosity: primate neural circuits for novelty and information seeking
Monosov, I. E. (2024) · Nature Reviews Neuroscience · doi:10.1038/s41583-023-00784-9 · Cited by 44
“…modulation of value by novelty afforded by this architecture can help to solve some forms of exploration– exploitation dilemmas 3,12,19,62,73 , particularly in contexts in which rewards are sparse, by allowing for flexible boosting (or bonusing) of RPEs by novelty to modulate exploration…”
Neural dynamics under active inference: plausibility and efficiency of information processing
Da Costa, L., Parr, T., Sengupta, B., & Friston, K. (2021) · Entropy · doi:10.3390/e23040454 · Cited by 38
Self-citationSurprise! Why enactivism and predictive processing are parting ways: The case of improvisation
Gallagher, S. (2023) · Possibility Studies & Society · doi:10.1177/27538699221132691 · Cited by 15
Active Inference, Novelty and Neglect
Parr, T., & Friston, K. J. (2018) · Current Topics in Behavioral Neurosciences · doi:10.1007/7854_2018_61 · Cited by 15
Self-citationRevealing human sensitivity to a latent temporal structure of changes
Marković, D., Reiter, A. M. F., & Kiebel, S. J. (2022) · Frontiers in Behavioral Neuroscience · doi:10.3389/fnbeh.2022.962494 · Cited by 2
Metadata
- Authors
- Schwartenbeck, P., FitzGerald, T., Dolan, R. J., & Friston, K.
- Venue
- Frontiers in Psychology
- Publisher
- Frontiers Media SA
- Published
- 2013
- Volume
- 4
- Pages
- 710-710
- ISSN
- 1664-1078 (ISSN-L)
- OpenAlex
- W2014114527
- Cited by
- 207
- References
- 20
- FWCI
- 4.04
- Type
- journal article
- Language
- en
- Open access
- gold
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.