The feeling of grip: novelty, error dynamics, and the predictive brain
Open accessThe DOI matches its Crossref record; 62 of the 64 references it cites were found in a registry or an index, and 1 are not checked yet.
Abstract
According to the free energy principle biological agents resist a tendency to disorder in their interactions with a dynamically changing environment by keeping themselves in sensory and physiological states that are expected given their embodiment and the niche they inhabit (Friston in Nat Rev Neurosci 11(2):127–138, 2010. doi: 10.1038/nrn2787 ). Why would a biological agent that aims at minimising uncertainty in its encounters with the world ever be motivated to seek out novelty? Novelty for such an agent would arrive in the form of sensory and physiological states that are unexpected. Such an agent ought therefore to avoid novel and surprising interactions with the world one might think. Yet humans and many other animals find play and other forms of novelty-seeking and exploration hugely rewarding. How can this be understood in frameworks for studying the mind that emphasise prediction error minimisation? This problem has been taken up in recent research concerned with epistemic action—actions an agent engages in to reduce uncertainty. However that work leaves two questions unanswered, which it is the aim of our paper to address. First, no account has been given yet of why it should feel good to the agent to engage the world playfully and with curiosity. Second an appeal is made to precision-estimation to explain epistemic action, yet it remains unclear how precision-weighting works in action more generally, or active inference. We argue that an answer to both questions may lie in the bodily states of an agent that track the rate at which free energy is being reduced. The recent literature on the predictive brain has connected the valence of emotional experiences to the rate of change in the reduction of prediction error (Joffily and Coricelli in PLoS Comput Biol 9(6):e1003094, 2013. doi: 10.1371/journal.pcbi.1003094 ; Van de Cruys, in Metzinger and Wiese (eds) Philosophy and predictive processing, vol 24, MIND Group, Frankfurt am Main, 2017. doi: 10.15502/9783958573253 ). In this literature valenced emotional experiences are hypothesised to be identical with changes in the rate at which prediction error is reduced. Experiences are negatively valenced when overall prediction error increases and are positively valenced when the sum of prediction errors decrease. We offer an ecological-enactive interpretation of the concept of valence and its connection to rate of change of prediction error. We show how rate of change should be understood in terms of embodied states of affordance-related action readiness. We then go on to apply this ecological-enactive account of error dynamics to provide an answer to the first question we have raised: It may explain why it should feel good to an agent to be curious and playful. Our ecological-enactive account also allows us to show how error dynamics may provide an answer to the second question we have raised regarding how precision-weighting works in active inference. An agent that is sensitive to rates of error reduction can tune precision on the fly. We show how this ability to tune precision on the go can allow agents to develop skills for adapting better and better to the unexpected, and search out opportunities for resolving uncertainty and progressing in its learning.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source says what the claim attributes to it.
In the citing paper
“agents don’t seek only to minimize prediction error in the present but are sensitive to rates of change in prediction error minimization over time, relative to their prior expectations about that rate”
From Active inference goes to school: the importance of active learning in the age of large language models (2024) · cites this paper as [44]
The cited source says what the claim attributes to it.
In the citing paper
“These affective states aren’t a mere by-product, though, but instead are a crucial feature of action selection.”
From Active inference goes to school: the importance of active learning in the age of large language models (2024) · cites this paper as [44]
The cited source says what the claim attributes to it.
In the citing paper
“Promoting self-evidencing outcomes on longer timescales requires organisms to be sensitive not only to prediction error reduction in the present, but the rate of prediction error reduction over time (Joffily and Coricelli 2013; Kiverstein et al. 2019; Van de Cruys 2017).”
From Consciousness in active inference: Deep self-models, other minds, and the challenge of psychedelic-induced ego-dissolution (2021) · cites this paper as Kiverstein et al. 2019
The cited source says what the claim attributes to it.
In the citing paper
“Inference about the reliability of the action model allows the system to increase or decrease precision on the current policy (Kiverstein et al. 2019; Hesp et al. 2021).”
From Consciousness in active inference: Deep self-models, other minds, and the challenge of psychedelic-induced ego-dissolution (2021) · cites this paper as Kiverstein et al. 2019
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Reinforcement Learning: An Introduction
Sutton, R. S., & Barto, A. G. (1998) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.1998.712192 · Cited by 27,225
Reinforcement Learning: An Introduction
(2005) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.2004.842673 · Cited by 24,496
A neural substrate of prediction and reward
Schultz, W., Dayan, P., & Montague, P. R. (1997) · Science · doi:10.1126/science.275.5306.1593 · Cited by 9,388 · cited 1×
“Traditional reinforcement learning models describe goaldirected behaviour as a product of the agent working out how best to maximize an expected reward (Schutz et al. 1997; Sutton and Barto 1998).”
BackgroundThe free-energy principle: a unified brain theory?
Friston, K. (2010) · Nature Reviews Neuroscience · doi:10.1038/nrn2787 · Cited by 8,120 · cited 3×
“…complex principle—the so-called “free energy principle” (FEP), which can be applied to every biological system that resists a tendency to disorder (Friston 2009, 2010, 2013; Kirchhoff and Froese 2017).1 Friston has proposed that everything that can change in the 1 Elsewhere we have provided a…”
BackgroundMethodJournal of Consciousness Studies
(2022) · Journal of Consciousness Studies · doi:10.53765/20512201 · Cited by 6,832
Whatever next? Predictive brains, situated agents, and the future of cognitive science
Clark, A. (2013) · Behavioral and Brain Sciences · doi:10.1017/s0140525x12000477 · Cited by 5,924
Mind in life: biology, phenomenology, and the sciences of mind
(2008) · Choice Reviews Online · doi:10.5860/choice.45-2543 · Cited by 2,696
The free-energy principle: a rough guide to the brain?
Friston, K. (2009) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2009.04.005 · Cited by 1,831
Frijda, N. H. (1988) · American Psychologist · doi:10.1037/0003-066x.43.5.349 · Cited by 1,528
A Rich Landscape of Affordances
Rietveld, E., & Kiverstein, J. (2014) · Ecological Psychology · doi:10.1080/10407413.2014.958035 · Cited by 1,160
Self-citationActive inference: a process theory
Friston, K., FitzGerald, T., Rigoli, F., Schwartenbeck, P., & Pezzulo, G. (2017) · Neural Computation · doi:10.1162/neco_a_00912 · Cited by 1,156 · cited 3×
“…of policies in terms of paths of activity some of which are more probable than others to lead you to be in the states you expect to be in. Policies should serve tominimise future free energy, or what Friston and colleagues understand in terms of “expected free energy” (Friston et al. 2015, 2017).”
BackgroundBerlyne, D. E. (1966) · Science · doi:10.1126/science.153.3731.25 · Cited by 1,137
Development as a dynamic system
Smith, L. B., & Thelen, E. (2003) · Trends in Cognitive Sciences · doi:10.1016/s1364-6613(03)00156-6 · Cited by 1,128
Intrinsic Motivation Systems for Autonomous Mental Development
Oudeyer, P.-Y., Kaplan, F., & Hafner, V. V. (2007) · IEEE Transactions on Evolutionary Computation · doi:10.1109/tevc.2006.890271 · Cited by 1,126
Superforecasting: the art and science of prediction
(2016) · Choice Reviews Online · doi:10.5860/choice.196400 · Cited by 919
Information-seeking, curiosity, and attention: computational and neural mechanisms
Gottlieb, J., Oudeyer, P.-Y., Lopes, M., & Baranes, A. (2013) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2013.09.001 · Cited by 784
Kidd, C., Piantadosi, S. T., & Aslin, R. N. (2012) · PLoS ONE · doi:10.1371/journal.pone.0036399 · Cited by 783
Feelings of Being: Phenomenology, psychiatry and the sense of reality
Ratcliffe, M. (2008) · Oxford University Press eBooks · doi:10.1093/med/9780199206469.001.0001 · Cited by 771
Friston, K. (2013) · Journal of The Royal Society Interface · doi:10.1098/rsif.2013.0475 · Cited by 754 · cited 1×
“…complex principle—the so-called “free energy principle” (FEP), which can be applied to every biological system that resists a tendency to disorder (Friston 2009, 2010, 2013; Kirchhoff and Froese 2017).1 Friston has proposed that everything that can change in the 1 Elsewhere we have provided a…”
BackgroundFormal theory of creativity, fun, and intrinsic motivation (1990–2010)
Schmidhuber, J. (2010) · IEEE Transactions on Autonomous Mental Development · doi:10.1109/tamd.2010.2056368 · Cited by 747
Active inference and epistemic value
Friston, K., Rigoli, F., Ognibene, D., Mathys, C., Fitzgerald, T., & Pezzulo, G. (2015) · Cognitive Neuroscience · doi:10.1080/17588928.2015.1020053 · Cited by 731
Friston, K. J., & Stephan, K. E. (2007) · Synthese · doi:10.1007/s11229-007-9237-y · Cited by 673
Self-organization, free energy minimization, and optimal grip on a field of affordances
Bruineberg, J., & Rietveld, E. (2014) · Frontiers in Human Neuroscience · doi:10.3389/fnhum.2014.00599 · Cited by 538
Self-citationFeelings of Being: Phenomenology, Psychiatry and the Sense of Reality
Ratcliffe, M. (2008) · Cited by 492
Active learning of inverse models with intrinsically motivated goal exploration in robots
Baranes, A., & Oudeyer, P.-Y. (2013) · Robotics and Autonomous Systems · doi:10.1016/j.robot.2012.05.008 · Cited by 455
Cerebral hierarchies: predictive processing, precision and the pulvinar
Kanai, R., Komura, Y., Shipp, S., & Friston, K. (2015) · Philosophical Transactions of the Royal Society B: Biological Sciences · doi:10.1098/rstb.2014.0169 · Cited by 432 · cited 3×
“If we think of error signals as broadcasting newsworthy information then we must also explain how the brain decides which channels it should “listen to” (Kanai et al. 2015; c.f. Kwisthout et al. 2017).”
BackgroundBruineberg, J., Kiverstein, J., & Rietveld, E. (2018) · Synthese · doi:10.1007/s11229-016-1239-1 · Cited by 383 · cited 7×
“In earlier work (Bruineberg and Rietveld 2014; Bruineberg et al. 2016), we’ve proposed an ecological-enactive reading of the FEP, providing an analysis of freeenergy in terms of disattunement of internal and external dynamics, a dynamic state of disequilibrium within the agent-environment system as…”
BackgroundMethodInfluentialSelf-citationDopamine, affordance and active inference
Friston, K. J., Shiner, T., FitzGerald, T., Galea, J. M., Adams, R., Brown, H., Dolan, R. J., Moran, R., Stephan, K. E., & Bestmann, S. (2012) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1002327 · Cited by 381 · cited 5×
“We agree with others that the dark room problem is in some ways a red herring (e.g. Friston et al. 2012b), however we will argue that a significant problem remains.”
BackgroundInfluentialFree-Energy Minimization and the Dark-Room Problem
Friston, K., Thornton, C., & Clark, A. (2012) · Frontiers in Psychology · doi:10.3389/fpsyg.2012.00130 · Cited by 367
Emotional Valence and the Free-Energy Principle
Joffily, M., & Coricelli, G. (2013) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1003094 · Cited by 322 · cited 3×
“In the recent literature a number of authors have begun to relate information about rate of change in error reduction to the valence of full-blooded emotional experience like happiness, disappointment, hope and fear (Joffily and Coricelli 2013; Van de Cruys 2017; Van de Cruys and Wagemans 2011).”
BackgroundResultPutting Reward in Art: A Tentative Prediction Error Account of Visual Art
Van de Cruys, S., & Wagemans, J. (2011) · i-Perception · doi:10.1068/i0466aap · Cited by 296
The anatomy of choice: dopamine and decision-making.
Friston, K., Schwartenbeck, P., FitzGerald, T., Moutoussis, M., Behrens, T., & Dolan, R. J. (2014) · Philosophical Transactions of the Royal Society B: Biological Sciences · doi:10.1098/rstb.2013.0481 · Cited by 289
The cybernetic Bayesian brain : from interoceptive inference to sensorimotor contingencies
Seth, A. K. (2015) · MIND Group · doi:10.25358/openscience-84 · Cited by 239
Frijda, N. H. (2004) · Feelings and Emotions · doi:10.1017/cbo9780511806582.010 · Cited by 230
The dopaminergic midbrain encodes the expected certainty about desired outcomes
Schwartenbeck, P., FitzGerald, T. H. B., Mathys, C., Dolan, R., & Friston, K. (2015) · Cerebral Cortex · doi:10.1093/cercor/bhu159 · Cited by 219
Seth, A. K. (2015) · Figshare · doi:10.15502/9783958570108 · Cited by 211
Exploration, novelty, surprise, and free energy minimization
Schwartenbeck, P., FitzGerald, T., Dolan, R. J., & Friston, K. (2013) · Frontiers in Psychology · doi:10.3389/fpsyg.2013.00710 · Cited by 207
Superforecasting : the art and science of prediction
Tetlock, P., & Gardner, D. (2015) · Cited by 154
In search of the neural circuits of intrinsic motivation
Kaplan, F. (2007) · Frontiers in Neuroscience · doi:10.3389/neuro.01.1.1.017.2007 · Cited by 140
Where There is Life There is Mind: In Support of a Strong Life-Mind Continuity Thesis
Kirchhoff, M. D., & Froese, T. (2017) · Entropy · doi:10.3390/e19040169 · Cited by 114 · cited 2×
“…so-called “free energy principle” (FEP), which can be applied to every biological system that resists a tendency to disorder (Friston 2009, 2010, 2013; Kirchhoff and Froese 2017).1 Friston has proposed that everything that can change in the 1 Elsewhere we have provided a detailed philosophical…”
BackgroundHappily entangled: prediction, emotion, and the embodied mind
Miller, M., & Clark, A. (2018) · Synthese · doi:10.1007/s11229-017-1399-7 · Cited by 114
Self-citationSchouppe, N., Braem, S., Houwer, J. D., Silvetti, M., Verguts, T., Ridderinkhof, K. R., & Notebaert, W. (2014) · Cognitive Affective & Behavioral Neuroscience · doi:10.3758/s13415-014-0318-3 · Cited by 110
Busting Out: Predictive Brains, Embodied Minds, and the Puzzle of the Evidentiary Veil
Clark, A. (2017) · Noûs · doi:10.1111/nous.12140 · Cited by 106
Affective Value in the Predictive Mind
Van De Cruys, S. (2017) · Theoretical Philosophy/MIND Group – JGU Mainz · doi:10.15502/9783958573253 · Cited by 100
The Feeling of Action Tendencies: On the Emotional Regulation of Goal-Directed Behavior
Lowe, R., & Ziemke, T. (2011) · Frontiers in Psychology · doi:10.3389/fpsyg.2011.00346 · Cited by 99
Intrinsically Motivated Learning of Real-World Sensorimotor Skills with Developmental Constraints
Oudeyer, P., Baranès, A., & Kaplan, F. (2012) · Springer eBooks · doi:10.1007/978-3-642-32375-1_13 · Cited by 83
Optimal Grip on Affordances in Architectural Design Practices: An Ethnography
Rietveld, E., & Brouwers, A. A. (2017) · Phenomenology and the Cognitive Sciences · doi:10.1007/s11097-016-9475-x · Cited by 77
Self-citationResponse by rats to differential stimulus complexity.
Dember, W. N., Earl, R. W., & Paradise, N. E. (1957) · Journal of Comparative and Physiological Psychology · doi:10.1037/h0046780 · Cited by 75
Tuning Your Priors to the World
Feldman, J. (2013) · Topics in Cognitive Science · doi:10.1111/tops.12003 · Cited by 73
Kwisthout, J., Bekkering, H., & van Rooij, I. (2017) · Brain and Cognition · doi:10.1016/j.bandc.2016.02.008 · Cited by 73 · cited 1×
“If we think of error signals as broadcasting newsworthy information then we must also explain how the brain decides which channels it should “listen to” (Kanai et al. 2015; c.f. Kwisthout et al. 2017).”
BackgroundOudeyer, P.-Y., & Kaplan, F. (2006) · Connection Science · doi:10.1080/09540090600768567 · Cited by 72
Nguyen, S. M., & Oudeyer, P. (2012) · Paladyn Journal of Behavioral Robotics · doi:10.2478/s13230-013-0110-z · Cited by 69
Rietveld, E. (2008) · Theory & Psychology · doi:10.1177/0959354308089789 · Cited by 59
Self-citationThe Neural Organ Explains the Mind
Hohwy, J. (2015) · doi:10.15502/9783958570016 · Cited by 54
(2018) · Cited by 54
Self-citationLearning tactile skills through curious exploration
Pape, L., Oddo, C. M., Controzzi, M., Cipriani, C., Förster, A., Carrozza, M. C., & Schmidhuber, J. (2012) · Frontiers in Neurorobotics · doi:10.3389/fnbot.2012.00006 · Cited by 51 · cited 2×
“…of skills in high dimensions and under strong time constraints and to avoid unfruitful activities that are either well learnt and trivial, or which are random and unlearnable (Pape et al. 2012; Ngo et al. 2012; Baranes and Oudeyer 2013; Nguyen and Oudeyer 2013)” (Gottlieb et al. 2013: p. 9).”
BackgroundSocially guided intrinsic motivation for robot learning of motor skills
Nguyen, S. M., & Oudeyer, P.-Y. (2014) · Autonomous Robots · doi:10.1007/s10514-013-9339-y · Cited by 46
Curiosity-driven phonetic learning
Moulin-Frier, C., & Oudeyer, P.-Y. (2012) · 2012 IEEE International Conference on Development and Learning and Epigenetic Robotics (ICDL) · doi:10.1109/devlrn.2012.6400583 · Cited by 36
Learning skills from play: Artificial curiosity on a Katana robot arm
Ngo, H., Luciw, M., Förster, A., & Schmidhuber, J. (2012) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn.2012.6252824 · Cited by 30
Inviting complementary perspectives on situated normativity in everyday life
Klaassen, P., Rietveld, E., & Topal, J. (2009) · Phenomenology and the Cognitive Sciences · doi:10.1007/s11097-009-9133-7 · Cited by 27
Self-citationKaplan, F., & Oudeyer, P. (2011) · Cambridge University Press eBooks · doi:10.1017/cbo9780511994838.011 · Cited by 11
Affective value in the predictive mind
Van de Cruys, S. (2017) · MIND Group · doi:10.25358/openscience-647 · Cited by 1
(2018) · Digital Commons@Becker (Washington University School of Medicine) · doi:10.1371/journal
Research works printed but not linked: 3
- Berlyne, D. E. (1966). Curiosity and exploration. Science , 153 (3731), 25-33. Jul 1.
- Frijda, N. H. (1986). The emotions: Studies in emotion and social interaction . Cambridge: Cambridge University Press.
- McEvoy, P. (2002). Classic theory: The theory of interacting systems . San Francisco: Microanalytix.
Cited by
Thinking through other minds: A variational approach to cognition and culture
Veissière, S. P. L., Constant, A., Ramstead, M. J. D., Friston, K. J., & Kirmayer, L. J. (2020) · Behavioral and Brain Sciences · doi:10.1017/s0140525x19001213 · Cited by 348
Playing With Fear: A Field Study in Recreational Horror
Andersen, M. M., Schjoedt, U., Price, H. C. W., Rosas, F. E., Scrivner, C., & Clasen, M. (2020) · Psychological Science · doi:10.1177/0956797620972116 · Cited by 137
Play in predictive minds: A cognitive theory of play.
Andersen, M. M., Kiverstein, J., Miller, M., & Roepstorff, A. (2023) · Psychological Review · doi:10.1037/rev0000369 · Cited by 122
Self-citationFree-energy minimization in joint agent-environment systems: A niche construction perspective
Bruineberg, J., Rietveld, E., Parr, T., van Maanen, L., & Friston, K. J. (2018) · Journal of Theoretical Biology · doi:10.1016/j.jtbi.2018.07.002 · Cited by 99
Self-citationNave, K., Deane, G., Miller, M., & Clark, A. (2020) · Wiley Interdisciplinary Reviews Cognitive Science · doi:10.1002/wcs.1542 · Cited by 110
Self-citation
Metadata
- Authors
- Kiverstein, J., Miller, M., & Rietveld, E.
- Affiliations
- University of Amsterdam (NL)
- University of Edinburgh (GB)
- Corresponding author
- Kiverstein, Julian
- Venue
- Synthese
- Published
- 2017-10-23
- Volume
- 196
- Issue
- 7
- Pages
- 2847-2869
- ISSN
- 0039-7857 (ISSN-L)
- 1573-0964
- OpenAlex
- W2765412958
- Cited by
- 143
- References
- 64
- Citations by year
- 2026: 17 · 2025: 22 · 2024: 20 · 2023: 23 · 2022: 22 · 2021: 13 · 2020: 14 · 2019: 8 · 2018: 3 · 2017: 1
- FWCI
- 4.09
- Type
- journal article
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- cc-by
- creativecommons.org/licenses/by/4.0 (tdm)
- Locations
- Synthese (publishedVersion, cc-by)
- Pure Amsterdam UMC (publishedVersion, cc-by)
- Data Archiving and Networked Services (DANS) (submittedVersion, other-oa)
- UvA-DARE (University of Amsterdam) (submittedVersion, cc-by)
- Funders
- H2020 European Research Council (ERC Starting Grant)
- Topics
- Life Sciences › Neuroscience › Cognitive Neuroscience › Embodied and Extended Cognition
- Social Sciences › Arts and Humanities › Philosophy › Free Will and Agency
- Life Sciences › Neuroscience › Cognitive Neuroscience › Cognitive Science and Education Research
- Keywords
- free energy principle, prediction error, novelty seeking, curiosity, valence, emotional experience, predictive processing, active inference, epistemic actions, precision estimation, exploration, play, embodied cognition, enactive philosophy, uncertainty reduction, motor control, error dynamics, predictive brain, precision weighting
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.