Learning action-oriented models through active inference
Open accessPublished version of a preprint
The DOI matches its Crossref record; 83 of the 84 references it cites were found in a registry or an index, and 1 are not checked yet.
Abstract
Converging theories suggest that organisms learn and exploit probabilistic models of their environment. However, it remains unclear how such models can be learned in practice. The open-ended complexity of natural environments means that it is generally infeasible for organisms to model their environment comprehensively. Alternatively, action-oriented models attempt to encode a parsimonious representation of adaptive agent-environment interactions. One approach to learning action-oriented models is to learn online in the presence of goal-directed behaviours. This constrains an agent to behaviourally relevant trajectories, reducing the diversity of the data a model need account for. Unfortunately, this approach can cause models to prematurely converge to sub-optimal solutions, through a process we refer to as a bad-bootstrap. Here, we exploit the normative framework of active inference to show that efficient action-oriented models can be learned by balancing goal-oriented and epistemic (information-seeking) behaviours in a principled manner. We illustrate our approach using a simple agent-based model of bacterial chemotaxis. We first demonstrate that learning via goal-directed behaviour indeed constrains models to behaviorally relevant aspects of the environment, but that this approach is prone to sub-optimal convergence. We then demonstrate that epistemic behaviours facilitate the construction of accurate and comprehensive models, but that these models are not tailored to any specific behavioural niche and are therefore less efficient in their use of data. Finally, we show that active inference agents learn models that are parsimonious, tailored to action, and which avoid bad bootstraps and sub-optimal convergence. Critically, our results indicate that models learned through active inference can support adaptive behaviour in spite of, and indeed because of, their departure from veridical representations of the environment. Our approach provides a principled method for learning adaptive models from limited interactions with an environment, highlighting a route to sample efficient learning algorithms.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Figures






Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“This unique formulation of confirmation bias as an epistemic phenomenon helps explain how individuals continuously forage their environment for information, but may become stuck in a so-called ‘bad bootstrap’ that simply reinforces existing beliefs about the world, which in the face of new information may lead to sub-optimal behaviour [131].”
From Epistemic Communities under Active Inference (2022) · cites this paper as [131]
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
The Ecological Approach to Visual Perception: Classic Edition
Gibson, J. J. (2014) · doi:10.4324/9781315740218 · Cited by 12,132
Playing Atari with Deep Reinforcement Learning
Mnih, V., Kavukcuoglu, K., Silver, D., Graves, A., Antonoglou, I., Wierstra, D., & Riedmiller, M. (2013) · arXiv · doi:10.48550/arxiv.1312.5602 · Cited by 9,471
The free-energy principle: a unified brain theory?
Friston, K. (2010) · Nature Reviews Neuroscience · doi:10.1038/nrn2787 · Cited by 8,120 · cited 1×
“Active inference is a normative theory that unifies perception, action and learning under a single imperative—the minimization of variational free energy [42, 43].”
BackgroundMarkov decision processes: Discrete stochastic dynamic programming
Hazeghi, K., & Puterman, M. L. (1995) · Journal of the American Statistical Association · doi:10.2307/2291177 · Cited by 8,037
Markov Decision Processes: Discrete Stochastic Dynamic Programming
Puterman, M. L. (1994) · Wiley Series in Probability and Statistics · doi:10.1002/9780470316887 · Cited by 6,024
Rao, R. P. N., & Ballard, D. H. (1999) · Nature Neuroscience · doi:10.1038/4580 · Cited by 5,483 · cited 2×
“This casts perception as a process of approximate inference, connecting active inference to influential theories such as the Bayesian brain hypothesis (Knill and Pouget, 2004; Gregory, 1980) and predictive coding (Rao and Ballard, 1999).”
MethodBengio, Y., Louradour, J., Collobert, R., & Weston, J. (2009) · Proceedings of the 26th Annual International Conference on Machine Learning · doi:10.1145/1553374.1553380 · Cited by 5,366
Watkins, C. (1989) · OpenGrey (Institut de l'Information Scientifique et Technique) · doi:10.17863/cam.132164 · Cited by 5,297
The Bayesian brain: The role of uncertainty in neural coding and computation
Knill, D. C., & Pouget, A. (2004) · Trends in Neurosciences · doi:10.1016/j.tins.2004.10.007 · Cited by 2,588
Chemotaxis in Escherichia coli analysed by Three-dimensional Tracking
BERG, H. C., & BROWN, D. A. (1972) · Nature · doi:10.1038/239500a0 · Cited by 2,231
Research works printed but not linked: 2
- Kuvayev L, Sutton RS. Model-Based Reinforcement Learning with an Approximate, Learned Model. In: in Proceedings of the Ninth Yale Workshop on Adaptive and Learning Systems; 1996. p. 101–105.
- Ha D, Schmidhuber J. World Models. arXiv:180310122. 2018;.
Cited by
A step-by-step tutorial on active inference and its application to empirical data
Smith, R., Friston, K. J., & Whyte, C. J. (2022) · Journal of Mathematical Psychology · doi:10.1016/j.jmp.2021.102632 · Cited by 246
Predictive processing as a systematic basis for identifying the neural correlates of consciousness
Hohwy, J., & Seth, A. (2020) · Philosophy and the Mind Sciences · doi:10.33735/phimisci.2020.ii.64 · Cited by 170
Self-citationThe Emperor's New Markov Blankets
Bruineberg, J., Dołęga, K., Dewhurst, J., & Baltieri, M. (2022) · Behavioral and Brain Sciences · doi:10.1017/s0140525x21002351 · Cited by 124
The evolution of brain architectures for predictive coding and active inference
Pezzulo, G., Parr, T., & Friston, K. (2022) · Philosophical Transactions of the Royal Society B · doi:10.1098/rstb.2020.0531 · Cited by 117
An Active Inference Approach to Modeling Structure Learning: Concept Learning as an Example Case
Smith, R., Schwartenbeck, P., Parr, T., & Friston, K. J. (2020) · Frontiers in Computational Neuroscience · doi:10.3389/fncom.2020.00041 · Cited by 99
Tschantz, A., Barca, L., Maisto, D., Buckley, C. L., Seth, A. K., & Pezzulo, G. (2022) · Biological Psychology · doi:10.1016/j.biopsycho.2022.108266 · Cited by 118
Self-citationLeptourgos, P., Fortier-Davy, M., Carhart-Harris, R., Corlett, P. R., Dupuis, D., Halberstadt, A. L., Kometer, M., Kozakova, E., LarØi, F., Noorani, T. N., et al. (2020) · Schizophrenia Bulletin · doi:10.1093/schbul/sbaa117 · Cited by 86
Gao, H., Ou, Y., Zhang, Z., Ni, M., Zhou, X., & Liao, L. (2021) · Frontiers in Psychology · doi:10.3389/fpsyg.2021.573779 · Cited by 84
From Generative Models to Generative Passages: A Computational Approach to (Neuro) Phenomenology
Ramstead, M. J. D., Seth, A. K., Hesp, C., Sandved-Smith, L., Mago, J., Lifshitz, M., Pagnoni, G., Smith, R., Dumas, G., Lutz, A., et al. (2022) · Review of Philosophy and Psychology · doi:10.1007/s13164-021-00604-y · Cited by 84
“, plan or sequence of actions) the agent must be pursuing on the assumption that what it does must be minimizing the difference between anticipated outcomes—conditional on its course of action—and what it, a priori, it expects to sense (Ramstead et al. 2019a, 2020b; Tschantz et al. 2020).”
BackgroundSelf-citationHow particular is the physics of the free energy principle?
Millidge, B. (2022) · Sussex Research Online (University of Sussex) · Cited by 83
Unpacking the complexities of consciousness: Theories and reflections
Mudrik, L., Boly, M., Dehaene, S., Fleming, S. M., Lamme, V., Seth, A., & Melloni, L. (2025) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2025.106053 · Cited by 81
“…the FEP, recent extensions have illustrated how minimization of ‘expected free energy ’ (i.e., approximate future prediction error under some action policy) can optimally balance a trade-off between exploratory (epistemic) and goal-directed actions (Friston et al., 2015; Tschantz et al., 2020).”
Self-citationGenerating meaning: active inference and the scope and limits of passive AI
Pezzulo, G., Parr, T., Cisek, P., Clark, A., & Friston, K. (2024) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2023.10.002 · Cited by 80
“…during evolution, free-energy minimization could endow animal brains with prior structure encoded in species-speci fi c circuitry [27,96,134]. representations of the environment [89 – 91] such as the generative models (e.g., central pattern generators) for action cycles in simple organisms [92].”
pymdp: A Python library for active inference in discrete state spaces
Heins, C., Millidge, B., Demekas, D., Klein, B., Friston, K., Couzin, I. D., & Tschantz, A. (2022) · Journal of Open Source Software · doi:10.21105/joss.04098 · Cited by 70
“A recent related, but largely non-overlapping project is ForneyLab, which provides a set of Julia libraries for performing approximate Bayesian inference via message passing on Forney Factor Graphs [49].”
BackgroundInfluentialSelf-citationTschantz, A., Baltieri, M., Seth, A. K., & Buckley, C. L. (2020) · Sussex Research Online (University of Sussex) · Cited by 69
Self-citationReinforcement Learning through Active Inference
Tschantz, A., Millidge, B., Seth, A. K., & Buckley, C. L. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2002.12636 · Cited by 55
“There have also been prior attempts to scale up active inference to continuous RL tasks (Tschantz et al., 2019a; Millidge, 2019; Ueltzhffer, 2018), which we build upon here.”
BackgroundSelf-citationOn the Relationship Between Active Inference and Control as Inference
Millidge, B., Tschantz, A., Seth, A. K., & Buckley, C. L. (2020) · Communications in computer and information science · doi:10.1007/978-3-030-64919-7_1 · Cited by 97
Self-citationThe neural coding framework for learning generative models
Ororbia, A., & Kifer, D. (2022) · Nature Communications · doi:10.1038/s41467-022-29632-7 · Cited by 57
“Doing so might also allow the machine learning community to make further progress on problems even harder than generative modeling, such the problem of active inference [23, 74] and continual temporal prediction [61] when an agent must interact with dynamic environments.”
BackgroundThe Predictive Dynamics of Happiness and Well-Being
Miller, M., Kiverstein, J., & Rietveld, E. (2022) · Emotion Review · doi:10.1177/17540739211063851 · Cited by 53
“A recent agent-based model showed that in order to optimize a model of the environment an agent must strike the right balance between epistemic actions that explore the environment for new policies, and pragmatic actions that exploit existing policies (Tschantz et al., 2020).”
BackgroundClark, A. (2022) · Australasian Journal of Philosophy · doi:10.1080/00048402.2022.2122523 · Cited by 72
“The ability of active inference systems to fi nd solutions of this kind has now been demonstrated in multiple simulation studies [Friston et al. 2015; Parr and Friston 2017; Tschantz et al. 2020].”
MethodPredictive processing in cognitive robotics: A review
Ciria, A., Schillaci, G., Pezzulo, G., Hafner, V. V., & Lara, B. (2021) · Neural Computation · doi:10.1162/neco_a_01383 · Cited by 51
“The few non-robotic models that focus on learning of generative models are based on the expected free energy formulations and use very simplified agents and behaviors (Tschantz et al., 2020; Baltieri and Buckley, 2017; Ueltzhöffer, 2018; Millidge, 2020).”
BackgroundInfluentialA Step-by-Step Tutorial on Active Inference and its Application to Empirical Data
Smith, R., Friston, K., & Whyte, C. (2021) · doi:10.31234/osf.io/b4jm6 · Cited by 51
Maisto, D., Barca, L., Van den Bergh, O., & Pezzulo, G. (2021) · Psychological Review · doi:10.1037/rev0000290 · Cited by 49
Hezemans, F. H., Wolpe, N., O’Callaghan, C., Ye, R., Rua, C., Jones, P. S., Murley, A. G., Holland, N., Regenthal, R., Tsvetanov, K. A., et al. (2022) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1010079 · Cited by 53
Epistemic Communities under Active Inference
Albarracin, M., Demekas, D., Ramstead, M. J. D., & Heins, C. (2022) · Entropy · doi:10.3390/e24040476 · Cited by 60
“This unique formulation of confirma- 1322 tion bias as an epistemic phenomenon helps explain how individuals continuously forage 1323 their environment for information, but may become stuck in a so-called ‘bad bootstrap’ that 1324 simply reinforces existing beliefs about the world, which in the face of new information…”
BackgroundActive inference models do not contradict folk psychology
Smith, R. S., Ramstead, M. J. D., & Kiefer, A. B. (2022) · Synthese · doi:10.1007/s11229-022-03480-w · Cited by 61
“…work has also explored how organisms learn action-oriented models of their ecological niche, where these models need not be fully accurate—but simply contain the generative structure and prior preferences most useful for guiding adaptive behavior within that niche (Tschantz et al., 2020).”
BackgroundSmith, R. S., Steklis, H. D., Steklis, N. G., Weihs, K. L., & Lane, R. D. (2020) · Biological Psychology · doi:10.1016/j.biopsycho.2020.107925 · Cited by 52
Affective experience in the predictive mind: a review and new integrative account
Fernandez Velasco, P., & Loev, S. (2021) · Synthese · doi:10.1007/s11229-020-02755-4 · Cited by 43
Designing ecosystems of intelligence from first principles
Friston, K. J., Ramstead, M. J. D., Kiefer, A. B., Tschantz, A., Buckley, C. L., Albarracin, M., Pitliya, R. J., Heins, C., Klein, B., Millidge, B., et al. (2024) · Collective Intelligence · doi:10.1177/26339137231222481 · Cited by 94
“…inference provides a promising framework for learning representations in which distinct generative factors are disentangled (Higgins et al., 2018), via the sensorimotor contingencies associated with controllable latent factors (Hinton et al., 2011; Tschantz et al., 2020; Van de Maele et al., 2021).”
BackgroundSelf-citationDeane, G. (2021) · Neuroscience of Consciousness · doi:10.1093/nc/niab024 · Cited by 38
“The breadth of explanations within this approach—from microscale explanations applied to understand the adaptive behaviour of bacteria (Tschantz et al. 2020) and plants (Calvo and Friston 2017), all theway up to social and cultural dynamics (Veissière et al.”
BackgroundPost-injury pain and behaviour: a control theory perspective
Seymour, B., Crook, R. J., & Chen, Z. S. (2023) · Nature reviews. Neuroscience · doi:10.1038/s41583-023-00699-5 · Cited by 45
Parr, T., Sajid, N., & Friston, K. J. (2020) · Entropy · doi:10.3390/e22050552 · Cited by 39
“It may be that the data-generating process is of a form that requires some transformation of the x variables, or even that the generative model is not an accurate description of the data-generating process [68].”
BackgroundNeural dynamics under active inference: plausibility and efficiency of information processing
Da Costa, L., Parr, T., Sengupta, B., & Friston, K. (2021) · Entropy · doi:10.3390/e23040454 · Cited by 38
Understanding, Explanation, and Active Inference
Parr, T., & Pezzulo, G. (2021) · Frontiers in Systems Neuroscience · doi:10.3389/fnsys.2021.772641 · Cited by 36
“It also represents beliefs about how the world should be—from the perspective of some (biological or synthetic) creature (Bruineberg et al., 2016; Tschantz et al., 2020).”
BackgroundNeural Dynamics under Active Inference: Plausibility and Efficiency of Information Processing
Costa, L. D., Parr, T., Sengupta, B., & Friston, K. J. (2021) · UCL Discovery (University College London) · Cited by 37
Reward maximization through discrete active inference
Da Costa, L., Sajid, N., Parr, T., Friston, K., & Smith, R. (2023) · Neural Computation · doi:10.1162/neco_a_01574 · Cited by 35
The Problem of Meaning: The Free Energy Principle and Artificial Agency
Kiverstein, J. D., Kirchhoff, M. D., & Froese, T. (2022) · Frontiers in Neurorobotics · doi:10.3389/fnbot.2022.844773 · Cited by 48
“As Tschantz et al. have noted, an active inference agent will tend to forage for information in parts of the environment expected to maximize instrumental value (Tschantz et al., 2020: p.7).”
BackgroundMethodInfluentialIs free-energy minimisation the mark of the cognitive?
Kiverstein, J., & Sims, M. (2021) · Biology & Philosophy · doi:10.1007/s10539-021-09788-0 · Cited by 23
Mann, S. F., Pain, R., & Kirchhoff, M. D. (2022) · Biology & Philosophy · doi:10.1007/s10539-022-09864-z · Cited by 34
“Tschantz et al. (2020) simulate bacterial chemotaxis, and give an active inference interpretation.”
BackgroundPezzulo, G., D’Amato, L., Mannella, F., Priorelli, M., de Maele, T. V., Stoianov, I. P., & Friston, K. J. (2024) · Annals of the New York Academy of Sciences · doi:10.1111/nyas.15118 · Cited by 32
“13,15,104,105 It is important to bear in mind that generative models encode the underlying causes of observations.”
BackgroundCognitive neurorobotics and self in the shared world, a focused review of ongoing research
Tani, J., & White, J. B. (2020) · Adaptive Behavior · doi:10.1177/1059712320962158 · Cited by 28
“Human infants or artificial agents may explore the world by seeking some intrinsic rewards such as novelty in interacting with their environments (Oudeyer et al., 2007; Schmidhuber, 1991; Tschantz et al., 2020).”
BackgroundKirchhoff, M. D., Kiverstein, J. D., & Robertson, I. (2022) · The British Journal for the Philosophy of Science · doi:10.1086/720861 · Cited by 36
Friston, K. J., Costa, L. D., Tschantz, A., Kiefer, A. B., Salvatori, T., Neacsu, V., Koudahl, M., Heins, C., Sajid, N., Marković, D., et al. (2024) · Biological Psychology · doi:10.1016/j.biopsycho.2024.108891 · Cited by 44
Self-citationCurious Inferences: Reply to Sun and Firestone on the Dark Room Problem
Seth, A. K., Millidge, B., Buckley, C. L., & Tschantz, A. (2020) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2020.05.011 · Cited by 23
Self-citationThe free energy principle: it’s not about what it takes, it’s about what took you there
Constant, A. (2021) · Biology & Philosophy · doi:10.1007/s10539-021-09787-1 · Cited by 24
Predictive processing and anti-representationalism
Facchin, M. (2021) · Synthese · doi:10.1007/s11229-021-03304-3 · Cited by 25
Active inference through whiskers
Mannella, F., Maggiore, F., Baltieri, M., & Pezzulo, G. (2021) · Neural Networks · doi:10.1016/j.neunet.2021.08.037 · Cited by 25
Useful misrepresentation: perception as embodied proactive inference
Martin, J. M., Solms, M., & Sterzer, P. (2021) · Trends in Neurosciences · doi:10.1016/j.tins.2021.04.007 · Cited by 28
Khajehnejad, M. A., Habibollahi, F., Loeffler, A., Paul, A., Razi, A., & Kagan, B. J. (2025) · Cyborg and Bionic Systems · doi:10.34133/cbsystems.0336 · Cited by 37
Resilience and active inference
Miller, M. M., Albarracin, M., Pitliya, R. J., Kiefer, A. B., Mago, J., Gorman, C., Friston, K. J., & Ramstead, M. J. D. (2022) · Frontiers in Psychology · doi:10.3389/fpsyg.2022.1059117 · Cited by 27
“As there is no guarantee that the models entailed by agents are accurate representations of a changing reality, there is no guarantee that behavior premised on these beliefs will be optimal (Tschantz et al., 2020).”
BackgroundTschantz, A., Barca, L., Maisto, D., Buckley, C. L., Seth, A. K., & Pezzulo, G. (2021) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2021.02.16.431365 · Cited by 18
Self-citationSkora, L. I., Seth, A. K., & Scott, R. B. Y. (2021) · Neuroscience of Consciousness · doi:10.1093/nc/niab003 · Cited by 17
“According to this view, the ‘winning’ predictions are not necessarily those which are the most veridical, but those which best support adaptive interactions with the world (Seth, 2014, 2015; Clark, 2015, 2016; Tschantz et al., 2020).”
BackgroundSelf-citationOptimal sensing and control of run-and-tumble chemotaxis
Nakamura, K., & Kobayashi, T. J. (2022) · Physical Review Research · doi:10.1103/physrevresearch.4.013120 · Cited by 17
“To treat such an extension, we may need to consider other formulations such as the active inference framework which uses variational approximation of posterior distributions [51].”
BackgroundStructure learning enhances concept formation in synthetic Active Inference agents
Neacsu, V., Mirza, M. B., Adams, R. A., & Friston, K. J. (2022) · PLoS ONE · doi:10.1371/journal.pone.0277199 · Cited by 27
“These results are reminiscent of work by [85]: here, the authors demonstrate that in Active Inference, uncertainties pertaining to the agents’ goals and preferences are prioritised over other types of uncertainty.”
BackgroundResultOn the minimal theory of consciousness implicit in active inference
Whyte, C. J., Corcoran, A. W., Robinson, J., Smith, R., Moran, R. J., Parr, T., Friston, K. J., Seth, A. K., & Hohwy, J. (2026) · Physics of Life Reviews · doi:10.1016/j.plrev.2025.11.002 · Cited by 15
Self-citationGeometric methods for sampling, optimization, inference, and adaptive agents
Barp, A., Costa, L. D., França, G., Friston, K. J., Girolami, M., Jordan, M. I., & Pavliotis, G. A. (2022) · Handbook of statistics · doi:10.1016/bs.host.2022.03.005 · Cited by 32
“Choosing priors that minimise free energy leads to parsimonious models that explain the data at hand [271].”
BackgroundPreferences need inferences: Learning, valuation, and curiosity in aesthetic experiencePreprint
de Cruys, S. V., Bervoets, J., & Moors, A. (2021) · PsyArXiv (OSF Preprints) · doi:10.31234/osf.io/zh6nt · Cited by 23
Phenomenal transparency, cognitive extension, and predictive processing
Facchin, M. (2022) · Phenomenology and the Cognitive Sciences · doi:10.1007/s11097-022-09831-9 · Cited by 18
Active Inference, Bayesian Optimal Design, and Expected Utility
Sajid, N., Costa, L. D., Parr, T., & Friston, K. J. (2022) · Cambridge University Press eBooks · doi:10.1017/9781009026949.007 · Cited by 27
Adversarial inference: predictive minds in the attention economy
Bruineberg, J. P. (2023) · Neuroscience of Consciousness · doi:10.1093/nc/niad019 · Cited by 19
“…follows, I will provide a relatively abstract and non-mathematical version of the argumentation (some of this literature tends to be a bit technical both in terms of mathematical formalisms and the concepts involved, see Friston et al. 2015; Tschantz et al., 2020; Parr et al. 2022 for details).”
BackgroundNeural representations unobserved—or: a dilemma for the cognitive neuroscience revolution
Facchin, M. (2023) · Synthese · doi:10.1007/s11229-023-04418-6 · Cited by 19
“But if the world is represented in such a distortive way, the representation must be somewhat false or inaccurate—in a way that is nevertheless conductive to an agent’s behavioral success (cf. Tschantz et al., 2020 for a proof of concept).”
A generative spiking neural-network model of goal-directed behaviour and one-step planning
Basanisi, R., Brovelli, A., Cartoni, E., & Baldassarre, G. (2020) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1007579 · Cited by 22
“Recently, it has been shown that the contextual learning and use-for-planning of the world model encounter a difficult problem for which the world model can prematurely converge to sub-optimal solutions (‘bad-bootstrapping’ problem, [46]).”
BackgroundMethodInfluentialTschantz, A., Baltieri, M., Seth, A. K., & Buckley, C. L. (2020) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn48605.2020.9207382 · Cited by 92
“This term is formally equivalent to a number of established quantities, such as (expected) Bayesian surprise, mutual information, and the expected reduction in posterior entropy [11, 31], and has been used to describe various epistemic foraging behaviors, such as saccades [32, 33, 34, 35] and sentence comprehension [1…”
BackgroundSelf-citationOn efficient computation in active inference
Paul, A., Sajid, N., Costa, L. D., & Razi, A. (2024) · Expert Systems with Applications · doi:10.1016/j.eswa.2024.124315 · Cited by 28
“, controlling the environment to sample observations that lead to a lower F [Tschantz et al., 2020]. So, instead of passively inferring what caused observations, the agent starts to ‘actively’ infer, exerting control over the environment using available actions in U . The central advantage of this formalism is that th…”
BackgroundMethodInfluentialBayesian mechanics of perceptual inference and motor control in the brain
Kim, C. S. (2021) · Biological Cybernetics · doi:10.1007/s00422-021-00859-9 · Cited by 12
Backprop-Free Reinforcement Learning with Active Neural Generative Coding
Ororbia, A. G., & Mali, A. (2022) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v36i1.19876 · Cited by 25
Bayesian Brains and the Rényi Divergence
Sajid, N., Faccio, F., Costa, L. D., Parr, T., Schmidhuber, J., & Friston, K. J. (2022) · Neural Computation · doi:10.1162/neco_a_01484 · Cited by 18
“This formulation underwrites practical applications that characterise the brain as performing Bayesian inference including predictive coding ([54, 34, 59, 69, 43]), and active inference ([61, 16, 22, 47, 63]).”
BackgroundWei, R., McDonald, A. D., Mehta, R. K., & García, A. (2024) · Human Factors The Journal of the Human Factors and Ergonomics Society · doi:10.1177/00187208241295932 · Cited by 10
“This surprise minimization framework, often called the Free Energy Principle (FEP), contains signi fi cant descriptive and explanatory power, ranging from neuronal activity (Bogacz, 2017), to the survival behavior of organisms (Tschantz et al., 2020), to mental illness (Schwartenbeck et al., 2015).”
How particular is the physics of the free energy principle?
Aguilera, M., Millidge, B., Tschantz, A., & Buckley, C. L. (2021) · Physics of Life Reviews · doi:10.1016/j.plrev.2021.11.001 · Cited by 103
Self-citationPiloting the Update: The Use of Therapeutic Relationship for Change – A Free Energy Account
Hauke, G., & Lohr, C. (2022) · Frontiers in Psychology · doi:10.3389/fpsyg.2022.842488 · Cited by 11
Wei, R., Garcia, A., McDonald, A. D., Markkula, G. M., Engström, J., & O’Kelly, M. (2025) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2025.3574552 · Cited by 16
“This term is usually referred to as “risk” because of its relationship to the deviation from an agent’s goal [38].”
Martínez, E., Kim, J. W., Barz, T., & Bournazou, M. N. C. (2021) · Computer-aided chemical engineering/Computer aided chemical engineering · doi:10.1016/b978-0-323-88506-5.50066-8 · Cited by 7
Extended Predictive Minds: do Markov Blankets Matter?
Facchin, M. (2021) · Review of Philosophy and Psychology · doi:10.1007/s13164-021-00607-9 · Cited by 15
“Importantly, in more complex systems9 free-energy minimization affords an optimal way to balance explorative (or epistemic) actions and exploitative (or pragmatic) actions, while making the agent learn the most efficient and minimalistic routes to success (e.g. Friston et al. 2016; Tschantz, 2020).”
BackgroundOn Epistemics in Expected Free Energy for Linear Gaussian State Space Models
Koudahl, M., Kouw, W. M., & de Vries, B. (2021) · Entropy · doi:10.3390/e23121565 · Cited by 14
“This sequence of actions is then considered either as an explicit vector of control signals [8,13–15] or amortised for instance by neural networks [16–19].”
BackgroundWorld Model Learning from Demonstrations with Active Inference: Application to Driving Behavior
Wei, R., García, A., McDonald, A. D., Markkula, G. M., Engström, J., Supeene, I., & O’Kelly, M. (2023) · Communications in computer and information science · doi:10.1007/978-3-031-28719-0_9 · Cited by 8
“Prior works addressed this by interacting with an environment [30] and receiving real-time expert feedback [24].”
BackgroundOn Kalman-Bucy filters, linear quadratic control and active inferencePreprint
Baltieri, M., & Buckley, C. L. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2005.06269 · Cited by 17
“a generative model is not necessarily one-to-one, and different generative models can ensure regulation while representing different sets of assumptions [9, 96].”
BackgroundSelf-citationResilience phenotypes derived from an active inference account of allostasis
Harrison, L., Gracias, A. J., Friston, K. J., & Buckwalter, J. G. (2025) · Frontiers in Behavioral Neuroscience · doi:10.3389/fnbeh.2025.1524722 · Cited by 12
Natural selection finds natural gradientPreprint
Costa, L. D., Parr, T., Sengupta, B., & Friston, K. J. (2020) · arXiv (Cornell University) · Cited by 10
Wang, Y., He, Y., Yu, F. R., Wu, K., & Chen, S. (2024) · IEEE Transactions on Mobile Computing · doi:10.1109/tmc.2024.3506161 · Cited by 6
Sims, M. (2023) · Biological Theory · doi:10.1007/s13752-022-00426-w · Cited by 8
“…statistical analogue in the notion of recognition models within the theory of active inference (Friston 2010; Pezzulo et al. 2015; Corcoran et al. 2020; Tschantz et al. 2020) where they, together with generative models, function to proactively control perception and action (Sims and Pezzulo 2021).”
BackgroundThe predictive brain model in diagnostic reasoning
Lim, T. K. (2021) · The Asia Pacific Scholar · doi:10.29060/taps.2021-6-2/ra2370 · Cited by 7
Active Strategies for Multisensory Conflict Suppression in the Virtual Hand IllusionPreprint
Lanillos, P., Franklin, S., Maselli, A., & Franklin, D. W. (2020) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2020.07.08.191304 · Cited by 5
A neural active inference model of perceptual-motor learning
Yang, Z., Diaz, G. J., Fajen, B. R., Bailey, R., & Ororbia, A. G. (2023) · Frontiers in Computational Neuroscience · doi:10.3389/fncom.2023.1099593 · Cited by 5
“The term rephrased as Rt,e is known as the epistemic (also known as intrinsic, uncertainty-reducing or information-seeking) component (Tschantz et al., 2020b), which measures the prediction error between the estimation of the future state by the transition model and the state predicted by the…”
BackgroundMethodInfluentialBackprop-Free Reinforcement Learning with Active Neural Generative CodingPreprint
Ororbia, A. G., & Mali, A. (2021) · doi:10.31219/osf.io/3nfcm · Cited by 3
“While the problems we examined varied in difficulty and certainly prove challenging for most RL models 7 , we were not unable (due to limited available computational resources) to examine the ideal setting with which we believe that the ANGC framework will yield the most interesting and important…”
BackgroundMethodInfluentialSocial Media Platforms: Trading with Prediction Error Minimization for Your AttentionPreprint
Ciria, A., Albarracin, M., Miller, M., & Lara, B. (2022) · Preprints.org · doi:10.20944/preprints202209.0259.v1 · Cited by 5
Structure and function in the predictive brain
Facchin, M., & Viola, M. (2025) · Biology & Philosophy · doi:10.1007/s10539-025-10000-w · Cited by 6
“In the case at hand, then, the ascribed function explains why we observe this particular neural structure, and the observed neural structure explains how the function is carried out.”
Adaptive visual selection: Predictive control of visual attention
Treviño, M. (2025) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2025.106523 · Cited by 3
Distributionally robust free energy principle for decision-making
Shafiei, A., Jesawada, H., Friston, K. J., & Russo, G. (2025) · Nature Communications · doi:10.1038/s41467-025-67348-6 · Cited by 9
“Run-and-tumble motions 51,52 might be an astute way to achieve this: interpreted through DR-FREE, tumbles might be driven by free energy maximization, needed to quantify across the environment a cost of ambiguity, and runs would be sampled from a free-energy minimizing policy that considers this…”
Where There Is Life There Is Mind… And Free Energy Minimisation?
Bogotá, J. D. (2024) · Interdisciplinary evolution research · doi:10.1007/978-3-031-70847-3_8 · Cited by 2
Learning to Listen, Listening to Learn: Music Perception and the Psychology of Enculturation
Pearce, M. T. (2025) · doi:10.1093/oso/9780198848004.001.0001 · Cited by 7
Iwashita, M., & Ishikawa, M. (2021) · IEEE Transactions on Cognitive and Developmental Systems · doi:10.1109/tcds.2021.3102993 · Cited by 3
“Active inference offers adaptive real-time perception and behavior control within a singular scheme [15].”
BackgroundA brain basis of dynamical intelligence for AI and computational neurosciencePreprint
Monaco, J. D., Rajan, K., & Hwang, G. M. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2105.07284 · Cited by 10
“Recent CN models have shown that this kind of sparse, action-driven plasticity may facilitate active inference for learning parsimonious and effective world models 166,167 .”
BackgroundActive strategies for multisensory conflict suppression in the virtual hand illusion
Lanillos, P., Franklin, S., Maselli, A., & Franklin, D. W. (2021) · Scientific Reports · doi:10.1038/s41598-021-02200-7 · Cited by 30
“Active inference 29 (AIF), a mathematical framework inspired by the Predictive (Bayesian) brain 30 that accounts for both perception and action, has been successfully applied, for instance, to model goal-driven behavior in organisms 31 and the physiology of dopamine 32 .”
Kouw, W. M. (2024) · Communications in computer and information science · doi:10.1007/978-3-031-77138-5_13 · Cited by 1
Wahidi, M., Rizk, A., & Imad, R. (2025) · IEEE Access · doi:10.1109/access.2025.3589514 · Cited by 3
“As shown in experiments simulating an agentic model of bacterial chemotaxis, AIF naturally balances these competing objectives to achieve high sample efficiency and robustness to suboptimal convergence [186].”
InfluentialDependable Task Offloading for CAVs in Web3: An Intelligence-Based Active Inference Approach
Li, X., Jing, T., Yu, F. R., Wang, H., Ren, Y., Zhu, M., Yan, Y., Ma, S., & Li, Y. (2026) · IEEE Transactions on Vehicular Technology · doi:10.1109/tvt.2026.3663258 · Cited by 1
“Additionally,unlike regression-basedDRL,FEP-drivenactiveinferencenaturally balancesexplorationandexploitationwithoutmanuallytuning parameters[25].”
Prediction in action: Toward an empirical science of active inference
Lageman, J., Fahrenfort, J. J., & Slagter, H. A. (2026) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2026.106817 · Cited by 2
The stuff matters: Consciousness, computation, and biology
Seth, A. K. (2026) · Behavioral and Brain Sciences · doi:10.1017/s0140525x26106906 · Cited by 3
Self-citationThinging Through Modelling. Active Inference meets Material EngagementPreprint
Paolo, L. D. D., Vindrola-Padrós, B., Clark, A., & Constant, A. (2025) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2025.08.17.670756 · Cited by 1
“The goal is to learn and use the least complex internal model (with the fewest parameters) that meets our needs [57].”
Exploring Action-Centric Representations Through the Lens of Rate-Distortion Theory
Varona, M. D. L., Buckley, C. L., & Millidge, B. (2023) · Communications in computer and information science · doi:10.1007/978-3-031-47958-8_12 · Cited by 2
“In the context of active inference, it has been shown that action-oriented models learn parsimonious representations of the environment by capturing relevant information for behaviour [21].”
BackgroundSelf-citationNeuron-level Prediction and Noise can Implement Flexible Reward-Seeking BehaviorPreprint
Li, C., Brenner, J. W., Boesky, A. P., Ramanathan, S., & Kreiman, G. (2024) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2024.05.22.595306 · Cited by 1
“Although these models have been successful in certain environments [23, 31, 46], one must still specify learned target distributions, and scaling them up is an ongoing problem [45].”
Background
Metadata
- Authors
- Tschantz, A., Seth, A. K., & Buckley, C. L.
- Affiliations
- University of Sussex (GB)
- Canadian Institute for Advanced Research (CA)
- Corresponding author
- Tschantz, Alexander
- Buckley, Christopher L.
- Venue
- PLOS Computational Biology
- Publisher
- Public Library of Science (PLoS)
- Published
- 2020-04-23
- Volume
- 16
- Issue
- 4
- Pages
- e1007805
- ISSN
- 1553-734X (ISSN-L)
- 1553-7358
- PMC
- PMC7200021
- OpenAlex
- W3017990858
- Semantic Scholar
- 6417d1bfc8f7dc8df7b04e50f8eb9188784b776a
- Cited by
- 121
- References
- 80
- Citations by year
- 2026: 8 · 2025: 25 · 2024: 9 · 2023: 9 · 2022: 23 · 2021: 31 · 2020: 16
- FWCI
- 10.61
- Type
- journal article
- Language
- en
- Update notices
- new_version (2020-05-05)
- Open access
- gold
- doi.org
- Licence
- Locations
- PLoS Computational Biology (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- RePEc: Research Papers in Economics (submittedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Figshare (submittedVersion, cc-by)
- Funders
- Canadian Institute for Advanced Research
- Biotechnology and Biological Sciences Research Council (BB/P022197/1)
- Topics
- Life Sciences › Neuroscience › Cognitive Neuroscience › Embodied and Extended Cognition
- Physical Sciences › Computer Science › Artificial Intelligence › Reinforcement Learning in Robotics
- Physical Sciences › Computer Science › Artificial Intelligence › Bayesian Modeling and Causal Inference
- Keywords
- active inference, Bayesian learning, goal-directed behavior, information seeking, model convergence, bacterial chemotaxis, sample efficiency, agent-environment interaction
- MeSH terms
- Algorithms, Chemotaxis, Environment, Goals, Learning, Models, Biological, Bacterial Physiological Phenomena, Computational Biology
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.