Active inference and agency: optimal control without cost functions
Open accessThe DOI matches its Crossref record; 85 of the 85 references it cites were found in a registry or an index.
Abstract
This paper describes a variational free-energy formulation of (partially observable) Markov decision problems in decision making under uncertainty. We show that optimal control can be cast as active inference. In active inference, both action and posterior beliefs about hidden states minimise a free energy bound on the negative log-likelihood of observed states, under a generative model. In this setting, reward or cost functions are absorbed into prior beliefs about state transitions and terminal states. Effectively, this converts optimal control into a pure inference problem, enabling the application of standard Bayesian filtering techniques. We then consider optimal trajectories that rest on posterior beliefs about hidden states in the future. Crucially, this entails modelling control as a hidden state that endows the generative model with a representation of agency. This leads to a distinction between models with and without inference on hidden control states; namely, agency-free and agency-based models, respectively.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Figures






Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Another avenue is training paradigms or biofeedback: patients could be given tasks that implicitly re-balance their predictive coding (58).”
From Predictive coding and neurocomputational psychiatry: a mechanistic framework for understanding mental disorders (2026) · cites this paper as [58]
Experimental — accuracy not yet measuredThe check wasn’t confident either way, so it gives no verdict — never disputed.
In the citing paper
“In OCD, the precision assigned to ϵ becomes pathologically high, effectively amplifying small discrepancies and driving excessive belief updates, even when the external world offers reassurance (58).”
From Predictive coding and neurocomputational psychiatry: a mechanistic framework for understanding mental disorders (2026) · cites this paper as [58]
Experimental — accuracy not yet measuredThe check wasn’t confident either way, so it gives no verdict — never disputed.
In the citing paper
“Computational models suggest that increasing the precision of certain prediction errors can produce OCD-like repetitive checking behaviour as the system tries to resolve an endless stream of perceived mismatches (56, 58).”
From Predictive coding and neurocomputational psychiatry: a mechanistic framework for understanding mental disorders (2026) · cites this paper as [58]
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
(1988) · Cited by 16,582
Rescorla, R. A. (1972) · Medical Entomology and Zoology · Cited by 9,661
Watkins, C. J. C. H., & Dayan, P. (1992) · Machine Learning · doi:10.1007/bf00992698 · Cited by 9,083
The free-energy principle: a unified brain theory?
Friston, K. (2010) · Nature Reviews Neuroscience · doi:10.1038/nrn2787 · Cited by 8,120 · cited 5×
“This formulation of behaviour is based on ergodic arguments about the nature of self organising systems (Ashby 1947)—for a fuller discussion please see Friston and Ao (2012) and Friston (2010) for their neurobiological implications .”
BackgroundMethodInfluentialSelf-citationRao, R. P. N., & Ballard, D. H. (1999) · Nature Neuroscience · doi:10.1038/4580 · Cited by 5,483 · cited 2×
“1995) and, under some simplifying assumptions, corresponds to predictive coding (Rao and Ballard 1999).”
BackgroundClassical conditioning II: Current research and theory.
Black, A. H., & Prokasy, W. F. (1972) · Appleton-Century-Crofts eBooks · Cited by 4,449
Planning and acting in partially observable stochastic domains
Kaelbling, L. P., Littman, M. L., & Cassandra, A. R. (1998) · Artificial Intelligence · doi:10.1016/s0004-3702(98)00023-x · Cited by 4,149 · cited 1×
“This leads to an extension of the MDP framework to accommodate partially observed states (Kaelbling et al. 1998).”
BackgroundGigerenzer, G., & Gaissmaier, W. (2011) · Annual Review of Psychology · doi:10.1146/annurev-psych-120709-145346 · Cited by 4,037 · cited 2×
“2009) and using heuristic policies (Gigerenzer and Gaissmaier 2011) based on the destruction of costly fixed point attractors (Friston 2010).”
BackgroundMethodWatkins, C. J. C. H., & Dayan, P. (1992) · Machine Learning · doi:10.1023/a:1022676722315 · Cited by 3,907 · cited 2×
“This paper considers discrete time (Markov) decision processes of the sort found in optimal control theory, models of behaviour and decision making (Bellman 1952; Watkins and Dayan 1992; Camerer 2003; Daw and Doya 2006; Todorov 2006; Dayan and Daw 2008).”
BackgroundSimple Statistical Gradient-Following Algorithms for Connectionist Reinforcement Learning
Williams, R. J. (1992) · Machine Learning · doi:10.1007/bf00992696 · Cited by 3,416
Dynamic Programming and Markov Processes.
Freimer, M., & Howard, R. A. (1961) · Journal of the American Statistical Association · doi:10.2307/2282268 · Cited by 3,378
High Gamma Power Is Phase-Locked to Theta Oscillations in Human Neocortex
Canolty, R. T., Edwards, E., Dalal, S. S., Soltani, M., Nagarajan, S. S., Kirsch, H. E., Berger, M. S., Barbaro, N. M., & Knight, R. T. (2006) · Science · doi:10.1126/science.1128115 · Cited by 2,650 · cited 2×
“These simulations produce dynamics that are not dissimilar to the nested oscillations seen in theta-gamma phase coupling between the prefrontal cortex and hippocampal system (Canolty et al. 2006; Axmacher et al. 2010), where there are about eight gamma cycles for each theta cycle.”
Background(1998) · Springer eBooks · doi:10.1007/978-94-011-5014-9 · Cited by 2,418
A View of the Em Algorithm that Justifies Incremental, Sparse, and other Variants
Neal, R. M., & Hinton, G. E. (1998) · Learning in Graphical Models · doi:10.1007/978-94-011-5014-9_12 · Cited by 2,187 · cited 2×
“2009), and expectation maximisation (Toussaint and Storkey 2006), both of which can be formulated as minimising variational free energy (Neal and Hinton 1998).”
BackgroundMethod(2003) · Cited by 1,680
Predictive coding under the free-energy principle
Friston, K., & Kiebel, S. (2009) · Philosophical Transactions of the Royal Society B: Biological Sciences · doi:10.1098/rstb.2008.0300 · Cited by 1,579
Self-citationToward a modern theory of adaptive networks: Expectation and prediction.
Sutton, R. S., & Barto, A. G. (1981) · Psychological Review · doi:10.1037/0033-295x.88.2.135 · Cited by 1,450 · cited 2×
“In these schemes (Rescorla and Wagner 1972; Sutton and Barto 1981; Watkins and Dayan 1992; Friston et al. 1994; Montague et al. 1995) the value function is itself learnt; this enables one to solve Markov decision problems without learning the model (transition probabilities), because the value function acts as a guida…”
BackgroundA free energy principle for the brain
Friston, K., Kilner, J., & Harrison, L. (2006) · Journal of Physiology-Paris · doi:10.1016/j.jphysparis.2006.10.001 · Cited by 1,369 · cited 2×
“…that active inference is a corollary of the free-energy principle: 3.1 The free-energy principle and active inference The free-energy principle (Friston et al. 2006) tries to explain how agents occupy a small number of attracting states in terms of minimising the Shannon entropy of the…”
BackgroundInfluentialSelf-citationAttention, uncertainty, and free-energy
Feldman, H., & Friston, K. J. (2010) · Frontiers in Human Neuroscience · doi:10.3389/fnhum.2010.00215 · Cited by 1,365 · cited 1×
“Attention and the Posner paradigm Feldman and Friston (2010) Attention and biased competition Retinal stabilization and oculomotor reflexes Friston et al.”
BackgroundSelf-citationMotivation concepts in behavioral neuroscience
Berridge, K. C. (2004) · Physiology & Behavior · doi:10.1016/j.physbeh.2004.02.004 · Cited by 1,302 · cited 1×
“A simple example here would be the context and time sensitive nature of motivational and physiological drives (Berridge 2004) that render the same states differentially rewarding, depending upon when they were last visited (for example, eating or drinking).”
BackgroundDayan, P., Hinton, G. E., Neal, R. M., & Zemel, R. S. (1995) · Neural Computation · doi:10.1162/neco.1995.7.5.889 · Cited by 1,207
Gläscher, J., Daw, N., Dayan, P., & O'Doherty, J. P. (2010) · Neuron · doi:10.1016/j.neuron.2010.04.016 · Cited by 1,166 · cited 1×
“It is in this sense that reinforcement learning is referred to as model free: see Dayan and Daw (2008) and Gläscher et al. (2010).”
MethodOn the theory of dynamic programming
Bellman, R. (1952) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.38.8.716 · Cited by 998 · cited 4×
“This paper considers discrete time (Markov) decision processes of the sort found in optimal control theory, models of behaviour and decision making (Bellman 1952; Watkins and Dayan 1992; Camerer 2003; Daw and Doya 2006; Todorov 2006; Dayan and Daw 2008).”
BackgroundMethodInfluentialKeeping the neural networks simple by minimizing the description length of the weights
Hinton, G. E., & van Camp, D. (1993) · Proceedings of the sixth annual conference on Computational learning theory - COLT '93 · doi:10.1145/168304.168306 · Cited by 994
Hierarchical Models in the Brain
Friston, K. (2008) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1000211 · Cited by 952 · cited 2×
“Neurobiologically, this generally reduces to some form of associative plasticity (Friston 2008).”
BackgroundSelf-citationJONES, J. E. (1971) · Nature · doi:10.1038/233571b0 · Cited by 952
Cross-frequency coupling supports multi-item working memory in the human hippocampus
Axmacher, N., Henseler, M. M., Jensen, O., Weinreich, I., Elger, C. E., & Fell, J. (2010) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.0911531107 · Cited by 936 · cited 1×
“These simulations produce dynamics that are not dissimilar to the nested oscillations seen in theta-gamma phase coupling between the prefrontal cortex and hippocampal system (Canolty et al. 2006; Axmacher et al. 2010), where there are about eight gamma cycles for each theta cycle.”
Birkhoff, G. D. (1931) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.17.2.656 · Cited by 867 · cited 1×
“Under ergodic assumptions, this entropy is (almost surely) the long-term time average of self information or surprise (Birkhoff 1931).”
BackgroundAction and behavior: a free-energy formulation
Friston, K. J., Daunizeau, J., Kilner, J., & Kiebel, S. J. (2010) · Biological Cybernetics · doi:10.1007/s00422-010-0364-z · Cited by 866 · cited 2×
“In other words, active inference extends the minimisation of variational free energy that underlies (approximate) Bayesian inference to include action (Friston et al. 2010).”
BackgroundSelf-citationSimple statistical gradient-following algorithms for connectionist reinforcement learning
Williams, R. J. (1992) · Machine Learning · doi:10.1023/a:1022672621406 · Cited by 817
Action understanding and active inference
Friston, K., Mattout, J., & Kilner, J. (2011) · Biological Cybernetics · doi:10.1007/s00422-011-0424-z · Cited by 665 · cited 3×
“This speaks to important questions that relate to theory of mind and the role of things like the mirror neuron system in active inference (Friston et al. 2011).”
BackgroundMethodSelf-citationPrinciples of the Self-Organizing Dynamic System
Ashby, W. R. (1947) · The Journal of General Psychology · doi:10.1080/00221309.1947.9918144 · Cited by 639 · cited 2×
“This formulation of behaviour is based on ergodic arguments about the nature of self organising systems (Ashby 1947)—for a fuller discussion please see Friston and Ao (2012) and Friston (2010) for their neurobiological implications .”
MethodDynamic Programming and Markov Processes.
Kaufman, H., & Howard, R. A. (1961) · The American Mathematical Monthly · doi:10.2307/2312519 · Cited by 638
Predictive Representations of State
Littman, M. L., & Sutton, R. S. (2001) · neural information processing systems · Cited by 600
Decision theory, reinforcement learning, and the brain
Dayan, P., & Daw, N. D. (2008) · Cognitive, Affective, & Behavioral Neuroscience · doi:10.3758/cabn.8.4.429 · Cited by 580 · cited 2×
“This paper considers discrete time (Markov) decision processes of the sort found in optimal control theory, models of behaviour and decision making (Bellman 1952; Watkins and Dayan 1992; Camerer 2003; Daw and Doya 2006; Todorov 2006; Dayan and Daw 2008).”
BackgroundMethodProbabilistic Inference and Influence Diagrams
Shachter, R. D. (1988) · Operations Research · doi:10.1287/opre.36.4.589 · Cited by 494 · cited 1×
“This makes maximizing utility equivalent to maximizing the likelihood of desired observations (Cooper 1988; Pearl 1988; Shachter 1988).”
BackgroundProbabilistic inference for solving discrete and continuous state Markov Decision Processes
Toussaint, M., & Storkey, A. (2006) · Proceedings of the 23rd international conference on Machine learning - ICML '06 · doi:10.1145/1143844.1143963 · Cited by 482 · cited 3×
“Later, Dayan and Hinton (1997) proposed an expectation maximization algorithm for reinforcement learning in the case of immediate rewards, while (Toussaint and Storkey 2006) cast the problem of computing optimal policies as a likelihood maximization problem.”
BackgroundThe computational neurobiology of learning and reward
Daw, N. D., & Doya, K. (2006) · Current Opinion in Neurobiology · doi:10.1016/j.conb.2006.03.006 · Cited by 465 · cited 1×
“This paper considers discrete time (Markov) decision processes of the sort found in optimal control theory, models of behaviour and decision making (Bellman 1952; Watkins and Dayan 1992; Camerer 2003; Daw and Doya 2006; Todorov 2006; Dayan and Daw 2008).”
BackgroundReinforcement learning or active inference?
Friston, K. J., Daunizeau, J., & Kiebel, S. J. (2009) · PLoS ONE · doi:10.1371/journal.pone.0006421 · Cited by 443 · cited 6×
“An example of learning policies through priors was presented in Friston et al. (2009), where an agent was immersed in a controlled environment that enforced optimal trajectories through state space.”
BackgroundInfluentialSelf-citationDopamine, affordance and active inference
Friston, K. J., Shiner, T., FitzGerald, T., Galea, J. M., Adams, R., Brown, H., Dolan, R. J., Moran, R., Stephan, K. E., & Bestmann, S. (2012) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1002327 · Cited by 381 · cited 2×
“2009), action selection (Friston et al. 2012) optimal motor control (Friston 2011) and dynamical systems theory (Friston and Ao 2012).”
BackgroundSelf-citationLinearly-solvable Markov decision problems
Todorov, E. (2007) · Advances in Neural Information Processing Systems 19 · doi:10.7551/mitpress/7503.003.0176 · Cited by 334 · cited 1×
“This paper considers discrete time (Markov) decision processes of the sort found in optimal control theory, models of behaviour and decision making (Bellman 1952; Watkins and Dayan 1992; Camerer 2003; Daw and Doya 2006; Todorov 2006; Dayan and Daw 2008).”
BackgroundWhat is optimal about motor control?
Friston, K. (2011) · Neuron · doi:10.1016/j.neuron.2011.10.018 · Cited by 332 · cited 2×
“2012) optimal motor control (Friston 2011) and dynamical systems theory (Friston and Ao 2012).”
BackgroundSelf-citationA tutorial on variational Bayesian inference
Fox, C. W., & Roberts, S. J. (2012) · Artificial Intelligence Review · doi:10.1007/s10462-011-9236-8 · Cited by 326
Bee foraging in uncertain environments using predictive hebbian learning
Montague, P. R., Dayan, P., Person, C., & Sejnowski, T. J. (1995) · Nature · doi:10.1038/377725a0 · Cited by 311 · cited 2×
“In these schemes (Rescorla and Wagner 1972; Sutton and Barto 1981; Watkins and Dayan 1992; Friston et al. 1994; Montague et al. 1995) the value function is itself learnt; this enables one to solve Markov decision problems without learning the model (transition probabilities), because the value function acts as a guida…”
BackgroundSelf-citationPath integrals and symmetry breaking for optimal control theory
Kappen, H. J. (2005) · Journal of Statistical Mechanics Theory and Experiment · doi:10.1088/1742-5468/2005/11/p11011 · Cited by 309 · cited 2×
“In particular, formulating the problem in terms of Kullback–Leibler minimization (Kappen 2005a,b) and path integrals of cost functions using the Feynman–Kac formula (Theodorou et al. 2010; Braun et al. 2011).”
BackgroundMethodBehavioural studies of strategic thinking in games
Camerer, C. F. (2003) · Trends in Cognitive Sciences · doi:10.1016/s1364-6613(03)00094-9 · Cited by 304 · cited 1×
“This paper considers discrete time (Markov) decision processes of the sort found in optimal control theory, models of behaviour and decision making (Bellman 1952; Watkins and Dayan 1992; Camerer 2003; Daw and Doya 2006; Todorov 2006; Dayan and Daw 2008).”
BackgroundLinear Theory for Control of Nonlinear Stochastic Systems
Kappen, H. J. (2005) · Physical Review Letters · doi:10.1103/physrevlett.95.200201 · Cited by 299 · cited 2×
“In particular, formulating the problem in terms of Kullback–Leibler minimization (Kappen 2005a,b) and path integrals of cost functions using the Feynman–Kac formula (Theodorou et al. 2010; Braun et al. 2011).”
BackgroundMethodOptimal control as a graphical model inference problem
Kappen, H. J., Gómez, V., & Opper, M. (2012) · Machine Learning · doi:10.1007/s10994-012-5278-7 · Cited by 297 · cited 1×
“Examples here include minimisation of Kullback–Leibler divergences (Todorov 2008; Kappen et al. 2009), and expectation maximisation (Toussaint and Storkey 2006), both of which can be formulated as minimising variational free energy (Neal and Hinton 1998).”
MethodA Generalized Path Integral Control Approach to Reinforcement Learning
TheodorouEvangelos, BuchliJonas, & SchaalStefan (2010) · Journal of Machine Learning Research · doi:10.5555/1756006.1953033 · Cited by 280 · cited 2×
“See also (Theodorou et al. 2010), who exploit the same formalism but with a more classical motivation.”
BackgroundMethodValue-dependent selection in the brain: Simulation in a synthetic neural model
Friston, K. J., Tononi, G., Reeke, G. N., Sporns, O., & Edelman, G. M. (1994) · Neuroscience · doi:10.1016/0306-4522(94)90592-4 · Cited by 268 · cited 2×
“In these schemes (Rescorla and Wagner 1972; Sutton and Barto 1981; Watkins and Dayan 1992; Friston et al. 1994; Montague et al. 1995) the value function is itself learnt; this enables one to solve Markov decision problems without learning the model (transition probabilities), because the value function acts as a guida…”
BackgroundSelf-citationGeneral duality between optimal control and estimation
Todorov, E. (2008) · 2008 47th IEEE Conference on Decision and Control · doi:10.1109/cdc.2008.4739438 · Cited by 248 · cited 2×
“Indeed, the formal equivalence between optimal control and estimation was acknowledged by Kalman at the inception of Bayesian filtering schemes (Todorov 2008).”
BackgroundMethodOptimal learning: computational procedures for bayes-adaptive markov decision processes
Duff, M. O., & Barto, A. G. (2002) · ScholarWorks@UMassAmherst (University of Massachusetts Amherst) · Cited by 239
From Influence Diagrams to Junction Trees
Jensen, F., Jensen, F. V., & Dittmer, S. L. (1994) · Uncertainty Proceedings 1994 · doi:10.1016/b978-1-55860-332-5.50051-1 · Cited by 207
(2009) · Control Theory · doi:10.1109/9780470544334.ch10 · Cited by 207
Rao, R. P. N. (2010) · Frontiers in Computational Neuroscience · doi:10.3389/fncom.2010.00146 · Cited by 191 · cited 1×
“In many scenarios this is unrealistic, in that an agent cannot know the exact state of the world, given noisy or partial observations (Rao 2010).”
BackgroundObservable Operator Models for Discrete Stochastic Time Series
Jaeger, H. (2000) · Neural Computation · doi:10.1162/089976600300015411 · Cited by 174 · cited 1×
“This conditioning of observations on action (as opposed to conditioning states on action) is not unrelated to treatments based on observer operator models and predictive representations of state: see Jaeger (2000) and Littman et al. (2002).”
MethodCortical circuits for perceptual inference
Friston, K., & Kiebel, S. (2009) · Neural Networks · doi:10.1016/j.neunet.2009.07.023 · Cited by 172 · cited 6×
“An example of learning policies through priors was presented in Friston et al. (2009), where an agent was immersed in a controlled environment that enforced optimal trajectories through state space.”
BackgroundInfluentialSelf-citationUsing Expectation-Maximization for Reinforcement Learning
Dayan, P., & Hinton, G. E. (1997) · Neural Computation · doi:10.1162/neco.1997.9.2.271 · Cited by 172 · cited 2×
“Later, Dayan and Hinton (1997) proposed an expectation maximization algorithm for reinforcement learning in the case of immediate rewards, while (Toussaint and Storkey 2006) cast the problem of computing optimal policies as a likelihood maximization problem .”
BackgroundTani, J. (2003) · Neural Networks · doi:10.1016/s0893-6080(02)00214-9 · Cited by 164 · cited 2×
“This form of learning has been used to produce some compelling and animate behaviours (Tani 2003; Namikawa et al. 2011).”
BackgroundMethodAccelerated Neural Evolution through Cooperatively Coevolved Synapses
GomezFaustino, SchmidhuberJürgen, & MiikkulainenRisto (2008) · Journal of Machine Learning Research · doi:10.5555/1390681.1390712 · Cited by 164 · cited 3×
“, (Gomez and Miikkulainen 2001; Gomez et al. 2009), where agents are selected to maximise their expected reward through successive exposures to environmental contingencies.”
BackgroundMethodStochastic Boolean Satisfiability
Littman, M. L., Majercik, S. M., & Pitassi, T. (2001) · Journal of Automated Reasoning · doi:10.1023/a:1017584715408 · Cited by 161 · cited 1×
“Indeed, it is well known that the computational complexity of a problem is not changed when reducing it to an inference problem: see (Littman et al. 2001) for a treatment of this in the setting of stochastic satisfiability problems and probabilistic inference.”
BackgroundBirkhoff, G. D. (1931) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.17.12.656 · Cited by 159
Free Energy, Value, and Attractors
Friston, K., & Ao, P. (2012) · Computational and Mathematical Methods in Medicine · doi:10.1155/2012/937860 · Cited by 139 · cited 6×
“This formulation of behaviour is based on ergodic arguments about the nature of self organising systems (Ashby 1947)—for a fuller discussion please see Friston and Ao (2012) and Friston (2010) for their neurobiological implications .”
BackgroundMethodInfluentialSelf-citationBayesian modelling of Jumping-to-Conclusions bias in delusional patients
Moutoussis, M., Bentall, R. P., El-Deredy, W., & Dayan, P. (2011) · Cognitive Neuropsychiatry · doi:10.1080/13546805.2010.548678 · Cited by 137 · cited 1×
“its neurochemical mediation (Kishida et al. 2010; Moutoussis et al. 2011).”
Experiments with Infinite-Horizon, Policy-Gradient Estimation
Baxter, J., Bartlett, P. L., & Weaver, L. (2001) · Journal of Artificial Intelligence Research · doi:10.1613/jair.807 · Cited by 137 · cited 2×
“In direct policy searches e.g., (Williams 1992; Baxter et al. 2001), a policy is optimised by mapping each state directly to an action, without reference to the value of the state.”
BackgroundProbabilistic Inference in Influence Diagrams
Zhang, N. L. (1998) · Computational Intelligence · doi:10.1111/0824-7935.00073 · Cited by 120 · cited 1×
“Subsequent work focussed on efficient methods to solve the ensuing inference problem (Jensen et al. 1994; Zhang 1998).”
MethodNeuroeconomic Approaches to Mental Disorders
Kishida, K. T., King-Casas, B., & Montague, P. R. (2010) · Neuron · doi:10.1016/j.neuron.2010.07.021 · Cited by 117
Filatov, N. M., & Unbehauen, H. (2004) · Lecture notes in control and information sciences · doi:10.1007/b96083 · Cited by 116 · cited 1×
“Early work in this area addressed dual control problems; for example, Feldbaum (1961) discussed control in the absence of complete information using a Bayesian framework: see Filatov and Unbehauen (2004) for review.”
Recognizing Sequences of Sequences
Kiebel, S. J., von Kriegstein, K., Daunizeau, J., & Friston, K. J. (2009) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1000464 · Cited by 115 · cited 1×
“This endows the scheme with a context sensitivity that may be important in hierarchical generative models, in which final states are themselves specified by trajectories over slower timescales (Kiebel et al. 2009a).”
BackgroundSelf-citationPerception and hierarchical dynamics
Kiebel, S. J. (2009) · Frontiers in Neuroinformatics · doi:10.3389/neuro.11.020.2009 · Cited by 102 · cited 1×
“This endows the scheme with a context sensitivity that may be important in hierarchical generative models, in which final states are themselves specified by trajectories over slower timescales (Kiebel et al. 2009a).”
BackgroundA non-equilibrium free energy theorem for deterministic systems
EVANS, D. J. (2003) · Molecular Physics · doi:10.1080/0026897031000085173 · Cited by 100 · cited 1×
“These arguments suggest that the long term average of variational free energy upper bounds the (Shannon) entropy of observations over time; which implies that action must minimise variational free energy to resist the dispersion of its states by random fluctuations (Evans 2003).”
BackgroundA complete class theorem for statistical problems with finite sample spaces
Brown, L. D. (1981) · The Annals of Statistics · doi:10.1214/aos/1176345645 · Cited by 97 · cited 1×
“The fact that one can replace cost functions with priors to produce the same behaviour is related to the complete class theorem (Brown 1981).”
BackgroundGraphical Model Inference in Optimal Control of Stochastic Multi-Agent Systems
Van den Broek, B., Wiegerinck, W., & Kappen, B. (2008) · Journal of Artificial Intelligence Research · doi:10.1613/jair.2473 · Cited by 83 · cited 1×
“Related work on stochastic optimal control (Kappen 2005a,b; van den Broek et al. 2008; Rawlik et al. 2010), exploits the reduction of control problems to inference problems by appealing to variational techniques to provide efficient and computationally tractable solutions.”
BackgroundA Method for Using Belief Networks as Influence Diagrams
Cooper, G. F. (2013) · arXiv (Cornell University) · doi:10.48550/arxiv.1304.2346 · Cited by 81 · cited 1×
“This makes maximizing utility equivalent to maximizing the likelihood of desired observations (Cooper 1988; Pearl 1988; Shachter 1988).”
BackgroundFree energy minimisation algorithmfor decoding and cryptanalysis
MacKay, D. J. C. (1995) · Electronics Letters · doi:10.1049/el:19950331 · Cited by 79 · cited 1×
“Active inference assumes that this function is the same variational free energy used in approximate Bayesian inference (Hinton and van Camp 1993; MacKay 1995; Neal and Hinton 1998; Dayan et al. 1995; Beal 2003).”
MethodHierarchical POMDP Controller Optimization by Likelihood Maximization
Toussaint, M., Charlin, L., & Poupart, P. (2012) · arXiv (Cornell University) · doi:10.48550/arxiv.1206.3291 · Cited by 74 · cited 1×
“More recently, this approach has been pursued by applying Bayesian procedures (or minimising Kullback–Leibler divergences ) to problems of optimal decision making in MDPs Botvinick and An 2008; Hoffman et al. 2009; Toussaint et al. 2008).”
MethodPath integral control and bounded rationality
Braun, D. A., Ortega, P. A., Theodorou, E., & Schaal, S. (2011) · 2011 IEEE Symposium on Adaptive Dynamic Programming and Reinforcement Learning (ADPRL) · doi:10.1109/adprl.2011.5967366 · Cited by 54 · cited 3×
“The notion that the Bellman optimality principle “can be derived as a limit case” from the variational principles that underlie active inference also emerges in recent information theoretic formulations of bounded rationality (Braun et al. 2011): Braun et al consider control costs in terms of the (cross) entropy of ch…”
BackgroundMethodGoal-directed decision making in prefrontal cortex: A computational framework.
Botvinick, M., & An, J. (2009) · PubMed · Cited by 54
A Neurodynamic Account of Spontaneous Behaviour
Namikawa, J., Nishimoto, R., & Tani, J. (2011) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1002221 · Cited by 44 · cited 1×
“This form of learning has been used to produce some compelling and animate behaviours (Tani 2003; Namikawa et al. 2011).”
MethodL'Analyse Statistique Bayesienne.
Angers, J., & Robert, C. P. (1993) · Journal of the American Statistical Association · doi:10.2307/2290751 · Cited by 39 · cited 1×
“The complete class theorem states that any admissible decision rule (behaviour) is Bayes-optimal for at least one pair of prior beliefs and cost function (Robert 1992).”
BackgroundAn Expectation Maximization Algorithm for Continuous Markov Decision Processes with Arbitrary Reward
Hoffman, M. D., de Freitas, N., Arnaud Doucet, B., & Peters, J. (2009) · TUbilio (Technical University of Darmstadt) · Cited by 27
Approximate Inference and Stochastic Optimal ControlPreprint
Rawlik, K., Toussaint, M., & Vijayakumar, S. (2010) · arXiv (Cornell University) · doi:10.48550/arxiv.1009.3958 · Cited by 23 · cited 1×
“Related work on stochastic optimal control (Kappen 2005a,b; van den Broek et al. 2008; Rawlik et al. 2010), exploits the reduction of control problems to inference problems by appealing to variational techniques to provide efficient and computationally tractable solutions.”
BackgroundImplicit motion perception in schizotypy and schizophrenia: A Representational Momentum study
Jarrett, C. B., Phillips, M., Parker, A. J., & Senior, C. (2002) · Cognitive Neuropsychiatry · doi:10.1080/13546800143000104 · Cited by 19
Best-response play in partially observable card games
Oliehoek, F. A., Spaan, M. T. J., & Vlassis, N. A. (2005) · Cited by 18
(2010) · DSpace@MIT (Massachusetts Institute of Technology)
Research works printed but not linked: 2
- 47. Morgan Kaufmann, San Fransisco
- 77. Watkins CJ, Dayan P (1992) Q-learning. Mach Learn 8:279-292
Cited by
REBUS and the anarchic brain: Toward a unified model of the brain action of psychedelics
Carhart-Harris, R. L., & Friston, K. J. (2019) · Pharmacological Reviews · doi:10.1124/pr.118.017160 · Cited by 1,180
“…to the free-energy principle, we know that an agent can minimize free energy via perceptual (Friston and Kiebel, 2009) and active inference (Friston et al., 2012b), both of which entail sampling and sculpting the world to refine internal representations of it and thus reduce uncertainty or…”
BackgroundSelf-citationActive inference and epistemic value
Friston, K., Rigoli, F., Ognibene, D., Mathys, C., Fitzgerald, T., & Pezzulo, G. (2015) · Cognitive Neuroscience · doi:10.1080/17588928.2015.1020053 · Cited by 731
“We have previously considered epistemic value and salience using continuous time predictive coding schemes and saccadic searches (Friston et al., 2012).”
BackgroundMethodSelf-citationHohwy, J. (2016) · Noûs · doi:10.1111/nous.12062 · Cited by 578
Navigating the Affordance Landscape: Feedback Control as a Process Model of Behavior and Cognition
Pezzulo, G., & Cisek, P. (2016) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2016.03.013 · Cited by 419
“, apple reachability) can influence the competition between proximal actions (picking the berry versus walking) [14].”
The anatomy of choice: active inference and agency
Friston, K., Schwartenbeck, P., FitzGerald, T., Moutoussis, M., Behrens, T., & Dolan, R. J. (2013) · Frontiers in Human Neuroscience · doi:10.3389/fnhum.2013.00598 · Cited by 340
“…treatments of hierarchical Bayesian inference (Mathys et al., 2011), from theoretical neurobiology (Frank et al., 2007; Fletcher and Frith, 2009; Friston et al., 2012b; Pellicano and Burr, 2012) and from empirical studies (Fiorillo et al., 2008; Coull et al., 2011; Galea et al., 2012; Zokaei et…”
BackgroundMethodInfluentialSelf-citation
Metadata
- Authors
- Friston, K., Samothrakis, S., & Montague, R.
- Affiliations
- Wellcome Centre for Human Neuroimaging (GB)
- University of Essex (GB)
- Corresponding author
- Friston, Karl
- Venue
- Biological Cybernetics
- Published
- 2012-08-03
- Volume
- 106
- Issue
- 8-9
- Pages
- 523-541
- ISSN
- 0340-1200 (ISSN-L)
- 1432-0770
- OpenAlex
- W2091405003
- Cited by
- 241
- References
- 83
- Citations by year
- 2026: 11 · 2025: 16 · 2024: 20 · 2023: 24 · 2022: 12 · 2021: 43 · 2020: 30 · 2019: 24 · 2018: 14 · 2017: 12 · 2016: 10 · 2015: 11 · 2014: 7 · 2013: 4 · 2012: 3
- FWCI
- 4.10
- Type
- journal article
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- cc-by
- Locations
- Biological Cybernetics (publishedVersion, cc-by)
- PubMed (publishedVersion)
- UCL Discovery (University College London) (submittedVersion)
- VTechWorks (Virginia Tech) (submittedVersion)
- Funders
- Wellcome Trust (088130)
- National Institute on Drug Abuse (R01DA011723)
- Topics
- Life Sciences › Neuroscience › Cognitive Neuroscience › Embodied and Extended Cognition
- Physical Sciences › Computer Science › Artificial Intelligence › Reinforcement Learning in Robotics
- Physical Sciences › Computer Science › Artificial Intelligence › Bayesian Modeling and Causal Inference
- Keywords
- active inference, agency, optimal control, free energy principle, Markov decision process, decision making under uncertainty, Bayesian filtering, prior beliefs, variational inference
- MeSH terms
- Algorithms, Bayes Theorem, Decision Making, Markov Chains, Models, Neurological
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.