Deep active inference as variational policy gradients
Open accessThe claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Although Millidge's mathematics uses a POMDP formalism, the code implements a fully observable MDP without an encoder/decoder architecture.
“Although the mathematics in the paper are based on the formalism of a partially observable Markov decision process (POMDP), the code does not implement an encoder/decoder architecture, that is, the code implements a fully observable decision process, i.e., MDP.”
From Deconstructing Deep Active Inference: A Contrarian Information Gatherer (2024) · cites this paper as Millidge (2020)
The DAI_VPG implementation is a mixture of POMDP and MDP settings, with an MDP generative model and expected free energy adapted from the POMDP setting.
“Thus, (as it stands) the implementation of the DAI V P G agent is a mixture between the POMDP and MDP setting, where the generative model corresponds to an MDP, and the expected free energy is adapted from the POMDP setting.”
From Deconstructing Deep Active Inference: A Contrarian Information Gatherer (2024) · cites this paper as Millidge (2020)
The DAI_VPG agent is able to solve the CartPole environment.
“DAI V P G is able to solve the CartPole environment.”
From Deconstructing Deep Active Inference: A Contrarian Information Gatherer (2024) · cites this paper as Millidge (2020)
The DAI_VPG implementation has no hidden states, which raises uncertainty about how the intrinsic value is computed.
“Now, remember that in the implementation, there is no encoder Q(s τ ) and no decoder P (o τ |s τ ).In other words, there are no hidden states s τ , raising some uncertainty about how the intrinsic value is computed.”
From Deconstructing Deep Active Inference: A Contrarian Information Gatherer (2024) · cites this paper as Millidge (2020)
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Human-level control through deep reinforcement learning
Mnih, V., Kavukcuoglu, K., Silver, D., Rusu, A. A., Veness, J., Bellemare, M. G., Graves, A., Riedmiller, M., Fidjeland, A. K., Ostrovski, G., et al. (2015) · Nature · doi:10.1038/nature14236 · Cited by 30,357 · see citing works
Auto-Encoding Variational Bayes
Kingma, D. P., & Welling, M. (2013) · arXiv · doi:10.48550/arxiv.1312.6114 · Cited by 20,067 · see citing works
(2016) · Cited by 11,679 · see citing works
Continuous control with deep reinforcement learning
Lillicrap, T. P., Hunt, J. J., Pritzel, A., Heess, N., Erez, T., Tassa, Y., Silver, D., & Wierstra, D. (2015) · arXiv · doi:10.48550/arxiv.1509.02971 · Cited by 10,782 · see citing works
Playing Atari with Deep Reinforcement Learning
Mnih, V., Kavukcuoglu, K., Silver, D., Graves, A., Antonoglou, I., Wierstra, D., & Riedmiller, M. (2013) · arXiv · doi:10.48550/arxiv.1312.5602 · Cited by 9,471 · see citing works
Mastering the game of Go without human knowledge
Silver, D., Schrittwieser, J., Simonyan, K., Antonoglou, I., Huang, A., Guez, A., Hubert, T., Baker, L., Lai, M., Bolton, A., et al. (2017) · Nature · doi:10.1038/nature24270 · Cited by 9,084 · see citing works
Reinforcement Learning: A Survey
Kaelbling, L. P., Littman, M. L., & Moore, A. W. (1996) · Journal of Artificial Intelligence Research · doi:10.1613/jair.301 · Cited by 8,972 · see citing works
The free-energy principle: a unified brain theory?
Friston, K. (2010) · Nature Reviews Neuroscience · doi:10.1038/nrn2787 · Cited by 8,120 · cited 3× · see citing works
“Another important parameter in Active Inference and predictive processing is the precision (Feldman and Friston, 2010; Kanai et al., 2015), which in Active Inference corresponds to the inverse temperature parameter in the softmax and so controls the stochasticity of action selection.”
BackgroundWhatever next? Predictive brains, situated agents, and the future of cognitive science
Clark, A. (2013) · Behavioral and Brain Sciences · doi:10.1017/s0140525x12000477 · Cited by 5,924 · cited 1× · see citing works
“…learning literature also focuses on adding an entropy regularization term to the loss (Haarnoja et al., 2017; Haarnoja, 2018; Ziebart et al., 2008; Rawlik et al., 2010, 2013; Haarnoja et al., 2018) and have shown this to empirically aid performance and exploration on many benchmark tasks.”
BackgroundSoft actor-critic Off-policy maximum entropy deep reinforcement learning with a stochastic actor
Haarnoja, T., Zhou, A., Abbeel, P., & Levine, S. (2018) · arXiv · doi:10.48550/arxiv.1801.01290 · Cited by 5,594 · see citing works
Rao, R. P. N., & Ballard, D. H. (1999) · Nature Neuroscience · doi:10.1038/4580 · Cited by 5,483 · see citing works
(1999) · Cited by 5,147 · see citing works
A theory of cortical responses.
Friston, K. (2005) · Philosophical Transactions of the Royal Society B: Biological Sciences · doi:10.1098/rstb.2005.1622 · Cited by 4,578 · see citing works
Variational Inference: A Review for Statisticians
Blei, D. M., Kucukelbir, A., & McAuliffe, J. D. (2017) · Journal of the American Statistical Association · doi:10.1080/01621459.2017.1285773 · Cited by 3,974 · see citing works
Addressing Function Approximation Error in Actor-Critic Methods
Fujimoto, S., van Hoof, H., & Meger, D. (2018) · arXiv · doi:10.48550/arxiv.1802.09477 · Cited by 3,650 · see citing works
Deep Reinforcement Learning with Double Q-Learning
Van Hasselt, H., Guez, A., & Silver, D. (2016) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v30i1.10295 · Cited by 3,318 · see citing works
The Bayesian brain: The role of uncertainty in neural coding and computation
Knill, D. C., & Pouget, A. (2004) · Trends in Neurosciences · doi:10.1016/j.tins.2004.10.007 · Cited by 2,588 · see citing works
Cortical substrates for exploratory decisions in humans
Daw, N. D., O'Doherty, J. P., Dayan, P., Seymour, B., & Dolan, R. J. (2006) · Nature · doi:10.1038/nature04766 · Cited by 2,224 · see citing works
Variational Inference: A Review for Statisticians
Blei, D. M., Kucukelbir, A., & McAuliffe, J. (2023) · Figshare · doi:10.6084/m9.figshare.5203696.v2 · Cited by 2,088 · see citing works
Maximum Entropy Inverse Reinforcement Learning
Ziebart, B. D., Maas, A. L., Bagnell, J. A., & Dey, A. K. (2018) · doi:10.1184/r1/6555512 · Cited by 1,868 · see citing works
The free-energy principle: a rough guide to the brain?
Friston, K. (2009) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2009.04.005 · Cited by 1,831 · see citing works
Stochastic variational inference
Hoffman, M. D., Blei, D. M., Wang, C., & Paisley, J. W. (2013) · Journal of Machine Learning Research · doi:10.5555/2567709.2502622 · Cited by 1,802 · see citing works
Recurrent World Models Facilitate Policy EvolutionPreprint
Ha, D. R., & Schmidhuber, J. (2018) · Neural Information Processing Systems · doi:10.48550/arxiv.1809.01999 · Cited by 1,592 · see citing works
Tutorial on Variational Autoencoders
Doersch, C. (2016) · arXiv · doi:10.48550/arxiv.1606.05908 · Cited by 1,470 · see citing works
A free energy principle for the brain
Friston, K., Kilner, J., & Harrison, L. (2006) · Journal of Physiology-Paris · doi:10.1016/j.jphysparis.2006.10.001 · Cited by 1,369 · see citing works
Attention, uncertainty, and free-energy
Feldman, H., & Friston, K. J. (2010) · Frontiers in Human Neuroscience · doi:10.3389/fnhum.2010.00215 · Cited by 1,365 · see citing works
(2015) · Cited by 1,141 · see citing works
PILCO: A Model-Based and Data-Efficient Approach to Policy Search
Deisenroth, M. P., & Rasmussen, C. E. (2011) · Cambridge University Engineering Department Publications Database · Cited by 1,136 · see citing works
Bayesian brain : probabilistic approaches to neural coding
賢治, 銅., 信, 石., Pouget, A., & Rao, R. P. N. (2006) · The MIT Press eBooks · doi:10.7551/mitpress/9780262042383.001.0001 · Cited by 823 · see citing works
What is intrinsic motivation? A typology of computational approaches
Oudeyer, P.-Y. (2007) · Frontiers in Neurorobotics · doi:10.3389/neuro.12.006.2007 · Cited by 811 · see citing works
Predictions not commands: Active inference in the motor system
Adams, R. A., Shipp, S., & Friston, K. J. (2013) · Brain Structure and Function · doi:10.1007/s00429-012-0475-5 · Cited by 765 · see citing works
Active inference and epistemic value
Friston, K., Rigoli, F., Ognibene, D., Mathys, C., Fitzgerald, T., & Pezzulo, G. (2015) · Cognitive Neuroscience · doi:10.1080/17588928.2015.1020053 · Cited by 731 · see citing works
Learning and inference in the brain
Friston, K. (2003) · Neural Networks · doi:10.1016/j.neunet.2003.06.005 · Cited by 724 · see citing works
Friston, K., FitzGerald, T., Rigoli, F., Schwartenbeck, P., O⿿Doherty, J., & Pezzulo, G. (2016) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2016.06.022 · Cited by 674 · see citing works
Friston, K. J., & Stephan, K. E. (2007) · Synthese · doi:10.1007/s11229-007-9237-y · Cited by 673 · see citing works
Reinforcement Learning with Deep Energy-Based PoliciesPreprint
Haarnoja, T., Tang, H., Abbeel, P., & Levine, S. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1702.08165 · Cited by 638 · see citing works
Brockman, G., Cheung, V., Pettersson, L., Schneider, J., Schulman, J., Tang, J., & Zaremba, W. (2016) · arXiv · doi:10.48550/arxiv.1606.01540 · Cited by 582 · see citing works
An active inference theory of allostasis and interoception in depression
Barrett, L. F., Quigley, K. S., & Hamilton, P. (2016) · Philosophical Transactions of the Royal Society B: Biological Sciences · doi:10.1098/rstb.2016.0011 · Cited by 554 · see citing works
Incentivizing Exploration In Reinforcement Learning With Deep Predictive ModelsPreprint
Stadie, B. C., Levine, S., & Abbeel, P. (2015) · arXiv.org · doi:10.48550/arxiv.1507.00814 · Cited by 545 · see citing works
Variational Information Maximisation for Intrinsically Motivated Reinforcement LearningPreprint
Mohamed, S., & Rezende, D. J. (2015) · Neural Information Processing Systems · doi:10.48550/arxiv.1509.08731 · Cited by 460 · see citing works
Reinforcement learning or active inference?
Friston, K. J., Daunizeau, J., & Kiebel, S. J. (2009) · PLoS ONE · doi:10.1371/journal.pone.0006421 · Cited by 443 · see citing works
Cerebral hierarchies: predictive processing, precision and the pulvinar
Kanai, R., Komura, Y., Shipp, S., & Friston, K. (2015) · Philosophical Transactions of the Royal Society B: Biological Sciences · doi:10.1098/rstb.2014.0169 · Cited by 432 · see citing works
Theories of Error Back-Propagation in the Brain
Whittington, J. C. R., & Bogacz, R. (2019) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2018.12.005 · Cited by 420 · see citing works
The graphical brain: Belief propagation and active inference
Friston, K. J., Parr, T., & de Vries, B. (2017) · Network Neuroscience · doi:10.1162/netn_a_00018 · Cited by 415 · see citing works
A free energy principle for biological systems
Karl, F. (2012) · Entropy · doi:10.3390/e14112100 · Cited by 377 · see citing works
Reinforcement Learning and Control as Probabilistic Inference: Tutorial and Review
Levine, S. (2018) · arXiv · doi:10.48550/arxiv.1805.00909 · Cited by 366 · see citing works
The free energy principle for action and perception: A mathematical review
Buckley, C. L., Kim, C. S., McGregor, S., & Seth, A. K. (2017) · Journal of Mathematical Psychology · doi:10.1016/j.jmp.2017.09.004 · Cited by 352 · see citing works
Active Inference, Curiosity and Insight
Friston, K. J., Lin, M., Frith, C., Pezzulo, G., Hobson, J. A., & Ondobaka, S. (2017) · Neural Computation · doi:10.1162/neco_a_00999 · Cited by 347 · see citing works
The history of the future of the Bayesian brain
Friston, K. J. (2011) · NeuroImage · doi:10.1016/j.neuroimage.2011.10.004 · Cited by 346 · see citing works
The anatomy of choice: active inference and agency
Friston, K., Schwartenbeck, P., FitzGerald, T., Moutoussis, M., Behrens, T., & Dolan, R. J. (2013) · Frontiers in Human Neuroscience · doi:10.3389/fnhum.2013.00598 · Cited by 340 · see citing works
What is optimal about motor control?
Friston, K. (2011) · Neuron · doi:10.1016/j.neuron.2011.10.018 · Cited by 332 · see citing works
Botvinick, M., & Toussaint, M. (2012) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2012.08.006 · Cited by 295 · see citing works
The anatomy of choice: dopamine and decision-making.
Friston, K., Schwartenbeck, P., FitzGerald, T., Moutoussis, M., Behrens, T., & Dolan, R. J. (2014) · Philosophical Transactions of the Royal Society B: Biological Sciences · doi:10.1098/rstb.2013.0481 · Cited by 289 · see citing works
A tutorial on the free-energy framework for modelling perception and learning
Bogacz, R. (2017) · Journal of Mathematical Psychology · doi:10.1016/j.jmp.2015.11.003 · Cited by 283 · see citing works
Whittington, J. C. R., & Bogacz, R. (2017) · Neural Computation · doi:10.1162/neco_a_00949 · Cited by 278 · see citing works
Deconstructing the human algorithms for exploration
Gershman, S. J. (2018) · Cognition · doi:10.1016/j.cognition.2017.12.014 · Cited by 267 · see citing works
Uncertainty, epistemics and active inference
Parr, T., & Friston, K. J. (2017) · Journal of The Royal Society Interface · doi:10.1098/rsif.2017.0376 · Cited by 252 · see citing works
Computational mechanisms of curiosity and goal-directed exploration
Schwartenbeck, P., Passecker, J., Hauser, T. U., FitzGerald, T. H. B., Kronbichler, M., & Friston, K. J. (2019) · eLife · doi:10.7554/elife.41703 · Cited by 250 · see citing works
Off-Policy Actor-Critic Structure for Optimal Control of Unknown Systems With Disturbances
Song, R., Lewis, F. L., Wei, Q., & Zhang, H. (2015) · IEEE Transactions on Cybernetics · doi:10.1109/tcyb.2015.2421338 · Cited by 241 · see citing works
Active inference and agency: optimal control without cost functions
Friston, K., Samothrakis, S., & Montague, R. (2012) · Biological Cybernetics · doi:10.1007/s00422-012-0512-8 · Cited by 241 · see citing works
Deep temporal models and active inference
Friston, K. J., Rosch, R. E., Parr, T., Price, C. J., & Bowman, H. (2017) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2017.04.009 · Cited by 240 · see citing works
A free energy principle for a particular physics
Friston, K. (2019) · arXiv · doi:10.48550/arxiv.1906.10184 · Cited by 222 · see citing works
The Anatomy of Inference: Generative Models and Brain Structure
Parr, T., & Friston, K. J. (2018) · Frontiers in Computational Neuroscience · doi:10.3389/fncom.2018.00090 · Cited by 198 · see citing works
Active Inference, Attention, and Motor Preparation
Brown, H., Friston, K., & Bestmann, S. (2011) · Frontiers in Psychology · doi:10.3389/fpsyg.2011.00218 · Cited by 165 · see citing works
Scene Construction, Visual Foraging, and Active Inference
Mirza, M. B., Adams, R. A., Mathys, C. D., & Friston, K. J. (2016) · Frontiers in Computational Neuroscience · doi:10.3389/fncom.2016.00056 · Cited by 165 · see citing works
Gershman, S. J. (2018) · Decision · doi:10.1037/dec0000101 · Cited by 146 · see citing works
Free Energy, Value, and Attractors
Friston, K. J., & Ao, P. (2011) · Computational and Mathematical Methods in Medicine · doi:10.1155/2012/937860 · Cited by 140 · see citing works
Unsupervised Predictive Memory in a Goal-Directed AgentPreprint
Wayne, G., Hung, C.-C., Amos, D., Mirza, M., Ahuja, A., Grabska‐Barwińska, A., Rae, J. W., Mirowski, P., Leibo, J. Z., Santoro, A., et al. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1803.10760 · Cited by 137 · see citing works
Clark, A. (2012) · Mind · doi:10.1093/mind/fzs106 · Cited by 133 · see citing works
Neuronal message passing using mean-field, bethe, and marginal approximations
Parr, T., Markovic, D., Kiebel, S. J., & Friston, K. J. (2019) · Scientific Reports · doi:10.1038/s41598-018-38246-3 · Cited by 128 · see citing works
Deep temporal models and active inference
Friston, K. J., Rosch, R., Parr, T., Price, C., & Bowman, H. (2018) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2018.04.004 · Cited by 119 · see citing works
Ueltzhöffer, K. (2018) · Biological Cybernetics · doi:10.1007/s00422-018-0785-7 · Cited by 98 · see citing works
Smooth pursuit and visual occlusion: active inference and oculomotor control in schizophrenia
Adams, R. A., Perrinet, L. U., & Friston, K. (2012) · PLoS ONE · doi:10.1371/journal.pone.0047502 · Cited by 91 · see citing works
On the Kalman filtering method in neural network training and pruning
Sum, J., Leung, C.-S., Young, G. H., et al. (1999) · IEEE Transactions on Neural Networks · doi:10.1109/72.737502 · Cited by 84 · see citing works
Curiosity-driven Exploration in Deep Reinforcement Learning via Bayesian Neural NetworksPreprint
Houthooft, R., Chen, X., Duan, Y., Schulman, J. D., de Turck, F., & Abbeel, P. (2016) · arXiv.org · doi:10.48550/arxiv.1605.09674 · Cited by 81 · see citing works
Active Inference, Evidence Accumulation, and the Urn Task
FitzGerald, T. H. B., Schwartenbeck, P., Moutoussis, M., Dolan, R. J., & Friston, K. J. (2014) · Neural Computation · doi:10.1162/neco_a_00699 · Cited by 77 · see citing works
Active Inference, epistemic value, and vicarious trial and errorPreprint
Pezzulo, G., Cartoni, E., Rigoli, F., Pio‐Lopez, L., & Friston, K. J. (2016) · Learning & Memory · doi:10.1101/lm.041780.116 · Cited by 75 · see citing works
Expected Policy Gradients for Reinforcement LearningPreprint
Ciosek, K., & Whiteson, S. (2018) · arXiv (Cornell University) · doi:10.1609/aaai.v32i1.11607 · Cited by 62 · see citing works
Ng, S., & McLachlan, G. J. (2004) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.2004.826217 · Cited by 61 · see citing works
Off-Policy Actor-CriticPreprint
Degris, T., White, M., & Sutton, R. S. (2012) · arXiv (Cornell University) · doi:10.48550/arxiv.1205.4839 · Cited by 52 · see citing works
Active inference and the anatomy of oculomotion
Parr, T., & Friston, K. J. (2018) · Neuropsychologia · doi:10.1016/j.neuropsychologia.2018.01.041 · Cited by 51 · see citing works
A new EM-based training algorithm for RBF networks
Lázaro, M., Santamarı́a, I., & Pantaleón, C. (2003) · Neural Networks · doi:10.1016/s0893-6080(02)00215-0 · Cited by 44 · see citing works
Curiosity-driven Exploration in Deep Reinforcement Learning via Bayesian Neural Networks.Preprint
Houthooft, R., Chen, X., Duan, Y., Schulman, J., Turck, F. D., & Abbeel, P. (2016) · arXiv (Cornell University) · Cited by 44 · see citing works
Simulating Active Inference Processes by Message Passing
van de Laar, T., & de Vries, B. (2019) · Frontiers in Robotics and AI · doi:10.3389/frobt.2019.00020 · Cited by 43 · see citing works
Fast training of recurrent networks based on the EM algorithm
Ma, S., & Ji, C. (1998) · IEEE Transactions on Neural Networks · doi:10.1109/72.655025 · Cited by 34 · see citing works
VIREL: A Variational Inference Framework for Reinforcement LearningPreprint
Fellows, M., Mahajan, A., Rudner, T. G. J., & Whiteson, S. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1811.01132 · Cited by 33 · see citing works
Bayesian policy selection using active inferencePreprint
Çatal, O., Nauta, J., Verbelen, T., Simoens, P., & Dhoedt, B. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1904.08149 · Cited by 28 · see citing works
Expanding the Active Inference Landscape: More Intrinsic Motivations in the Perception-Action Loop
Biehl, M., Guckelsberger, C., Salge, C., Smith, S. C., & Polani, D. (2018) · Frontiers in Neurorobotics · doi:10.3389/fnbot.2018.00045 · Cited by 27 · see citing works
An Active Inference Approach to Modeling Structure Learning: Concept Learning as an Example Case
Smith, R., Schwartenbeck, P., Parr, T., & Friston, K. J. (2020) · UCL Discovery (University College London) · Cited by 26 · see citing works
Generalised free energy and active inference: can the future cause the past?Preprint
Parr, T., & Friston, K. J. (2018) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/304782 · Cited by 24 · see citing works
An active inference and epistemic value view of metacognition
Moulin, C. J. A., & Souchay, C. (2015) · Cognitive Neuroscience · doi:10.1080/17588928.2015.1051015 · Cited by 21 · see citing works
Millidge, B. (2019) · doi:10.31234/osf.io/kf6wc · Cited by 21 · see citing works
Self-citationApproximate Inference and Stochastic Optimal ControlPreprint
Rawlik, K., Toussaint, M., & Vijayakumar, S. (2010) · arXiv (Cornell University) · doi:10.48550/arxiv.1009.3958 · Cited by 19 · see citing works
Mean Actor CriticPreprint
Allen, C., Asadi, K., Roderick, M., et al. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1709.00503 · Cited by 18 · see citing works
Impulsivity and Active Inference
Mirza, M. B., Adams, R. A., Parr, T., & Friston, K. J. (2019) · Journal of Cognitive Neuroscience · doi:10.1162/jocn_a_01352 · Cited by 16 · see citing works
Acquiring Diverse Robot Skills via Maximum Entropy Deep Reinforcement Learning
Haarnoja, T. (2018) · UC Berkeley · Cited by 11 · see citing works
Flagged first, then by citations
Research works printed but not linked: 6
- [13]13. Cesa-Bianchi, N., Gentile, C., Lugosi, G., and Neu, G. (2017). Boltzmann exploration done right. In Advances in Neural Information Processing Systems, pages 6284-6293.
- [14]14. Ciosek, K. and Whiteson, S. (2018). Expected policy gradients. In Thirty-Second AAAI Conference on Artificial Intelligence.
- [55]55. Houthooft, R., Chen, X., Duan, Y., Schulman, J., De Turck, F., and Abbeel, P. (2016b). Vime: Variational information maximizing exploration. In Advances in Neural Information Processing Systems, pages 1109-1117.
- [79]79. Rawlik, K., Toussaint, M., and Vijayakumar, S. (2013). On stochastic optimal control and reinforcement learning by approximate inference. In Twenty-Third International Joint Conference on Artificial Intelligence.
- [82]82. Smith, R., Schwartenbeck, P., Parr, T., and Friston, K. J. (2019). An active inference approach to modeling concept learning. bioRxiv, page 633677.
- [85]85. Sutton, R. S., Barto, A. G., et al. (1998). Introduction to reinforcement learning, volume 135. MIT press Cambridge.
Cited by
Sorted by citations
Interoception as modeling, allostasis as control
Sennesh, E., Theriault, J., Brooks, D., van de Meent, J.-W., Barrett, L. F., & Quigley, K. S. (2022) · Biological Psychology · doi:10.1016/j.biopsycho.2021.108242 · Cited by 107 · see citing works
Bayesian mechanics for stationary processes
Da Costa, L., Friston, K., Heins, C., & Pavliotis, G. A. (2021) · Proceedings of the Royal Society A: Mathematical, Physical and Engineering Sciences · doi:10.1098/rspa.2021.0518 · Cited by 63 · see citing works
Predictive processing in cognitive robotics: A review
Ciria, A., Schillaci, G., Pezzulo, G., Hafner, V. V., & Lara, B. (2021) · Neural Computation · doi:10.1162/neco_a_01383 · Cited by 51 · see citing works
Deane, G. (2021) · Neuroscience of Consciousness · doi:10.1093/nc/niab024 · Cited by 38 · see citing works
“Active inference formalizes allostasis (and action) as an inference problem, ‘planning as inference’ (Kaplan and Friston 2018) under the free energy principle (Friston 2019).”
BackgroundInfluentialNeural dynamics under active inference: plausibility and efficiency of information processing
Da Costa, L., Parr, T., Sengupta, B., & Friston, K. (2021) · Entropy · doi:10.3390/e23040454 · Cited by 38 · see citing works
Learned uncertainty: The free energy principle in anxiety
McGovern, H. T., De Foe, A., Biddell, H., Leptourgos, P., Corlett, P., Bandara, K., & Hutchinson, B. T. (2022) · Frontiers in Psychology · doi:10.3389/fpsyg.2022.943785 · Cited by 33 · see citing works
Reward maximization through discrete active inference
Da Costa, L., Sajid, N., Parr, T., Friston, K., & Smith, R. (2023) · Neural Computation · doi:10.1162/neco_a_01574 · Cited by 35 · see citing works
Deconstructing Deep Active Inference: A Contrarian Information Gatherer
Champion, T., Grześ, M., Bonheme, L., & Bowman, H. (2024) · Neural Computation · doi:10.1162/neco_a_01697 · Cited by 5 · see citing works
Emergence of Goal‐Directed Behaviors via Active Inference With Self‐Prior
Kim, D., Kanazawa, H., Yoshida, N., & Kuniyoshi, Y. (2026) · Developmental Science · doi:10.1111/desc.70286
“In contrast, approaches utilizing deep neural networks do not directly mimic biological details, but they provide a practical method to extend the core computational principles of active inference to high-dimensional problems (Millidge 2020 ).”
- Authors
- Millidge, B.
- Affiliations
- University of Edinburgh (GB)
- Corresponding author
- Millidge, Beren
- ORCID
- Millidge, Beren
- Venue
- Journal of Mathematical Psychology
- Publisher
- Elsevier BV
- Published
- 2020-06
- Volume
- 96
- Pages
- 102348
- ISSN
- 0022-2496 (ISSN-L)
- OpenAlex
- W2959937473
- Cited by
- 115
- References
- 96
- Citations by year
- 2026: 12 · 2025: 9 · 2024: 8 · 2023: 12 · 2022: 22 · 2021: 31 · 2020: 21
- FWCI
- 8.45
- Type
- journal article
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- Journal of Mathematical Psychology (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Funders
- Engineering and Physical Sciences Research Council
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Reinforcement Learning in Robotics
- Life Sciences › Neuroscience › Cognitive Neuroscience › Embodied and Extended Cognition
- Physical Sciences › Computer Science › Artificial Intelligence › Psychiatry, Mental Health, Neuroscience
- Keywords
- active inference, variational free energy, perception-action coupling, maximum entropy reinforcement learning, reinforcement learning
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
