The DOI matches its Crossref record; 83 of the 84 references it cites were found in a registry or an index, and 1 are not checked yet.
Learning action-oriented models through active inference
Open accessPublished version of a preprint
Abstract
Converging theories suggest that organisms learn and exploit probabilistic models of their environment. However, it remains unclear how such models can be learned in practice. The open-ended complexity of natural environments means that it is generally infeasible for organisms to model their environment comprehensively. Alternatively, action-oriented models attempt to encode a parsimonious representation of adaptive agent-environment interactions. One approach to learning action-oriented models is to learn online in the presence of goal-directed behaviours. This constrains an agent to behaviourally relevant trajectories, reducing the diversity of the data a model need account for. Unfortunately, this approach can cause models to prematurely converge to sub-optimal solutions, through a process we refer to as a bad-bootstrap. Here, we exploit the normative framework of active inference to show that efficient action-oriented models can be learned by balancing goal-oriented and epistemic (information-seeking) behaviours in a principled manner. We illustrate our approach using a simple agent-based model of bacterial chemotaxis. We first demonstrate that learning via goal-directed behaviour indeed constrains models to behaviorally relevant aspects of the environment, but that this approach is prone to sub-optimal convergence. We then demonstrate that epistemic behaviours facilitate the construction of accurate and comprehensive models, but that these models are not tailored to any specific behavioural niche and are therefore less efficient in their use of data. Finally, we show that active inference agents learn models that are parsimonious, tailored to action, and which avoid bad bootstraps and sub-optimal convergence. Critically, our results indicate that models learned through active inference can support adaptive behaviour in spite of, and indeed because of, their departure from veridical representations of the environment. Our approach provides a principled method for learning adaptive models from limited interactions with an environment, highlighting a route to sample efficient learning algorithms.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Figures






Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“This unique formulation of confirmation bias as an epistemic phenomenon helps explain how individuals continuously forage their environment for information, but may become stuck in a so-called ‘bad bootstrap’ that simply reinforces existing beliefs about the world, which in the face of new information may lead to sub-optimal behaviour [131].”
From Epistemic Communities under Active Inference (2022) · cites this paper as [131]
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
The Ecological Approach to Visual Perception: Classic Edition
Gibson, J. J. (2014) · doi:10.4324/9781315740218 · Cited by 12,132
Playing Atari with Deep Reinforcement Learning
Mnih, V., Kavukcuoglu, K., Silver, D., Graves, A., Antonoglou, I., Wierstra, D., & Riedmiller, M. (2013) · arXiv · doi:10.48550/arxiv.1312.5602 · Cited by 9,471
The free-energy principle: a unified brain theory?
Friston, K. (2010) · Nature Reviews Neuroscience · doi:10.1038/nrn2787 · Cited by 8,120 · cited 1×
“Active inference is a normative theory that unifies perception, action and learning under a single imperative—the minimization of variational free energy [42, 43].”
BackgroundMarkov decision processes: Discrete stochastic dynamic programming
Hazeghi, K., & Puterman, M. L. (1995) · Journal of the American Statistical Association · doi:10.2307/2291177 · Cited by 8,037
Markov Decision Processes: Discrete Stochastic Dynamic Programming
Puterman, M. L. (1994) · Wiley Series in Probability and Statistics · doi:10.1002/9780470316887 · Cited by 6,024
Rao, R. P. N., & Ballard, D. H. (1999) · Nature Neuroscience · doi:10.1038/4580 · Cited by 5,483 · cited 2×
“This casts perception as a process of approximate inference, connecting active inference to influential theories such as the Bayesian brain hypothesis (Knill and Pouget, 2004; Gregory, 1980) and predictive coding (Rao and Ballard, 1999).”
MethodBengio, Y., Louradour, J., Collobert, R., & Weston, J. (2009) · Proceedings of the 26th Annual International Conference on Machine Learning · doi:10.1145/1553374.1553380 · Cited by 5,366
Watkins, C. (1989) · OpenGrey (Institut de l'Information Scientifique et Technique) · doi:10.17863/cam.132164 · Cited by 5,297
The Bayesian brain: The role of uncertainty in neural coding and computation
Knill, D. C., & Pouget, A. (2004) · Trends in Neurosciences · doi:10.1016/j.tins.2004.10.007 · Cited by 2,588
Chemotaxis in Escherichia coli analysed by Three-dimensional Tracking
BERG, H. C., & BROWN, D. A. (1972) · Nature · doi:10.1038/239500a0 · Cited by 2,231
Research works printed but not linked: 2
- Kuvayev L, Sutton RS. Model-Based Reinforcement Learning with an Approximate, Learned Model. In: in Proceedings of the Ninth Yale Workshop on Adaptive and Learning Systems; 1996. p. 101–105.
- Ha D, Schmidhuber J. World Models. arXiv:180310122. 2018;.
Cited by
A step-by-step tutorial on active inference and its application to empirical data
Smith, R., Friston, K. J., & Whyte, C. J. (2022) · Journal of Mathematical Psychology · doi:10.1016/j.jmp.2021.102632 · Cited by 246
Predictive processing as a systematic basis for identifying the neural correlates of consciousness
Hohwy, J., & Seth, A. (2020) · Philosophy and the Mind Sciences · doi:10.33735/phimisci.2020.ii.64 · Cited by 170
Self-citationThe Emperor's New Markov Blankets
Bruineberg, J. P., Dołęga, K., Dewhurst, J., & Baltieri, M. (2021) · Behavioral and Brain Sciences · doi:10.1017/s0140525x21002351 · Cited by 169
The evolution of brain architectures for predictive coding and active inference
Pezzulo, G., Parr, T., & Friston, K. J. (2021) · Philosophical Transactions of the Royal Society B Biological Sciences · doi:10.1098/rstb.2020.0531 · Cited by 162
An Active Inference Approach to Modeling Structure Learning: Concept Learning as an Example Case
Smith, R. S., Schwartenbeck, P., Parr, T., & Friston, K. J. (2020) · Frontiers in Computational Neuroscience · doi:10.3389/fncom.2020.00041 · Cited by 132
Metadata
- Authors
- Tschantz, A., Seth, A. K., & Buckley, C. L.
- Affiliations
- University of Sussex (GB)
- Canadian Institute for Advanced Research (CA)
- Corresponding author
- Tschantz, Alexander
- Buckley, Christopher L.
- Venue
- PLOS Computational Biology
- Publisher
- Public Library of Science (PLoS)
- Published
- 2020-04-23
- Volume
- 16
- Issue
- 4
- Pages
- e1007805
- ISSN
- 1553-734X (ISSN-L)
- 1553-7358
- PMC
- PMC7200021
- OpenAlex
- W3017990858
- Semantic Scholar
- 6417d1bfc8f7dc8df7b04e50f8eb9188784b776a
- Cited by
- 121
- References
- 80
- Citations by year
- 2026: 8 · 2025: 25 · 2024: 9 · 2023: 9 · 2022: 23 · 2021: 31 · 2020: 16
- FWCI
- 10.61
- Type
- journal article
- Language
- en
- Update notices
- new_version (2020-05-05)
- Open access
- gold
- doi.org
- Licence
- Locations
- PLoS Computational Biology (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- RePEc: Research Papers in Economics (submittedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Figshare (submittedVersion, cc-by)
- Funders
- Canadian Institute for Advanced Research
- Biotechnology and Biological Sciences Research Council (BB/P022197/1)
- Topics
- Life Sciences › Neuroscience › Cognitive Neuroscience › Embodied and Extended Cognition
- Physical Sciences › Computer Science › Artificial Intelligence › Reinforcement Learning in Robotics
- Physical Sciences › Computer Science › Artificial Intelligence › Bayesian Modeling and Causal Inference
- Keywords
- active inference, Bayesian learning, goal-directed behavior, information seeking, model convergence, bacterial chemotaxis, sample efficiency, agent-environment interaction
- MeSH terms
- Algorithms, Chemotaxis, Environment, Goals, Learning, Models, Biological, Bacterial Physiological Phenomena, Computational Biology
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Learning action-oriented models through active inference. https://app.cochat.ai/papers/10.1371/journal.pcbi.1007805
- The paper
- Tschantz, A., Seth, A. K., & Buckley, C. L. (2020). Learning action-oriented models through active inference. PLOS Computational Biology. https://doi.org/10.1371/journal.pcbi.1007805
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.