Active Inference, Evidence Accumulation, and the Urn Task
Open accessThe DOI matches its Crossref record; 30 of the 30 references it cites were found in a registry or an index.
Abstract
Deciding how much evidence to accumulate before making a decision is a problem we and other animals often face, but one that is not completely understood. This issue is particularly important because a tendency to sample less information (often known as reflection impulsivity) is a feature in several psychopathologies, such as psychosis. A formal understanding of information sampling may therefore clarify the computational anatomy of psychopathology. In this theoretical letter, we consider evidence accumulation in terms of active (Bayesian) inference using a generic model of Markov decision processes. Here, agents are equipped with beliefs about their own behavior—in this case, that they will make informed decisions. Normative decision making is then modeled using variational Bayes to minimize surprise about choice outcomes. Under this scheme, different facets of belief updating map naturally onto the functional anatomy of the brain (at least at a heuristic level). Of particular interest is the key role played by the expected precision of beliefs about control, which we have previously suggested may be encoded by dopaminergic neurons in the midbrain. We show that manipulating expected precision strongly affects how much information an agent characteristically samples, and thus provides a possible link between impulsivity and dopaminergic dysfunction. Our study therefore represents a step toward understanding evidence accumulation in terms of neurobiologically plausible Bayesian inference and may cast light on why this process is disordered in psychopathology.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
The free-energy principle: a unified brain theory?
Friston, K. (2010) · Nature Reviews Neuroscience · doi:10.1038/nrn2787 · Cited by 8,120 · cited 2×
“This motivates and explains both functional segregation (Zeki et al., 1991) (since subsets are conditionally independent of one another) and functional integration (Friston, 2002) (since updating requires message passing between subsets) (Friston, 2010).”
BackgroundSelf-citationA theory of cortical responses.
Friston, K. (2005) · Philosophical Transactions of the Royal Society B: Biological Sciences · doi:10.1098/rstb.2005.1622 · Cited by 4,578 · cited 1×
“These gradient descent schemes can be implemented as (biologically plausible) Bayesian filtering or predictive coding (Mumford, 1992; Friston, 2005; Friston et al., 2007, 2008).”
BackgroundSelf-citationNeural systems of reinforcement for drug addiction: from actions to habits to compulsion
Everitt, B. J., & Robbins, T. W. (2005) · Nature Neuroscience · doi:10.1038/nn1579 · Cited by 3,785 · cited 1×
“The effect of varying prior precision on impulsivity is particularly interesting, given the proposal that precision is encoded by dopaminergic activity in the midbrain and the fact that dopaminergic abnormalities are implicated in a number of psychopathologies (Everitt and Robbins, 2005; Howes and Kapur, 2009; Steeves…”
BackgroundThe Neural Basis of Decision Making
Gold, J. I., & Shadlen, M. N. (2007) · Annual Review of Neuroscience · doi:10.1146/annurev.neuro.29.051605.113038 · Cited by 3,764 · cited 2×
“Evidence accumulation, in its various forms, has been the subject of much interest in cognitive neuroscience (Gold and Shadlen, 2007; Yang and Shadlen, 2007; Moutoussis et al., 2011; Hunt et al., 2012).”
BackgroundMethodThe Dopamine Hypothesis of Schizophrenia: Version III--The Final Common Pathway
Howes, O. D., & Kapur, S. (2009) · Schizophrenia Bulletin · doi:10.1093/schbul/sbp006 · Cited by 2,693 · cited 5×
“In particular, as disorders such as schizophrenia, Parkinson's disease and drug abuse are characterised by aberrant dopaminergic function (Dagher and Robbins, 2009; Howes and Kapur, 2009), we focus on the effects of varying expected precision (which we assume to be encoded by”
BackgroundInfluentialDissociable Roles of Ventral and Dorsal Striatum in Instrumental Conditioning
O'Doherty, J., Dayan, P., Schultz, J., Deichmann, R., Friston, K., & Dolan, R. J. (2004) · Science · doi:10.1126/science.1094285 · Cited by 2,088
Self-citationBy Carrot or by Stick: Cognitive Reinforcement Learning in Parkinsonism
Frank, M. J., Seeberger, L. C., & O'Reilly, R. C. (2004) · Science · doi:10.1126/science.1102941 · Cited by 1,964
Neuron-type-specific signals for reward and punishment in the ventral tegmental area
Cohen, J. Y., Haesler, S., Vong, L., Lowell, B. B., & Uchida, N. (2012) · Nature · doi:10.1038/nature10754 · Cited by 1,318 · cited 2×
“The hypothesis that dopamine encodes expected precision is also supported by the characteristic time course of expected precision during evidence accumulation, which shows a pattern similar to that observed in dopaminergic neurons in response to omitted and expected reward (Schultz et al., 1997; Cohen et al., 2012).”
ResultOn the computational architecture of the neocortex: II The role of cortico-cortical loops
Mumford, D. (1992) · Biological Cybernetics · doi:10.1007/bf00198477 · Cited by 1,074
On the computational architecture of the neocortex: I. The role of the thalamo-cortical loop
Mumford, D. (1991) · Biological Cybernetics · doi:10.1007/bf00202389 · Cited by 929 · cited 2×
“filtering or predictive coding (Mumford, 1992; Friston, 2005; Friston, Mattout, Trujillo-Barreto, Ashburner, & Penny, 2007; Friston, TrujilloBarreto, & Daunizeau, 2008).”
Background
Research works printed but not linked: 32
- Attias H., 2000, Advances in neural information processing systems, 12, 209
- 10.1037/a0032079 · DOI · Paper page
- 10.1016/j.schres.2010.08.009 · DOI · Paper page
- Beal M. J., 2003, Variational algorithms for approximate Bayesian inference
- 10.1016/j.biopsych.2005.11.007 · DOI · Paper page
- 10.1017/S0033291707002152 · DOI · Paper page
- 10.1016/j.neuron.2009.01.031 · DOI · Paper page
- 10.1038/nature04766 · DOI · Paper page
- 10.3389/fnins.2012.00075 · DOI · Paper page
- Djamshidian, A., O’Sullivan, S. S., Sanotsky, Y., Sharman, S., Matviyenko, Y., Foltynie, T. … Averbeck, B. B. (2012). Decision making, impulsivity, and addictions: Do Parkinson’s disease patients jump to conclusions? Mov. Disord., 27, 1137–1145. · DOI
- 10.1371/journal.pone.0058536 · DOI
- 10.1017/S0033291796004175 · DOI · Paper page
- 10.1016/S0301-0082(02)00076-X · DOI · Paper page
- 10.1523/JNEUROSCI.4236-11.2011 · DOI · Paper page
- 10.1097/00005053-199104000-00003 · DOI · Paper page
- 10.1371/journal.pone.0053344 · DOI
- 10.1037/h0022886 · DOI · Paper page
- 10.1523/JNEUROSCI.17-19-07450.1997 · DOI · Paper page
- 10.1111/j.1748-6653.2011.02005.x · DOI · Paper page
- 10.1016/j.schres.2007.08.021 · DOI
- 10.1348/014466505X35678 · DOI · Paper page
- 10.1126/science.275.5306.1593 · DOI · Paper page
- 10.3389/fpsyg.2013.00710 · DOI · Paper page
- Schwartenbeck P., 2014, Cereb. Cortex.
- 10.4249/scholarpedia.3737 · doi:10.4249/scholarpedia.3737
- 10.1093/brain/aws083 · DOI · Paper page
- 10.1073/pnas.91.12.5720 · DOI · Paper page
- 10.1002/ana.21809 · DOI · Paper page
- 10.1016/j.biopsych.2006.09.042 · DOI · Paper page
- 10.1038/nature05852 · DOI · Paper page
- 10.1016/j.bcp.2008.01.018 · DOI
- 10.1523/JNEUROSCI.11-03-00641.1991 · DOI · Paper page
Cited by
Active inference: a process theory
Friston, K., FitzGerald, T., Rigoli, F., Schwartenbeck, P., & Pezzulo, G. (2017) · Neural Computation · doi:10.1162/neco_a_00912 · Cited by 1,156
Self-citationFriston, K., FitzGerald, T., Rigoli, F., Schwartenbeck, P., ODoherty, J., & Pezzulo, G. (2016) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2016.06.022 · Cited by 674
Self-citationActive Inference, Curiosity and Insight
Friston, K. J., Lin, M., Frith, C. D., Pezzulo, G., Hobson, J. A., & Ondobaka, S. (2017) · Neural Computation · doi:10.1162/neco_a_00999 · Cited by 351
Self-citationA tutorial on the free-energy framework for modelling perception and learning
Bogacz, R. (2017) · Journal of Mathematical Psychology · doi:10.1016/j.jmp.2015.11.003 · Cited by 283
The dopaminergic midbrain encodes the expected certainty about desired outcomes
Schwartenbeck, P., FitzGerald, T. H. B., Mathys, C., Dolan, R., & Friston, K. (2015) · Cerebral Cortex · doi:10.1093/cercor/bhu159 · Cited by 219
Self-citationDeep temporal models and active inference
Friston, K. J., Rosch, R., Parr, T., Price, C., & Bowman, H. (2017) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2017.04.009 · Cited by 237
Self-citationGeneralised free energy and active inference
Parr, T., & Friston, K. J. (2019) · Biological Cybernetics · doi:10.1007/s00422-019-00805-w · Cited by 236
Self-citationActive inference on discrete state-spaces: A synthesis
Da Costa, L., Parr, T., Sajid, N., Veselic, S., Neacsu, V., & Friston, K. (2020) · Journal of Mathematical Psychology · doi:10.1016/j.jmp.2020.102447 · Cited by 226
“or risk sensitive control in an engineering benchmark – the mountain car problem. [14,16] Evidence accumulation Illustrating the role of evidence accumulation in decision-making through an urns task. [31,32] Psychopathology Simulation of addictive choice behaviour. [33] Dopamine The precision of beliefs about policies…”
BackgroundSelf-citationA free energy principle for a particular physics
Friston, K. (2019) · arXiv · doi:10.48550/arxiv.1906.10184 · Cited by 222
“It provides a generic (active) inference scheme that has been used in a wide variety of applications and simulations; ranging from games in behavioural economics (FitzGerald et al., 2015) and reinforcement learning (Schwartenbeck et al.”
BackgroundMethodSelf-citationPredictive processing as a systematic basis for identifying the neural correlates of consciousness
Hohwy, J., & Seth, A. (2020) · Philosophy and the Mind Sciences · doi:10.33735/phimisci.2020.ii.64 · Cited by 170
Scene Construction, Visual Foraging, and Active Inference
Mirza, M. B., Adams, R. A., Mathys, C. D., & Friston, K. J. (2016) · Frontiers in Computational Neuroscience · doi:10.3389/fncom.2016.00056 · Cited by 165
Self-citationThe free energy principle made simpler but not too simple
Friston, K., Da Costa, L., Sajid, N., Heins, C., Ueltzhöffer, K., Pavliotis, G. A., & Parr, T. (2023) · Physics Reports · doi:10.1016/j.physrep.2023.07.001 · Cited by 163
“It describes active inference based upon autonomous paths or policies and has been used in a variety of applications and simulations; ranging from games in behavioural economics [152] and reinforcement learning [153,154] through to language understanding [155] and scene construction [156].”
Self-citationDeep temporal models and active inference
Friston, K. J., Rosch, R., Parr, T., Price, C., & Bowman, H. (2018) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2018.04.004 · Cited by 119
Self-citationDeep active inference as variational policy gradients
Millidge, B. (2020) · Journal of Mathematical Psychology · doi:10.1016/j.jmp.2020.102348 · Cited by 115
Dopamine, reward learning, and active inference
FitzGerald, T. H. B., Dolan, R. J., & Friston, K. (2015) · Frontiers in Computational Neuroscience · doi:10.3389/fncom.2015.00136 · Cited by 112
Self-citationEvidence for surprise minimization over value maximization in choice behavior
Schwartenbeck, P., FitzGerald, T. H. B., Mathys, C., Dolan, R., Kronbichler, M., & Friston, K. (2015) · Scientific Reports · doi:10.1038/srep16575 · Cited by 96
Self-citationIntroducing a Bayesian model of selective attention based on active inference
Mirza, M. B., Adams, R. A., Friston, K., & Parr, T. (2019) · Scientific Reports · doi:10.1038/s41598-019-50138-8 · Cited by 83
“Extrinsic value is the expected utility defined in terms of prior preferences over the outcomes 28 .”
BackgroundSelf-citationKnowing when to stop: aberrant precision and evidence accumulation in schizophrenia
Limongi, R., Bohaterewicz, B., Nowicka, M., Plewka, A., & Friston, K. J. (2018) · Schizophrenia Research · doi:10.1016/j.schres.2017.12.018 · Cited by 38
Self-citationEvidence and implications of abnormal predictive coding in dementia
Kocagoncu, E., Klimovich-Gray, A., Hughes, L. E., & Rowe, J. B. (2021) · Brain · doi:10.1093/brain/awab254 · Cited by 3
“This can lead to ‘jumping to conclusions’.(109) Dopamine dysregulation may explain some types of impulsivity (e.”
Background
Metadata
- Authors
- FitzGerald, T. H. B., Schwartenbeck, P., Moutoussis, M., Dolan, R. J., & Friston, K.
- Venue
- Neural Computation
- Publisher
- MIT Press - Journals
- Published
- 2015-02
- Volume
- 27
- Issue
- 2
- Pages
- 306-328
- ISSN
- 0899-7667 (ISSN-L)
- 1530-888X
- OpenAlex
- W2012418261
- Cited by
- 81
- References
- 30
- FWCI
- 3.23
- Type
- journal article
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.