Active Inference, Evidence Accumulation, and the Urn Task
Open accessThe DOI matches its Crossref record; 28 of the 28 references it cites were found in a registry or an index.
Abstract
Deciding how much evidence to accumulate before making a decision is a problem we and other animals often face, but one that is not completely understood. This issue is particularly important because a tendency to sample less information (often known as reflection impulsivity) is a feature in several psychopathologies, such as psychosis. A formal understanding of information sampling may therefore clarify the computational anatomy of psychopathology. In this theoretical letter, we consider evidence accumulation in terms of active (Bayesian) inference using a generic model of Markov decision processes. Here, agents are equipped with beliefs about their own behavior—in this case, that they will make informed decisions. Normative decision making is then modeled using variational Bayes to minimize surprise about choice outcomes. Under this scheme, different facets of belief updating map naturally onto the functional anatomy of the brain (at least at a heuristic level). Of particular interest is the key role played by the expected precision of beliefs about control, which we have previously suggested may be encoded by dopaminergic neurons in the midbrain. We show that manipulating expected precision strongly affects how much information an agent characteristically samples, and thus provides a possible link between impulsivity and dopaminergic dysfunction. Our study therefore represents a step toward understanding evidence accumulation in terms of neurobiologically plausible Bayesian inference and may cast light on why this process is disordered in psychopathology.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
The free-energy principle: a unified brain theory?
Friston, K. (2010) · Nature Reviews Neuroscience · doi:10.1038/nrn2787 · Cited by 8,120 · cited 2×
“This motivates and explains both functional segregation (Zeki et al., 1991) (since subsets are conditionally independent of one another) and functional integration (Friston, 2002) (since updating requires message passing between subsets) (Friston, 2010).”
BackgroundSelf-citationNeural systems of reinforcement for drug addiction: from actions to habits to compulsion
Everitt, B. J., & Robbins, T. W. (2005) · Nature Neuroscience · doi:10.1038/nn1579 · Cited by 3,785 · cited 1×
“The effect of varying prior precision on impulsivity is particularly interesting, given the proposal that precision is encoded by dopaminergic activity in the midbrain and the fact that dopaminergic abnormalities are implicated in a number of psychopathologies (Everitt and Robbins, 2005; Howes and Kapur, 2009; Steeves…”
BackgroundThe Neural Basis of Decision Making
Gold, J. I., & Shadlen, M. N. (2007) · Annual Review of Neuroscience · doi:10.1146/annurev.neuro.29.051605.113038 · Cited by 3,764 · cited 2×
“Evidence accumulation, in its various forms, has been the subject of much interest in cognitive neuroscience (Gold and Shadlen, 2007; Yang and Shadlen, 2007; Moutoussis et al., 2011; Hunt et al., 2012).”
BackgroundMethodThe Dopamine Hypothesis of Schizophrenia: Version III--The Final Common Pathway
Howes, O. D., & Kapur, S. (2009) · Schizophrenia Bulletin · doi:10.1093/schbul/sbp006 · Cited by 2,693 · cited 5×
“In particular, as disorders such as schizophrenia, Parkinson's disease and drug abuse are characterised by aberrant dopaminergic function (Dagher and Robbins, 2009; Howes and Kapur, 2009), we focus on the effects of varying expected precision (which we assume to be encoded by”
BackgroundInfluentialDissociable Roles of Ventral and Dorsal Striatum in Instrumental Conditioning
O'Doherty, J., Dayan, P., Schultz, J., Deichmann, R., Friston, K., & Dolan, R. J. (2004) · Science · doi:10.1126/science.1094285 · Cited by 2,088
Self-citationBy Carrot or by Stick: Cognitive Reinforcement Learning in Parkinsonism
Frank, M. J., Seeberger, L. C., & O'Reilly, R. C. (2004) · Science · doi:10.1126/science.1102941 · Cited by 1,964
Neuron-type-specific signals for reward and punishment in the ventral tegmental area
Cohen, J. Y., Haesler, S., Vong, L., Lowell, B. B., & Uchida, N. (2012) · Nature · doi:10.1038/nature10754 · Cited by 1,318 · cited 2×
“The hypothesis that dopamine encodes expected precision is also supported by the characteristic time course of expected precision during evidence accumulation, which shows a pattern similar to that observed in dopaminergic neurons in response to omitted and expected reward (Schultz et al., 1997; Cohen et al., 2012).”
ResultOn the computational architecture of the neocortex: II The role of cortico-cortical loops
Mumford, D. (1992) · Biological Cybernetics · doi:10.1007/bf00198477 · Cited by 1,074
On the computational architecture of the neocortex: I. The role of the thalamo-cortical loop
Mumford, D. (1991) · Biological Cybernetics · doi:10.1007/bf00202389 · Cited by 929 · cited 2×
“filtering or predictive coding (Mumford, 1992; Friston, 2005; Friston, Mattout, Trujillo-Barreto, Ashburner, & Penny, 2007; Friston, TrujilloBarreto, & Daunizeau, 2008).”
BackgroundVariational free energy and the Laplace approximation
Friston, K., Mattout, J., Trujillo-Barreto, N., Ashburner, J., & Penny, W. (2007) · NeuroImage · doi:10.1016/j.neuroimage.2006.08.035 · Cited by 880 · cited 1×
“These gradient descent schemes can be implemented as (biologically plausible) Bayesian filtering or predictive coding (Mumford, 1992; Friston, 2005; Friston et al., 2007, 2008).”
BackgroundSelf-citationThe Computational Anatomy of Psychosis
Adams, R. A., Stephan, K. E., Brown, H. R., Frith, C. D., & Friston, K. J. (2013) · Frontiers in Psychiatry · doi:10.3389/fpsyt.2013.00047 · Cited by 850 · cited 2×
“…particularly interesting, given the proposal that precision is encoded by dopaminergic activity in the midbrain and the fact that dopaminergic abnormalities are implicated in a number of psychopathologies (Everitt and Robbins, 2005; Howes and Kapur, 2009; Steeves et al., 2010; Adams et al., 2013).”
BackgroundSelf-citationCognitive approaches to delusions: a critical review of theories and evidence.
Garety, P. A., & Freeman, D. (1999) · British Journal of Clinical Psychology · doi:10.1348/014466599162700 · Cited by 794 · cited 1×
“Interestingly, delusional patients ‘jump to conclusions’ in the first version, while they are more willing to revise their decision in light of contradictory evidence in the second (Garety and Freeman, 1999).”
BackgroundA Bayesian foundation for individual learning under uncertainty
Mathys, C. (2011) · Frontiers in Human Neuroscience · doi:10.3389/fnhum.2011.00039 · Cited by 655
Probabilistic judgements in deluded and non-deluded subjects
Huq, S. F., Garety, P. A., & Hemsley, D. R. (1988) · The Quarterly Journal of Experimental Psychology Section A · doi:10.1080/14640748808402300 · Cited by 596 · cited 4×
“This paper considers the nature of evidence accumulation in terms of active (Bayesian) inference using the urn or beads task (Huq et al., 1988; Moritz and Woodward, 2005).”
BackgroundInfluentialThe Cost of Accumulating Evidence in Perceptual Decision Making
Drugowitsch, J., Moreno-Bote, R., Churchland, A. K., Shadlen, M. N., & Pouget, A. (2012) · The Journal of Neuroscience · doi:10.1523/jneurosci.4010-11.2012 · Cited by 577 · cited 1×
“Drugowitsch et al. take a similar approach, to produce a hybrid model combining dynamic programming with drift diffusion models to capture collapsing bounds (Drugowitsch et al., 2012).”
BackgroundActive inference, sensory attenuation and illusions
Brown, H., Adams, R. A., Parees, I., Edwards, M., & Friston, K. (2013) · Cognitive Processing · doi:10.1007/s10339-013-0571-3 · Cited by 488 · cited 2×
“Prior work has considered how variations in observed behaviour can result from optimal inference, when the parameters governing estimates of environmental volatility (Mathys et al., 2011; Vossel et al., 2013) and sensory precision (Adams et al., 2012; Brown et al., 2013) are altered.”
BackgroundSelf-citationMechanisms underlying cortical activity during value-guided choice
Hunt, L. T., Kolling, N., Soltani, A., Woolrich, M. W., Rushworth, M. F. S., & Behrens, T. E. J. (2012) · Nature Neuroscience · doi:10.1038/nn.3017 · Cited by 454 · cited 1×
“Evidence accumulation, in its various forms, has been the subject of much interest in cognitive neuroscience (Gold and Shadlen, 2007; Yang and Shadlen, 2007; Moutoussis et al., 2011; Hunt et al., 2012).”
BackgroundDopamine, affordance and active inference
Friston, K. J., Shiner, T., FitzGerald, T., Galea, J. M., Adams, R., Brown, H., Dolan, R. J., Moran, R., Stephan, K. E., & Bestmann, S. (2012) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1002327 · Cited by 381 · cited 4×
“…while our discrete time models are further abstracted from brain function than models we have considered elsewhere (for example (Adams et al., 2012; Friston et al., 2012)) they are still capable of embodying mechanistic hypotheses about cognition, which is essential if we want to relate model…”
BackgroundMethodInfluentialSelf-citationThe anatomy of choice: active inference and agency
Friston, K., Schwartenbeck, P., FitzGerald, T., Moutoussis, M., Behrens, T., & Dolan, R. J. (2013) · Frontiers in Human Neuroscience · doi:10.3389/fnhum.2013.00598 · Cited by 340 · cited 15×
“In this paper, we focus on the formal aspects of this (Bayes) optimal evidence accumulation – using a generic scheme that has been applied previously to limited offer and trust games (Friston et al., 2013; Moutoussis et al., 2014; Schwartenbeck et al., 2014).”
BackgroundMethodInfluentialSelf-citationThe anatomy of choice: dopamine and decision-making.
Friston, K., Schwartenbeck, P., FitzGerald, T., Moutoussis, M., Behrens, T., & Dolan, R. J. (2014) · Philosophical Transactions of the Royal Society B: Biological Sciences · doi:10.1098/rstb.2013.0481 · Cited by 289
Self-citationThe Role of Human Orbitofrontal Cortex in Value Comparison for Incommensurable Objects
FitzGerald, T. H. B., Seymour, B., & Dolan, R. J. (2009) · Journal of Neuroscience · doi:10.1523/jneurosci.0717-09.2009 · Cited by 279
Self-citationDEM: A variational treatment of dynamic systems
Friston, K. J., Trujillo-Barreto, N., & Daunizeau, J. (2008) · NeuroImage · doi:10.1016/j.neuroimage.2008.02.054 · Cited by 278 · cited 1×
“These gradient descent schemes can be implemented as (biologically plausible) Bayesian filtering or predictive coding (Mumford, 1992; Friston, 2005; Friston et al., 2007, 2008).”
BackgroundThe dopaminergic midbrain encodes the expected certainty about desired outcomes
Schwartenbeck, P., FitzGerald, T. H. B., Mathys, C., Dolan, R., & Friston, K. (2015) · Cerebral Cortex · doi:10.1093/cercor/bhu159 · Cited by 219
Self-citationSpeechley, W. J., Whitman, J. C., & Woodward, T. S. (2010) · Journal of Psychiatry and Neuroscience · doi:10.1503/jpn.090025 · Cited by 202
Spatial attention, precision, and Bayesian inference: a study of saccadic response speed
Vossel, S., Mathys, C., Daunizeau, J., Bauer, M., Driver, J., Friston, K. J., & Stephan, K. E. (2014) · Cerebral Cortex · doi:10.1093/cercor/bhs418 · Cited by 146 · cited 2×
“Prior work has considered how variations in observed behaviour can result from optimal inference, when the parameters governing estimates of environmental volatility (Mathys et al., 2011; Vossel et al., 2013) and sensory precision (Adams et al., 2012; Brown et al., 2013) are altered.”
BackgroundSelf-citationBayesian inferences about the self (and others): a review.
Moutoussis, M., Fearon, P., El-Deredy, W., Dolan, R. J., & Friston, K. J. (2014) · Consciousness and Cognition · doi:10.1016/j.concog.2014.01.009 · Cited by 141 · cited 1×
“In this paper, we focus on the formal aspects of this (Bayes) optimal evidence accumulation – using a generic scheme that has been applied previously to limited offer and trust games (Friston et al., 2013; Moutoussis et al., 2014; Schwartenbeck et al., 2014).”
MethodSelf-citationBayesian modelling of Jumping-to-Conclusions bias in delusional patients
Moutoussis, M., Bentall, R. P., El-Deredy, W., & Dayan, P. (2011) · Cognitive Neuropsychiatry · doi:10.1080/13546805.2010.548678 · Cited by 137 · cited 15×
“Indeed, while gross measures of reflection impulsivity such as draws-to-decision are normalized by antidopaminergic therapy (Menon et al., 2008), the finer statistical structure of decision making in the Urns task is not (Moutoussis et al., 2011).”
BackgroundMethodInfluentialSelf-citationSmooth pursuit and visual occlusion: active inference and oculomotor control in schizophrenia
Adams, R. A., Perrinet, L. U., & Friston, K. (2012) · PLoS ONE · doi:10.1371/journal.pone.0047502 · Cited by 91 · cited 4×
“This means that while our discrete time models are further abstracted from brain function than models we have considered elsewhere (for example (Adams et al., 2012; Friston et al., 2012)) they are still capable of embodying mechanistic hypotheses about cognition, which is essential if we want to…”
BackgroundMethodInfluentialSelf-citation
Research works printed but not linked: 34
- Attias H., 2000, Advances in neural information processing systems, 12, 209
- 10.1037/a0032079 · DOI · Paper page
- 10.1016/j.schres.2010.08.009 · DOI · Paper page
- Beal M. J., 2003, Variational algorithms for approximate Bayesian inference
- 10.1016/j.biopsych.2005.11.007 · DOI · Paper page
- 10.1017/S0033291707002152 · DOI · Paper page
- 10.1016/j.neuron.2009.01.031 · DOI · Paper page
- 10.1038/nature04766 · DOI · Paper page
- 10.3389/fnins.2012.00075 · DOI · Paper page
- Djamshidian, A., O’Sullivan, S. S., Sanotsky, Y., Sharman, S., Matviyenko, Y., Foltynie, T. … Averbeck, B. B. (2012). Decision making, impulsivity, and addictions: Do Parkinson’s disease patients jump to conclusions? Mov. Disord., 27, 1137–1145. · DOI
- 10.1371/journal.pone.0058536 · DOI
- 10.1017/S0033291796004175 · DOI · Paper page
- 10.1016/S0301-0082(02)00076-X · DOI · Paper page
- 10.1098/rstb.2005.1622 · DOI · Paper page
- 10.1523/JNEUROSCI.4236-11.2011 · DOI · Paper page
- 10.1097/00005053-199104000-00003 · DOI · Paper page
- 10.1371/journal.pone.0053344 · DOI
- 10.1037/h0022886 · DOI · Paper page
- 10.1523/JNEUROSCI.17-19-07450.1997 · DOI · Paper page
- 10.1111/j.1748-6653.2011.02005.x · DOI · Paper page
- 10.1016/j.schres.2007.08.021 · DOI
- 10.1016/j.tics.2011.11.018 · DOI · Paper page
- 10.1348/014466505X35678 · DOI · Paper page
- 10.1126/science.275.5306.1593 · DOI · Paper page
- 10.3389/fpsyg.2013.00710 · DOI · Paper page
- Schwartenbeck P., 2014, Cereb. Cortex.
- 10.4249/scholarpedia.3737 · doi:10.4249/scholarpedia.3737
- 10.1093/brain/aws083 · DOI · Paper page
- 10.1073/pnas.91.12.5720 · DOI · Paper page
- 10.1002/ana.21809 · DOI · Paper page
- 10.1016/j.biopsych.2006.09.042 · DOI · Paper page
- 10.1038/nature05852 · DOI · Paper page
- 10.1016/j.bcp.2008.01.018 · DOI
- 10.1523/JNEUROSCI.11-03-00641.1991 · DOI · Paper page
Cited by
Active inference: a process theory
Friston, K., FitzGerald, T., Rigoli, F., Schwartenbeck, P., & Pezzulo, G. (2017) · Neural Computation · doi:10.1162/neco_a_00912 · Cited by 1,156
Self-citationFriston, K., FitzGerald, T., Rigoli, F., Schwartenbeck, P., ODoherty, J., & Pezzulo, G. (2016) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2016.06.022 · Cited by 674
Self-citationActive Inference, Curiosity and Insight
Friston, K. J., Lin, M., Frith, C. D., Pezzulo, G., Hobson, J. A., & Ondobaka, S. (2017) · Neural Computation · doi:10.1162/neco_a_00999 · Cited by 351
Self-citationA tutorial on the free-energy framework for modelling perception and learning
Bogacz, R. (2017) · Journal of Mathematical Psychology · doi:10.1016/j.jmp.2015.11.003 · Cited by 283
The dopaminergic midbrain encodes the expected certainty about desired outcomes
Schwartenbeck, P., FitzGerald, T. H. B., Mathys, C., Dolan, R., & Friston, K. (2015) · Cerebral Cortex · doi:10.1093/cercor/bhu159 · Cited by 219
Self-citation
Metadata
- Authors
- FitzGerald, T. H. B., Schwartenbeck, P., Moutoussis, M., Dolan, R. J., & Friston, K.
- Venue
- Neural Computation
- Publisher
- MIT Press - Journals
- Published
- 2015-02
- Volume
- 27
- Issue
- 2
- Pages
- 306-328
- ISSN
- 0899-7667 (ISSN-L)
- 1530-888X
- OpenAlex
- W2012418261
- Cited by
- 81
- References
- 28
- FWCI
- 3.23
- Type
- journal article
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.