Optimal inference with suboptimal models: Addiction and active Bayesian inference
Open accessThe DOI matches its Crossref record; 48 of the 48 references it cites were found in a registry or an index.
Abstract
When casting behaviour as active (Bayesian) inference, optimal inference is defined with respect to an agent's beliefs - based on its generative model of the world. This contrasts with normative accounts of choice behaviour, in which optimal actions are considered in relation to the true structure of the environment - as opposed to the agent's beliefs about worldly states (or the task). This distinction shifts an understanding of suboptimal or pathological behaviour away from aberrant inference as such, to understanding the prior beliefs of a subject that cause them to behave less 'optimally' than our prior beliefs suggest they should behave. Put simply, suboptimal or pathological behaviour does not speak against understanding behaviour in terms of (Bayes optimal) inference, but rather calls for a more refined understanding of the subject's generative model upon which their (optimal) Bayesian inference is based. Here, we discuss this fundamental distinction and its implications for understanding optimality, bounded rationality and pathological (choice) behaviour. We illustrate our argument using addictive choice behaviour in a recently described 'limited offer' task. Our simulations of pathological choices and addictive behaviour also generate some clear hypotheses, which we hope to pursue in ongoing empirical work.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Tversky, A., & Kahneman, D. (1974) · Science · doi:10.1126/science.185.4157.1124 · Cited by 28,810
Anxiety and cognitive performance: Attentional control theory
Eysenck, M. W., Derakshan, N., Santos, R., & Calvo, M. G. (2007) · Emotion · doi:10.1037/1528-3542.7.2.336 · Cited by 5,289 · cited 1×
“, a lack of confidence or precision) [33,34,44].”
BackgroundA theory of cortical responses.
Friston, K. (2005) · Philosophical Transactions of the Royal Society B: Biological Sciences · doi:10.1098/rstb.2005.1622 · Cited by 4,578 · cited 1×
“Recent theoretical attempts to provide a unified understanding of behaviour and brain function in a dynamic and uncertain world cast cognition as probabilistic Bayesian inference [4,47,48,76], culminating in the Bayesian brain hypothesis ([27,36,37,54,65].”
BackgroundSelf-citationGigerenzer, G., & Gaissmaier, W. (2011) · Annual Review of Psychology · doi:10.1146/annurev-psych-120709-145346 · Cited by 4,037 · cited 1×
“ing some to question whether humans truly perform Bayesian inference and suggesting that in many situations non-optimal models are employed; for example, simple heuristics when making decisions [17,43,52,77,78].”
BackgroundJ Neuropsychiatry Clin Neurosci
Yudofsky, S. C., & Hales, R. E. (1992) · The Journal of Neuropsychiatry and Clinical Neurosciences · doi:10.1176/jnp.4.4.359 · Cited by 3,818
Neural systems of reinforcement for drug addiction: from actions to habits to compulsion
Everitt, B. J., & Robbins, T. W. (2005) · Nature Neuroscience · doi:10.1038/nn1579 · Cited by 3,785 · cited 2×
“, a lack of confidence or precision) [33,34,44].”
BackgroundBounded rationality and organizational learning
Simon, H. A. (1991) · Organization Science · doi:10.1287/orsc.2.1.125 · Cited by 3,028 · cited 1×
“closely related to the concept of bounded rationality and satisficing as opposed to maximising [70,71].”
BackgroundThe Bayesian brain: The role of uncertainty in neural coding and computation
Knill, D. C., & Pouget, A. (2004) · Trends in Neurosciences · doi:10.1016/j.tins.2004.10.007 · Cited by 2,588 · cited 1×
“Recent theoretical attempts to provide a unified understanding of behaviour and brain function in a dynamic and uncertain world cast cognition as probabilistic Bayesian inference [4,47,48,76], culminating in the Bayesian brain hypothesis ([27,36,37,54,65].”
BackgroundThe debate over dopamine’s role in reward: the case for incentive salience
Berridge, K. C. (2007) · Psychopharmacology · doi:10.1007/s00213-006-0578-x · Cited by 2,347
Decision making, impulse control and loss of willpower to resist drugs: a neurocognitive perspective
Bechara, A. (2005) · Nature Neuroscience · doi:10.1038/nn1584 · Cited by 2,149 · cited 2×
“Individuals with addiction have been found to be more impulsive [3,6,32], and neuronal activation asso-”
BackgroundThe free-energy principle: a rough guide to the brain?
Friston, K. (2009) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2009.04.005 · Cited by 1,831
Self-citationHabits, Rituals, and the Evaluative Brain
Graybiel, A. M. (2008) · Annual Review of Neuroscience · doi:10.1146/annurev.neuro.29.051605.112851 · Cited by 1,697 · cited 1×
“to stimulus–response control [29], resulting in a predominance of less flexible, habitual modes of behaviour [32,45,67].”
BackgroundHierarchical Bayesian inference in the visual cortex
Lee, T. S., & Mumford, D. (2003) · Journal of the Optical Society of America A · doi:10.1364/josaa.20.001434 · Cited by 1,473 · cited 1×
“that the brain may indeed perform (close to) ‘optimal’ probabilistic inference [18,48,55,82,83].”
BackgroundObject perception as Bayesian inference
Kersten, D., Mamassian, P., & Yuille, A. (2004) · Annual Review of Psychology · doi:10.1146/annurev.psych.55.090902.142005 · Cited by 1,269 · cited 1×
“which model is best, inducing the need for Bayesian model averaging [35], or incompatible sensory data and priors; for example, in the case of perceptual illusions [42,53,79].”
BackgroundDayan, P., Hinton, G. E., Neal, R. M., & Zemel, R. S. (1995) · Neural Computation · doi:10.1162/neco.1995.7.5.889 · Cited by 1,207
When the world becomes ‘too real’: a Bayesian explanation of autistic perception
Pellicano, E., & Burr, D. (2012) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2012.08.009 · Cited by 1,156 · cited 1×
“approaches have been proposed in the field of autism [62], schizophrenia [1] or functional motor and sensory symptoms [31].”
BackgroundMotion illusions as optimal percepts
Weiss, Y., Simoncelli, E. P., & Adelson, E. H. (2002) · Nature Neuroscience · doi:10.1038/nn0602-858 · Cited by 1,076
High Impulsivity Predicts the Switch to Compulsive Cocaine-Taking
Belin, D., Mar, A. C., Dalley, J. W., Robbins, T. W., & Everitt, B. J. (2008) · Science · doi:10.1126/science.1158136 · Cited by 989
Dolan, R. J., & Dayan, P. (2013) · Neuron · doi:10.1016/j.neuron.2013.09.007 · Cited by 977
Self-citationThe Computational Anatomy of Psychosis
Adams, R. A., Stephan, K. E., Brown, H. R., Frith, C. D., & Friston, K. J. (2013) · Frontiers in Psychiatry · doi:10.3389/fpsyt.2013.00047 · Cited by 850 · cited 2×
“Indeed, identifying particular characteristics of generative models associated with a particular psychiatric disorder marks a central aim of computational psychiatry [2,58,60,72].”
BackgroundSelf-citationA Bayesian account of ‘hysteria’
Edwards, M. J., Adams, R. A., Brown, H., Parees, I., & Friston, K. J. (2012) · Brain · doi:10.1093/brain/aws129 · Cited by 825 · cited 1×
“approaches have been proposed in the field of autism [62], schizophrenia [1] or functional motor and sensory symptoms [31].”
BackgroundSelf-citationMontague, P. R., Dolan, R. J., Friston, K. J., & Dayan, P. (2012) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2011.11.018 · Cited by 809
Self-citationEvery Good Regulator of a System Must Be a Model of That System
Conant, R. C., & Ashby, W. R. (1991) · Facets of Systems Science · doi:10.1007/978-1-4899-0718-9_37 · Cited by 712
From prediction error to incentive salience: mesolimbic computation of reward motivation.
Berridge, K. C. (2012) · European Journal of Neuroscience · doi:10.1111/j.1460-9568.2012.07990.x · Cited by 700 · cited 1×
“Other perspectives invoke a shift from a predominance of liking to wanting in the context of incentive salience [8,9], implying that addicts show compulsive ‘wanting’ of drugs or drug-like behaviour without actually ‘liking’ it [68].”
BackgroundThe neural mechanisms of inter-temporal decision-making: understanding variability
Peters, J., & Büchel, C. (2011) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2011.03.002 · Cited by 693 · cited 1×
“Addiction is associated with characteristic patterns of maladaptive choice behaviour [49,61,63], and thus appears to be ideally suited for a computational approach to understand the basis of aberrant decision processes.”
BackgroundProbabilistic population codes for Bayesian decision making
Beck, J. M., Ma, W. J., Kiani, R., Hanks, T., Churchland, A. K., Roitman, J., Shadlen, M. N., Latham, P. E., & Pouget, A. (2008) · Neuron · doi:10.1016/j.neuron.2008.09.021 · Cited by 645
Reward deficiency syndrome: genetic aspects of behavioral disorders
Comings, D. E., & Blum, K. (2000) · Progress in Brain Research · doi:10.1016/s0079-6123(00)26022-6 · Cited by 621
Probabilistic brains: knowns and unknowns
Pouget, A., Beck, J. M., Ma, W. J., & Latham, P. E. (2013) · Nature Neuroscience · doi:10.1038/nn.3495 · Cited by 615 · cited 2×
“Recent theoretical attempts to provide a unified understanding of behaviour and brain function in a dynamic and uncertain world cast cognition as probabilistic Bayesian inference [4,47,48,76], culminating in the Bayesian brain hypothesis ([27,36,37,54,65].”
BackgroundGillan, C. M., Papmeyer, M., Morein-Zamir, S., Sahakian, B. J., Fineberg, N. A., Robbins, T. W., & de Wit, S. (2011) · American Journal of Psychiatry · doi:10.1176/appi.ajp.2011.10071062 · Cited by 609 · cited 1×
“, a lack of confidence or precision) [33,34,44].”
BackgroundBickel, W. K., Miller, M. L., Yi, R., Kowal, B. P., Lindquist, D. M., & Pitcock, J. A. (2007) · Drug and Alcohol Dependence · doi:10.1016/j.drugalcdep.2006.09.016 · Cited by 571
Computational psychiatry: the brain as a phantastic organ
Friston, K. J., Stephan, K. E., Montague, R., & Dolan, R. J. (2014) · The Lancet Psychiatry · doi:10.1016/s2215-0366(14)70275-5 · Cited by 531 · cited 1×
“is the central procedure in the field of computational psychiatry [29,39,41,60,72].”
BackgroundSelf-citationA unified framework for addiction: Vulnerabilities in the decision process
Redish, A. D., Jensen, S., & Johnson, A. (2008) · Behavioral and Brain Sciences · doi:10.1017/s0140525x0800472x · Cited by 501
Bayesian just-so stories in psychology and neuroscience.
Bowers, J. S., & Davis, C. J. (2012) · Psychological Bulletin · doi:10.1037/a0026450 · Cited by 436 · cited 1×
“In other instances, it has been shown that organisms substantially deviate from Bayes-optimality defined with respect to the true generative process [15,16,19,46,51], lead-”
BackgroundA free energy principle for biological systems
Karl, F. (2012) · Entropy · doi:10.3390/e14112100 · Cited by 377 · cited 1×
“imising reward [74] or minimising surprise [38].”
Redish, A. D., Jensen, S., Johnson, A., & Kurth-Nelson, Z. (2007) · Psychological Review · doi:10.1037/0033-295x.114.3.784 · Cited by 346 · cited 1×
“to stimulus–response control [29], resulting in a predominance of less flexible, habitual modes of behaviour [32,45,67].”
BackgroundThe anatomy of choice: active inference and agency
Friston, K., Schwartenbeck, P., FitzGerald, T., Moutoussis, M., Behrens, T., & Dolan, R. J. (2013) · Frontiers in Human Neuroscience · doi:10.3389/fnhum.2013.00598 · Cited by 340 · cited 1×
“backwards induction and dynamic programming, or whether one takes into account the actual cognitive resources and neuronal message-passing necessary to perform a process given the computational resources of the brain [40].”
BackgroundSelf-citationNot Noisy, Just Wrong: The Role of Suboptimal Inference in Behavioral Variability
Beck, J. M., Ma, W. J., Pitkow, X., Latham, P. E., & Pouget, A. (2012) · Neuron · doi:10.1016/j.neuron.2012.03.016 · Cited by 316 · cited 1×
“In computational neuroscience one tries to understand how a certain problem could be solved in the brain to motivate a generative model of a task that might be implemented in the brain in a biologically plausible way.”
BackgroundComputational approaches to psychiatry
Stephan, K. E., & Mathys, C. (2014) · Current Opinion in Neurobiology · doi:10.1016/j.conb.2013.12.007 · Cited by 260
Self-citationThe dopamine hypothesis of drug addiction and its potential therapeutic value
Diana, M. (2011) · Frontiers in Psychiatry · doi:10.3389/fpsyt.2011.00064 · Cited by 251
Illusions, perception and Bayes
Geisler, W. S., & Kersten, D. (2002) · Nature Neuroscience · doi:10.1038/nn0602-508 · Cited by 249 · cited 1×
“which model is best, inducing the need for Bayesian model averaging [35], or incompatible sensory data and priors; for example, in the case of perceptual illusions [42,53,79].”
BackgroundMiedl, S. F. (2012) · Archives of General Psychiatry · doi:10.1001/archgenpsychiatry.2011.1552 · Cited by 229 · cited 2×
“Furthermore, addictive choice behaviour has been related to steeper delay discounting [59,64,66,73] – with the implication that the negative future consequences of addictive behaviour are strongly devalued – as well”
BackgroundThe dopaminergic midbrain encodes the expected certainty about desired outcomes
Schwartenbeck, P., FitzGerald, T. H. B., Mathys, C., Dolan, R., & Friston, K. (2015) · Cerebral Cortex · doi:10.1093/cercor/bhu159 · Cited by 219
Self-citationProbabilistic models in human sensorimotor control
Wolpert, D. M. (2007) · Human Movement Science · doi:10.1016/j.humov.2007.05.005 · Cited by 212
Observing the observer (I): meta-bayesian models of learning and decision-making
Daunizeau, J., den Ouden, H. E. M., Pessiglione, M., Kiebel, S. J., Stephan, K. E., & Friston, K. J. (2010) · PLoS ONE · doi:10.1371/journal.pone.0015554 · Cited by 167 · cited 1×
“We performed maximum a posteriori (MAP) modelling of taskrelevant parameters (expected precision, hazard rate and sensitivity) by inverting the generative model of each subject, given observed behaviour.”
MethodSelf-citationImaging brain response to reward in addictive disorders
Hommer, D. W., Bjork, J. M., & Gilman, J. M. (2011) · Annals of the New York Academy of Sciences · doi:10.1111/j.1749-6632.2010.05898.x · Cited by 163
Computational and dynamic models in neuroimaging
Friston, K. J., & Dolan, R. J. (2010) · NeuroImage · doi:10.1016/j.neuroimage.2009.12.068 · Cited by 131
Self-citationModel averaging, optimal inference, and habit formation
FitzGerald, T. H. B., Dolan, R. J., & Friston, K. J. (2014) · Frontiers in Human Neuroscience · doi:10.3389/fnhum.2014.00457 · Cited by 121
Self-citationSmooth pursuit and visual occlusion: active inference and oculomotor control in schizophrenia
Adams, R. A., Perrinet, L. U., & Friston, K. (2012) · PLoS ONE · doi:10.1371/journal.pone.0047502 · Cited by 91 · cited 1×
“approaches have been proposed in the field of autism [62], schizophrenia [1] or functional motor and sensory symptoms [31].”
BackgroundSelf-citation
Cited by
Active inference: a process theory
Friston, K., FitzGerald, T., Rigoli, F., Schwartenbeck, P., & Pezzulo, G. (2017) · Neural Computation · doi:10.1162/neco_a_00912 · Cited by 1,156
Self-citationFriston, K., FitzGerald, T., Rigoli, F., Schwartenbeck, P., ODoherty, J., & Pezzulo, G. (2016) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2016.06.022 · Cited by 674
Self-citationAllostatic Self-efficacy: A Metacognitive Theory of Dyshomeostasis-Induced Fatigue and Depression
Stephan, K. E., Manjaly, Z. M., Mathys, C. D., Weber, L. A. E., Paliwal, S., Gard, T., Tittgemeyer, M., Fleming, S. M., Haker, H., Seth, A. K., et al. (2016) · Frontiers in Human Neuroscience · doi:10.3389/fnhum.2016.00550 · Cited by 437
Self-citationThe neurobiology of interoception in health and disease
Quadt, L., Critchley, H. D., & Garfinkel, S. N. (2018) · Annals of the New York Academy of Sciences · doi:10.1111/nyas.13915 · Cited by 434
From cognitivism to autopoiesis: towards a computational framework for the embodied mind
Allen, M., & Friston, K. J. (2018) · Synthese · doi:10.1007/s11229-016-1288-5 · Cited by 386
Self-citation
Metadata
- Authors
- Schwartenbeck, P., FitzGerald, T. H. B., Mathys, C., Dolan, R., Wurst, F., Kronbichler, M., & Friston, K.
- Venue
- Medical Hypotheses
- Publisher
- Elsevier BV
- Published
- 2015-02
- Volume
- 84
- Issue
- 2
- Pages
- 109-117
- ISSN
- 0306-9877 (ISSN-L)
- OpenAlex
- W2135713127
- Cited by
- 108
- References
- 48
- FWCI
- 5.29
- Type
- journal article
- Language
- en
- Open access
- hybrid
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.