Learning the value of information in an uncertain world
The DOI matches its Crossref record; 52 of the 52 references it cites were found in a registry or an index.
Abstract
Our decisions are guided by outcomes that are associated with decisions made in the past. However, the amount of influence each past outcome has on our next decision remains unclear. To ensure optimal decision-making, the weight given to decision outcomes should reflect their salience in predicting future outcomes, and this salience should be modulated by the volatility of the reward environment. We show that human subjects assess volatility in an optimal manner and adjust decision-making accordingly. This optimal estimate of volatility is reflected in the fMRI signal in the anterior cingulate cortex (ACC) when each trial outcome is observed. When a new piece of information is witnessed, activity levels reflect its salience for predicting future outcomes. Furthermore, variations in this ACC signal across the population predict variations in subject learning rates. Our results provide a formal account of how we weigh our different experiences in guiding our future actions.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“When the environment is volatile people should give more weight to recent sensory outcomes in building expectations about what they will see next (e.g. adopt a high α), in contrast they should give information from the distant past more weight when the environment is stable (e.g. adopt a low α) 8,9.”
From Adults with autism overestimate the volatility of the sensory environment (2017) · cites this paper as [8]
In the citing paper
“While computational studies of decision-making about rewards and punishments show that participants adapt their rate of learning about action-outcome contingencies in response to changes in environmental volatility 8–10”
From Adults with autism overestimate the volatility of the sensory environment (2017) · cites this paper as [8]
In the citing paper
“Previous work has shown that human learners indeed adjust their learning rate according to experimentally manipulated levels of volatility (Behrens et al., 2007).”
From The generalized Hierarchical Gaussian Filter (2026) · cites this paper as Behrens et al., 2007
In the citing paper
“This is known as hierarchical filtering and emphasizes the importance of taking into account the volatility of the environment, i.e., the temporal instability of its statistical structure, such as the probabilities by which one event causes another (Behrens et al., 2007; Mathys et al., 2011).”
From Allostatic Self-efficacy: A Metacognitive Theory of Dyshomeostasis-Induced Fatigue and Depression (2016) · cites this paper as Behrens et al., 2007
In the citing paper
“The causal structure of the world, with its nested spatial and temporal scales, implies that the brain’s internal model also possesses a hierarchical structure, which is a natural form for Bayesian inference: hierarchical models allow to encode information about the precision of beliefs at one level by values of hierarchically higher levels (81–83) (Figure 3D).”
From Can Bayesian Theories of Autism Spectrum Disorder Help Improve Clinical Practice? (2016) · cites this paper as [82]
1 more claim about this paper
Create a free account to see every claim other papers make about this one.
Citations
Cites
Reinforcement Learning: An Introduction
Sutton, R. S., & Barto, A. G. (1998) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.1998.712192 · Cited by 27,225
Reinforcement Learning: An Introduction
(2005) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.2004.842673 · Cited by 24,496
Engle, R. F. (1982) · Econometrica · doi:10.2307/1912773 · Cited by 20,795
Advances in functional and structural MR image analysis and implementation as FSL
Smith, S. M., Jenkinson, M., Woolrich, M. W., Beckmann, C. F., Behrens, T. E. J., Johansen-Berg, H., Bannister, P. R., De Luca, M., Drobnjak, I., Flitney, D. E., et al. (2004) · NeuroImage · doi:10.1016/j.neuroimage.2004.07.051 · Cited by 13,630
Self-citationKahneman, D., & Tversky, A. (1984) · American Psychologist · doi:10.1037/0003-066x.39.4.341 · Cited by 6,544
Humans integrate visual and haptic information in a statistically optimal fashion
Ernst, M. O., & Banks, M. S. (2002) · Nature · doi:10.1038/415429a · Cited by 4,844
Aston-Jones, G., & Cohen, J. D. (2005) · Annual Review of Neuroscience · doi:10.1146/annurev.neuro.28.061604.135709 · Cited by 4,472
Classical conditioning II: Current research and theory.
Black, A. H., & Prokasy, W. F. (1972) · Appleton-Century-Crofts eBooks · Cited by 4,449
Conflict monitoring and anterior cingulate cortex: an update
Botvinick, M. M., Cohen, J. D., & Carter, C. S. (2004) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2004.10.003 · Cited by 3,623
Pearce, J. M., & Hall, G. (1980) · Psychological Review · doi:10.1037/0033-295x.87.6.532 · Cited by 3,023
Cited by
The expected value of control: an integrative theory of anterior cingulate cortex function
Shenhav, A., Botvinick, M. M., & Cohen, J. D. (2013) · Neuron · doi:10.1016/j.neuron.2013.07.007 · Cited by 2,321
The Social Brain: Neural Basis of Social Knowledge
Adolphs, R. (2009) · Annual Review of Psychology · doi:10.1146/annurev.psych.60.110707.163514 · Cited by 1,827
Gut feelings: the emerging biology of gut-brain communication
Mayer, E. A. (2011) · Nature Reviews Neuroscience · doi:10.1038/nrn3071 · Cited by 1,726
A common role of insula in feelings, empathy and uncertainty
Singer, T., Critchley, H. D., & Preuschoff, K. (2009) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2009.05.001 · Cited by 1,270
What Is a Cognitive Map? Organizing Knowledge for Flexible Behavior
Behrens, T. E. J., Muller, T. H., Whittington, J. C. R., Mark, S., Baram, A. B., Stachenfeld, K. L., & Kurth-Nelson, Z. (2018) · Neuron · doi:10.1016/j.neuron.2018.10.002 · Cited by 1,055
Self-citationGläscher, J., Daw, N., Dayan, P., & O'Doherty, J. P. (2010) · Neuron · doi:10.1016/j.neuron.2010.04.016 · Cited by 1,166
Frontal cortex and reward-guided learning and decision-making
Rushworth, M. F. S., Noonan, M. P., Boorman, E. D., Walton, M. E., & Behrens, T. E. (2011) · Neuron · doi:10.1016/j.neuron.2011.05.014 · Cited by 1,111
Self-citationMedial prefrontal cortex as an action-outcome predictor
Alexander, W. H., & Brown, J. W. (2011) · Nature Neuroscience · doi:10.1038/nn.2921 · Cited by 1,000
The Neurobiology of Decision: Consensus and Controversy
Kable, J. W., & Glimcher, P. W. (2009) · Neuron · doi:10.1016/j.neuron.2009.09.003 · Cited by 928
Associative learning of social value
Behrens, T. E. J., Hunt, L. T., Woolrich, M. W., & Rushworth, M. F. S. (2008) · Nature · doi:10.1038/nature07538 · Cited by 925
Self-citationLiu, X., Hairston, J., Schrier, M., & Fan, J. (2011) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2010.12.012 · Cited by 916
Precise minds in uncertain worlds: Predictive coding in autism.
Van de Cruys, S., Evers, K., Van der Hallen, R., Van Eylen, L., Boets, B., de-Wit, L., & Wagemans, J. (2014) · Psychological Review · doi:10.1037/a0037665 · Cited by 885
Choice, uncertainty and value in prefrontal and cingulate cortex
Rushworth, M. F. S., & Behrens, T. E. J. (2008) · Nature Neuroscience · doi:10.1038/nn2066 · Cited by 866
Self-citationNeural correlates, computation and behavioural impact of decision confidence
Kepecs, A., Uchida, N., Zariwala, H. A., & Mainen, Z. F. (2008) · Nature · doi:10.1038/nature07200 · Cited by 809
The orbitofrontal cortex and beyond: From affect to decision-making
Rolls, E. T., & Grabenhorst, F. (2008) · Progress in Neurobiology · doi:10.1016/j.pneurobio.2008.09.001 · Cited by 807
Human insula activation reflects risk prediction errors as well as risk
Preuschoff, K., Quartz, S. R., & Bossaerts, P. (2008) · The Journal of Neuroscience · doi:10.1523/jneurosci.4286-07.2008 · Cited by 784
Fox, K. C. R., Nijeboer, S., Dixon, M. L., Floman, J. L., Ellamil, M., Rumak, S. P., Sedlmeier, P., & Christoff, K. (2014) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2014.03.016 · Cited by 804
Beckmann, M., Johansen-Berg, H., & Rushworth, M. F. S. (2009) · The Journal of Neuroscience · doi:10.1523/jneurosci.3328-08.2009 · Cited by 789
Self-citationInformatic parcellation of the network involved in the computation of subjective value
Clithero, J. A., & Rangel, A. (2014) · Social Cognitive and Affective Neuroscience · doi:10.1093/scan/nst106 · Cited by 706
Reinforcement learning in the brain
Niv, Y. (2009) · Journal of Mathematical Psychology · doi:10.1016/j.jmp.2008.12.005 · Cited by 717
Rational regulation of learning dynamics by pupil-linked arousal systems
Nassar, M. R., Rumsey, K. M., Wilson, R. C., Parikh, K., Heasly, B., & Gold, J. I. (2012) · Nature Neuroscience · doi:10.1038/nn.3130 · Cited by 722
Navigating Into the Future or Driven by the Past
Seligman, M. E. P., Railton, P., Baumeister, R. F., & Sripada, C. (2013) · Perspectives on Psychological Science · doi:10.1177/1745691612474317 · Cited by 720
Reinforcement learning: The Good, The Bad and The Ugly
Dayan, P., & Niv, Y. (2008) · Current Opinion in Neurobiology · doi:10.1016/j.conb.2008.08.003 · Cited by 704
Decision Making in Recurrent Neuronal Circuits
Wang, X.-J. (2008) · Neuron · doi:10.1016/j.neuron.2008.09.034 · Cited by 679
“Some studies aimed at identifying distinct brain systems that are recruited by different types of uncertainty (Yu and Dayan, 2005; Hsu et al., 2005; Huettel et al., 2006; Behrens et al., 2007).”
BackgroundMetacognition in human decision-making: confidence and error monitoring
Yeung, N., & Summerfield, C. (2012) · Philosophical Transactions of the Royal Society B: Biological Sciences · doi:10.1098/rstb.2011.0416 · Cited by 667
Boorman, E. D., Behrens, T. E. J., Woolrich, M. W., & Rushworth, M. F. S. (2009) · Neuron · doi:10.1016/j.neuron.2009.05.014 · Cited by 665
Self-citationNeuroeconomics: Decision Making and the Brain
Rushworth, M. F. S., & Walton, M. E. (2009) · Neuron · doi:10.1016/j.neuron.2009.07.005 · Cited by 658
Self-citationA Bayesian foundation for individual learning under uncertainty
Mathys, C. (2011) · Frontiers in Human Neuroscience · doi:10.3389/fnhum.2011.00039 · Cited by 655
Uncertainty and stress: Why it causes diseases and how it is mastered by the brain
Peters, A., McEwen, B. S., & Friston, K. (2017) · Progress in Neurobiology · doi:10.1016/j.pneurobio.2017.05.004 · Cited by 652
“…stress-related responses are broadcast by the ACC to the anterior insular and onto autonomic centers to elicit autonomic reflexes (Barrett and Simmons, 2015; Behrens et al., 2007; Feinstein et al., 2006; Karlsson et al., 2012; Liljeholm et al., 2013; Paulus et al., 2002; Sarinopoulos et al., 2010).”
Doya, K. (2008) · Nature Neuroscience · doi:10.1038/nn2077 · Cited by 642
An aberrant precision account of autism
Lawson, R. P., Rees, G., & Friston, K. J. (2014) · Frontiers in Human Neuroscience · doi:10.3389/fnhum.2014.00302 · Cited by 641
Neurophysiology of performance monitoring and adaptive behavior
Ullsperger, M., Danielmeier, C., & Jocham, G. (2014) · Physiological Reviews · doi:10.1152/physrev.00041.2012 · Cited by 634
Frontal Cortex and the Hierarchical Control of Behavior
Badre, D., & Nee, D. E. (2018) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2017.11.005 · Cited by 611
Prefrontal cortex as a meta-reinforcement learning system
Wang, J. X., Kurth-Nelson, Z., Kumaran, D., Tirumala, D., Soyer, H., Leibo, J. Z., Hassabis, D., & Botvinick, M. (2018) · Nature Neuroscience · doi:10.1038/s41593-018-0147-8 · Cited by 596
Learning to reinforcement learn
Wang, J. X., Kurth-Nelson, Z., Tirumala, D., Soyer, H., Leibo, J. Z., Munos, R., Blundell, C., Kumaran, D., & Botvinick, M. (2016) · arXiv · doi:10.48550/arxiv.1611.05763 · Cited by 579
Nassar, M. R., Wilson, R. C., Heasly, B., & Gold, J. I. (2010) · The Journal of Neuroscience · doi:10.1523/jneurosci.0822-10.2010 · Cited by 560
Conflict-induced behavioural adjustment: a clue to the executive functions of the prefrontal cortex
Mansouri, F. A., Tanaka, K., & Buckley, M. J. (2009) · Nature Reviews Neuroscience · doi:10.1038/nrn2538 · Cited by 559
Reasoning, Learning, and Creativity: Frontal Lobe Function and Human Decision-Making
Collins, A., & Koechlin, E. (2012) · PLoS Biology · doi:10.1371/journal.pbio.1001293 · Cited by 558
“Neuroimaging studies further show that dorsomedial prefrontal regions evaluate the discrepancies between actual and predicted action outcomes [17,52] and estimate the volatility of external contingencies [14].”
BackgroundMethodResultInfluentialNeural Basis of Reinforcement Learning and Decision Making
Lee, D., Seo, H., & Jung, M. W. (2012) · Annual Review of Neuroscience · doi:10.1146/annurev-neuro-062111-150512 · Cited by 546
Midbrain Dopamine Neurons Signal Preference for Advance Information about Upcoming Rewards
Bromberg-Martin, E. S., & Hikosaka, O. (2009) · Neuron · doi:10.1016/j.neuron.2009.06.009 · Cited by 548
Brain plasticity through the life span: Learning to learn and action video games
Bavelier, D., Green, C. S., Pouget, A., & Schrater, P. (2012) · Annual Review of Neuroscience · doi:10.1146/annurev-neuro-060909-152832 · Cited by 541
Neural computations associated with goal-directed choice
Rangel, A., & Hare, T. (2010) · Current Opinion in Neurobiology · doi:10.1016/j.conb.2010.03.001 · Cited by 532
Adults with autism overestimate the volatility of the sensory environment
Lawson, R. P., Mathys, C., & Rees, G. (2017) · Nature Neuroscience · doi:10.1038/nn.4615 · Cited by 470
Pupil dilation signals surprise: evidence for noradrenaline’s role in decision making
Preuschoff, K. (2011) · Frontiers in Neuroscience · doi:10.3389/fnins.2011.00115 · Cited by 519
How prediction errors shape perception, attention, and motivation
den Ouden, H. E. M., Kok, P., & de Lange, F. P. (2012) · Frontiers in Psychology · doi:10.3389/fpsyg.2012.00548 · Cited by 512
Robust group analysis using outlier inference
Woolrich, M. (2008) · NeuroImage · doi:10.1016/j.neuroimage.2008.02.042 · Cited by 504
Self-citationPosterior cingulate cortex: adapting behavior to a changing world
Pearson, J. M., Heilbronner, S. R., Barack, D. L., Hayden, B. Y., & Platt, M. L. (2011) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2011.02.002 · Cited by 489
Modalities, Modes, and Models in Functional Neuroimaging
Friston, K. J. (2009) · Science · doi:10.1126/science.1174521 · Cited by 489
Mapping anhedonia onto reinforcement learning: a behavioural meta-analysis
Huys, Q. J., Pizzagalli, D. A., Bogdan, R., & Dayan, P. (2013) · Biology of Mood & Anxiety Disorders · doi:10.1186/2045-5380-3-12 · Cited by 480
Anxious individuals have difficulty learning the causal statistics of aversive environments
Browning, M., Behrens, T. E., Jocham, G., O'Reilly, J. X., & Bishop, S. J. (2015) · Nature Neuroscience · doi:10.1038/nn.3961 · Cited by 476
Self-citationLesion mapping of cognitive control and value-based decision making in the prefrontal cortex
Gläscher, J., Adolphs, R., Damasio, H., Bechara, A., Rudrauf, D., Calamia, M., Paul, L. K., & Tranel, D. (2012) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1206608109 · Cited by 476
Allostatic Self-efficacy: A Metacognitive Theory of Dyshomeostasis-Induced Fatigue and Depression
Stephan, K. E., Manjaly, Z. M., Mathys, C. D., Weber, L. A. E., Paliwal, S., Gard, T., Tittgemeyer, M., Fleming, S. M., Haker, H., Seth, A. K., et al. (2016) · Frontiers in Human Neuroscience · doi:10.3389/fnhum.2016.00550 · Cited by 437
“This is known as hierarchical filtering and emphasizes the importance of taking into account the volatility of the environment, i.e., the temporal instability of its statistical structure, such as the probabilities by which one event causes another (Behrens et al., 2007; Mathys et al., 2011).”
BackgroundTrial-by-trial data analysis using computational models
Daw, N. D. (2011) · Decision Making, Affect, and Learning · doi:10.1093/acprof:oso/9780199600434.003.0001 · Cited by 456
Uncertainty in perception and the Hierarchical Gaussian Filter
Mathys, C. D., Lomakina, E. I., Daunizeau, J., Iglesias, S., Brodersen, K. H., Friston, K. J., & Stephan, K. E. (2014) · Frontiers in Human Neuroscience · doi:10.3389/fnhum.2014.00825 · Cited by 465
Testing the behavioral interaction and integration of attentional networks
Fan, J., Gu, X., Guise, K. G., Liu, X., Fossella, J., Wang, H., & Posner, M. I. (2009) · Brain and Cognition · doi:10.1016/j.bandc.2009.02.002 · Cited by 458
Dorsal Anterior Cingulate Cortex: A Bottom-Up View
Heilbronner, S. R., & Hayden, B. Y. (2016) · Annual Review of Neuroscience · doi:10.1146/annurev-neuro-070815-013952 · Cited by 455
Double dissociation of value computations in orbitofrontal and anterior cingulate neurons
Kennerley, S. W., Behrens, T. E. J., & Wallis, J. D. (2011) · Nature Neuroscience · doi:10.1038/nn.2961 · Cited by 454
Self-citationNeurons in the frontal lobe encode the value of multiple decision variables
Kennerley, S. W., Dahmubed, A. F., Lara, A. H., & Wallis, J. D. (2009) · Journal of Cognitive Neuroscience · doi:10.1162/jocn.2009.21100 · Cited by 453
Neural correlates of value, risk, and risk aversion contributing to decision making under risk
Christopoulos, G. I., Tobler, P. N., Bossaerts, P., Dolan, R. J., & Schultz, W. (2009) · The Journal of Neuroscience · doi:10.1523/jneurosci.2614-09.2009 · Cited by 451
Value, search, persistence and model updating in anterior cingulate cortex
Kolling, N., Wittmann, M. K., Behrens, T. E. J., Boorman, E. D., Mars, R. B., & Rushworth, M. F. S. (2016) · Nature Neuroscience · doi:10.1038/nn.4382 · Cited by 444
Self-citationNeural mechanisms and temporal dynamics of performance monitoring
Ullsperger, M., Fischer, A. G., Nigbur, R., & Endrass, T. (2014) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2014.02.009 · Cited by 435
Hauser, T. U., Iannaccone, R., Stämpfli, P., Drechsler, R., Brandeis, D., Walitza, S., & Brem, S. (2014) · NeuroImage · doi:10.1016/j.neuroimage.2013.08.028 · Cited by 428
Attention reverses the effect of prediction in silencing sensory signals
Kok, P., Rahnev, D., Jehee, J. F. M., Lau, H. C., & de Lange, F. P. (2012) · Cerebral Cortex · doi:10.1093/cercor/bhr310 · Cited by 430
Cognitive control over learning: Creating, clustering, and generalizing task-set structure.
Collins, A. G. E., & Frank, M. J. (2013) · Psychological Review · doi:10.1037/a0030852 · Cited by 426
Mohr, P. N. C., Biele, G., & Heekeren, H. R. (2010) · The Journal of Neuroscience · doi:10.1523/jneurosci.0003-10.2010 · Cited by 411
Collins, A. G. E., & Frank, M. J. (2012) · European Journal of Neuroscience · doi:10.1111/j.1460-9568.2011.07980.x · Cited by 417
Re-evaluating the role of TPJ in attentional control: Contextual updating?
Geng, J. J., & Vossel, S. (2013) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2013.08.010 · Cited by 411
Gambling near-misses enhance motivation to gamble and recruit win-related brain circuitry
Clark, L., Lawrence, A. J., Astley-Jones, F., & Gray, N. (2009) · Neuron · doi:10.1016/j.neuron.2008.12.031 · Cited by 410
The Computation of Social Behavior
Behrens, T. E. J., Hunt, L. T., & Rushworth, M. F. S. (2009) · Science · doi:10.1126/science.1169694 · Cited by 407
Self-citationFoundations of human reasoning in the prefrontal cortex
Donoso, M., Collins, A. G. E., & Koechlin, E. (2014) · Science · doi:10.1126/science.1252254 · Cited by 406
The neural correlates of tinnitus-related distress
Vanneste, S., Plazier, M., der Loo, E. V., de Heyning, P. V., Congedo, M., & De Ridder, D. (2010) · NeuroImage · doi:10.1016/j.neuroimage.2010.04.029 · Cited by 401
Walton, M. E., Behrens, T. E. J., Buckley, M. J., Rudebeck, P. H., & Rushworth, M. F. S. (2010) · Neuron · doi:10.1016/j.neuron.2010.02.027 · Cited by 397
Self-citationConfidence as Bayesian probability: From neural origins to behavior
Meyniel, F., Sigman, M., & Mainen, Z. F. (2015) · Neuron · doi:10.1016/j.neuron.2015.09.039 · Cited by 349
Hierarchical Prediction Errors in Midbrain and Basal Forebrain during Sensory Learning
Iglesias, S., Mathys, C., Brodersen, K. H., Kasper, L., Piccirelli, M., den Ouden, H. E. M., & Stephan, K. E. (2013) · Neuron · doi:10.1016/j.neuron.2013.09.009 · Cited by 342
Dissociable effects of surprise and model update in parietal and anterior cingulate cortex
O’Reilly, J. X., Schüffelgen, U., Cuell, S. F., Behrens, T. E. J., Mars, R. B., & Rushworth, M. F. S. (2013) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1305373110 · Cited by 378
Self-citationHayden, B. Y., Heilbronner, S. R., Pearson, J. M., & Platt, M. L. (2011) · The Journal of Neuroscience · doi:10.1523/jneurosci.4652-10.2011 · Cited by 379
Neubert, F.-X., Mars, R. B., Sallet, J., & Rushworth, M. F. S. (2015) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1410767112 · Cited by 375
Self-citationAhn, W.-Y., Haines, N., & Zhang, L. (2017) · Computational Psychiatry · doi:10.1162/cpsy_a_00002 · Cited by 362
Adaptation by binding: a learning account of cognitive control
Verguts, T., & Notebaert, W. (2009) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2009.02.007 · Cited by 356
Differential roles of human striatum and amygdala in associative learning
Li, J., Schiller, D., Schoenbaum, G., Phelps, E. A., & Daw, N. D. (2011) · Nature Neuroscience · doi:10.1038/nn.2904 · Cited by 353
Frontal cortex subregions play distinct roles in choices between actions and stimuli
Rudebeck, P. H., Behrens, T. E., Kennerley, S. W., Baxter, M. G., Buckley, M. J., Walton, M. E., & Rushworth, M. F. S. (2008) · The Journal of Neuroscience · doi:10.1523/jneurosci.3541-08.2008 · Cited by 350
Self-citationKnowing how much you don't know: a neural organization of uncertainty estimates
Bach, D. R., & Dolan, R. J. (2012) · Nature Reviews Neuroscience · doi:10.1038/nrn3289 · Cited by 339
Behavioral Shifts and Action Valuation in the Anterior Cingulate Cortex
Quilodran, R., Rothé, M., & Procyk, E. (2008) · Neuron · doi:10.1016/j.neuron.2007.11.031 · Cited by 334
Bayesian approaches to autism: Towards volatility, action, and behavior.
Palmer, C. J., Lawson, R. P., & Hohwy, J. (2017) · Psychological Bulletin · doi:10.1037/bul0000097 · Cited by 307
Dunsmoor, J. E., Niv, Y., Daw, N., & Phelps, E. A. (2015) · Neuron · doi:10.1016/j.neuron.2015.09.028 · Cited by 329
Attention and associative learning in humans: An integrative review.
Le Pelley, M. E., Mitchell, C. J., Beesley, T., George, D. N., & Wills, A. J. (2016) · Psychological Bulletin · doi:10.1037/bul0000064 · Cited by 329
Learning latent structure: carving nature at its joints
Gershman, S. J., & Niv, Y. (2010) · Current Opinion in Neurobiology · doi:10.1016/j.conb.2010.02.008 · Cited by 326
Emotional Valence and the Free-Energy Principle
Joffily, M., & Coricelli, G. (2013) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1003094 · Cited by 322
The prefrontal cortex, pathological anxiety, and anxiety disorders
Kenwood, M. M., Kalin, N. H., & Barbas, H. (2022) · Neuropsychopharmacology · doi:10.1038/s41386-021-01109-z · Cited by 312
Dissociable components of rule-guided behavior depend on distinct medial and prefrontal regions
Buckley, M. J., Mansouri, F. A., Hoda, H., Mahboubi, M., Browning, P. G. F., Kwok, S. C., Phillips, A., & Tanaka, K. (2009) · Science · doi:10.1126/science.1172377 · Cited by 319
Mapping value based planning and extensively trained choice in the human brain
Wunderlich, K., Dayan, P., & Dolan, R. J. (2012) · Nature Neuroscience · doi:10.1038/nn.3068 · Cited by 310
Functionally dissociable influences on learning rate in a dynamic environment
McGuire, J. T., Nassar, M. R., Gold, J. I., & Kable, J. W. (2014) · Neuron · doi:10.1016/j.neuron.2014.10.013 · Cited by 299
Striatal Prediction Error Modulates Cortical Coupling
den Ouden, H. E. M., Daunizeau, J., Roiser, J., Friston, K. J., & Stephan, K. E. (2010) · The Journal of Neuroscience · doi:10.1523/jneurosci.4458-09.2010 · Cited by 304
Connectivity-based parcellation of the human orbitofrontal cortex
Kahnt, T., Chang, L. J., Park, S. Q., Heinzle, J., & Haynes, J.-D. (2012) · The Journal of Neuroscience · doi:10.1523/jneurosci.0257-12.2012 · Cited by 299
“Interestingly, this cluster was also connected to executive regions in the dorsomedial frontal cortex that have been linked to representing and updating action values (Behrens et al., 2007; Gläscher et al., 2009), suggesting a role for this subdivision in decision making.”
BackgroundGrounding cognitive control in associative learning
Abrahamse, E., Braem, S., Notebaert, W., & Verguts, T. (2016) · Psychological Bulletin · doi:10.1037/bul0000047 · Cited by 276
Frontal Theta Reflects Uncertainty and Unexpectedness during Exploration and Exploitation
Cavanagh, J. F., Figueroa, C. M., Cohen, M. X., & Frank, M. J. (2012) · Cerebral Cortex · doi:10.1093/cercor/bhr332 · Cited by 291
A reservoir of time constants for memory traces in cortical neurons
Bernacchia, A., Seo, H., Lee, D., & Wang, X.-J. (2011) · Nature Neuroscience · doi:10.1038/nn.2752 · Cited by 292
“Any parameters resulting in unrealistic values were discarded, such as negative values of τ, τ1 or τ2, values of (6)”
BackgroundBehavioural and neural characterization of optimistic reinforcement learning
Lefebvre, G., Lebreton, M., Meyniel, F., Bourgeois-Gironde, S., & Palminteri, S. (2017) · Nature Human Behaviour · doi:10.1038/s41562-017-0067 · Cited by 290
Neural Coding of Uncertainty and Probability
Ma, W. J., & Jazayeri, M. (2014) · Annual Review of Neuroscience · doi:10.1146/annurev-neuro-071013-014017 · Cited by 290
Reward Signals, Attempted Suicide, and Impulsivity in Late-Life Depression
Dombrovski, A. Y., Szanto, K., Clark, L., Reynolds, C. F., & Siegle, G. J. (2013) · JAMA Psychiatry · doi:10.1001/jamapsychiatry.2013.75 · Cited by 289
Metadata
- Authors
- Behrens, T. E. J., Woolrich, M. W., Walton, M. E., & Rushworth, M. F. S.
- Affiliations
- John Radcliffe Hospital (GB)
- University of Oxford (GB)
- Corresponding author
- Behrens, Timothy E J
- Venue
- Nature Neuroscience
- Published
- 2007-08-05
- Volume
- 10
- Issue
- 9
- Pages
- 1214-1221
- ISSN
- 1097-6256 (ISSN-L)
- 1546-1726
- OpenAlex
- W2123429050
- Cited by
- 2,053
- References
- 52
- Citations by year
- 2026: 122 · 2025: 163 · 2024: 156 · 2023: 135 · 2022: 139 · 2021: 128 · 2020: 133 · 2019: 134 · 2018: 108 · 2017: 108 · 2016: 121 · 2015: 101 · 2014: 87 · 2013: 91 · 2012: 88
- FWCI
- 27.97
- Type
- journal article
- Language
- en
- Open access
- closed
- Licence
- www.springer.com/tdm (tdm)
- Locations
- Nature Neuroscience (publishedVersion)
- PubMed (publishedVersion)
- Funders
- Wellcome Trust
- Topics
- Life Sciences › Neuroscience › Cognitive Neuroscience › Neural and Behavioral Psychology Studies
- Social Sciences › Decision Sciences › General Decision Sciences › Decision-Making and Behavioral Economics
- Life Sciences › Neuroscience › Cellular and Molecular Neuroscience › Neurotransmitter Receptor Influence on Behavior
- Keywords
- value of information, uncertainty, optimal learning, anterior cingulate cortex, learning rate, Bayesian decision making, human learning, fMRI decoding
- MeSH terms
- Adolescent, Adult, Bayes Theorem, Brain Mapping, Decision Making, Female, Gyrus Cinguli, Humans, Image Processing, Computer-Assisted, Learning, Magnetic Resonance Imaging, Male, Oxygen, Pattern Recognition, Visual, Photic Stimulation, Probability, Reaction Time, Reinforcement, Psychology
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.