Learning the value of information in an uncertain world
The DOI matches its Crossref record; 52 of the 52 references it cites were found in a registry or an index.
Abstract
Our decisions are guided by outcomes that are associated with decisions made in the past. However, the amount of influence each past outcome has on our next decision remains unclear. To ensure optimal decision-making, the weight given to decision outcomes should reflect their salience in predicting future outcomes, and this salience should be modulated by the volatility of the reward environment. We show that human subjects assess volatility in an optimal manner and adjust decision-making accordingly. This optimal estimate of volatility is reflected in the fMRI signal in the anterior cingulate cortex (ACC) when each trial outcome is observed. When a new piece of information is witnessed, activity levels reflect its salience for predicting future outcomes. Furthermore, variations in this ACC signal across the population predict variations in subject learning rates. Our results provide a formal account of how we weigh our different experiences in guiding our future actions.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“When the environment is volatile people should give more weight to recent sensory outcomes in building expectations about what they will see next (e.g. adopt a high α), in contrast they should give information from the distant past more weight when the environment is stable (e.g. adopt a low α) 8,9.”
From Adults with autism overestimate the volatility of the sensory environment (2017) · cites this paper as [8]
In the citing paper
“While computational studies of decision-making about rewards and punishments show that participants adapt their rate of learning about action-outcome contingencies in response to changes in environmental volatility 8–10”
From Adults with autism overestimate the volatility of the sensory environment (2017) · cites this paper as [8]
In the citing paper
“Previous work has shown that human learners indeed adjust their learning rate according to experimentally manipulated levels of volatility (Behrens et al., 2007).”
From The generalized Hierarchical Gaussian Filter (2026) · cites this paper as Behrens et al., 2007
In the citing paper
“This is known as hierarchical filtering and emphasizes the importance of taking into account the volatility of the environment, i.e., the temporal instability of its statistical structure, such as the probabilities by which one event causes another (Behrens et al., 2007; Mathys et al., 2011).”
From Allostatic Self-efficacy: A Metacognitive Theory of Dyshomeostasis-Induced Fatigue and Depression (2016) · cites this paper as Behrens et al., 2007
In the citing paper
“The causal structure of the world, with its nested spatial and temporal scales, implies that the brain’s internal model also possesses a hierarchical structure, which is a natural form for Bayesian inference: hierarchical models allow to encode information about the precision of beliefs at one level by values of hierarchically higher levels (81–83) (Figure 3D).”
From Can Bayesian Theories of Autism Spectrum Disorder Help Improve Clinical Practice? (2016) · cites this paper as [82]
1 more claim about this paper
Create a free account to see every claim other papers make about this one.
Citations
Cites
Reinforcement Learning: An Introduction
Sutton, R. S., & Barto, A. G. (1998) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.1998.712192 · Cited by 27,225
Reinforcement Learning: An Introduction
(2005) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.2004.842673 · Cited by 24,496
Engle, R. F. (1982) · Econometrica · doi:10.2307/1912773 · Cited by 20,795
Advances in functional and structural MR image analysis and implementation as FSL
Smith, S. M., Jenkinson, M., Woolrich, M. W., Beckmann, C. F., Behrens, T. E. J., Johansen-Berg, H., Bannister, P. R., De Luca, M., Drobnjak, I., Flitney, D. E., et al. (2004) · NeuroImage · doi:10.1016/j.neuroimage.2004.07.051 · Cited by 13,630
Self-citationKahneman, D., & Tversky, A. (1984) · American Psychologist · doi:10.1037/0003-066x.39.4.341 · Cited by 6,544
Humans integrate visual and haptic information in a statistically optimal fashion
Ernst, M. O., & Banks, M. S. (2002) · Nature · doi:10.1038/415429a · Cited by 4,844
Aston-Jones, G., & Cohen, J. D. (2005) · Annual Review of Neuroscience · doi:10.1146/annurev.neuro.28.061604.135709 · Cited by 4,472
Classical conditioning II: Current research and theory.
Black, A. H., & Prokasy, W. F. (1972) · Appleton-Century-Crofts eBooks · Cited by 4,449
Conflict monitoring and anterior cingulate cortex: an update
Botvinick, M. M., Cohen, J. D., & Carter, C. S. (2004) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2004.10.003 · Cited by 3,623
Pearce, J. M., & Hall, G. (1980) · Psychological Review · doi:10.1037/0033-295x.87.6.532 · Cited by 3,023
Cited by
Items 101–200 of 1,013
Emotion and decision-making: affect-driven belief systems in anxiety and depression
Paulus, M. P., & Yu, A. J. (2012) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2012.07.009 · Cited by 287
Adaptive learning under expected and unexpected uncertainty
Soltani, A., & Izquierdo, A. (2019) · Nature reviews. Neuroscience · doi:10.1038/s41583-019-0180-y · Cited by 286
Anterior cingulate engagement in a foraging context reflects choice difficulty, not foraging value
Shenhav, A., Straccia, M. A., Cohen, J. D., & Botvinick, M. M. (2014) · Nature Neuroscience · doi:10.1038/nn.3771 · Cited by 277
Resolving uncertainty in a social world
FeldmanHall, O., & Shenhav, A. (2019) · Nature Human Behaviour · doi:10.1038/s41562-019-0590-x · Cited by 274
Kragel, P. A., Kano, M., Van Oudenhove, L., Ly, H. G., Dupont, P., Rubio, A., Delon-Martin, C., Bonaz, B. L., Manuck, S. B., Gianaros, P. J., et al. (2018) · Nature Neuroscience · doi:10.1038/s41593-017-0051-7 · Cited by 272
Tan, H., Wade, C., & Brown, P. (2016) · The Journal of Neuroscience · doi:10.1523/jneurosci.3204-15.2016 · Cited by 267
Risk, unexpected uncertainty, and estimation uncertainty: Bayesian learning in unstable settings
Payzan-LeNestour, E., & Bossaerts, P. (2011) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1001048 · Cited by 267
Mars, R. B., Debener, S., Gladwin, T. E., Harrison, L. M., Haggard, P., Rothwell, J. C., & Bestmann, S. (2008) · The Journal of Neuroscience · doi:10.1523/jneurosci.2925-08.2008 · Cited by 264
The neural representation of unexpected uncertainty during value-based decision making
Payzan-LeNestour, E., Dunne, S., Bossaerts, P., & O’Doherty, J. P. (2013) · Neuron · doi:10.1016/j.neuron.2013.04.037 · Cited by 257
Egner, T. (2014) · Frontiers in Psychology · doi:10.3389/fpsyg.2014.01247 · Cited by 253
Multiple signals in anterior cingulate cortex
Kolling, N., Behrens, T., Wittmann, M., & Rushworth, M. (2016) · Current Opinion in Neurobiology · doi:10.1016/j.conb.2015.12.007 · Cited by 249
“In both cases, and in other reports [20,34,35], ACC is linked to behavioural change, invigoration of new responses, novel response strategies, and exploration.”
Self-citationA distributed, hierarchical and recurrent framework for reward-based choice
Hunt, L. T., & Hayden, B. Y. (2017) · Nature Reviews Neuroscience · doi:10.1038/nrn.2017.7 · Cited by 244
Medial and orbital frontal cortex in decision-making and flexible behavior
Klein-Flügge, M. C., Bongioanni, A., & Rushworth, M. F. S. (2022) · Neuron · doi:10.1016/j.neuron.2022.05.022 · Cited by 244
Self-citationAnxiety, Depression, and Decision Making: A Computational Perspective
Bishop, S. J., & Gagne, C. (2018) · Annual Review of Neuroscience · doi:10.1146/annurev-neuro-080317-062007 · Cited by 240
“…contingency volatility from contingency noise and adjust their behavior by moderating their learning rate from the human and basic neuroscience literature implicate the anterior cingulate cortex (ACC) and the amygdala in the use of contingency volatility to modulate rate of learning (Behrens et .”
BackgroundNeurocomputational models of basal ganglia function in learning, memory and choice
Cohen, M. X., & Frank, M. J. (2009) · Behavioural Brain Research · doi:10.1016/j.bbr.2008.09.029 · Cited by 237
Learning the opportunity cost of time in a patch-foraging task
Constantino, S. M., & Daw, N. D. (2015) · Cognitive, Affective, & Behavioral Neuroscience · doi:10.3758/s13415-015-0350-y · Cited by 236
Confirmation bias in human reinforcement learning: Evidence from counterfactual feedback processing
Palminteri, S., Lefebvre, G., Kilford, E. J., & Blakemore, S.-J. (2017) · PLOS Computational Biology · doi:10.1371/journal.pcbi.1005684 · Cited by 236
Gershman, S. J., & Uchida, N. (2019) · Nature Reviews Neuroscience · doi:10.1038/s41583-019-0220-7 · Cited by 236
Network Resets in Medial Prefrontal Cortex Mark the Onset of Behavioral Uncertainty
Karlsson, M. P., Tervo, D. G. R., & Karpova, A. Y. (2012) · Science · doi:10.1126/science.1226518 · Cited by 236
Prefrontal cortical control of a brainstem social behavior circuit
Franklin, T. B., Silva, B. A., Perova, Z., Marrone, L., Masferrer, M. E., Zhan, Y., Kaplan, A., Greetham, L., Verrechia, V., Halman, A., et al. (2017) · Nature Neuroscience · doi:10.1038/nn.4470 · Cited by 231
Chase, H. W., Kumar, P., Eickhoff, S. B., & Dombrovski, A. Y. (2015) · Cognitive, Affective, & Behavioral Neuroscience · doi:10.3758/s13415-015-0338-7 · Cited by 231
Haber, S. N., & Behrens, T. E. J. (2014) · Neuron · doi:10.1016/j.neuron.2014.08.031 · Cited by 230
Self-citationKrugel, L. K., Biele, G., Mohr, P. N. C., Li, S.-C., & Heekeren, H. R. (2009) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.0905191106 · Cited by 229
“In a recent study, participants’ choices in a two-option probabilistic reversal task were predicted using a dynamic learning rate that is modulated by the volatility of the environment as estimated by a Bayesian learner (31).”
ResultMice alternate between discrete strategies during perceptual decision-making
Ashwood, Z. C., Roy, N. A., Stone, I. R., Laboratory, T. I. B., Urai, A. E., Churchland, A. K., Pouget, A., & Pillow, J. W. (2022) · Nature Neuroscience · doi:10.1038/s41593-021-01007-z · Cited by 229
Learning to Simulate Others' Decisions
Suzuki, S., Harasawa, N., Ueno, K., Gardner, J. L., Ichinohe, N., Haruno, M., Cheng, K., & Nakahara, H. (2012) · Neuron · doi:10.1016/j.neuron.2012.04.030 · Cited by 226
The Affective Dog and Its Rational Tale: Intuition and Attunement
Railton, P. (2014) · Ethics · doi:10.1086/675876 · Cited by 228
Sequential effects: Superstition or rational behavior?
Yu, A. J., & Cohen, J. D. (2008) · PubMed · Cited by 282
Neural correlates of change in major depressive disorder anhedonia following open-label ketamine
Lally, N., Nugent, A. C., Luckenbaugh, D. A., Niciu, M. J., Roiser, J. P., & Zarate, C. A. (2015) · Journal of Psychopharmacology · doi:10.1177/0269881114568041 · Cited by 224
Interaction of language, auditory and memory brain networks in auditory verbal hallucinations
Ćurčić-Blake, B., Ford, J. M., Hubl, D., Orlov, N. D., Sommer, I. E., Waters, F., Allen, P., Jardri, R., Woodruff, P. W., David, O., et al. (2017) · Progress in Neurobiology · doi:10.1016/j.pneurobio.2016.11.002 · Cited by 224
Chase, H. W., Swainson, R., Durham, L., Benham, L., & Cools, R. (2011) · Journal of Cognitive Neuroscience · doi:10.1162/jocn.2010.21456 · Cited by 223
Triple dissociation of attention and decision computations across prefrontal cortex
Hunt, L. T., Malalasekera, W. M. N., de Berker, A. O., Miranda, B., Farmer, S. F., Behrens, T. E. J., & Kennerley, S. W. (2018) · Nature Neuroscience · doi:10.1038/s41593-018-0239-5 · Cited by 222
Self-citationSeparate Neural Mechanisms Underlie Choices and Strategic Preferences in Risky Decision Making
Venkatraman, V., Payne, J. W., Bettman, J. R., Luce, M. F., & Huettel, S. A. (2009) · Neuron · doi:10.1016/j.neuron.2009.04.007 · Cited by 222
Reminders of past choices bias decisions for reward in humans
Bornstein, A. M., Khaw, M. W., Shohamy, D., & Daw, N. D. (2017) · Nature Communications · doi:10.1038/ncomms15958 · Cited by 223
Reinforcement learning across development: What insights can we draw from a decade of research?
Nussenbaum, K., & Hartley, C. A. (2019) · Developmental Cognitive Neuroscience · doi:10.1016/j.dcn.2019.100733 · Cited by 217
Potential Vulnerabilities of Neuronal Reward, Risk, and Decision Mechanisms to Addictive Drugs
Schultz, W. (2011) · Neuron · doi:10.1016/j.neuron.2011.02.014 · Cited by 220
Inferring on the intentions of others by hierarchical Bayesian learning
Diaconescu, A. O., Mathys, C., Weber, L. A. E., Daunizeau, J., Kasper, L., Lomakina, E. I., Fehr, E., & Stephan, K. E. (2014) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1003810 · Cited by 219
Counterfactual Choice and Learning in a Neural Network Centered on Human Lateral Frontopolar Cortex
Boorman, E. D., Behrens, T. E., & Rushworth, M. F. (2011) · PLoS Biology · doi:10.1371/journal.pbio.1001093 · Cited by 218
Self-citationA Primer on Foraging and the Explore/Exploit Trade-Off for Psychiatry Research
Addicott, M. A., Pearson, J. M., Sweitzer, M. M., Barack, D. L., & Platt, M. L. (2017) · Neuropsychopharmacology · doi:10.1038/npp.2017.108 · Cited by 219
Who Knows? Metacognitive Social Learning Strategies
Heyes, C. (2016) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2015.12.007 · Cited by 218
Approaches to analysis in model-based cognitive neuroscience
Turner, B. M., Forstmann, B. U., Love, B. C., Palmeri, T. J., & Van Maanen, L. (2017) · Journal of Mathematical Psychology · doi:10.1016/j.jmp.2016.01.001 · Cited by 218
Vanneste, S., & De Ridder, D. (2012) · Frontiers in Systems Neuroscience · doi:10.3389/fnsys.2012.00031 · Cited by 216
Fouragnan, E. F., Chau, B. K. H., Folloni, D., Kolling, N., Verhagen, L., Klein-Flügge, M., Tankelevitch, L., Papageorgiou, G. K., Aubry, J.-F., Sallet, J., et al. (2019) · Nature Neuroscience · doi:10.1038/s41593-019-0375-6 · Cited by 213
Self-citationWhen Does Model-Based Control Pay Off?
Kool, W., Cushman, F. A., & Gershman, S. J. (2016) · PLOS Computational Biology · doi:10.1371/journal.pcbi.1005090 · Cited by 211
“However, it has been shown that people adapt RL parameters such as the learning rate and choice randomness to maximize reward in the environment [55], providing alleviation for this concern.”
BackgroundKennerley, S. W., & Wallis, J. D. (2009) · European Journal of Neuroscience · doi:10.1111/j.1460-9568.2009.06743.x · Cited by 210
Dopaminergic control of the exploration-exploitation trade-off via the basal ganglia
Humphries, M. (2012) · Frontiers in Neuroscience · doi:10.3389/fnins.2012.00009 · Cited by 206
Uncertainty of feedback and state estimation determines the speed of motor adaptation
Wei, K. (2010) · Frontiers in Computational Neuroscience · doi:10.3389/fncom.2010.00011 · Cited by 205
An information theory account of cognitive control
Fan, J. (2014) · Frontiers in Human Neuroscience · doi:10.3389/fnhum.2014.00680 · Cited by 200
Modeling Trait Anxiety: From Computational Processes to Personality
Raymond, J. G., Steele, J. D., & Seriès, P. (2017) · Frontiers in Psychiatry · doi:10.3389/fpsyt.2017.00001 · Cited by 203
The role of dopamine in the brain - lessons learned from Parkinson's disease
Meder, D., Herz, D. M., Rowe, J. B., Lehéricy, S., & Siebner, H. R. (2019) · NeuroImage · doi:10.1016/j.neuroimage.2018.11.021 · Cited by 201
Posterior Cingulate Cortex Mediates Outcome-Contingent Allocation of Behavior
Hayden, B. Y., Nair, A. C., McCoy, A. N., & Platt, M. L. (2008) · Neuron · doi:10.1016/j.neuron.2008.09.012 · Cited by 199
Klein, J. C., Rushworth, M. F. S., Behrens, T. E. J., Mackay, C. E., de Crespigny, A. J., D'Arceuil, H., & Johansen-Berg, H. (2010) · NeuroImage · doi:10.1016/j.neuroimage.2010.02.062 · Cited by 198
Self-citationComputational noise in reward-guided learning drives behavioral variability in volatile environments
Findling, C., Skvortsova, V., Dromnelle, R., Palminteri, S., & Wyart, V. (2019) · Nature Neuroscience · doi:10.1038/s41593-019-0518-9 · Cited by 196
Computational Models of Performance Monitoring and Cognitive Control
Alexander, W. H., & Brown, J. W. (2010) · Topics in Cognitive Science · doi:10.1111/j.1756-8765.2010.01085.x · Cited by 193
Beyond working memory: the role of persistent activity in decision making
Curtis, C. E., & Lee, D. (2010) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2010.03.006 · Cited by 191
Surprise! Neural correlates of Pearce–Hall and Rescorla–Wagner coexist within the brain
Roesch, M. R., Esber, G. R., Li, J., Daw, N. D., & Schoenbaum, G. (2012) · European Journal of Neuroscience · doi:10.1111/j.1460-9568.2011.07986.x · Cited by 191
Decision Making: The Neuroethological Turn
Pearson, J. M., Watson, K. K., & Platt, M. L. (2014) · Neuron · doi:10.1016/j.neuron.2014.04.037 · Cited by 190
Hierarchical reasoning by neural circuits in the frontal cortex
Sarafyazd, M., & Jazayeri, M. (2019) · Science · doi:10.1126/science.aav8911 · Cited by 191
Twenty-Five Lessons from Computational Neuromodulation
Dayan, P. (2012) · Neuron · doi:10.1016/j.neuron.2012.09.027 · Cited by 190
Tsuchida, A., Doll, B. B., & Fellows, L. K. (2010) · The Journal of Neuroscience · doi:10.1523/jneurosci.1958-10.2010 · Cited by 188
“…studies in humans and lesion work in monkeys, it has been suggested that this region is particularly important for choices in stochastic environments in which integration of outcome history is required for behavioral adjustment (Kennerley et al., 2006; Behrens et al., 2007; Jocham et al., 2009b).”
BackgroundHeterogeneous reward signals in prefrontal cortex
Wallis, J. D., & Kennerley, S. W. (2010) · Current Opinion in Neurobiology · doi:10.1016/j.conb.2010.02.009 · Cited by 186
“7 different actions [50], and encodes “fictive” rewards, counterfactual rewards that might have been obtained had the subject selected an alternate course of action [51].”
BackgroundStratified medicine for mental disorders
Schumann, G., Binder, E. B., Holte, A., de Kloet, E. R., Oedegaard, K. J., Robbins, T. W., Walker-Tilley, T. R., Bitter, I., Brown, V. J., Buitelaar, J., et al. (2014) · European Neuropsychopharmacology · doi:10.1016/j.euroneuro.2013.09.010 · Cited by 186
Bayesian Prediction and Evaluation in the Anterior Cingulate Cortex
Ide, J. S., Shenoy, P., Yu, A. J., & Li, C.-S. R. (2013) · The Journal of Neuroscience · doi:10.1523/jneurosci.2201-12.2013 · Cited by 185
“Along a somewhat different line, it has been suggested that dACC encodes a volatility signal that correlates with how rapidly statistical contingencies in the external world change over time (Behrens et al., 2007).”
BackgroundInfluentialRisk-dependent reward value signal in human prefrontal cortex
Tobler, P. N., Christopoulos, G. I., O'Doherty, J. P., Dolan, R. J., & Schultz, W. (2009) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.0809599106 · Cited by 183
When is an error not a prediction error? An electrophysiological investigation
Holroyd, C. B., Krigolson, O. E., Baker, R., Lee, S., & Gibson, J. (2009) · Cognitive, Affective, & Behavioral Neuroscience · doi:10.3758/cabn.9.1.59 · Cited by 183
Robust averaging during perceptual judgment
de Gardelle, V., & Summerfield, C. (2011) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1104517108 · Cited by 183
Solway, A., & Botvinick, M. M. (2012) · Psychological Review · doi:10.1037/a0026435 · Cited by 182
Unexpected arousal modulates the influence of sensory noise on confidence
Allen, M., Frank, D., Schwarzkopf, D. S., Fardo, F., Winston, J. S., Hauser, T. U., & Rees, G. (2016) · eLife · doi:10.7554/elife.18103 · Cited by 182
Computational Precision of Mental Inference as Critical Source of Human Choice Suboptimality
Drugowitsch, J., Wyart, V., Devauchelle, A.-D., & Koechlin, E. (2016) · Neuron · doi:10.1016/j.neuron.2016.11.005 · Cited by 181
Petzschner, F. H., Weber, L. A. E., Gard, T., & Stephan, K. E. (2017) · Biological Psychiatry · doi:10.1016/j.biopsych.2017.05.012 · Cited by 181
Frontal Oscillatory Dynamics Predict Feedback Learning and Action Adjustment
van de Vijver, I., Ridderinkhof, K. R., & Cohen, M. X. (2011) · Journal of Cognitive Neuroscience · doi:10.1162/jocn_a_00110 · Cited by 180
Advances in the computational understanding of mental illness
Huys, Q. J. M., Browning, M., Paulus, M. P., & Frank, M. J. (2021) · Neuropsychopharmacology · doi:10.1038/s41386-020-0746-4 · Cited by 179
Real and Fictive Outcomes Are Processed Differently but Converge on a Common Adaptive Mechanism
Fischer, A. G., & Ullsperger, M. (2013) · Neuron · doi:10.1016/j.neuron.2013.07.006 · Cited by 178
Perceptual Learning and Decision-Making in Human Medial Frontal Cortex
Kahnt, T., Grueschow, M., Speck, O., & Haynes, J.-D. (2011) · Neuron · doi:10.1016/j.neuron.2011.02.054 · Cited by 179
Uncertainty and Exploration in a Restless Bandit Problem
Speekenbrink, M., & Konstantinidis, E. (2015) · Topics in Cognitive Science · doi:10.1111/tops.12145 · Cited by 177
Normative evidence accumulation in unpredictable environments
Glaze, C. M., Kable, J. W., & Gold, J. I. (2015) · eLife · doi:10.7554/elife.08825 · Cited by 175
Skvortsova, V., Palminteri, S., & Pessiglione, M. (2014) · The Journal of Neuroscience · doi:10.1523/jneurosci.1350-14.2014 · Cited by 177
“In particular, we did not test more complex architectures that would track higher-order dynamics (Hampton et al., 2006; Behrens et al., 2007; Mathys et al., 2011; Doll et al., 2012).”
BackgroundReciprocal relations between cognitive neuroscience and formal cognitive models: opposites attract?
Forstmann, B. U., Wagenmakers, E.-J., Eichele, T., Brown, S., & Serences, J. T. (2011) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2011.04.002 · Cited by 177
Kennerley, S. W., & Walton, M. E. (2011) · Behavioral Neuroscience · doi:10.1037/a0023575 · Cited by 176
Self-citationFouragnan, E., Retzler, C., & Philiastides, M. G. (2018) · Human Brain Mapping · doi:10.1002/hbm.24047 · Cited by 176
A perceptual inference mechanism for hallucinations linked to striatal dopamine
Cassidy, C. M., Balsam, P. D., Weinstein, J. J., Rosengard, R. J., Slifstein, M., Daw, N. D., Abi-Dargham, A., & Horga, G. (2018) · Current Biology · doi:10.1016/j.cub.2017.12.059 · Cited by 176
Social Network Modulation of Reward-Related Signals
Fareri, D. S., Niznikiewicz, M. A., Lee, V. K., & Delgado, M. R. (2012) · Journal of Neuroscience · doi:10.1523/jneurosci.0610-12.2012 · Cited by 175
“Ventral mPFC regions differentiate monetary outcomes of varying magnitude (Knutson et al., 2003), and portions of the anterior cingulate track action outcome history, facilitating learning of optimal behavior in both nonsocial (Behrens et al., 2007) and social (Behrens et al., 2008) situations.”
BackgroundPike, A. C., & Robinson, O. J. (2022) · JAMA Psychiatry · doi:10.1001/jamapsychiatry.2022.0051 · Cited by 167
Zhang, L., Lengersdorff, L., Mikus, N., Gläscher, J., & Lamm, C. (2020) · Social Cognitive and Affective Neuroscience · doi:10.1093/scan/nsaa089 · Cited by 173
Meta-learning in natural and artificial intelligence
Wang, J. X. (2021) · Current Opinion in Behavioral Sciences · doi:10.1016/j.cobeha.2021.01.002 · Cited by 171
Differential Representations of Prior and Likelihood Uncertainty in the Human Brain
Vilares, I., Howard, J. D., Fernandes, H. L., Gottfried, J. A., & Kording, K. P. (2012) · Current Biology · doi:10.1016/j.cub.2012.07.010 · Cited by 168
Children’s and Apes’ Preparatory Responses to Two Mutually Exclusive Possibilities
Redshaw, J., & Suddendorf, T. (2016) · Current Biology · doi:10.1016/j.cub.2016.04.062 · Cited by 168
Summerfield, C., & Tsetsos, K. (2012) · Frontiers in Neuroscience · doi:10.3389/fnins.2012.00070 · Cited by 167
Observing the observer (I): meta-bayesian models of learning and decision-making
Daunizeau, J., den Ouden, H. E. M., Pessiglione, M., Kiebel, S. J., Stephan, K. E., & Friston, K. J. (2010) · PLoS ONE · doi:10.1371/journal.pone.0015554 · Cited by 167
“This is manifest at many temporal and processing scales; e.g. , low-level visual processing ([10–16]), multimodal sensory integration ([17–21]), sensorimotor learning ([22–27]), conditioning in a volatile environment ([28–29]), attention ([30–31]), and even reasoning ([32–33]).”
Decision Making: From Neuroscience to Psychiatry
Lee, D. (2013) · Neuron · doi:10.1016/j.neuron.2013.04.008 · Cited by 166
A Unifying Probabilistic View of Associative Learning
Gershman, S. J. (2015) · PLOS Computational Biology · doi:10.1371/journal.pcbi.1004567 · Cited by 166
Serotonin neurons modulate learning rate through uncertainty
Grossman, C. D., Bari, B. A., & Cohen, J. Y. (2022) · Current Biology · doi:10.1016/j.cub.2021.12.006 · Cited by 164
Boorman, E. D., Rushworth, M. F., & Behrens, T. E. (2013) · The Journal of Neuroscience · doi:10.1523/jneurosci.3022-12.2013 · Cited by 164
Self-citationAnterior cingulate is a source of valence-specific information about value and uncertainty
Monosov, I. E. (2017) · Nature Communications · doi:10.1038/s41467-017-00072-y · Cited by 163
On the functional anatomy of the urge-for-action
Jackson, S. R., Parkinson, A., Kim, S. Y., Schüermann, M., & Eickhoff, S. B. (2011) · Cognitive Neuroscience · doi:10.1080/17588928.2011.604717 · Cited by 158
The Sense of Confidence during Probabilistic Learning: A Normative Account
Meyniel, F., Schlunegger, D., & Dehaene, S. (2015) · PLOS Computational Biology · doi:10.1371/journal.pcbi.1004305 · Cited by 163
Klein-Flügge, M. C., Kennerley, S. W., Saraiva, A. C., Penny, W. D., & Bestmann, S. (2015) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1004116 · Cited by 163
“, hyperbolic versus exponential model for delay discounting [59]; an improved volatility learning model over a standard reinforcement learning model; see [60]).”
BackgroundPezzulo, G., Rigoli, F., & Chersi, F. (2013) · Frontiers in Psychology · doi:10.3389/fpsyg.2013.00092 · Cited by 163
“However, it is plausible that living organisms explicitly model volatility (Behrens et al., 2007; Kepecs et al., 2008).”
Interoception of breathing and its relationship with anxiety
Harrison, O. K., Köchli, L., Marino, S., Luechinger, R., Hennel, F., Brand, K., Hess, A. J., Frässle, S., Iglesias, S., Vinckier, F., et al. (2021) · Neuron · doi:10.1016/j.neuron.2021.09.045 · Cited by 163
Decoding different roles for vmPFC and dlPFC in multi-attribute decision making
Kahnt, T., Heinzle, J., Park, S. Q., & Haynes, J.-D. (2011) · NeuroImage · doi:10.1016/j.neuroimage.2010.05.058 · Cited by 161
Corcoran, A. W., Pezzulo, G., & Hohwy, J. (2020) · Biology & Philosophy · doi:10.1007/s10539-020-09746-2 · Cited by 161
Metadata
- Authors
- Behrens, T. E. J., Woolrich, M. W., Walton, M. E., & Rushworth, M. F. S.
- Affiliations
- John Radcliffe Hospital (GB)
- University of Oxford (GB)
- Corresponding author
- Behrens, Timothy E J
- Venue
- Nature Neuroscience
- Published
- 2007-08-05
- Volume
- 10
- Issue
- 9
- Pages
- 1214-1221
- ISSN
- 1097-6256 (ISSN-L)
- 1546-1726
- OpenAlex
- W2123429050
- Cited by
- 2,053
- References
- 52
- Citations by year
- 2026: 122 · 2025: 163 · 2024: 156 · 2023: 135 · 2022: 139 · 2021: 128 · 2020: 133 · 2019: 134 · 2018: 108 · 2017: 108 · 2016: 121 · 2015: 101 · 2014: 87 · 2013: 91 · 2012: 88
- FWCI
- 27.97
- Type
- journal article
- Language
- en
- Open access
- closed
- Licence
- www.springer.com/tdm (tdm)
- Locations
- Nature Neuroscience (publishedVersion)
- PubMed (publishedVersion)
- Funders
- Wellcome Trust
- Topics
- Life Sciences › Neuroscience › Cognitive Neuroscience › Neural and Behavioral Psychology Studies
- Social Sciences › Decision Sciences › General Decision Sciences › Decision-Making and Behavioral Economics
- Life Sciences › Neuroscience › Cellular and Molecular Neuroscience › Neurotransmitter Receptor Influence on Behavior
- Keywords
- value of information, uncertainty, optimal learning, anterior cingulate cortex, learning rate, Bayesian decision making, human learning, fMRI decoding
- MeSH terms
- Adolescent, Adult, Bayes Theorem, Brain Mapping, Decision Making, Female, Gyrus Cinguli, Humans, Image Processing, Computer-Assisted, Learning, Magnetic Resonance Imaging, Male, Oxygen, Pattern Recognition, Visual, Photic Stimulation, Probability, Reaction Time, Reinforcement, Psychology
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.