Learning the value of information in an uncertain world
The DOI matches its Crossref record; 52 of the 52 references it cites were found in a registry or an index.
Abstract
Our decisions are guided by outcomes that are associated with decisions made in the past. However, the amount of influence each past outcome has on our next decision remains unclear. To ensure optimal decision-making, the weight given to decision outcomes should reflect their salience in predicting future outcomes, and this salience should be modulated by the volatility of the reward environment. We show that human subjects assess volatility in an optimal manner and adjust decision-making accordingly. This optimal estimate of volatility is reflected in the fMRI signal in the anterior cingulate cortex (ACC) when each trial outcome is observed. When a new piece of information is witnessed, activity levels reflect its salience for predicting future outcomes. Furthermore, variations in this ACC signal across the population predict variations in subject learning rates. Our results provide a formal account of how we weigh our different experiences in guiding our future actions.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“When the environment is volatile people should give more weight to recent sensory outcomes in building expectations about what they will see next (e.g. adopt a high α), in contrast they should give information from the distant past more weight when the environment is stable (e.g. adopt a low α) 8,9.”
From Adults with autism overestimate the volatility of the sensory environment (2017) · cites this paper as [8]
In the citing paper
“While computational studies of decision-making about rewards and punishments show that participants adapt their rate of learning about action-outcome contingencies in response to changes in environmental volatility 8–10”
From Adults with autism overestimate the volatility of the sensory environment (2017) · cites this paper as [8]
In the citing paper
“Previous work has shown that human learners indeed adjust their learning rate according to experimentally manipulated levels of volatility (Behrens et al., 2007).”
From The generalized Hierarchical Gaussian Filter (2026) · cites this paper as Behrens et al., 2007
In the citing paper
“This is known as hierarchical filtering and emphasizes the importance of taking into account the volatility of the environment, i.e., the temporal instability of its statistical structure, such as the probabilities by which one event causes another (Behrens et al., 2007; Mathys et al., 2011).”
From Allostatic Self-efficacy: A Metacognitive Theory of Dyshomeostasis-Induced Fatigue and Depression (2016) · cites this paper as Behrens et al., 2007
In the citing paper
“The causal structure of the world, with its nested spatial and temporal scales, implies that the brain’s internal model also possesses a hierarchical structure, which is a natural form for Bayesian inference: hierarchical models allow to encode information about the precision of beliefs at one level by values of hierarchically higher levels (81–83) (Figure 3D).”
From Can Bayesian Theories of Autism Spectrum Disorder Help Improve Clinical Practice? (2016) · cites this paper as [82]
1 more claim about this paper
Create a free account to see every claim other papers make about this one.
Citations
Cites
Reinforcement Learning: An Introduction
Sutton, R. S., & Barto, A. G. (1998) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.1998.712192 · Cited by 27,225
Reinforcement Learning: An Introduction
(2005) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.2004.842673 · Cited by 24,496
Engle, R. F. (1982) · Econometrica · doi:10.2307/1912773 · Cited by 20,795
Advances in functional and structural MR image analysis and implementation as FSL
Smith, S. M., Jenkinson, M., Woolrich, M. W., Beckmann, C. F., Behrens, T. E. J., Johansen-Berg, H., Bannister, P. R., De Luca, M., Drobnjak, I., Flitney, D. E., et al. (2004) · NeuroImage · doi:10.1016/j.neuroimage.2004.07.051 · Cited by 13,630
Self-citationKahneman, D., & Tversky, A. (1984) · American Psychologist · doi:10.1037/0003-066x.39.4.341 · Cited by 6,544
Humans integrate visual and haptic information in a statistically optimal fashion
Ernst, M. O., & Banks, M. S. (2002) · Nature · doi:10.1038/415429a · Cited by 4,844
Aston-Jones, G., & Cohen, J. D. (2005) · Annual Review of Neuroscience · doi:10.1146/annurev.neuro.28.061604.135709 · Cited by 4,472
Classical conditioning II: Current research and theory.
Black, A. H., & Prokasy, W. F. (1972) · Appleton-Century-Crofts eBooks · Cited by 4,449
Conflict monitoring and anterior cingulate cortex: an update
Botvinick, M. M., Cohen, J. D., & Carter, C. S. (2004) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2004.10.003 · Cited by 3,623
Pearce, J. M., & Hall, G. (1980) · Psychological Review · doi:10.1037/0033-295x.87.6.532 · Cited by 3,023
Cited by
Items 701–800 of 1,013
Li, Y., Nassar, M. R., Kable, J. W., & Gold, J. I. (2019) · Journal of Neuroscience · doi:10.1523/jneurosci.0159-19.2019 · Cited by 40
Hein, T. P., & Herrojo Ruiz, M. (2022) · NeuroImage · doi:10.1016/j.neuroimage.2022.118895 · Cited by 37
The computational relationship between reinforcement learning, social inference, and paranoia
Barnby, J. M., Mehta, M. A., & Moutoussis, M. (2022) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1010326 · Cited by 38
Multistage Valuation Signals and Common Neural Currencies
Platt, M. L., & Plaßmann, H. (2013) · Elsevier eBooks · doi:10.1016/b978-0-12-416008-8.00013-9 · Cited by 41
Large-scale citizen science reveals predictors of sensorimotor adaptation
Tsay, J. S., Asmerian, H., Germine, L. T., Wilmer, J. B., Ivry, R. B., & Nakayama, K. (2024) · Nature Human Behaviour · doi:10.1038/s41562-023-01798-0 · Cited by 46
Variational Bayesian mixed-effects inference for classification studies
Brodersen, K. H., Daunizeau, J., Mathys, C. D., Chumbley, J. R., Buhmann, J. M., & Stephan, K. E. (2013) · NeuroImage · doi:10.1016/j.neuroimage.2013.03.008 · Cited by 44
Working Memory and Anticipatory Set Modulate Midbrain and Putamen Activity
Yu, Y., FitzGerald, T. H. B., & Friston, K. J. (2013) · Journal of Neuroscience · doi:10.1523/jneurosci.1176-13.2013 · Cited by 36
Breakdown of long-range temporal correlations in brain oscillations during general anesthesia
Krzemiński, D., Kamiński, M., Marchewka, A., & Bola, M. (2017) · NeuroImage · doi:10.1016/j.neuroimage.2017.07.047 · Cited by 35
Harlé, K. M., Guo, D., Zhang, S., Paulus, M. P., & Yu, A. J. (2017) · PLOS ONE · doi:10.1371/journal.pone.0186473 · Cited by 36
Maurer, C., Chambon, V., Bourgeois‐Gironde, S., Leboyer, M., & Zalla, T. (2017) · Cognition · doi:10.1016/j.cognition.2017.11.007 · Cited by 41
Active Inference, Belief Propagation, and the Bethe Approximation
Schwöbel, S., Kiebel, S., & Marković, D. (2018) · Neural Computation · doi:10.1162/neco_a_01108 · Cited by 36
The human as delta-rule learner.
Lee, S., Gold, J. I., & Kable, J. W. (2020) · Decision · doi:10.1037/dec0000112 · Cited by 36
Li, G., Zhang, S., Le, T. M., Tang, X., & Li, C. R. (2020) · Cerebral Cortex Communications · doi:10.1093/texcom/tgaa025 · Cited by 41
Effects of positive mood on probabilistic learning: Behavioral and electrophysiological correlates
Bakic, J., Jepma, M., Raedt, R. D., & Pourtois, G. (2014) · Biological Psychology · doi:10.1016/j.biopsycho.2014.09.012 · Cited by 43
Computation noise promotes cognitive resilience to adverse conditions during decision-makingPreprint
Findling, C., & Wyart, V. (2020) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2020.06.10.145300 · Cited by 36
Human inference reflects a normative balance of complexity and accuracy
Tavoni, G., Doi, T., Pizzica, C., Balasubramanian, V., & Gold, J. I. (2022) · Nature Human Behaviour · doi:10.1038/s41562-022-01357-z · Cited by 35
A translational perspective on the anti-anhedonic effect of ketamine and its neural underpinnings
Pulcu, E., Guinea, C., Cowen, P. J., Murphy, S. E., & Harmer, C. J. (2022) · Molecular Psychiatry · doi:10.1038/s41380-021-01183-1 · Cited by 35
Rubaltelli, E., Agnoli, S., & Franchin, L. (2015) · Journal of Behavioral Decision Making · doi:10.1002/bdm.1885 · Cited by 42
Impaired error-likelihood prediction in medial prefrontal cortex in schizophrenia
Krawitz, A., Braver, T. S., Barch, D. M., & Brown, J. W. (2011) · NeuroImage · doi:10.1016/j.neuroimage.2010.09.027 · Cited by 35
A Spiking Neural Integrator Model of the Adaptive Control of Action by the Medial Prefrontal Cortex
Bekolay, T., Laubach, M., & Eliasmith, C. (2014) · Journal of Neuroscience · doi:10.1523/jneurosci.2421-13.2014 · Cited by 38
Decision-Making Based on Emotional Images
Katahira, K., Fujimura, T., Okanoya, K., & Okada, M. (2011) · Frontiers in Psychology · doi:10.3389/fpsyg.2011.00311 · Cited by 36
Boschin, E. A., Brkic, M. M., Simons, J. S., & Buckley, M. J. (2016) · Cerebral Cortex · doi:10.1093/cercor/bhw350 · Cited by 44
Adaptive Learning through Temporal Dynamics of State Representation
Razmi, N., & Nassar, M. R. (2022) · Journal of Neuroscience · doi:10.1523/jneurosci.0387-21.2022 · Cited by 38
Dynamic Estimation of Task-Relevant Variance in Movement under Risk
Landy, M. S., Trommershäuser, J., & Daw, N. D. (2012) · Journal of Neuroscience · doi:10.1523/jneurosci.6160-11.2012 · Cited by 38
The Self and Its Reputation in Autism
Frith, C. D., & Frith, U. (2008) · Neuron · doi:10.1016/j.neuron.2008.01.014 · Cited by 35
Metacognitive Feelings: A Predictive-Processing Perspective
Velasco, P. F., & Loev, S. (2024) · Perspectives on Psychological Science · doi:10.1177/17456916231221976 · Cited by 73
Thalamocortical architectures for flexible cognition and efficient learning
Scott, D. N., Mukherjee, A., Nassar, M. R., & Halassa, M. M. (2024) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2024.05.006 · Cited by 62
The Role of Dopamine in Motor Flexibility
Bestmann, S., Ruge, D., Rothwell, J. C., & Galea, J. M. (2014) · Journal of Cognitive Neuroscience · doi:10.1162/jocn_a_00706 · Cited by 35
Risk seeking behavior of preschool children in a gambling task
Moreira, B. C. D. M., Matsushita, R. Y., & Silva, S. D. (2010) · Journal of Economic Psychology · doi:10.1016/j.joep.2010.03.005 · Cited by 41
The Yin and the Yang of Prediction: An fMRI Study of Semantic Predictive Processing
Weber, K., Lau, E. F., Stillerman, B. S., & Kuperberg, G. R. (2016) · PLoS ONE · doi:10.1371/journal.pone.0148637 · Cited by 41
Stress enhances model-free reinforcement learning only after negative outcome
Park, H., Lee, D., & Chey, J. (2017) · PLoS ONE · doi:10.1371/journal.pone.0180588 · Cited by 42
A unifying Bayesian account of contextual effects in value-based choice
Rigoli, F., Mathys, C. D., Friston, K. J., & Dolan, R. J. (2017) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1005769 · Cited by 37
Vassena, E., Deraeve, J., & Alexander, W. H. (2019) · Cognitive Affective & Behavioral Neuroscience · doi:10.3758/s13415-018-00685-w · Cited by 37
Uncertainty in learning, choice, and visual fixation
Stojić, H., Orquin, J. L., Dayan, P., Dolan, R. J., & Speekenbrink, M. (2020) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1911348117 · Cited by 40
Lambert, K. G., Hyer, M. M., Rzucidlo, A., Bergeron, T., Landis, T., & Bardi, M. (2014) · Frontiers in Behavioral Neuroscience · doi:10.3389/fnbeh.2014.00124 · Cited by 36
A Computational Analysis of Aberrant Delay Discounting in Psychiatric Disorders
Story, G. W., Moutoussis, M., & Dolan, R. J. (2016) · Frontiers in Psychology · doi:10.3389/fpsyg.2015.01948 · Cited by 37
Risk Aversion, Gender, and Constitutional Change
Verge, T., Guinjoan, M., & Rodón, T. (2015) · Politics & Gender · doi:10.1017/s1743923x15000264 · Cited by 47
Spatial Attention, Motor Intention, and Bayesian Cue Predictability in the Human Brain
Kuhns, A. B., Dombert, P. L., Mengotti, P., Fink, G. R., & Vossel, S. (2017) · Journal of Neuroscience · doi:10.1523/jneurosci.3255-16.2017 · Cited by 36
Developmental asymmetries in learning to adjust to cooperative and uncooperative environments
Westhoff, B., Molleman, L., Viding, E. M., van den Bos, W., & van Duijvenvoorde, A. C. K. (2020) · Scientific Reports · doi:10.1038/s41598-020-78546-1 · Cited by 36
Kóbor, A., Horváth, K., Kardos, Z., Németh, D., & Janacsek, K. (2020) · Cognition · doi:10.1016/j.cognition.2020.104413 · Cited by 38
Learning to Synchronize: Midfrontal Theta Dynamics during Rule Switching
Verbeke, P., Ergo, K., Loof, E. D., & Verguts, T. (2020) · Journal of Neuroscience · doi:10.1523/jneurosci.1874-20.2020 · Cited by 40
Pupil diameter tracked during motor adaptation in humans
Yokoi, A., & Weiler, J. (2022) · Journal of Neurophysiology · doi:10.1152/jn.00021.2022 · Cited by 37
Social Information and Economic Decision-Making in the Ultimatum Game
Gaertig, C., Moser, A., Alguacil, S., & Ruz, M. (2012) · Frontiers in Neuroscience · doi:10.3389/fnins.2012.00103 · Cited by 35
Learning About Unstable, Publicly Unobservable Payoffs
Payzan-LeNestour, É., & Bossaerts, P. L. (2014) · Review of Financial Studies · doi:10.1093/rfs/hhu069 · Cited by 38
Social anxiety and dynamic social reinforcement learning in a volatile environment
Beltzer, M. L., Adams, S., Beling, P. A., & Teachman, B. A. (2019) · Clinical Psychological Science · doi:10.1177/2167702619858425 · Cited by 32
Intact predictive processing in autistic adults: evidence from statistical learning
Pesthy, O., Farkas, K., Sapey‐Triomphe, L., Guttengéber, A., Komoróczy, E., Janacsek, K., Réthelyi, J., & Németh, D. (2023) · Scientific Reports · doi:10.1038/s41598-023-38708-3 · Cited by 34
Dopamine release in human associative striatum during reversal learning
Grill, F., Guitart‐Masip, M., Johansson, J. J., Jonasson, L. S., Axelsson, J., Nyberg, L. H., & Rieckmann, A. (2024) · Nature Communications · doi:10.1038/s41467-023-44358-w · Cited by 38
Hierarchical gradients of multiple timescales in the mammalian forebrain
Song, M. J., Shin, E. J., Seo, H., Soltani, A., Steinmetz, N. A., Lee, D., Jung, M. W., & Paik, S. (2024) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2415695121 · Cited by 45
The Bivalency effect in task switching: Event‐related potentials
Grundy, J. G., Benarroch, M. F. F., Woodward, T. S., Metzak, P. D., Whitman, J. C., & Shedden, J. M. (2011) · Human Brain Mapping · doi:10.1002/hbm.21488 · Cited by 36
Wessel, J. R., & Huber, D. E. (2019) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1006927 · Cited by 33
Neural arbitration between social and individual learning systems
Diaconescu, A. O., Stecy, M., Kasper, L., Burke, C. J., Nagy, Z., Mathys, C. D., & Tobler, P. N. (2020) · eLife · doi:10.7554/elife.54051 · Cited by 34
Disrupted state transition learning as a computational marker of compulsivity
Sharp, P. B., Dolan, R. J., & Eldar, E. (2021) · Psychological Medicine · doi:10.1017/s0033291721003846 · Cited by 37
Effects of methylphenidate on reinforcement learning depend on working memory capacity
Kandroodi, M. R., Cook, J. L., Swart, J. C., Froböse, M. I., Geurts, D. E. M., Vahabie, A., Ahmadabadi, M. N., Cools, R., & den Ouden, H. E. M. (2021) · Psychopharmacology · doi:10.1007/s00213-021-05974-w · Cited by 38
An evolutionary model of sensitive periods when the reliability of cues varies across ontogeny
Walasek, N., Frankenhuis, W. E., & Panchanathan, K. (2021) · Behavioral Ecology · doi:10.1093/beheco/arab113 · Cited by 33
Hein, T. P., Gong, Z., Ivanova, M., Fedele, T., Nikulin, V. V., & Ruiz, M. H. (2023) · Communications Biology · doi:10.1038/s42003-023-04628-1 · Cited by 37
Thorndike's Law 2.0: Dopamine and the Regulation of Thrift
Beeler, J. A. (2012) · Frontiers in Neuroscience · doi:10.3389/fnins.2012.00116 · Cited by 43
Randomised prior feedback modulates neural signals of outcome monitoring
Mushtaq, F., Wilkie, R. M., Mon-Williams, M. A., & Schaefer, A. (2015) · NeuroImage · doi:10.1016/j.neuroimage.2015.10.046 · Cited by 37
Karuza, E. A., Li, P., Weiss, D. J., Bulgarelli, F., Zinszer, B. D., & Aslin, R. Ν. (2016) · Journal of Cognitive Neuroscience · doi:10.1162/jocn_a_00990 · Cited by 37
The Neurobiological Foundations of Valuation in Human Decision Making Under Uncertainty
Bossaerts, P. L., Preuschoff, K., & Hsu, M. (2009) · Elsevier eBooks · doi:10.1016/b978-0-12-374176-9.00023-3 · Cited by 35
Preparing humanity for change and artificial intelligencePreprint
Laukkonen, R., Biddell, H., & Gallagher, R. (2019) · PsyArXiv (OSF Preprints) · doi:10.31234/osf.io/g5qwc · Cited by 40
Learning environment-specific learning rates
Simoens, J., Verguts, T., & Braem, S. (2024) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1011978 · Cited by 40
Tanaka, S., O’Doherty, J. P., & Sakagami, M. (2019) · Nature Communications · doi:10.1038/s41467-019-11334-2 · Cited by 32
Rational inattention and tonic dopamine
Mikhael, J. G., Lai, L., & Gershman, S. J. (2021) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1008659 · Cited by 39
Wilk, H. A., & Morton, J. B. (2012) · NeuroImage · doi:10.1016/j.neuroimage.2012.06.069 · Cited by 36
A Normative Theory of Forgetting: Lessons from the Fruit Fly
Brea, J., Urbanczik, R., & Senn, W. (2014) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1003640 · Cited by 35
Brazil, I. A., Mathys, C. D., Popma, A., Hoppenbrouwers, S. S., & Cohn, M. D. (2017) · Biological Psychiatry Cognitive Neuroscience and Neuroimaging · doi:10.1016/j.bpsc.2017.04.005 · Cited by 33
Reward prediction errors create event boundaries in memoryPreprint
Rouhani, N., Norman, K. A., Niv, Y., & Bornstein, A. M. (2019) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/725440 · Cited by 32
Expecting the unexpected: a review of learning under uncertainty across development
Topel, S., Ma, I., Sleutels, J. J. M., van Steenbergen, H., de Bruijn, E. R. A., & van Duijvenvoorde, A. C. K. (2023) · Cognitive Affective & Behavioral Neuroscience · doi:10.3758/s13415-023-01098-0 · Cited by 37
The role of prediction and outcomes in adaptive cognitive control
Schiffer, A., Waszak, F., & Yeung, N. (2015) · Journal of Physiology-Paris · doi:10.1016/j.jphysparis.2015.02.001 · Cited by 37
Neta, M., Nelson, S. M., & Petersen, S. E. (2016) · Cerebral Cortex · doi:10.1093/cercor/bhw053 · Cited by 35
Evaluation of ambiguous associations in the amygdala by learning the structure of the environment
Madarász, T. J., Díaz-Mataix, L., Akhand, O., Ycu, E. A., LeDoux, J. E., & Johansen, J. P. (2016) · Nature Neuroscience · doi:10.1038/nn.4308 · Cited by 32
Fraud detection in banking using deep reinforcement learning
Bouchti, A. E., Chakroun, A., Abbar, H., & Okar, C. (2017) · International Conference on Innovative Computing Technology · doi:10.1109/intech.2017.8102446 · Cited by 33
Sallet, J., Noonan, M. P., Thomas, A. G., O’Reilly, J. X., Anderson, J., Papageorgiou, G., Neubert, F. X., Ahmed, B., Smith, J. E. T., Bell, A. H., et al. (2020) · PLoS Biology · doi:10.1371/journal.pbio.3000605 · Cited by 30
Self-citationSlooten, J. C. V., Jahfari, S., & Theeuwes, J. (2019) · Scientific Reports · doi:10.1038/s41598-019-53805-y · Cited by 39
Human Decision-Making beyond the Rational Decision Theory
Koechlin, É. (2019) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2019.11.001 · Cited by 30
Adaptive Empathy: Empathic Response Selection as a Dynamic, Feedback-Based Learning Process
Arbel, E. K., Shamay‐Tsoory, S., & Hertz, U. (2021) · Frontiers in Psychiatry · doi:10.3389/fpsyt.2021.706474 · Cited by 41
Jiang, S., Wang, S., & Wan, X. (2022) · PLOS Biology · doi:10.1371/journal.pbio.3001301 · Cited by 30
Transfer of Learned Cognitive Flexibility to Novel Stimuli and Task Sets
Wen, T., Geddert, R., Madlon‐Kay, S., & Egner, T. (2023) · Psychological Science · doi:10.1177/09567976221141854 · Cited by 43
Loosen, A. M., Seow, T. X. F., & Hauser, T. U. (2024) · Behavior Research Methods · doi:10.3758/s13428-024-02427-y · Cited by 35
Desmet, C., Deschrijver, E., & Braß, M. (2013) · Social Cognitive and Affective Neuroscience · doi:10.1093/scan/nst002 · Cited by 33
The object space task shows cumulative memory expression in both mice and rats
Genzel, L., Schut, E. H. S., Schröder, T., Eichler, R., Khamassi, M., Gomez, A., Navarro‐Lobato, I., & Battaglia, F. P. (2019) · PLoS Biology · doi:10.1371/journal.pbio.3000322 · Cited by 35
Model-based reinforcement learning under concurrent schedules of reinforcement in rodents
Huh, N., Jo, S., Kim, H., Sul, J. H., & Jung, M. W. (2009) · Learning & Memory · doi:10.1101/lm.1295509 · Cited by 37
Diarra, M., Marchitto, M., Bressolle, M.-C., Thierry Baccino, B., & Drai-Zerbib, V. (2023) · Frontiers in Neuroergonomics · doi:10.3389/fnrgo.2023.1059476 · Cited by 33
Ecological expected utility and the mythical neural code
Feldman, J. A. (2009) · Cognitive Neurodynamics · doi:10.1007/s11571-009-9090-4 · Cited by 29
Bai, Y., Katahira, K., & Ohira, H. (2014) · Frontiers in Psychology · doi:10.3389/fpsyg.2014.00871 · Cited by 32
Kheifets, A., Freestone, D. M., & Gallistel, C. R. (2017) · Journal of the Experimental Analysis of Behavior · doi:10.1002/jeab.261 · Cited by 29
Easdale, L. C., Pelley, M. E. L., & Beesley, T. (2018) · Quarterly Journal of Experimental Psychology · doi:10.1080/17470218.2017.1363257 · Cited by 42
The basal ganglia in reward and decision making
Doll, B. B., & Frank, M. J. (2009) · Elsevier eBooks · doi:10.1016/b978-0-12-374620-7.00019-4 · Cited by 31
The temporal representation of experience in subjective mood
Keren, H., Zheng, C. Y., Jangraw, D. C., Chang, K. K., Vitale, A. D., Nielson, D., Rutledge, R. B., Pereira, F., & Stringaris, A. K. (2021) · eLife · doi:10.7554/elife.62051 · Cited by 37
Neuromodulation and Neurophysiology on the Timescale of Learning and Decision-Making
Grossman, C. D., & Cohen, J. Y. (2022) · Annual Review of Neuroscience · doi:10.1146/annurev-neuro-092021-125059 · Cited by 34
Duration between rewards controls the rate of behavioral and dopaminergic learning
Burke, D. A., Taylor, A., Jeong, H., Lee, S., Zsembik, L. C. P., Wu, B., Floeder, J. R., Naik, G. A., Chen, R., & Namboodiri, V. M. K. (2026) · Nature Neuroscience · doi:10.1038/s41593-026-02206-2 · Cited by 30
Premkumar, P., Parbhakar, V. A., Fannon, D. G., Lythgoe, D. J., Williams, S., Kuipers, E., & Kumari, V. (2010) · Psychiatry Research Neuroimaging · doi:10.1016/j.pscychresns.2010.02.008 · Cited by 31
Adaptive Behavior: Humans Act as Bayesian Learners
Yu, A. J. (2007) · Current Biology · doi:10.1016/j.cub.2007.09.007 · Cited by 33
Nonparametric learning rules from bandit experiments: The eyes have it!
Hu, Y., Kayaba, Y., & Shum, M. (2013) · Games and Economic Behavior · doi:10.1016/j.geb.2013.05.003 · Cited by 34
Clark, J. J., Nasrallah, N. A., Hart, A. S., Collins, A. L., Bernstein, I. L., & Phillips, P. E. M. (2012) · PLoS ONE · doi:10.1371/journal.pone.0037357 · Cited by 29
Harlé, K. M., Zhang, S., Ma, N., Yu, A. J., & Paulus, M. P. (2016) · Biological Psychiatry: Cognitive Neuroscience and Neuroimaging · doi:10.1016/j.bpsc.2016.06.008 · Cited by 29
Anggraini, D., Glasauer, S., & Wunderlich, K. (2018) · Scientific Reports · doi:10.1038/s41598-018-28241-z · Cited by 35
Sensitivity to value-driven attention is predicted by how we learn from value
Jahfari, S., & Theeuwes, J. (2017) · Psychonomic Bulletin & Review · doi:10.3758/s13423-016-1106-6 · Cited by 28
Botelho, C., Fernandes, C., Campos, C., Seixas, C., Pasion, R., Garcez, H., Ferreira‐Santos, F., Barbosa, F., Teixeira, J. M., & Paiva, T. O. (2023) · Cognitive Affective & Behavioral Neuroscience · doi:10.3758/s13415-023-01101-8 · Cited by 38
A Control Theoretic Model of Adaptive Learning in Dynamic Environments
Ritz, H., Nassar, M. R., Frank, M. J., & Shenhav, A. (2018) · Journal of Cognitive Neuroscience · doi:10.1162/jocn_a_01289 · Cited by 33
Metadata
- Authors
- Behrens, T. E. J., Woolrich, M. W., Walton, M. E., & Rushworth, M. F. S.
- Affiliations
- John Radcliffe Hospital (GB)
- University of Oxford (GB)
- Corresponding author
- Behrens, Timothy E J
- Venue
- Nature Neuroscience
- Published
- 2007-08-05
- Volume
- 10
- Issue
- 9
- Pages
- 1214-1221
- ISSN
- 1097-6256 (ISSN-L)
- 1546-1726
- OpenAlex
- W2123429050
- Cited by
- 2,053
- References
- 52
- Citations by year
- 2026: 122 · 2025: 163 · 2024: 156 · 2023: 135 · 2022: 139 · 2021: 128 · 2020: 133 · 2019: 134 · 2018: 108 · 2017: 108 · 2016: 121 · 2015: 101 · 2014: 87 · 2013: 91 · 2012: 88
- FWCI
- 27.97
- Type
- journal article
- Language
- en
- Open access
- closed
- Licence
- www.springer.com/tdm (tdm)
- Locations
- Nature Neuroscience (publishedVersion)
- PubMed (publishedVersion)
- Funders
- Wellcome Trust
- Topics
- Life Sciences › Neuroscience › Cognitive Neuroscience › Neural and Behavioral Psychology Studies
- Social Sciences › Decision Sciences › General Decision Sciences › Decision-Making and Behavioral Economics
- Life Sciences › Neuroscience › Cellular and Molecular Neuroscience › Neurotransmitter Receptor Influence on Behavior
- Keywords
- value of information, uncertainty, optimal learning, anterior cingulate cortex, learning rate, Bayesian decision making, human learning, fMRI decoding
- MeSH terms
- Adolescent, Adult, Bayes Theorem, Brain Mapping, Decision Making, Female, Gyrus Cinguli, Humans, Image Processing, Computer-Assisted, Learning, Magnetic Resonance Imaging, Male, Oxygen, Pattern Recognition, Visual, Photic Stimulation, Probability, Reaction Time, Reinforcement, Psychology
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.