Learning the value of information in an uncertain world
The DOI matches its Crossref record; 52 of the 52 references it cites were found in a registry or an index.
Abstract
Our decisions are guided by outcomes that are associated with decisions made in the past. However, the amount of influence each past outcome has on our next decision remains unclear. To ensure optimal decision-making, the weight given to decision outcomes should reflect their salience in predicting future outcomes, and this salience should be modulated by the volatility of the reward environment. We show that human subjects assess volatility in an optimal manner and adjust decision-making accordingly. This optimal estimate of volatility is reflected in the fMRI signal in the anterior cingulate cortex (ACC) when each trial outcome is observed. When a new piece of information is witnessed, activity levels reflect its salience for predicting future outcomes. Furthermore, variations in this ACC signal across the population predict variations in subject learning rates. Our results provide a formal account of how we weigh our different experiences in guiding our future actions.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“When the environment is volatile people should give more weight to recent sensory outcomes in building expectations about what they will see next (e.g. adopt a high α), in contrast they should give information from the distant past more weight when the environment is stable (e.g. adopt a low α) 8,9.”
From Adults with autism overestimate the volatility of the sensory environment (2017) · cites this paper as [8]
In the citing paper
“While computational studies of decision-making about rewards and punishments show that participants adapt their rate of learning about action-outcome contingencies in response to changes in environmental volatility 8–10”
From Adults with autism overestimate the volatility of the sensory environment (2017) · cites this paper as [8]
In the citing paper
“Previous work has shown that human learners indeed adjust their learning rate according to experimentally manipulated levels of volatility (Behrens et al., 2007).”
From The generalized Hierarchical Gaussian Filter (2026) · cites this paper as Behrens et al., 2007
In the citing paper
“This is known as hierarchical filtering and emphasizes the importance of taking into account the volatility of the environment, i.e., the temporal instability of its statistical structure, such as the probabilities by which one event causes another (Behrens et al., 2007; Mathys et al., 2011).”
From Allostatic Self-efficacy: A Metacognitive Theory of Dyshomeostasis-Induced Fatigue and Depression (2016) · cites this paper as Behrens et al., 2007
In the citing paper
“The causal structure of the world, with its nested spatial and temporal scales, implies that the brain’s internal model also possesses a hierarchical structure, which is a natural form for Bayesian inference: hierarchical models allow to encode information about the precision of beliefs at one level by values of hierarchically higher levels (81–83) (Figure 3D).”
From Can Bayesian Theories of Autism Spectrum Disorder Help Improve Clinical Practice? (2016) · cites this paper as [82]
1 more claim about this paper
Create a free account to see every claim other papers make about this one.
Citations
Cites
Reinforcement Learning: An Introduction
Sutton, R. S., & Barto, A. G. (1998) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.1998.712192 · Cited by 27,225
Reinforcement Learning: An Introduction
(2005) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.2004.842673 · Cited by 24,496
Engle, R. F. (1982) · Econometrica · doi:10.2307/1912773 · Cited by 20,795
Advances in functional and structural MR image analysis and implementation as FSL
Smith, S. M., Jenkinson, M., Woolrich, M. W., Beckmann, C. F., Behrens, T. E. J., Johansen-Berg, H., Bannister, P. R., De Luca, M., Drobnjak, I., Flitney, D. E., et al. (2004) · NeuroImage · doi:10.1016/j.neuroimage.2004.07.051 · Cited by 13,630
Self-citationKahneman, D., & Tversky, A. (1984) · American Psychologist · doi:10.1037/0003-066x.39.4.341 · Cited by 6,544
Humans integrate visual and haptic information in a statistically optimal fashion
Ernst, M. O., & Banks, M. S. (2002) · Nature · doi:10.1038/415429a · Cited by 4,844
Aston-Jones, G., & Cohen, J. D. (2005) · Annual Review of Neuroscience · doi:10.1146/annurev.neuro.28.061604.135709 · Cited by 4,472
Classical conditioning II: Current research and theory.
Black, A. H., & Prokasy, W. F. (1972) · Appleton-Century-Crofts eBooks · Cited by 4,449
Conflict monitoring and anterior cingulate cortex: an update
Botvinick, M. M., Cohen, J. D., & Carter, C. S. (2004) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2004.10.003 · Cited by 3,623
Pearce, J. M., & Hall, G. (1980) · Psychological Review · doi:10.1037/0033-295x.87.6.532 · Cited by 3,023
Cited by
Items 401–500 of 1,013
Mas-Herrero, E., & Marco-Pallarés, J. (2014) · Journal of Cognitive Neuroscience · doi:10.1162/jocn_a_00516 · Cited by 85
Specific frontal neural dynamics contribute to decisions to check
Stoll, F. M., Fontanier, V., & Procyk, E. (2016) · Nature Communications · doi:10.1038/ncomms11990 · Cited by 86
Meder, D., Kolling, N., Verhagen, L., Wittmann, M. K., Scholl, J., Madsen, K. H., Hulme, O. J., Behrens, T. E. J., & Rushworth, M. F. S. (2017) · Nature Communications · doi:10.1038/s41467-017-02169-w · Cited by 85
Self-citationScholl, J., Kolling, N., Nelissen, N., Wittmann, M. K., Harmer, C. J., & Rushworth, M. F. S. (2015) · Journal of Neuroscience · doi:10.1523/jneurosci.0396-15.2015 · Cited by 85
Self-citationDisrupted prediction errors index social deficits in autism spectrum disorder
Balsters, J. H., Apps, M. A. J., Bolis, D., Lehner, R., Gallagher, L., & Wenderoth, N. (2017) · Brain · doi:10.1093/brain/aww287 · Cited by 85
“This is in keeping with studies that have shown that the ACCs encodes prediction errors for all agents in neurotypical cohorts (Behrens et al., 2007, 2008; Matsumoto et al., 2007; Apps et al., 2013b; Chang et al., 2013), while the ACCg exclusively encodes social prediction errors.”
ResultThe statistical structures of reinforcement learning with asymmetric value updates
Katahira, K. (2018) · Journal of Mathematical Psychology · doi:10.1016/j.jmp.2018.09.002 · Cited by 96
Hikosaka, O., Yamamoto, S., Yasuda, M., & Kim, H. F. (2013) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2013.07.001 · Cited by 83
The Predictive Brain as a Stubborn Scientist
Yon, D., de Lange, F. P., & Press, C. (2019) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2018.10.003 · Cited by 84
Is Model Fitting Necessary for Model-Based fMRI?
Wilson, R. C., & Niv, Y. (2015) · PLOS Computational Biology · doi:10.1371/journal.pcbi.1004237 · Cited by 83
Computational models of adaptive behavior and prefrontal cortex
Soltani, A., & Koechlin, É. (2021) · Neuropsychopharmacology · doi:10.1038/s41386-021-01123-1 · Cited by 96
Dynamic shaping of dopamine signals during probabilistic Pavlovian conditioning
Hart, A. S., Clark, J. J., & Phillips, P. E. M. (2015) · Neurobiology of Learning and Memory · doi:10.1016/j.nlm.2014.07.010 · Cited by 82
The Temporal Dynamics of Surprise
Noordewier, M. K., Topolinski, S., & van Dijk, E. (2016) · Social and Personality Psychology Compass · doi:10.1111/spc3.12242 · Cited by 107
Anticipating cognitive effort: roles of perceived error-likelihood and time demands
Dunn, T. L., Inzlicht, M., & Risko, E. F. (2017) · Psychological Research · doi:10.1007/s00426-017-0943-x · Cited by 94
Dopamine and reward: a view from the prefrontal cortex
Chau, B. K. H., Jarvis, H., Law, C.-K., & Chong, T. T. ‐. (2018) · Behavioural Pharmacology · doi:10.1097/fbp.0000000000000424 · Cited by 97
Probabilistic reversal learning is impaired in Parkinson's disease
Peterson, D. A., Elliott, C., Song, D. D., Makeig, S., Sejnowski, T. J., & Poizner, H. (2009) · Neuroscience · doi:10.1016/j.neuroscience.2009.07.033 · Cited by 83
Neural correlates of dueling affective reactions to win–win choices
Shenhav, A., & Buckner, R. L. (2014) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1405725111 · Cited by 82
Taming the beast: extracting generalizable knowledge from computational models of cognition
Nassar, M. R., & Frank, M. J. (2016) · Current Opinion in Behavioral Sciences · doi:10.1016/j.cobeha.2016.04.003 · Cited by 81
Cognitive Control as Cost‐Benefit Decision Making
Kool, W., Shenhav, A., & Botvinick, M. (2017) · Wiley eBooks · doi:10.1002/9781118920497.ch10 · Cited by 110
Contextual influence on confidence judgments in human reinforcement learning
Lebreton, M., Bacily, K., Palminteri, S., & Engelmann, J. B. (2019) · PLOS Computational Biology · doi:10.1371/journal.pcbi.1006973 · Cited by 81
Evaluating the evidence for expectation suppression in the visual system
Feuerriegel, D., Vogels, R., & Kovács, G. (2021) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2021.04.002 · Cited by 97
Increased Dependence of Action Selection on Recent Motor History in Parkinson's Disease
Helmich, R. C. G., Aarts, E., de Lange, F. P., Bloem, B. R., & Toni, I. (2009) · Journal of Neuroscience · doi:10.1523/jneurosci.0704-09.2009 · Cited by 99
Rewards and Cognitive Control in the Human Prefrontal Cortex
Duverne, S., & Koechlin, E. (2017) · Cerebral Cortex · doi:10.1093/cercor/bhx210 · Cited by 79
A tutorial on particle filters
Speekenbrink, M. (2016) · Journal of Mathematical Psychology · doi:10.1016/j.jmp.2016.05.006 · Cited by 78
The neurobiology of uncertainty: implications for statistical learning
Hasson, U. (2017) · Philosophical Transactions of the Royal Society B Biological Sciences · doi:10.1098/rstb.2016.0048 · Cited by 88
Glaze, C. M., Filipowicz, A. L. S., Kable, J. W., Balasubramanian, V., & Gold, J. I. (2018) · Nature Human Behaviour · doi:10.1038/s41562-018-0297-4 · Cited by 92
Henco, L., Diaconescu, A. O., Lahnakoski, J. M., Brandi, M.-L., Hörmann, S., Hennings, J., Hasan, A., Papazova, I., Strube, W., Bolis, D., et al. (2020) · PLOS Computational Biology · doi:10.1371/journal.pcbi.1008162 · Cited by 78
Neural Correlates of Probabilistic Category Learning in Patients with Schizophrenia
Weickert, T. W., Goldberg, T. E., Callicott, J. H., Chen, Q., Apud, J. A., Das, S., Zoltick, B. J., Egan, M. F., Meeter, M., Myers, C., et al. (2009) · The Journal of Neuroscience · doi:10.1523/jneurosci.4341-08.2009 · Cited by 78
Imitation as an inheritance system
Shea, N. (2009) · Philosophical Transactions of the Royal Society B: Biological Sciences · doi:10.1098/rstb.2009.0061 · Cited by 78
Single-Trial Event-Related Potential Correlates of Belief Updating
Bennett, D., Murawski, C., & Bode, S. (2015) · eneuro · doi:10.1523/eneuro.0076-15.2015 · Cited by 78
Monosov, I. E., & Hikosaka, O. (2013) · Nature Neuroscience · doi:10.1038/nn.3398 · Cited by 77
Strategic control in decision‐making under uncertainty
Venkatraman, V., & Huettel, S. A. (2012) · European Journal of Neuroscience · doi:10.1111/j.1460-9568.2012.08009.x · Cited by 77
Tea With Milk? A Hierarchical Generative Framework of Sequential Event Comprehension
Kuperberg, G. R. (2020) · Topics in Cognitive Science · doi:10.1111/tops.12518 · Cited by 96
Human Pavlovian fear conditioning conforms to probabilistic learning
Tzovara, A., Korn, C. W., & Bach, D. R. (2018) · PLOS Computational Biology · doi:10.1371/journal.pcbi.1006243 · Cited by 76
Dissociable reward and timing signals in human midbrain and ventral striatum
Klein-Flügge, M. C., Hunt, L. T., Bach, D. R., Dolan, R. J., & Behrens, T. E. J. (2011) · Neuron · doi:10.1016/j.neuron.2011.08.024 · Cited by 75
“…et al., 2009; Krugel et al., 2009), others have chosen tomaximize BOLD signal sensitivity by introducing significant randomness (up to 10 s) in the interval between conditioned stimulus and outcome (Behrens et al., 2007, 2008; Hare et al., 2008; Cohen et al., 2010; Daniel and Pollmann, 2010).”
Self-citationDistinct Regions within Medial Prefrontal Cortex Process Pain and Cognition
Jahn, A., Nee, D. E., Alexander, W. H., & Brown, J. W. (2016) · The Journal of Neuroscience · doi:10.1523/jneurosci.2180-16.2016 · Cited by 75
“In particular, the mPFC is responsive to conflict (Botvinick et al., 2001), error detection (Gehring et al., 1993), error likelihood (Brown and Braver, 2005), pain (Rainville et al., 1997), time on task (Grinband et al., 2011), and volatility (Behrens et al., 2007).”
BackgroundRyskin, R., Levy, R. P., & Fedorenko, E. (2020) · Neuropsychologia · doi:10.1016/j.neuropsychologia.2019.107258 · Cited by 75
Computation noise in human learning and decision-making: origin, impact, function
Findling, C., & Wyart, V. (2021) · Current Opinion in Behavioral Sciences · doi:10.1016/j.cobeha.2021.02.018 · Cited by 96
Valence-dependent influence of serotonin depletion on model-based choice strategy
Worbe, Y., Palminteri, S., Savulich, G., Daw, N. D., Fernandez-Egea, E., Robbins, T. W., & Voon, V (2016) · Molecular Psychiatry · doi:10.1038/mp.2015.46 · Cited by 75
Perceptual Classification in a Rapidly Changing Environment
Summerfield, C., Behrens, T. E., & Koechlin, E. (2011) · Neuron · doi:10.1016/j.neuron.2011.06.022 · Cited by 75
Self-citationStress, genotype and norepinephrine in the prediction of mouse behavior using reinforcement learning
Lukšys, G., Gerstner, W., & Sandi, C. (2009) · Nature Neuroscience · doi:10.1038/nn.2374 · Cited by 86
An Integrative Way for Studying Neural Basis of Basic Emotions With fMRI
Gu, S., Wang, F., Cao, C., Wu, E., Tang, Y., & Huang, J. H. (2019) · Frontiers in Neuroscience · doi:10.3389/fnins.2019.00628 · Cited by 86
Inactivation of the prelimbic or infralimbic cortex impairs decision-making in the rat gambling task
Zeeb, F. D., Baarendse, P. J. J., Vanderschuren, L. J. M. J., & Winstanley, C. A. (2015) · Psychopharmacology · doi:10.1007/s00213-015-4075-y · Cited by 92
Pupil diameter encodes the idiosyncratic, cognitive complexity of belief updating
Filipowicz, A. L. S., Glaze, C. M., Kable, J. W., & Gold, J. I. (2020) · eLife · doi:10.7554/elife.57872 · Cited by 93
Predictive waves in the autism-schizophrenia continuum: A novel biobehavioral model
Tarasi, L., Trajkovic, J., Diciotti, S., di Pellegrino, G., Ferri, F., Ursino, M., & Romei, V. (2021) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2021.11.006 · Cited by 79
Langner, R., Kellermann, T., Eickhoff, S. B., Boers, F., Chatterjee, A., Willmes, K., & Sturm, W. (2012) · Human Brain Mapping · doi:10.1002/hbm.21220 · Cited by 74
Silvetti, M., Seurinck, R., & Verguts, T. (2012) · Cortex · doi:10.1016/j.cortex.2012.05.008 · Cited by 92
Amemori, K.-I., Gibb, L. G., & Graybiel, A. M. (2011) · Frontiers in Human Neuroscience · doi:10.3389/fnhum.2011.00047 · Cited by 73
Lai, M.-C., Lombardo, M. V., Chakrabarti, B., Ecker, C., Sadek, S. A., Wheelwright, S. J., Murphy, D. G. M., Suckling, J., Bullmore, E. T., & Baron-Cohen, S. (2012) · NeuroImage · doi:10.1016/j.neuroimage.2012.03.018 · Cited by 74
Amiez, C., Sallet, J., Procyk, E., & Petrides, M. (2012) · NeuroImage · doi:10.1016/j.neuroimage.2012.06.023 · Cited by 92
Decoding representations of scenes in the medial temporal lobes
Bonnici, H. M., Kumaran, D., Chadwick, M. J., Weiskopf, N., Hassabis, D., & Maguire, E. A. (2012) · Hippocampus · doi:10.1002/hipo.20960 · Cited by 74
Neural Correlates of Error Monitoring Modulated by Atomoxetine in Healthy Volunteers
Graf, H., Abler, B., Freudenmann, R. W., Beschoner, P., Schaeffeler, E., Spitzer, M., Schwab, M. G., & Grön, G. (2010) · Biological Psychiatry · doi:10.1016/j.biopsych.2010.10.018 · Cited by 80
Atypical processing of uncertainty in individuals at risk for psychosis
Cole, D. M., Diaconescu, A. O., Pfeiffer, U. J., Brodersen, K. H., Mathys, C. D., Julkowski, D., Ruhrmann, S., Schilbach, L., Tittgemeyer, M., Vogeley, K., et al. (2020) · NeuroImage: Clinical · doi:10.1016/j.nicl.2020.102239 · Cited by 72
Yoo, A. H., & Collins, A. G. E. (2022) · Journal of Cognitive Neuroscience · doi:10.1162/jocn_a_01808 · Cited by 97
Kheifets, A., & Gallistel, C. R. (2012) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1205131109 · Cited by 72
Model-Based Reasoning in Humans Becomes Automatic with Training
Economides, M., Kurth-Nelson, Z., Lübbert, A., Guitart-Masip, M., & Dolan, R. J. (2015) · PLOS Computational Biology · doi:10.1371/journal.pcbi.1004463 · Cited by 72
When the context matters: Identity, secession and the spatial dimension in Catalonia
Rodón, T., & Guinjoan, M. (2018) · Political Geography · doi:10.1016/j.polgeo.2018.01.004 · Cited by 72
Distributional Reinforcement Learning in the Brain
Lowet, A. S., Qiao, Z., Matias, S., Drugowitsch, J., & Uchida, N. (2020) · Trends in Neurosciences · doi:10.1016/j.tins.2020.09.004 · Cited by 93
Najar, A., Bonnet, E., Bahrami, B., & Palminteri, S. (2020) · PLoS Biology · doi:10.1371/journal.pbio.3001028 · Cited by 89
A new model of decision processing in instrumental learning tasks
Miletić, S., Boag, R. J., Trutti, A. C., Stevenson, N., Forstmann, B. U., & Heathcote, A. (2021) · eLife · doi:10.7554/elife.63055 · Cited by 87
Buss, A. T., & Spencer, J. P. (2014) · Monographs of the Society for Research in Child Development · doi:10.1002/mono.12096 · Cited by 87
Action and Outcome Activity State Patterns in the Anterior Cingulate Cortex
Hyman, J. M., Whitman, J., Emberly, E. G., Woodward, T. S., & Seamans, J. K. (2012) · Cerebral Cortex · doi:10.1093/cercor/bhs104 · Cited by 80
Control of entropy in neural models of environmental state
Muller, T. H., Mars, R. B., Behrens, T. E., & O'Reilly, J. X. (2019) · eLife · doi:10.7554/elife.39404 · Cited by 67
Self-citationScaling prediction errors to reward variability benefits error-driven learning in humans
Diederen, K. M. J., & Schultz, W. (2015) · Journal of Neurophysiology · doi:10.1152/jn.00483.2015 · Cited by 69
“Indeed, previous studies reported that BOLD responses in the paracingulate and anterior cingulate cortex, the cuneus, and the prefrontal cortex reflect variations in absolute learning rate (Behrens et al. 2007; Krugel et al. 2009; Payzan-LeNestour et al. 2013; Vilares et al. 2012).”
BackgroundResponse outcomes gate the impact of expectations on perceptual decisions
Hermoso-Mendizabal, A., Hyafil, A., Rueda‐Orozco, P. E., Jaramillo, S., Robbe, D., & de la Rocha, J. (2020) · Nature Communications · doi:10.1038/s41467-020-14824-w · Cited by 72
Reward Prediction Error and Declarative Memory
Ergo, K., Loof, E. D., & Verguts, T. (2020) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2020.02.009 · Cited by 71
Decision Salience Signals in Posterior Cingulate Cortex
Heilbronner, S. R., Hayden, B. Y., & Platt, M. L. (2011) · Frontiers in Neuroscience · doi:10.3389/fnins.2011.00055 · Cited by 69
Control without Controllers: Toward a Distributed Neuroscience of Executive Control
Eisenreich, B. R., Akaishi, R., & Hayden, B. Y. (2017) · Journal of Cognitive Neuroscience · doi:10.1162/jocn_a_01139 · Cited by 68
Balancing new against old information: the role of puzzlement surprise in learning
Faraji, M., Preuschoff, K., & Gerstner, W. (2018) · Neural Computation · doi:10.1162/neco_a_01025 · Cited by 68
“The first is a simple, one-dimensional dynamic decision-making task that has been used in behavioral studies (Behrens et al., 2007; Nassar et al., 2012) of learning under uncertainty.”
BackgroundMethodInfluentialTheory of mind and decision science: Towards a typology of tasks and computational models
Rusch, T., Kumar, S., Doshi, P., Spezio, M., & Gläscher, J. P. (2020) · Neuropsychologia · doi:10.1016/j.neuropsychologia.2020.107488 · Cited by 96
Individuals with adverse childhood experiences explore less and underweight reward feedback
Lloyd, A., McKay, R. T., & Furl, N. (2022) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2109373119 · Cited by 68
Neurophysiological mechanisms of error monitoring in human and non-human primates
Fu, Z., Sajad, A., Errington, S. P., Schall, J. D., & Rutishauser, U. (2023) · Nature Reviews Neuroscience · doi:10.1038/s41583-022-00670-w · Cited by 66
Word Wins Over Face: Emotional Stroop Effect Activates the Frontal Cortical Network
Ovaysikia, S., Tahir, K. A., Chan, J. L., & DeSouza, J. F. X. (2011) · Frontiers in Human Neuroscience · doi:10.3389/fnhum.2010.00234 · Cited by 68
A general role for medial prefrontal cortex in event prediction
Alexander, W. H., & Brown, J. W. (2014) · Frontiers in Computational Neuroscience · doi:10.3389/fncom.2014.00069 · Cited by 68
Dissociable forms of uncertainty-driven representational change across the human brain
Nassar, M. R., McGuire, J. T., Ritz, H., & Kable, J. (2018) · The Journal of Neuroscience · doi:10.1523/jneurosci.1713-18.2018 · Cited by 68
Dezza, I. C., Yu, A. J., Cleeremans, A., & Alexander, W. (2017) · Scientific Reports · doi:10.1038/s41598-017-17237-w · Cited by 420
Prefrontal reinstatement of contextual task demand is predicted by separable hippocampal patterns
Jiang, J., Wang, S.-F., Guo, W., Fernandez, C., & Wagner, A. D. (2020) · Nature Communications · doi:10.1038/s41467-020-15928-z · Cited by 70
Pedersen, M. L., & Frank, M. J. (2020) · Computational Brain & Behavior · doi:10.1007/s42113-020-00084-w · Cited by 73
Effort, success, and nonuse determine arm choice
Schweighofer, N., Xiao, Y., Kim, S., Yoshioka, T., Gordon, J., & Osu, R. (2015) · Journal of Neurophysiology · doi:10.1152/jn.00593.2014 · Cited by 69
Integration of individual and social information for decision-making in groups of different sizes
Park, S. A., Goïame, S., O'Connor, D. A., & Dreher, J.-C. (2017) · PLOS Biology · doi:10.1371/journal.pbio.2001958 · Cited by 67
Neural evidence for Bayesian trial-by-trial adaptation on the N400 during semantic priming
Delaney-Busch, N., Morgan, E., Lau, E. F., & Kuperberg, G. R. (2019) · Cognition · doi:10.1016/j.cognition.2019.01.001 · Cited by 80
Differentiable Neural Substrates for Learned and Described Value and Risk
FitzGerald, T. H. B., Seymour, B., Bach, D. R., & Dolan, R. J. (2010) · Current Biology · doi:10.1016/j.cub.2010.08.048 · Cited by 66
Nastase, S., Iacovella, V., & Hasson, U. (2014) · Human Brain Mapping · doi:10.1002/hbm.22238 · Cited by 66
Nassar, M. R., Waltz, J. A., Albrecht, M. A., Gold, J. M., & Frank, M. J. (2020) · Brain · doi:10.1093/brain/awaa453 · Cited by 75
Early adversity and the development of explore–exploit tradeoffs
Frankenhuis, W. E., & Gopnik, A. (2023) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2023.04.001 · Cited by 89
Schiffer, A.-M., Ahlheim, C., Wurm, M. F., & Schubotz, R. I. (2012) · PLoS ONE · doi:10.1371/journal.pone.0036445 · Cited by 65
Group independent component analysis of resting state EEG in large normative samples
Congedo, M., John, R. E., Ridder, D. D., & Prichep, L. S. (2010) · International Journal of Psychophysiology · doi:10.1016/j.ijpsycho.2010.06.003 · Cited by 72
Neural Responses to Ambiguity Involve Domain-general and Domain-specific Emotion Processing Systems
Neta, M., Kelley, W. M., & Whalen, P. J. (2013) · Journal of Cognitive Neuroscience · doi:10.1162/jocn_a_00363 · Cited by 76
Scholl, J., & Klein-Flügge, M. C. (2017) · Behavioural Brain Research · doi:10.1016/j.bbr.2017.09.050 · Cited by 83
Reinforcement Learning Enables Resource Partitioning in Foraging Bats
Goldshtein, A., Handel, M., Eitan, O., Bonstein, A., Shaler, T., Collet, S., Greif, S., Medellín, R. A., Emek, Y., Korman, A., et al. (2020) · Current Biology · doi:10.1016/j.cub.2020.07.079 · Cited by 65
Discovering cognitive strategies with tiny recurrent neural networks
Ji-An, L., Benna, M. K., & Mattar, M. G. (2025) · Nature · doi:10.1038/s41586-025-09142-4 · Cited by 64
“They do, however, reflect established principles of learning—adaptive learning rates 35–37 , trial-to-trial variability 38 and side biases 12 (Supplementary Logit change patterns suggest similarities between meta-RL agent and Bayesian inference model, and differences relative to model-based RL…”
What makes the dorsomedial frontal cortex active during reading the mental states of others?
Isoda, M., & Noritake, A. (2013) · Frontiers in Neuroscience · doi:10.3389/fnins.2013.00232 · Cited by 64
“Indeed, uncertainty driven by volatility modulates pre-SMA activity (Behrens et al., 2007).”
BackgroundRandles, D., Inzlicht, M., Proulx, T., Tullett, A. M., & Heine, S. J. (2015) · Journal of Personality and Social Psychology · doi:10.1037/a0038933 · Cited by 64
The limits of learning: Exploration, generalization, and the development of learning traps.
Rich, A. S., & Gureckis, T. M. (2018) · Journal of Experimental Psychology General · doi:10.1037/xge0000466 · Cited by 84
Boosting perceptual learning by fake feedback
Shibata, K., Yamagishi, N., Ishii, S., & Kawato, M. (2009) · Vision Research · doi:10.1016/j.visres.2009.06.009 · Cited by 74
Cholinergic Stimulation Enhances Bayesian Belief Updating in the Deployment of Spatial Attention
Vossel, S., Bauer, M., Mathys, C., Adams, R. A., Dolan, R. J., Stephan, K. E., & Friston, K. J. (2014) · The Journal of Neuroscience · doi:10.1523/jneurosci.0091-14.2014 · Cited by 64
Neural mechanisms and computations underlying stress effects on learning and memory
Lukšys, G., & Sandi, C. (2011) · Current Opinion in Neurobiology · doi:10.1016/j.conb.2011.03.003 · Cited by 68
Unexpected but Incidental Positive Outcomes Predict Real-World Gambling
Otto, A. R., Fleming, S. M., & Glimcher, P. W. (2016) · Psychological Science · doi:10.1177/0956797615618366 · Cited by 71
Gagne, C., Dayan, P., & Bishop, S. J. (2018) · Current Opinion in Behavioral Sciences · doi:10.1016/j.cobeha.2018.03.013 · Cited by 63
Tonic exploration governs both flexibility and lapses
Ebitz, R. B., Sleezer, B. J., Jedema, H. P., Bradberry, C. W., & Hayden, B. Y. (2019) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1007475 · Cited by 71
Enax, L., Krapp, V., Piehl, A., & Weber, B. (2015) · Frontiers in Behavioral Neuroscience · doi:10.3389/fnbeh.2015.00247 · Cited by 80
Metadata
- Authors
- Behrens, T. E. J., Woolrich, M. W., Walton, M. E., & Rushworth, M. F. S.
- Affiliations
- John Radcliffe Hospital (GB)
- University of Oxford (GB)
- Corresponding author
- Behrens, Timothy E J
- Venue
- Nature Neuroscience
- Published
- 2007-08-05
- Volume
- 10
- Issue
- 9
- Pages
- 1214-1221
- ISSN
- 1097-6256 (ISSN-L)
- 1546-1726
- OpenAlex
- W2123429050
- Cited by
- 2,053
- References
- 52
- Citations by year
- 2026: 122 · 2025: 163 · 2024: 156 · 2023: 135 · 2022: 139 · 2021: 128 · 2020: 133 · 2019: 134 · 2018: 108 · 2017: 108 · 2016: 121 · 2015: 101 · 2014: 87 · 2013: 91 · 2012: 88
- FWCI
- 27.97
- Type
- journal article
- Language
- en
- Open access
- closed
- Licence
- www.springer.com/tdm (tdm)
- Locations
- Nature Neuroscience (publishedVersion)
- PubMed (publishedVersion)
- Funders
- Wellcome Trust
- Topics
- Life Sciences › Neuroscience › Cognitive Neuroscience › Neural and Behavioral Psychology Studies
- Social Sciences › Decision Sciences › General Decision Sciences › Decision-Making and Behavioral Economics
- Life Sciences › Neuroscience › Cellular and Molecular Neuroscience › Neurotransmitter Receptor Influence on Behavior
- Keywords
- value of information, uncertainty, optimal learning, anterior cingulate cortex, learning rate, Bayesian decision making, human learning, fMRI decoding
- MeSH terms
- Adolescent, Adult, Bayes Theorem, Brain Mapping, Decision Making, Female, Gyrus Cinguli, Humans, Image Processing, Computer-Assisted, Learning, Magnetic Resonance Imaging, Male, Oxygen, Pattern Recognition, Visual, Photic Stimulation, Probability, Reaction Time, Reinforcement, Psychology
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.