Cited by 286Neural and Behavioral Psychology StudiesMemory and Neural MechanismsDecision-Making and Behavioral Economics

The DOI matches its Crossref record; 118 of the 118 references it cites were found in a registry or an index.

Adaptive learning under expected and unexpected uncertainty

Open access

Soltani, Alireza, Izquierdo, Alicia·2019·Nature reviews. Neuroscience

DOI 10.1038/s41583-019-0180-yPMC PMC6752962

Open this paper in CoChat — free account

Abstract

The outcome of a decision is often uncertain, and outcomes can vary over repeated decisions. Whether decision outcomes should substantially affect behaviour and learning depends on whether they are representative of a typically experienced range of outcomes or signal a change in the reward environment. Successful learning and decision-making therefore require the ability to estimate expected uncertainty (related to the variability of outcomes) and unexpected uncertainty (related to the variability of the environment). Understanding the bases and effects of these two types of uncertainty and the interactions between them - at the computational and the neural level - is crucial for understanding adaptive learning. Here, we examine computational models and experimental findings to distil computational principles and neural mechanisms for adaptive learning under uncertainty.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

Integrity signals

Registry

Crossref · 10.1038/s41583-019-0180-y

Cites

118 of 118 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Reinforcement Learning: An Introduction
Sutton, R. S., & Barto, A. G. (1998) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.1998.712192 · Cited by 27,225
Reinforcement Learning: An Introduction
Sutton, R. S., & Barto, A. (2005) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.2004.842673 · Cited by 25,268
A neural substrate of prediction and reward
Schultz, W., Dayan, P., & Montague, P. R. (1997) · Science · doi:10.1126/science.275.5306.1593 · Cited by 9,388
What is the role of dopamine in reward: hedonic impact, reward learning, or incentive salience?
Berridge, K. C., & Robinson, T. E. (1998) · Brain Research Reviews · doi:10.1016/s0165-0173(98)00019-8 · Cited by 4,127
A model for Pavlovian learning: Variations in the effectiveness of conditioned but not of unconditioned stimuli.
Pearce, J. M., & Hall, G. (1980) · Psychological Review · doi:10.1037/0033-295x.87.6.532 · Cited by 3,023
The expected value of control: an integrative theory of anterior cingulate cortex function
Shenhav, A., Botvinick, M. M., & Cohen, J. D. (2013) · Neuron · doi:10.1016/j.neuron.2013.07.007 · Cited by 2,321
Cortical substrates for exploratory decisions in humans
Daw, N. D., O'Doherty, J. P., Dayan, P., Seymour, B., & Dolan, R. J. (2006) · Nature · doi:10.1038/nature04766 · Cited by 2,224
Learning the value of information in an uncertain world
Behrens, T. E. J., Woolrich, M. W., Walton, M. E., & Rushworth, M. F. S. (2007) · Nature Neuroscience · doi:10.1038/nn1954 · Cited by 2,053
Uncertainty and anticipation in anxiety: an integrated neurobiological and psychological perspective
Grupe, D. W., & Nitschke, J. B. (2013) · Nature Reviews Neuroscience · doi:10.1038/nrn3524 · Cited by 2,007
Uncertainty, Neuromodulation, and Attention
Yu, A. J., & Dayan, P. (2005) · Neuron · doi:10.1016/j.neuron.2005.04.026 · Cited by 1,774
Primate anterior cingulate cortex: Where motor control, drive and cognition interface
Paus, T. (2001) · Nature Reviews Neuroscience · doi:10.1038/35077500 · Cited by 1,722
Metaplasticity: tuning synapses and networks for plasticity
Abraham, W. C. (2008) · Nature reviews. Neuroscience · doi:10.1038/nrn2356 · Cited by 1,166
Amygdalo‐cortical projections in the monkey ( Macaca fascicularis )
Amaral, D. G., & Price, J. L. (1984) · Journal of Comparative Neurology · doi:10.1002/cne.902300402 · Cited by 1,162
Frontal cortex and reward-guided learning and decision-making
Rushworth, M. F. S., Noonan, M. P., Boorman, E. D., Walton, M. E., & Behrens, T. E. (2011) · Neuron · doi:10.1016/j.neuron.2011.05.014 · Cited by 1,111
Optimal decision making and the anterior cingulate cortex
Kennerley, S. W., Walton, M. E., Behrens, T. E. J., Buckley, M. J., & Rushworth, M. F. S. (2006) · Nature Neuroscience · doi:10.1038/nn1724 · Cited by 948
Choice, uncertainty and value in prefrontal and cingulate cortex
Rushworth, M. F. S., & Behrens, T. E. J. (2008) · Nature Neuroscience · doi:10.1038/nn2066 · Cited by 866
The short-latency dopamine signal: a role in discovering novel actions?
Redgrave, P., & Gurney, K. (2006) · Nature Reviews Neuroscience · doi:10.1038/nrn2022 · Cited by 762
The neural basis of reversal learning: An updated perspective
Izquierdo, A., Brigman, J. L., Radke, A. K., Rudebeck, P. H., & Holmes, A. (2017) · Neuroscience · doi:10.1016/j.neuroscience.2016.03.021 · Cited by 670
Self-citation
A Bayesian foundation for individual learning under uncertainty
Mathys, C. (2011) · Frontiers in Human Neuroscience · doi:10.3389/fnhum.2011.00039 · Cited by 655
Modulators of decision making
Doya, K. (2008) · Nature Neuroscience · doi:10.1038/nn2077 · Cited by 642
An Approximately Bayesian Delta-Rule Model Explains the Dynamics of Belief Updating in a Changing Environment
Nassar, M. R., Wilson, R. C., Heasly, B. S., & Gold, J. I. (2010) · Journal of Neuroscience · doi:10.1523/jneurosci.0822-10.2010 · Cited by 607
What the orbitofrontal cortex does not do
Stalnaker, T. A., Cooch, N. K., & Schoenbaum, G. (2015) · Nature Neuroscience · doi:10.1038/nn.3982 · Cited by 531
Dorsal Anterior Cingulate Cortex: A Bottom-Up View
Heilbronner, S. R., & Hayden, B. Y. (2016) · Annual Review of Neuroscience · doi:10.1146/annurev-neuro-070815-013952 · Cited by 503
Learning and selective attention
Dayan, P., Kakade, S., & Montague, P. R. (2000) · Nature Neuroscience · doi:10.1038/81504 · Cited by 495
Reward Value Coding Distinct From Risk Attitude-Related Uncertainty Coding in Human Reward Systems
Tobler, P. N., O'Doherty, J. P., Dolan, R. J., & Schultz, W. (2007) · Journal of Neurophysiology · doi:10.1152/jn.00745.2006 · Cited by 472
Bayesian theories of conditioning in a changing world
Courville, A. C., Daw, N. D., & Touretzky, D. S. (2006) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2006.05.004 · Cited by 457
Beyond the reward hypothesis: alternative functions of nucleus accumbens dopamine
Salamone, J. D., Correa, M., Mingote, S., & Weber, S. (2004) · Current Opinion in Pharmacology · doi:10.1016/j.coph.2004.09.004 · Cited by 454
Effects of lesions of the orbitofrontal cortex on sensitivity to delayed and probabilistic reinforcement
S., M., S., B., M.-Y., H., C., B., E., S., J., D., & I., A. (2002) · Psychopharmacology · doi:10.1007/s00213-001-0983-0 · Cited by 409
Separable learning systems in the macaque brain and the role of orbitofrontal cortex in contingent learning
Walton, M. E., Behrens, T. E. J., Buckley, M. J., Rudebeck, P. H., & Rushworth, M. F. S. (2010) · Neuron · doi:10.1016/j.neuron.2010.02.027 · Cited by 397
Cross-species studies of orbitofrontal cortex and value-based decision-making
Wallis, J. D. (2011) · Nature Neuroscience · doi:10.1038/nn.2956 · Cited by 384
Surprise Signals in Anterior Cingulate Cortex: Neuronal Encoding of Unsigned Reward Prediction Errors Driving Adjustment in Behavior
Hayden, B. Y., Heilbronner, S. R., Pearson, J. M., & Platt, M. L. (2011) · The Journal of Neuroscience · doi:10.1523/jneurosci.4652-10.2011 · Cited by 379
Distinct Roles of Rodent Orbitofrontal and Medial Prefrontal Cortex in Decision Making
Sul, J. H., Kim, H., Huh, N., Lee, D., & Jung, M. W. (2010) · Neuron · doi:10.1016/j.neuron.2010.03.033 · Cited by 378
Decoding subjective decisions from orbitofrontal cortex
Rich, E. L., & Wallis, J. D. (2016) · Nature Neuroscience · doi:10.1038/nn.4320 · Cited by 368
Over the river, through the woods: cognitive maps in the hippocampus and orbitofrontal cortex
Wikenheiser, A. M., & Schoenbaum, G. M. (2016) · Nature reviews. Neuroscience · doi:10.1038/nrn.2016.56 · Cited by 363
The Cognitive Thalamus as a Gateway to Mental Representations
Wolff, M., & Vann, S. D. (2018) · Journal of Neuroscience · doi:10.1523/jneurosci.0479-18.2018 · Cited by 361
Cytoarchitecture of mouse and rat cingulate cortex with human homologies
Vogt, B. A., & Paxinos, G. (2014) · Brain Structure and Function · doi:10.1007/s00429-012-0493-3 · Cited by 348
Separate value comparison and learning mechanisms in macaque medial and lateral orbitofrontal cortex
Noonan, M. P., Walton, M. E., Behrens, T. E. J., Sallet, J., Buckley, M. J., & Rushworth, M. F. S. (2010) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1012246107 · Cited by 340
Knowing how much you don't know: a neural organization of uncertainty estimates
Bach, D. R., & Dolan, R. J. (2012) · Nature Reviews Neuroscience · doi:10.1038/nrn3289 · Cited by 339
Neural systems implicated in delayed and probabilistic reinforcement
Cardinal, R. N. (2006) · Neural Networks · doi:10.1016/j.neunet.2006.03.004 · Cited by 330
Temporal Filtering of Reward Signals in the Dorsal Anterior Cingulate Cortex during a Mixed-Strategy Game
Seo, H., & Lee, D. (2007) · The Journal of Neuroscience · doi:10.1523/jneurosci.2369-07.2007 · Cited by 324
An Unexpected Sequence of Events: Mismatch Detection in the Human Hippocampus
Kumaran, D., & Maguire, E. A. (2006) · PLoS Biology · doi:10.1371/journal.pbio.0040424 · Cited by 324
Circuit-Based Corticostriatal Homologies Between Rat and Primate
Heilbronner, S. R., Rodríguez-Romaguera, J., Quirk, G. J., Groenewegen, H. J., & Haber, S. N. (2016) · Biological Psychiatry · doi:10.1016/j.biopsych.2016.05.012 · Cited by 321
The basolateral amygdala in reward learning and addiction
Wassum, K. M., & Izquierdo, A. (2015) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2015.08.017 · Cited by 306
Self-citation
Functionally dissociable influences on learning rate in a dynamic environment
McGuire, J. T., Nassar, M. R., Gold, J. I., & Kable, J. W. (2014) · Neuron · doi:10.1016/j.neuron.2014.10.013 · Cited by 299
Risk, unexpected uncertainty, and estimation uncertainty: Bayesian learning in unstable settings
Payzan-LeNestour, E., & Bossaerts, P. (2011) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1001048 · Cited by 267
The Roles of Dopamine and Serotonin in Decision Making: Evidence from Pharmacological Experiments in Humans
Rogers, R. D. (2010) · Neuropsychopharmacology · doi:10.1038/npp.2010.165 · Cited by 262
Fundamental contribution by the basolateral amygdala to different forms of decision making
Ghods-Sharifi, S., St. Onge, J. R., & Floresco, S. B. (2009) · The Journal of Neuroscience · doi:10.1523/jneurosci.0315-09.2009 · Cited by 257
The neural representation of unexpected uncertainty during value-based decision making
Payzan-LeNestour, E., Dunne, S., Bossaerts, P., & O’Doherty, J. P. (2013) · Neuron · doi:10.1016/j.neuron.2013.04.037 · Cited by 257
Anterior cingulate error‐related activity is modulated by predicted reward
Amiez, C., Joseph, J., & Procyk, E. (2005) · European Journal of Neuroscience · doi:10.1111/j.1460-9568.2005.04170.x · Cited by 252
Projections of the medial orbital and ventral orbital cortex in the rat
Hoover, W. B., & Vertes, R. P. (2011) · The Journal of Comparative Neurology · doi:10.1002/cne.22733 · Cited by 251
Functional Heterogeneity within Rat Orbitofrontal Cortex in Reward Learning and Decision Making
Izquierdo, A. (2017) · Journal of Neuroscience · doi:10.1523/jneurosci.1678-17.2017 · Cited by 251
Self-citation
Coding of Reward Risk by Orbitofrontal Neurons Is Mostly Distinct from Coding of Reward Value
O’Neill, M. J., & Schultz, W. (2010) · Neuron · doi:10.1016/j.neuron.2010.09.031 · Cited by 249
Effects of lesions of the nucleus accumbens core on choice between small certain rewards and large uncertain rewards in rats
Cardinal, R. N., & Howes, N. J. (2005) · BMC Neuroscience · doi:10.1186/1471-2202-6-37 · Cited by 248
Genetic variation in dopaminergic neuromodulation influences the ability to rapidly and flexibly adapt decisions
Krugel, L. K., Biele, G., Mohr, P. N. C., Li, S.-C., & Heekeren, H. R. (2009) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.0905191106 · Cited by 229
Triple dissociation of attention and decision computations across prefrontal cortex
Hunt, L. T., Malalasekera, W. M. N., de Berker, A. O., Miranda, B., Farmer, S. F., Behrens, T. E. J., & Kennerley, S. W. (2018) · Nature Neuroscience · doi:10.1038/s41593-018-0239-5 · Cited by 222
Medial Orbitofrontal Cortex Mediates Outcome Retrieval in Partially Observable Task Situations
Bradfield, L. A., Dezfouli, A., van Holstein, M., Chieng, B. C. H., & Balleine, B. W. (2015) · Neuron · doi:10.1016/j.neuron.2015.10.044 · Cited by 211
Surprise! Neural correlates of Pearce–Hall and Rescorla–Wagner coexist within the brain
Roesch, M. R., Esber, G. R., Li, J., Daw, N. D., & Schoenbaum, G. M. (2012) · European Journal of Neuroscience · doi:10.1111/j.1460-9568.2011.07986.x · Cited by 208
Multifaceted Contributions by Different Regions of the Orbitofrontal and Medial Prefrontal Cortex to Probabilistic Reversal Learning
Dalton, G. L., Wang, N. Y., Phillips, A. G., & Floresco, S. B. (2016) · The Journal of Neuroscience · doi:10.1523/jneurosci.3366-15.2016 · Cited by 184
Abstract context representations in primate amygdala and prefrontal cortex
Saez, A., Rigotti, M., Ostojic, S., Fusi, S., & Salzman, C. D. (2015) · Neuron · doi:10.1016/j.neuron.2015.07.024 · Cited by 183
Motivational neural circuits underlying reinforcement learning
Averbeck, B. B., & Costa, V. D. (2017) · Nature Neuroscience · doi:10.1038/nn.4506 · Cited by 181
Separate Prefrontal-Subcortical Circuits Mediate Different Components of Risk-Based Decision Making
Onge, J. R. S., Stopper, C. M., Zahm, D. S., & Floresco, S. (2012) · Journal of Neuroscience · doi:10.1523/jneurosci.5625-11.2012 · Cited by 172
Advances in Understanding Mechanisms of Thalamic Relays in Cognition and Behavior
Mitchell, A. S., Sherman, S. M., Sommer, M. A., Mair, R. G., Vertes, R. P., & Chudasama, Y. (2014) · Journal of Neuroscience · doi:10.1523/jneurosci.3289-14.2014 · Cited by 171
Reversal learning and dopamine: a Bayesian perspective
Costa, V. D., Tran, V. L., Turchi, J., & Averbeck, B. B. (2015) · The Journal of Neuroscience · doi:10.1523/jneurosci.1989-14.2015 · Cited by 170
Specialized Representations of Value in the Orbital and Ventrolateral Prefrontal Cortex: Desirability versus Availability of Outcomes
Rudebeck, P. H., Saunders, R. C., Lundgren, D. A., & Murray, E. A. (2017) · Neuron · doi:10.1016/j.neuron.2017.07.042 · Cited by 170
Anterior cingulate is a source of valence-specific information about value and uncertainty
Monosov, I. E. (2017) · Nature Communications · doi:10.1038/s41467-017-00072-y · Cited by 163
Topography of projections from the medial prefrontal cortex to the amygdala in the rat.
Cassell, M., & Wright, D. J. (1986) · Brain Research Bulletin · doi:10.1016/0361-9230(86)90237-6 · Cited by 161
Model-based predictions for dopamine
Langdon, A. J., Sharpe, M. J., Schoenbaum, G., & Niv, Y. (2018) · Current Opinion in Neurobiology · doi:10.1016/j.conb.2017.10.006 · Cited by 161
Mediodorsal Thalamus Hypofunction Impairs Flexible Goal-Directed Behavior
Parnaudeau, S., Taylor, K., Bolkan, S., Ward, R. D., Balsam, P., & Kellendonk, C. (2014) · Biological Psychiatry · doi:10.1016/j.biopsych.2014.03.020 · Cited by 156
Synaptic Targets of Medial Septal Projections in the Hippocampus and Extrahippocampal Cortices of the Mouse
Ünal, G., Joshi, A., Viney, T. J., Kis, V., & Somogyi, P. (2015) · Journal of Neuroscience · doi:10.1523/jneurosci.2639-15.2015 · Cited by 154
Dopamine, uncertainty and TD learning
Niv, Y., Duff, M. O., & Dayan, P. (2005) · Behavioral and Brain Functions · doi:10.1186/1744-9081-1-6 · Cited by 153
Neural Correlates of Variations in Event Processing during Learning in Basolateral Amygdala
Roesch, M. R., Calu, D. J., Esber, G. R., & Schoenbaum, G. M. (2010) · Journal of Neuroscience · doi:10.1523/jneurosci.5781-09.2010 · Cited by 151
Compulsivity reveals a novel dissociation between action and confidence
Vaghi, M. M., Luyckx, F., Sule, A., Fineberg, N. A., Robbins, T. W., & De Martino, B. (2017) · Neuron · doi:10.1016/j.neuron.2017.09.006 · Cited by 145
Amygdala and Ventral Striatum Make Distinct Contributions to Reinforcement Learning
Costa, V. D., Monte, O. D., Lucas, D. R., Murray, E. A., & Averbeck, B. B. (2016) · Neuron · doi:10.1016/j.neuron.2016.09.025 · Cited by 141
Overriding Phasic Dopamine Signals Redirects Action Selection during Risk/Reward Decision Making
Stopper, C. M., Tse, M. T. L., Montes, D. R., Wiedman, C. R., & Floresco, S. B. (2014) · Neuron · doi:10.1016/j.neuron.2014.08.033 · Cited by 136
Adding prediction risk to the theory of reward learning
PREUSCHOFF, K., & BOSSAERTS, P. (2007) · Annals of the New York Academy of Sciences · doi:10.1196/annals.1390.005 · Cited by 133
A Biophysically Based Neural Model of Matching Law Behavior: Melioration by Stochastic Synapses
Soltani, A., & Wang, X. (2006) · Journal of Neuroscience · doi:10.1523/jneurosci.5159-05.2006 · Cited by 129
Self-citation
Making predictions in a changing world—inference, uncertainty, and learning
O’Reilly, J. X. (2013) · Frontiers in Neuroscience · doi:10.3389/fnins.2013.00105 · Cited by 124
A Novel Neural Prediction Error Found in Anterior Cingulate Cortex Ensembles
Hyman, J. M., Holroyd, C. B., & Seamans, J. K. (2017) · Neuron · doi:10.1016/j.neuron.2017.06.021 · Cited by 124
Deciphering Decision Making: Variation in Animal Models of Effort- and Uncertainty-Based Choice Reveals Distinct Neural Circuitries Underlying Core Cognitive Processes
Winstanley, C. A., & Floresco, S. (2016) · Journal of Neuroscience · doi:10.1523/jneurosci.1713-16.2016 · Cited by 123
Different Time Courses for Learning-Related Changes in Amygdala and Orbitofrontal Cortex
Morrison, S. E., Saez, A., Lau, B., & Salzman, C. D. (2011) · Neuron · doi:10.1016/j.neuron.2011.07.016 · Cited by 121
Model-Based Control in Dimensional Psychiatry
Voon, V., Reiter, A. M. F., Sebold, M., & Groman, S. M. (2017) · Biological Psychiatry · doi:10.1016/j.biopsych.2017.04.006 · Cited by 121
Theory of Choice in Bandit, Information Sampling and Foraging Tasks
Averbeck, B. B. (2015) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1004164 · Cited by 110
Metaplasticity as a Neural Substrate for Adaptive Learning and Choice under Uncertainty
Farashahi, S. G., Donahue, C. H., Khorsand, P., Seo, H., Lee, D., & Soltani, A. (2017) · Neuron · doi:10.1016/j.neuron.2017.03.044 · Cited by 106
Self-citation
Dynamic routing of task-relevant signals for decision making in dorsolateral prefrontal cortex
Donahue, C. H., & Lee, D. (2015) · Nature Neuroscience · doi:10.1038/nn.3918 · Cited by 104
A Mixture of Delta-Rules Approximation to Bayesian Inference in Change-Point Problems
Wilson, R. C., Nassar, M. R., & Gold, J. I. (2013) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1003150 · Cited by 103
Decoding the cognitive map: ensemble hippocampal sequences and decision making
Wikenheiser, A. M., & Redish, A. D. (2014) · Current Opinion in Neurobiology · doi:10.1016/j.conb.2014.10.002 · Cited by 92
Preferential Involvement by Nucleus Accumbens Shell in Mediating Probabilistic Learning and Reversal Shifts
Dalton, G. L., Phillips, A. G., & Floresco, S. (2014) · Journal of Neuroscience · doi:10.1523/jneurosci.5058-13.2014 · Cited by 92
Model‐based learning and the contribution of the orbitofrontal cortex to the model‐free world
McDannald, M. A., Takahashi, Y. K., Lopatina, N., Pietras, B. W., Jones, J. L., & Schoenbaum, G. M. (2012) · European Journal of Neuroscience · doi:10.1111/j.1460-9568.2011.07982.x · Cited by 88
Simultaneous representation of a spectrum of dynamically changing value estimates during decision making
Meder, D., Kolling, N., Verhagen, L., Wittmann, M. K., Scholl, J., Madsen, K. H., Hulme, O. J., Behrens, T. E. J., & Rushworth, M. F. S. (2017) · Nature Communications · doi:10.1038/s41467-017-02169-w · Cited by 86
Amygdala Contributions to Stimulus–Reward Encoding in the Macaque Medial and Orbital Frontal Cortex during Learning
Rudebeck, P. H., Ripple, J. A., Mitz, A. R., Averbeck, B. B., & Murray, E. A. (2017) · Journal of Neuroscience · doi:10.1523/jneurosci.0933-16.2017 · Cited by 85
The Role of Frontal Cortical and Medial-Temporal Lobe Brain Areas in Learning a Bayesian Prior Belief on Reversals
Jang, A. I., Costa, V. D., Rudebeck, P. H., Chudasama, Y., Murray, E. A., & Averbeck, B. B. (2015) · Journal of Neuroscience · doi:10.1523/jneurosci.1594-15.2015 · Cited by 81
Selective and graded coding of reward uncertainty by neurons in the primate anterodorsal septal region
Monosov, I. E., & Hikosaka, O. (2013) · Nature Neuroscience · doi:10.1038/nn.3398 · Cited by 81
Dissociable Performance on Scene Learning and Strategy Implementation after Lesions to Magnocellular Mediodorsal Thalamic Nucleus
Mitchell, A. S., Baxter, M. G., & Gaffan, D. (2007) · Journal of Neuroscience · doi:10.1523/jneurosci.1835-07.2007 · Cited by 77
Functional Interaction of Medial Mediodorsal Thalamic Nucleus But Not Nucleus Accumbens with Amygdala and Orbital Prefrontal Cortex Is Essential for Adaptive Response Selection after Reinforcer Devaluation
Izquierdo, A., & Murray, E. A. (2010) · Journal of Neuroscience · doi:10.1523/jneurosci.3795-09.2010 · Cited by 77
Self-citation
Critical role for the mediodorsal thalamus in permitting rapid reward-guided updating in stochastic reward environments
Chakraborty, S., Kolling, N., Walton, M. E., & Mitchell, A. S. (2016) · eLife · doi:10.7554/elife.13588 · Cited by 75
Distinct roles for the amygdala and orbitofrontal cortex in representing the relative amount of expected reward
Saez, R. A., Saez, A., Paton, J. J., Lau, B., & Salzman, C. D. (2017) · Neuron · doi:10.1016/j.neuron.2017.06.012 · Cited by 75
Scaling prediction errors to reward variability benefits error-driven learning in humans
Diederen, K. M. J., & Schultz, W. (2015) · Journal of Neurophysiology · doi:10.1152/jn.00483.2015 · Cited by 69
Reward Stability Determines the Contribution of Orbitofrontal Cortex to Adaptive Behavior
Riceberg, J. S., & Shapiro, M. L. (2012) · Journal of Neuroscience · doi:10.1523/jneurosci.0776-12.2012 · Cited by 68
Balancing new against old information: the role of puzzlement surprise in learning
Faraji, M., Preuschoff, K., & Gerstner, W. (2018) · Neural Computation · doi:10.1162/neco_a_01025 · Cited by 68
Volatility Facilitates Value Updating in the Prefrontal Cortex
Massi, B., Donahue, C. H., & Lee, D. (2018) · Neuron · doi:10.1016/j.neuron.2018.06.033 · Cited by 64
Chronic Exposure to Methamphetamine Disrupts Reinforcement-Based Decision Making in Rats
Groman, S. M., Rich, K. M., Smith, N. J., Lee, D., & Taylor, J. R. (2017) · Neuropsychopharmacology · doi:10.1038/npp.2017.159 · Cited by 62
Back to basics: Making predictions in the orbitofrontal–amygdala circuit
Sharpe, M. J., & Schoenbaum, G. M. (2016) · Neurobiology of Learning and Memory · doi:10.1016/j.nlm.2016.04.009 · Cited by 61
Orbitofrontal Cortex Signals Expected Outcomes with Predictive Codes When Stable Contingencies Promote the Integration of Reward History
Riceberg, J. S., & Shapiro, M. L. (2017) · Journal of Neuroscience · doi:10.1523/jneurosci.2951-16.2016 · Cited by 52
Computational modeling of epiphany learning
Chen, W. J., & Krajbich, I. M. (2017) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1618161114 · Cited by 52
The Hippocampus Codes the Uncertainty of Cue–Outcome Associations: An Intracranial Electrophysiological Study in Humans
Vanni-Mercier, G., Mauguı̀ere, F., Isnard, J., & Dreher, J. (2009) · Journal of Neuroscience · doi:10.1523/jneurosci.5298-08.2009 · Cited by 50
Orbitofrontal neurons signal reward predictions, not reward prediction errors
Stalnaker, T. A., Liu, T., Takahashi, Y. K., & Schoenbaum, G. M. (2018) · Neurobiology of Learning and Memory · doi:10.1016/j.nlm.2018.01.013 · Cited by 49
Orbitofrontal cortex reflects changes in response–outcome contingencies during probabilistic reversal learning
Amodeo, L. R., McMurray, M. S., & Roitman, J. D. (2016) · Neuroscience · doi:10.1016/j.neuroscience.2016.03.034 · Cited by 48
Adaptive learning and decision-making under uncertainty by metaplastic synapses guided by a surprise detection system
Iigaya, K. (2016) · eLife · doi:10.7554/elife.18073 · Cited by 45
Neurons in the primate dorsal striatum signal the uncertainty of object–reward associations
White, J. K., & Monosov, I. E. (2016) · Nature Communications · doi:10.1038/ncomms12735 · Cited by 43
Mice infer probabilistic models for timing
Li, Y., & Dudman, J. T. (2013) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1310666110 · Cited by 40
Dopamine, inference, and uncertainty
Gershman, S. J. (2017) · Neural Computation · doi:10.1162/neco_a_01023 · Cited by 40
Complementary contributions of basolateral amygdala and orbitofrontal cortex to value learning under uncertainty
Stolyarova, A., & Izquierdo, A. (2017) · eLife · doi:10.7554/elife.27483 · Cited by 39
Self-citation
Dopamine D 3 Receptor Availability Is Associated with Inflexible Decision Making
Groman, S. M., Smith, N. J., Petrullli, J. R., Massi, B., Chen, L., Ropchan, J. R., Huang, Y. H., Lee, D., Morris, E. D., & Taylor, J. R. (2016) · Journal of Neuroscience · doi:10.1523/jneurosci.3253-15.2016 · Cited by 37
From biophysics to cognition: reward-dependent adaptive choice behavior
Soltani, A., & Wang, X. (2008) · Current Opinion in Neurobiology · doi:10.1016/j.conb.2008.07.003 · Cited by 34
Self-citation
Neural Estimates of Imagined Outcomes in Basolateral Amygdala Depend on Orbitofrontal Cortex
Lucantonio, F., Gardner, M., Mirenzi, A., Newman, L. E., Takahashi, Y. K., & Schoenbaum, G. M. (2015) · Journal of Neuroscience · doi:10.1523/jneurosci.3126-15.2015 · Cited by 34
Optimal structure of metaplasticity for adaptive learning
Khorsand, P., & Soltani, A. (2017) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1005630 · Cited by 28
Self-citation
Uncertainty in action‐value estimation affects both action choice and learning rate of the choice behaviors of rats
Funamizu, A., Ito, M., Doya, K., Kanzaki, R., & Takahashi, H. (2012) · European Journal of Neuroscience · doi:10.1111/j.1460-9568.2012.08025.x · Cited by 26
Differential coding of uncertain reward in rat insular and orbitofrontal cortex
Jo, S., & Jung, M. W. (2016) · Scientific Reports · doi:10.1038/srep24085 · Cited by 23

Research works printed but not linked: 2

Dayan, P. & Long, T. in Advances in Neural Information Processing Systems 10: Proceedings of the 1997 Conference (eds Jordan, M. I., Kearns, M. J. & Solla, S. A.) 117–123 (MIT Press, 1998).
Soltani, A. C., W. & Wang, X. J. in Decision Neuroscience: An Integrative Perspective (eds Dreher, J.-C. & Tremblay, L.) 163–222 (Elsevier Academic Press, 2017).

Sorted by citations

Rethinking Concepts and Categories for Understanding the Neurodevelopmental Effects of Childhood Adversity
Smith, K. E., & Pollak, S. D. (2020) · Perspectives on Psychological Science · doi:10.1177/1745691620920725 · Cited by 379
The prefrontal cortex, pathological anxiety, and anxiety disorders
Kenwood, M. M., Kalin, N. H., & Barbas, H. (2021) · Neuropsychopharmacology · doi:10.1038/s41386-021-01109-z · Cited by 379
Medial and orbital frontal cortex in decision-making and flexible behavior
Klein-Flügge, M. C., Bongioanni, A., & Rushworth, M. F. S. (2022) · Neuron · doi:10.1016/j.neuron.2022.05.022 · Cited by 244
Using reinforcement learning models in social neuroscience: frameworks, pitfalls and suggestions of best practices
Zhang, L., Lengersdorff, L., Mikuš, N., Gläscher, J. P., & Lamm, C. (2020) · Social Cognitive and Affective Neuroscience · doi:10.1093/scan/nsaa089 · Cited by 168
Serotonin neurons modulate learning rate through uncertainty
Grossman, C. D., Bari, B. A., & Cohen, J. Y. (2022) · Current Biology · doi:10.1016/j.cub.2021.12.006 · Cited by 164

See all 286

Authors: Soltani, A., & Izquierdo, A.
Affiliations: Dartmouth College (US)
University of California, Los Angeles (US)
Corresponding author: Soltani, Alireza
Izquierdo, Alicia
ORCID: Soltani, Alireza
Izquierdo, Alicia
Venue: Nature reviews. Neuroscience
Publisher: Springer Science and Business Media LLC
Published: 2019-05-30
Volume: 20
Issue: 10
Pages: 635-644
ISSN: 1471-003X (ISSN-L)
1471-0048
DOI: 10.1038/s41583-019-0180-y
PMC: PMC6752962
OpenAlex: W2946961766
Cited by: 286
References: 118
Citations by year: 2026: 31 · 2025: 63 · 2024: 51 · 2023: 42 · 2022: 29 · 2021: 36 · 2020: 26 · 2019: 8
FWCI: 3.10
Type: journal article
Language: en
Open access: green
www.ncbi.nlm.nih.gov
Licence: www.springer.com/tdm (tdm)
www.springer.com/tdm (vor)
Locations: Nature reviews. Neuroscience (publishedVersion)
PubMed (publishedVersion)
Europe PMC (PubMed Central) (acceptedVersion)
PubMed Central (acceptedVersion)
eScholarship (California Digital Library) (submittedVersion)
Funders: National Institute on Drug Abuse (R01DA047870)
Topics: Life Sciences › Neuroscience › Cognitive Neuroscience › Neural and Behavioral Psychology Studies
Life Sciences › Neuroscience › Cognitive Neuroscience › Memory and Neural Mechanisms
Social Sciences › Decision Sciences › General Decision Sciences › Decision-Making and Behavioral Economics
Keywords: adaptive learning, uncertainty estimation, decision-making, computational model, neural mechanisms, outcome variability, reinforcement learning
MeSH terms: Adaptation, Biological, Animals, Brain, Choice Behavior, Decision Making, Humans, Learning, Nerve Net, Uncertainty

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

Adaptive learning under expected and unexpected uncertainty

Abstract

Overview

Integrity signals

Verification

Claims the paper makes

Claims other papers make about this one

Self-check

Citations

Cites

Cited by

Related

Metadata

Record

About these checks