Adaptive learning under expected and unexpected uncertainty
Open accessAbstract
The outcome of a decision is often uncertain, and outcomes can vary over repeated decisions. Whether decision outcomes should substantially affect behaviour and learning depends on whether they are representative of a typically experienced range of outcomes or signal a change in the reward environment. Successful learning and decision-making therefore require the ability to estimate expected uncertainty (related to the variability of outcomes) and unexpected uncertainty (related to the variability of the environment). Understanding the bases and effects of these two types of uncertainty and the interactions between them - at the computational and the neural level - is crucial for understanding adaptive learning. Here, we examine computational models and experimental findings to distil computational principles and neural mechanisms for adaptive learning under uncertainty.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Normatively, learning rates should vary as a function of uncertainty.
“Normatively, learning rates should vary as a function of uncertainty.7–9”
From Serotonin neurons modulate learning rate through uncertainty (2022) · cites this paper as [9]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Reinforcement Learning: An Introduction
Sutton, R. S., & Barto, A. G. (1998) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.1998.712192 · Cited by 27,225 · see citing works
Reinforcement Learning: An Introduction
Sutton, R. S., & Barto, A. (2005) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.2004.842673 · Cited by 25,263 · see citing works
A neural substrate of prediction and reward
Schultz, W., Dayan, P., & Montague, P. R. (1997) · Science · doi:10.1126/science.275.5306.1593 · Cited by 9,388 · see citing works
What is the role of dopamine in reward: hedonic impact, reward learning, or incentive salience?
Berridge, K. C., & Robinson, T. E. (1998) · Brain Research Reviews · doi:10.1016/s0165-0173(98)00019-8 · Cited by 4,127 · see citing works
Pearce, J. M., & Hall, G. (1980) · Psychological Review · doi:10.1037/0033-295x.87.6.532 · Cited by 3,023 · see citing works
The expected value of control: an integrative theory of anterior cingulate cortex function
Shenhav, A., Botvinick, M. M., & Cohen, J. D. (2013) · Neuron · doi:10.1016/j.neuron.2013.07.007 · Cited by 2,321 · see citing works
Cortical substrates for exploratory decisions in humans
Daw, N. D., O'Doherty, J. P., Dayan, P., Seymour, B., & Dolan, R. J. (2006) · Nature · doi:10.1038/nature04766 · Cited by 2,224 · see citing works
Learning the value of information in an uncertain world
Behrens, T. E. J., Woolrich, M. W., Walton, M. E., & Rushworth, M. F. S. (2007) · Nature Neuroscience · doi:10.1038/nn1954 · Cited by 2,053 · see citing works
Uncertainty and anticipation in anxiety: an integrated neurobiological and psychological perspective
Grupe, D. W., & Nitschke, J. B. (2013) · Nature Reviews Neuroscience · doi:10.1038/nrn3524 · Cited by 2,007 · see citing works
Uncertainty, Neuromodulation, and Attention
Yu, A. J., & Dayan, P. (2005) · Neuron · doi:10.1016/j.neuron.2005.04.026 · Cited by 1,774 · see citing works
Primate anterior cingulate cortex: Where motor control, drive and cognition interface
Paus, T. (2001) · Nature Reviews Neuroscience · doi:10.1038/35077500 · Cited by 1,722 · see citing works
Amygdalo‐cortical projections in the monkey ( <i>Macaca fascicularis</i> )
Amaral, D. G., & Price, J. L. (1984) · Journal of Comparative Neurology · doi:10.1002/cne.902300402 · Cited by 1,162 · see citing works
Metaplasticity: tuning synapses and networks for plasticity
Abraham, W. C. (2008) · Nature reviews. Neuroscience · doi:10.1038/nrn2356 · Cited by 1,119 · see citing works
Frontal cortex and reward-guided learning and decision-making
Rushworth, M. F. S., Noonan, M. P., Boorman, E. D., Walton, M. E., & Behrens, T. E. (2011) · Neuron · doi:10.1016/j.neuron.2011.05.014 · Cited by 1,111 · see citing works
Optimal decision making and the anterior cingulate cortex
Kennerley, S. W., Walton, M. E., Behrens, T. E. J., Buckley, M. J., & Rushworth, M. F. S. (2006) · Nature Neuroscience · doi:10.1038/nn1724 · Cited by 948 · see citing works
Choice, uncertainty and value in prefrontal and cingulate cortex
Rushworth, M. F. S., & Behrens, T. E. J. (2008) · Nature Neuroscience · doi:10.1038/nn2066 · Cited by 866 · see citing works
The short-latency dopamine signal: a role in discovering novel actions?
Redgrave, P., & Gurney, K. (2006) · Nature Reviews Neuroscience · doi:10.1038/nrn2022 · Cited by 762 · see citing works
The neural basis of reversal learning: An updated perspective
Izquierdo, A., Brigman, J. L., Radke, A. K., Rudebeck, P. H., & Holmes, A. (2017) · Neuroscience · doi:10.1016/j.neuroscience.2016.03.021 · Cited by 670 · see citing works
Self-citationA Bayesian foundation for individual learning under uncertainty
Mathys, C. (2011) · Frontiers in Human Neuroscience · doi:10.3389/fnhum.2011.00039 · Cited by 655 · see citing works
Doya, K. (2008) · Nature Neuroscience · doi:10.1038/nn2077 · Cited by 642 · see citing works
Nassar, M. R., Wilson, R. C., Heasly, B. S., & Gold, J. I. (2010) · Journal of Neuroscience · doi:10.1523/jneurosci.0822-10.2010 · Cited by 564 · see citing works
What the orbitofrontal cortex does not do
Stalnaker, T. A., Cooch, N. K., & Schoenbaum, G. (2015) · Nature Neuroscience · doi:10.1038/nn.3982 · Cited by 531 · see citing works
Learning and selective attention
Dayan, P., Kakade, S., & Montague, P. R. (2000) · Nature Neuroscience · doi:10.1038/81504 · Cited by 495 · see citing works
Reward Value Coding Distinct From Risk Attitude-Related Uncertainty Coding in Human Reward Systems
Tobler, P. N., O'Doherty, J. P., Dolan, R. J., & Schultz, W. (2007) · Journal of Neurophysiology · doi:10.1152/jn.00745.2006 · Cited by 472 · see citing works
Dorsal Anterior Cingulate Cortex: A Bottom-Up View
Heilbronner, S. R., & Hayden, B. Y. (2016) · Annual Review of Neuroscience · doi:10.1146/annurev-neuro-070815-013952 · Cited by 469 · see citing works
Bayesian theories of conditioning in a changing world
Courville, A. C., Daw, N. D., & Touretzky, D. S. (2006) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2006.05.004 · Cited by 457 · see citing works
Beyond the reward hypothesis: alternative functions of nucleus accumbens dopamine
Salamone, J. D., Correa, M., Mingote, S., & Weber, S. (2004) · Current Opinion in Pharmacology · doi:10.1016/j.coph.2004.09.004 · Cited by 454 · see citing works
S., M., S., B., M.-Y., H., C., B., E., S., J., D., & I., A. (2002) · Psychopharmacology · doi:10.1007/s00213-001-0983-0 · Cited by 409 · see citing works
Walton, M. E., Behrens, T. E. J., Buckley, M. J., Rudebeck, P. H., & Rushworth, M. F. S. (2010) · Neuron · doi:10.1016/j.neuron.2010.02.027 · Cited by 397 · see citing works
Hayden, B. Y., Heilbronner, S. R., Pearson, J. M., & Platt, M. L. (2011) · The Journal of Neuroscience · doi:10.1523/jneurosci.4652-10.2011 · Cited by 379 · see citing works
Distinct Roles of Rodent Orbitofrontal and Medial Prefrontal Cortex in Decision Making
Sul, J. H., Kim, H., Huh, N., Lee, D., & Jung, M. W. (2010) · Neuron · doi:10.1016/j.neuron.2010.03.033 · Cited by 372 · see citing works
Decoding subjective decisions from orbitofrontal cortex
Rich, E. L., & Wallis, J. D. (2016) · Nature Neuroscience · doi:10.1038/nn.4320 · Cited by 360 · see citing works
Cross-species studies of orbitofrontal cortex and value-based decision-making
Wallis, J. D. (2011) · Nature Neuroscience · doi:10.1038/nn.2956 · Cited by 356 · see citing works
Over the river, through the woods: cognitive maps in the hippocampus and orbitofrontal cortex
Wikenheiser, A. M., & Schoenbaum, G. M. (2016) · Nature reviews. Neuroscience · doi:10.1038/nrn.2016.56 · Cited by 353 · see citing works
Cytoarchitecture of mouse and rat cingulate cortex with human homologies
Vogt, B. A., & Paxinos, G. (2014) · Brain Structure and Function · doi:10.1007/s00429-012-0493-3 · Cited by 348 · see citing works
Separate value comparison and learning mechanisms in macaque medial and lateral orbitofrontal cortex
Noonan, M. P., Walton, M. E., Behrens, T. E. J., Sallet, J., Buckley, M. J., & Rushworth, M. F. S. (2010) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1012246107 · Cited by 340 · see citing works
The Cognitive Thalamus as a Gateway to Mental Representations
Wolff, M., & Vann, S. D. (2018) · Journal of Neuroscience · doi:10.1523/jneurosci.0479-18.2018 · Cited by 340 · see citing works
Knowing how much you don't know: a neural organization of uncertainty estimates
Bach, D. R., & Dolan, R. J. (2012) · Nature Reviews Neuroscience · doi:10.1038/nrn3289 · Cited by 339 · see citing works
Seo, H., & Lee, D. (2007) · The Journal of Neuroscience · doi:10.1523/jneurosci.2369-07.2007 · Cited by 324 · see citing works
An Unexpected Sequence of Events: Mismatch Detection in the Human Hippocampus
Kumaran, D., & Maguire, E. A. (2006) · PLoS Biology · doi:10.1371/journal.pbio.0040424 · Cited by 324 · see citing works
Circuit-Based Corticostriatal Homologies Between Rat and Primate
Heilbronner, S. R., Rodríguez-Romaguera, J., Quirk, G. J., Groenewegen, H. J., & Haber, S. N. (2016) · Biological Psychiatry · doi:10.1016/j.biopsych.2016.05.012 · Cited by 321 · see citing works
Neural systems implicated in delayed and probabilistic reinforcement
Cardinal, R. N. (2006) · Neural Networks · doi:10.1016/j.neunet.2006.03.004 · Cited by 318 · see citing works
The basolateral amygdala in reward learning and addiction
Wassum, K. M., & Izquierdo, A. (2015) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2015.08.017 · Cited by 306 · see citing works
Self-citationFunctionally dissociable influences on learning rate in a dynamic environment
McGuire, J. T., Nassar, M. R., Gold, J. I., & Kable, J. W. (2014) · Neuron · doi:10.1016/j.neuron.2014.10.013 · Cited by 299 · see citing works
Risk, unexpected uncertainty, and estimation uncertainty: Bayesian learning in unstable settings
Payzan-LeNestour, E., & Bossaerts, P. (2011) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1001048 · Cited by 267 · see citing works
Fundamental contribution by the basolateral amygdala to different forms of decision making
Ghods-Sharifi, S., St. Onge, J. R., & Floresco, S. B. (2009) · The Journal of Neuroscience · doi:10.1523/jneurosci.0315-09.2009 · Cited by 257 · see citing works
The neural representation of unexpected uncertainty during value-based decision making
Payzan-LeNestour, E., Dunne, S., Bossaerts, P., & O’Doherty, J. P. (2013) · Neuron · doi:10.1016/j.neuron.2013.04.037 · Cited by 257 · see citing works
Projections of the medial orbital and ventral orbital cortex in the rat
Hoover, W. B., & Vertes, R. P. (2011) · The Journal of Comparative Neurology · doi:10.1002/cne.22733 · Cited by 251 · see citing works
Functional Heterogeneity within Rat Orbitofrontal Cortex in Reward Learning and Decision Making
Izquierdo, A. (2017) · Journal of Neuroscience · doi:10.1523/jneurosci.1678-17.2017 · Cited by 251 · see citing works
Self-citationCoding of Reward Risk by Orbitofrontal Neurons Is Mostly Distinct from Coding of Reward Value
O’Neill, M. J., & Schultz, W. (2010) · Neuron · doi:10.1016/j.neuron.2010.09.031 · Cited by 249 · see citing works
Rogers, R. D. (2010) · Neuropsychopharmacology · doi:10.1038/npp.2010.165 · Cited by 248 · see citing works
Anterior cingulate error‐related activity is modulated by predicted reward
Amiez, C., Joseph, J., & Procyk, E. (2005) · European Journal of Neuroscience · doi:10.1111/j.1460-9568.2005.04170.x · Cited by 236 · see citing works
Cardinal, R. N., & Howes, N. J. (2005) · BMC Neuroscience · doi:10.1186/1471-2202-6-37 · Cited by 231 · see citing works
Krugel, L. K., Biele, G., Mohr, P. N. C., Li, S.-C., & Heekeren, H. R. (2009) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.0905191106 · Cited by 229 · see citing works
Triple dissociation of attention and decision computations across prefrontal cortex
Hunt, L. T., Malalasekera, W. M. N., de Berker, A. O., Miranda, B., Farmer, S. F., Behrens, T. E. J., & Kennerley, S. W. (2018) · Nature Neuroscience · doi:10.1038/s41593-018-0239-5 · Cited by 222 · see citing works
Medial Orbitofrontal Cortex Mediates Outcome Retrieval in Partially Observable Task Situations
Bradfield, L. A., Dezfouli, A., van Holstein, M., Chieng, B. C. H., & Balleine, B. W. (2015) · Neuron · doi:10.1016/j.neuron.2015.10.044 · Cited by 206 · see citing works
Surprise! Neural correlates of Pearce–Hall and Rescorla–Wagner coexist within the brain
Roesch, M. R., Esber, G. R., Li, J., Daw, N. D., & Schoenbaum, G. M. (2012) · European Journal of Neuroscience · doi:10.1111/j.1460-9568.2011.07986.x · Cited by 186 · see citing works
Dalton, G. L., Wang, N. Y., Phillips, A. G., & Floresco, S. B. (2016) · The Journal of Neuroscience · doi:10.1523/jneurosci.3366-15.2016 · Cited by 184 · see citing works
Abstract context representations in primate amygdala and prefrontal cortex
Saez, A., Rigotti, M., Ostojic, S., Fusi, S., & Salzman, C. D. (2015) · Neuron · doi:10.1016/j.neuron.2015.07.024 · Cited by 183 · see citing works
Motivational neural circuits underlying reinforcement learning
Averbeck, B. B., & Costa, V. D. (2017) · Nature Neuroscience · doi:10.1038/nn.4506 · Cited by 175 · see citing works
Reversal learning and dopamine: a Bayesian perspective
Costa, V. D., Tran, V. L., Turchi, J., & Averbeck, B. B. (2015) · The Journal of Neuroscience · doi:10.1523/jneurosci.1989-14.2015 · Cited by 170 · see citing works
Rudebeck, P. H., Saunders, R. C., Lundgren, D. A., & Murray, E. A. (2017) · Neuron · doi:10.1016/j.neuron.2017.07.042 · Cited by 170 · see citing works
Advances in Understanding Mechanisms of Thalamic Relays in Cognition and Behavior
Mitchell, A. S., Sherman, S. M., Sommer, M. A., Mair, R. G., Vertes, R. P., & Chudasama, Y. (2014) · Journal of Neuroscience · doi:10.1523/jneurosci.3289-14.2014 · Cited by 164 · see citing works
Separate Prefrontal-Subcortical Circuits Mediate Different Components of Risk-Based Decision Making
Onge, J. R. S., Stopper, C. M., Zahm, D. S., & Floresco, S. (2012) · Journal of Neuroscience · doi:10.1523/jneurosci.5625-11.2012 · Cited by 164 · see citing works
Anterior cingulate is a source of valence-specific information about value and uncertainty
Monosov, I. E. (2017) · Nature Communications · doi:10.1038/s41467-017-00072-y · Cited by 163 · see citing works
Topography of projections from the medial prefrontal cortex to the amygdala in the rat.
Cassell, M., & Wright, D. J. (1986) · Brain Research Bulletin · doi:10.1016/0361-9230(86)90237-6 · Cited by 161 · see citing works
Model-based predictions for dopamine
Langdon, A. J., Sharpe, M. J., Schoenbaum, G., & Niv, Y. (2018) · Current Opinion in Neurobiology · doi:10.1016/j.conb.2017.10.006 · Cited by 161 · see citing works
Mediodorsal Thalamus Hypofunction Impairs Flexible Goal-Directed Behavior
Parnaudeau, S., Taylor, K., Bolkan, S., Ward, R. D., Balsam, P., & Kellendonk, C. (2014) · Biological Psychiatry · doi:10.1016/j.biopsych.2014.03.020 · Cited by 156 · see citing works
Dopamine, uncertainty and TD learning
Niv, Y., Duff, M. O., & Dayan, P. (2005) · Behavioral and Brain Functions · doi:10.1186/1744-9081-1-6 · Cited by 153 · see citing works
Neural Correlates of Variations in Event Processing during Learning in Basolateral Amygdala
Roesch, M. R., Calu, D. J., Esber, G. R., & Schoenbaum, G. M. (2010) · Journal of Neuroscience · doi:10.1523/jneurosci.5781-09.2010 · Cited by 150 · see citing works
Ünal, G., Joshi, A., Viney, T. J., Kis, V., & Somogyi, P. (2015) · Journal of Neuroscience · doi:10.1523/jneurosci.2639-15.2015 · Cited by 149 · see citing works
Compulsivity reveals a novel dissociation between action and confidence
Vaghi, M. M., Luyckx, F., Sule, A., Fineberg, N. A., Robbins, T. W., & De Martino, B. (2017) · Neuron · doi:10.1016/j.neuron.2017.09.006 · Cited by 145 · see citing works
Overriding Phasic Dopamine Signals Redirects Action Selection during Risk/Reward Decision Making
Stopper, C. M., Tse, M. T. L., Montes, D. R., Wiedman, C. R., & Floresco, S. B. (2014) · Neuron · doi:10.1016/j.neuron.2014.08.033 · Cited by 136 · see citing works
Adding prediction risk to the theory of reward learning
PREUSCHOFF, K., & BOSSAERTS, P. (2007) · Annals of the New York Academy of Sciences · doi:10.1196/annals.1390.005 · Cited by 133 · see citing works
Amygdala and Ventral Striatum Make Distinct Contributions to Reinforcement Learning
Costa, V. D., Monte, O. D., Lucas, D. R., Murray, E. A., & Averbeck, B. B. (2016) · Neuron · doi:10.1016/j.neuron.2016.09.025 · Cited by 133 · see citing works
A Biophysically Based Neural Model of Matching Law Behavior: Melioration by Stochastic Synapses
Soltani, A., & Wang, X. (2006) · Journal of Neuroscience · doi:10.1523/jneurosci.5159-05.2006 · Cited by 127 · see citing works
Self-citationMaking predictions in a changing world—inference, uncertainty, and learning
O’Reilly, J. X. (2013) · Frontiers in Neuroscience · doi:10.3389/fnins.2013.00105 · Cited by 126 · see citing works
A Novel Neural Prediction Error Found in Anterior Cingulate Cortex Ensembles
Hyman, J. M., Holroyd, C. B., & Seamans, J. K. (2017) · Neuron · doi:10.1016/j.neuron.2017.06.021 · Cited by 123 · see citing works
Different Time Courses for Learning-Related Changes in Amygdala and Orbitofrontal Cortex
Morrison, S. E., Saez, A., Lau, B., & Salzman, C. D. (2011) · Neuron · doi:10.1016/j.neuron.2011.07.016 · Cited by 121 · see citing works
Winstanley, C. A., & Floresco, S. (2016) · Journal of Neuroscience · doi:10.1523/jneurosci.1713-16.2016 · Cited by 117 · see citing works
Model-Based Control in Dimensional Psychiatry
Voon, V., Reiter, A. M. F., Sebold, M., & Groman, S. M. (2017) · Biological Psychiatry · doi:10.1016/j.biopsych.2017.04.006 · Cited by 112 · see citing works
Theory of Choice in Bandit, Information Sampling and Foraging Tasks
Averbeck, B. B. (2015) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1004164 · Cited by 105 · see citing works
A Mixture of Delta-Rules Approximation to Bayesian Inference in Change-Point Problems
Wilson, R. C., Nassar, M. R., & Gold, J. I. (2013) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1003150 · Cited by 103 · see citing works
Dynamic routing of task-relevant signals for decision making in dorsolateral prefrontal cortex
Donahue, C. H., & Lee, D. (2015) · Nature Neuroscience · doi:10.1038/nn.3918 · Cited by 100 · see citing works
Metaplasticity as a Neural Substrate for Adaptive Learning and Choice under Uncertainty
Farashahi, S. G., Donahue, C. H., Khorsand, P., Seo, H., Lee, D., & Soltani, A. (2017) · Neuron · doi:10.1016/j.neuron.2017.03.044 · Cited by 98 · see citing works
Self-citationDalton, G. L., Phillips, A. G., & Floresco, S. (2014) · Journal of Neuroscience · doi:10.1523/jneurosci.5058-13.2014 · Cited by 92 · see citing works
Model‐based learning and the contribution of the orbitofrontal cortex to the model‐free world
McDannald, M. A., Takahashi, Y. K., Lopatina, N., Pietras, B. W., Jones, J. L., & Schoenbaum, G. M. (2012) · European Journal of Neuroscience · doi:10.1111/j.1460-9568.2011.07982.x · Cited by 84 · see citing works
Meder, D., Kolling, N., Verhagen, L., Wittmann, M. K., Scholl, J., Madsen, K. H., Hulme, O. J., Behrens, T. E. J., & Rushworth, M. F. S. (2017) · Nature Communications · doi:10.1038/s41467-017-02169-w · Cited by 84 · see citing works
Rudebeck, P. H., Ripple, J. A., Mitz, A. R., Averbeck, B. B., & Murray, E. A. (2017) · Journal of Neuroscience · doi:10.1523/jneurosci.0933-16.2017 · Cited by 83 · see citing works
Decoding the cognitive map: ensemble hippocampal sequences and decision making
Wikenheiser, A. M., & Redish, A. D. (2014) · Current Opinion in Neurobiology · doi:10.1016/j.conb.2014.10.002 · Cited by 82 · see citing works
Jang, A. I., Costa, V. D., Rudebeck, P. H., Chudasama, Y., Murray, E. A., & Averbeck, B. B. (2015) · Journal of Neuroscience · doi:10.1523/jneurosci.1594-15.2015 · Cited by 77 · see citing works
Mitchell, A. S., Baxter, M. G., & Gaffan, D. (2007) · Journal of Neuroscience · doi:10.1523/jneurosci.1835-07.2007 · Cited by 77 · see citing works
Izquierdo, A., & Murray, E. A. (2010) · Journal of Neuroscience · doi:10.1523/jneurosci.3795-09.2010 · Cited by 77 · see citing works
Self-citationMonosov, I. E., & Hikosaka, O. (2013) · Nature Neuroscience · doi:10.1038/nn.3398 · Cited by 77 · see citing works
Chakraborty, S., Kolling, N., Walton, M. E., & Mitchell, A. S. (2016) · eLife · doi:10.7554/elife.13588 · Cited by 75 · see citing works
Saez, R. A., Saez, A., Paton, J. J., Lau, B., & Salzman, C. D. (2017) · Neuron · doi:10.1016/j.neuron.2017.06.012 · Cited by 75 · see citing works
Scaling prediction errors to reward variability benefits error-driven learning in humans
Diederen, K. M. J., & Schultz, W. (2015) · Journal of Neurophysiology · doi:10.1152/jn.00483.2015 · Cited by 69 · see citing works
Reward Stability Determines the Contribution of Orbitofrontal Cortex to Adaptive Behavior
Riceberg, J. S., & Shapiro, M. L. (2012) · Journal of Neuroscience · doi:10.1523/jneurosci.0776-12.2012 · Cited by 68 · see citing works
Balancing new against old information: the role of puzzlement surprise in learning
Faraji, M., Preuschoff, K., & Gerstner, W. (2018) · Neural Computation · doi:10.1162/neco_a_01025 · Cited by 68 · see citing works
Back to basics: Making predictions in the orbitofrontal–amygdala circuit
Sharpe, M. J., & Schoenbaum, G. M. (2016) · Neurobiology of Learning and Memory · doi:10.1016/j.nlm.2016.04.009 · Cited by 61 · see citing works
Volatility Facilitates Value Updating in the Prefrontal Cortex
Massi, B., Donahue, C. H., & Lee, D. (2018) · Neuron · doi:10.1016/j.neuron.2018.06.033 · Cited by 61 · see citing works
Chronic Exposure to Methamphetamine Disrupts Reinforcement-Based Decision Making in Rats
Groman, S. M., Rich, K. M., Smith, N. J., Lee, D., & Taylor, J. R. (2017) · Neuropsychopharmacology · doi:10.1038/npp.2017.159 · Cited by 59 · see citing works
Riceberg, J. S., & Shapiro, M. L. (2017) · Journal of Neuroscience · doi:10.1523/jneurosci.2951-16.2016 · Cited by 52 · see citing works
Computational modeling of epiphany learning
Chen, W. J., & Krajbich, I. M. (2017) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1618161114 · Cited by 52 · see citing works
Vanni-Mercier, G., Mauguı̀ere, F., Isnard, J., & Dreher, J. (2009) · Journal of Neuroscience · doi:10.1523/jneurosci.5298-08.2009 · Cited by 50 · see citing works
Orbitofrontal neurons signal reward predictions, not reward prediction errors
Stalnaker, T. A., Liu, T., Takahashi, Y. K., & Schoenbaum, G. M. (2018) · Neurobiology of Learning and Memory · doi:10.1016/j.nlm.2018.01.013 · Cited by 49 · see citing works
Amodeo, L. R., McMurray, M. S., & Roitman, J. D. (2016) · Neuroscience · doi:10.1016/j.neuroscience.2016.03.034 · Cited by 46 · see citing works
Iigaya, K. (2016) · eLife · doi:10.7554/elife.18073 · Cited by 43 · see citing works
Neurons in the primate dorsal striatum signal the uncertainty of object–reward associations
White, J. K., & Monosov, I. E. (2016) · Nature Communications · doi:10.1038/ncomms12735 · Cited by 43 · see citing works
Dopamine, inference, and uncertainty
Gershman, S. J. (2017) · Neural Computation · doi:10.1162/neco_a_01023 · Cited by 40 · see citing works
Stolyarova, A., & Izquierdo, A. (2017) · eLife · doi:10.7554/elife.27483 · Cited by 39 · see citing works
Self-citationDopamine D 3 Receptor Availability Is Associated with Inflexible Decision Making
Groman, S. M., Smith, N. J., Petrullli, J. R., Massi, B., Chen, L., Ropchan, J. R., Huang, Y. H., Lee, D., Morris, E. D., & Taylor, J. R. (2016) · Journal of Neuroscience · doi:10.1523/jneurosci.3253-15.2016 · Cited by 37 · see citing works
Mice infer probabilistic models for timing
Li, Y., & Dudman, J. T. (2013) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1310666110 · Cited by 36 · see citing works
From biophysics to cognition: reward-dependent adaptive choice behavior
Soltani, A., & Wang, X. (2008) · Current Opinion in Neurobiology · doi:10.1016/j.conb.2008.07.003 · Cited by 34 · see citing works
Self-citationNeural Estimates of Imagined Outcomes in Basolateral Amygdala Depend on Orbitofrontal Cortex
Lucantonio, F., Gardner, M., Mirenzi, A., Newman, L. E., Takahashi, Y. K., & Schoenbaum, G. M. (2015) · Journal of Neuroscience · doi:10.1523/jneurosci.3126-15.2015 · Cited by 34 · see citing works
Optimal structure of metaplasticity for adaptive learning
Khorsand, P., & Soltani, A. (2017) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1005630 · Cited by 26 · see citing works
Self-citationFunamizu, A., Ito, M., Doya, K., Kanzaki, R., & Takahashi, H. (2012) · European Journal of Neuroscience · doi:10.1111/j.1460-9568.2012.08025.x · Cited by 24 · see citing works
Differential coding of uncertain reward in rat insular and orbitofrontal cortex
Jo, S., & Jung, M. W. (2016) · Scientific Reports · doi:10.1038/srep24085 · Cited by 21 · see citing works
Flagged first, then by citations
Research works printed but not linked: 2
- [30]Dayan PLT Statistical models of conditioning. Adv Neural Inf Process Syst, 1998: p. 117– 123.
- [32]Soltani ACW; Wang XJ, Neural circuit mechanisms of value-based decision-making and reinforcement learning, in Decision Neuroscience: An Integrative Perspective, Dreher JCT, L, Editor 2017, Elsevier Academic Press: London, U.K p. 163– 222.
Cited by
Sorted by citations
Smith, K. E., & Pollak, S. D. (2020) · Perspectives on Psychological Science · doi:10.1177/1745691620920725 · Cited by 351 · see citing works
The prefrontal cortex, pathological anxiety, and anxiety disorders
Kenwood, M. M., Kalin, N. H., & Barbas, H. (2021) · Neuropsychopharmacology · doi:10.1038/s41386-021-01109-z · Cited by 339 · see citing works
Medial and orbital frontal cortex in decision-making and flexible behavior
Klein-Flügge, M. C., Bongioanni, A., & Rushworth, M. F. S. (2022) · Neuron · doi:10.1016/j.neuron.2022.05.022 · Cited by 244 · see citing works
Zhang, L., Lengersdorff, L., Mikuš, N., Gläscher, J. P., & Lamm, C. (2020) · Social Cognitive and Affective Neuroscience · doi:10.1093/scan/nsaa089 · Cited by 158 · see citing works
Serotonin neurons modulate learning rate through uncertainty
Grossman, C. D., Bari, B. A., & Cohen, J. Y. (2022) · Current Biology · doi:10.1016/j.cub.2021.12.006 · Cited by 164 · see citing works
A brain network supporting social influences in human decision-making
Zhang, L., & Gläscher, J. P. (2020) · Science Advances · doi:10.1126/sciadv.abb4159 · Cited by 154 · see citing works
A model for learning based on the joint estimation of stochasticity and volatility
Piray, P., & Daw, N. D. (2021) · Nature Communications · doi:10.1038/s41467-021-26731-9 · Cited by 153 · see citing works
Paranoia and belief updating during the COVID-19 crisis
Suthaharan, P., Reed, E. J., Leptourgos, P., Kenney, J. G., Uddenberg, S., Mathys, C. D., Litman, L., Robinson, J., Moss, A. J., Taylor, J. R., et al. (2021) · Nature Human Behaviour · doi:10.1038/s41562-021-01176-8 · Cited by 125 · see citing works
Anterior Cingulate Cortex and the Control of Dynamic Behavior in Primates
Monosov, I. E., Haber, S. N., Leuthardt, E. C., & Jezzini, A. (2020) · Current Biology · doi:10.1016/j.cub.2020.10.009 · Cited by 118 · see citing works
Flexible combination of reward information across primates
Farashahi, S., Donahue, C. H., Hayden, B. Y., Lee, D., & Soltani, A. (2019) · Nature Human Behaviour · doi:10.1038/s41562-019-0714-3 · Cited by 112 · see citing works
Self-citationEye pupil signals information gain
Zénon, A. (2019) · Proceedings of the Royal Society B Biological Sciences · doi:10.1098/rspb.2019.1593 · Cited by 108 · see citing works
Thalamocortical excitability modulation guides human perception under uncertainty
Kosciessa, J. Q., Lindenberger, U., & Garrett, D. D. (2021) · Nature Communications · doi:10.1038/s41467-021-22511-7 · Cited by 101 · see citing works
A simple model for learning in volatile environments
Piray, P., & Daw, N. D. (2020) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1007963 · Cited by 104 · see citing works
Stolyarova, A., Rakhshan, M., Hart, E. E., O’Dell, T. J., Peters, M. A. K., Lau, H. C., Soltani, A., & Izquierdo, A. (2019) · Nature Communications · doi:10.1038/s41467-019-12725-1 · Cited by 90 · see citing works
Self-citationLevy, I., & Schiller, D. (2021) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2020.11.007 · Cited by 95 · see citing works
How Outcome Uncertainty Mediates Attention, Learning, and Decision-Making
Monosov, I. E. (2020) · Trends in Neurosciences · doi:10.1016/j.tins.2020.06.009 · Cited by 87 · see citing works
Predictive Models for Educational Purposes: A Systematic Review
Almalawi, A., Soh, B., Li, A., & Samra, H. E. (2024) · Big Data and Cognitive Computing · doi:10.3390/bdcc8120187 · Cited by 91 · see citing works
Uncertainty Makes Me Emotional: Uncertainty as an Elicitor and Modulator of Emotional States
Morriss, J., Tupitsa, E., Dodd, H. F., & Hirsch, C. R. (2022) · Frontiers in Psychology · doi:10.3389/fpsyg.2022.777025 · Cited by 89 · see citing works
Computational models of adaptive behavior and prefrontal cortex
Soltani, A., & Koechlin, É. (2021) · Neuropsychopharmacology · doi:10.1038/s41386-021-01123-1 · Cited by 85 · see citing works
Self-citationAI-Enabled UAV Communications: Challenges and Future Directions
Hashesh, A. O., Hashima, S., Zaki, R. M., Fouda, M. M., Hatano, K., & Eldien, A. S. T. (2022) · IEEE Access · doi:10.1109/access.2022.3202956 · Cited by 97 · see citing works
Divergent Strategies for Learning in Males and Females
Chen, C. S., Ebitz, R. B., Bindas, S. R., Redish, A. D., Hayden, B. Y., & Grissom, N. M. (2020) · Current Biology · doi:10.1016/j.cub.2020.09.075 · Cited by 84 · see citing works
Distributional Reinforcement Learning in the Brain
Lowet, A. S., Qiao, Z., Matias, S., Drugowitsch, J., & Uchida, N. (2020) · Trends in Neurosciences · doi:10.1016/j.tins.2020.09.004 · Cited by 69 · see citing works
Timescales of cognition in the brain
Soltani, A., Murray, J. D., Seo, H., & Lee, D. (2021) · Current Opinion in Behavioral Sciences · doi:10.1016/j.cobeha.2021.03.003 · Cited by 60 · see citing works
Self-citationA Decision Architecture for Safety Computations
Tashjian, S. M., Zbozinek, T. D., & Mobbs, D. (2021) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2021.01.013 · Cited by 60 · see citing works
A taxonomy of surprise definitions
Modirshanechi, A., Brea, J., & Gerstner, W. (2022) · Journal of Mathematical Psychology · doi:10.1016/j.jmp.2022.102712 · Cited by 60 · see citing works
Olivo, G., Gaudio, S., & Schiöth, H. B. (2019) · Nutrients · doi:10.3390/nu11081907 · Cited by 56 · see citing works
Uncertainty-driven regulation of learning and exploration in adolescents: A computational account
Jepma, M., Schaaf, J. V., Visser, I., & Huizenga, H. M. (2020) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1008276 · Cited by 48 · see citing works
Transdiagnostic computations of uncertainty: towards a new lens on intolerance of uncertainty
Sandhu, T. R., Xiao, B., & Lawson, R. P. (2023) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2023.105123 · Cited by 51 · see citing works
Monosov, I. E., & Rushworth, M. F. S. (2021) · Neuropsychopharmacology · doi:10.1038/s41386-021-01079-2 · Cited by 48 · see citing works
Ji, Y., Wang, L., Long, Y., Wang, J., Zheng, H., Yu, Z. G., Zhang, Y., & Ang, K. (2025) · Nature Communications · doi:10.1038/s41467-025-62306-8 · Cited by 51 · see citing works
Aging, uncertainty, and decision making—A review
Frank, C. C., & Seaman, K. L. (2023) · Cognitive Affective & Behavioral Neuroscience · doi:10.3758/s13415-023-01064-w · Cited by 48 · see citing works
Foraging with the frontal cortex: A cross-species evaluation of reward-guided behavior
Rudebeck, P. H., & Izquierdo, A. (2021) · Neuropsychopharmacology · doi:10.1038/s41386-021-01140-0 · Cited by 48 · see citing works
Self-citationComputational processes of simultaneous learning of stochasticity and volatility in humans
Piray, P., & Daw, N. D. (2024) · Nature Communications · doi:10.1038/s41467-024-53459-z · Cited by 45 · see citing works
An empirical evaluation of active inference in multi-armed bandits
Marković, D., Stojić, H., Schwöbel, S., & Kiebel, S. J. (2021) · Neural Networks · doi:10.1016/j.neunet.2021.08.018 · Cited by 44 · see citing works
Goal-directed learning in adolescence: neurocognitive development and contextual influences
Wilbrecht, L., & Davidow, J. Y. (2024) · Nature reviews. Neuroscience · doi:10.1038/s41583-023-00783-w · Cited by 44 · see citing works
Brown, V. M., Price, R. B., & Dombrovski, A. Y. (2023) · Cognitive Affective & Behavioral Neuroscience · doi:10.3758/s13415-023-01080-w · Cited by 40 · see citing works
Ugarte, E., & Hastings, P. D. (2023) · Development and Psychopathology · doi:10.1017/s0954579423000305 · Cited by 42 · see citing works
Novelty is not surprise: Human exploratory and adaptive behavior in sequential decision-making
He, X., Modirshanechi, A., Lehmann, M. P., Wulfram Gerstner, B., & Herzog, M. H. (2021) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1009070 · Cited by 45 · see citing works
Suthaharan, P., & Corlett, P. R. (2023) · Scientific Reports · doi:10.1038/s41598-023-33305-w · Cited by 38 · see citing works
Hein, T. P., & Ruiz, M. H. (2022) · NeuroImage · doi:10.1016/j.neuroimage.2022.118895 · Cited by 37 · see citing works
Liu, S., & Fisher, P. A. (2022) · Developmental Cognitive Neuroscience · doi:10.1016/j.dcn.2022.101091 · Cited by 35 · see citing works
Adaptive Learning through Temporal Dynamics of State Representation
Razmi, N., & Nassar, M. R. (2022) · Journal of Neuroscience · doi:10.1523/jneurosci.0387-21.2022 · Cited by 32 · see citing works
Probabilistic Reinforcement Learning and Anhedonia
Kangas, B. D., Der‐Avakian, A., & Pizzagalli, D. A. (2022) · Current topics in behavioral neurosciences · doi:10.1007/7854_2022_349 · Cited by 34 · see citing works
Smith, K. E., & Pollak, S. D. (2021) · Cognitive Affective & Behavioral Neuroscience · doi:10.3758/s13415-021-00901-0 · Cited by 33 · see citing works
Sustained Pupil Responses Are Modulated by Predictability of Auditory Sequences
Milne, A. E., Zhao, S., Tampakaki, C., Bury, G., & Maria Chait, A. (2021) · Journal of Neuroscience · doi:10.1523/jneurosci.2879-20.2021 · Cited by 34 · see citing works
The orbitofrontal cortex in temporal cognition.
Sosa, J. L. R., Buonomano, D. V., & Izquierdo, A. (2021) · Behavioral Neuroscience · doi:10.1037/bne0000430 · Cited by 35 · see citing works
Self-citationPupil diameter tracked during motor adaptation in humans
Yokoi, A., & Weiler, J. (2022) · Journal of Neurophysiology · doi:10.1152/jn.00021.2022 · Cited by 34 · see citing works
Uncertainty in a context of pain: disliked but also more painful?
Zaman, J., Oudenhove, L. V., & Vlaeyen, J. W. S. (2020) · Pain · doi:10.1097/j.pain.0000000000002106 · Cited by 32 · see citing works
Yu-peng, L., Niu, R., & Yu, Z. (2023) · Language Teaching Research · doi:10.1177/13621688231216201 · Cited by 35 · see citing works
Expecting the unexpected: a review of learning under uncertainty across development
Topel, S., Ma, I., Sleutels, J. J. M., van Steenbergen, H., de Bruijn, E. R. A., & van Duijvenvoorde, A. C. K. (2023) · Cognitive Affective & Behavioral Neuroscience · doi:10.3758/s13415-023-01098-0 · Cited by 34 · see citing works
Surprise and recency in novelty detection in the primate brain
Zhang, K., Bromberg-Martin, E. S., Sogukpinar, F., Kocher, K., & Monosov, I. E. (2022) · Current Biology · doi:10.1016/j.cub.2022.03.064 · Cited by 27 · see citing works
Brainstem fMRI signaling of surprise across different types of deviant stimuli
Mazancieux, A., Mauconduit, F., Amadon, A., de Gee, J. W., Donner, T. H., & Meyniel, F. (2023) · Cell Reports · doi:10.1016/j.celrep.2023.113405 · Cited by 27 · see citing works
Mixing memory and desire: How memory reactivation supports deliberative decision‐making
Wang, S., Feng, S. F., & Bornstein, A. M. (2021) · Wiley Interdisciplinary Reviews Cognitive Science · doi:10.1002/wcs.1581 · Cited by 26 · see citing works
Unique features of stimulus-based probabilistic reversal learning.
Harris, C., Aguirre, C. G., Kolli, S., Das, K., Izquierdo, A., & Soltani, A. (2021) · Behavioral Neuroscience · doi:10.1037/bne0000474 · Cited by 27 · see citing works
Self-citationRelative salience signaling within a thalamo-orbitofrontal circuit governs learning rate
Namboodiri, V. M. K., Hobbs, T. G., Trujillo‐Pisanty, I., Simon, R. C., Hjort, M. M., & Stuber, G. D. (2021) · Current Biology · doi:10.1016/j.cub.2021.09.037 · Cited by 25 · see citing works
Association between panic disorder and childhood adversities: a systematic review and meta-analysis
Zhang, J., Wiecaszek, P., Sami, S., & Meiser‐Stedman, R. (2021) · Psychological Medicine · doi:10.1017/s0033291721004505 · Cited by 26 · see citing works
Apergis-Schoute, A. M., van der Flier, F. E., Ip, S., Kanen, J. W., Vaghi, M. M. S., Fineberg, N. A., Sahakian, B. J., Cardinal, R. N., & Robbins, T. W. (2023) · Biological Psychiatry Global Open Science · doi:10.1016/j.bpsgos.2023.06.004 · Cited by 26 · see citing works
Surprise and novelty in the brain
Modirshanechi, A., Becker, S., Brea, J., & Gerstner, W. (2023) · Current Opinion in Neurobiology · doi:10.1016/j.conb.2023.102758 · Cited by 29 · see citing works
Marković, D., Goschke, T., & Kiebel, S. J. (2020) · Cognitive Affective & Behavioral Neuroscience · doi:10.3758/s13415-020-00837-x · Cited by 24 · see citing works
Kreis, I. V., Zhang, L., Moritz, S., & Pfuhl, G. (2021) · Schizophrenia Research · doi:10.1016/j.schres.2021.06.038 · Cited by 21 · see citing works
Sandini, T. M., Onofrychuk, T. J., Roebuck, A. J., Hammond, S. A., Udenze, D., Hayat, S., Herdzik, M. A., McElroy, D. L., Orvold, S. N., Greba, Q., et al. (2023) · eNeuro · doi:10.1523/eneuro.0100-23.2023 · Cited by 24 · see citing works
Pupil dilation and skin conductance as measures of prediction error in aversive learning
Stemerding, L., van Ast, V. A., Gerlicher, A. M. V., & Kindt, M. (2022) · Behaviour Research and Therapy · doi:10.1016/j.brat.2022.104164 · Cited by 22 · see citing works
Understanding the development of reward learning through the lens of meta-learning
Nussenbaum, K., & Hartley, C. A. (2024) · Nature Reviews Psychology · doi:10.1038/s44159-024-00304-1 · Cited by 28 · see citing works
Frith, C., & Frith, U. (2023) · The MIT Press eBooks · doi:10.7551/mitpress/10400.001.0001 · Cited by 22 · see citing works
Understanding learning through uncertainty and bias
Bruckner, R., Heekeren, H. R., & Nassar, M. R. (2025) · Communications Psychology · doi:10.1038/s44271-025-00203-y · Cited by 34 · see citing works
Accumbofrontal tract integrity is related to early life adversity and feedback learning
Kennedy, B. V., Hanson, J. L., Buser, N. J., van den Bos, W., Rudolph, K. D., Davidson, R. J., & Pollak, S. D. (2021) · Neuropsychopharmacology · doi:10.1038/s41386-021-01129-9 · Cited by 20 · see citing works
Woo, J. H., Aguirre, C. G., Bari, B. A., Tsutsui, K., Fabian Grabenhorst, B., Cohen, J. Y., Schultz, W., Izquierdo, A., & Soltani, A. (2023) · Cognitive Affective & Behavioral Neuroscience · doi:10.3758/s13415-022-01059-z · Cited by 21 · see citing works
Self-citationAdaptive tuning of human learning and choice variability to unexpected uncertainty
Lee, J. K., Rouault, M., & Wyart, V. (2023) · Science Advances · doi:10.1126/sciadv.add0501 · Cited by 20 · see citing works
Temporal chunking as a mechanism for unsupervised learning of task-sets
Bouchacourt, F., Palminteri, S., Koechlin, É., & Ostojic, S. (2020) · eLife · doi:10.7554/elife.50469 · Cited by 20 · see citing works
The challenge of memory destabilisation: From prediction error to prior expectations and biomarkers
Milton, A. L., Das, R., & Merlo, E. (2023) · Brain Research Bulletin · doi:10.1016/j.brainresbull.2023.01.010 · Cited by 21 · see citing works
Stroink, M. L. (2020) · Frontiers in Sustainable Cities · doi:10.3389/frsc.2020.00031 · Cited by 19 · see citing works
Suthaharan, P., Thompson, S. L., Rossi-Goldthorpe, R. A., Rudebeck, P. H., Walton, M. E., Chakraborty, S., Noonan, M. P., Costa, V. D., Murray, E. A., Mathys, C. D., et al. (2024) · Cell Reports · doi:10.1016/j.celrep.2024.114355 · Cited by 19 · see citing works
Dopamine transients encode reward prediction errors independent of learning rates
Mah, A., Golden, C. E. M., & Constantinople, C. M. (2024) · Cell Reports · doi:10.1016/j.celrep.2024.114840 · Cited by 21 · see citing works
Consciousness, (meta)cognition, and culture
Frith, C. (2023) · Quarterly Journal of Experimental Psychology · doi:10.1177/17470218231164502 · Cited by 20 · see citing works
Ez‐zizi, A., Farrell, S., Leslie, D. S., Malhotra, G., & Ludwig, C. J. H. (2023) · Computational Brain & Behavior · doi:10.1007/s42113-022-00165-y · Cited by 17 · see citing works
The rat frontal orienting field dynamically encodes value for economic decisions under risk
Bao, C., Zhu, X., Mōller-Mara, J., Li, J., Dubroqua, S., & Erlich, J. C. (2023) · Nature Neuroscience · doi:10.1038/s41593-023-01461-x · Cited by 17 · see citing works
Prior cocaine self-administration impairs attention signals in anterior cingulate cortex
Vázquez, D., Pribut, H. J., Burton, A. C., Tennyson, S. S., & Roesch, M. R. (2019) · Neuropsychopharmacology · doi:10.1038/s41386-019-0578-2 · Cited by 18 · see citing works
Efficient stabilization of imprecise statistical inference through conditional belief updating
Drevet, J., Drugowitsch, J., & Wyart, V. (2022) · Nature Human Behaviour · doi:10.1038/s41562-022-01445-0 · Cited by 17 · see citing works
Womelsdorf, T., Watson, M. R., & Tiesinga, P. H. E. (2021) · Journal of Cognitive Neuroscience · doi:10.1162/jocn_a_01780 · Cited by 17 · see citing works
Synthetic surprise as the foundation of the psychedelic experience
Filippo, R. D., & Schmitz, D. (2024) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2024.105538 · Cited by 15 · see citing works
Fast adaptation to rule switching using neuronal surprise
Barry, M., & Wulfram Gerstner, B. (2024) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1011839 · Cited by 18 · see citing works
Farashahi, S. G., Xu, J. Y., Wu, S., & Soltani, A. (2020) · Cognition · doi:10.1016/j.cognition.2020.104425 · Cited by 16 · see citing works
Self-citationSex difference in the weighting of expected uncertainty under chronic stress
Lei, H., Mochizuki, Y., Chen, C., Hagiwara, K., Hirotsu, M., Matsubara, T., & Nakagawa, S. (2021) · Scientific Reports · doi:10.1038/s41598-021-88155-1 · Cited by 15 · see citing works
The influence of anxiety on exploration: A review of computational modeling studies
Chou, K.-P., Wilson, R. C., & Smith, R. S. (2024) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2024.105940 · Cited by 15 · see citing works
Leveraging Basic Science for the Clinic—From Bench to Bedside
Corlett, P. R., & Schoenbaum, G. M. (2020) · JAMA Psychiatry · doi:10.1001/jamapsychiatry.2020.3656 · Cited by 15 · see citing works
Foucault, C., & Meyniel, F. (2021) · eLife · doi:10.7554/elife.71801 · Cited by 14 · see citing works
Pajkossy, P., Gesztesi, G., & Racsmány, M. (2023) · Cognitive Affective & Behavioral Neuroscience · doi:10.3758/s13415-023-01072-w · Cited by 13 · see citing works
Dopamine neurons encode a multidimensional probabilistic map of future rewardPreprint
Sousa, M., Bujalski, P., Cruz, B. F., Louie, K., McNamee, D., & Paton, J. J. (2023) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2023.11.12.566727 · Cited by 15 · see citing works
Aguirre, C. G., Woo, J. H., Romero-Sosa, J. L., Rivera, Z. M., Tejada, A. N., Munier, J. J., Perez, J., Goldfarb, M., Das, K., Gomez, M., et al. (2023) · Journal of Neuroscience · doi:10.1523/jneurosci.0622-23.2023 · Cited by 14 · see citing works
Self-citationGonzález, V. V., Zhang, Y., Ashikyan, S. A., Rickard, A., Yassine, I., Romero-Sosa, J. L., Blaisdell, A. P., & Izquierdo, A. (2024) · Cerebral Cortex · doi:10.1093/cercor/bhae135 · Cited by 15 · see citing works
Self-citationA multidimensional distributional map of future reward in dopamine neurons
Sousa, M., Bujalski, P., Cruz, B. F., Louie, K., McNamee, D., & Paton, J. J. (2025) · Nature · doi:10.1038/s41586-025-09089-6 · Cited by 30 · see citing works
Guttman, Z., Mandelkern, M. A., Ghahremani, D. G., Kohno, M., Dean, A. C., & London, E. D. (2023) · Drug and Alcohol Dependence · doi:10.1016/j.drugalcdep.2023.109860 · Cited by 13 · see citing works
Acetylcholine demixes heterogeneous dopamine signals for learning and moving
Jang, H. J., Ward, R. M., Golden, C. E. M., & Constantinople, C. M. (2026) · Nature Neuroscience · doi:10.1038/s41593-026-02227-x · Cited by 15 · see citing works
Schirru, M., Véronneau‐Veilleux, F., Nekka, F., & Ursino, M. (2022) · International Journal of Molecular Sciences · doi:10.3390/ijms23073452 · Cited by 13 · see citing works
Understanding Learning Through Uncertainty and BiasPreprint
Bruckner, R., Heekeren, H. R., & Nassar, M. R. (2022) · doi:10.31234/osf.io/xjkbg · Cited by 13 · see citing works
Chronic ethanol exposure produces sex-dependent impairments in value computations in the striatum
Cheng, Y., Magnard, R., Langdon, A., Lee, D., & Janak, P. H. (2025) · Science Advances · doi:10.1126/sciadv.adt0200 · Cited by 12 · see citing works
Change point estimation by the mouse medial frontal cortex during probabilistic reward learningPreprint
Murphy, C. E., Wang, H., Ortega, H. K., Kwan, A. C., & Atilgan, H. (2022) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2022.05.26.493245 · Cited by 12 · see citing works
Experience-driven recalibration of learning from surprising events
Bakst, L., & McGuire, J. T. (2022) · Cognition · doi:10.1016/j.cognition.2022.105343 · Cited by 16 · see citing works
Vázquez, D., Maulhardt, S., Stalnaker, T. A., Solway, A., Charpentier, C. J., & Roesch, M. R. (2024) · Journal of Neuroscience · doi:10.1523/jneurosci.1850-23.2024 · Cited by 11 · see citing works
The evolution of sensitive periods beyond early ontogeny: Bridging theory and data
Walasek, N., Panchanathan, K., & Frankenhuis, W. E. (2024) · Functional Ecology · doi:10.1111/1365-2435.14615 · Cited by 14 · see citing works
- Authors
- Soltani, A., & Izquierdo, A.
- Affiliations
- Dartmouth College (US)
- University of California, Los Angeles (US)
- Corresponding author
- Soltani, Alireza
- Izquierdo, Alicia
- Venue
- Nature reviews. Neuroscience
- Published
- 2019-05-30
- Volume
- 20
- Issue
- 10
- Pages
- 635-644
- ISSN
- 1471-003X (ISSN-L)
- 1471-0048
- PMC
- PMC6752962
- OpenAlex
- W2946961766
- Cited by
- 286
- References
- 118
- Citations by year
- 2026: 31 · 2025: 63 · 2024: 51 · 2023: 42 · 2022: 29 · 2021: 36 · 2020: 26 · 2019: 8
- FWCI
- 3.10
- Type
- journal article
- Language
- en
- Open access
- green
- www.ncbi.nlm.nih.gov
- Locations
- Nature reviews. Neuroscience (publishedVersion)
- PubMed (publishedVersion)
- Europe PMC (PubMed Central) (acceptedVersion)
- PubMed Central (acceptedVersion)
- eScholarship (California Digital Library) (submittedVersion)
- Funders
- National Institute on Drug Abuse (R01DA047870)
- Topics
- Life Sciences › Neuroscience › Cognitive Neuroscience › Neural and Behavioral Psychology Studies
- Life Sciences › Neuroscience › Cognitive Neuroscience › Memory and Neural Mechanisms
- Social Sciences › Decision Sciences › General Decision Sciences › Decision-Making and Behavioral Economics
- Keywords
- adaptive learning, uncertainty estimation, decision-making, computational model, neural mechanisms, outcome variability, reinforcement learning
- MeSH terms
- Adaptation, Biological, Animals, Brain, Choice Behavior, Decision Making, Humans, Learning, Nerve Net, Uncertainty
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
