Cited by 286Neural and Behavioral Psychology StudiesMemory and Neural MechanismsDecision-Making and Behavioral Economics

The DOI matches its Crossref record; 118 of the 118 references it cites were found in a registry or an index.

Adaptive learning under expected and unexpected uncertainty

Open access

Soltani, Alireza, Izquierdo, Alicia·2019·Nature reviews. Neuroscience

DOI 10.1038/s41583-019-0180-yPMC PMC6752962

Open this paper in CoChat — free account

Abstract

The outcome of a decision is often uncertain, and outcomes can vary over repeated decisions. Whether decision outcomes should substantially affect behaviour and learning depends on whether they are representative of a typically experienced range of outcomes or signal a change in the reward environment. Successful learning and decision-making therefore require the ability to estimate expected uncertainty (related to the variability of outcomes) and unexpected uncertainty (related to the variability of the environment). Understanding the bases and effects of these two types of uncertainty and the interactions between them - at the computational and the neural level - is crucial for understanding adaptive learning. Here, we examine computational models and experimental findings to distil computational principles and neural mechanisms for adaptive learning under uncertainty.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

Integrity signals

Registry

Crossref · 10.1038/s41583-019-0180-y

Cites

118 of 118 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Reinforcement Learning: An Introduction
Sutton, R. S., & Barto, A. G. (1998) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.1998.712192 · Cited by 27,225
Reinforcement Learning: An Introduction
Sutton, R. S., & Barto, A. (2005) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.2004.842673 · Cited by 25,268
A neural substrate of prediction and reward
Schultz, W., Dayan, P., & Montague, P. R. (1997) · Science · doi:10.1126/science.275.5306.1593 · Cited by 9,388
What is the role of dopamine in reward: hedonic impact, reward learning, or incentive salience?
Berridge, K. C., & Robinson, T. E. (1998) · Brain Research Reviews · doi:10.1016/s0165-0173(98)00019-8 · Cited by 4,127
A model for Pavlovian learning: Variations in the effectiveness of conditioned but not of unconditioned stimuli.
Pearce, J. M., & Hall, G. (1980) · Psychological Review · doi:10.1037/0033-295x.87.6.532 · Cited by 3,023
The expected value of control: an integrative theory of anterior cingulate cortex function
Shenhav, A., Botvinick, M. M., & Cohen, J. D. (2013) · Neuron · doi:10.1016/j.neuron.2013.07.007 · Cited by 2,321
Cortical substrates for exploratory decisions in humans
Daw, N. D., O'Doherty, J. P., Dayan, P., Seymour, B., & Dolan, R. J. (2006) · Nature · doi:10.1038/nature04766 · Cited by 2,224
Learning the value of information in an uncertain world
Behrens, T. E. J., Woolrich, M. W., Walton, M. E., & Rushworth, M. F. S. (2007) · Nature Neuroscience · doi:10.1038/nn1954 · Cited by 2,053
Uncertainty and anticipation in anxiety: an integrated neurobiological and psychological perspective
Grupe, D. W., & Nitschke, J. B. (2013) · Nature Reviews Neuroscience · doi:10.1038/nrn3524 · Cited by 2,007
Uncertainty, Neuromodulation, and Attention
Yu, A. J., & Dayan, P. (2005) · Neuron · doi:10.1016/j.neuron.2005.04.026 · Cited by 1,774

Show all 118 references

Research works printed but not linked: 2

Dayan, P. & Long, T. in Advances in Neural Information Processing Systems 10: Proceedings of the 1997 Conference (eds Jordan, M. I., Kearns, M. J. & Solla, S. A.) 117–123 (MIT Press, 1998).
Soltani, A. C., W. & Wang, X. J. in Decision Neuroscience: An Integrative Perspective (eds Dreher, J.-C. & Tremblay, L.) 163–222 (Elsevier Academic Press, 2017).

Sorted by citations

Rethinking Concepts and Categories for Understanding the Neurodevelopmental Effects of Childhood Adversity
Smith, K. E., & Pollak, S. D. (2020) · Perspectives on Psychological Science · doi:10.1177/1745691620920725 · Cited by 379
The prefrontal cortex, pathological anxiety, and anxiety disorders
Kenwood, M. M., Kalin, N. H., & Barbas, H. (2021) · Neuropsychopharmacology · doi:10.1038/s41386-021-01109-z · Cited by 379
Medial and orbital frontal cortex in decision-making and flexible behavior
Klein-Flügge, M. C., Bongioanni, A., & Rushworth, M. F. S. (2022) · Neuron · doi:10.1016/j.neuron.2022.05.022 · Cited by 244
Using reinforcement learning models in social neuroscience: frameworks, pitfalls and suggestions of best practices
Zhang, L., Lengersdorff, L., Mikuš, N., Gläscher, J. P., & Lamm, C. (2020) · Social Cognitive and Affective Neuroscience · doi:10.1093/scan/nsaa089 · Cited by 168
Serotonin neurons modulate learning rate through uncertainty
Grossman, C. D., Bari, B. A., & Cohen, J. Y. (2022) · Current Biology · doi:10.1016/j.cub.2021.12.006 · Cited by 164
A brain network supporting social influences in human decision-making
Zhang, L., & Gläscher, J. P. (2020) · Science Advances · doi:10.1126/sciadv.abb4159 · Cited by 158
A model for learning based on the joint estimation of stochasticity and volatility
Piray, P., & Daw, N. D. (2021) · Nature Communications · doi:10.1038/s41467-021-26731-9 · Cited by 153
Paranoia and belief updating during the COVID-19 crisis
Suthaharan, P., Reed, E., Leptourgos, P., Kenney, J. G., Uddenberg, S., Mathys, C. D., Litman, L., Robinson, J., Moss, A. J., Taylor, J. R., et al. (2021) · Nature Human Behaviour · doi:10.1038/s41562-021-01176-8 · Cited by 123
Anterior Cingulate Cortex and the Control of Dynamic Behavior in Primates
Monosov, I. E., Haber, S. N., Leuthardt, E. C., & Jezzini, A. (2020) · Current Biology · doi:10.1016/j.cub.2020.10.009 · Cited by 118
Flexible combination of reward information across primates
Farashahi, S., Donahue, C. H., Hayden, B. Y., Lee, D., & Soltani, A. (2019) · Nature Human Behaviour · doi:10.1038/s41562-019-0714-3 · Cited by 112
Self-citation
Eye pupil signals information gain
Zénon, A. (2019) · Proceedings of the Royal Society B Biological Sciences · doi:10.1098/rspb.2019.1593 · Cited by 110
Thalamocortical excitability modulation guides human perception under uncertainty
Kosciessa, J. Q., Lindenberger, U., & Garrett, D. D. (2021) · Nature Communications · doi:10.1038/s41467-021-22511-7 · Cited by 102
A simple model for learning in volatile environments
Piray, P., & Daw, N. D. (2020) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1007963 · Cited by 104
Contributions of anterior cingulate cortex and basolateral amygdala to decision confidence and learning under uncertainty
Stolyarova, A., Rakhshan, M., Hart, E. E., O’Dell, T. J., Peters, M. A. K., Lau, H. C., Soltani, A., & Izquierdo, A. (2019) · Nature Communications · doi:10.1038/s41467-019-12725-1 · Cited by 93
Self-citation
Neural computations of threat
Levy, I., & Schiller, D. (2021) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2020.11.007 · Cited by 95
How Outcome Uncertainty Mediates Attention, Learning, and Decision-Making
Monosov, I. E. (2020) · Trends in Neurosciences · doi:10.1016/j.tins.2020.06.009 · Cited by 95
Predictive Models for Educational Purposes: A Systematic Review
Almalawi, A., Soh, B., Li, A., & Samra, H. E. (2024) · Big Data and Cognitive Computing · doi:10.3390/bdcc8120187 · Cited by 91
Uncertainty Makes Me Emotional: Uncertainty as an Elicitor and Modulator of Emotional States
Morriss, J., Tupitsa, E., Dodd, H. F., & Hirsch, C. R. (2022) · Frontiers in Psychology · doi:10.3389/fpsyg.2022.777025 · Cited by 98
Computational models of adaptive behavior and prefrontal cortex
Soltani, A., & Koechlin, É. (2021) · Neuropsychopharmacology · doi:10.1038/s41386-021-01123-1 · Cited by 86
Self-citation
AI-Enabled UAV Communications: Challenges and Future Directions
Hashesh, A. O., Hashima, S., Zaki, R. M., Fouda, M. M., Hatano, K., & Eldien, A. S. T. (2022) · IEEE Access · doi:10.1109/access.2022.3202956 · Cited by 97
Divergent Strategies for Learning in Males and Females
Chen, C. S., Ebitz, R. B., Bindas, S. R., Redish, A. D., Hayden, B. Y., & Grissom, N. M. (2020) · Current Biology · doi:10.1016/j.cub.2020.09.075 · Cited by 90
Distributional Reinforcement Learning in the Brain
Lowet, A. S., Qiao, Z., Matias, S., Drugowitsch, J., & Uchida, N. (2020) · Trends in Neurosciences · doi:10.1016/j.tins.2020.09.004 · Cited by 75
Timescales of cognition in the brain
Soltani, A., Murray, J. D., Seo, H., & Lee, D. (2021) · Current Opinion in Behavioral Sciences · doi:10.1016/j.cobeha.2021.03.003 · Cited by 60
Self-citation
A Decision Architecture for Safety Computations
Tashjian, S. M., Zbozinek, T. D., & Mobbs, D. (2021) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2021.01.013 · Cited by 62
A taxonomy of surprise definitions
Modirshanechi, A., Brea, J., & Wulfram Gerstner, B. (2022) · Journal of Mathematical Psychology · doi:10.1016/j.jmp.2022.102712 · Cited by 67
Brain and Cognitive Development in Adolescents with Anorexia Nervosa: A Systematic Review of fMRI Studies
Olivo, G., Gaudio, S., & Schiöth, H. B. (2019) · Nutrients · doi:10.3390/nu11081907 · Cited by 56
Uncertainty-driven regulation of learning and exploration in adolescents: A computational account
Jepma, M., Schaaf, J. V., Visser, I., & Huizenga, H. M. (2020) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1008276 · Cited by 48
Transdiagnostic computations of uncertainty: towards a new lens on intolerance of uncertainty
Sandhu, T. R., Xiao, B., & Lawson, R. P. (2023) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2023.105123 · Cited by 51
Interactions between ventrolateral prefrontal and anterior cingulate cortex during learning and behavioural change
Monosov, I. E., & Rushworth, M. F. S. (2021) · Neuropsychopharmacology · doi:10.1038/s41386-021-01079-2 · Cited by 53
Ultralow energy adaptive neuromorphic computing using reconfigurable zinc phosphorus trisulfide memristors
Ji, Y., Wang, L., Long, Y., Wang, J., Zheng, H., Yu, Z. G., Zhang, Y., & Ang, K. (2025) · Nature Communications · doi:10.1038/s41467-025-62306-8 · Cited by 51
Aging, uncertainty, and decision making—A review
Frank, C. C., & Seaman, K. L. (2023) · Cognitive Affective & Behavioral Neuroscience · doi:10.3758/s13415-023-01064-w · Cited by 60
Foraging with the frontal cortex: A cross-species evaluation of reward-guided behavior
Rudebeck, P. H., & Izquierdo, A. (2021) · Neuropsychopharmacology · doi:10.1038/s41386-021-01140-0 · Cited by 55
Self-citation
Computational processes of simultaneous learning of stochasticity and volatility in humans
Piray, P., & Daw, N. D. (2024) · Nature Communications · doi:10.1038/s41467-024-53459-z · Cited by 45
An empirical evaluation of active inference in multi-armed bandits
Marković, D., Stojić, H., Schwöbel, S., & Kiebel, S. J. (2021) · Neural Networks · doi:10.1016/j.neunet.2021.08.018 · Cited by 44
Goal-directed learning in adolescence: neurocognitive development and contextual influences
Wilbrecht, L., & Davidow, J. Y. (2024) · Nature reviews. Neuroscience · doi:10.1038/s41583-023-00783-w · Cited by 47
Anxiety as a disorder of uncertainty: implications for understanding maladaptive anxiety, anxious avoidance, and exposure therapy
Brown, V. M., Price, R. B., & Dombrovski, A. Y. (2023) · Cognitive Affective & Behavioral Neuroscience · doi:10.3758/s13415-023-01080-w · Cited by 51
Assessing unpredictability in caregiver–child relationships: Insights from theoretical and empirical perspectives
Ugarte, E., & Hastings, P. D. (2023) · Development and Psychopathology · doi:10.1017/s0954579423000305 · Cited by 51
Novelty is not surprise: Human exploratory and adaptive behavior in sequential decision-making
He, X., Modirshanechi, A., Lehmann, M. P., Wulfram Gerstner, B., & Herzog, M. H. (2021) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1009070 · Cited by 49
Assumed shared belief about conspiracy theories in social networks protects paranoid individuals against distress
Suthaharan, P., & Corlett, P. R. (2023) · Scientific Reports · doi:10.1038/s41598-023-33305-w · Cited by 38
State anxiety alters the neural oscillatory correlates of predictions and prediction errors during reward-based learning
Hein, T. P., & Ruiz, M. H. (2022) · NeuroImage · doi:10.1016/j.neuroimage.2022.118895 · Cited by 37
Early experience unpredictability in child development as a model for understanding the impact of the COVID-19 pandemic: A translational neuroscience perspective
Liu, S., & Fisher, P. A. (2022) · Developmental Cognitive Neuroscience · doi:10.1016/j.dcn.2022.101091 · Cited by 40
Adaptive Learning through Temporal Dynamics of State Representation
Razmi, N., & Nassar, M. R. (2022) · Journal of Neuroscience · doi:10.1523/jneurosci.0387-21.2022 · Cited by 32
Probabilistic Reinforcement Learning and Anhedonia
Kangas, B. D., Der‐Avakian, A., & Pizzagalli, D. A. (2022) · Current topics in behavioral neurosciences · doi:10.1007/7854_2022_349 · Cited by 36
Early life stress and neural development: Implications for understanding the developmental effects of COVID-19
Smith, K. E., & Pollak, S. D. (2021) · Cognitive Affective & Behavioral Neuroscience · doi:10.3758/s13415-021-00901-0 · Cited by 38
Sustained Pupil Responses Are Modulated by Predictability of Auditory Sequences
Milne, A. E., Zhao, S., Tampakaki, C., Bury, G., & Maria Chait, A. (2021) · Journal of Neuroscience · doi:10.1523/jneurosci.2879-20.2021 · Cited by 42
The orbitofrontal cortex in temporal cognition.
Sosa, J. L. R., Buonomano, D. V., & Izquierdo, A. (2021) · Behavioral Neuroscience · doi:10.1037/bne0000430 · Cited by 36
Self-citation
Pupil diameter tracked during motor adaptation in humans
Yokoi, A., & Weiler, J. (2022) · Journal of Neurophysiology · doi:10.1152/jn.00021.2022 · Cited by 35
Uncertainty in a context of pain: disliked but also more painful?
Zaman, J., Oudenhove, L. V., & Vlaeyen, J. W. S. (2020) · Pain · doi:10.1097/j.pain.0000000000002106 · Cited by 32
Roles of ambiguity tolerance and learning effectiveness: Structural equation modeling evidence from EFL students’ perceptions of factors influencing peer collaboration
Yu-peng, L., Niu, R., & Yu, Z. (2023) · Language Teaching Research · doi:10.1177/13621688231216201 · Cited by 35
Expecting the unexpected: a review of learning under uncertainty across development
Topel, S., Ma, I., Sleutels, J. J. M., van Steenbergen, H., de Bruijn, E. R. A., & van Duijvenvoorde, A. C. K. (2023) · Cognitive Affective & Behavioral Neuroscience · doi:10.3758/s13415-023-01098-0 · Cited by 37
Surprise and recency in novelty detection in the primate brain
Zhang, K., Bromberg-Martin, E. S., Sogukpinar, F., Kocher, K., & Monosov, I. E. (2022) · Current Biology · doi:10.1016/j.cub.2022.03.064 · Cited by 27
Brainstem fMRI signaling of surprise across different types of deviant stimuli
Mazancieux, A., Mauconduit, F., Amadon, A., de Gee, J. W., Donner, T. H., & Meyniel, F. (2023) · Cell Reports · doi:10.1016/j.celrep.2023.113405 · Cited by 29
Mixing memory and desire: How memory reactivation supports deliberative decision‐making
Wang, S., Feng, S. F., & Bornstein, A. M. (2021) · Wiley Interdisciplinary Reviews Cognitive Science · doi:10.1002/wcs.1581 · Cited by 28
Unique features of stimulus-based probabilistic reversal learning.
Harris, C., Aguirre, C. G., Kolli, S., Das, K., Izquierdo, A., & Soltani, A. (2021) · Behavioral Neuroscience · doi:10.1037/bne0000474 · Cited by 27
Self-citation
Relative salience signaling within a thalamo-orbitofrontal circuit governs learning rate
Namboodiri, V. M. K., Hobbs, T. G., Trujillo‐Pisanty, I., Simon, R. C., Hjort, M. M., & Stuber, G. D. (2021) · Current Biology · doi:10.1016/j.cub.2021.09.037 · Cited by 26
Association between panic disorder and childhood adversities: a systematic review and meta-analysis
Zhang, J., Wiecaszek, P., Sami, S., & Meiser‐Stedman, R. (2021) · Psychological Medicine · doi:10.1017/s0033291721004505 · Cited by 28
Perseveration and Shifting in Obsessive-Compulsive Disorder as a Function of Uncertainty, Punishment, and Serotonergic Medication
Apergis-Schoute, A. M., van der Flier, F. E., Ip, S., Kanen, J. W., Vaghi, M. M. S., Fineberg, N. A., Sahakian, B. J., Cardinal, R. N., & Robbins, T. W. (2023) · Biological Psychiatry Global Open Science · doi:10.1016/j.bpsgos.2023.06.004 · Cited by 28
Surprise and novelty in the brain
Modirshanechi, A., Becker, S., Brea, J., & Wulfram Gerstner, B. (2023) · Current Opinion in Neurobiology · doi:10.1016/j.conb.2023.102758 · Cited by 31
Meta-control of the exploration-exploitation dilemma emerges from probabilistic inference over a hierarchy of time scales
Marković, D., Goschke, T., & Kiebel, S. J. (2020) · Cognitive Affective & Behavioral Neuroscience · doi:10.3758/s13415-020-00837-x · Cited by 24
Spared performance but increased uncertainty in schizophrenia: Evidence from a probabilistic decision-making task
Kreis, I. V., Zhang, L., Moritz, S., & Pfuhl, G. (2021) · Schizophrenia Research · doi:10.1016/j.schres.2021.06.038 · Cited by 21
Repeated Exposure to High-THC Cannabis Smoke during Gestation Alters Sex Ratio, Behavior, and Amygdala Gene Expression of Sprague Dawley Rat Offspring
Sandini, T. M., Onofrychuk, T. J., Roebuck, A. J., Hammond, S. A., Udenze, D., Hayat, S., Herdzik, M. A., McElroy, D. L., Orvold, S. N., Greba, Q., et al. (2023) · eNeuro · doi:10.1523/eneuro.0100-23.2023 · Cited by 24
Pupil dilation and skin conductance as measures of prediction error in aversive learning
Stemerding, L., van Ast, V. A., Gerlicher, A. M. V., & Kindt, M. (2022) · Behaviour Research and Therapy · doi:10.1016/j.brat.2022.104164 · Cited by 24
Understanding the development of reward learning through the lens of meta-learning
Nussenbaum, K., & Hartley, C. A. (2024) · Nature Reviews Psychology · doi:10.1038/s44159-024-00304-1 · Cited by 35
What Makes Us Social?
Frith, C., & Frith, U. (2023) · The MIT Press eBooks · doi:10.7551/mitpress/10400.001.0001 · Cited by 22
Understanding learning through uncertainty and bias
Bruckner, R., Heekeren, H. R., & Nassar, M. R. (2025) · Communications Psychology · doi:10.1038/s44271-025-00203-y · Cited by 39
Accumbofrontal tract integrity is related to early life adversity and feedback learning
Kennedy, B. V., Hanson, J. L., Buser, N. J., van den Bos, W., Rudolph, K. D., Davidson, R. J., & Pollak, S. D. (2021) · Neuropsychopharmacology · doi:10.1038/s41386-021-01129-9 · Cited by 22
Mechanisms of adjustments to different types of uncertainty in the reward environment across mice and monkeys
Woo, J. H., Aguirre, C. G., Bari, B. A., Tsutsui, K., Fabian Grabenhorst, B., Cohen, J. Y., Schultz, W., Izquierdo, A., & Soltani, A. (2023) · Cognitive Affective & Behavioral Neuroscience · doi:10.3758/s13415-022-01059-z · Cited by 23
Self-citation
Adaptive tuning of human learning and choice variability to unexpected uncertainty
Lee, J. K., Rouault, M., & Wyart, V. (2023) · Science Advances · doi:10.1126/sciadv.add0501 · Cited by 23
Temporal chunking as a mechanism for unsupervised learning of task-sets
Bouchacourt, F., Palminteri, S., Koechlin, É., & Ostojic, S. (2020) · eLife · doi:10.7554/elife.50469 · Cited by 22
The challenge of memory destabilisation: From prediction error to prior expectations and biomarkers
Milton, A. L., Das, R., & Merlo, E. (2023) · Brain Research Bulletin · doi:10.1016/j.brainresbull.2023.01.010 · Cited by 22
The Dynamics of Psycho-Social-Ecological Resilience in the Urban Environment: A Complex Adaptive Systems Theory Perspective
Stroink, M. L. (2020) · Frontiers in Sustainable Cities · doi:10.3389/frsc.2020.00031 · Cited by 19
Lesions to the mediodorsal thalamus, but not orbitofrontal cortex, enhance volatility beliefs linked to paranoia
Suthaharan, P., Thompson, S. L., Rossi-Goldthorpe, R. A., Rudebeck, P. H., Walton, M. E., Chakraborty, S., Noonan, M. P., Costa, V. D., Murray, E. A., Mathys, C. D., et al. (2024) · Cell Reports · doi:10.1016/j.celrep.2024.114355 · Cited by 19
Dopamine transients encode reward prediction errors independent of learning rates
Mah, A., Golden, C. E. M., & Constantinople, C. M. (2024) · Cell Reports · doi:10.1016/j.celrep.2024.114840 · Cited by 22
Consciousness, (meta)cognition, and culture
Frith, C. (2023) · Quarterly Journal of Experimental Psychology · doi:10.1177/17470218231164502 · Cited by 20
Reinforcement Learning Under Uncertainty: Expected Versus Unexpected Uncertainty and State Versus Reward Uncertainty
Ez‐zizi, A., Farrell, S., Leslie, D. S., Malhotra, G., & Ludwig, C. J. H. (2023) · Computational Brain & Behavior · doi:10.1007/s42113-022-00165-y · Cited by 17
The rat frontal orienting field dynamically encodes value for economic decisions under risk
Bao, C., Zhu, X., Mōller-Mara, J., Li, J., Dubroqua, S., & Erlich, J. C. (2023) · Nature Neuroscience · doi:10.1038/s41593-023-01461-x · Cited by 17
Prior cocaine self-administration impairs attention signals in anterior cingulate cortex
Vázquez, D., Pribut, H. J., Burton, A. C., Tennyson, S. S., & Roesch, M. R. (2019) · Neuropsychopharmacology · doi:10.1038/s41386-019-0578-2 · Cited by 18
Efficient stabilization of imprecise statistical inference through conditional belief updating
Drevet, J., Drugowitsch, J., & Wyart, V. (2022) · Nature Human Behaviour · doi:10.1038/s41562-022-01445-0 · Cited by 21
Learning at Variable Attentional Load Requires Cooperation of Working Memory, Meta-learning, and Attention-augmented Reinforcement Learning
Womelsdorf, T., Watson, M. R., & Tiesinga, P. H. E. (2021) · Journal of Cognitive Neuroscience · doi:10.1162/jocn_a_01780 · Cited by 20
Synthetic surprise as the foundation of the psychedelic experience
Filippo, R. D., & Schmitz, D. (2024) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2024.105538 · Cited by 16
Fast adaptation to rule switching using neuronal surprise
Barry, M., & Wulfram Gerstner, B. (2024) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1011839 · Cited by 20
Learning arbitrary stimulus-reward associations for naturalistic stimuli involves transition from learning about features to learning about objects
Farashahi, S. G., Xu, J. Y., Wu, S., & Soltani, A. (2020) · Cognition · doi:10.1016/j.cognition.2020.104425 · Cited by 16
Self-citation
Sex difference in the weighting of expected uncertainty under chronic stress
Lei, H., Mochizuki, Y., Chen, C., Hagiwara, K., Hirotsu, M., Matsubara, T., & Nakagawa, S. (2021) · Scientific Reports · doi:10.1038/s41598-021-88155-1 · Cited by 15
The influence of anxiety on exploration: A review of computational modeling studies
Chou, K.-P., Wilson, R. C., & Smith, R. S. (2024) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2024.105940 · Cited by 18
Leveraging Basic Science for the Clinic—From Bench to Bedside
Corlett, P. R., & Schoenbaum, G. M. (2020) · JAMA Psychiatry · doi:10.1001/jamapsychiatry.2020.3656 · Cited by 15
Gated recurrence enables simple and accurate sequence prediction in stochastic, changing, and structured environments
Foucault, C., & Meyniel, F. (2021) · eLife · doi:10.7554/elife.71801 · Cited by 16
How uncertain are you? Disentangling expected and unexpected uncertainty in pupil-linked brain arousal during reversal learning
Pajkossy, P., Gesztesi, G., & Racsmány, M. (2023) · Cognitive Affective & Behavioral Neuroscience · doi:10.3758/s13415-023-01072-w · Cited by 15
Dopamine neurons encode a multidimensional probabilistic map of future rewardPreprint
Sousa, M., Bujalski, P., Cruz, B. F., Louie, K., McNamee, D., & Paton, J. J. (2023) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2023.11.12.566727 · Cited by 15
Dissociable Contributions of Basolateral Amygdala and Ventrolateral Orbitofrontal Cortex to Flexible Learning Under Uncertainty
Aguirre, C. G., Woo, J. H., Romero-Sosa, J. L., Rivera, Z. M., Tejada, A. N., Munier, J. J., Perez, J., Goldfarb, M., Das, K., Gomez, M., et al. (2023) · Journal of Neuroscience · doi:10.1523/jneurosci.0622-23.2023 · Cited by 14
Self-citation
A special role for anterior cingulate cortex, but not orbitofrontal cortex or basolateral amygdala, in choices involving information
González, V. V., Zhang, Y., Ashikyan, S. A., Rickard, A., Yassine, I., Romero-Sosa, J. L., Blaisdell, A. P., & Izquierdo, A. (2024) · Cerebral Cortex · doi:10.1093/cercor/bhae135 · Cited by 16
Self-citation
A multidimensional distributional map of future reward in dopamine neurons
Sousa, M., Bujalski, P., Cruz, B. F., Louie, K., McNamee, D., & Paton, J. J. (2025) · Nature · doi:10.1038/s41586-025-09089-6 · Cited by 34
Decomposing risky decision-making in methamphetamine use disorder: Behavioral updating and D2 dopamine receptors
Guttman, Z., Mandelkern, M. A., Ghahremani, D. G., Kohno, M., Dean, A. C., & London, E. D. (2023) · Drug and Alcohol Dependence · doi:10.1016/j.drugalcdep.2023.109860 · Cited by 13
Acetylcholine demixes heterogeneous dopamine signals for learning and moving
Jang, H. J., Ward, R. M., Golden, C. E. M., & Constantinople, C. M. (2026) · Nature Neuroscience · doi:10.1038/s41593-026-02227-x · Cited by 20
Phasic Dopamine Changes and Hebbian Mechanisms during Probabilistic Reversal Learning in Striatal Circuits: A Computational Study
Schirru, M., Véronneau‐Veilleux, F., Nekka, F., & Ursino, M. (2022) · International Journal of Molecular Sciences · doi:10.3390/ijms23073452 · Cited by 13
Understanding Learning Through Uncertainty and BiasPreprint
Bruckner, R., Heekeren, H. R., & Nassar, M. R. (2022) · doi:10.31234/osf.io/xjkbg · Cited by 13
Chronic ethanol exposure produces sex-dependent impairments in value computations in the striatum
Cheng, Y., Magnard, R., Langdon, A., Lee, D., & Janak, P. H. (2025) · Science Advances · doi:10.1126/sciadv.adt0200 · Cited by 16
Change point estimation by the mouse medial frontal cortex during probabilistic reward learningPreprint
Murphy, C. E., Wang, H., Ortega, H. K., Kwan, A. C., & Atilgan, H. (2022) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2022.05.26.493245 · Cited by 12
Experience-driven recalibration of learning from surprising events
Bakst, L., & McGuire, J. T. (2022) · Cognition · doi:10.1016/j.cognition.2022.105343 · Cited by 16
Optogenetic Inhibition of Rat Anterior Cingulate Cortex Impairs the Ability to Initiate and Stay on Task
Vázquez, D., Maulhardt, S., Stalnaker, T. A., Solway, A., Charpentier, C. J., & Roesch, M. R. (2024) · Journal of Neuroscience · doi:10.1523/jneurosci.1850-23.2024 · Cited by 11
The evolution of sensitive periods beyond early ontogeny: Bridging theory and data
Walasek, N., Panchanathan, K., & Frankenhuis, W. E. (2024) · Functional Ecology · doi:10.1111/1365-2435.14615 · Cited by 14

Authors: Soltani, A., & Izquierdo, A.
Affiliations: Dartmouth College (US)
University of California, Los Angeles (US)
Corresponding author: Soltani, Alireza
Izquierdo, Alicia
ORCID: Soltani, Alireza
Izquierdo, Alicia
Venue: Nature reviews. Neuroscience
Publisher: Springer Science and Business Media LLC
Published: 2019-05-30
Volume: 20
Issue: 10
Pages: 635-644
ISSN: 1471-003X (ISSN-L)
1471-0048
DOI: 10.1038/s41583-019-0180-y
PMC: PMC6752962
OpenAlex: W2946961766
Cited by: 286
References: 118
Citations by year: 2026: 31 · 2025: 63 · 2024: 51 · 2023: 42 · 2022: 29 · 2021: 36 · 2020: 26 · 2019: 8
FWCI: 3.10
Type: journal article
Language: en
Open access: green
www.ncbi.nlm.nih.gov
Licence: www.springer.com/tdm (tdm)
www.springer.com/tdm (vor)
Locations: Nature reviews. Neuroscience (publishedVersion)
PubMed (publishedVersion)
Europe PMC (PubMed Central) (acceptedVersion)
PubMed Central (acceptedVersion)
eScholarship (California Digital Library) (submittedVersion)
Funders: National Institute on Drug Abuse (R01DA047870)
Topics: Life Sciences › Neuroscience › Cognitive Neuroscience › Neural and Behavioral Psychology Studies
Life Sciences › Neuroscience › Cognitive Neuroscience › Memory and Neural Mechanisms
Social Sciences › Decision Sciences › General Decision Sciences › Decision-Making and Behavioral Economics
Keywords: adaptive learning, uncertainty estimation, decision-making, computational model, neural mechanisms, outcome variability, reinforcement learning
MeSH terms: Adaptation, Biological, Animals, Brain, Choice Behavior, Decision Making, Humans, Learning, Nerve Net, Uncertainty

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

Adaptive learning under expected and unexpected uncertainty

Abstract

Overview

Integrity signals

Verification

Claims the paper makes

Claims other papers make about this one

Self-check

Citations

Cites

Cited by

Related

Metadata

Record

About these checks