Risk, unexpected uncertainty, and estimation uncertainty: Bayesian learning in unstable settings
Open accessAbstract
Recently, evidence has emerged that humans approach learning using Bayesian updating rather than (model-free) reinforcement algorithms in a six-arm restless bandit problem. Here, we investigate what this implies for human appreciation of uncertainty. In our task, a Bayesian learner distinguishes three equally salient levels of uncertainty. First, the Bayesian perceives irreducible uncertainty or risk: even knowing the payoff probabilities of a given arm, the outcome remains uncertain. Second, there is (parameter) estimation uncertainty or ambiguity: payoff probabilities are unknown and need to be estimated. Third, the outcome probabilities of the arms change: the sudden jumps are referred to as unexpected uncertainty. We document how the three levels of uncertainty evolved during the course of our experiment and how it affected the learning rate. We then zoom in on estimation uncertainty, which has been suggested to be a driving force in exploration, in spite of evidence of widespread aversion to ambiguity. Our data corroborate the latter. We discuss neural evidence that foreshadowed the ability of humans to distinguish between the three levels of uncertainty. Finally, we investigate the boundaries of human capacity to implement Bayesian learning. We repeat the experiment with different instructions, reflecting varying levels of structural uncertainty. Under this fourth notion of uncertainty, choices were no better explained by Bayesian updating than by (model-free) reinforcement learning. Exit questionnaires revealed that participants remained unaware of the presence of unexpected uncertainty and failed to acquire the right model with which to implement Bayesian updating.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Deviations from expected uncertainty should increase learning rates.
“Here, deviations from expected uncertainty (“unexpected uncertainty”) should increase learning rates.8,12,13,15–18”
From Serotonin neurons modulate learning rate through uncertainty (2022) · cites this paper as [15]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Kass, R. E., & Raftery, A. E. (1995) · Journal of the American Statistical Association · doi:10.1080/01621459.1995.10476572 · Cited by 12,558 · see citing works
Nudge: improving decisions about health, wealth, and happiness
(2008) · Choice Reviews Online · doi:10.5860/choice.46-0977 · Cited by 11,614 · see citing works
Knight, F. H. (1921) · Cited by 9,967 · see citing works
“Risk, Ambiguity, and the Savage Axioms.”
Ellsberg, D. (1961) · The Quarterly Journal of Economics · doi:10.2307/1884324 · Cited by 7,891 · see citing works
Statistical Decision Theory and Bayesian Analysis
Berger, J. O. (1985) · Springer Series in Statistics · doi:10.1007/978-1-4757-4286-2 · Cited by 7,722 · see citing works
Rescorla, R. A. (1972) · Medical Entomology and Zoology · Cited by 7,268 · see citing works
Nudge: Improving Decisions About Health, Wealth and Happiness
Adkisson, R. V. (2008) · The Social Science Journal · doi:10.1016/j.soscij.2008.09.003 · Cited by 5,680 · see citing works
Aston-Jones, G., & Cohen, J. D. (2005) · Annual Review of Neuroscience · doi:10.1146/annurev.neuro.28.061604.135709 · Cited by 4,472 · see citing works
Classical conditioning II: Current research and theory.
Black, A. H., & Prokasy, W. F. (1972) · Appleton-Century-Crofts eBooks · Cited by 4,449 · see citing works
Allais, M. (1953) · Econometrica · doi:10.2307/1907921 · Cited by 4,170 · see citing works
Pearce, J. M., & Hall, G. (1980) · Psychological Review · doi:10.1037/0033-295x.87.6.532 · Cited by 3,023 · see citing works
Keynes, J. M. (1973) · Palgrave Macmillan UK eBooks · doi:10.1007/978-1-349-00843-8 · Cited by 2,576 · see citing works
Daw, N. D., Niv, Y., & Dayan, P. (2005) · Nature Neuroscience · doi:10.1038/nn1560 · Cited by 2,533 · see citing works
Stochastic Volatility: Likelihood Inference and Comparison with ARCH Models
Kim, S., Shepherd, N., & Chib, S. (1998) · The Review of Economic Studies · doi:10.1111/1467-937x.00050 · Cited by 2,338 · see citing works
Cortical substrates for exploratory decisions in humans
Daw, N. D., O'Doherty, J. P., Dayan, P., Seymour, B., & Dolan, R. J. (2006) · Nature · doi:10.1038/nature04766 · Cited by 2,224 · see citing works
Discrete coding of reward probability and uncertainty by dopamine neurons
Fiorillo, C. D., Tobler, P. N., & Schultz, W. (2003) · Science · doi:10.1126/science.1077349 · Cited by 2,085 · see citing works
Learning the value of information in an uncertain world
Behrens, T. E. J., Woolrich, M. W., Walton, M. E., & Rushworth, M. F. S. (2007) · Nature Neuroscience · doi:10.1038/nn1954 · Cited by 2,053 · see citing works
Craik, K. J. W. (1943) · Cited by 1,845 · see citing works
Uncertainty, Neuromodulation, and Attention
Yu, A. J., & Dayan, P. (2005) · Neuron · doi:10.1016/j.neuron.2005.04.026 · Cited by 1,774 · see citing works
Assessment and Propagation of Model Uncertainty
Draper, D. (1995) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/j.2517-6161.1995.tb02015.x · Cited by 1,599 · see citing works
Neural systems responding to degrees of uncertainty in human decision-making
Hsu, M., Bhatt, M., Adolphs, R., Tranel, D., & Camerer, C. F. (2005) · Science · doi:10.1126/science.1115327 · Cited by 1,411 · see citing works
Cohen, J. D., McClure, S. M., & Yu, A. J. (2007) · Philosophical Transactions of the Royal Society B Biological Sciences · doi:10.1098/rstb.2007.2098 · Cited by 1,121 · see citing works
Choice, uncertainty and value in prefrontal and cingulate cortex
Rushworth, M. F. S., & Behrens, T. E. J. (2008) · Nature Neuroscience · doi:10.1038/nn2066 · Cited by 866 · see citing works
Uncertainty Aversion, Risk Aversion, and the Optimal Choice of Portfolio
Dow, J., & Werlang, S. R. D. C. (1992) · Econometrica · doi:10.2307/2951685 · Cited by 806 · see citing works
Human insula activation reflects risk prediction errors as well as risk
Preuschoff, K., Quartz, S. R., & Bossaerts, P. (2008) · The Journal of Neuroscience · doi:10.1523/jneurosci.4286-07.2008 · Cited by 784 · see citing works
Paulus, M. P., Rogalsky, C., Simmons, A., Feinstein, J. S., & Stein, M. B. (2003) · NeuroImage · doi:10.1016/s1053-8119(03)00251-9 · Cited by 745 · see citing works
Neural Signatures of Economic Preferences for Risk and Ambiguity
Huettel, S. A., Stowe, C. J., Gordon, E. M., Warner, B. T., & Platt, M. L. (2006) · Neuron · doi:10.1016/j.neuron.2006.01.024 · Cited by 713 · see citing works
Neural differentiation of expected reward and risk in human subcortical structures
Preuschoff, K., Bossaerts, P., & Quartz, S. R. (2006) · Neuron · doi:10.1016/j.neuron.2006.06.024 · Cited by 692 · see citing works
Doya, K. (2008) · Nature Neuroscience · doi:10.1038/nn2077 · Cited by 642 · see citing works
On the Consistency of Bayes Estimates
Diaconis, P. W., & Freedman, D. A. (1986) · The Annals of Statistics · doi:10.1214/aos/1176349830 · Cited by 614 · see citing works
Hampton, A. N., Bossaerts, P., & O’Doherty, J. P. (2006) · The Journal of Neuroscience · doi:10.1523/jneurosci.1010-06.2006 · Cited by 498 · see citing works
Dopamine: generalization and bonuses
Kakade, S. M., & Dayan, P. (2002) · Neural Networks · doi:10.1016/s0893-6080(02)00048-5 · Cited by 477 · see citing works
Reward Value Coding Distinct From Risk Attitude-Related Uncertainty Coding in Human Reward Systems
Tobler, P. N., O'Doherty, J. P., Dolan, R. J., & Schultz, W. (2007) · Journal of Neurophysiology · doi:10.1152/jn.00745.2006 · Cited by 472 · see citing works
Huettel, S. A., Song, A. W., & McCarthy, G. (2005) · Journal of Neuroscience · doi:10.1523/jneurosci.5070-04.2005 · Cited by 457 · see citing works
Bayesian theories of conditioning in a changing world
Courville, A. C., Daw, N. D., & Touretzky, D. S. (2006) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2006.05.004 · Cited by 457 · see citing works
Neural correlates of value, risk, and risk aversion contributing to decision making under risk
Christopoulos, G. I., Tobler, P. N., Bossaerts, P., Dolan, R. J., & Schultz, W. (2009) · The Journal of Neuroscience · doi:10.1523/jneurosci.2614-09.2009 · Cited by 451 · see citing works
Rutishauser, U., Mamelak, A. N., & Schuman, E. M. (2006) · Neuron · doi:10.1016/j.neuron.2006.02.015 · Cited by 296 · see citing works
Information theory, novelty and hippocampal responses: unpredicted or unpredictable?
Strange, B. A., Duggins, A., Penny, W. D., Dolan, R. J., & Friston, K. J. (2005) · Neural Networks · doi:10.1016/j.neunet.2004.12.004 · Cited by 250 · see citing works
Acknowledging Misspecification in Macroeconomic Theory
Hansen, L. P., & Sargent, T. J. (2001) · Review of Economic Dynamics · doi:10.1006/redy.2001.0132 · Cited by 211 · see citing works
Control of exploitation–exploration meta-parameter in reinforcement learning
Ishii, S., Yoshida, W., & Yoshimoto, J. (2002) · Neural Networks · doi:10.1016/s0893-6080(02)00056-4 · Cited by 209 · see citing works
Resolution of Uncertainty in Prefrontal Cortex
Yoshida, W., & Ishii, S. (2006) · Neuron · doi:10.1016/j.neuron.2006.05.006 · Cited by 196 · see citing works
Adding prediction risk to the theory of reward learning
PREUSCHOFF, K., & BOSSAERTS, P. (2007) · Annals of the New York Academy of Sciences · doi:10.1196/annals.1390.005 · Cited by 133 · cited 1× · see citing works
“There, uncertainty is monolithic: it is the expected magnitude of the prediction error [6].”
BackgroundNeural correlates of risk prediction error during reinforcement learning in humans
d'Acremont, M., Lu, Z.-L., Li, X., Van der Linden, M., & Bechara, A. (2009) · NeuroImage · doi:10.1016/j.neuroimage.2009.04.096 · Cited by 94 · see citing works
Perceptual Set as an Artifact of Recency
EPSTEIN, W. M., & Rock, I. (1960) · The American Journal of Psychology · doi:10.2307/1419898 · Cited by 66 · see citing works
Vanni-Mercier, G., Mauguı̀ere, F., Isnard, J., & Dreher, J. (2009) · Journal of Neuroscience · doi:10.1523/jneurosci.5298-08.2009 · Cited by 50 · see citing works
Ambiguity and uncertainty in Ellsberg and Shackle
Basili, M., & Zappia, C. (2009) · Cambridge Journal of Economics · doi:10.1093/cje/bep008 · Cited by 44 · see citing works
Statistical Models of Conditioning
Dayan, P., & Long, T. W. (1997) · Cited by 34 · see citing works
Learning for non‐stationary Dirichlet processes
Quinn, A., & Kárný, M. (2007) · International Journal of Adaptive Control and Signal Processing · doi:10.1002/acs.949 · Cited by 10 · see citing works
Bayesian Learning in UnstableSettings: Experimental Evidence Based on the Bandit ProblemPreprint
Nestour, É. P. L. (2010) · RePEc: Research Papers in Economics · Cited by 8 · see citing works
Bayesian Learning in an Unstable World: Experimental Evidence Based on the Bandit ProblemPreprint
Payzan-LeNestour, É. (2012) · SSRN Electronic Journal · doi:10.2139/ssrn.1628657 · Cited by 6 · see citing works
Self-citation
Flagged first, then by citations
2 of the 52 references couldn’t be matched to a known paper.
Cited by
Sorted by citations
Neural computations underlying arbitration between model-based and model-free learning
Lee, S. W., Shimojo, S., & O’Doherty, J. P. (2014) · Neuron · doi:10.1016/j.neuron.2013.11.028 · Cited by 642 · see citing works
Humans use directed and random exploration to solve the explore–exploit dilemma.
Wilson, R. C., Geana, A., White, J. M., Ludvig, E. A., & Cohen, J. D. (2014) · Journal of Experimental Psychology: General · doi:10.1037/a0038199 · Cited by 580 · see citing works
Uncertainty in perception and the Hierarchical Gaussian Filter
Mathys, C. D., Lomakina, E. I., Daunizeau, J., Iglesias, S., Brodersen, K. H., Friston, K. J., & Stephan, K. E. (2014) · Frontiers in Human Neuroscience · doi:10.3389/fnhum.2014.00825 · Cited by 465 · see citing works
PhysiCell: An open source physics-based cell simulator for 3-D multicellular systems
Ghaffarizadeh, A., Heiland, R. W., Friedman, S. H., Mumenthaler, S. M., & Macklin, P. (2018) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1005991 · Cited by 456 · see citing works
Confidence as Bayesian probability: From neural origins to behavior
Meyniel, F., Sigman, M., & Mainen, Z. F. (2015) · Neuron · doi:10.1016/j.neuron.2015.09.039 · Cited by 349 · see citing works
Computations of uncertainty mediate acute stress responses in humans
de Berker, A. O., Rutledge, R. B., Mathys, C., Marshall, L., Cross, G. F., Dolan, R. J., & Bestmann, S. (2016) · Nature Communications · doi:10.1038/ncomms10996 · Cited by 330 · see citing works
Emotional Valence and the Free-Energy Principle
Joffily, M., & Coricelli, G. (2013) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1003094 · Cited by 322 · see citing works
Bayesian approaches to autism: Towards volatility, action, and behavior.
Palmer, C. J., Lawson, R. P., & Hohwy, J. (2017) · Psychological Bulletin · doi:10.1037/bul0000097 · Cited by 307 · see citing works
Computational psychiatry: towards a mathematically informed understanding of mental illness
Adams, R. A., Huys, Q. J. M., & Roiser, J. P. (2015) · Journal of Neurology, Neurosurgery & Psychiatry · doi:10.1136/jnnp-2015-310737 · Cited by 285 · see citing works
Rostrolateral Prefrontal Cortex and Individual Differences in Uncertainty-Driven Exploration
Badre, D., Doll, B. B., Long, N. M., & Frank, M. J. (2012) · Neuron · doi:10.1016/j.neuron.2011.12.025 · Cited by 309 · see citing works
Frontal Theta Reflects Uncertainty and Unexpectedness during Exploration and Exploitation
Cavanagh, J. F., y Figueroa, C. M. L. F., Cohen, M. X., & Frank, M. J. (2011) · Cerebral Cortex · doi:10.1093/cercor/bhr332 · Cited by 294 · see citing works
Adaptive learning under expected and unexpected uncertainty
Soltani, A., & Izquierdo, A. (2019) · Nature reviews. Neuroscience · doi:10.1038/s41583-019-0180-y · Cited by 286 · see citing works
Computational approaches to psychiatry
Stephan, K. E., & Mathys, C. (2014) · Current Opinion in Neurobiology · doi:10.1016/j.conb.2013.12.007 · Cited by 260 · see citing works
Deconstructing the human algorithms for exploration
Gershman, S. J. (2018) · Cognition · doi:10.1016/j.cognition.2017.12.014 · Cited by 267 · see citing works
The neural representation of unexpected uncertainty during value-based decision making
Payzan-LeNestour, E., Dunne, S., Bossaerts, P., & O’Doherty, J. P. (2013) · Neuron · doi:10.1016/j.neuron.2013.04.037 · Cited by 257 · see citing works
Self-citationAttention, learning, and the value of information
Gottlieb, J. (2012) · Neuron · doi:10.1016/j.neuron.2012.09.034 · Cited by 241 · see citing works
Modeling Trait Anxiety: From Computational Processes to Personality
Raymond, J. G., Steele, J. D., & Seriès, P. (2017) · Frontiers in Psychiatry · doi:10.3389/fpsyt.2017.00001 · Cited by 203 · see citing works
Deciding Fast and Slow: The Role of Cognitive Biases in AI-assisted Decision-making
Rastogi, C., Zhang, Y., Wei, D., Varshney, K. R., Dhurandhar, A., & Tomsett, R. (2022) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3512930 · Cited by 216 · see citing works
Pharmacological Fingerprints of Contextual Uncertainty
Marshall, L., Mathys, C., Ruge, D., de Berker, A. O., Dayan, P., Stephan, K. E., & Bestmann, S. (2016) · PLOS Biology · doi:10.1371/journal.pbio.1002575 · Cited by 158 · see citing works
Serotonin neurons modulate learning rate through uncertainty
Grossman, C. D., Bari, B. A., & Cohen, J. Y. (2022) · Current Biology · doi:10.1016/j.cub.2021.12.006 · Cited by 164 · see citing works
The Sense of Confidence during Probabilistic Learning: A Normative Account
Meyniel, F., Schlunegger, D., & Dehaene, S. (2015) · PLOS Computational Biology · doi:10.1371/journal.pcbi.1004305 · Cited by 163 · see citing works
Corcoran, A. W., Pezzulo, G., & Hohwy, J. (2020) · Biology & Philosophy · doi:10.1007/s10539-020-09746-2 · Cited by 161 · see citing works
Social learning strategies regulate the wisdom and madness of interactive crowds
Toyokawa, W., Whalen, A., & Laland, K. N. (2019) · Nature Human Behaviour · doi:10.1038/s41562-018-0518-x · Cited by 145 · see citing works
Cavanagh, J. F. (2015) · NeuroImage · doi:10.1016/j.neuroimage.2015.02.007 · Cited by 148 · see citing works
Gershman, S. J. (2018) · Decision · doi:10.1037/dec0000101 · Cited by 146 · see citing works
Paranoia as a deficit in non-social belief updating
Reed, E., Uddenberg, S., Suthaharan, P., Mathys, C. D., Taylor, J. R., Groman, S. M., & Corlett, P. R. (2020) · eLife · doi:10.7554/elife.56345 · Cited by 135 · see citing works
Different Varieties of Uncertainty in Human Decision-Making
Bland, A. R., & Schaefer, A. (2012) · Frontiers in Neuroscience · doi:10.3389/fnins.2012.00085 · Cited by 126 · see citing works
O’Doherty, J. P. (2014) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2014.03.027 · Cited by 129 · see citing works
Making predictions in a changing world—inference, uncertainty, and learning
O’Reilly, J. X. (2013) · Frontiers in Neuroscience · doi:10.3389/fnins.2013.00105 · Cited by 126 · see citing works
Nassar, M. R., Bruckner, R., Gold, J. I., Li, S., Heekeren, H. R., & Eppinger, B. (2016) · Nature Communications · doi:10.1038/ncomms11609 · Cited by 123 · see citing works
Arousal-related adjustments of perceptual biases optimize perception in dynamic environments
Krishnamurthy, K., Nassar, M. R., Sarode, S., & Gold, J. I. (2017) · Nature Human Behaviour · doi:10.1038/s41562-017-0107 · Cited by 117 · see citing works
Boorman, E. D., Rajendran, V. G., O’Reilly, J. X., & Behrens, T. E. J. (2016) · Neuron · doi:10.1016/j.neuron.2016.02.014 · Cited by 117 · see citing works
Heinz, A., Murray, G. K., Schlagenhauf, F., Sterzer, P., Grace, A. A., & Waltz, J. A. (2018) · Schizophrenia Bulletin · doi:10.1093/schbul/sby154 · Cited by 118 · see citing works
Morriss, J., Zuj, D. V., & Mertens, G. (2021) · International Journal of Psychophysiology · doi:10.1016/j.ijpsycho.2021.05.011 · Cited by 108 · see citing works
Cortical coupling reflects Bayesian belief updating in the deployment of spatial attention
Vossel, S., Mathys, C., Stephan, K. E., & Friston, K. J. (2015) · The Journal of Neuroscience · doi:10.1523/jneurosci.1382-15.2015 · Cited by 99 · see citing works
Theory of Choice in Bandit, Information Sampling and Foraging Tasks
Averbeck, B. B. (2015) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1004164 · Cited by 105 · see citing works
Neural Computations Mediating One-Shot Learning in the Human Brain
Lee, S. W., O’Doherty, J. P., & Shinsuke Shimojo, A. (2015) · PLoS Biology · doi:10.1371/journal.pbio.1002137 · Cited by 102 · see citing works
Metaplasticity as a Neural Substrate for Adaptive Learning and Choice under Uncertainty
Farashahi, S. G., Donahue, C. H., Khorsand, P., Seo, H., Lee, D., & Soltani, A. (2017) · Neuron · doi:10.1016/j.neuron.2017.03.044 · Cited by 98 · see citing works
Dissociable neural correlates of uncertainty underlie different exploration strategies
Tomov, M. S., Truong, V. Q., Hundia, R., & Gershman, S. J. (2020) · Nature Communications · doi:10.1038/s41467-020-15766-z · Cited by 101 · see citing works
Levy, I., & Schiller, D. (2021) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2020.11.007 · Cited by 95 · see citing works
Novelty and Inductive Generalization in Human Reinforcement Learning
Gershman, S., & Niv, Y. (2015) · Topics in Cognitive Science · doi:10.1111/tops.12138 · Cited by 102 · see citing works
A simple model for learning in volatile environments
Piray, P., & Daw, N. D. (2020) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1007963 · Cited by 104 · see citing works
Anxiety Impedes Adaptive Social Learning Under Uncertainty
Lamba, A., Frank, M. J., & FeldmanHall, O. (2020) · Psychological Science · doi:10.1177/0956797620910993 · Cited by 90 · see citing works
The structure of reinforcement-learning mechanisms in the human brain
O’Doherty, J. P., Lee, S. W., & McNamee, D. (2014) · Current Opinion in Behavioral Sciences · doi:10.1016/j.cobeha.2014.10.004 · Cited by 97 · see citing works
Franklin, N. T., & Frank, M. J. (2015) · eLife · doi:10.7554/elife.12029 · Cited by 88 · see citing works
Predictive Models for Educational Purposes: A Systematic Review
Almalawi, A., Soh, B., Li, A., & Samra, H. E. (2024) · Big Data and Cognitive Computing · doi:10.3390/bdcc8120187 · Cited by 91 · see citing works
Vinckier, F., Gaillard, R., Palminteri, S., Rigoux, L., Salvador, A., Fornito, A., Adapa, R., Krebs, M. O., Pessiglione, M., & Fletcher, P. C. (2016) · Molecular Psychiatry · doi:10.1038/mp.2015.73 · Cited by 87 · see citing works
Learning to Represent a Multi-Context Environment: More than Detecting Changes
Qian, T., Jaeger, T. F., & Aslin, R. Ν. (2012) · Frontiers in Psychology · doi:10.3389/fpsyg.2012.00228 · Cited by 87 · see citing works
Computational models of adaptive behavior and prefrontal cortex
Soltani, A., & Koechlin, É. (2021) · Neuropsychopharmacology · doi:10.1038/s41386-021-01123-1 · Cited by 85 · see citing works
Kwisthout, J., Bekkering, H., & van Rooij, I. (2017) · Brain and Cognition · doi:10.1016/j.bandc.2016.02.008 · Cited by 73 · see citing works
Balancing new against old information: the role of puzzlement surprise in learning
Faraji, M., Preuschoff, K., & Gerstner, W. (2018) · Neural Computation · doi:10.1162/neco_a_01025 · Cited by 68 · see citing works
Scaling prediction errors to reward variability benefits error-driven learning in humans
Diederen, K. M. J., & Schultz, W. (2015) · Journal of Neurophysiology · doi:10.1152/jn.00483.2015 · Cited by 69 · see citing works
Prediction and perception: Insights for (and from) tinnitus
Hullfish, J., Sedley, W., & Vanneste, S. (2019) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2019.04.008 · Cited by 68 · see citing works
Dezza, I. C., Yu, A. J., Cleeremans, A., & Alexander, W. (2017) · Scientific Reports · doi:10.1038/s41598-017-17237-w · Cited by 67 · see citing works
Neural evidence for Bayesian trial-by-trial adaptation on the N400 during semantic priming
Delaney-Busch, N., Morgan, E., Lau, E. F., & Kuperberg, G. R. (2019) · Cognition · doi:10.1016/j.cognition.2019.01.001 · Cited by 68 · see citing works
Why and how the brain weights contributions from a mixture of experts
O’Doherty, J. P., Lee, S. W., Tadayonnejad, R., Cockburn, J., Iigaya, K., & Charpentier, C. J. (2021) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2020.10.022 · Cited by 67 · see citing works
What makes the dorsomedial frontal cortex active during reading the mental states of others?
Isoda, M., & Noritake, A. (2013) · Frontiers in Neuroscience · doi:10.3389/fnins.2013.00232 · Cited by 64 · see citing works
Dezza, I. C., Cleeremans, A., & Alexander, W. (2019) · Journal of Experimental Psychology General · doi:10.1037/xge0000546 · Cited by 62 · see citing works
Nicotinic receptors in the ventral tegmental area promote uncertainty-seeking
Naudé, J., Tolu, S., Dongelmans, M., Torquet, N., Valverde, S., Rodriguez, G., Pons, S., Maskos, U., Mourot, A., Marti, F., et al. (2016) · Nature Neuroscience · doi:10.1038/nn.4223 · Cited by 61 · see citing works
Boredom, Information-Seeking and Exploration.
Geana, A., Wilson, R. C., Daw, N. D., & Cohen, J. (2016) · eScholarship (California Digital Library) · Cited by 60 · see citing works
A Decision Architecture for Safety Computations
Tashjian, S. M., Zbozinek, T. D., & Mobbs, D. (2021) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2021.01.013 · Cited by 60 · see citing works
de Boer, L., Axelsson, J., Riklund, K., Nyberg, L. H., Dayan, P., Bäckman, L., & Guitart‐Masip, M. (2017) · eLife · doi:10.7554/elife.26424 · Cited by 57 · see citing works
Computational Models Reveal a Passive Mechanism for Cell Migration in the Crypt
Dunn, S.-J., Näthke, I. S., & Osborne, J. M. (2013) · PLoS ONE · doi:10.1371/journal.pone.0080516 · Cited by 55 · see citing works
Payzan-LeNestour, É., & Bossaerts, P. L. (2012) · Frontiers in Neuroscience · doi:10.3389/fnins.2012.00150 · Cited by 54 · see citing works
Self-citationConfidence resets reveal hierarchical adaptive learning in humans
Heilbron, M., & Meyniel, F. (2019) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1006972 · Cited by 52 · see citing works
Rungskunroch, P., Jack, A., & Kaewunruen, S. (2021) · Reliability Engineering & System Safety · doi:10.1016/j.ress.2021.107684 · Cited by 53 · see citing works
Neural Mechanisms of Updating under Reducible and Irreducible Uncertainty
Kobayashi, K., & Hsu, M. (2017) · Journal of Neuroscience · doi:10.1523/jneurosci.0535-17.2017 · Cited by 50 · see citing works
It’s new, but is it good? How generalization and uncertainty guide the exploration of novel options.
Stojić, H., Schulz, E., Analytis, P. P., & Speekenbrink, M. (2020) · Journal of Experimental Psychology General · doi:10.1037/xge0000749 · Cited by 52 · see citing works
Transdiagnostic computations of uncertainty: towards a new lens on intolerance of uncertainty
Sandhu, T. R., Xiao, B., & Lawson, R. P. (2023) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2023.105123 · Cited by 51 · see citing works
Dunn, S.-J., Appleton, P. L., Nelson, S. A., Näthke, I. S., Gavaghan, D. J., & Osborne, J. M. (2012) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1002515 · Cited by 48 · see citing works
Brain Systems for Probabilistic and Dynamic Prediction: Computational Specificity and Integration
O’Reilly, J. X., Jbabdi, S., Rushworth, M. F. S., & Behrens, T. E. J. (2013) · PLoS Biology · doi:10.1371/journal.pbio.1001662 · Cited by 47 · see citing works
Neural Mechanisms Behind Identification of Leptokurtic Noise and Adaptive Behavioral Response
d’Acremont, M., & Bossaerts, P. L. (2016) · Cerebral Cortex · doi:10.1093/cercor/bhw013 · Cited by 50 · see citing works
Queirazza, F., Fouragnan, E. F., Steele, J. D., Cavanagh, J., & Philiastides, M. G. (2019) · Science Advances · doi:10.1126/sciadv.aav4962 · Cited by 47 · see citing works
Imprecise neural computations as a source of adaptive behaviour in volatile environments
Findling, C., Chopin, N., & Koechlin, É. (2020) · Nature Human Behaviour · doi:10.1038/s41562-020-00971-z · Cited by 46 · see citing works
Aberg, K. C., Toren, I., & Paz, R. (2021) · Molecular Psychiatry · doi:10.1038/s41380-021-01363-z · Cited by 44 · see citing works
Neural surprise in somatosensory Bayesian learning
Gijsen, S. C. J., Grundei, M., Lange, R. T., Ostwald, D., & Blankenburg, F. (2021) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1008068 · Cited by 44 · see citing works
Brown, V. M., Hallquist, M. N., Frank, M. J., & Dombrovski, A. Y. (2022) · Cognition · doi:10.1016/j.cognition.2022.105233 · Cited by 43 · see citing works
Toyokawa, W., Saito, Y., & Kameda, T. (2016) · Evolution and Human Behavior · doi:10.1016/j.evolhumbehav.2016.11.001 · Cited by 42 · see citing works
Lombard, E., Kometer, M., & Preuschoff, K. (2017) · Organizational Research Methods · doi:10.1177/1094428117730891 · Cited by 49 · see citing works
Brown, V. M., Price, R. B., & Dombrovski, A. Y. (2023) · Cognitive Affective & Behavioral Neuroscience · doi:10.3758/s13415-023-01080-w · Cited by 40 · see citing works
Prior probabilities modulate cortical surprise responses: A study of event-related potentials
Seer, C., Lange, F., Boos, M., Dengler, R., & Kopp, B. (2016) · Brain and Cognition · doi:10.1016/j.bandc.2016.04.011 · Cited by 39 · see citing works
Stolyarova, A., & Izquierdo, A. (2017) · eLife · doi:10.7554/elife.27483 · Cited by 39 · see citing works
Neural and psychophysiological markers of intolerance of uncertainty.
Morriss, J., Abend, R., Zika, O., Bradford, D. E., & Mertens, G. (2023) · International Journal of Psychophysiology · doi:10.1016/j.ijpsycho.2023.01.003 · Cited by 38 · see citing works
d’Acremont, M., Fornari, E., & Bossaerts, P. L. (2013) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1002895 · Cited by 34 · see citing works
The Predictive Processing Hypothesis
Hohwy, J. (2018) · Oxford University Press eBooks · doi:10.1093/oxfordhb/9780198735410.013.7 · Cited by 34 · see citing works
Uncertainty in learning, choice, and visual fixation
Stojić, H., Orquin, J. L., Dayan, P., Dolan, R. J., & Speekenbrink, M. (2020) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1911348117 · Cited by 32 · see citing works
Neurons in human pre-supplementary motor area encode key computations for value-based choice
Aquino, T. G., Cockburn, J., Mamelak, A. N., Rutishauser, U., & O’Doherty, J. P. (2023) · Nature Human Behaviour · doi:10.1038/s41562-023-01548-2 · Cited by 33 · see citing works
Human noise blindness drives suboptimal cognitive inference
Castañón, S. H., Moran, R., Ding, J., Egner, T., Bang, D., & Summerfield, C. (2019) · Nature Communications · doi:10.1038/s41467-019-09330-7 · Cited by 34 · see citing works
A Normative Theory of Forgetting: Lessons from the Fruit Fly
Brea, J., Urbanczik, R., & Senn, W. (2014) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1003640 · Cited by 31 · see citing works
Nonparametric learning rules from bandit experiments: The eyes have it!
Hu, Y., Kayaba, Y., & Shum, M. (2013) · Games and Economic Behavior · doi:10.1016/j.geb.2013.05.003 · Cited by 28 · see citing works
The Manager’s Notepad: Working Memory, Exploration, and Performance
Martinez, D. L., Brusoni, S., Tata, A., & Zollo, M. (2019) · Journal of Management Studies · doi:10.1111/joms.12528 · Cited by 29 · see citing works
Neural responses in macaque prefrontal cortex are linked to strategic exploration
Jahn, C. I., Grohn, J., Cuell, S. F., Emberton, A., Bouret, S., Walton, M. E., Kolling, N., & Sallet, J. (2023) · PLoS Biology · doi:10.1371/journal.pbio.3001985 · Cited by 28 · see citing works
Optimal structure of metaplasticity for adaptive learning
Khorsand, P., & Soltani, A. (2017) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1005630 · Cited by 26 · see citing works
Inference and Search on Graph-Structured Spaces
Wu, C. M., Schulz, E., & Gershman, S. J. (2020) · Computational Brain & Behavior · doi:10.1007/s42113-020-00091-x · Cited by 26 · see citing works
Empowerment contributes to exploration behaviour in a creative video game
Brändle, F., Stocks, L. J., Tenenbaum, J. B., Gershman, S. J., & Schulz, E. (2023) · Nature Human Behaviour · doi:10.1038/s41562-023-01661-2 · Cited by 34 · see citing works
Valence and magnitude ambiguity in feedback processing
Gu, R., Feng, X., Broster, L. S., Lu, Y., Xu, P., & Luo, Y. (2017) · Brain and Behavior · doi:10.1002/brb3.672 · Cited by 24 · see citing works
Schadenberg, B. R., Reidsma, D., Heylen, D., & Evers, V. (2021) · ACM Transactions on Human-Robot Interaction · doi:10.1145/3461534 · Cited by 25 · see citing works
Causal Role of the Dorsolateral Prefrontal Cortex in Belief Updating under Uncertainty
Schulreich, S., & Schwabe, L. (2020) · Cerebral Cortex · doi:10.1093/cercor/bhaa219 · Cited by 24 · see citing works
Generic properties of a computational task predict human effort and performance
Franco, J. P., Yadav, N., Bossaerts, P. L., & Murawski, C. (2021) · Journal of Mathematical Psychology · doi:10.1016/j.jmp.2021.102592 · Cited by 18 · see citing works
Chasing Unknown Bandits: Uncertainty Guidance in Learning and Decision Making
Speekenbrink, M. (2022) · Current Directions in Psychological Science · doi:10.1177/09637214221105051 · Cited by 23 · see citing works
- Authors
- Payzan-LeNestour, E., & Bossaerts, P.
- Affiliations
- California Institute of Technology (US)
- ORCID
- Bossaerts, Peter
- Venue
- PLoS Computational Biology
- Publisher
- Public Library of Science (PLoS)
- Published
- 2011-01-20
- Volume
- 7
- Issue
- 1
- Pages
- e1001048
- ISSN
- 1553-734X (ISSN-L)
- 1553-7358
- PMC
- PMC3024253
- OpenAlex
- W2027554764
- Cited by
- 267
- References
- 52
- Citations by year
- 2026: 10 · 2025: 31 · 2024: 24 · 2023: 16 · 2022: 16 · 2021: 18 · 2020: 22 · 2019: 22 · 2018: 19 · 2017: 20 · 2016: 19 · 2015: 18 · 2014: 11 · 2013: 12 · 2012: 8
- FWCI
- 5.59
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Locations
- PLoS Computational Biology (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- Minerva Access (University of Melbourne) (submittedVersion, cc-by)
- CaltechAUTHORS (California Institute of Technology) (submittedVersion, cc-by)
- RePEc: Research Papers in Economics (submittedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Figshare (submittedVersion, cc-by)
- Infoscience (Ecole Polytechnique Fédérale de Lausanne) (submittedVersion)
- Infoscience (Ecole Polytechnique Fédérale de Lausanne) (submittedVersion, cc-by-nc-nd)
- Funders
- National Science Foundation
- Swiss Finance Institute
- Schweizerischer Nationalfonds zur Förderung der Wissenschaftlichen Forschung
- Topics
- Social Sciences › Decision Sciences › General Decision Sciences › Decision-Making and Behavioral Economics
- Life Sciences › Neuroscience › Cognitive Neuroscience › Neural and Behavioral Psychology Studies
- Social Sciences › Decision Sciences › Management Science and Operations Research › Advanced Bandit Algorithms Research
- Keywords
- Bayesian learning, restless bandits, risk, estimation uncertainty, ambiguity, structural uncertainty, exploration, learning rate, model-free reinforcement learning, human decision-making, perceived uncertainty, ambiguity aversion
- MeSH terms
- Bayes Theorem, Probability, Risk, Uncertainty
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
