CoChat
Sign Up
Cited by 267Decision-Making and Behavioral EconomicsNeural and Behavioral Psychology StudiesAdvanced Bandit Algorithms Research

The DOI matches its Crossref record; 50 of the 52 references it cites were found in a registry or an index.

Risk, Unexpected Uncertainty, and Estimation Uncertainty: Bayesian Learning in Unstable Settings

Open access
Payzan-LeNestour, Elise, Bossaerts, Peter·2011·PLoS Computational Biology
DOI 10.1371/journal.pcbi.1001048PMC PMC3024253
Open this paper in CoChat — free accountOpen PDF

Abstract

Recently, evidence has emerged that humans approach learning using Bayesian updating rather than (model-free) reinforcement algorithms in a six-arm restless bandit problem. Here, we investigate what this implies for human appreciation of uncertainty. In our task, a Bayesian learner distinguishes three equally salient levels of uncertainty. First, the Bayesian perceives irreducible uncertainty or risk: even knowing the payoff probabilities of a given arm, the outcome remains uncertain. Second, there is (parameter) estimation uncertainty or ambiguity: payoff probabilities are unknown and need to be estimated. Third, the outcome probabilities of the arms change: the sudden jumps are referred to as unexpected uncertainty. We document how the three levels of uncertainty evolved during the course of our experiment and how it affected the learning rate. We then zoom in on estimation uncertainty, which has been suggested to be a driving force in exploration, in spite of evidence of widespread aversion to ambiguity. Our data corroborate the latter. We discuss neural evidence that foreshadowed the ability of humans to distinguish between the three levels of uncertainty. Finally, we investigate the boundaries of human capacity to implement Bayesian learning. We repeat the experiment with different instructions, reflecting varying levels of structural uncertainty. Under this fourth notion of uncertainty, choices were no better explained by Bayesian updating than by (model-free) reinforcement learning. Exit questionnaires revealed that participants remained unaware of the presence of unexpected uncertainty and failed to acquire the right model with which to implement Bayesian updating.

Overview

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1371/journal.pcbi.1001048

Cites

50 of 50 references verified0 citation errors · 0 not checkable

Statistics

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Flagged first, then by citations

Show all 52 references

2 of the 52 references couldn’t be matched to a known paper.

Cited by

Sorted by citations

Items 201–265 of 265

Related

See all

Metadata

Authors
Payzan-LeNestour, E., & Bossaerts, P.
Affiliations
California Institute of Technology (US)
ORCID
Bossaerts, Peter
Venue
PLoS Computational Biology
Publisher
Public Library of Science (PLoS)
Published
2011-01-20
Volume
7
Issue
1
Pages
e1001048
ISSN
  • 1553-734X (ISSN-L)
  • 1553-7358
DOI
10.1371/journal.pcbi.1001048
PMC
PMC3024253
OpenAlex
W2027554764
Cited by
267
References
52
Citations by year
2026: 10 · 2025: 31 · 2024: 24 · 2023: 16 · 2022: 16 · 2021: 18 · 2020: 22 · 2019: 22 · 2018: 19 · 2017: 20 · 2016: 19 · 2015: 18 · 2014: 11 · 2013: 12 · 2012: 8
FWCI
5.59
Type
journal article
Language
en
Open access
Licence
Locations
  • PLoS Computational Biology (publishedVersion, cc-by)
  • PubMed Central (publishedVersion, cc-by)
  • PubMed (publishedVersion)
  • Minerva Access (University of Melbourne) (submittedVersion, cc-by)
  • CaltechAUTHORS (California Institute of Technology) (submittedVersion, cc-by)
  • RePEc: Research Papers in Economics (submittedVersion)
  • DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
  • Figshare (submittedVersion, cc-by)
  • Infoscience (Ecole Polytechnique Fédérale de Lausanne) (submittedVersion)
  • Infoscience (Ecole Polytechnique Fédérale de Lausanne) (submittedVersion, cc-by-nc-nd)
Funders
  • National Science Foundation
  • Swiss Finance Institute
  • Schweizerischer Nationalfonds zur Förderung der Wissenschaftlichen Forschung
Topics
  • Social Sciences › Decision Sciences › General Decision Sciences › Decision-Making and Behavioral Economics
  • Life Sciences › Neuroscience › Cognitive Neuroscience › Neural and Behavioral Psychology Studies
  • Social Sciences › Decision Sciences › Management Science and Operations Research › Advanced Bandit Algorithms Research
Keywords
Bayesian learning, restless bandits, risk, estimation uncertainty, ambiguity, structural uncertainty, exploration, learning rate, model-free reinforcement learning, human decision-making, perceived uncertainty, ambiguity aversion
MeSH terms
Bayes Theorem, Probability, Risk, Uncertainty

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review