CoChat

Dopamine, Inference, and Uncertainty

Open access
Gershman, Samuel J. · 2017 · Neural Computation

Published version of a preprint

The DOI matches its Crossref record; 66 of the 66 references it cites were found in a registry or an index.

Abstract

The hypothesis that the phasic dopamine response reports a reward prediction error has become deeply entrenched. However, dopamine neurons exhibit several notable deviations from this hypothesis. A coherent explanation for these deviations can be obtained by analyzing the dopamine response in terms of Bayesian reinforcement learning. The key idea is that prediction errors are modulated by probabilistic beliefs about the relationship between cues and outcomes, updated through Bayesian inference. This account can explain dopamine responses to inferred value in sensory preconditioning, the effects of cue preexposure (latent inhibition), and adaptive coding of prediction errors when rewards vary across orders of magnitude. We further postulate that orbitofrontal cortex transforms the stimulus representation through recurrent dynamics, such that a simple error-driven learning rule operating on the transformed representation can implement the Bayesian reinforcement learning update.

Overview

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1162/neco_a_01023

Cites

66 of 66 references verified0 citation errors · 0 not checkable

Statistics

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Show all 66 references

Research works printed but not linked: 2

  1. Dayan, P. and Kakade, S. (2001). Explaining away in weight space. In Leen, T., Dietterich, T., and Tresp, V., editors, Advances in Neural Information Processing Systems 13 , pages 451-457. MIT Press.
  2. Pearce, J., Kaye, H., and Hall, G. (1982). Predictive accuracy and stimulus associability: Development of a model for Pavlovian learning. Quantitative analyses of behavior , 3:241-255.

Cited by

See all 40

Metadata

Authors
Gershman, S. J.
Affiliations
Harvard University (US)
Corresponding author
Gershman, Samuel J.
Venue
Neural Computation
Publisher
MIT Press
Published
2017-12
Volume
29
Issue
12
Pages
3311-3326
ISSN
  • 0899-7667 (ISSN-L)
  • 1530-888X
OpenAlex
W2951636479
Cited by
40
References
66
Citations by year
2026: 3 · 2025: 1 · 2024: 3 · 2023: 3 · 2022: 7 · 2021: 5 · 2020: 3 · 2019: 5 · 2018: 6 · 2017: 4
FWCI
2.41
Type
journal article
Language
en
Open access
Licence
cc-by-nc-nd
Locations
  • Neural Computation (publishedVersion)
  • bioRxiv (Cold Spring Harbor Laboratory) (acceptedVersion, cc-by-nc-nd)
  • PubMed (publishedVersion)
Funders
  • National Science Foundation
  • China Scholarship Council
Topics
  • Life Sciences › Neuroscience › Cellular and Molecular Neuroscience › Neurotransmitter Receptor Influence on Behavior
  • Life Sciences › Neuroscience › Cognitive Neuroscience › Neural and Behavioral Psychology Studies
  • Life Sciences › Neuroscience › Cognitive Neuroscience › Memory and Neural Mechanisms
Keywords
dopamine, reward prediction error, Bayesian reinforcement learning, Bayesian inference, uncertainty, sensory preconditioning, latent inhibition, orbitofrontal cortex, recurrent dynamics, error-based learning, probabilistic beliefs

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.