CoChat
Sign Up
Cited by 1,304Neurotransmitter Receptor Influence on BehaviorNeural and Behavioral Psychology StudiesReinforcement Learning in Robotics

Midbrain dopamine neurons encode a quantitative reward prediction error signal

Open access
Bayer, Hannah M., Glimcher, Paul W.·2005·Neuron
DOI 10.1016/j.neuron.2005.05.020PMC PMC1564381
Open this paper in CoChat — free accountOpen PDF

Abstract

The midbrain dopamine neurons are hypothesized to provide a physiological correlate of the reward prediction error signal required by current models of reinforcement learning. We examined the activity of single dopamine neurons during a task in which subjects learned by trial and error when to make an eye movement for a juice reward. We found that these neurons encoded the difference between the current reward and a weighted average of previous rewards, a reward prediction error, but only for outcomes that were better than expected. Thus, the firing rate of midbrain dopamine neurons is quantitatively predicted by theoretical descriptions of the reward prediction error signal used in reinforcement learning models for circumstances in which this signal has a positive value. We also found that the dopamine system continued to compute the reward prediction error even when the behavioral policy of the animal was only weakly influenced by this computation.

Integrity signals

Registry

Crossref · 10.1016/j.neuron.2005.05.020

Cites

34 of 34 references verified0 citation errors · 0 not checkable

Statistics

Recomputing p-values needs the paper’s full text.

Claims the paper makes

We couldn’t retrieve this paper’s full text, so its claims and self-check weren’t run. The registry and citation checks still apply.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Associative learning is driven by the activity of dopamine neurons in the midbrain, where spiking activity resembles the reward prediction error (RPE) signal in temporal difference (TD) learning.

    “one successful theoretical model posits that associative learning is driven by the activity of dopamine neurons in the midbrain, where spiking activity resembles the reward prediction error (RPE) signal in an RL algorithm called temporal difference (TD) learning [1][2][3][4].”

    From Emergence of belief-like representations through reinforcement learning (2023) · cites this paper as [2]

  • Biphasic filtering in dopamine neurons might provide a reward prediction error.

    “It was already noted previously that a biphasic filtering in dopamine neurons might provide a reward prediction error40.”

    From A reservoir of time constants for memory traces in cortical neurons (2011) · cites this paper as [40]

  • Bayer and Glimcher showed that the number of action potentials discharged by individual SNC neurons is proportional to the magnitude of the difference between the expected and obtained reinforcer, supporting the hypothesis that dopamine neurons underlie computation of prediction error.

    “They showed that the number of action potentials discharged by individual SNC neurons is proportional to the magnitude of the difference between the reinforcer expected (vi−1) and the reinforcer actually obtained (ri), thereby lending strong support to the hypothesis that dopamine neurons underlie the computation of prediction error.”

    From DYNAMIC RESPONSE-BY-RESPONSE MODELS OF MATCHING BEHAVIOR IN RHESUS MONKEYS (2005) · cites this paper as Bayer and Glimcher (2005)

  • Bayer and Glimcher (2005) linked SNC activity to reinforcement-rate estimation using the exponentially weighted average.

    “Bayer and Glimcher (2005) linked SNC activity to reinforcement-rate estimation using the exponentially weighted average described above.”

    From DYNAMIC RESPONSE-BY-RESPONSE MODELS OF MATCHING BEHAVIOR IN RHESUS MONKEYS (2005) · cites this paper as Bayer and Glimcher (2005)

  • In reinforcement learning theory, the error signal is postulated to update the reward expectation, which in turn is used to compute the error signal.

    “In reinforcement learning theory, the error signal is postulated to update the reward expectation, which in turn is used to compute the error signal (Sutton and Barto, 1998; Bayer and Glimcher, 2005; Rushworth and Behrens, 2008).”

    From Decision Making in Recurrent Neuronal Circuits (2008) · cites this paper as Bayer and Glimcher, 2005

Cites

Flagged first, then by citations

Cited by

Sorted by citations

Related

Authors
Bayer, H. M., & Glimcher, P. W.
Affiliations
New York University (US)
Corresponding author
Glimcher, Paul W.
ORCID
Venue
Neuron
Publisher
Elsevier BV
Published
2005-07
Volume
47
Issue
1
Pages
129-141
ISSN
0896-6273 (ISSN-L)
DOI
10.1016/j.neuron.2005.05.020
PMC
PMC1564381
OpenAlex
W2119170562
Cited by
1,304
References
34
Citations by year
2026: 46 · 2025: 52 · 2024: 50 · 2023: 41 · 2022: 57 · 2021: 60 · 2020: 108 · 2019: 72 · 2018: 65 · 2017: 67 · 2016: 75 · 2015: 66 · 2014: 77 · 2013: 69 · 2012: 81
FWCI
16.81
Type
journal article
Language
en
Open access
Licence
Locations
  • Neuron (publishedVersion)
  • PubMed (publishedVersion)
  • Europe PMC (PubMed Central) (acceptedVersion)
Funders
  • National Institutes of Health (EY-10536)
  • Israel National Road Safety Authority (MH12790)
Topics
  • Life Sciences › Neuroscience › Cellular and Molecular Neuroscience › Neurotransmitter Receptor Influence on Behavior
  • Life Sciences › Neuroscience › Cognitive Neuroscience › Neural and Behavioral Psychology Studies
  • Physical Sciences › Computer Science › Artificial Intelligence › Reinforcement Learning in Robotics
Keywords
midbrain dopaminergic neurons, reward prediction error, reinforcement learning, single-unit activity, trial-and-error learning, dopaminergic system, behavioral public policy
MeSH terms
Algorithms, Animals, Conditioning, Operant, Cues, Dopamine, Macaca mulatta, Male, Mesencephalon, Neurons, Photic Stimulation, Reinforcement Schedule, Reward, Saccades

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review