CoChat
Sign Up
Cited by 156Neural and Behavioral Psychology StudiesDecision-Making and Behavioral EconomicsReinforcement Learning in Robotics

Reinforcement learning and decision making in monkeys during a competitive game

Lee, Daeyeol, Conroy, Michelle L., McGreevy, Benjamin P., Barraclough, Dominic J.·2004·Cognitive Brain Research
DOI 10.1016/j.cogbrainres.2004.07.007
Open this paper in CoChat — free account

Abstract

Animals living in a dynamic environment must adjust their decision-making strategies through experience. To gain insights into the neural basis of such adaptive decision-making processes, we trained monkeys to play a competitive game against a computer in an oculomotor free-choice task. The animal selected one of two visual targets in each trial and was rewarded only when it selected the same target as the computer opponent. To determine how the animal's decision-making strategy can be affected by the opponent's strategy, the computer opponent was programmed with three different algorithms that exploited different aspects of the animal's choice and reward history. When the computer selected its targets randomly with equal probabilities, animals selected one of the targets more often, violating the prediction of probability matching, and their choices were systematically influenced by the choice history of the two players. When the computer exploited only the animal's choice history but not its reward history, animal's choice became more independent of its own choice history but was still related to the choice history of the opponent. This bias was substantially reduced, but not completely eliminated, when the computer used the choice history of both players in making its predictions. These biases were consistent with the predictions of reinforcement learning, suggesting that the animals sought optimal decision-making strategies using reinforcement learning algorithms.

Integrity signals

Registry

Crossref · 10.1016/j.cogbrainres.2004.07.007

Cites

81 of 81 references verified0 citation errors · 0 not checkable

Statistics

Recomputing p-values needs the paper’s full text.

Claims the paper makes

This paper’s full text isn’t openly available, so its claims and self-check weren’t run. The registry and citation checks still apply.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • A few studies have estimated the time constant of the reward filter from behavior, and assessed how past rewards affect choice selection.

    “So far a few studies have estimated the time constant of this filter from behavior, and assessed how past rewards affect choice selection8-12”

    From A reservoir of time constants for memory traces in cortical neurons (2011) · cites this paper as [11]

  • In the matching pennies task, the animals made their decisions largely on the basis of their reward history.

    “Although the global optimal strategy for the matching pennies task is to choose randomly and therefore does not require memory, the animals made their decisions largely on the basis of their reward history11,19-22.”

    From A reservoir of time constants for memory traces in cortical neurons (2011) · cites this paper as [11]

  • Although the overall performance of monkeys was nearly optimal, their trial-by-trial decisions, locally in time, were influenced by previous rewards and actions.

    “Although the overall performance of monkeys was nearly optimal, their trial-by-trial decisions, locally in time, were influenced by previous rewards and actions11,19-22.”

    From A reservoir of time constants for memory traces in cortical neurons (2011) · cites this paper as [11]

  • Incorporating choice as a covariate in dynamic response-by-response models allows one to isolate changes due to past choices from those due to past reinforcers.

    “Incorporating choice as a covariate in dynamic response-by-response models allows one to isolate these changes from those due to past reinforcers (Lee et al., 2004; Mookherjee & Sopher, 1994).”

    From DYNAMIC RESPONSE-BY-RESPONSE MODELS OF MATCHING BEHAVIOR IN RHESUS MONKEYS (2005) · cites this paper as Lee et al., 2004

Cites

Flagged first, then by citations

Cited by

Sorted by citations

Related

Authors
Lee, D., Conroy, M. L., McGreevy, B. P., & Barraclough, D. J.
Affiliations
University of Rochester (US)
Corresponding author
Lee, Daeyeol
ORCID
Venue
Cognitive Brain Research
Publisher
Elsevier BV
Published
2004-12
Volume
22
Issue
1
Pages
45-58
ISSN
0926-6410 (ISSN-L)
DOI
10.1016/j.cogbrainres.2004.07.007
OpenAlex
W2011971614
Cited by
156
References
81
Citations by year
2026: 8 · 2025: 5 · 2024: 8 · 2023: 5 · 2022: 8 · 2021: 4 · 2020: 4 · 2019: 5 · 2018: 10 · 2017: 4 · 2016: 6 · 2015: 4 · 2014: 5 · 2013: 9 · 2012: 8
FWCI
3.87
Type
journal article
Language
en
Open access
closed
Licence
www.elsevier.com/tdm/userlicense/1.0/ (tdm)
Locations
  • Cognitive Brain Research (publishedVersion)
  • PubMed (publishedVersion)
Funders
  • Foundation for the National Institutes of Health
  • James S. McDonnell Foundation
  • National Eye Institute (P30EY001319)
  • National Institute of Neurological Disorders and Stroke (R01NS044270)
Topics
  • Life Sciences › Neuroscience › Cognitive Neuroscience › Neural and Behavioral Psychology Studies
  • Social Sciences › Decision Sciences › General Decision Sciences › Decision-Making and Behavioral Economics
  • Physical Sciences › Computer Science › Artificial Intelligence › Reinforcement Learning in Robotics
Keywords
reinforcement learning, adaptive decision-making, competitive games, probability matching, opponent modeling, reward history, computer opponents, primate decision making, optimal foraging theory
MeSH terms
Algorithms, Animals, Behavior, Animal, Competitive Behavior, Decision Making, Feedback, Learning, Macaca mulatta, Male, Models, Psychological, Probability, Reinforcement, Psychology, Time Factors, Logistic Models

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review