CoChat
Sign Up
Cited by 346Neural and Behavioral Psychology StudiesNeurotransmitter Receptor Influence on BehaviorReinforcement Learning in Robotics

Speed/accuracy trade-off between the habitual and the goal-directed processes

Open access
Keramati, Mehdi, Dezfouli, Amir, Piray, Payam·2011·PLoS Computational Biology
DOI 10.1371/journal.pcbi.1002055PMC PMC3102758
Open this paper in CoChat — free accountOpen PDF

Abstract

Instrumental responses are hypothesized to be of two kinds: habitual and goal-directed, mediated by the sensorimotor and the associative cortico-basal ganglia circuits, respectively. The existence of the two heterogeneous associative learning mechanisms can be hypothesized to arise from the comparative advantages that they have at different stages of learning. In this paper, we assume that the goal-directed system is behaviourally flexible, but slow in choice selection. The habitual system, in contrast, is fast in responding, but inflexible in adapting its behavioural strategy to new conditions. Based on these assumptions and using the computational theory of reinforcement learning, we propose a normative model for arbitration between the two processes that makes an approximately optimal balance between search-time and accuracy in decision making. Behaviourally, the model can explain experimental evidence on behavioural sensitivity to outcome at the early stages of learning, but insensitivity at the later stages. It also explains that when two choices with equal incentive values are available concurrently, the behaviour remains outcome-sensitive, even after extensive training. Moreover, the model can explain choice reaction time variations during the course of learning, as well as the experimental observation that as the number of choices increases, the reaction time also increases. Neurobiologically, by assuming that phasic and tonic activities of midbrain dopamine neurons carry the reward prediction error and the average reward signals used by the model, respectively, the model predicts that whereas phasic dopamine indirectly affects behaviour through reinforcing stimulus-response associations, tonic dopamine can directly affect behaviour through manipulating the competition between the habitual and the goal-directed systems and thus, affect reaction time.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1371/journal.pcbi.1002055

Cites

83 of 83 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • More recent formulations emphasize cooperation, rather than only competition, between behavioural controllers.

    “More recent formulations emphasize cooperation, rather than only competition (Daw and Dayan, 2014, Dolan and Dayan, 2013, Keramati et al., 2011, O’Doherty et al., 2015, Pezzulo et al., 2013; see also Daw et al., 2011, Lee et al., 2014) for relevant empirical evidence that supports an integrative viewpoint.”

    From Active Inference, homeostatic regulation and adaptive behavioural control (2015) · cites this paper as Keramati et al., 2011

    In Discussion > Future Directions, the paper explicitly says that the only interaction in its proposed model is competition. It discusses collaboration as a possible future extension, not as a feature emphasized by its formulation. (LLM, conf=0.88)

    Checked by a reasoning model — may be wrong.

    Experimental — accuracy not yet measured

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Flagged first, then by citations

Research works printed but not linked: 2

  1. [2]Dickinson A Balleine BW 2002 The role of learning in motivation. Gallistel CR Steven's Handbook of Experimental Psychology: Learning, Motivation, and Emotion New York Wiley 497 533 Volume 3 3rd edition
  2. [56]Simon H 1982 Volume 2, Models of bounded rationality. Cambridge MIT Press

Cited by

Sorted by citations

Related

Authors
Keramati, M., Dezfouli, A., & Piray, P.
Affiliations
  • Sharif University of Technology (IR)
  • University of Tehran (IR)
Corresponding author
Keramati, Mehdi
ORCID
Venue
PLoS Computational Biology
Publisher
Public Library of Science (PLoS)
Published
2011-05-26
Volume
7
Issue
5
Pages
e1002055
ISSN
  • 1553-734X (ISSN-L)
  • 1553-7358
DOI
10.1371/journal.pcbi.1002055
PMC
PMC3102758
OpenAlex
W2080834410
Cited by
346
References
82
Citations by year
2026: 11 · 2025: 23 · 2024: 22 · 2023: 23 · 2022: 15 · 2021: 39 · 2020: 28 · 2019: 31 · 2018: 28 · 2017: 29 · 2016: 22 · 2015: 26 · 2014: 21 · 2013: 13 · 2012: 11
FWCI
7.96
Type
journal article
Language
en
Open access
Licence
Locations
  • PLoS Computational Biology (publishedVersion, cc-by)
  • PubMed Central (publishedVersion, cc-by)
  • PubMed (publishedVersion)
  • City Research Online (City University London) (acceptedVersion)
  • RePEc: Research Papers in Economics (submittedVersion)
  • DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
Topics
  • Life Sciences › Neuroscience › Cognitive Neuroscience › Neural and Behavioral Psychology Studies
  • Life Sciences › Neuroscience › Cellular and Molecular Neuroscience › Neurotransmitter Receptor Influence on Behavior
  • Physical Sciences › Computer Science › Artificial Intelligence › Reinforcement Learning in Robotics
Keywords
habitual behavior, goal-directed behavior, speed-accuracy trade-off, reinforcement learning, cortico-basal ganglia circuits, sensorimotor circuits, decision-making, choice reaction time, outcome sensitivity, dopamine, phasic dopamine, tonic dopamine, reward prediction error, average reward, associative learning, behavioral flexibility
MeSH terms
Algorithms, Animals, Behavior, Animal, Choice Behavior, Computer Simulation, Decision Making, Dopamine, Goals, Humans, Learning, Markov Chains, Models, Neurological, Neurons, Reinforcement, Psychology, Reproducibility of Results, Maze Learning, Rats

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review