Cited by 673Reinforcement Learning in RoboticsNeural and Behavioral Psychology StudiesPsychiatry, Mental Health, Neuroscience

Reinforcement learning, fast and slow

Open access
Botvinick, Matthew, Ritter, Sam, Wang, Jane X., Kurth-Nelson, Zeb, Blundell, Charles, Hassabis, Demis·2019·Trends in Cognitive Sciences

Abstract

Deep reinforcement learning (RL) methods have driven impressive advances in artificial intelligence in recent years, exceeding human performance in domains ranging from Atari to Go to no-limit poker. This progress has drawn the attention of cognitive scientists interested in understanding human learning. However, the concern has been raised that deep RL may be too sample-inefficient - that is, it may simply be too slow - to provide a plausible model of how humans learn. In the present review, we counter this critique by describing recently developed techniques that allow deep RL to operate more nimbly, solving problems much more quickly than previous methods. Although these techniques were developed in an AI context, we propose that they may have rich implications for psychology and neuroscience. A key insight, arising from these AI methods, concerns the fundamental connection between fast RL and slower, more incremental forms of learning.

Integrity signals

Registry

Crossref · 10.1016/j.tics.2019.02.006

Cites

105 of 105 references verified0 citation errors · 0 not checkable

Statistics

Recomputing p-values needs the paper’s full text.

Claims the paper makes

We couldn’t retrieve this paper’s full text, so its claims and self-check weren’t run. The registry and citation checks still apply.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Cites

Flagged first, then by citations

7 of the 112 references couldn’t be matched to a known paper.

Cited by

Sorted by citations

Authors
Botvinick, M., Ritter, S., Wang, J. X., Kurth-Nelson, Z., Blundell, C., & Hassabis, D.
Affiliations
  • Google DeepMind (United Kingdom) (GB)
  • Princeton University (US)
Corresponding author
Botvinick, Matthew
Venue
Trends in Cognitive Sciences
Publisher
Elsevier BV
Published
2019-05
Volume
23
Issue
5
Pages
408-422
ISSN
1364-6613 (ISSN-L)
OpenAlex
W2938321354
Cited by
667
References
112
Citations by year
2026: 41 · 2025: 85 · 2024: 116 · 2023: 104 · 2022: 120 · 2021: 90 · 2020: 89 · 2019: 21
FWCI
5.26
Type
journal article
Language
en
Open access
Locations
  • Trends in Cognitive Sciences (publishedVersion, cc-by)
  • PubMed (publishedVersion)
Funders
DeepMind
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › Reinforcement Learning in Robotics
  • Life Sciences › Neuroscience › Cognitive Neuroscience › Neural and Behavioral Psychology Studies
  • Physical Sciences › Computer Science › Artificial Intelligence › Psychiatry, Mental Health, Neuroscience
Keywords
deep reinforcement learning, sample efficiency, human learning, fast learning, incremental learning, learning speed, computational models of learning, cognitive psychology, neuroscience of learning
MeSH terms
Animals, Artificial Intelligence, Humans, Reinforcement, Psychology, Time Factors, Neural Networks, Computer, Memory, Episodic

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review