Reinforcement learning: Computational theory and biological mechanisms
Open accessThe DOI matches its Crossref record; 76 of the 76 references it cites were found in a registry or an index.
Abstract
Reinforcement learning is a computational framework for an active agent to learn behaviors on the basis of a scalar reward signal. The agent can be an animal, a human, or an artificial system such as a robot or a computer program. The reward can be food, water, money, or whatever measure of the performance of the agent. The theory of reinforcement learning, which was developed in an artificial intelligence community with intuitions from animal learning theory, is now giving a coherent account on the function of the basal ganglia. It now serves as the "common language" in which biologists, engineers, and social scientists can exchange their problems and findings. This article reviews the basic theoretical framework of reinforcement learning and discusses its recent and future contributions toward the understanding of animal behaviors and human decision making.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source says what the claim attributes to it.
In the citing paper
“The striatum is ideally suited for such learning because the conditions required for LTP at cortico-striatal synapses closely match the conditions for reinforcement learning [34–36].”
From Cortical and basal ganglia contributions to habit learning and automaticity (2010) · cites this paper as [34]
In the citing paper
“An extensive literature shows that dopamine cells exhibit phasic increases in discharge rate following unexpected primary or conditioned rewards and depress their discharge below baseline following an unexpected failure to receive a reward [34, 36].”
From Cortical and basal ganglia contributions to habit learning and automaticity (2010) · cites this paper as [34]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
A neural substrate of prediction and reward
Schultz, W., Dayan, P., & Montague, P. R. (1997) · Science · doi:10.1126/science.275.5306.1593 · Cited by 9,388
Watkins, C. J. C. H., & Dayan, P. (1992) · Machine Learning · doi:10.1007/bf00992698 · Cited by 9,083 · cited 1×
“Another popular reinforcement learning algorithm is called Q -learning (Watkins, 1989; Watkins and Dayan, 1992), in which the agent learns the so-called action value function Q (cid”
Quantum phase transition from a superfluid to a Mott insulator in a gas of ultracold atoms
Greiner, M., Mandel, O., Esslinger, T., Hänsch, T. W., & Bloch, I. (2002) · Nature · doi:10.1038/415039a · Cited by 5,935
Watkins, C. (1989) · OpenGrey (Institut de l'Information Scientifique et Technique) · doi:10.17863/cam.132164 · Cited by 5,297
Learning to predict by the methods of temporal differences
Sutton, R. S. (1988) · Machine Learning · doi:10.1007/bf00115009 · Cited by 5,148 · cited 2×
“The deviation is called the temporal difference TD error ,o r theTD signal (Barto et al., 1983; Sutton, 1988 )”
Predictive reward signal of dopamine neurons
Schultz, W. (1998) · Journal of Neurophysiology · doi:10.1152/jn.1998.80.1.1 · Cited by 4,564
Planning and acting in partially observable stochastic domains
Kaelbling, L. P., Littman, M. L., & Cassandra, A. R. (1998) · Artificial Intelligence · doi:10.1016/s0004-3702(98)00023-x · Cited by 4,149
Watkins, C. J. C. H., & Dayan, P. (1992) · Machine Learning · doi:10.1023/a:1022676722315 · Cited by 3,907
Neuronlike adaptive elements that can solve difficult learning control problems
Barto, A. G., Sutton, R. S., & Anderson, C. W. (1983) · IEEE Transactions on Systems, Man, and Cybernetics · doi:10.1109/tsmc.1983.6313077 · Cited by 3,233
Barto, A. G. (1998) · IFAC Proceedings Volumes · doi:10.1016/s1474-6670(17)38315-5 · Cited by 2,976
Research works printed but not linked: 4
- Dayan P, 1996, Mach. Learn., 25, 5
- Houk JC, 1995, Models of Information Processing, 249
- Sutton R S, 1988, Mach. Learn., 3, 9
- Thorndike E L, 1898, Psychol. Rev., 2, 1
Cited by
The neural basis of the speed–accuracy tradeoff
Bogacz, R., Wagenmakers, E.-J., Forstmann, B. U., & Nieuwenhuis, S. (2010) · Trends in Neurosciences · doi:10.1016/j.tins.2009.09.002 · Cited by 724
Doya, K. (2008) · Nature Neuroscience · doi:10.1038/nn2077 · Cited by 642
“In the following sections, I first review theoretical models of the other three steps—evaluation, action selection and learning—based on the framework of reinforcement learnin”
BackgroundSelf-citationCortical and basal ganglia contributions to habit learning and automaticity
Ashby, F. G., Turner, B. O., & Horvitz, J. C. (2010) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2010.02.001 · Cited by 481
“An extensive literature shows that dopamine cells exhibit phasic increases in discharge rate following unexpected primary or conditioned rewards and depress their discharge below baseline following an unexpected failure to receive a reward [ 34 , 36].”
BackgroundMethodNeuromodulated Spike-Timing-Dependent Plasticity, and Theory of Three-Factor Learning Rules
Frémaux, N., & Gerstner, W. (2016) · Frontiers in Neural Circuits · doi:10.3389/fncir.2015.00085 · Cited by 413
Foundations of human reasoning in the prefrontal cortex
Donoso, M., Collins, A. G. E., & Koechlin, E. (2014) · Science · doi:10.1126/science.1252254 · Cited by 406
Metadata
- Authors
- Doya, K.
- Affiliations
- Okinawa Institute of Science and Technology Graduate University (JP)
- Corresponding author
- Doya, Kenji
- ORCID
- Doya, Kenji
- Venue
- HFSP journal
- Publisher
- Informa UK Limited
- Published
- 2007-05
- Volume
- 1
- Issue
- 1
- Pages
- 30-40
- ISSN
- 1955-205X (ISSN-L)
- 1955-2068
- PMC
- PMC2645553
- OpenAlex
- W2091789872
- Cited by
- 150
- References
- 76
- Citations by year
- 2026: 7 · 2025: 11 · 2024: 8 · 2023: 7 · 2022: 6 · 2021: 9 · 2020: 6 · 2019: 6 · 2018: 10 · 2017: 4 · 2016: 12 · 2015: 5 · 2014: 13 · 2013: 6 · 2012: 12
- FWCI
- 7.06
- Type
- journal article
- Language
- en
- Open access
- bronze
- doi.org
- Locations
- HFSP journal (publishedVersion)
- PubMed Central (publishedVersion)
- PubMed (publishedVersion)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Reinforcement Learning in Robotics
- Life Sciences › Neuroscience › Cognitive Neuroscience › Neural and Behavioral Psychology Studies
- Life Sciences › Neuroscience › Cellular and Molecular Neuroscience › Neurotransmitter Receptor Influence on Behavior
- Keywords
- reinforcement learning, basal ganglia, reward signals, animal learning, human decision-making, theory of computation, biological mechanisms, artificial intelligence
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Reinforcement learning: Computational theory and biological mechanisms. https://app.cochat.ai/papers/10.2976/1.2732246
- The paper
- Doya, K. (2007). Reinforcement learning: Computational theory and biological mechanisms. HFSP journal. https://doi.org/10.2976/1.2732246
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.