Oculomotor learning revisited: a model of reinforcement learning in the basal ganglia incorporating an efference copy of motor actions
Open accessThe DOI matches its Crossref record; 30 of the 30 references it cites were found in a registry or an index.
Abstract
In its simplest formulation, reinforcement learning is based on the idea that if an action taken in a particular context is followed by a favorable outcome, then, in the same context, the tendency to produce that action should be strengthened, or reinforced. While reinforcement learning forms the basis of many current theories of basal ganglia (BG) function, these models do not incorporate distinct computational roles for signals that convey context, and those that convey what action an animal takes. Recent experiments in the songbird suggest that vocal-related BG circuitry receives two functionally distinct excitatory inputs. One input is from a cortical region that carries context information about the current "time" in the motor sequence. The other is an efference copy of motor commands from a separate cortical brain region that generates vocal variability during learning. Based on these findings, I propose here a general model of vertebrate BG function that combines context information with a distinct motor efference copy signal. The signals are integrated by a learning rule in which efference copy inputs gate the potentiation of context inputs (but not efference copy inputs) onto medium spiny neurons in response to a rewarded action. The hypothesis is described in terms of a circuit that implements the learning of visually guided saccades. The model makes testable predictions about the anatomical and functional properties of hypothesized context and efference copy inputs to the striatum from both thalamic and cortical sources.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
The role of the basal ganglia in habit formation
Yin, H. H., & Knowlton, B. J. (2006) · Nature Reviews Neuroscience · doi:10.1038/nrn1919 · Cited by 2,402
A framework for mesencephalic dopamine systems based on predictive Hebbian learning
Montague, P., Dayan, P., & Sejnowski, T. (1996) · The Journal of Neuroscience · doi:10.1523/jneurosci.16-05-01936.1996 · Cited by 2,066 · cited 2×
“, 2006), whose firing rate signals the appearance of unexpected rewards or the absence of rewards that were expected (Montague et al., 1996; Schultz et al., 1997; Schultz, 2002; Cohen et al., 2012).”
BackgroundLearning and Memory Functions of the Basal Ganglia
Packard, M. G., & Knowlton, B. J. (2002) · Annual Review of Neuroscience · doi:10.1146/annurev.neuro.25.112701.142937 · Cited by 1,717 · cited 3×
“Nor do striatal lesions prevent the animal from learning spatial aspects of the task, which are instead affected by hippocampal lesions (Packard et al., 1989; Packard and Knowlton, 2002; Featherstone and McDonald, 2004).”
BackgroundHabits, Rituals, and the Evaluative Brain
Graybiel, A. M. (2008) · Annual Review of Neuroscience · doi:10.1146/annurev.neuro.29.051605.112851 · Cited by 1,697 · cited 3×
“” The implementation of this principle, which forms the basis of reinforcement learning, instrumental learning, and stimulus-response learning (Sutton and Barto, 1998; Packard and Knowlton, 2002; Graybiel, 2008), requires three pieces of information: the action (response) that the animal makes, the context (situation)…”
BackgroundNeuron-type-specific signals for reward and punishment in the ventral tegmental area
Cohen, J. Y., Haesler, S., Vong, L., Lowell, B. B., & Uchida, N. (2012) · Nature · doi:10.1038/nature10754 · Cited by 1,318 · cited 2×
“, 2006), whose firing rate signals the appearance of unexpected rewards or the absence of rewards that were expected (Montague et al., 1996; Schultz et al., 1997; Schultz, 2002; Cohen et al., 2012).”
BackgroundThe basal ganglia: a vertebrate solution to the selection problem?
Redgrave, P., Prescott, T. J., & Gurney, K. (1999) · Neuroscience · doi:10.1016/s0306-4522(98)00319-4 · Cited by 1,199
Dichotomous dopaminergic control of striatal synaptic plasticity
Shen, W., Flajolet, M., Greengard, P., & Surmeier, D. J. (2008) · Science · doi:10.1126/science.1160575 · Cited by 1,190
Revised nomenclature for avian telencephalon and some related brainstem nuclei
Reiner, A., Perkel, D. J., Bruce, L. L., Butler, A. B., Csillag, A., Kuenzel, W., Medina, L., Paxinos, G., Shimizu, T., Striedter, G., et al. (2004) · Journal of Comparative Neurology · doi:10.1002/cne.20118 · Cited by 1,148
The Basal Ganglia and Chunking of Action Repertoires
Graybiel, A. M. (1998) · Neurobiology of Learning and Memory · doi:10.1006/nlme.1998.3843 · Cited by 1,042 · cited 3×
“Neural circuitry in the basal ganglia (BG) is intimately involved in the control of learned behaviors (Graybiel et al., 1994; Graybiel, 1998), and is thought to be essential for the modification of behavior through reinforcement (Barto, 1995; Doya, 2000; Daw and Doya, 2006).”
BackgroundComplementary roles of basal ganglia and cerebellum in learning and motor control
Doya, K. (2000) · Current Opinion in Neurobiology · doi:10.1016/s0959-4388(00)00153-7 · Cited by 964
Cited by
Differential Innervation of Direct- and Indirect-Pathway Striatal Projection Neurons
Wall, N. R., De La Parra, M., Callaway, E. M., & Kreitzer, A. C. (2013) · Neuron · doi:10.1016/j.neuron.2013.05.014 · Cited by 489
Birds, primates, and spoken language origins: behavioral phenotypes and neurobiological substrates
Petkov, C. I., & Jarvis, E. D. (2012) · Frontiers in Evolutionary Neuroscience · doi:10.3389/fnevo.2012.00012 · Cited by 409
Dopamine neurons encode performance error in singing birds
Gadagkar, V., Puzerey, P. A., Chen, R., Baird-Daniel, E., Farhang, A. R., & Goldberg, J. H. (2016) · Science · doi:10.1126/science.aah6837 · Cited by 268
The functional logic of corticostriatal connections
Shipp, S. (2017) · Brain Structure and Function · doi:10.1007/s00429-016-1250-9 · Cited by 118
“Yet the role-reversal model places the onus for action selection entirely with the broader origins of contextual input—sufficient for some circumstances, as illustrated by a schematic of a cue-driven oculomotor choice task (Fee 2012, 2014).”
BackgroundLearning Enhances Sensory Processing in Mouse V1 before Improving Behavior
Jurjut, O., Georgieva, P., Busse, L., & Katzner, S. (2017) · The Journal of Neuroscience · doi:10.1523/jneurosci.3485-16.2017 · Cited by 99
Metadata
- Authors
- Fee, M. S.
- Venue
- Frontiers in Neural Circuits
- Publisher
- Frontiers Media SA
- Published
- 2012
- Volume
- 6
- Pages
- 38-38
- ISSN
- 1662-5110 (ISSN-L)
- OpenAlex
- W1971295726
- Cited by
- 40
- References
- 30
- FWCI
- 2.00
- Type
- journal article
- Language
- en
- Open access
- gold
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Oculomotor learning revisited: a model of reinforcement learning in the basal ganglia incorporating an efference copy of motor actions. https://app.cochat.ai/papers/10.3389/fncir.2012.00038
- The paper
- Fee, M. S. (2012). Oculomotor learning revisited: a model of reinforcement learning in the basal ganglia incorporating an efference copy of motor actions. Frontiers in Neural Circuits. https://doi.org/10.3389/fncir.2012.00038
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.