The role of efference copy in striatal learning
Open accessThe DOI matches its Crossref record; 24 of the 24 references it cites were found in a registry or an index.
Abstract
Reinforcement learning requires the convergence of signals representing context, action, and reward. While models of basal ganglia function have well-founded hypotheses about the neural origin of signals representing context and reward, the function and origin of signals representing action are less clear. Recent findings suggest that exploratory or variable behaviors are initiated by a wide array of ‘action-generating’ circuits in the midbrain, brainstem, and cortex. Thus, in order to learn, the striatum must incorporate an efference copy of action decisions made in these action-generating circuits. Here we review several recent neural models of reinforcement learning that emphasize the role of efference copy signals, as well as ideas about how these signals might be integrated with inputs signaling context and reward.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Watkins, C. J. C. H., & Dayan, P. (1992) · Machine Learning · doi:10.1007/bf00992698 · Cited by 9,083
Dopamine in motivational control: rewarding, aversive, and alerting
Bromberg-Martin, E. S., Matsumoto, M., & Hikosaka, O. (2010) · Neuron · doi:10.1016/j.neuron.2010.11.022 · Cited by 2,396
Synaptic tagging and long-term potentiation
Frey, U., & Morris, R. G. M. (1997) · Nature · doi:10.1038/385533a0 · Cited by 1,736
Learning and Memory Functions of the Basal Ganglia
Packard, M. G., & Knowlton, B. J. (2002) · Annual Review of Neuroscience · doi:10.1146/annurev.neuro.25.112701.142937 · Cited by 1,717 · cited 1×
“This simple statement, known as Thorndike's Law of Effect, is one of the central tenets of animal behavior, and forms the basis of instrumental learning, or operant conditioning [2,3].”
Habits, Rituals, and the Evaluative Brain
Graybiel, A. M. (2008) · Annual Review of Neuroscience · doi:10.1146/annurev.neuro.29.051605.112851 · Cited by 1,697 · cited 1×
“This simple statement, known as Thorndike's Law of Effect, is one of the central tenets of animal behavior, and forms the basis of instrumental learning, or operant conditioning [2,3].”
The Basal Ganglia and Adaptive Motor Control
Graybiel, A. M., Aosaki, T., Flaherty, A. W., & Kimura, M. (1994) · Science · doi:10.1126/science.8091209 · Cited by 1,324
The basal ganglia: a vertebrate solution to the selection problem?
Redgrave, P., Prescott, T. J., & Gurney, K. (1999) · Neuroscience · doi:10.1016/s0306-4522(98)00319-4 · Cited by 1,199
Dichotomous dopaminergic control of striatal synaptic plasticity
Shen, W., Flajolet, M., Greengard, P., & Surmeier, D. J. (2008) · Science · doi:10.1126/science.1160575 · Cited by 1,190
The Basal Ganglia and Chunking of Action Repertoires
Graybiel, A. M. (1998) · Neurobiology of Learning and Memory · doi:10.1006/nlme.1998.3843 · Cited by 1,042 · cited 1×
“Neural circuitry in the basal ganglia (BG) is well known to be involved in the control of learned behaviors [5,6], and the striatum, the input structure of the BG, is well established as a key structure in the neural implementation of reinforcement learning [7-10].”
Mesolimbocortical and nigrostriatal dopamine responses to salient non-reward events
Horvitz, J. C. (2000) · Neuroscience · doi:10.1016/s0306-4522(00)00019-1 · Cited by 967
Complementary roles of basal ganglia and cerebellum in learning and motor control
Doya, K. (2000) · Current Opinion in Neurobiology · doi:10.1016/s0959-4388(00)00153-7 · Cited by 964
Frank, M. J. (2005) · Journal of Cognitive Neuroscience · doi:10.1162/0898929052880093 · Cited by 925
Visual-Motor Function of the Primate Superior Colliculus
Wurtz, R. H., & Albano, J. E. (1980) · Annual Review of Neuroscience · doi:10.1146/annurev.ne.03.030180.001201 · Cited by 904
A cellular mechanism of reward-related learning
Reynolds, J. N. J., Hyland, B. I., & Wickens, J. R. (2001) · Nature · doi:10.1038/35092560 · Cited by 879
Parallel neural networks for learning sequential procedures
Hikosaka, O., Nakahara, H., Rand, M. K., Sakai, K., Lu, X., Nakamura, K., Miyachi, S., & Doya, K. (1999) · Trends in Neurosciences · doi:10.1016/s0166-2236(99)01439-3 · Cited by 799
The short-latency dopamine signal: a role in discovering novel actions?
Redgrave, P., & Gurney, K. (2006) · Nature Reviews Neuroscience · doi:10.1038/nrn2022 · Cited by 762
Different time courses of learning-related activity in the prefrontal cortex and striatum.
Pasupathy, A., & Miller, E. K. (2005) · Nature · doi:10.1038/nature03287 · Cited by 712 · cited 1×
“ganglia circuitry in oculomotor learning, in which animals are trained, using rewards, to make saccades in a particular direction depending on which visual stimulus is presented [11-13].”
Importance of unpredictability for reward responses in primate dopamine neurons
Mirenowicz, J., & Schultz, W. (1994) · Journal of Neurophysiology · doi:10.1152/jn.1994.72.2.1024 · Cited by 569
Brain Circuits for the Internal Monitoring of Movements
Sommer, M. A., & Wurtz, R. H. (2008) · Annual Review of Neuroscience · doi:10.1146/annurev.neuro.31.060407.125627 · Cited by 508
Precise auditory–vocal mirroring in neurons for learned vocal communication
Prather, J. F., Peters, S., Nowicki, S., & Mooney, R. (2008) · Nature · doi:10.1038/nature06492 · Cited by 463
Choi, G. B., Dong, H.-W., Murphy, A. J., Valenzuela, D. M., Yancopoulos, G. D., Swanson, L. W., & Anderson, D. J. (2005) · Neuron · doi:10.1016/j.neuron.2005.04.011 · Cited by 462
Basal ganglia orient eyes to reward
Hikosaka, O., Nakamura, K., & Nakahara, H. (2006) · Journal of Neurophysiology · doi:10.1152/jn.00458.2005 · Cited by 377
Fiete, I. R., Fee, M. S., & Seung, H. S. (2007) · Journal of Neurophysiology · doi:10.1152/jn.01311.2006 · Cited by 167
Self-citationAronov, D., Veit, L., Goldberg, J. H., & Fee, M. S. (2011) · The Journal of Neuroscience · doi:10.1523/jneurosci.3009-11.2011 · Cited by 64
Cited by
Schizophrenia, dopamine and the striatum: From biology to symptoms
McCutcheon, R. A., Abi-Dargham, A., & Howes, O. D. (2019) · Trends in Neurosciences · doi:10.1016/j.tins.2018.12.004 · Cited by 763
“Efference copies accompanying internally generated motor cortex activity travel via pyramidal tract neurons to the dorsal striatum [87,88].”
BackgroundThe connectome of an insect brain
Winding, M., Pedigo, B. D., Barnes, C. L., Patsolic, H. G., Park, Y., Kazimiers, T., Fushiki, A., Andrade, I. V., Khandelwal, A., Valdes-Aleman, J., et al. (2023) · Science · doi:10.1126/science.add9330 · Cited by 435
“In summary, the observed architecture gives DANs access to multiple kinds of information that is thought to be critical in driving reinforcement learning: actual rewards (122), predicted values (145), selected actions (185), and proprioceptive feedback about ongoing actions (184).”
BackgroundThe functional logic of corticostriatal connections
Shipp, S. (2017) · Brain Structure and Function · doi:10.1007/s00429-016-1250-9 · Cited by 118
“A recent model of corticostriatal plasticity incorporating the ‘action signal’ noted above also stipulates distinct roles for contextual and efference copy inputs to an SPN (Fee 2014).”
BackgroundInfluential
Metadata
- Authors
- Fee, M. S.
- Venue
- Current Opinion in Neurobiology
- Publisher
- Elsevier BV
- Published
- 2014-04
- Volume
- 25
- Pages
- 194-200
- ISSN
- 0959-4388 (ISSN-L)
- OpenAlex
- W2005291990
- Cited by
- 87
- References
- 24
- FWCI
- 0.39
- Type
- journal article
- Language
- en
- Open access
- bronze
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.