The DOI matches its Crossref record; 62 of the 62 references it cites were found in a registry or an index.
Striatal and Tegmental Neurons Code Critical Signals for Temporal-Difference Learning of State Value in Domestic Chicks
Open accessAbstract
To ensure survival, animals must update the internal representations of their environment in a trial-and-error fashion. Psychological studies of associative learning and neurophysiological analyses of dopaminergic neurons have suggested that this updating process involves the temporal-difference (TD) method in the basal ganglia network. However, the way in which the component variables of the TD method are implemented at the neuronal level is unclear. To investigate the underlying neural mechanisms, we trained domestic chicks to associate color cues with food rewards. We recorded neuronal activities from the medial striatum or tegmentum in a freely behaving condition and examined how reward omission changed neuronal firing. To compare neuronal activities with the signals assumed in the TD method, we simulated the behavioral task in the form of a finite sequence composed of discrete steps of time. The three signals assumed in the simulated task were the prediction signal, the target signal for updating, and the TD-error signal. In both the medial striatum and tegmentum, the majority of recorded neurons were categorized into three types according to their fitness for three models, though these neurons tended to form a continuum spectrum without distinct differences in the firing rate. Specifically, two types of striatal neurons successfully mimicked the target signal and the prediction signal. A linear summation of these two types of striatum neurons was a good fit for the activity of one type of tegmental neurons mimicking the TD-error signal. The present study thus demonstrates that the striatum and tegmentum can convey the signals critically required for the TD method. Based on the theoretical and neurophysiological studies, together with tract-tracing data, we propose a novel model to explain how the convergence of signals represented in the striatum could lead to the computation of TD error in tegmental dopaminergic neurons.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Reinforcement Learning: An Introduction
Sutton, R. S., & Barto, A. G. (1998) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.1998.712192 · Cited by 27,225
Mastering the game of Go with deep neural networks and tree search
Silver, D., Huang, A., Maddison, C. J., Guez, A., Sifre, L., van den Driessche, G., Schrittwieser, J., Antonoglou, I., Panneershelvam, V., Lanctot, M., et al. (2016) · Nature · doi:10.1038/nature16961 · Cited by 15,612
A neural substrate of prediction and reward
Schultz, W., Dayan, P., & Montague, P. R. (1997) · Science · doi:10.1126/science.275.5306.1593 · Cited by 9,388
Some Studies in Machine Learning Using the Game of Checkers
Samuel, A. L. (1959) · IBM Journal of Research and Development · doi:10.1147/rd.33.0210 · Cited by 4,443
Conditioned Reflexes: An Investigation of the Physiological Activity of the Cerebral Cortex.
(1930) · Archives of Neurology And Psychiatry · doi:10.1001/archneurpsyc.1930.02220110256018 · Cited by 2,673
Discrete coding of reward probability and uncertainty by dopamine neurons
Fiorillo, C. D., Tobler, P. N., & Schultz, W. (2003) · Science · doi:10.1126/science.1077349 · Cited by 2,085
A framework for mesencephalic dopamine systems based on predictive Hebbian learning
Montague, P., Dayan, P., & Sejnowski, T. (1996) · The Journal of Neuroscience · doi:10.1523/jneurosci.16-05-01936.1996 · Cited by 2,066
Conditioned Reflexes: an Investigation of the Physiological Activity of the Cerebral Cortex
DENNY-BROWN, D. (1928) · Nature · doi:10.1038/121662a0 · Cited by 1,836
Temporal difference learning and TD-Gammon
Tesauro, G. (1995) · Communications of the ACM · doi:10.1145/203330.203343 · Cited by 1,618
Conditioned reflexes: An investigation of the physiological activity of the cerebral cortex
Pavlov, I. P. (2010) · Annals of neurosciences · doi:10.5214/ans.0972-7531.1017309 · Cited by 1,460
Neuron-type-specific signals for reward and punishment in the ventral tegmental area
Cohen, J. Y., Haesler, S., Vong, L., Lowell, B. B., & Uchida, N. (2012) · Nature · doi:10.1038/nature10754 · Cited by 1,318
Lateral habenula as a source of negative reward signals in dopamine neurons
Matsumoto, M., & Hikosaka, O. (2007) · Nature · doi:10.1038/nature05860 · Cited by 1,292
Whole-Brain Mapping of Direct Inputs to Midbrain Dopamine Neurons
Watabe-Uchida, M., Zhu, L., Ogawa, S. K., Vamanrao, A., & Uchida, N. (2012) · Neuron · doi:10.1016/j.neuron.2012.03.017 · Cited by 1,247
Models of Information Processing in the Basal Ganglia
(1994) · doi:10.7551/mitpress/4708.001.0001 · Cited by 1,190
Revised nomenclature for avian telencephalon and some related brainstem nuclei
Reiner, A., Perkel, D. J., Bruce, L. L., Butler, A. B., Csillag, A., Kuenzel, W., Medina, L., Paxinos, G., Shimizu, T., Striedter, G., et al. (2004) · Journal of Comparative Neurology · doi:10.1002/cne.20118 · Cited by 1,148
Neuronal Reward and Decision Signals: From Theories to Data
Schultz, W. (2015) · Physiological Reviews · doi:10.1152/physrev.00023.2014 · Cited by 1,126
Representation of Action-Specific Reward Values in the Striatum
Samejima, K., Ueda, Y., Doya, K., & Kimura, M. (2005) · Science · doi:10.1126/science.1115270 · Cited by 939
Temporal Difference Learning and TD-Gammon
Tesauro, G. (1995) · ICGA Journal · doi:10.3233/icg-1995-18207 · Cited by 764
A stereotaxic atlas of the brain of the chick (Gallus domesticus)
Kuenzel, W. J., & Masson, M. (1988) · Journal of the Arkansas Academy of Science · Cited by 710
Expectation of reward modulates cognitive signals in the basal ganglia
Kawagoe, R., Takikawa, Y., & Hikosaka, O. (1998) · Nature Neuroscience · doi:10.1038/1625 · Cited by 631
Roesch, M. R., Calu, D. J., & Schoenbaum, G. (2007) · Nature Neuroscience · doi:10.1038/nn2013 · Cited by 597
Mechanisms for selection of basic motor programs – roles for the striatum and pallidum
Grillner, S., Hellgren, J., Ménard, A., Saitoh, K., & Wikström, M. A. (2005) · Trends in Neurosciences · doi:10.1016/j.tins.2005.05.004 · Cited by 592
Coincident but Distinct Messages of Midbrain Dopamine and Striatal Tonically Active Neurons
Morris, G., Arkadir, D., Nevet, A., Vaadia, E., & Bergman, H. (2004) · Neuron · doi:10.1016/j.neuron.2004.06.012 · Cited by 574
Anterior Cingulate: Single Neuronal Signals Related to Degree of Reward Expectancy
Shidara, M., & Richmond, B. J. (2002) · Science · doi:10.1126/science.1069504 · Cited by 518
Midbrain dopamine neurons encode decisions for future action
Morris, G., Nevet, A., Arkadir, D., Vaadia, E., & Bergman, H. (2006) · Nature Neuroscience · doi:10.1038/nn1743 · Cited by 503
Actor–critic models of the basal ganglia: new anatomical and computational perspectives
Joel, D., Niv, Y., & Ruppin, E. (2002) · Neural Networks · doi:10.1016/s0893-6080(02)00047-3 · Cited by 500
Arithmetic and local circuitry underlying dopamine prediction errors
Eshel, N., Bukwich, M., Rao, V., Hemmelder, V., Tian, J., & Uchida, N. (2015) · Nature · doi:10.1038/nature14855 · Cited by 392
Seo, H., & Lee, D. (2007) · The Journal of Neuroscience · doi:10.1523/jneurosci.2369-07.2007 · Cited by 324
Bocklisch, C., Pascoli, V., Wong, J. C. Y., House, D. R. C., Yvon, C., de Roo, M., Tan, K. R., & Lüscher, C. (2013) · Science · doi:10.1126/science.1237059 · Cited by 310
Reiner, A., & Anderson, K. D. (1990) · Brain Research Reviews · doi:10.1016/0165-0173(90)90003-7 · Cited by 272
Modifications of Reward Expectation-Related Neuronal Activity During Learning in Primate Striatum
Tremblay, L., Hollerman, J. R., & Schultz, W. (1998) · Journal of Neurophysiology · doi:10.1152/jn.1998.80.2.964 · Cited by 268
Kr�ner, S., & G�nt�rk�n, O. (1999) · The Journal of Comparative Neurology · doi:10.1002/(sici)1096-9861(19990503)407:2<228::aid-cne6>3.0.co;2-2 · Cited by 265
Clarke, H. F., Robbins, T. W., & Roberts, A. C. (2008) · The Journal of Neuroscience · doi:10.1523/jneurosci.1521-08.2008 · Cited by 263
Distributed and Mixed Information in Monosynaptic Inputs to Dopamine Neurons
Tian, J., Huang, R., Cohen, J. Y., Osakada, F., Kobak, D., Machens, C. K., Callaway, E. M., Uchida, N., & Watabe-Uchida, M. (2016) · Neuron · doi:10.1016/j.neuron.2016.08.018 · Cited by 224
Role of Striatum in Updating Values of Chosen Actions
Kim, H., Sul, J. H., Huh, N., Lee, D., & Jung, M. W. (2009) · The Journal of Neuroscience · doi:10.1523/jneurosci.2728-09.2009 · Cited by 224
Annett, L. E., McGregor, A., & Robbins, T. W. (1989) · Behavioural Brain Research · doi:10.1016/0166-4328(89)90005-3 · Cited by 220
Veenman, C. L., Wild, J. M., & Reiner, A. (1995) · Journal of Comparative Neurology · doi:10.1002/cne.903540108 · Cited by 217
Shidara, M., Aigner, T. G., & Richmond, B. J. (1998) · The Journal of Neuroscience · doi:10.1523/jneurosci.18-07-02613.1998 · Cited by 205
Selective lesions of the dorsomedial striatum impair serial spatial reversal learning in rats
Castañé, A., Theobald, D. E. H., & Robbins, T. W. (2010) · Behavioural Brain Research · doi:10.1016/j.bbr.2010.02.017 · Cited by 201
Veenman, C. L., & Reiner, A. (1994) · Journal of Comparative Neurology · doi:10.1002/cne.903390205 · Cited by 184
Reinforcement learning: Computational theory and biological mechanisms
Doya, K. (2007) · HFSP journal · doi:10.2976/1.2732246 · Cited by 150
Working Memory Neurons in Pigeons
Diekamp, B., Kalt, T., & Güntürkün, O. (2002) · Journal of Neuroscience · doi:10.1523/jneurosci.22-04-j0002.2002 · Cited by 135
Reward Prediction Error Coding in Dorsal Striatal Neurons
Oyama, K., Hernádi, I., Iijima, T., & Tsutsui, K. (2010) · Journal of Neuroscience · doi:10.1523/jneurosci.1719-10.2010 · Cited by 103
The Mind Through Chick Eyes : Memory, Cognition and Anticipation
Matsushima, T., Izawa, E., Aoki, N., & Yanagihara, S. (2003) · ZOOLOGICAL SCIENCE · doi:10.2108/zsj.20.395 · Cited by 87
Self-citationIzawa, E., Zachár, G., Yanagihara, S., & Matsushima, T. (2003) · Journal of Neuroscience · doi:10.1523/jneurosci.23-05-01894.2003 · Cited by 84
Self-citationApicella, P., Deffains, M., Ravel, S., & Legallet, E. (2009) · European Journal of Neuroscience · doi:10.1111/j.1460-9568.2009.06872.x · Cited by 79
Theoretical and Empirical Studies of Learning
Niv, Y., & Montague, P. R. (2009) · Elsevier eBooks · doi:10.1016/b978-0-12-374176-9.00022-1 · Cited by 73
Székely, A. D., Boxer, M. I., Stewart, M. G., & Csillag, A. (1994) · The Journal of Comparative Neurology · doi:10.1002/cne.903480305 · Cited by 72
Gale, S. D., & Perkel, D. J. (2006) · Journal of Neurophysiology · doi:10.1152/jn.01040.2005 · Cited by 67
Selective striatal connections of midbrain dopaminergic nuclei in the chick ( Gallus domesticus )
Mezey, S. E., & Csillag, A. (2002) · Cell and Tissue Research · doi:10.1007/s00441-002-0514-2 · Cited by 63
Rueda‐Orozco, P. E., Montes-Rodríguez, C. J., Gómez, E. S., Méndez‐Díaz, M., & Prospero-Garcia, O. E. (2008) · Neuropharmacology · doi:10.1016/j.neuropharm.2008.04.013 · Cited by 48
Bálint, E., Mezey, S. E., & Csillag, A. (2011) · The Journal of Comparative Neurology · doi:10.1002/cne.22672 · Cited by 45
The role of basal ganglia in reinforcement learning and imprinting in domestic chicks
Izawa, E., Yanagihara, S., Atsumi, T., & Matsushima, T. (2001) · Neuroreport · doi:10.1097/00001756-200106130-00045 · Cited by 45
Self-citationAoki, N., Csillag, A., & Matsushima, T. (2006) · European Journal of Neuroscience · doi:10.1111/j.1460-9568.2006.05090.x · Cited by 43
Self-citationAnderson, K. D., Karle, E. J., & Reiner, A. J. (1991) · The Journal of Comparative Neurology · doi:10.1002/cne.903090305 · Cited by 42
Dynamics of neural coding in the accumbens during extinction and reinstatement of rewarded behavior
Janak, P. H., Chen, M.-T., & Caulder, T. (2004) · Behavioural Brain Research · doi:10.1016/j.bbr.2004.02.003 · Cited by 38
Izawa, E., Aoki, N., & Matsushima, T. (2005) · European Journal of Neuroscience · doi:10.1111/j.1460-9568.2005.04311.x · Cited by 37
Self-citationOgura, Y., & Matsushima, T. (2011) · Frontiers in Neuroscience · doi:10.3389/fnins.2011.00091 · Cited by 30
Self-citationOyama, K., Tateyama, Y., Hernádi, I., Tobler, P. N., Iijima, T., & Tsutsui, K. (2015) · Journal of Neurophysiology · doi:10.1152/jn.00097.2015 · Cited by 29
Ogura, Y., Izumi, T., Yoshioka, M., & Matsushima, T. (2015) · Behavioural Brain Research · doi:10.1016/j.bbr.2015.07.052 · Cited by 24
Self-citationIchikawa, Y., Izawa, E., & Matsushima, T. (2004) · Cognitive Brain Research · doi:10.1016/j.cogbrainres.2004.08.001 · Cited by 15
Self-citationAmita, H., & Matsushima, T. (2014) · Behavioural Brain Research · doi:10.1016/j.bbr.2014.04.004 · Cited by 14
Self-citation
Cited by
Bryois, J., Skene, N. G., Hansen, T. F., Kogelman, L. J. A., Watson, H. J., Liu, Z., Eating Disorders Working Group of the Psychiatric Genomics Consortium, Adan, R., Alfredsson, L., Ando, T., et al. (2020) · Nature Genetics · doi:10.1038/s41588-020-0610-9 · Cited by 366
Alpha-synuclein oligomers: a new hope
Bengoa-Vergniory, N., Roberts, R. F., Wade-Martins, R., & Alegre-Abarrategui, J. (2017) · Acta Neuropathologica · doi:10.1007/s00401-017-1755-1 · Cited by 338
Alber, J., Goldfarb, D., Thompson, L. I., Arthur, E., Hernandez, K., Cheng, D., DeBuc, D. C., Cordeiro, F., Provetti‐Cunha, L., den Haan, J., et al. (2020) · Alzheimer's & Dementia · doi:10.1002/alz.12006 · Cited by 154
Naturalizing psychedelic spirituality
Letheby, C. (2017) · Zygon: Journal of Religion and Science · doi:10.1111/zygo.12353 · Cited by 18
A machine learning framework to optimize optic nerve electrical stimulation for vision restoration
Romeni, S., Zoccolan, D., & Micera, S. (2021) · Patterns · doi:10.1016/j.patter.2021.100286 · Cited by 17
Metadata
- Authors
- Wen, C., Ogura, Y., & Matsushima, T.
- Venue
- Frontiers in Neuroscience
- Publisher
- Frontiers Media SA
- Published
- 2016-11-08
- Volume
- 10
- Pages
- 476-476
- ISSN
- 1662-453X (ISSN-L)
- OpenAlex
- W2552043181
- Cited by
- 259
- References
- 62
- FWCI
- 23.46
- Type
- journal article
- Language
- en
- Open access
- gold
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.