The DOI matches its Crossref record; 52 of the 52 references it cites were found in a registry or an index.
Reinforcement learning can account for associative and perceptual learning on a visual-decision task
Open accessAbstract
We recently showed that improved perceptual performance on a visual motion direction–discrimination task corresponds to changes in how an unmodified sensory representation in the brain is interpreted to form a decision that guides behavior. Here we found that these changes can be accounted for using a reinforcement-learning rule to shape functional connectivity between the sensory and decision neurons. We modeled performance on the basis of the readout of simulated responses of direction-selective sensory neurons in the middle temporal area (MT) of monkey cortex. A reward prediction error guided changes in connections between these sensory neurons and the decision process, first establishing the association between motion direction and response direction, and then gradually improving perceptual sensitivity by selectively strengthening the connections from the most sensitive neurons in the sensory population. The results suggest a common, feedback-driven mechanism for some forms of associative and perceptual learning.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Reinforcement Learning: An Introduction
Sutton, R. S., & Barto, A. G. (1998) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.1998.712192 · Cited by 27,225
Reinforcement Learning: An Introduction
(2005) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.2004.842673 · Cited by 24,496
Advances in neural information processing systems 7
Cios, K., & Shields, M. (1997) · Neurocomputing · doi:10.1016/s0925-2312(97)90018-7 · Cited by 22,059
Advances in Neural Information Processing Systems 14: Proceedings of the 2001 Conference
(2002) · The MIT Press eBooks · doi:10.7551/mitpress/1120.001.0001 · Cited by 8,890
The Neural Basis of Decision Making
Gold, J. I., & Shadlen, M. N. (2007) · Annual Review of Neuroscience · doi:10.1146/annurev.neuro.29.051605.113038 · Cited by 3,764
Self-citationSimple Statistical Gradient-Following Algorithms for Connectionist Reinforcement Learning
Williams, R. J. (1992) · Machine Learning · doi:10.1007/bf00992696 · Cited by 3,416
The debate over dopamine’s role in reward: the case for incentive salience
Berridge, K. C. (2007) · Psychopharmacology · doi:10.1007/s00213-006-0578-x · Cited by 2,347
The analysis of visual motion: a comparison of neuronal and psychophysical performance
Britten, K., Shadlen, M., Newsome, W., & Movshon, J. (1992) · The Journal of Neuroscience · doi:10.1523/jneurosci.12-12-04745.1992 · Cited by 1,958
Neural correlates of decision variables in parietal cortex
Platt, M. L., & Glimcher, P. W. (1999) · Nature · doi:10.1038/22268 · Cited by 1,721
SPACE AND ATTENTION IN PARIETAL CORTEX
Colby, C. L., & Goldberg, M. E. (1999) · Annual Review of Neuroscience · doi:10.1146/annurev.neuro.22.1.319 · Cited by 1,710
Behavioral Theories and the Neurophysiology of Reward
Schultz, W. (2006) · Annual Review of Psychology · doi:10.1146/annurev.psych.56.091103.070229 · Cited by 1,637
Newsome, W., & Pare, E. (1988) · The Journal of Neuroscience · doi:10.1523/jneurosci.08-06-02201.1988 · Cited by 1,615
Roitman, J. D., & Shadlen, M. N. (2002) · The Journal of Neuroscience · doi:10.1523/jneurosci.22-21-09475.2002 · Cited by 1,589
Multiple dopamine functions at different time courses
Schultz, W. (2007) · Annual Review of Neuroscience · doi:10.1146/annurev.neuro.28.061604.135722 · Cited by 1,415
Intentional Maps in Posterior Parietal Cortex
Andersen, R. A., & Buneo, C. A. (2002) · Annual Review of Neuroscience · doi:10.1146/annurev.neuro.25.112701.142922 · Cited by 1,294
Correlated neuronal discharge rate and its implications for psychophysical performance
Zohary, E., Shadlen, M. N., & Newsome, W. T. (1994) · Nature · doi:10.1038/370140a0 · Cited by 1,237
A relationship between behavioral choice and the visual responses of neurons in macaque MT
Britten, K. H., Newsome, W. T., Shadlen, M. N., Celebrini, S., & Movshon, J. A. (1996) · Visual Neuroscience · doi:10.1017/s095252380000715x · Cited by 1,049
Cortical microstimulation influences perceptual judgements of motion direction
Salzman, C. D., Britten, K. H., & Newsome, W. T. (1990) · Nature · doi:10.1038/346174a0 · Cited by 881 · cited 1×
“In contrast, there was no apparent change in the motion-driven responses of neurons in the MT, which encodes motion information that is used to solve the tas”
BackgroundNeural computations that underlie decisions about sensory stimuli
Gold, J. I., & Shadlen, M. N. (2001) · Trends in Cognitive Sciences · doi:10.1016/s1364-6613(00)01567-9 · Cited by 881
Self-citationShadlen, M., Britten, K., Newsome, W., & Movshon, J. (1996) · The Journal of Neuroscience · doi:10.1523/jneurosci.16-04-01486.1996 · Cited by 824
Task difficulty and the specificity of perceptual learning
Ahissar, M., & Hochstein, S. (1997) · Nature · doi:10.1038/387401a0 · Cited by 824
Neural correlates, computation and behavioural impact of decision confidence
Kepecs, A., Uchida, N., Zariwala, H. A., & Mainen, Z. F. (2008) · Nature · doi:10.1038/nature07200 · Cited by 809
The Neural Basis of Perceptual Learning
Gilbert, C. D., Sigman, M., & Crist, R. E. (2001) · Neuron · doi:10.1016/s0896-6273(01)00424-x · Cited by 691
A role for neural integrators in perceptual decision making
Mazurek, M. E., Roitman, J. D., Ditterich, J., & Shadlen, M. N. (2003) · Cerebral Cortex · doi:10.1093/cercor/bhg097 · Cited by 620
Responses of neurons in macaque MT to stochastic motion signals
Britten, K. H., Shadlen, M. N., Newsome, W. T., & Movshon, J. A. (1993) · Visual Neuroscience · doi:10.1017/s0952523800010269 · Cited by 594
Spatial and Temporal Scales of Neuronal Correlation in Primary Visual Cortex
Smith, M. A., & Kohn, A. (2008) · The Journal of Neuroscience · doi:10.1523/jneurosci.2929-08.2008 · Cited by 589
Decision theory, reinforcement learning, and the brain
Dayan, P., & Daw, N. D. (2008) · Cognitive, Affective, & Behavioral Neuroscience · doi:10.3758/cabn.8.4.429 · Cited by 580
Cortical remodelling induced by activity of ventral tegmental dopamine neurons
Bao, S., Chan, V. T., & Merzenich, M. M. (2001) · Nature · doi:10.1038/35083586 · Cited by 569
Sequential updating of conditional probabilities on directed graphical structures
Spiegelhalter, D. J., & Lauritzen, S. L. (1990) · Networks · doi:10.1002/net.3230200507 · Cited by 521
Neural correlates of perceptual learning in a sensory-motor, but not a sensory, cortical area
Law, C.-T., & Gold, J. I. (2008) · Nature Neuroscience · doi:10.1038/nn2070 · Cited by 516
Self-citationCorrelated Firing in Macaque Visual Area MT: Time Scales and Relationship to Behavior
Bair, W., Zohary, E., & Newsome, W. T. (2001) · The Journal of Neuroscience · doi:10.1523/jneurosci.21-05-01676.2001 · Cited by 515
Direction-specific improvement in motion discrimination
Ball, K., & Sekuler, R. (1987) · Vision Research · doi:10.1016/0042-6989(87)90011-3 · Cited by 509
Kiani, R., Hanks, T. D., & Shadlen, M. N. (2008) · The Journal of Neuroscience · doi:10.1523/jneurosci.4761-07.2008 · Cited by 500
Mechanisms of perceptual learning
Dosher, B. A., & Lu, Z.-L. (1999) · Vision Research · doi:10.1016/s0042-6989(99)00059-0 · Cited by 394
Microstimulation of macaque area LIP affects decision-making in a motion discrimination task
Hanks, T. D., Ditterich, J., & Shadlen, M. N. (2006) · Nature Neuroscience · doi:10.1038/nn1683 · Cited by 360
Conservation of total synaptic weight through balanced synaptic depression and potentiation
Royer, S., & Paré, D. (2003) · Nature · doi:10.1038/nature01530 · Cited by 356
A unified model for perceptual learning
SEITZ, A., & WATANABE, T. (2005) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2005.05.010 · Cited by 356
Population coding in neuronal systems with correlated noise
Sompolinsky, H., Yoon, H., Kang, K., & Shamir, M. (2001) · Physical Review E · doi:10.1103/physreve.64.051904 · Cited by 322
Neural population code for fine perceptual decisions in area MT
Purushothaman, G., & Bradley, D. C. (2005) · Nature Neuroscience · doi:10.1038/nn1373 · Cited by 309
Pattern-recognizing stochastic learning automata
Barto, A. G., & Anandan, P. (1985) · IEEE Transactions on Systems, Man, and Cybernetics · doi:10.1109/tsmc.1985.6313371 · Cited by 305
Perceptual learning: specificity versus generalization
Fahle, M. (2005) · Current Opinion in Neurobiology · doi:10.1016/j.conb.2005.03.010 · Cited by 284
Gold, J. I., & Shadlen, M. N. (2003) · The Journal of Neuroscience · doi:10.1523/jneurosci.23-02-00632.2003 · Cited by 281
Self-citationWhat is reinforced by phasic dopamine signals?
Redgrave, P., Gurney, K., & Reynolds, J. (2008) · Brain Research Reviews · doi:10.1016/j.brainresrev.2007.10.007 · Cited by 261
Gibson, E. J. (1963) · Annual Review of Psychology · doi:10.1146/annurev.ps.14.020163.000333 · Cited by 259
Celebrini, S., & Newsome, W. (1994) · The Journal of Neuroscience · doi:10.1523/jneurosci.14-07-04109.1994 · Cited by 232
Mukai, I., Kim, D., Fukunaga, M., Japee, S., Marrett, S., & Ungerleider, L. G. (2007) · The Journal of Neuroscience · doi:10.1523/jneurosci.3002-07.2007 · Cited by 176
A new perceptual illusion reveals mechanisms of sensory decoding
Jazayeri, M., & Movshon, J. A. (2007) · Nature · doi:10.1038/nature05739 · Cited by 173
Error Detection, Correction, and Prevention in the Brain: A Brief Review of Data and Theories
van Veen, V., & Carter, C. S. (2006) · Clinical EEG and Neuroscience · doi:10.1177/155005940603700411 · Cited by 129
Different populations of neurons contribute to the detection and discrimination of visual motion
Hol, K., & Treue, S. (2001) · Vision Research · doi:10.1016/s0042-6989(00)00314-x · Cited by 103
Loewenstein, Y., & Seung, H. S. (2006) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.0505220103 · Cited by 102
The Effect of Correlations on the Fisher Information of Population Codes
Yoon, H., & Sompolinsky, H. (1998) · Cited by 42
Correlates of Perceptual Learning in an Oculomotor Decision Variable
Connolly, P. M., Bennur, S., & Gold, J. I. (2009) · Journal of Neuroscience · doi:10.1523/jneurosci.3962-08.2009 · Cited by 19
Self-citation
Cited by
Human-level control through deep reinforcement learning
Mnih, V., Kavukcuoglu, K., Silver, D., Rusu, A. A., Veness, J., Bellemare, M. G., Graves, A., Riedmiller, M., Fidjeland, A. K., Ostrovski, G., et al. (2015) · Nature · doi:10.1038/nature14236 · Cited by 30,357
Cortical layer-specific critical dynamics triggering perception
Marshel, J. H., Kim, Y. S., Machado, T. A., Quirin, S., Benson, B., Kadmon, J., Raja, C., Chibukhchyan, A., Ramakrishnan, C., Inoue, M., et al. (2019) · Science · doi:10.1126/science.aaw5202 · Cited by 652
Inside the brain of an elite athlete: the neural processes that support high achievement in sports
Yarrow, K., Brown, P., & Krakauer, J. W. (2009) · Nature Reviews Neuroscience · doi:10.1038/nrn2672 · Cited by 637
From drugs to deprivation: A Bayesian framework for understanding models of psychosis
Corlett, P. R., Frith, C. D., & Fletcher, P. C. (2009) · Psychopharmacology · doi:10.1007/s00213-009-1561-0 · Cited by 367
“Recent computational neuroscience models highlight the role of prediction error in perceptual learning and decision making, both in forging associations between sensory representation and in tuning perceptual sensitivity in middle temporal cortex (Law and Gold 2009).”
BackgroundHierarchical Prediction Errors in Midbrain and Basal Forebrain during Sensory Learning
Iglesias, S., Mathys, C., Brodersen, K. H., Kasper, L., Piccirelli, M., den Ouden, H. E. M., & Stephan, K. E. (2013) · Neuron · doi:10.1016/j.neuron.2013.09.009 · Cited by 342
Metadata
- Authors
- Law, C.-T., & Gold, J. I.
- Venue
- Nature Neuroscience
- Published
- 2009-04-19
- Volume
- 12
- Issue
- 5
- Pages
- 655-663
- ISSN
- 1097-6256 (ISSN-L)
- 1546-1726
- OpenAlex
- W2040414277
- Cited by
- 291
- References
- 52
- FWCI
- 8.68
- Type
- journal article
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.