A more biologically plausible learning rule for neural networks.
Open accessThe DOI matches its Crossref record; 1 of the 1 references it cites were found in a registry or an index.
Abstract
Many recent studies have used artificial neural network algorithms to model how the brain might process information. However, back-propagation learning, the method that is generally used to train these networks, is distinctly "unbiological." We describe here a more biologically plausible learning rule, using reinforcement learning, which we have applied to the problem of how area 7a in the posterior parietal cortex of monkeys might represent visual space in head-centered coordinates. The network behaves similarly to networks trained by using back-propagation and to neurons recorded in area 7a. These results show that a neural network does not require back propagation to acquire biologically interesting properties.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Visual and Eye Movement Functions of the Posterior Parietal Cortex
Andersen, R. A. (1989) · Annual Review of Neuroscience · doi:10.1146/annurev.ne.12.030189.002113 · Cited by 382
Self-citation
Cited by
McClelland, J. L., McNaughton, B. L., & O'Reilly, R. C. (1995) · Psychological Review · doi:10.1037/0033-295x.102.3.419 · Cited by 5,256
“Mazzoni, Andersen, and Jordan (1991) have compared an associative reinforcement learning algorithm and the back-propagation algorithm as procedures for discovering representations that are useful for the transformation of visual space from retinal to head-centered coordinates and for development of…”
BackgroundHolroyd, C. B., & Coles, M. G. H. (2002) · Psychological Review · doi:10.1037/0033-295x.109.4.679 · Cited by 4,027
Softky, W., & Koch, C. (1993) · The Journal of Neuroscience · doi:10.1523/jneurosci.13-01-00334.1993 · Cited by 1,689
Lillicrap, T. P., Santoro, A., Marris, L., Akerman, C. J., & Hinton, G. (2020) · Nature Reviews Neuroscience · doi:10.1038/s41583-020-0277-3 · Cited by 908
Random synaptic feedback weights support error backpropagation for deep learning
Lillicrap, T. P., Cownden, D., Tweed, D. B., & Akerman, C. J. (2016) · Nature Communications · doi:10.1038/ncomms13276 · Cited by 812
The parallel distributed processing approach to semantic cognition
McClelland, J. L., & Rogers, T. T. (2003) · Nature Reviews Neuroscience · doi:10.1038/nrn1076 · Cited by 795
Metalearning and neuromodulation
Doya, K. (2002) · Neural Networks · doi:10.1016/s0893-6080(02)00044-8 · Cited by 704
Neuromodulation and cortical function: modeling the physiological basis of behavior
Hasselmo, M. E. (1995) · Behavioural Brain Research · doi:10.1016/0166-4328(94)00113-t · Cited by 556
Theories of Error Back-Propagation in the Brain
Whittington, J. C. R., & Bogacz, R. (2019) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2018.12.005 · Cited by 420
“The lack of local error representation has been addressed by early theories by proposing that the errors associated with individual neurons are not computed, but instead the synaptic plasticity is driven by a global error signal carried by neuromodulators [16 – 19].”
BackgroundReading population codes: a neural implementation of ideal observers
Deneve, S., Latham, P. E., & Pouget, A. (1999) · Nature Neuroscience · doi:10.1038/11205 · Cited by 392
Reinforcement Learning Through Modulation of Spike-Timing-Dependent Synaptic Plasticity
Florian, R. V. (2007) · Neural Computation · doi:10.1162/neco.2007.19.6.1468 · Cited by 358
O'Reilly, R. C. (1996) · Neural Computation · doi:10.1162/neco.1996.8.5.895 · Cited by 352
“Another approach uses a global reinforcement signal instead of specificerror signals ( Mazzoni, Andersen, & Jordan, 1991 ), but this is not as powerful as standard backpropagation.”
MethodEncoding of Intention and Spatial Location in the Posterior Parietal Cortex
Andersen, R. A. (1995) · Cerebral Cortex · doi:10.1093/cercor/5.5.457 · Cited by 351
Self-citationLearning in spiking neural networks by reinforcement of stochastic synaptic transmission
Seung, H. S. (2003) · Neuron · doi:10.1016/s0896-6273(03)00761-x · Cited by 340
“This extra explored by a number of previous models (Williams, difference can help suppress the effects of fluctuations 1992; Barto et al., 1983; Mazzoni et al., 1991).”
BackgroundCoordinate transformations in the representation of spatial information
Andersen, R. A., Snyder, L. H., Li, C.-S., & Stricanne, B. (1993) · Current Opinion in Neurobiology · doi:10.1016/0959-4388(93)90206-e · Cited by 328
Self-citationNeuropsychological inference with an interactive brain: A critique of the “locality” assumption
Farah, M. J. (1994) · Behavioral and Brain Sciences · doi:10.1017/s0140525x00033306 · Cited by 311
To recognize shapes, first learn to generate images
Hinton, G. E. (2007) · Progress in Brain Research · doi:10.1016/s0079-6123(06)65034-6 · Cited by 309
Whittington, J. C. R., & Bogacz, R. (2017) · Neural Computation · doi:10.1162/neco_a_00949 · Cited by 278
“Models in the first class assume that neurons (Barto & Jordan, 1987; Mazzoni, Andersen, & Jordan, 1991; Williams, 1992) or synapses (Unnikrishnan & Venugopal, 1994; Seung, 2003) behave stochastically and receive a global signal describing the error on the output (e.g., via a neuromodulator).”
BackgroundMcClelland, J. L. (2013) · Journal of Experimental Psychology: General · doi:10.1037/a0033812 · Cited by 275
Attention-Gated Reinforcement Learning of Internal Representations for Classification
Roelfsema, P. R., & Ooyen, A. V. (2005) · Neural Computation · doi:10.1162/0899766054615699 · Cited by 260
Motor primitives in space and time via targeted gain modulation in cortical networks
Stroud, J. P., Porter, M. A., Hennequin, G., & Vogels, T. P. (2018) · Nature Neuroscience · doi:10.1038/s41593-018-0276-0 · Cited by 119
“Our rule, which acts on the modulatory pathway of our model but is similar to proposed synaptic plasticity rules for reward-based learning [29, 30, 31, 32], uses a global scalar signal of recent performance to iteratively adjust each neuron’s gain while the initial condition x 0 and the network…”
Activity-difference training of deep neural networks using memristor crossbars
Yi, S.-I., Kendall, J. D., Williams, R. S., & Kumar, S. (2022) · Nature Electronics · doi:10.1038/s41928-022-00869-w · Cited by 75
(2001) · doi:10.4324/9781410600646-11 · Cited by 43
Identification and integration of sensory modalities: Neural basis and relation to consciousness
Pennartz, C. M. A. (2009) · Consciousness and Cognition · doi:10.1016/j.concog.2009.03.003 · Cited by 40
Metadata
- Authors
- Mazzoni, P., Andersen, R. A., & Jordan, M. I.
- Venue
- Proceedings of the National Academy of Sciences
- Published
- 1991-05-15
- Volume
- 88
- Issue
- 10
- Pages
- 4433-4437
- ISSN
- 0027-8424 (ISSN-L)
- 1091-6490
- OpenAlex
- W2008008116
- Cited by
- 227
- References
- 1
- FWCI
- 16.73
- Type
- journal article
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.