How Attention Can Create Synaptic Tags for the Learning of Working Memories in Sequential Tasks
Open accessThe DOI matches its Crossref record; 36 of the 36 references it cites were found in a registry or an index.
Abstract
Intelligence is our ability to learn appropriate responses to new stimuli and situations. Neurons in association cortex are thought to be essential for this ability. During learning these neurons become tuned to relevant features and start to represent them with persistent activity during memory delays. This learning process is not well understood. Here we develop a biologically plausible learning scheme that explains how trial-and-error learning induces neuronal selectivity and working memory representations for task-relevant information. We propose that the response selection stage sends attentional feedback signals to earlier processing levels, forming synaptic tags at those connections responsible for the stimulus-response mapping. Globally released neuromodulators then interact with tagged synapses to determine their plasticity. The resulting learning rule endows neural networks with the capacity to create new working memory representations of task relevant information as persistent activity. It is remarkably generic: it explains how association neurons learn to store task-relevant information for linear as well as non-linear stimulus-response mappings, how they become tuned to category boundaries or analog variables, depending on the task demands, and how they learn to integrate probabilistic evidence for perceptual decisions.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Hochreiter, S., & Schmidhuber, J. (1997) · Neural Computation · doi:10.1162/neco.1997.9.8.1735 · Cited by 102,318
Learning representations by back-propagating errors
Rumelhart, D. E., Hinton, G. E., & Williams, R. J. (1986) · Nature · doi:10.1038/323533a0 · Cited by 31,892
The Neural Basis of Decision Making
Gold, J. I., & Shadlen, M. N. (2007) · Annual Review of Neuroscience · doi:10.1146/annurev.neuro.29.051605.113038 · Cited by 3,764 · cited 1×
“Persistent activity in area LIP has also been related to perceptual decision making, because LIP neurons integrate sensory information over time in decision making tasks [43].”
BackgroundMnemonic coding of visual space in the monkey's dorsolateral prefrontal cortex
Funahashi, S., Bruce, C. J., & Goldman-Rakic, P. S. (1989) · Journal of Neurophysiology · doi:10.1152/jn.1989.61.2.331 · Cited by 2,685 · cited 2×
“These properties resemble those of neurons in LIP [2–5] and the frontal cortex [24] of monkeys.”
BackgroundResultGetting formal with dopamine and reward
Schultz, W. (2002) · Neuron · doi:10.1016/s0896-6273(02)00967-4 · Cited by 2,574
The time course of perceptual choice: The leaky, competing accumulator model.
Usher, M., & McClelland, J. L. (2001) · Psychological Review · doi:10.1037/0033-295x.108.3.550 · Cited by 2,152
The multiple-demand (MD) system of the primate brain: mental programs for intelligent behaviour
Duncan, J. (2010) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2010.01.004 · Cited by 1,846 · cited 1×
“Training induces new stimulus and memory representations in ‘multiple-demand’ regions of the cortex [1].”
BackgroundSynaptic tagging and long-term potentiation
Frey, U., & Morris, R. G. M. (1997) · Nature · doi:10.1038/385533a0 · Cited by 1,736 · cited 2×
“Once an action is selected, the unit that codes the winning action a feeds back to earlier processing levels to create synaptic tags [31,32], also known as eligibility traces [7,16] on the responsible synapses (orange pentagons in Fig.”
BackgroundThe importance of mixed selectivity in complex cognitive tasks
Rigotti, M., Barak, O., Warden, M. R., Wang, X.-J., Daw, N. D., Miller, E. K., & Fusi, S. (2013) · Nature · doi:10.1038/nature12160 · Cited by 1,639 · cited 2×
“Furthermore, our results illustrate that AuGMEnT not only trains the association units to integrate stochastic sensory evidence but that it also endows them with the required mixed selectivity for target color and symbol sequence that is required to solve this non-linear task [41].”
BackgroundNeurons in the orbitofrontal cortex encode economic value
Padoa-Schioppa, C., & Assad, J. A. (2006) · Nature · doi:10.1038/nature04676 · Cited by 1,598 · cited 1×
“Neurons representing action values have been found in the frontal cortex, basal ganglia and midbrain [12,35,36] and some orbitofrontal neurons specifically code the chosen value, qa [37].”
BackgroundMultiple dopamine functions at different time courses
Schultz, W. (2007) · Annual Review of Neuroscience · doi:10.1146/annurev.neuro.28.061604.135722 · Cited by 1,415 · cited 2×
“Furthermore, the hypothesis that a neuromodulatory signal, like dopamine, broadcasts the RPE to all synapses in the network is supported by neurobiological findings [9,10,38].”
BackgroundModels of Information Processing in the Basal Ganglia
(1994) · doi:10.7551/mitpress/4708.001.0001 · Cited by 1,190
Categorical Representation of Visual Stimuli in the Primate Prefrontal Cortex
Freedman, D. J., Riesenhuber, M., Poggio, T., & Miller, E. K. (2001) · Science · doi:10.1126/science.291.5502.312 · Cited by 1,123
Memory related motor planning activity in posterior parietal cortex of macaque
Gnadt, J. W., & Andersen, R. A. (1988) · Experimental Brain Research · doi:10.1007/bf00271862 · Cited by 998 · cited 16×
“However, if the animals learn a visual categorization task, persistent activity of LIP cells becomes tuned to the boundary between categories [4] whereas the neurons integrate probabilistic evidence if the task is sensory decision making [5].”
BackgroundInfluentialComputational roles for dopamine in behavioural control
Montague, P. R., Hyman, S. E., & Cohen, J. D. (2004) · Nature · doi:10.1038/nature03015 · Cited by 966
Representation of Action-Specific Reward Values in the Striatum
Samejima, K., Ueda, Y., Doya, K., & Kimura, M. (2005) · Science · doi:10.1126/science.1115270 · Cited by 939 · cited 2×
“The scheme uses units inspired by transient and sustained neurons in sensory cortices [19], actionvalue coding neurons in frontal cortex, basal ganglia and midbrain [12,35,36] and neurons with mnemonic activity that integrate input in association cortex.”
BackgroundMethodReward, Motivation, and Reinforcement Learning
Dayan, P., & Balleine, B. W. (2002) · Neuron · doi:10.1016/s0896-6273(02)00963-7 · Cited by 855
Neuronal correlates of parametric working memory in the prefrontal cortex
Romo, R., Brody, C. D., Hernández, A., & Lemus, L. (1999) · Nature · doi:10.1038/20939 · Cited by 822 · cited 3×
“Our last simulation investigated a task that has been used to study vibrotactile working memory [6,45].”
BackgroundMethodParallel processing strategies of the primate visual system
Nassi, J. J., & Callaway, E. M. (2009) · Nature Reviews Neuroscience · doi:10.1038/nrn2619 · Cited by 802
Graded persistent activity in entorhinal cortex neurons
Egorov, A. V., Hamam, B. N., Fransén, E., Hasselmo, M. E., & Alonso, A. A. (2002) · Nature · doi:10.1038/nature01171 · Cited by 769
On the actions that one nerve cell can have on another: distinguishing “drivers” from “modulators”
Sherman, S. M., & Guillery, R. W. (1998) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.95.12.7121 · Cited by 746
A critical time window for dopamine actions on the structural plasticity of dendritic spines
Yagishita, S., Hayashi-Takagi, A., Ellis-Davies, G. C. R., Urakubo, H., Ishii, S., & Kasai, H. (2014) · Science · doi:10.1126/science.1255514 · Cited by 614
Cascade Models of Synaptically Stored Memories
Fusi, S., Drew, P. J., & Abbott, L. F. (2005) · Neuron · doi:10.1016/j.neuron.2005.02.001 · Cited by 569
Experience-dependent representation of visual categories in parietal cortex
Freedman, D. J., & Assad, J. A. (2006) · Nature · doi:10.1038/nature05078 · Cited by 542 · cited 18×
“However, if the animals learn a visual categorization task, persistent activity of LIP cells becomes tuned to the boundary between categories [4] whereas the neurons integrate probabilistic evidence if the task is sensory decision making [5].”
BackgroundInfluentialCortico–basal ganglia circuit mechanism for a decision threshold in reaction time tasks
Lo, C.-C., & Wang, X.-J. (2006) · Nature Neuroscience · doi:10.1038/nn1722 · Cited by 505
Midbrain dopamine neurons encode decisions for future action
Morris, G., Nevet, A., Arkadir, D., Vaadia, E., & Bergman, H. (2006) · Nature Neuroscience · doi:10.1038/nn1743 · Cited by 503
Attentional control of early perceptual learning.
Ahissar, M., & Hochstein, S. (1993) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.90.12.5718 · Cited by 497 · cited 1×
“If observers select a stimulus for an action, attention invariably shifts to this stimulus [55] and this selective attention signal gates perceptual learning so that attended objects have larger impact on future behavior [56–58].”
BackgroundDorsal raphe neurons signal reward through 5-HT and glutamate
Liu, Z., Zhou, J., Li, Y., Hu, F., Lu, Y., Ma, M., Feng, Q., Zhang, J.-E., Wang, D., Zeng, J., et al. (2014) · Neuron · doi:10.1016/j.neuron.2014.02.010 · Cited by 469 · cited 1×
“Furthermore, a recent study demonstrated that serotonergic neurons also carry a reward-predicting signal and that the optogenetic activation of serotonergic neurons acts as a positive reinforcer [65].”
BackgroundNeuronal correlates of goal-based motor selection in the prefrontal cortex.
Matsumoto, K., Suzuki, W., & Tanaka, K. (2003) · Science · doi:10.1126/science.1084204 · Cited by 384
Basal ganglia orient eyes to reward
Hikosaka, O., Nakamura, K., & Nakahara, H. (2006) · Journal of Neurophysiology · doi:10.1152/jn.00458.2005 · Cited by 377 · cited 2×
“The scheme uses units inspired by transient and sustained neurons in sensory cortices [19], actionvalue coding neurons in frontal cortex, basal ganglia and midbrain [12,35,36] and neurons with mnemonic activity that integrate input in association cortex.”
BackgroundMethodLearning in spiking neural networks by reinforcement of stochastic synaptic transmission
Seung, H. S. (2003) · Neuron · doi:10.1016/s0896-6273(03)00761-x · Cited by 340
Neuronal Correlates of a Perceptual Decision in Ventral Premotor Cortex
Romo, R., Hernández, A., & Zainos, A. (2004) · Neuron · doi:10.1016/s0896-6273(03)00817-1 · Cited by 340
Mechanism of Graded Persistent Cellular Activity of Entorhinal Cortex Layer V Neurons
Fransén, E., Tahvildari, B., Egorov, A. V., Hasselmo, M. E., & Alonso, A. A. (2006) · Neuron · doi:10.1016/j.neuron.2006.01.036 · Cited by 279 · cited 2×
“To keep the model simple, we do not simulate the mechanisms responsible for persistent activity, which have been addressed in previous work [20–22].”
BackgroundMethodDifferent glutamate receptors convey feedforward and recurrent processing in macaque V1
Self, M. W., Kooijmans, R. N., Supèr, H., Lamme, V. A., & Roelfsema, P. R. (2012) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1119527109 · Cited by 184 · cited 1×
“In this context it is of interest that previous studies demonstrated that feedforward propagation of activity to higher cortical areas mainly utilizes the AMPA receptor, whereas feedback effects rely more on the NMDA receptor [85], which plays an important role in synaptic plasticity.”
BackgroundSelf-citationMoncada, D., Ballarini, F., Martinez, M. C., Frey, J. U., & Viola, H. (2011) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1104495108 · Cited by 171
De Pasquale, R., & Sherman, S. M. (2011) · The Journal of Neuroscience · doi:10.1523/jneurosci.3664-11.2011 · Cited by 114 · cited 1×
“NMDA receptors also modify neuronal activity in lower areas, and another candidate receptor that could have a specific role in the influence of feedback connections on plasticity are metabotropic glutamate receptors, which are prominent in feedback pathways [86,87] and known to influence synaptic plasticity [88].”
Background
Cited by
Conscious Processing and the Global Neuronal Workspace Hypothesis
Mashour, G. A., Roelfsema, P., Changeux, J.-P., & Dehaene, S. (2020) · Neuron · doi:10.1016/j.neuron.2020.01.026 · Cited by 1,164
“…ways to train artificial neuronal networks to form task-relevant sequences of memory states (LeCun et al., 2015), and researchers have also started to model the formation of relevant working memories and the transitions between them in the brain (O’Reilly and Frank, 2006; Rombouts et al., 2015).”
BackgroundSelf-citationThe Distributed Nature of Working Memory
Christophel, T. B., Klink, P. C., Spitzer, B., Roelfsema, P. R., & Haynes, J.-D. (2017) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2016.12.007 · Cited by 882
“Indeed, recent computational studies have reported biologically plausible learning rules that can induce persistent activity for working memory, specifically for those features that need to be kept online during memory delays [150,151].”
BackgroundSelf-citationGerstner, W., Lehmann, M., Liakoni, V., Corneil, D., & Brea, J. (2018) · Frontiers in Neural Circuits · doi:10.3389/fncir.2018.00053 · Cited by 290
Toward an integration of deep learning and neuroscience
Marblestone, A. H., Wayne, G., & Kording, K. P. (2016) · Frontiers in Computational Neuroscience · doi:10.3389/fncom.2016.00094 · Cited by 119
Reinforcement learning of linking and tracing contours in recurrent neural networks
Brosch, T., Neumann, H., & Roelfsema, P. R. (2015) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1004489 · Cited by 33
“feedback from the response selection stage in combination with a reward-prediction error, to solve delayed reward tasks [35, 36].”
BackgroundSelf-citation
Metadata
- Authors
- Rombouts, J. O., Bohte, S. M., & Roelfsema, P. R.
- Venue
- PLOS Computational Biology
- Publisher
- Public Library of Science (PLoS)
- Published
- 2015-03-05
- Volume
- 11
- Issue
- 3
- Pages
- e1004060
- ISSN
- 1553-734X (ISSN-L)
- 1553-7358
- OpenAlex
- W1994016657
- Cited by
- 59
- References
- 36
- FWCI
- 4.06
- Type
- journal article
- Language
- en
- Open access
- gold
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.