Synaptic and Spiking Dynamics underlying Reward Reversal in the Orbitofrontal Cortex
The DOI matches its Crossref record; 19 of the 19 references it cites were found in a registry or an index.
Abstract
Cognitive and emotional flexibility involve a coordinated interaction between working memory, attention, reward expectations, and the evaluation of rewards and punishers so that behaviour can be changed if necessary. We describe a model at the integrate-and-fire neuronal level of the synaptic and spiking mechanisms which can hold an expectation of a reward rule in working memory, and can reverse the reward rule if expected rewards are not obtained. An example of a reward rule is that stimulus 1 is currently associated with reward, and stimulus 2 with punishment. The attractor-based reward rule working memory incorporates a spike-frequency synaptic adaptation mechanism which supports the neural switching between rules by being shut down by a general inhibitory input produced by punishment, so that when the attractor starts up again is in the opposite state. The mechanism can implement one-trial reward reversal, which is a property of orbitofrontal cortex neurons. We show how this reward rule input can operate in a biased competition way to influence which one of two stimuli is currently associated with reward and which with punishment, and to map the stimuli correctly to the reward or punishment representations, providing a basis for action selection required to obtain the reinforcer.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Neural networks and physical systems with emergent collective computational abilities.
Hopfield, J. J. (1982) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.79.8.2554 · Cited by 19,270 · cited 2×
“We therefore investigate here a mechanism that by utilizing an attractor recurrent autoassociative network (Hopfield , 1982; Rolls and Treves, 1998; Rolls and Deco, 2002) can maintain the rule that is current in a continuing active state of firing until the rule is reversed by punishment or the…”
BackgroundSynaptic plasticity: taming the beast
Abbott, L. F., & Nelson, S. B. (2000) · Nature Neuroscience · doi:10.1038/81453 · Cited by 2,263 · cited 2×
“We also tried other spike-frequency-adapting mechanisms, including Ca 2+ -activated K + hyper-polarizing currents (Liu and Wang, 2001) and short-term synaptic depression (Abbott and Nelson, 2000).”
BackgroundThe Orbitofrontal Cortex and Reward
Rolls, E. T. (2000) · Cerebral Cortex · doi:10.1093/cercor/10.3.284 · Cited by 1,595 · cited 7×
“... might be implemented by having long-term synaptic depression for synapses that represented the reward-associated stimulus before the reversal, and long-term potentiation of the new stimulus that after reversal is associated with reward, this would require one-trial long-term potentiation and one-trial heterosynapt…”
BackgroundInfluentialSelf-citationThe functions of the orbitofrontal cortex
Rolls, E. T. (2004) · Brain and Cognition · doi:10.1016/s0278-2626(03)00277-x · Cited by 1,327
Self-citationA neural basis for visual search in inferior temporal cortex
Chelazzi, L., Miller, E. K., Duncan, J., & Desimone, R. (1993) · Nature · doi:10.1038/363345a0 · Cited by 1,114 · cited 2×
“In fact, the system is dynamically driven according to the biased competition hypothesis (Moran and Desimone, 1985; Spitzer et al., 1988; Chelazzi et al., 1993; Miller et al., 1993; Motter, 1993; Chelazzi, 1998; Reynolds and Desimone, 1999).”
MethodRolls, E. T., Hornak, J., Wade, D., & McGrath, J. (1994) · Journal of Neurology, Neurosurgery & Psychiatry · doi:10.1136/jnnp.57.12.1518 · Cited by 1,005 · cited 4×
“…rapidly to changing reinforcement contingencies produced by damage to brain regions such as the orbitofrontal cortex may be related to the emotional and behavioural changes that follow alteration of the function of these regions (Damasio, 1994; Rolls et al., 1994; Hornak et al., 2003, 2004).”
BackgroundInfluentialInteractions between frontal cortex and basal ganglia in working memory: A computational model
FRANK, M. J., LOUGHRY, B., & O'REILLY, R. C. (2001) · Cognitive, Affective, & Behavioral Neuroscience · doi:10.3758/cabn.1.2.137 · Cited by 879 · cited 2×
“…implemented this gating mechanism by means of a multiplicative term implemented either through phasic dopamine neuromodulation of the frontal cortex by the ventral tegmental area (O'Reilly et al., 2002), or through the interactions between the basal ganglia and frontal cortex (Frank et al., 2001).”
BackgroundMethodSynaptic basis of cortical persistent activity: the importance of NMDA receptors to working memory
Wang, X.-J. (1999) · The Journal of Neuroscience · doi:10.1523/jneurosci.19-21-09587.1999 · Cited by 859 · cited 2×
“As was noted by Lisman et al. (1998), Wang (1999) and Brunel and Wang (2001), the recurrent excitation was assumed to be largely mediated by the NMDA receptors, in order to provide more robust persistent activity during the short-term memory related delay period; and the amplitude of recurrent…”
BackgroundThe orbitofrontal cortex: Neuronal activity in the behaving monkey
Thorpe, S. J., Rolls, E. T., & Maddison, S. (1983) · Experimental Brain Research · doi:10.1007/bf00235545 · Cited by 836 · cited 31×
“In order to investigate the neurodynamics underlying the rapid stimulus--reward reversal in the context of the findings of Thorpe et al. (1983) and Rolls et al. (1996), we explicitly model the level of processes occurring at the AMPA, NMDA and GABA synapses in the integrate-and-fire implementation…”
BackgroundMethodResultInfluentialChanges in emotion after circumscribed surgical lesions of the orbitofrontal and cingulate cortices
Hornak, J. (2003) · Brain · doi:10.1093/brain/awg168 · Cited by 725
Cited by
KRINGELBACH, M. (2004) · Progress in Neurobiology · doi:10.1016/j.pneurobio.2004.03.006 · Cited by 1,977
“These error detection neurons are likely to play an important role in the behavioural changes that are required when reinforcement contingencies change (Deco and Rolls, 2004).”
BackgroundThe functions of the orbitofrontal cortex
Rolls, E. T. (2004) · Brain and Cognition · doi:10.1016/s0278-2626(03)00277-x · Cited by 1,327
Self-citationThe orbitofrontal cortex and beyond: From affect to decision-making
Rolls, E. T., & Grabenhorst, F. (2008) · Progress in Neurobiology · doi:10.1016/j.pneurobio.2008.09.001 · Cited by 807
Triadic model of the neurobiology of motivated behavior in adolescence.
ERNST, M., PINE, D. S., & HARDIN, M. (2006) · Psychological Medicine · doi:10.1017/s0033291705005891 · Cited by 718
Frank, M. J., & Claus, E. D. (2006) · Psychological Review · doi:10.1037/0033-295x.113.2.300 · Cited by 585
The orbitofrontal cortex: reward, emotion and depression
Rolls, E. T., Cheng, W., & Feng, J. (2020) · Brain Communications · doi:10.1093/braincomms/fcaa196 · Cited by 474
Taste, olfactory, and food texture processing in the brain, and the control of food intake
ROLLS, E. (2005) · Physiology & Behavior · doi:10.1016/j.physbeh.2005.04.012 · Cited by 430
Limbic systems for emotion and for memory, but no single limbic system
Rolls, E. T. (2015) · Cortex · doi:10.1016/j.cortex.2013.12.005 · Cited by 428
Stochastic dynamics as a principle of brain function
Deco, G., Rolls, E. T., & Romo, R. (2009) · Progress in Neurobiology · doi:10.1016/j.pneurobio.2009.01.006 · Cited by 382
Self-citationThe orbitofrontal cortex and emotion in health and disease, including depression
Rolls, E. T. (2019) · Neuropsychologia · doi:10.1016/j.neuropsychologia.2017.09.021 · Cited by 329
Attention, short-term memory, and action selection: A unifying theory
Deco, G., & Rolls, E. T. (2005) · Progress in Neurobiology · doi:10.1016/j.pneurobio.2005.08.004 · Cited by 325
Self-citationRolls, E. T. (2023) · Brain Structure and Function · doi:10.1007/s00429-023-02644-9 · Cited by 312
Taste, olfactory, and food reward value processing in the brain
Rolls, E. T. (2015) · Progress in Neurobiology · doi:10.1016/j.pneurobio.2015.03.002 · Cited by 288
Gershman, S. J., & Uchida, N. (2019) · Nature Reviews Neuroscience · doi:10.1038/s41583-019-0220-7 · Cited by 236
Rolls, E. (2008) · Acta Physiologica Hungarica · doi:10.1556/aphysiol.95.2008.2.1 · Cited by 214
Self-citationSensory processing in the brain related to the control of food intake
Rolls, E. T. (2007) · Proceedings of the Nutrition Society · doi:10.1017/s0029665107005332 · Cited by 209
Fusi, S., Asaad, W. F., Miller, E. K., & Wang, X.-J. (2007) · Neuron · doi:10.1016/j.neuron.2007.03.017 · Cited by 199
“…suggest a novel and specific scenario for conditional associative learning, in contrast to a different scenario in which the system does not learn after reversal but switches between prewired neural representations depending on different contexts (Deco and Rolls, 2003, 2005; Salinas, 2004).”
BackgroundA non-reward attractor theory of depression
Rolls, E. T. (2016) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2016.05.007 · Cited by 180
Face-selective and auditory neurons in the primate orbitofrontal cortex
Rolls, E. T., Critchley, H. D., Browning, A. S., & Inoue, K. (2006) · Experimental Brain Research · doi:10.1007/s00221-005-0191-y · Cited by 151
Self-citationA neural network model of individual differences in task switching abilities
Herd, S. A., O׳Reilly, R. C., Hazy, T. E., Chatham, C. H., Brant, A. M., & Friedman, N. P. (2014) · Neuropsychologia · doi:10.1016/j.neuropsychologia.2014.04.014 · Cited by 128
Reward Sensitivity Is Associated with Brain Activity during Erotic Stimulus Processing
Costumero, V., Barrós-Loscertales, A., Bustamante, J. C., Ventura-Campos, N., Fuentes, P., Rosell-Negre, P., & Ávila, C. (2013) · PLoS ONE · doi:10.1371/journal.pone.0066940 · Cited by 45
“Deco and Rolls (2005) proposed that the OFC encodes reward rules (expectations about stimulus-contingency associations) that can be quickly reversed if expected rewards are not obtained [43].”
BackgroundDopamine, inference, and uncertainty
Gershman, S. J. (2017) · Neural Computation · doi:10.1162/neco_a_01023 · Cited by 40
“The idea that recurrent dynamics in OFC play an important role in stimulus representation for reinforcement learning and reward expectation has also figured in earlier models (Deco and Rolls, 2005; Frank and Claus, 2006).”
ResultRolls, E. T. (2012) · Pharmacology Biochemistry and Behavior · doi:10.1016/j.pbb.2011.06.017 · Cited by 39
Metadata
- Authors
- Deco, G.
- Venue
- Cerebral Cortex
- Publisher
- Oxford University Press (OUP)
- Published
- 2004-07-06
- Volume
- 15
- Issue
- 1
- Pages
- 15-30
- ISSN
- 1047-3211 (ISSN-L)
- 1460-2199
- OpenAlex
- W2149646554
- Cited by
- 114
- References
- 19
- FWCI
- 7.04
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.