CoChat
Sign Up
Cited by 294Neural and Behavioral Psychology StudiesNeural dynamics and brain functionMemory and Neural Mechanisms

A reservoir of time constants for memory traces in cortical neurons

Open access
Bernacchia, Alberto, Seo, Hyojung, Lee, Daeyeol, Wang, Xiao-Jing·2011·Nature Neuroscience
DOI 10.1038/nn.2752PMC PMC3079398
Open this paper in CoChat — free account

Abstract

According to reinforcement learning theory of decision making, reward expectation is computed by integrating past rewards with a fixed timescale. In contrast, we found that a wide range of time constants is available across cortical neurons recorded from monkeys performing a competitive game task. By recognizing that reward modulates neural activity multiplicatively, we found that one or two time constants of reward memory can be extracted for each neuron in prefrontal, cingulate and parietal cortex. These timescales ranged from hundreds of milliseconds to tens of seconds, according to a power law distribution, which is consistent across areas and reproduced by a 'reservoir' neural network model. These neuronal memory timescales were weakly, but significantly, correlated with those of monkey's decisions. Our findings suggest a flexible memory system in which neural subpopulations with distinct sets of long or short memory timescales may be selectively deployed according to the task demands.

AI snapshot

Written from the paper’s full text.

TL;DR

Cortical neurons recorded from monkeys playing a competitive game carry reward-memory traces with a wide range of time constants, distributed as a power law across ACCd, DLPFC, and LIP. A reservoir network model operating near criticality reproduces this distribution, and neural timescales are weakly but significantly correlated with behavioral timescales.

Key findings

  • Across 681 neurons from ACCd (154), DLPFC (322), and LIP (205), reward memory was best fit by a single exponential for 269 neurons, double exponentials for 268 neurons, and no memory for 144 neurons; 79% (537/681) showed exponential decay.
  • Memory timescales ranged from hundreds of milliseconds to tens of seconds and followed a power-law distribution with exponent −2, consistent across the three cortical areas; about 20% of neurons (133/681) had timescales larger than one trial (29% ACCd, 19% DLPFC, 13% LIP, χ²-test p=0.0005).
  • Behavioral timescales from a reinforcement learning fit also followed a power law, and neural and behavioral timescales were weakly but significantly correlated across sessions (R=0.12, p=0.003); simultaneously recorded neuron pairs showed no significant correlation (312 pairs, R=0.07, p=0.2).
  • A randomly connected reservoir network model reproduced the power-law distribution of timescales when operating at the critical point, with exponentially distributed amplitudes and normal connection matrix.

Methods

  • Single-unit activity was recorded from six rhesus monkeys performing a matching pennies task; firing rates were computed in twelve 250-ms epochs per trial, and reward history up to five trials back was used.
  • Memory traces were estimated with multiple linear regression, then fit with single or double exponential functions multiplied by the epoch code; model selection used Bayesian Information Criterion, and reshuffling trials served as a control (96%, 656/681, showed no memory after reshuffling).
  • Behavior was fit per session with a standard reinforcement learning model using maximum likelihood, with significance assessed against 100 reshuffled surrogate sessions; the network model used 1,000 neurons with dynamics dv/dt = J·v(t) + h·Rew(t).

Limitations

  • The paper states that the matching pennies task does not necessarily require reward memory, that the neural–behavioral correlation is weak and might be caused by unknown factors, that the neuronal and behavioral ranges differ, and that behavioral data were not fit with a multi-timescale RL model.
  • The paper also states that the network model is linear and would need nonlinearities if connection-width exceeds the critical level, and that the readout driving behavior is not explicitly modeled.
  • Evident from the text: only about two neurons were simultaneously recorded on average, limiting pairwise analyses, and the study reports correlational rather than causal links between neural timescales and behavior.

Model-written summary. Check it against the paper before relying on it.

Integrity signals

Registry

Crossref · 10.1038/nn.2752

Cites

51 of 51 references verified0 citation errors · 0 not checkable

Statistics

Nothing to recompute.

Claims the paper makes

0 of 59 cited sources checked · 0 supported · 0 flagged

  • Findingown data

    A randomly connected circuit model, akin to a 'reservoir' network, can reproduce the observed distribution of timescales, provided that the network operates at the critical point (or 'edge of chaos').

    “Finally, we show that a randomly connected circuit model, akin to a “reservoir” network13-15, can reproduce the observed distribution of timescales, provided that the network operates at the critical point (or “edge of chaos”)16-18.”
    Quote in the paperSubstantial claim0 of 6 sources checked
    • cites [13] Maass, W., Natschläger, T. & Markram, H. Real-time computing without stable states: a new framework for neural computation based on perturbations. Neural Comput. 14, 2531–2560 (2002). source

      Not checked in the free run.

    • cites [14] Jaeger, H., Lukosevicius, M., Popovici, D. & Siewert, U. Optimization and applications of echo state networks with leaky-integrator neurons. Neural Netw. 20, 335–352 (2007). source

      Not checked in the free run.

    • cites [15] Verstraeten, D., Schrauwen, B., D′Haene, M. & Stroobandt, D. An experimental unification of reservoir computing methods. Neural Netw. 20, 391–403 (2007). source

      Not checked in the free run.

    • cites [16] Sussillo, D. & Abbott, L.F. Generating coherent patterns of activity from chaotic neural networks. Neuron 63, 544–557 (2009). source

      Not checked in the free run.

    • cites [17] Bertschinger, N. & Natschlager, T. Real-time computation at the edge of chaos in recurrent neural networks. Neural Comput. 16, 1413–1436 (2004). source

      Not checked in the free run.

    • cites [18] Langton, C.G. Computation at the edge of chaos: phase transitions and emergent computations. Physica D 42, 12–37 (1990). source

      Not checked in the free run.

    • The authors state this as an assumption: The specific form of the input signals is not important; the results depend only on the synaptic matrix J.
    • The authors state this as an assumption: The connection weights (entries of matrix J) are randomly distributed.
    • The authors state this as an assumption: The linear model is sufficient for reproducing the observed power-law distribution of timescales under specific conditions.
  • Findingown data

    The activity of neurons varied substantially in different trial epochs (99% of neurons, 675/681, ANOVA p<0.05).

    “Consistent with previous studies23-25, we found that the activity of neurons varied substantially in different trial epochs (99% of neurons, 675/681, ANOVA p<0.05).”
    Quote in the paperNot a substantial claim0 of 3 sources checked
    • cites [23] Lapish, C.C., Durstewitz, D., Chandler, L.J. & Seamans, J.K. Successful choice behavior is associated with distinct and coherent network states in anterior cingulate cortex. Proc. Natl. Acad. Sci. USA 105, 11963–11968 (2008). source

      Not checked in the free run.

    • cites [24] Sigala, N., Kusonoki, M., Nimmo-Smith, I., Gaffan, D. & Duncan, J. Hierarchical coding for sequential task events in the monkey prefrontal cortex. Proc. Natl. Acad. Sci. USA 105, 11969–11974 (2008). source

      Not checked in the free run.

    • cites [25] Jin, D.Z., Fujii, N. & Graybiel, A.N. Neural representation of time in cortico-basal ganglia circuits. Proc. Natl. Acad. Sci. USA 106, 19156–19161 (2009). source

      Not checked in the free run.

  • Findingown data

    A 'reservoir' neural network model reproduces the observed distribution of timescales.

    “reproduced by a “reservoir” neural network model.”
    Quote in the paperSubstantial claim
    • The authors state this as an assumption: The specific form of the input signals is not important; the results depend only on the synaptic matrix J.
    • The authors state this as an assumption: The connection weights (entries of matrix J) are randomly distributed.
    • The authors state this as an assumption: The linear model is sufficient for reproducing the observed power-law distribution of timescales under specific conditions.
  • Findingown data

    In some cases, the sum of two exponential functions fitted the data better than a single exponential.

    “In some cases, the sum of two exponential functions, ex(t) = A1·e−t/τ1+A2·e−t/τ2, fitted the data better than a single exponential”
    Quote in the paperNot a substantial claim
    • The authors state this as an assumption: The memory trace f is factorized into the epoch code g(k) and an exponential function ex(t).
  • Findingown data

    Memory time constants can be extracted from the activity of single neurons.

    “we show that memory time constants can be extracted from the activity of single neurons.”
    Quote in the paperSubstantial claim
    • The authors state this as an assumption: The memory trace f is factorized into the epoch code g(k) and an exponential function ex(t).
  • May combine two claims. 26
  • Classified as probably not a substantial claim. 31

  • May combine two claims.

    Findingown data

    The memory trace does not decay monotonically, but its strength is modulated throughout the trial consistent with the epoch code.

    “The memory trace does not decay monotonically, but its strength is modulated throughout the trial consistent with the epoch code.”
    Quote in the paperSubstantial claim
    • The authors state this as an assumption: The memory trace f is factorized into the epoch code g(k) and an exponential function ex(t).

  • May combine two claims.

    Findingown data

    There is a wide range of timescales across neurons, obeying a power law distribution.

    “and there is a wide range of timescales across neurons, obeying a power law distribution.”
    Quote in the paperSubstantial claim
    • The authors state this as an assumption: The memory trace f is factorized into the epoch code g(k) and an exponential function ex(t).

  • May combine two claims.

    Findingown data

    The best fit was a single exponential for 269 neurons and double exponentials for 268 neurons, while the remaining 144 neurons were fitted best by a model with ex(t)=0.

    “Using the Bayesian Information Criterion (BIC), we found that the best fit was a single exponential for 269 neurons and double exponentials for 268 neurons, while the remaining 144 neurons were fitted best by a model with ex(t)=0.”
    Quote in the paperSubstantial claim
    • The authors state this as an assumption: The memory trace f is factorized into the epoch code g(k) and an exponential function ex(t).

  • May combine two claims.

    Findingown data

    These timescales ranged from hundreds of milliseconds to tens of seconds, according to a power-law distribution.

    “These timescales ranged from hundreds of milliseconds to tens of seconds, according to a power-law distribution”
    Quote in the paperSubstantial claim
    • The authors state this as an assumption: The memory trace f is factorized into the epoch code g(k) and an exponential function ex(t).

  • May combine two claims.

    Findingown data

    Different neurons have different magnitudes and time courses of the firing rate, and they all show a decaying memory trace modulated by the epoch code.

    “Different neurons have different magnitudes and time courses of the firing rate, and they all show a decaying memory trace modulated by the epoch code.”
    Quote in the paperSubstantial claim
    • The authors state this as an assumption: The memory trace f is factorized into the epoch code g(k) and an exponential function ex(t).

  • May combine two claims.

    Findingown data

    If J is normal, the amplitudes of the memory traces follow an exponential distribution, consistent with the experimental observations.

    “If J is normal, we showed that the amplitudes of the memory traces follow an exponential distribution (Fig.8b), consistent with the experimental observations (Fig.8a).”
    Quote in the paperSubstantial claim
    • The authors state this as an assumption: The specific form of the input signals is not important; the results depend only on the synaptic matrix J.
    • The authors state this as an assumption: The connection weights (entries of matrix J) are randomly distributed.

  • May combine two claims.

    Findingown data

    The distribution of behavioral timescales followed a power law distribution, and the exponent was consistent with that measured in the neural distribution.

    “For the 196 sessions fitted by the RL model, the distribution of behavioral timescales followed a power law distribution (Fig.5a), and the exponent was consistent with that measured in the neural distribution.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    Both the behavioral and neural memory timescales were fairly stable within a single session.

    “Results are shown in Fig.6, suggesting that both the behavioral (panel (a)) and neural memory timescales (panel (b)) were fairly stable within a single session.”
    Quote in the paperNot a substantial claim
    • The authors state this as an assumption: The memory trace f is factorized into the epoch code g(k) and an exponential function ex(t).

  • May combine two claims.

    Findingown data

    Multiplicative modulation and a power law distribution of memory timescales also hold for memory trace of past choices.

    “Our results indicate that multiplicative modulation and a power law distribution of memory timescales also hold for memory trace of past choices”
    Quote in the paperNot a substantial claim
    • The authors state this as an assumption: The memory trace f is factorized into the epoch code g(k) and an exponential function ex(t).

  • May combine two claims.

    Findingown data

    The connection weights must be broadly distributed among neuron pairs, and this endows the network with a wide variety of timescales.

    “First, we found that the connection weights must be broadly distributed among neuron pairs, and this endows the network with a wide variety of timescales.”
    Quote in the paperSubstantial claim
    • The authors state this as an assumption: The specific form of the input signals is not important; the results depend only on the synaptic matrix J.
    • The authors state this as an assumption: The connection weights (entries of matrix J) are randomly distributed.

  • May combine two claims.

    Findingown data

    About 20% of all recorded neurons (133/681) had timescale larger than one trial (29% in ACCd, 19% in DLPFC and 13% in LIP, χ2-test p=0.0005).

    “About 20% of all recorded neurons (133/681) had timescale larger than one trial (29% in ACCd, 19% in DLPFC and 13% in LIP, χ2-test p=0.0005”
    Quote in the paperSubstantial claim
    • The authors state this as an assumption: The memory trace f is factorized into the epoch code g(k) and an exponential function ex(t).

  • Classified as probably not a substantial claim.

    Interpretationown data

    Neurons fitted best by ex(t)=0 are interpreted as having no memory.

    “The latter is interpreted as no memory, and the corresponding neurons were excluded from further analysis.”
    Quote in the paperSubstantial claim
    • The authors state this as an assumption: The memory trace f is factorized into the epoch code g(k) and an exponential function ex(t).

  • Classified as probably not a substantial claim.

    Premisecitation

    Neural correlates of reward valuation have been interpreted in the framework of reinforcement learning (RL) theory.

    “and interpreted in the framework of reinforcement learning (RL) theory4-5.”
    Quote in the paperSubstantial claim0 of 2 sources checked
    • cites [4] Soltani, A., Lee, D. & Wang, X.-J. Neural mechanism for stochastic behavior during a competitive game. Neural Netw. 19, 1075–1090 (2006). source

      Not checked in the free run.

    • cites [5] Sutton, R.S. & Barto,, A.G. Reinforcement Learning, an Introduction (MIT Press, Cambridge, Massachusetts, 1998). source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Premisecitation

    A few studies have estimated the time constant of the reward filter from behavior, and assessed how past rewards affect choice selection.

    “So far a few studies have estimated the time constant of this filter from behavior, and assessed how past rewards affect choice selection8-12”
    Quote in the paperNot a substantial claim0 of 5 sources checked
    • cites [8] Lau, B. & Glimcher, P.W. Dynamic response-by-response models of matching behavior in rhesus monkeys. J. Exp. Anal. Behav. 84, 555–579 (2005). source

      Not checked in the free run.

    • cites [9] Corrado, G.S., Sugrue, L.P., Seung, H.S. & Newsome, W.T. Linear-nonlinear-Poisson models of primate choice dynamics. J. Exp. Anal. Behav. 84, 581–617 (2005). source

      Not checked in the free run.

    • cites [10] Kennerley, S.W., Walton, M.E., Behrens, T.E., Buckley, M.J. & Rushworth, M.F. Optimal decision making and the anterior cingulate cortex. Nat. Neurosci. 9, 940–947 (2006). source

      Not checked in the free run.

    • cites [11] Lee, D., Conroy, M.L., McGreevy, B.P. & Barraclough, D.J. Reinforcement learning and decision making in monkeys during a competitive game. Brain Res. Cogn. Brain Res. 22, 45–58 (2004). source

      Not checked in the free run.

    • cites [12] Kim, S., Hwang, J., Seo, H. & Lee, D. Valuation of uncertain and delayed rewards in primate prefrontal cortex. Neural Netw. 22, 294–304 (2009). source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    Neural correlates of reward valuation have been observed in different studies.

    “Neural correlates of reward valuation have been observed in different studies1-3”
    Quote in the paperNot a substantial claim0 of 3 sources checked
    • cites [1] Kable, J.W. & Glimcher, P.W. The neurobiology of decision: consensus and controversy. Neuron 63, 733–745 (2009). source

      Not checked in the free run.

    • cites [2] Rushworth, M.F. & Behrens, T.E. Choice, uncertainty and value in prefrontal and cingulate cortex. Nat. Neurosci. 11, 389–397 (2008). source

      Not checked in the free run.

    • cites [3] Wang, X.-J. Decision making in recurrent neural circuits. Neuron 60, 215–234 (2008). source

      Not checked in the free run.

  • May combine two claims.

    Interpretationown data

    Each area is endowed with a reservoir of time constants for reward memory, which are distributed heterogeneously across neurons.

    “Hence, each area is endowed with a reservoir of time constants for reward memory, which are distributed heterogeneously across neurons.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Study design and conductown data

    The fitting procedure is valid: after randomly reshuffling the order of trials in each session, 96% of neurons (656/681) show no memory.

    “We tested the validity of the fitting procedure by randomly reshuffling the order of trials in each session, and we consistently found that 96% of neurons (656/681) show no memory after reshuffling.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Interpretationreasoning

    The double exponential decay of memory may correspond to a reward prediction error signal.

    “Similarly, the double exponential decay of memory may correspond to a reward prediction error signal: if the short timescale (τ1) is small enough (about one trial or smaller) then the corresponding exponential filter will respond primarily to the reward in the present trial, while the long timescale (τ2) may provide a value signal by weighting the rewards in the past few trials.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Interpretationreasoning

    If the short timescale τ1 is about one trial or smaller, the corresponding exponential filter will respond primarily to the reward in the present trial.

    “Similarly, the double exponential decay of memory may correspond to a reward prediction error signal: if the short timescale (τ1) is small enough (about one trial or smaller) then the corresponding exponential filter will respond primarily to the reward in the present trial, while the long timescale (τ2) may provide a value signal by weighting the rewards in the past few trials.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

    Interpretationreasoning

    The animal's behavior may be determined by a readout system that is able to sample, at different times, from a variety of timescales present in the reservoir.

    “The animal's behavior may be determined by a readout system that is able to sample, at different times, from a variety of timescales present in the reservoir.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Interpretationreasoning

    For a power law distribution ~ τ−2, the variance increases with the sample size and arbitrarily large timescales would be observed with a proportionally large increment in the number of recorded neurons.

    “In fact, for a power law distribution ~ τ−2, the variance increases with the sample size and, in principle, arbitrarily large timescales would be observed with a proportionally large increment in the number of recorded neurons.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Premisecitation

    The time course of memory-related activity varies across different neurons and different task protocols, including persistent, ramping, and multi-phasic activity.

    “The time course of memory-related activity varies across different neurons and different task protocols, including persistent, ramping, and multi-phasic activity41-43.”
    Quote in the paperNot a substantial claim0 of 3 sources checked
    • cites [41] Rainer, G. & Miller, E.K. Time course of object-related neural activity in the primate prefrontal cortex during a short-term memory task. Eur. J. Neurosci. 15, 1244–1254 (2002). source

      Not checked in the free run.

    • cites [42] Machens, C.K., Romo, R. & Brody, C.D. Functional, but not anatomical, separation of “what” and “when” in prefrontal cortex. J. Neurosci. 30, 350–360 (2010). source

      Not checked in the free run.

    • cites [43] Shafi, M. et al. Variability in neuronal activity in primate cortex during working memory tasks. Neuroscience 146, 1082–1108 (2007). source

      Not checked in the free run.

  • May combine two claims.

    Interpretationown data

    The same diversity is observed across three cortical areas, suggesting that the computation of reward memory is a distributed process.

    “The same diversity is observed across three cortical areas, suggesting that the computation of reward memory is a distributed process.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Premisecitation

    Anterior cingulate cortex was implicated in monitoring conflict between incompatible response processes or detecting performance errors.

    “While anterior cingulate cortex was implicated in monitoring conflict between incompatible response processes35 or detecting performance errors36”
    Quote in the paperNot a substantial claim0 of 2 sources checked
    • cites [35] Botvinick, M.M., Braver, T.S., Barch, D.M., Carter, C.S. & Cohen, J.D. Conflict monitoring and cognitive control. Psychol. Rev. 108, 624–652 (2001). source

      Not checked in the free run.

    • cites [36] Holroyd, C.B. & Coles, M.G.H. The neural basis of human error processing: reinforcement learning, dopamine and error-related negativity. Psychol. Rev. 109, 679–709 (2002). source

      Not checked in the free run.

  • Classified as probably not a substantial claim.

    Premisereasoning

    The heterogeneous encoding of the input allows the flexible learning of different output functions.

    “The heterogeneous encoding of the input allows the flexible learning of different output functions.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Premisecitation

    Although the overall performance of monkeys was nearly optimal, their trial-by-trial decisions, locally in time, were influenced by previous rewards and actions.

    “Although the overall performance of monkeys was nearly optimal, their trial-by-trial decisions, locally in time, were influenced by previous rewards and actions11,19-22.”
    Quote in the paperSubstantial claim0 of 5 sources checked
    • cites [11] Lee, D., Conroy, M.L., McGreevy, B.P. & Barraclough, D.J. Reinforcement learning and decision making in monkeys during a competitive game. Brain Res. Cogn. Brain Res. 22, 45–58 (2004). source

      Not checked in the free run.

    • cites [19] Seo, H. & Lee, D. Temporal filtering of reward signals in the dorsal anterior cingulate cortex during a mixed-strategy game. J. Neurosci. 27, 8366–8377 (2007). source

      Not checked in the free run.

    • cites [20] Seo, H., Barraclough, D.J. & Lee, D. Dynamic signals related to choices and outcomes in the dorsolateral prefrontal cortex. Cereb. Cortex 17, i110–i117 (2007). source

      Not checked in the free run.

    • cites [21] Seo, H., Barraclough, D.J. & Lee, D. Lateral intraparietal cortex and reinforcement learning during a mixed-strategy game. J. Neurosci. 29, 7278–7289 (2009). source

      Not checked in the free run.

    • cites [22] Barraclough, D.J., Conroy, M.L. & Lee, D. Prefrontal cortex and decision making in a mixed-strategy game. Nat. Neurosci. 7, 404–410 (2004). source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Interpretationown data

    The behaviour might be driven by a mechanism that appropriately samples from a range of timescales in a neural network.

    “This suggests the possibility that the behavior might be driven by a mechanism that appropriately samples from a range of timescales in a neural network, which has yet to be elucidated.”
    Quote in the paperNot a substantial claim

  • Classified as probably not a substantial claim. May combine two claims.

    Premisecitation

    Reservoir models predict that the memory of input signals is stored in a large, recurrent and heterogeneous network in a distributed manner, and that a desired output is obtained by a trainable combination of the response signals in the reservoir.

    “Those models predict that the memory of input signals is stored in a large, recurrent and heterogeneous network (reservoir), in a distributed manner, and that a desired output is obtained by a trainable combination of the response signals in the reservoir.”
    Quote in the paperSubstantial claim0 of 3 sources checked
    • cites [13] Maass, W., Natschläger, T. & Markram, H. Real-time computing without stable states: a new framework for neural computation based on perturbations. Neural Comput. 14, 2531–2560 (2002). source

      Not checked in the free run.

    • cites [14] Jaeger, H., Lukosevicius, M., Popovici, D. & Siewert, U. Optimization and applications of echo state networks with leaky-integrator neurons. Neural Netw. 20, 335–352 (2007). source

      Not checked in the free run.

    • cites [15] Verstraeten, D., Schrauwen, B., D′Haene, M. & Stroobandt, D. An experimental unification of reservoir computing methods. Neural Netw. 20, 391–403 (2007). source

      Not checked in the free run.

  • May combine two claims.

    Study design and conductown data

    The timescales from one-exponential and two-exponential functions were distributed similarly, so they were pooled.

    “Since the timescales from one (τ) and two-exponential functions (τ1, τ2) were distributed similarly (see Fig.S1a,b in the Supplementary Material), we pooled all timescales”
    Quote in the paperSubstantial claim
    • The authors state this as an assumption: The memory trace f is factorized into the epoch code g(k) and an exponential function ex(t).

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Interpretationreasoning

    The long timescale τ2 may provide a value signal by weighting the rewards in the past few trials.

    “Similarly, the double exponential decay of memory may correspond to a reward prediction error signal: if the short timescale (τ1) is small enough (about one trial or smaller) then the corresponding exponential filter will respond primarily to the reward in the present trial, while the long timescale (τ2) may provide a value signal by weighting the rewards in the past few trials.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    Biphasic filtering in dopamine neurons might provide a reward prediction error.

    “It was already noted previously that a biphasic filtering in dopamine neurons might provide a reward prediction error40.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [40] Bayer, H.M. & Glimcher, P.W. Midbrain dopamine neurons encode a quantitative reward prediction error signal. Neuron 47, 129–141 (2005). source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    Prefrontal cortex plays a major role in dynamic decision processes encoding and updating values.

    “Prefrontal cortex plays a major role in dynamic decision processes encoding and updating values1-4.”
    Quote in the paperNot a substantial claim0 of 4 sources checked
    • cites [1] Kable, J.W. & Glimcher, P.W. The neurobiology of decision: consensus and controversy. Neuron 63, 733–745 (2009). source

      Not checked in the free run.

    • cites [2] Rushworth, M.F. & Behrens, T.E. Choice, uncertainty and value in prefrontal and cingulate cortex. Nat. Neurosci. 11, 389–397 (2008). source

      Not checked in the free run.

    • cites [3] Wang, X.-J. Decision making in recurrent neural circuits. Neuron 60, 215–234 (2008). source

      Not checked in the free run.

    • cites [4] Soltani, A., Lee, D. & Wang, X.-J. Neural mechanism for stochastic behavior during a competitive game. Neural Netw. 19, 1075–1090 (2006). source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    Recent studies have placed more emphasis on anterior cingulate cortex's role in representing both positive and negative values.

    “recent studies have placed more emphasis on its role in representing both positive and negative values19,37.”
    Quote in the paperNot a substantial claim0 of 2 sources checked
    • cites [19] Seo, H. & Lee, D. Temporal filtering of reward signals in the dorsal anterior cingulate cortex during a mixed-strategy game. J. Neurosci. 27, 8366–8377 (2007). source

      Not checked in the free run.

    • cites [37] Wallis, J.D. & Kennerley, S.W. Heterogeneous reward signals in prefrontal cortex. Curr. Opin. Neurobiol. 20, 191–198 (2010). source

      Not checked in the free run.

  • Classified as probably not a substantial claim. May combine two claims.

    Premisenone given

    Reservoir-type networks have been the subject of active research in computational neuroscience and machine learning, but experimental support that such networks are adopted by the brain has been lacking.

    “Reservoir-type networks have been the subject of active research in computational neuroscience and machine learning13-15, but so far experimental support that such networks are adopted by the brain has been lacking.”
    Quote in the paperSubstantial claim0 of 3 sources checked
    • cites [13] Maass, W., Natschläger, T. & Markram, H. Real-time computing without stable states: a new framework for neural computation based on perturbations. Neural Comput. 14, 2531–2560 (2002). source

      Not checked in the free run.

    • cites [14] Jaeger, H., Lukosevicius, M., Popovici, D. & Siewert, U. Optimization and applications of echo state networks with leaky-integrator neurons. Neural Netw. 20, 335–352 (2007). source

      Not checked in the free run.

    • cites [15] Verstraeten, D., Schrauwen, B., D′Haene, M. & Stroobandt, D. An experimental unification of reservoir computing methods. Neural Netw. 20, 391–403 (2007). source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Premisecitation

    Parietal cortex has been implicated in decision making based on the value representation as well as on the accumulation of sensory evidence.

    “Parietal cortex has also been implicated in decision making based on the value representation as well as on the accumulation of sensory evidence38,39.”
    Quote in the paperNot a substantial claim0 of 2 sources checked
    • cites [38] Platt, M.L. & Glimcher, P.W. Neural correlates of decision variables in parietal cortex. Nature 400, 233–238 (1999). source

      Not checked in the free run.

    • cites [39] Roitman, J.D. & Shadlen, M.N. Response of neurons in the lateral intraparietal area during a combined visual discrimination reaction time task. J. Neurosci. 22, 9475–9489 (2002). source

      Not checked in the free run.

  • May combine two claims.

    Interpretationreasoning

    Reward filtering in the cortex involves a dynamic reservoir network operating at the critical point, leading to a power-law distribution of time constants.

    “This empirical finding is reproduced by a simple computational model, which suggests that reward filtering in the cortex involves a dynamic reservoir network operating at the critical point, leading to a power-law distribution of time constants.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Study design and conductnone given

    The placement of the recording chamber was guided by magnetic resonance images and confirmed by metal pins inserted in known anatomical locations at the end of the experiment in some animals.

    “The placement of the recording chamber was guided by magnetic resonance (MR) images, and confirmed by metal pins inserted in known anatomical locations at the end of the experiment in some animals.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    A shift from an exploitive to an exploratory strategy as the environment becomes uncertain is often assessed by an increase in the learning rate.

    “This is noteworthy, since a shift from an exploitive to an exploratory strategy as the environment becomes uncertain is often assessed by an increase in the learning rate10.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [10] Kennerley, S.W., Walton, M.E., Behrens, T.E., Buckley, M.J. & Rushworth, M.F. Optimal decision making and the anterior cingulate cortex. Nat. Neurosci. 9, 940–947 (2006). source

      Not checked in the free run.

  • May combine two claims.

    Study design and conductreasoning

    The matrix product (RewT·Rew) is well conditioned, and the solution fsol minimizing the variance of the noise is robust.

    “This implies that the matrix product (RewT·Rew) is well conditioned, and the solution fsol minimizing the variance of the noise (or squared error) is robust and given by (M.6)fsol=(RewT⋅Rew)−1⋅RewT⋅FR”
    Quote in the paperSubstantial claim
    • Rests on an assumption: The sequence of rewards is nearly random.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductnone given

    As a control of the fitting procedure, the label n in the firing rates FR(n,k) was reshuffled, assigning to each firing rate the value of a random trial, and the entire procedure was repeated.

    “As a control of the fitting procedure, we reshuffled the label n in the firing rates FR(n,k), assigning to each firing rate the value of a random trial, and we repeated the entire procedure.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductnone given

    Fitting was repeated ten times for each neuron and each model, in the search for a global minimum of the error.

    “Fitting was repeated ten times for each neuron and each model, in the search for a global minimum of the error.”
    Quote in the paperNot a substantial claim

  • Classified as probably not a substantial claim.

    Study design and conductnone given

    Parameters resulting in unrealistic values, such as negative values of τ, τ1 or τ2, values of τ larger than 20 trials, and the absolute value of A or (A1+A2) larger than 4, were discarded.

    “Any parameters resulting in unrealistic values were discarded, such as negative values of τ, τ1 or τ2, values of τ larger than 20 trials, and the absolute value of A or (A1+A2) larger than 4.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S4 (based).

    Premisereasoning

    A diversity of time constants means a broad range of learning rates, since the two are inversely related to each other.

    “A diversity of time constants also means a broad range of learning rates, since the two are inversely related to each other.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

    Premisenone given

    The matrix product (RewT·Rew) is approximately proportional to the identity matrix.

    “The matrix product (RewT·Rew) is approximately proportional to the identity matrix.”
    Quote in the paperNot a substantial claim
    • Rests on an assumption: The sequence of rewards is nearly random.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisereasoning

    When RewT·Rew=I, the filter is equal to the firing rate averaged over all trials, where the average is conditioned on the past rewards.

    “When RewT·Rew=I, the filter is equal to the firing rate averaged over all trials, where the average is conditioned on the past rewards.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductnone given

    The significance of the reinforcement learning estimation was assessed, for each session, by constructing 100 surrogate sessions obtained by reshuffling the order of trials.

    “The significance of the estimation was assessed, for each session, by constructing 100 surrogate sessions, each one obtained by reshuffling of the order of trials.”
    Quote in the paperNot a substantial claim

  • Classified as probably not a substantial claim.

    Study design and conductown data

    The selection of the appropriate exponential model for each neuron was determined according to the Bayesian information criterion (BIC).

    “Among the three models, the selection of the appropriate one for each neuron was determined according to the Bayesian information criterion (BIC):”
    Quote in the paperSubstantial claim

  • Classified as probably not a substantial claim.

    Premisenone given

    Power-law distributions imply a high probability for both large and small time constants.

    “Power-law distributions are unusual, as they imply a high probability for both large and small time constants.”
    Quote in the paperSubstantial claim

  • Classified as probably not a substantial claim.

    Premisereasoning

    The physical time t depends on all indices k, n and n', because the time elapsed between different epochs and between successive trials is variable due to variability in the time taken by the animal to start a trial and to make a saccade to one of the two targets.

    “The physical time t depends on all indices k, n and n', because the time elapsed between different epochs and between successive trials is variable, due to the variability in the time taken by the animal to start a trial and to make a saccade to one of the two targets.”
    Quote in the paperSubstantial claim

  • Classified as probably not a substantial claim.

    Study design and conductnone given

    All neurons were recorded without pre-screening.

    “All the neurons were recorded without pre-screening.”
    Quote in the paperSubstantial claim
50 of 50

  • The paper relies on this without support.

    The neural mechanism underlying switching between long and short time constants for computing reward expectation remains poorly understood.

    “The neural mechanism underlying switching between long and short time constants for computing reward expectation remains poorly understood.”

  • The paper relies on this without support.

    The neural mechanisms responsible for such timescales are still unknown.

    “but the neural mechanisms responsible for such timescales are still unknown.”

  • The paper relies on this without support.

    The matching pennies task used in this study does not necessarily require the memory of past rewards, and the optimal strategy for the monkey is to choose randomly and unpredictably.

    “The matching pennies task used in this study does not necessarily require the memory of past rewards, and the optimal strategy for the monkey is to choose randomly and unpredictably.”

  • The authors state this as an assumption.

    The effects of rewards in successive trials are additive.

    “assuming that the effects of rewards in successive trials are additive.”

  • The authors state this as an assumption.

    The memory trace f is factorized into the epoch code g(k) and an exponential function ex(t).

    “we assumed that the memory trace f is factorized into the epoch code g(k) and an exponential function ex(t)”

  • The authors state this as an assumption.

    The specific form of the input signals is not important; the results depend only on the synaptic matrix J.

    “For our purposes, the specific form of the input signals is not important; the results will depend only on the synaptic matrix J.”

  • The authors state this as an assumption.

    The connection weights (entries of matrix J) are randomly distributed.

    “We assumed the connection weights (the entries of the matrix J) to be randomly distributed”

  • The authors state this as an assumption.

    The linear model is sufficient for reproducing the observed power-law distribution of timescales under specific conditions.

    “here we limit ourselves to the linear model, which is sufficient for the purpose of reproducing the observed power-law distribution of timescales under specific conditions.”

  • The authors state this as an assumption.

    Memory traces are exponential functions ex(t) rescaled by the epoch code g(k).

    “we assumed that memory traces are exponential function ex(t) rescaled by the epoch code g(k)”

  • The authors state this as an assumption.

    The saccade reaction time of the animal in all trials is equal to 120 ms on average.

    “we assumed that the saccade reaction time of the animal in all trials is equal to 120ms (average)”

  • The authors state this as an assumption.

    The time elapsed between the initiation of two successive trials is 3.4 s (median).

    “and that the time elapsed between the initiation of two successive trials is 3.4s (median).”

  • The paper relies on this without support.

    The sequence of rewards is nearly random.

    “Because the sequence of rewards is nearly random and N is large, different columns of the matrix Rew are nearly orthogonal.”

  • The paper relies on this without support.

    In the matching pennies task, the global optimal strategy is to choose randomly and therefore does not require memory.

    “Although the global optimal strategy for the matching pennies task is to choose randomly and therefore does not require memory, the animals made their decisions largely on the basis of their reward history11,19-22.”
13 of 13

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Similar results are obtained when target desirability is manipulated by relative magnitude, probability, or delay of expected reward.

    “Similar results are obtained in tasks that manipulate the desirability of a target using different methods, for example by controlling the relative magnitude, probability or delay of its expected reward (Bernacchia et al., 2011; Louie et al., 2011; Sugrue et al., 2004; Yang and Shadlen, 2007).”

    From Attention, learning, and the value of information (2012) · cites this paper as Bernacchia et al., 2011

Cites

Flagged first, then by citations

1 of the 52 references couldn’t be matched to a known paper.

Cited by

Sorted by citations

Related

Authors
Bernacchia, A., Seo, H., Lee, D., & Wang, X.-J.
Affiliations
Yale University (US)
Corresponding author
Wang, Xiao-Jing
ORCID
Venue
Nature Neuroscience
Publisher
Springer Science and Business Media LLC
Published
2011-02-13
Volume
14
Issue
3
Pages
366-372
ISSN
  • 1097-6256 (ISSN-L)
  • 1546-1726
DOI
10.1038/nn.2752
PMC
PMC3079398
OpenAlex
W2139630667
Semantic Scholar
87fa62529ba1b9dd0ff3d6e9fcd5b62a26363247
Cited by
292
References
52
Citations by year
2026: 14 · 2025: 13 · 2024: 19 · 2023: 15 · 2022: 23 · 2021: 25 · 2020: 28 · 2019: 19 · 2018: 21 · 2017: 21 · 2016: 16 · 2015: 20 · 2014: 18 · 2013: 15 · 2012: 16
FWCI
9.41
Type
journal article
Language
en
Open access
Licence
www.springer.com/tdm (tdm)
Locations
  • Nature Neuroscience (publishedVersion)
  • PubMed (publishedVersion)
  • PubMed Central (acceptedVersion)
Funders
  • National Institute of Mental Health (R01MH062349, R01MH073246)
  • National Institute on Drug Abuse (DA029330, R01DA029330)
Topics
  • Life Sciences › Neuroscience › Cognitive Neuroscience › Neural and Behavioral Psychology Studies
  • Life Sciences › Neuroscience › Cognitive Neuroscience › Neural dynamics and brain function
  • Life Sciences › Neuroscience › Cognitive Neuroscience › Memory and Neural Mechanisms
Keywords
reward memory, power-law distribution, reinforcement learning, decision-making, prefrontal cortex, cingulate cortex, parietal cortex, reward anticipation, decision time, cortical neurons
MeSH terms
Animals, Haplorhini, Cerebral Cortex, Decision Making, Humans, Memory, Nerve Net, Neurons, Neuropsychological Tests, Reinforcement, Psychology, Reward, Time Factors

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Claims read by deepseek-v4.1-flash under definition version 3 (extract-v3) on 2026-10-10.

Classified by jev-1.13-20260917 (gold-guideline-v0.2) on 2026-10-10.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review