The DOI matches its Crossref record; 51 of the 52 references it cites were found in a registry or an index; none of the 59 sources its claims cite is checked for support yet.
A reservoir of time constants for memory traces in cortical neurons
Open accessAbstract
According to reinforcement learning theory of decision making, reward expectation is computed by integrating past rewards with a fixed timescale. In contrast, we found that a wide range of time constants is available across cortical neurons recorded from monkeys performing a competitive game task. By recognizing that reward modulates neural activity multiplicatively, we found that one or two time constants of reward memory can be extracted for each neuron in prefrontal, cingulate and parietal cortex. These timescales ranged from hundreds of milliseconds to tens of seconds, according to a power law distribution, which is consistent across areas and reproduced by a 'reservoir' neural network model. These neuronal memory timescales were weakly, but significantly, correlated with those of monkey's decisions. Our findings suggest a flexible memory system in which neural subpopulations with distinct sets of long or short memory timescales may be selectively deployed according to the task demands.
Overview
AI snapshot
Written from the paper’s full text.
TL;DR
Cortical neurons recorded from monkeys playing a competitive game carry reward-memory traces with a wide range of time constants, distributed as a power law across ACCd, DLPFC, and LIP. A reservoir network model operating near criticality reproduces this distribution, and neural timescales are weakly but significantly correlated with behavioral timescales.
Key findings
- Across 681 neurons from ACCd (154), DLPFC (322), and LIP (205), reward memory was best fit by a single exponential for 269 neurons, double exponentials for 268 neurons, and no memory for 144 neurons; 79% (537/681) showed exponential decay.
- Memory timescales ranged from hundreds of milliseconds to tens of seconds and followed a power-law distribution with exponent −2, consistent across the three cortical areas; about 20% of neurons (133/681) had timescales larger than one trial (29% ACCd, 19% DLPFC, 13% LIP, χ²-test p=0.0005).
- Behavioral timescales from a reinforcement learning fit also followed a power law, and neural and behavioral timescales were weakly but significantly correlated across sessions (R=0.12, p=0.003); simultaneously recorded neuron pairs showed no significant correlation (312 pairs, R=0.07, p=0.2).
- A randomly connected reservoir network model reproduced the power-law distribution of timescales when operating at the critical point, with exponentially distributed amplitudes and normal connection matrix.
Methods
- Single-unit activity was recorded from six rhesus monkeys performing a matching pennies task; firing rates were computed in twelve 250-ms epochs per trial, and reward history up to five trials back was used.
- Memory traces were estimated with multiple linear regression, then fit with single or double exponential functions multiplied by the epoch code; model selection used Bayesian Information Criterion, and reshuffling trials served as a control (96%, 656/681, showed no memory after reshuffling).
- Behavior was fit per session with a standard reinforcement learning model using maximum likelihood, with significance assessed against 100 reshuffled surrogate sessions; the network model used 1,000 neurons with dynamics dv/dt = J·v(t) + h·Rew(t).
Limitations
- The paper states that the matching pennies task does not necessarily require reward memory, that the neural–behavioral correlation is weak and might be caused by unknown factors, that the neuronal and behavioral ranges differ, and that behavioral data were not fit with a multi-timescale RL model.
- The paper also states that the network model is linear and would need nonlinearities if connection-width exceeds the critical level, and that the readout driving behavior is not explicitly modeled.
- Evident from the text: only about two neurons were simultaneously recorded on average, limiting pairwise analyses, and the study reports correlational rather than causal links between neural timescales and behavior.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
0 of 59 cited sources checked · 0 supported · 0 flagged
None of the sources cited for this claim have been checked yet.
In the paper
“Finally, we show that a randomly connected circuit model, akin to a “reservoir” network13-15, can reproduce the observed distribution of timescales, provided that the network operates at the critical point (or “edge of chaos”)16-18.”
§ A reservoir of time constants for memory traces in cortical neurons > Abstract
What the sources say
- Not checked yetcites [13] Maass, W., Natschläger, T. & Markram, H. Real-time computing without stable states: a new framework for neural computation based on perturbations. Neural Comput. 14, 2531–2560 (2002). source
- Not checked yetcites [14] Jaeger, H., Lukosevicius, M., Popovici, D. & Siewert, U. Optimization and applications of echo state networks with leaky-integrator neurons. Neural Netw. 20, 335–352 (2007). source
- Not checked yetcites [15] Verstraeten, D., Schrauwen, B., D′Haene, M. & Stroobandt, D. An experimental unification of reservoir computing methods. Neural Netw. 20, 391–403 (2007). source
- Not checked yetcites [16] Sussillo, D. & Abbott, L.F. Generating coherent patterns of activity from chaotic neural networks. Neuron 63, 544–557 (2009). source
- Not checked yetcites [17] Bertschinger, N. & Natschlager, T. Real-time computation at the edge of chaos in recurrent neural networks. Neural Comput. 16, 1413–1436 (2004). source
- Not checked yetcites [18] Langton, C.G. Computation at the edge of chaos: phase transitions and emergent computations. Physica D 42, 12–37 (1990). source
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passes- The authors state this as an assumption: The specific form of the input signals is not important; the results depend only on the synaptic matrix J.
- The authors state this as an assumption: The connection weights (entries of matrix J) are randomly distributed.
- The authors state this as an assumption: The linear model is sufficient for reproducing the observed power-law distribution of timescales under specific conditions.
None of the sources cited for this claim have been checked yet.
In the paper
“Consistent with previous studies23-25, we found that the activity of neurons varied substantially in different trial epochs (99% of neurons, 675/681, ANOVA p<0.05).”
§ A reservoir of time constants for memory traces in cortical neurons > Results > Multiplicative memory traces in cortical neurons
What the sources say
- Not checked yetcites [23] Lapish, C.C., Durstewitz, D., Chandler, L.J. & Seamans, J.K. Successful choice behavior is associated with distinct and coherent network states in anterior cingulate cortex. Proc. Natl. Acad. Sci. USA 105, 11963–11968 (2008). source
- Not checked yetcites [24] Sigala, N., Kusonoki, M., Nimmo-Smith, I., Gaffan, D. & Duncan, J. Hierarchical coding for sequential task events in the monkey prefrontal cortex. Proc. Natl. Acad. Sci. USA 105, 11969–11974 (2008). source
- Not checked yetcites [25] Jin, D.Z., Fujii, N. & Graybiel, A.N. Neural representation of time in cortico-basal ganglia circuits. Proc. Natl. Acad. Sci. USA 106, 19156–19161 (2009). source
Strength:plain· not yet calibratedCausal:correlational· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes? Load-bearing: unclear✓ Based: passesA result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“reproduced by a “reservoir” neural network model.”
§ A reservoir of time constants for memory traces in cortical neurons > Abstract
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passes- The authors state this as an assumption: The specific form of the input signals is not important; the results depend only on the synaptic matrix J.
- The authors state this as an assumption: The connection weights (entries of matrix J) are randomly distributed.
- The authors state this as an assumption: The linear model is sufficient for reproducing the observed power-law distribution of timescales under specific conditions.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“In some cases, the sum of two exponential functions, ex(t) = A1·e−t/τ1+A2·e−t/τ2, fitted the data better than a single exponential”
§ A reservoir of time constants for memory traces in cortical neurons > Results > Multiplicative memory traces in cortical neurons
Strength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes? Load-bearing: unclear✓ Based: passes- The authors state this as an assumption: The memory trace f is factorized into the epoch code g(k) and an exponential function ex(t).
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“we show that memory time constants can be extracted from the activity of single neurons.”
§ A reservoir of time constants for memory traces in cortical neurons > Abstract
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passes- The authors state this as an assumption: The memory trace f is factorized into the epoch code g(k) and an exponential function ex(t).
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“a fraction of them did not show a modulation of the memory by the epoch code.”
§ A reservoir of time constants for memory traces in cortical neurons > Results > Multiplicative memory traces in cortical neurons
Strength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes? Load-bearing: unclear✓ Based: passes- The authors state this as an assumption: The memory trace f is factorized into the epoch code g(k) and an exponential function ex(t).
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“our analysis of animal's behavior suggests that the timescale over which reward events affect decisions changes across experimental sessions”
§ A reservoir of time constants for memory traces in cortical neurons > Abstract
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesA result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The time constants for reward memory at the behavioural and neuronal levels were weakly but significantly correlated across experimental sessions.”
§ A reservoir of time constants for memory traces in cortical neurons > Abstract
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passes- The authors state this as an assumption: The memory trace f is factorized into the epoch code g(k) and an exponential function ex(t).
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“These neuronal memory timescales were weakly but significantly correlated with those of monkey's decisions.”
§ A reservoir of time constants for memory traces in cortical neurons > Abstract
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passes- The authors state this as an assumption: The memory trace f is factorized into the epoch code g(k) and an exponential function ex(t).
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“By recognizing that reward modulates neural activity multiplicatively, we found that one or two time constants of reward memory can be extracted for each neuron in prefrontal, cingulate, and parietal cortex.”
§ A reservoir of time constants for memory traces in cortical neurons > Abstract
Load-bearingStrength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passes- The authors state this as an assumption: The memory trace f is factorized into the epoch code g(k) and an exponential function ex(t).
- It may state two separate claims as one. 26
- Probably not a substantial claim. 31
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The memory trace does not decay monotonically, but its strength is modulated throughout the trial consistent with the epoch code.”
§ A reservoir of time constants for memory traces in cortical neurons > Results > Multiplicative memory traces in cortical neurons
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesIt may state two separate claims as one.
- The authors state this as an assumption: The memory trace f is factorized into the epoch code g(k) and an exponential function ex(t).
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“and there is a wide range of timescales across neurons, obeying a power law distribution.”
§ A reservoir of time constants for memory traces in cortical neurons > Abstract
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesIt may state two separate claims as one.
- The authors state this as an assumption: The memory trace f is factorized into the epoch code g(k) and an exponential function ex(t).
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Using the Bayesian Information Criterion (BIC), we found that the best fit was a single exponential for 269 neurons and double exponentials for 268 neurons, while the remaining 144 neurons were fitted best by a model with ex(t)=0.”
§ A reservoir of time constants for memory traces in cortical neurons > Results > Multiplicative memory traces in cortical neurons
Load-bearingStrength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesIt may state two separate claims as one.
- The authors state this as an assumption: The memory trace f is factorized into the epoch code g(k) and an exponential function ex(t).
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“These timescales ranged from hundreds of milliseconds to tens of seconds, according to a power-law distribution”
§ A reservoir of time constants for memory traces in cortical neurons > Abstract
Load-bearingStrength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesIt may state two separate claims as one.
- The authors state this as an assumption: The memory trace f is factorized into the epoch code g(k) and an exponential function ex(t).
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Different neurons have different magnitudes and time courses of the firing rate, and they all show a decaying memory trace modulated by the epoch code.”
§ A reservoir of time constants for memory traces in cortical neurons > Results > Multiplicative memory traces in cortical neurons
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesIt may state two separate claims as one.
- The authors state this as an assumption: The memory trace f is factorized into the epoch code g(k) and an exponential function ex(t).
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“If J is normal, we showed that the amplitudes of the memory traces follow an exponential distribution (Fig.8b), consistent with the experimental observations (Fig.8a).”
§ A reservoir of time constants for memory traces in cortical neurons > Results > Neural network model for memory traces
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesIt may state two separate claims as one.
- The authors state this as an assumption: The specific form of the input signals is not important; the results depend only on the synaptic matrix J.
- The authors state this as an assumption: The connection weights (entries of matrix J) are randomly distributed.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“For the 196 sessions fitted by the RL model, the distribution of behavioral timescales followed a power law distribution (Fig.5a), and the exponent was consistent with that measured in the neural distribution.”
§ A reservoir of time constants for memory traces in cortical neurons > Results > Comparison with behavior
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesIt may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Results are shown in Fig.6, suggesting that both the behavioral (panel (a)) and neural memory timescales (panel (b)) were fairly stable within a single session.”
§ A reservoir of time constants for memory traces in cortical neurons > Results > Comparison with behavior
Strength:hedged· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes? Load-bearing: unclear✓ Based: passesIt may state two separate claims as one.
- The authors state this as an assumption: The memory trace f is factorized into the epoch code g(k) and an exponential function ex(t).
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Our results indicate that multiplicative modulation and a power law distribution of memory timescales also hold for memory trace of past choices”
§ A reservoir of time constants for memory traces in cortical neurons > Results > Comparison with behavior
Strength:hedged· not yet calibratedCausal:correlational· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes? Load-bearing: unclear✓ Based: passesIt may state two separate claims as one.
- The authors state this as an assumption: The memory trace f is factorized into the epoch code g(k) and an exponential function ex(t).
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“First, we found that the connection weights must be broadly distributed among neuron pairs, and this endows the network with a wide variety of timescales.”
§ A reservoir of time constants for memory traces in cortical neurons > Results > Neural network model for memory traces
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesIt may state two separate claims as one.
- The authors state this as an assumption: The specific form of the input signals is not important; the results depend only on the synaptic matrix J.
- The authors state this as an assumption: The connection weights (entries of matrix J) are randomly distributed.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“About 20% of all recorded neurons (133/681) had timescale larger than one trial (29% in ACCd, 19% in DLPFC and 13% in LIP, χ2-test p=0.0005”
§ A reservoir of time constants for memory traces in cortical neurons > Results > Multiplicative memory traces in cortical neurons
Load-bearingStrength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesIt may state two separate claims as one.
- The authors state this as an assumption: The memory trace f is factorized into the epoch code g(k) and an exponential function ex(t).
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The latter is interpreted as no memory, and the corresponding neurons were excluded from further analysis.”
§ A reservoir of time constants for memory traces in cortical neurons > Results > Multiplicative memory traces in cortical neurons
Load-bearingStrength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesProbably not a substantial claim.
- The authors state this as an assumption: The memory trace f is factorized into the epoch code g(k) and an exponential function ex(t).
None of the sources cited for this claim have been checked yet.
In the paper
“and interpreted in the framework of reinforcement learning (RL) theory4-5.”
§ A reservoir of time constants for memory traces in cortical neurons > Abstract
What the sources say
- Not checked yetcites [4] Soltani, A., Lee, D. & Wang, X.-J. Neural mechanism for stochastic behavior during a competitive game. Neural Netw. 19, 1075–1090 (2006). source
- Not checked yetcites [5] Sutton, R.S. & Barto,, A.G. Reinforcement Learning, an Introduction (MIT Press, Cambridge, Massachusetts, 1998). source
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesProbably not a substantial claim.
None of the sources cited for this claim have been checked yet.
In the paper
“So far a few studies have estimated the time constant of this filter from behavior, and assessed how past rewards affect choice selection8-12”
§ A reservoir of time constants for memory traces in cortical neurons > Abstract
What the sources say
- Not checked yetcites [8] Lau, B. & Glimcher, P.W. Dynamic response-by-response models of matching behavior in rhesus monkeys. J. Exp. Anal. Behav. 84, 555–579 (2005). source
- Not checked yetcites [9] Corrado, G.S., Sugrue, L.P., Seung, H.S. & Newsome, W.T. Linear-nonlinear-Poisson models of primate choice dynamics. J. Exp. Anal. Behav. 84, 581–617 (2005). source
- Not checked yetcites [10] Kennerley, S.W., Walton, M.E., Behrens, T.E., Buckley, M.J. & Rushworth, M.F. Optimal decision making and the anterior cingulate cortex. Nat. Neurosci. 9, 940–947 (2006). source
- Not checked yetcites [11] Lee, D., Conroy, M.L., McGreevy, B.P. & Barraclough, D.J. Reinforcement learning and decision making in monkeys during a competitive game. Brain Res. Cogn. Brain Res. 22, 45–58 (2004). source
- Not checked yetcites [12] Kim, S., Hwang, J., Seo, H. & Lee, D. Valuation of uncertain and delayed rewards in primate prefrontal cortex. Neural Netw. 22, 294–304 (2009). source
Strength:plain· not yet calibratedCausal:correlational· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.
None of the sources cited for this claim have been checked yet.
In the paper
“Neural correlates of reward valuation have been observed in different studies1-3”
§ A reservoir of time constants for memory traces in cortical neurons > Abstract
What the sources say
- Not checked yetcites [1] Kable, J.W. & Glimcher, P.W. The neurobiology of decision: consensus and controversy. Neuron 63, 733–745 (2009). source
- Not checked yetcites [2] Rushworth, M.F. & Behrens, T.E. Choice, uncertainty and value in prefrontal and cingulate cortex. Nat. Neurosci. 11, 389–397 (2008). source
- Not checked yetcites [3] Wang, X.-J. Decision making in recurrent neural circuits. Neuron 60, 215–234 (2008). source
Strength:plain· not yet calibratedCausal:correlational· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Hence, each area is endowed with a reservoir of time constants for reward memory, which are distributed heterogeneously across neurons.”
§ A reservoir of time constants for memory traces in cortical neurons > Abstract
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesIt may state two separate claims as one.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“We tested the validity of the fitting procedure by randomly reshuffling the order of trials in each session, and we consistently found that 96% of neurons (656/681) show no memory after reshuffling.”
§ A reservoir of time constants for memory traces in cortical neurons > Results > Multiplicative memory traces in cortical neurons
Load-bearingStrength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesIt may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Similarly, the double exponential decay of memory may correspond to a reward prediction error signal: if the short timescale (τ1) is small enough (about one trial or smaller) then the corresponding exponential filter will respond primarily to the reward in the present trial, while the long timescale (τ2) may provide a value signal by weighting the rewards in the past few trials.”
§ A reservoir of time constants for memory traces in cortical neurons > Discussion
Strength:hedged· not yet calibratedCausal:conditional· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Similarly, the double exponential decay of memory may correspond to a reward prediction error signal: if the short timescale (τ1) is small enough (about one trial or smaller) then the corresponding exponential filter will respond primarily to the reward in the present trial, while the long timescale (τ2) may provide a value signal by weighting the rewards in the past few trials.”
§ A reservoir of time constants for memory traces in cortical neurons > Discussion
Strength:hedged· not yet calibratedCausal:conditional· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The animal's behavior may be determined by a readout system that is able to sample, at different times, from a variety of timescales present in the reservoir.”
§ A reservoir of time constants for memory traces in cortical neurons > Results > Comparison with behavior
Strength:hedged· not yet calibratedCausal:conditional· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✗ Based: failsProbably not a substantial claim. The paper’s conclusions may not depend on it. The paper may give no reason why it holds.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“In fact, for a power law distribution ~ τ−2, the variance increases with the sample size and, in principle, arbitrarily large timescales would be observed with a proportionally large increment in the number of recorded neurons.”
§ A reservoir of time constants for memory traces in cortical neurons > Results > Multiplicative memory traces in cortical neurons
Strength:hedged· not yet calibratedCausal:direct· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes? Load-bearing: unclear✓ Based: passesIt may state two separate claims as one.
None of the sources cited for this claim have been checked yet.
In the paper
“The time course of memory-related activity varies across different neurons and different task protocols, including persistent, ramping, and multi-phasic activity41-43.”
§ A reservoir of time constants for memory traces in cortical neurons > Discussion
What the sources say
- Not checked yetcites [41] Rainer, G. & Miller, E.K. Time course of object-related neural activity in the primate prefrontal cortex during a short-term memory task. Eur. J. Neurosci. 15, 1244–1254 (2002). source
- Not checked yetcites [42] Machens, C.K., Romo, R. & Brody, C.D. Functional, but not anatomical, separation of “what” and “when” in prefrontal cortex. J. Neurosci. 30, 350–360 (2010). source
- Not checked yetcites [43] Shafi, M. et al. Variability in neuronal activity in primate cortex during working memory tasks. Neuroscience 146, 1082–1108 (2007). source
Strength:plain· not yet calibratedCausal:correlational· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The same diversity is observed across three cortical areas, suggesting that the computation of reward memory is a distributed process.”
§ A reservoir of time constants for memory traces in cortical neurons > Discussion
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesIt may state two separate claims as one.
None of the sources cited for this claim have been checked yet.
In the paper
“While anterior cingulate cortex was implicated in monitoring conflict between incompatible response processes35 or detecting performance errors36”
§ A reservoir of time constants for memory traces in cortical neurons > Discussion
What the sources say
- Not checked yetcites [35] Botvinick, M.M., Braver, T.S., Barch, D.M., Carter, C.S. & Cohen, J.D. Conflict monitoring and cognitive control. Psychol. Rev. 108, 624–652 (2001). source
- Not checked yetcites [36] Holroyd, C.B. & Coles, M.G.H. The neural basis of human error processing: reinforcement learning, dopamine and error-related negativity. Psychol. Rev. 109, 679–709 (2002). source
Strength:plain· not yet calibratedCausal:correlational· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“The heterogeneous encoding of the input allows the flexible learning of different output functions.”
§ A reservoir of time constants for memory traces in cortical neurons > Discussion
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesProbably not a substantial claim.
None of the sources cited for this claim have been checked yet.
In the paper
“Although the overall performance of monkeys was nearly optimal, their trial-by-trial decisions, locally in time, were influenced by previous rewards and actions11,19-22.”
§ A reservoir of time constants for memory traces in cortical neurons > Results > Comparison with behavior
What the sources say
- Not checked yetcites [11] Lee, D., Conroy, M.L., McGreevy, B.P. & Barraclough, D.J. Reinforcement learning and decision making in monkeys during a competitive game. Brain Res. Cogn. Brain Res. 22, 45–58 (2004). source
- Not checked yetcites [19] Seo, H. & Lee, D. Temporal filtering of reward signals in the dorsal anterior cingulate cortex during a mixed-strategy game. J. Neurosci. 27, 8366–8377 (2007). source
- Not checked yetcites [20] Seo, H., Barraclough, D.J. & Lee, D. Dynamic signals related to choices and outcomes in the dorsolateral prefrontal cortex. Cereb. Cortex 17, i110–i117 (2007). source
- Not checked yetcites [21] Seo, H., Barraclough, D.J. & Lee, D. Lateral intraparietal cortex and reinforcement learning during a mixed-strategy game. J. Neurosci. 29, 7278–7289 (2009). source
- Not checked yetcites [22] Barraclough, D.J., Conroy, M.L. & Lee, D. Prefrontal cortex and decision making in a mixed-strategy game. Nat. Neurosci. 7, 404–410 (2004). source
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesIt may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“This suggests the possibility that the behavior might be driven by a mechanism that appropriately samples from a range of timescales in a neural network, which has yet to be elucidated.”
§ A reservoir of time constants for memory traces in cortical neurons > Discussion
Strength:hedged· not yet calibratedCausal:conditional· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it.
None of the sources cited for this claim have been checked yet.
In the paper
“Those models predict that the memory of input signals is stored in a large, recurrent and heterogeneous network (reservoir), in a distributed manner, and that a desired output is obtained by a trainable combination of the response signals in the reservoir.”
§ A reservoir of time constants for memory traces in cortical neurons > Discussion
What the sources say
- Not checked yetcites [13] Maass, W., Natschläger, T. & Markram, H. Real-time computing without stable states: a new framework for neural computation based on perturbations. Neural Comput. 14, 2531–2560 (2002). source
- Not checked yetcites [14] Jaeger, H., Lukosevicius, M., Popovici, D. & Siewert, U. Optimization and applications of echo state networks with leaky-integrator neurons. Neural Netw. 20, 335–352 (2007). source
- Not checked yetcites [15] Verstraeten, D., Schrauwen, B., D′Haene, M. & Stroobandt, D. An experimental unification of reservoir computing methods. Neural Netw. 20, 391–403 (2007). source
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesProbably not a substantial claim. It may state two separate claims as one.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“Since the timescales from one (τ) and two-exponential functions (τ1, τ2) were distributed similarly (see Fig.S1a,b in the Supplementary Material), we pooled all timescales”
§ A reservoir of time constants for memory traces in cortical neurons > Results > Multiplicative memory traces in cortical neurons
Load-bearingStrength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesIt may state two separate claims as one.
- The authors state this as an assumption: The memory trace f is factorized into the epoch code g(k) and an exponential function ex(t).
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Similarly, the double exponential decay of memory may correspond to a reward prediction error signal: if the short timescale (τ1) is small enough (about one trial or smaller) then the corresponding exponential filter will respond primarily to the reward in the present trial, while the long timescale (τ2) may provide a value signal by weighting the rewards in the past few trials.”
§ A reservoir of time constants for memory traces in cortical neurons > Discussion
Strength:hedged· not yet calibratedCausal:conditional· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it.
In the paper
“It was already noted previously that a biphasic filtering in dopamine neurons might provide a reward prediction error40.”
§ A reservoir of time constants for memory traces in cortical neurons > Discussion
What the sources say
- Not checked yetcites [40] Bayer, H.M. & Glimcher, P.W. Midbrain dopamine neurons encode a quantitative reward prediction error signal. Neuron 47, 129–141 (2005). source
Strength:hedged· not yet calibratedCausal:conditional· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it.
None of the sources cited for this claim have been checked yet.
In the paper
“Prefrontal cortex plays a major role in dynamic decision processes encoding and updating values1-4.”
§ A reservoir of time constants for memory traces in cortical neurons > Discussion
What the sources say
- Not checked yetcites [1] Kable, J.W. & Glimcher, P.W. The neurobiology of decision: consensus and controversy. Neuron 63, 733–745 (2009). source
- Not checked yetcites [2] Rushworth, M.F. & Behrens, T.E. Choice, uncertainty and value in prefrontal and cingulate cortex. Nat. Neurosci. 11, 389–397 (2008). source
- Not checked yetcites [3] Wang, X.-J. Decision making in recurrent neural circuits. Neuron 60, 215–234 (2008). source
- Not checked yetcites [4] Soltani, A., Lee, D. & Wang, X.-J. Neural mechanism for stochastic behavior during a competitive game. Neural Netw. 19, 1075–1090 (2006). source
Strength:plain· not yet calibratedCausal:correlational· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it.
None of the sources cited for this claim have been checked yet.
In the paper
“recent studies have placed more emphasis on its role in representing both positive and negative values19,37.”
§ A reservoir of time constants for memory traces in cortical neurons > Discussion
What the sources say
- Not checked yetcites [19] Seo, H. & Lee, D. Temporal filtering of reward signals in the dorsal anterior cingulate cortex during a mixed-strategy game. J. Neurosci. 27, 8366–8377 (2007). source
- Not checked yetcites [37] Wallis, J.D. & Kennerley, S.W. Heterogeneous reward signals in prefrontal cortex. Curr. Opin. Neurobiol. 20, 191–198 (2010). source
Strength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it.
None of the sources cited for this claim have been checked yet.
In the paper
“Reservoir-type networks have been the subject of active research in computational neuroscience and machine learning13-15, but so far experimental support that such networks are adopted by the brain has been lacking.”
§ A reservoir of time constants for memory traces in cortical neurons > Discussion
What the sources say
- Not checked yetcites [13] Maass, W., Natschläger, T. & Markram, H. Real-time computing without stable states: a new framework for neural computation based on perturbations. Neural Comput. 14, 2531–2560 (2002). source
- Not checked yetcites [14] Jaeger, H., Lukosevicius, M., Popovici, D. & Siewert, U. Optimization and applications of echo state networks with leaky-integrator neurons. Neural Netw. 20, 335–352 (2007). source
- Not checked yetcites [15] Verstraeten, D., Schrauwen, B., D′Haene, M. & Stroobandt, D. An experimental unification of reservoir computing methods. Neural Netw. 20, 391–403 (2007). source
Load-bearingStrength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesProbably not a substantial claim. It may state two separate claims as one.
None of the sources cited for this claim have been checked yet.
In the paper
“Parietal cortex has also been implicated in decision making based on the value representation as well as on the accumulation of sensory evidence38,39.”
§ A reservoir of time constants for memory traces in cortical neurons > Discussion
What the sources say
- Not checked yetcites [38] Platt, M.L. & Glimcher, P.W. Neural correlates of decision variables in parietal cortex. Nature 400, 233–238 (1999). source
- Not checked yetcites [39] Roitman, J.D. & Shadlen, M.N. Response of neurons in the lateral intraparietal area during a combined visual discrimination reaction time task. J. Neurosci. 22, 9475–9489 (2002). source
Strength:plain· not yet calibratedCausal:correlational· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“This empirical finding is reproduced by a simple computational model, which suggests that reward filtering in the cortex involves a dynamic reservoir network operating at the critical point, leading to a power-law distribution of time constants.”
§ A reservoir of time constants for memory traces in cortical neurons > Discussion
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesIt may state two separate claims as one.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“The placement of the recording chamber was guided by magnetic resonance (MR) images, and confirmed by metal pins inserted in known anatomical locations at the end of the experiment in some animals.”
§ A reservoir of time constants for memory traces in cortical neurons > Methods > Animal preparation and electrophysiological recording
Strength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.
In the paper
“This is noteworthy, since a shift from an exploitive to an exploratory strategy as the environment becomes uncertain is often assessed by an increase in the learning rate10.”
§ A reservoir of time constants for memory traces in cortical neurons > Discussion
What the sources say
- Not checked yetcites [10] Kennerley, S.W., Walton, M.E., Behrens, T.E., Buckley, M.J. & Rushworth, M.F. Optimal decision making and the anterior cingulate cortex. Nat. Neurosci. 9, 940–947 (2006). source
Strength:plain· not yet calibratedCausal:correlational· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“This implies that the matrix product (RewT·Rew) is well conditioned, and the solution fsol minimizing the variance of the noise (or squared error) is robust and given by (M.6)fsol=(RewT⋅Rew)−1⋅RewT⋅FR”
§ A reservoir of time constants for memory traces in cortical neurons > Methods > Multiple regression analysis of memory traces
Load-bearingStrength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesIt may state two separate claims as one.
- Rests on an assumption: The sequence of rewards is nearly random.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“As a control of the fitting procedure, we reshuffled the label n in the firing rates FR(n,k), assigning to each firing rate the value of a random trial, and we repeated the entire procedure.”
§ A reservoir of time constants for memory traces in cortical neurons > Methods > Exponential memory traces and model selection
Strength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“Fitting was repeated ten times for each neuron and each model, in the search for a global minimum of the error.”
§ A reservoir of time constants for memory traces in cortical neurons > Methods > Exponential memory traces and model selection
Strength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“Any parameters resulting in unrealistic values were discarded, such as negative values of τ, τ1 or τ2, values of τ larger than 20 trials, and the absolute value of A or (A1+A2) larger than 4.”
§ A reservoir of time constants for memory traces in cortical neurons > Methods > Exponential memory traces and model selection
Load-bearingStrength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesProbably not a substantial claim.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“A diversity of time constants also means a broad range of learning rates, since the two are inversely related to each other.”
§ A reservoir of time constants for memory traces in cortical neurons > Discussion
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✗ Based: failsProbably not a substantial claim. The paper may give no reason why it holds.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“The matrix product (RewT·Rew) is approximately proportional to the identity matrix.”
§ A reservoir of time constants for memory traces in cortical neurons > Methods > Multiple regression analysis of memory traces
Strength:hedged· not yet calibratedCausal:correlational· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✗ Based: failsProbably not a substantial claim. The paper’s conclusions may not depend on it. The paper may give no reason why it holds.
- Rests on an assumption: The sequence of rewards is nearly random.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“When RewT·Rew=I, the filter is equal to the firing rate averaged over all trials, where the average is conditioned on the past rewards.”
§ A reservoir of time constants for memory traces in cortical neurons > Methods > Multiple regression analysis of memory traces
Strength:plain· not yet calibratedCausal:correlational· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“The significance of the estimation was assessed, for each session, by constructing 100 surrogate sessions, each one obtained by reshuffling of the order of trials.”
§ A reservoir of time constants for memory traces in cortical neurons > Methods > Reinforcement learning fit of behavior
Strength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“Among the three models, the selection of the appropriate one for each neuron was determined according to the Bayesian information criterion (BIC):”
§ A reservoir of time constants for memory traces in cortical neurons > Methods > Exponential memory traces and model selection
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesProbably not a substantial claim.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“Power-law distributions are unusual, as they imply a high probability for both large and small time constants.”
§ A reservoir of time constants for memory traces in cortical neurons > Discussion
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesProbably not a substantial claim.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“The physical time t depends on all indices k, n and n', because the time elapsed between different epochs and between successive trials is variable, due to the variability in the time taken by the animal to start a trial and to make a saccade to one of the two targets.”
§ A reservoir of time constants for memory traces in cortical neurons > Methods > Exponential memory traces and model selection
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesProbably not a substantial claim.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“All the neurons were recorded without pre-screening.”
§ A reservoir of time constants for memory traces in cortical neurons > Methods > Animal preparation and electrophysiological recording
Load-bearingStrength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesProbably not a substantial claim.
The paper relies on this without support.
The neural mechanism underlying switching between long and short time constants for computing reward expectation remains poorly understood.
“The neural mechanism underlying switching between long and short time constants for computing reward expectation remains poorly understood.”
The paper relies on this without support.
The neural mechanisms responsible for such timescales are still unknown.
“but the neural mechanisms responsible for such timescales are still unknown.”
The paper relies on this without support.
The matching pennies task used in this study does not necessarily require the memory of past rewards, and the optimal strategy for the monkey is to choose randomly and unpredictably.
“The matching pennies task used in this study does not necessarily require the memory of past rewards, and the optimal strategy for the monkey is to choose randomly and unpredictably.”
The authors state this as an assumption.
The effects of rewards in successive trials are additive.
“assuming that the effects of rewards in successive trials are additive.”
The authors state this as an assumption.
The memory trace f is factorized into the epoch code g(k) and an exponential function ex(t).
“we assumed that the memory trace f is factorized into the epoch code g(k) and an exponential function ex(t)”
The authors state this as an assumption.
The specific form of the input signals is not important; the results depend only on the synaptic matrix J.
“For our purposes, the specific form of the input signals is not important; the results will depend only on the synaptic matrix J.”
The authors state this as an assumption.
The connection weights (entries of matrix J) are randomly distributed.
“We assumed the connection weights (the entries of the matrix J) to be randomly distributed”
The authors state this as an assumption.
The linear model is sufficient for reproducing the observed power-law distribution of timescales under specific conditions.
“here we limit ourselves to the linear model, which is sufficient for the purpose of reproducing the observed power-law distribution of timescales under specific conditions.”
The authors state this as an assumption.
Memory traces are exponential functions ex(t) rescaled by the epoch code g(k).
“we assumed that memory traces are exponential function ex(t) rescaled by the epoch code g(k)”
The authors state this as an assumption.
The saccade reaction time of the animal in all trials is equal to 120 ms on average.
“we assumed that the saccade reaction time of the animal in all trials is equal to 120ms (average)”
The authors state this as an assumption.
The time elapsed between the initiation of two successive trials is 3.4 s (median).
“and that the time elapsed between the initiation of two successive trials is 3.4s (median).”
The paper relies on this without support.
The sequence of rewards is nearly random.
“Because the sequence of rewards is nearly random and N is large, different columns of the matrix Rew are nearly orthogonal.”
The paper relies on this without support.
In the matching pennies task, the global optimal strategy is to choose randomly and therefore does not require memory.
“Although the global optimal strategy for the matching pennies task is to choose randomly and therefore does not require memory, the animals made their decisions largely on the basis of their reward history11,19-22.”
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Similar results are obtained in tasks that manipulate the desirability of a target using different methods, for example by controlling the relative magnitude, probability or delay of its expected reward (Bernacchia et al., 2011; Louie et al., 2011; Sugrue et al., 2004; Yang and Shadlen, 2007).”
From Attention, learning, and the value of information (2012) · cites this paper as Bernacchia et al., 2011
Citations
Cites
Reinforcement Learning: An Introduction
Sutton, R. S., & Barto, A. G. (1998) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.1998.712192 · Cited by 27,225 · cited 1×
“(5) The seven unknown variables have been rewritten by a single vector f”
BackgroundReinforcement Learning: An Introduction
Sutton, R. S., & Barto, A. (2005) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.2004.842673 · Cited by 25,268
Conflict monitoring and cognitive control
Botvinick, M. M., Braver, T. S., Barch, D. M., Carter, C. S., & Cohen, J. D. (2001) · Psychological Review · doi:10.1037/0033-295x.108.3.624 · Cited by 7,351 · cited 1×
“Although anterior cingulate cortex has been implicated in monitoring conflict between incompatible response processe”
MethodHolroyd, C. B., & Coles, M. G. H. (2002) · Psychological Review · doi:10.1037/0033-295x.109.4.679 · Cited by 4,027
Maass, W., Natschläger, T., & Markram, H. (2002) · Neural Computation · doi:10.1162/089976602760407955 · Cited by 4,025 · cited 2×
“If a single value function (for a given stimulus/action) and a single reward (delivered at time zero) are considered, the solution of equation (13) can be approximated by an exponential response,”
BackgroundCortical substrates for exploratory decisions in humans
Daw, N. D., O'Doherty, J. P., Dayan, P., Seymour, B., & Dolan, R. J. (2006) · Nature · doi:10.1038/nature04766 · Cited by 2,224 · cited 1×
“If the payoffs for the same option change often and unpredictably, then rewards should be filtered on short timescales to track the fast changes in a volatile environment; in contrast, if past rewards reliably predict future ones, then they should be filtered on long timescales to exploit a stable environmen”
BackgroundLearning the value of information in an uncertain world
Behrens, T. E. J., Woolrich, M. W., Walton, M. E., & Rushworth, M. F. S. (2007) · Nature Neuroscience · doi:10.1038/nn1954 · Cited by 2,053 · cited 1×
“Any parameters resulting in unrealistic values were discarded, such as negative values of τ, τ1 or τ2, values of (6)”
BackgroundWeak pairwise correlations imply strongly correlated network states in a neural population
Schneidman, E., Berry, M. J., Segev, R., & Bialek, W. (2006) · Nature · doi:10.1038/nature04701 · Cited by 1,745 · cited 1×
“To the best of our knowledge, our results provide the first complete statistical description of a network connection matrix based on in vivo neuronal recordings of behaving animals (see also refs”
ResultNeural correlates of decision variables in parietal cortex
Platt, M. L., & Glimcher, P. W. (1999) · Nature · doi:10.1038/22268 · Cited by 1,721 · cited 1×
“Parietal cortex has also been implicated in decision making on the basis of the value representation and the accumulation of sensory evidenc”
BackgroundRoitman, J. D., & Shadlen, M. N. (2002) · The Journal of Neuroscience · doi:10.1523/jneurosci.22-21-09475.2002 · Cited by 1,589 · cited 1×
“Parietal cortex has also been implicated in decision making on the basis of the value representation and the accumulation of sensory evidenc”
BackgroundSpectra and Pseudospectra: The Behavior of Nonnormal Matrices and Operators
Trefethen, L. N., & Embree, M. (2005) · doi:10.1515/9780691213101 · Cited by 1,586
Computation at the edge of chaos: Phase transitions and emergent computation
Langton, C. G. (1990) · Physica D: Nonlinear Phenomena · doi:10.1016/0167-2789(90)90064-v · Cited by 1,527
Midbrain dopamine neurons encode a quantitative reward prediction error signal
Bayer, H. M., & Glimcher, P. W. (2005) · Neuron · doi:10.1016/j.neuron.2005.05.020 · Cited by 1,304
An experimental unification of reservoir computing methods
Verstraeten, D., Schrauwen, B., D’Haene, M., & Stroobandt, D. (2007) · Neural Networks · doi:10.1016/j.neunet.2007.04.003 · Cited by 1,135
Generating Coherent Patterns of Activity from Chaotic Neural Networks
Sussillo, D., & Abbott, L. F. (2009) · Neuron · doi:10.1016/j.neuron.2009.07.018 · Cited by 1,092
Optimization and applications of echo state networks with leaky- integrator neurons
Jaeger, H., Lukoševičius, M., Popovici, D., & Siewert, U. (2007) · Neural Networks · doi:10.1016/j.neunet.2007.04.016 · Cited by 948
Optimal decision making and the anterior cingulate cortex
Kennerley, S. W., Walton, M. E., Behrens, T. E. J., Buckley, M. J., & Rushworth, M. F. S. (2006) · Nature Neuroscience · doi:10.1038/nn1724 · Cited by 948 · cited 1×
“The epoch code g(k) was fixed by the firing rates averaged across trials, whereas the parameters of the exponential function (two parameters (A,τ) when using equation (10) and four parameters (A1,τ1, A2,τ2) when using equation (11)) were estimated using a nonlinear curve-fitting procedure, implemented by the Matlab fu…”
MethodThe Neurobiology of Decision: Consensus and Controversy
Kable, J. W., & Glimcher, P. W. (2009) · Neuron · doi:10.1016/j.neuron.2009.09.003 · Cited by 928 · cited 5×
“We therefore tested whether any memory trace of choice exists in the recorded neurons by applying the same analysis of equations (1) and (2) and substituting reward with choice.”
BackgroundMethodInfluentialAn adaptive coding model of neural function in prefrontal cortex
Duncan, J. (2001) · Nature Reviews Neuroscience · doi:10.1038/35097575 · Cited by 874
Choice, uncertainty and value in prefrontal and cingulate cortex
Rushworth, M. F. S., & Behrens, T. E. J. (2008) · Nature Neuroscience · doi:10.1038/nn2066 · Cited by 866 · cited 3×
“Using the Bayesian Information Criterion, we found that the best fit was a single exponential for 269 neurons and double exponentials for 268 neurons, whereas the (2) remaining 144 neurons were fitted best by a model with ex(t) = 0.”
BackgroundMethodReal-Time Computation at the Edge of Chaos in Recurrent Neural Networks
Bertschinger, N., & Natschläger, T. (2004) · Neural Computation · doi:10.1162/089976604323057443 · Cited by 833 · cited 1×
“, can reproduce the observed distribution of timescales, provided that the network operates at the critical point (or edge of chaos”
BackgroundDecision Making in Recurrent Neuronal Circuits
Wang, X.-J. (2008) · Neuron · doi:10.1016/j.neuron.2008.09.034 · Cited by 679 · cited 4×
“The filter f considered in equation (3) is replaced by g(k)·ex(t).”
BackgroundInfluentialSelf-citationPrefrontal cortex and decision making in a mixed-strategy game
Barraclough, D. J., Conroy, M. L., & Lee, D. (2004) · Nature Neuroscience · doi:10.1038/nn1209 · Cited by 678 · cited 1×
“(205 neurons) of six monkeys performing a matching pennies tas”
BackgroundSelf-citationPrefrontal neuronal activity in rhesus monkeys performing a delayed anti-saccade task
Funahashi, S., Chafee, M. V., & Goldman, P. S. (1993) · Nature · doi:10.1038/365753a0 · Cited by 662
DYNAMIC RESPONSE-BY-RESPONSE MODELS OF MATCHING BEHAVIOR IN RHESUS MONKEYS
Lau, B., & Glimcher, P. W. (2005) · Journal of the Experimental Analysis of Behavior · doi:10.1901/jeab.2005.110-04 · Cited by 485
Prospective coding for objects in primate prefrontal cortex
Rainer, G., Rao, S. C., & Miller, E. K. (1999) · The Journal of Neuroscience · doi:10.1523/jneurosci.19-13-05493.1999 · Cited by 432
Timing and Neural Encoding of Somatosensory Parametric Working Memory in Macaque Prefrontal Cortex
Brody, C. D. (2003) · Cerebral Cortex · doi:10.1093/cercor/bhg100 · Cited by 350
Memory without Feedback in a Neural Network
Goldman, M. S. (2009) · Neuron · doi:10.1016/j.neuron.2008.12.012 · Cited by 342
Balanced Amplification: A New Mechanism of Selective Amplification of Neural Activity Patterns
Murphy, B. K., & Miller, K. D. (2009) · Neuron · doi:10.1016/j.neuron.2009.02.005 · Cited by 338
Seo, H., & Lee, D. (2007) · The Journal of Neuroscience · doi:10.1523/jneurosci.2369-07.2007 · Cited by 324 · cited 1×
“Results Multiplicative memory traces in cortical neurons We analyzed single-neuron activity recorded from three cortical areas, ACC”
MethodSelf-citationMemory traces in dynamical systems
Ganguli, S., Huh, D., & Sompolinsky, H. (2008) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.0804451105 · Cited by 295
Optimal Information Storage and the Distribution of Synaptic Weights
Brunel, N., Hakim, V., Isope, P., Nadal, J., & Barbour, B. (2004) · Neuron · doi:10.1016/j.neuron.2004.08.023 · Cited by 292
Representation of Attended Versus Remembered Locations in Prefrontal Cortex
Lebedev, M., Messinger, A., Kralik, J. D., & Wise, S. P. (2004) · PLoS Biology · doi:10.1371/journal.pbio.0020365 · Cited by 275 · cited 1×
“and it has been debated as to whether other processes involved in goal-directed behavior could be inter-mixed with a memory trace, such as spatial attentio”
BackgroundFunctional, But Not Anatomical, Separation of “What” and “When” in Prefrontal Cortex
Machens, C. K., Romo, R., & Brody, C. D. (2010) · The Journal of Neuroscience · doi:10.1523/jneurosci.3276-09.2010 · Cited by 260
Neural representation of time in cortico-basal ganglia circuits
Jin, D. Z., Fujii, N., & Graybiel, A. M. (2009) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.0909881106 · Cited by 246
Variability in neuronal activity in primate cortex during working memory tasks
Shafi, M., Zhou, Y., Quintana, J., Chow, C., Fuster, J., & Bodner, M. (2007) · Neuroscience · doi:10.1016/j.neuroscience.2006.12.072 · Cited by 215
The prefrontal cortex and working memory: physiology and brain imaging
Passingham, D., & Sakai, K. (2004) · Current Opinion in Neurobiology · doi:10.1016/j.conb.2004.03.003 · Cited by 207 · cited 1×
“Memory traces in the neural signals are mixed with other task-dependent factor”
MethodLINEAR-NONLINEAR-POISSON MODELS OF PRIMATE CHOICE DYNAMICS
Corrado, G. S., Sugrue, L. P., Seung, H. S., & Newsome, W. T. (2005) · Journal of the Experimental Analysis of Behavior · doi:10.1901/jeab.2005.23-05 · Cited by 206
Beyond working memory: the role of persistent activity in decision making
Curtis, C. E., & Lee, D. (2010) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2010.03.006 · Cited by 191
Self-citationHeterogeneous reward signals in prefrontal cortex
Wallis, J. D., & Kennerley, S. W. (2010) · Current Opinion in Neurobiology · doi:10.1016/j.conb.2010.02.009 · Cited by 186 · cited 1×
“, recent studies have placed more emphasis on its role in representing both positive and negative value”
BackgroundRigotti, M. (2010) · Frontiers in Computational Neuroscience · doi:10.3389/fncom.2010.00024 · Cited by 184
Reinforcement learning and decision making in monkeys during a competitive game
Lee, D., Conroy, M. L., McGreevy, B. P., & Barraclough, D. J. (2004) · Cognitive Brain Research · doi:10.1016/j.cogbrainres.2004.07.007 · Cited by 156
Self-citationOne-dimensional dynamics of attention and decision making in LIP
Ganguli, S., Bisley, J. W., Roitman, J. D., Shadlen, M. N., Goldberg, M. E., & Miller, K. D. (2008) · Neuron · doi:10.1016/j.neuron.2008.01.038 · Cited by 152
Dynamic Signals Related to Choices and Outcomes in the Dorsolateral Prefrontal Cortex
Seo, H., Barraclough, D. J., & Lee, D. (2007) · Cerebral Cortex · doi:10.1093/cercor/bhm064 · Cited by 143
Self-citationHierarchical coding for sequential task events in the monkey prefrontal cortex
Sigala, N., Kusunoki, M., Nimmo-Smith, I., Gaffan, D., & Duncan, J. (2008) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.0802569105 · Cited by 140
Lateral Intraparietal Cortex and Reinforcement Learning during a Mixed-Strategy Game
Seo, H., Barraclough, D. J., & Lee, D. (2009) · The Journal of Neuroscience · doi:10.1523/jneurosci.1479-09.2009 · Cited by 125
Self-citationLapish, C. C., Durstewitz, D., Chandler, L. J., & Seamans, J. K. (2008) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.0804045105 · Cited by 119
Rainer, G., & Miller, E. K. (2002) · European Journal of Neuroscience · doi:10.1046/j.1460-9568.2002.01958.x · Cited by 113
Multiple Timescales of Memory in Lateral Habenula and Dopamine Neurons
Bromberg-Martin, E. S., Matsumoto, M., Nakahara, H., & Hikosaka, O. (2010) · Neuron · doi:10.1016/j.neuron.2010.06.031 · Cited by 97 · cited 1×
“2752-S1">Supplementary Text ), as observed in a recent stud”
BackgroundNeural mechanism for stochastic behaviour during a competitive game
Soltani, A., Lee, D., & Wang, X.-J. (2006) · Neural Networks · doi:10.1016/j.neunet.2006.05.044 · Cited by 77 · cited 1×
“For example, f(3,4) describes the effect of a reward after 3 trials during epoch 4.”
BackgroundSelf-citationValuation of uncertain and delayed rewards in primate prefrontal cortex
Kim, S., Hwang, J., Seo, H., & Lee, D. (2009) · Neural Networks · doi:10.1016/j.neunet.2009.03.010 · Cited by 37 · cited 1×
“where Rew(t) denotes the reward received by the animal in trial t, and the term inside square is commonly defined as the reward prediction error; that is, the (12)”
BackgroundSelf-citation
Flagged first, then by citations
1 of the 52 references couldn’t be matched to a known paper.
Cited by
Sorted by citations
Recent advances in physical reservoir computing: A review
Tanaka, G., Yamane, T., Héroux, J. B., Nakane, R., Kanazawa, N., Takeda, S., Numata, H., Nakano, D., & Hirose, A. (2019) · Neural Networks · doi:10.1016/j.neunet.2019.03.005 · Cited by 2,146
“…& Jaeger, 2011), the reservoir model for storing different time constants of memory traces for reward expectation in re-inforcement learning (Bernacchia, Seo, Lee, & Wang, 2011), the autonomously emerged working memory model through reward-modulated online learning rule that enables the…”
BackgroundA hierarchy of intrinsic timescales across primate cortex
Murray, J. D., Bernacchia, A., Freedman, D. J., Romo, R., Wallis, J. D., Cai, X., Padoa-Schioppa, C., Pasternak, T., Seo, H., Lee, D., et al. (2014) · Nature Neuroscience · doi:10.1038/nn.3862 · Cited by 990
Self-citationCortical tracking of hierarchical linguistic structures in connected speech
Ding, N., Melloni, L., Zhang, H., Tian, X., & Poeppel, D. (2016) · Nature Neuroscience · doi:10.1038/nn.4186 · Cited by 1,059
‘Activity-silent’ working memory in prefrontal cortex: a dynamic coding framework
Stokes, M. G. (2015) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2015.05.004 · Cited by 850
Slow cortical dynamics and the accumulation of information over long timescales
Honey, C. J., Thesen, T., Donner, T. H., Silbert, L. J., Carlson, C. E., Devinsky, O., Doyle, W. K., Rubin, N., Heeger, D. J., & Hasson, U. (2012) · Neuron · doi:10.1016/j.neuron.2012.08.011 · Cited by 532
“There is a growing realization, however, of the importance of the neural mechanisms by which information can be accumulated over time (Ben-Yakov et al., 2012; Bernacchia et al., 2011; Brody et al., 2003; Maass et al., 2007; Wang, 2002).”
Metadata
- Authors
- Bernacchia, A., Seo, H., Lee, D., & Wang, X.-J.
- Affiliations
- Yale University (US)
- Corresponding author
- Wang, Xiao-Jing
- Venue
- Nature Neuroscience
- Published
- 2011-02-13
- Volume
- 14
- Issue
- 3
- Pages
- 366-372
- ISSN
- 1097-6256 (ISSN-L)
- 1546-1726
- PMC
- PMC3079398
- OpenAlex
- W2139630667
- Semantic Scholar
- 87fa62529ba1b9dd0ff3d6e9fcd5b62a26363247
- Cited by
- 292
- References
- 52
- Citations by year
- 2026: 14 · 2025: 13 · 2024: 19 · 2023: 15 · 2022: 23 · 2021: 25 · 2020: 28 · 2019: 19 · 2018: 21 · 2017: 21 · 2016: 16 · 2015: 20 · 2014: 18 · 2013: 15 · 2012: 16
- FWCI
- 9.41
- Type
- journal article
- Language
- en
- Open access
- green
- www.ncbi.nlm.nih.gov
- Licence
- www.springer.com/tdm (tdm)
- Locations
- Nature Neuroscience (publishedVersion)
- PubMed (publishedVersion)
- PubMed Central (acceptedVersion)
- Funders
- National Institute of Mental Health (R01MH062349, R01MH073246)
- National Institute on Drug Abuse (DA029330, R01DA029330)
- Topics
- Life Sciences › Neuroscience › Cognitive Neuroscience › Neural and Behavioral Psychology Studies
- Life Sciences › Neuroscience › Cognitive Neuroscience › Neural dynamics and brain function
- Life Sciences › Neuroscience › Cognitive Neuroscience › Memory and Neural Mechanisms
- Keywords
- reward memory, power-law distribution, reinforcement learning, decision-making, prefrontal cortex, cingulate cortex, parietal cortex, reward anticipation, decision time, cortical neurons
- MeSH terms
- Animals, Haplorhini, Cerebral Cortex, Decision Making, Humans, Memory, Nerve Net, Neurons, Neuropsychological Tests, Reinforcement, Psychology, Reward, Time Factors
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
A language model drew the claims from the paper on 2026-10-10, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-10-10 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
