The DOI matches its Crossref record; 51 of the 52 references it cites were found in a registry or an index; none of the 59 sources its claims cite is checked for support yet.
A reservoir of time constants for memory traces in cortical neurons
Open accessAbstract
According to reinforcement learning theory of decision making, reward expectation is computed by integrating past rewards with a fixed timescale. In contrast, we found that a wide range of time constants is available across cortical neurons recorded from monkeys performing a competitive game task. By recognizing that reward modulates neural activity multiplicatively, we found that one or two time constants of reward memory can be extracted for each neuron in prefrontal, cingulate and parietal cortex. These timescales ranged from hundreds of milliseconds to tens of seconds, according to a power law distribution, which is consistent across areas and reproduced by a 'reservoir' neural network model. These neuronal memory timescales were weakly, but significantly, correlated with those of monkey's decisions. Our findings suggest a flexible memory system in which neural subpopulations with distinct sets of long or short memory timescales may be selectively deployed according to the task demands.
Overview
AI snapshot
Written from the paper’s full text.
TL;DR
Cortical neurons recorded from monkeys playing a competitive game carry reward-memory traces with a wide range of time constants, distributed as a power law across ACCd, DLPFC, and LIP. A reservoir network model operating near criticality reproduces this distribution, and neural timescales are weakly but significantly correlated with behavioral timescales.
Key findings
- Across 681 neurons from ACCd (154), DLPFC (322), and LIP (205), reward memory was best fit by a single exponential for 269 neurons, double exponentials for 268 neurons, and no memory for 144 neurons; 79% (537/681) showed exponential decay.
- Memory timescales ranged from hundreds of milliseconds to tens of seconds and followed a power-law distribution with exponent −2, consistent across the three cortical areas; about 20% of neurons (133/681) had timescales larger than one trial (29% ACCd, 19% DLPFC, 13% LIP, χ²-test p=0.0005).
- Behavioral timescales from a reinforcement learning fit also followed a power law, and neural and behavioral timescales were weakly but significantly correlated across sessions (R=0.12, p=0.003); simultaneously recorded neuron pairs showed no significant correlation (312 pairs, R=0.07, p=0.2).
- A randomly connected reservoir network model reproduced the power-law distribution of timescales when operating at the critical point, with exponentially distributed amplitudes and normal connection matrix.
Methods
- Single-unit activity was recorded from six rhesus monkeys performing a matching pennies task; firing rates were computed in twelve 250-ms epochs per trial, and reward history up to five trials back was used.
- Memory traces were estimated with multiple linear regression, then fit with single or double exponential functions multiplied by the epoch code; model selection used Bayesian Information Criterion, and reshuffling trials served as a control (96%, 656/681, showed no memory after reshuffling).
- Behavior was fit per session with a standard reinforcement learning model using maximum likelihood, with significance assessed against 100 reshuffled surrogate sessions; the network model used 1,000 neurons with dynamics dv/dt = J·v(t) + h·Rew(t).
Limitations
- The paper states that the matching pennies task does not necessarily require reward memory, that the neural–behavioral correlation is weak and might be caused by unknown factors, that the neuronal and behavioral ranges differ, and that behavioral data were not fit with a multi-timescale RL model.
- The paper also states that the network model is linear and would need nonlinearities if connection-width exceeds the critical level, and that the readout driving behavior is not explicitly modeled.
- Evident from the text: only about two neurons were simultaneously recorded on average, limiting pairwise analyses, and the study reports correlational rather than causal links between neural timescales and behavior.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
0 of 59 cited sources checked · 0 supported · 0 flagged
None of the sources cited for this claim have been checked yet.
In the paper
“Finally, we show that a randomly connected circuit model, akin to a “reservoir” network13-15, can reproduce the observed distribution of timescales, provided that the network operates at the critical point (or “edge of chaos”)16-18.”
§ A reservoir of time constants for memory traces in cortical neurons > Abstract
What the sources say
- Not checked yetcites [13] Maass, W., Natschläger, T. & Markram, H. Real-time computing without stable states: a new framework for neural computation based on perturbations. Neural Comput. 14, 2531–2560 (2002). source
- Not checked yetcites [14] Jaeger, H., Lukosevicius, M., Popovici, D. & Siewert, U. Optimization and applications of echo state networks with leaky-integrator neurons. Neural Netw. 20, 335–352 (2007). source
- Not checked yetcites [15] Verstraeten, D., Schrauwen, B., D′Haene, M. & Stroobandt, D. An experimental unification of reservoir computing methods. Neural Netw. 20, 391–403 (2007). source
- Not checked yetcites [16] Sussillo, D. & Abbott, L.F. Generating coherent patterns of activity from chaotic neural networks. Neuron 63, 544–557 (2009). source
- Not checked yetcites [17] Bertschinger, N. & Natschlager, T. Real-time computation at the edge of chaos in recurrent neural networks. Neural Comput. 16, 1413–1436 (2004). source
- Not checked yetcites [18] Langton, C.G. Computation at the edge of chaos: phase transitions and emergent computations. Physica D 42, 12–37 (1990). source
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passes- The authors state this as an assumption: The specific form of the input signals is not important; the results depend only on the synaptic matrix J.
- The authors state this as an assumption: The connection weights (entries of matrix J) are randomly distributed.
- The authors state this as an assumption: The linear model is sufficient for reproducing the observed power-law distribution of timescales under specific conditions.
None of the sources cited for this claim have been checked yet.
In the paper
“Consistent with previous studies23-25, we found that the activity of neurons varied substantially in different trial epochs (99% of neurons, 675/681, ANOVA p<0.05).”
§ A reservoir of time constants for memory traces in cortical neurons > Results > Multiplicative memory traces in cortical neurons
What the sources say
- Not checked yetcites [23] Lapish, C.C., Durstewitz, D., Chandler, L.J. & Seamans, J.K. Successful choice behavior is associated with distinct and coherent network states in anterior cingulate cortex. Proc. Natl. Acad. Sci. USA 105, 11963–11968 (2008). source
- Not checked yetcites [24] Sigala, N., Kusonoki, M., Nimmo-Smith, I., Gaffan, D. & Duncan, J. Hierarchical coding for sequential task events in the monkey prefrontal cortex. Proc. Natl. Acad. Sci. USA 105, 11969–11974 (2008). source
- Not checked yetcites [25] Jin, D.Z., Fujii, N. & Graybiel, A.N. Neural representation of time in cortico-basal ganglia circuits. Proc. Natl. Acad. Sci. USA 106, 19156–19161 (2009). source
Strength:plain· not yet calibratedCausal:correlational· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes? Load-bearing: unclear✓ Based: passesA result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“reproduced by a “reservoir” neural network model.”
§ A reservoir of time constants for memory traces in cortical neurons > Abstract
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passes- The authors state this as an assumption: The specific form of the input signals is not important; the results depend only on the synaptic matrix J.
- The authors state this as an assumption: The connection weights (entries of matrix J) are randomly distributed.
- The authors state this as an assumption: The linear model is sufficient for reproducing the observed power-law distribution of timescales under specific conditions.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“In some cases, the sum of two exponential functions, ex(t) = A1·e−t/τ1+A2·e−t/τ2, fitted the data better than a single exponential”
§ A reservoir of time constants for memory traces in cortical neurons > Results > Multiplicative memory traces in cortical neurons
Strength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes? Load-bearing: unclear✓ Based: passes- The authors state this as an assumption: The memory trace f is factorized into the epoch code g(k) and an exponential function ex(t).
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“we show that memory time constants can be extracted from the activity of single neurons.”
§ A reservoir of time constants for memory traces in cortical neurons > Abstract
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passes- The authors state this as an assumption: The memory trace f is factorized into the epoch code g(k) and an exponential function ex(t).
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“a fraction of them did not show a modulation of the memory by the epoch code.”
§ A reservoir of time constants for memory traces in cortical neurons > Results > Multiplicative memory traces in cortical neurons
Strength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes? Load-bearing: unclear✓ Based: passes- The authors state this as an assumption: The memory trace f is factorized into the epoch code g(k) and an exponential function ex(t).
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“our analysis of animal's behavior suggests that the timescale over which reward events affect decisions changes across experimental sessions”
§ A reservoir of time constants for memory traces in cortical neurons > Abstract
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesA result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The time constants for reward memory at the behavioural and neuronal levels were weakly but significantly correlated across experimental sessions.”
§ A reservoir of time constants for memory traces in cortical neurons > Abstract
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passes- The authors state this as an assumption: The memory trace f is factorized into the epoch code g(k) and an exponential function ex(t).
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“These neuronal memory timescales were weakly but significantly correlated with those of monkey's decisions.”
§ A reservoir of time constants for memory traces in cortical neurons > Abstract
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passes- The authors state this as an assumption: The memory trace f is factorized into the epoch code g(k) and an exponential function ex(t).
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“By recognizing that reward modulates neural activity multiplicatively, we found that one or two time constants of reward memory can be extracted for each neuron in prefrontal, cingulate, and parietal cortex.”
§ A reservoir of time constants for memory traces in cortical neurons > Abstract
Load-bearingStrength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passes- The authors state this as an assumption: The memory trace f is factorized into the epoch code g(k) and an exponential function ex(t).
- It may state two separate claims as one. 26
- Probably not a substantial claim. 31
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The memory trace does not decay monotonically, but its strength is modulated throughout the trial consistent with the epoch code.”
§ A reservoir of time constants for memory traces in cortical neurons > Results > Multiplicative memory traces in cortical neurons
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesIt may state two separate claims as one.
- The authors state this as an assumption: The memory trace f is factorized into the epoch code g(k) and an exponential function ex(t).
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“and there is a wide range of timescales across neurons, obeying a power law distribution.”
§ A reservoir of time constants for memory traces in cortical neurons > Abstract
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesIt may state two separate claims as one.
- The authors state this as an assumption: The memory trace f is factorized into the epoch code g(k) and an exponential function ex(t).
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Using the Bayesian Information Criterion (BIC), we found that the best fit was a single exponential for 269 neurons and double exponentials for 268 neurons, while the remaining 144 neurons were fitted best by a model with ex(t)=0.”
§ A reservoir of time constants for memory traces in cortical neurons > Results > Multiplicative memory traces in cortical neurons
Load-bearingStrength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesIt may state two separate claims as one.
- The authors state this as an assumption: The memory trace f is factorized into the epoch code g(k) and an exponential function ex(t).
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“These timescales ranged from hundreds of milliseconds to tens of seconds, according to a power-law distribution”
§ A reservoir of time constants for memory traces in cortical neurons > Abstract
Load-bearingStrength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesIt may state two separate claims as one.
- The authors state this as an assumption: The memory trace f is factorized into the epoch code g(k) and an exponential function ex(t).
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Different neurons have different magnitudes and time courses of the firing rate, and they all show a decaying memory trace modulated by the epoch code.”
§ A reservoir of time constants for memory traces in cortical neurons > Results > Multiplicative memory traces in cortical neurons
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesIt may state two separate claims as one.
- The authors state this as an assumption: The memory trace f is factorized into the epoch code g(k) and an exponential function ex(t).
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“If J is normal, we showed that the amplitudes of the memory traces follow an exponential distribution (Fig.8b), consistent with the experimental observations (Fig.8a).”
§ A reservoir of time constants for memory traces in cortical neurons > Results > Neural network model for memory traces
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesIt may state two separate claims as one.
- The authors state this as an assumption: The specific form of the input signals is not important; the results depend only on the synaptic matrix J.
- The authors state this as an assumption: The connection weights (entries of matrix J) are randomly distributed.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“For the 196 sessions fitted by the RL model, the distribution of behavioral timescales followed a power law distribution (Fig.5a), and the exponent was consistent with that measured in the neural distribution.”
§ A reservoir of time constants for memory traces in cortical neurons > Results > Comparison with behavior
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesIt may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Results are shown in Fig.6, suggesting that both the behavioral (panel (a)) and neural memory timescales (panel (b)) were fairly stable within a single session.”
§ A reservoir of time constants for memory traces in cortical neurons > Results > Comparison with behavior
Strength:hedged· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes? Load-bearing: unclear✓ Based: passesIt may state two separate claims as one.
- The authors state this as an assumption: The memory trace f is factorized into the epoch code g(k) and an exponential function ex(t).
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Our results indicate that multiplicative modulation and a power law distribution of memory timescales also hold for memory trace of past choices”
§ A reservoir of time constants for memory traces in cortical neurons > Results > Comparison with behavior
Strength:hedged· not yet calibratedCausal:correlational· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes? Load-bearing: unclear✓ Based: passesIt may state two separate claims as one.
- The authors state this as an assumption: The memory trace f is factorized into the epoch code g(k) and an exponential function ex(t).
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“First, we found that the connection weights must be broadly distributed among neuron pairs, and this endows the network with a wide variety of timescales.”
§ A reservoir of time constants for memory traces in cortical neurons > Results > Neural network model for memory traces
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesIt may state two separate claims as one.
- The authors state this as an assumption: The specific form of the input signals is not important; the results depend only on the synaptic matrix J.
- The authors state this as an assumption: The connection weights (entries of matrix J) are randomly distributed.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“About 20% of all recorded neurons (133/681) had timescale larger than one trial (29% in ACCd, 19% in DLPFC and 13% in LIP, χ2-test p=0.0005”
§ A reservoir of time constants for memory traces in cortical neurons > Results > Multiplicative memory traces in cortical neurons
Load-bearingStrength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesIt may state two separate claims as one.
- The authors state this as an assumption: The memory trace f is factorized into the epoch code g(k) and an exponential function ex(t).
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The latter is interpreted as no memory, and the corresponding neurons were excluded from further analysis.”
§ A reservoir of time constants for memory traces in cortical neurons > Results > Multiplicative memory traces in cortical neurons
Load-bearingStrength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesProbably not a substantial claim.
- The authors state this as an assumption: The memory trace f is factorized into the epoch code g(k) and an exponential function ex(t).
None of the sources cited for this claim have been checked yet.
In the paper
“and interpreted in the framework of reinforcement learning (RL) theory4-5.”
§ A reservoir of time constants for memory traces in cortical neurons > Abstract
What the sources say
- Not checked yetcites [4] Soltani, A., Lee, D. & Wang, X.-J. Neural mechanism for stochastic behavior during a competitive game. Neural Netw. 19, 1075–1090 (2006). source
- Not checked yetcites [5] Sutton, R.S. & Barto,, A.G. Reinforcement Learning, an Introduction (MIT Press, Cambridge, Massachusetts, 1998). source
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesProbably not a substantial claim.
None of the sources cited for this claim have been checked yet.
In the paper
“So far a few studies have estimated the time constant of this filter from behavior, and assessed how past rewards affect choice selection8-12”
§ A reservoir of time constants for memory traces in cortical neurons > Abstract
What the sources say
- Not checked yetcites [8] Lau, B. & Glimcher, P.W. Dynamic response-by-response models of matching behavior in rhesus monkeys. J. Exp. Anal. Behav. 84, 555–579 (2005). source
- Not checked yetcites [9] Corrado, G.S., Sugrue, L.P., Seung, H.S. & Newsome, W.T. Linear-nonlinear-Poisson models of primate choice dynamics. J. Exp. Anal. Behav. 84, 581–617 (2005). source
- Not checked yetcites [10] Kennerley, S.W., Walton, M.E., Behrens, T.E., Buckley, M.J. & Rushworth, M.F. Optimal decision making and the anterior cingulate cortex. Nat. Neurosci. 9, 940–947 (2006). source
- Not checked yetcites [11] Lee, D., Conroy, M.L., McGreevy, B.P. & Barraclough, D.J. Reinforcement learning and decision making in monkeys during a competitive game. Brain Res. Cogn. Brain Res. 22, 45–58 (2004). source
- Not checked yetcites [12] Kim, S., Hwang, J., Seo, H. & Lee, D. Valuation of uncertain and delayed rewards in primate prefrontal cortex. Neural Netw. 22, 294–304 (2009). source
Strength:plain· not yet calibratedCausal:correlational· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.
None of the sources cited for this claim have been checked yet.
In the paper
“Neural correlates of reward valuation have been observed in different studies1-3”
§ A reservoir of time constants for memory traces in cortical neurons > Abstract
What the sources say
- Not checked yetcites [1] Kable, J.W. & Glimcher, P.W. The neurobiology of decision: consensus and controversy. Neuron 63, 733–745 (2009). source
- Not checked yetcites [2] Rushworth, M.F. & Behrens, T.E. Choice, uncertainty and value in prefrontal and cingulate cortex. Nat. Neurosci. 11, 389–397 (2008). source
- Not checked yetcites [3] Wang, X.-J. Decision making in recurrent neural circuits. Neuron 60, 215–234 (2008). source
Strength:plain· not yet calibratedCausal:correlational· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Hence, each area is endowed with a reservoir of time constants for reward memory, which are distributed heterogeneously across neurons.”
§ A reservoir of time constants for memory traces in cortical neurons > Abstract
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesIt may state two separate claims as one.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“We tested the validity of the fitting procedure by randomly reshuffling the order of trials in each session, and we consistently found that 96% of neurons (656/681) show no memory after reshuffling.”
§ A reservoir of time constants for memory traces in cortical neurons > Results > Multiplicative memory traces in cortical neurons
Load-bearingStrength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesIt may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Similarly, the double exponential decay of memory may correspond to a reward prediction error signal: if the short timescale (τ1) is small enough (about one trial or smaller) then the corresponding exponential filter will respond primarily to the reward in the present trial, while the long timescale (τ2) may provide a value signal by weighting the rewards in the past few trials.”
§ A reservoir of time constants for memory traces in cortical neurons > Discussion
Strength:hedged· not yet calibratedCausal:conditional· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Similarly, the double exponential decay of memory may correspond to a reward prediction error signal: if the short timescale (τ1) is small enough (about one trial or smaller) then the corresponding exponential filter will respond primarily to the reward in the present trial, while the long timescale (τ2) may provide a value signal by weighting the rewards in the past few trials.”
§ A reservoir of time constants for memory traces in cortical neurons > Discussion
Strength:hedged· not yet calibratedCausal:conditional· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The animal's behavior may be determined by a readout system that is able to sample, at different times, from a variety of timescales present in the reservoir.”
§ A reservoir of time constants for memory traces in cortical neurons > Results > Comparison with behavior
Strength:hedged· not yet calibratedCausal:conditional· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✗ Based: failsProbably not a substantial claim. The paper’s conclusions may not depend on it. The paper may give no reason why it holds.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“In fact, for a power law distribution ~ τ−2, the variance increases with the sample size and, in principle, arbitrarily large timescales would be observed with a proportionally large increment in the number of recorded neurons.”
§ A reservoir of time constants for memory traces in cortical neurons > Results > Multiplicative memory traces in cortical neurons
Strength:hedged· not yet calibratedCausal:direct· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes? Load-bearing: unclear✓ Based: passesIt may state two separate claims as one.
None of the sources cited for this claim have been checked yet.
In the paper
“The time course of memory-related activity varies across different neurons and different task protocols, including persistent, ramping, and multi-phasic activity41-43.”
§ A reservoir of time constants for memory traces in cortical neurons > Discussion
What the sources say
- Not checked yetcites [41] Rainer, G. & Miller, E.K. Time course of object-related neural activity in the primate prefrontal cortex during a short-term memory task. Eur. J. Neurosci. 15, 1244–1254 (2002). source
- Not checked yetcites [42] Machens, C.K., Romo, R. & Brody, C.D. Functional, but not anatomical, separation of “what” and “when” in prefrontal cortex. J. Neurosci. 30, 350–360 (2010). source
- Not checked yetcites [43] Shafi, M. et al. Variability in neuronal activity in primate cortex during working memory tasks. Neuroscience 146, 1082–1108 (2007). source
Strength:plain· not yet calibratedCausal:correlational· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The same diversity is observed across three cortical areas, suggesting that the computation of reward memory is a distributed process.”
§ A reservoir of time constants for memory traces in cortical neurons > Discussion
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesIt may state two separate claims as one.
None of the sources cited for this claim have been checked yet.
In the paper
“While anterior cingulate cortex was implicated in monitoring conflict between incompatible response processes35 or detecting performance errors36”
§ A reservoir of time constants for memory traces in cortical neurons > Discussion
What the sources say
- Not checked yetcites [35] Botvinick, M.M., Braver, T.S., Barch, D.M., Carter, C.S. & Cohen, J.D. Conflict monitoring and cognitive control. Psychol. Rev. 108, 624–652 (2001). source
- Not checked yetcites [36] Holroyd, C.B. & Coles, M.G.H. The neural basis of human error processing: reinforcement learning, dopamine and error-related negativity. Psychol. Rev. 109, 679–709 (2002). source
Strength:plain· not yet calibratedCausal:correlational· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“The heterogeneous encoding of the input allows the flexible learning of different output functions.”
§ A reservoir of time constants for memory traces in cortical neurons > Discussion
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesProbably not a substantial claim.
None of the sources cited for this claim have been checked yet.
In the paper
“Although the overall performance of monkeys was nearly optimal, their trial-by-trial decisions, locally in time, were influenced by previous rewards and actions11,19-22.”
§ A reservoir of time constants for memory traces in cortical neurons > Results > Comparison with behavior
What the sources say
- Not checked yetcites [11] Lee, D., Conroy, M.L., McGreevy, B.P. & Barraclough, D.J. Reinforcement learning and decision making in monkeys during a competitive game. Brain Res. Cogn. Brain Res. 22, 45–58 (2004). source
- Not checked yetcites [19] Seo, H. & Lee, D. Temporal filtering of reward signals in the dorsal anterior cingulate cortex during a mixed-strategy game. J. Neurosci. 27, 8366–8377 (2007). source
- Not checked yetcites [20] Seo, H., Barraclough, D.J. & Lee, D. Dynamic signals related to choices and outcomes in the dorsolateral prefrontal cortex. Cereb. Cortex 17, i110–i117 (2007). source
- Not checked yetcites [21] Seo, H., Barraclough, D.J. & Lee, D. Lateral intraparietal cortex and reinforcement learning during a mixed-strategy game. J. Neurosci. 29, 7278–7289 (2009). source
- Not checked yetcites [22] Barraclough, D.J., Conroy, M.L. & Lee, D. Prefrontal cortex and decision making in a mixed-strategy game. Nat. Neurosci. 7, 404–410 (2004). source
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesIt may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“This suggests the possibility that the behavior might be driven by a mechanism that appropriately samples from a range of timescales in a neural network, which has yet to be elucidated.”
§ A reservoir of time constants for memory traces in cortical neurons > Discussion
Strength:hedged· not yet calibratedCausal:conditional· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it.
None of the sources cited for this claim have been checked yet.
In the paper
“Those models predict that the memory of input signals is stored in a large, recurrent and heterogeneous network (reservoir), in a distributed manner, and that a desired output is obtained by a trainable combination of the response signals in the reservoir.”
§ A reservoir of time constants for memory traces in cortical neurons > Discussion
What the sources say
- Not checked yetcites [13] Maass, W., Natschläger, T. & Markram, H. Real-time computing without stable states: a new framework for neural computation based on perturbations. Neural Comput. 14, 2531–2560 (2002). source
- Not checked yetcites [14] Jaeger, H., Lukosevicius, M., Popovici, D. & Siewert, U. Optimization and applications of echo state networks with leaky-integrator neurons. Neural Netw. 20, 335–352 (2007). source
- Not checked yetcites [15] Verstraeten, D., Schrauwen, B., D′Haene, M. & Stroobandt, D. An experimental unification of reservoir computing methods. Neural Netw. 20, 391–403 (2007). source
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesProbably not a substantial claim. It may state two separate claims as one.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“Since the timescales from one (τ) and two-exponential functions (τ1, τ2) were distributed similarly (see Fig.S1a,b in the Supplementary Material), we pooled all timescales”
§ A reservoir of time constants for memory traces in cortical neurons > Results > Multiplicative memory traces in cortical neurons
Load-bearingStrength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesIt may state two separate claims as one.
- The authors state this as an assumption: The memory trace f is factorized into the epoch code g(k) and an exponential function ex(t).
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Similarly, the double exponential decay of memory may correspond to a reward prediction error signal: if the short timescale (τ1) is small enough (about one trial or smaller) then the corresponding exponential filter will respond primarily to the reward in the present trial, while the long timescale (τ2) may provide a value signal by weighting the rewards in the past few trials.”
§ A reservoir of time constants for memory traces in cortical neurons > Discussion
Strength:hedged· not yet calibratedCausal:conditional· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it.
In the paper
“It was already noted previously that a biphasic filtering in dopamine neurons might provide a reward prediction error40.”
§ A reservoir of time constants for memory traces in cortical neurons > Discussion
What the sources say
- Not checked yetcites [40] Bayer, H.M. & Glimcher, P.W. Midbrain dopamine neurons encode a quantitative reward prediction error signal. Neuron 47, 129–141 (2005). source
Strength:hedged· not yet calibratedCausal:conditional· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it.
None of the sources cited for this claim have been checked yet.
In the paper
“Prefrontal cortex plays a major role in dynamic decision processes encoding and updating values1-4.”
§ A reservoir of time constants for memory traces in cortical neurons > Discussion
What the sources say
- Not checked yetcites [1] Kable, J.W. & Glimcher, P.W. The neurobiology of decision: consensus and controversy. Neuron 63, 733–745 (2009). source
- Not checked yetcites [2] Rushworth, M.F. & Behrens, T.E. Choice, uncertainty and value in prefrontal and cingulate cortex. Nat. Neurosci. 11, 389–397 (2008). source
- Not checked yetcites [3] Wang, X.-J. Decision making in recurrent neural circuits. Neuron 60, 215–234 (2008). source
- Not checked yetcites [4] Soltani, A., Lee, D. & Wang, X.-J. Neural mechanism for stochastic behavior during a competitive game. Neural Netw. 19, 1075–1090 (2006). source
Strength:plain· not yet calibratedCausal:correlational· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it.
None of the sources cited for this claim have been checked yet.
In the paper
“recent studies have placed more emphasis on its role in representing both positive and negative values19,37.”
§ A reservoir of time constants for memory traces in cortical neurons > Discussion
What the sources say
- Not checked yetcites [19] Seo, H. & Lee, D. Temporal filtering of reward signals in the dorsal anterior cingulate cortex during a mixed-strategy game. J. Neurosci. 27, 8366–8377 (2007). source
- Not checked yetcites [37] Wallis, J.D. & Kennerley, S.W. Heterogeneous reward signals in prefrontal cortex. Curr. Opin. Neurobiol. 20, 191–198 (2010). source
Strength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it.
None of the sources cited for this claim have been checked yet.
In the paper
“Reservoir-type networks have been the subject of active research in computational neuroscience and machine learning13-15, but so far experimental support that such networks are adopted by the brain has been lacking.”
§ A reservoir of time constants for memory traces in cortical neurons > Discussion
What the sources say
- Not checked yetcites [13] Maass, W., Natschläger, T. & Markram, H. Real-time computing without stable states: a new framework for neural computation based on perturbations. Neural Comput. 14, 2531–2560 (2002). source
- Not checked yetcites [14] Jaeger, H., Lukosevicius, M., Popovici, D. & Siewert, U. Optimization and applications of echo state networks with leaky-integrator neurons. Neural Netw. 20, 335–352 (2007). source
- Not checked yetcites [15] Verstraeten, D., Schrauwen, B., D′Haene, M. & Stroobandt, D. An experimental unification of reservoir computing methods. Neural Netw. 20, 391–403 (2007). source
Load-bearingStrength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesProbably not a substantial claim. It may state two separate claims as one.
None of the sources cited for this claim have been checked yet.
In the paper
“Parietal cortex has also been implicated in decision making based on the value representation as well as on the accumulation of sensory evidence38,39.”
§ A reservoir of time constants for memory traces in cortical neurons > Discussion
What the sources say
- Not checked yetcites [38] Platt, M.L. & Glimcher, P.W. Neural correlates of decision variables in parietal cortex. Nature 400, 233–238 (1999). source
- Not checked yetcites [39] Roitman, J.D. & Shadlen, M.N. Response of neurons in the lateral intraparietal area during a combined visual discrimination reaction time task. J. Neurosci. 22, 9475–9489 (2002). source
Strength:plain· not yet calibratedCausal:correlational· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“This empirical finding is reproduced by a simple computational model, which suggests that reward filtering in the cortex involves a dynamic reservoir network operating at the critical point, leading to a power-law distribution of time constants.”
§ A reservoir of time constants for memory traces in cortical neurons > Discussion
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesIt may state two separate claims as one.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“The placement of the recording chamber was guided by magnetic resonance (MR) images, and confirmed by metal pins inserted in known anatomical locations at the end of the experiment in some animals.”
§ A reservoir of time constants for memory traces in cortical neurons > Methods > Animal preparation and electrophysiological recording
Strength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.
In the paper
“This is noteworthy, since a shift from an exploitive to an exploratory strategy as the environment becomes uncertain is often assessed by an increase in the learning rate10.”
§ A reservoir of time constants for memory traces in cortical neurons > Discussion
What the sources say
- Not checked yetcites [10] Kennerley, S.W., Walton, M.E., Behrens, T.E., Buckley, M.J. & Rushworth, M.F. Optimal decision making and the anterior cingulate cortex. Nat. Neurosci. 9, 940–947 (2006). source
Strength:plain· not yet calibratedCausal:correlational· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“This implies that the matrix product (RewT·Rew) is well conditioned, and the solution fsol minimizing the variance of the noise (or squared error) is robust and given by (M.6)fsol=(RewT⋅Rew)−1⋅RewT⋅FR”
§ A reservoir of time constants for memory traces in cortical neurons > Methods > Multiple regression analysis of memory traces
Load-bearingStrength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesIt may state two separate claims as one.
- Rests on an assumption: The sequence of rewards is nearly random.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“As a control of the fitting procedure, we reshuffled the label n in the firing rates FR(n,k), assigning to each firing rate the value of a random trial, and we repeated the entire procedure.”
§ A reservoir of time constants for memory traces in cortical neurons > Methods > Exponential memory traces and model selection
Strength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“Fitting was repeated ten times for each neuron and each model, in the search for a global minimum of the error.”
§ A reservoir of time constants for memory traces in cortical neurons > Methods > Exponential memory traces and model selection
Strength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“Any parameters resulting in unrealistic values were discarded, such as negative values of τ, τ1 or τ2, values of τ larger than 20 trials, and the absolute value of A or (A1+A2) larger than 4.”
§ A reservoir of time constants for memory traces in cortical neurons > Methods > Exponential memory traces and model selection
Load-bearingStrength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesProbably not a substantial claim.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“A diversity of time constants also means a broad range of learning rates, since the two are inversely related to each other.”
§ A reservoir of time constants for memory traces in cortical neurons > Discussion
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✗ Based: failsProbably not a substantial claim. The paper may give no reason why it holds.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“The matrix product (RewT·Rew) is approximately proportional to the identity matrix.”
§ A reservoir of time constants for memory traces in cortical neurons > Methods > Multiple regression analysis of memory traces
Strength:hedged· not yet calibratedCausal:correlational· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✗ Based: failsProbably not a substantial claim. The paper’s conclusions may not depend on it. The paper may give no reason why it holds.
- Rests on an assumption: The sequence of rewards is nearly random.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“When RewT·Rew=I, the filter is equal to the firing rate averaged over all trials, where the average is conditioned on the past rewards.”
§ A reservoir of time constants for memory traces in cortical neurons > Methods > Multiple regression analysis of memory traces
Strength:plain· not yet calibratedCausal:correlational· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“The significance of the estimation was assessed, for each session, by constructing 100 surrogate sessions, each one obtained by reshuffling of the order of trials.”
§ A reservoir of time constants for memory traces in cortical neurons > Methods > Reinforcement learning fit of behavior
Strength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“Among the three models, the selection of the appropriate one for each neuron was determined according to the Bayesian information criterion (BIC):”
§ A reservoir of time constants for memory traces in cortical neurons > Methods > Exponential memory traces and model selection
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesProbably not a substantial claim.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“Power-law distributions are unusual, as they imply a high probability for both large and small time constants.”
§ A reservoir of time constants for memory traces in cortical neurons > Discussion
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesProbably not a substantial claim.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“The physical time t depends on all indices k, n and n', because the time elapsed between different epochs and between successive trials is variable, due to the variability in the time taken by the animal to start a trial and to make a saccade to one of the two targets.”
§ A reservoir of time constants for memory traces in cortical neurons > Methods > Exponential memory traces and model selection
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesProbably not a substantial claim.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“All the neurons were recorded without pre-screening.”
§ A reservoir of time constants for memory traces in cortical neurons > Methods > Animal preparation and electrophysiological recording
Load-bearingStrength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesProbably not a substantial claim.
The paper relies on this without support.
The neural mechanism underlying switching between long and short time constants for computing reward expectation remains poorly understood.
“The neural mechanism underlying switching between long and short time constants for computing reward expectation remains poorly understood.”
The paper relies on this without support.
The neural mechanisms responsible for such timescales are still unknown.
“but the neural mechanisms responsible for such timescales are still unknown.”
The paper relies on this without support.
The matching pennies task used in this study does not necessarily require the memory of past rewards, and the optimal strategy for the monkey is to choose randomly and unpredictably.
“The matching pennies task used in this study does not necessarily require the memory of past rewards, and the optimal strategy for the monkey is to choose randomly and unpredictably.”
The authors state this as an assumption.
The effects of rewards in successive trials are additive.
“assuming that the effects of rewards in successive trials are additive.”
The authors state this as an assumption.
The memory trace f is factorized into the epoch code g(k) and an exponential function ex(t).
“we assumed that the memory trace f is factorized into the epoch code g(k) and an exponential function ex(t)”
The authors state this as an assumption.
The specific form of the input signals is not important; the results depend only on the synaptic matrix J.
“For our purposes, the specific form of the input signals is not important; the results will depend only on the synaptic matrix J.”
The authors state this as an assumption.
The connection weights (entries of matrix J) are randomly distributed.
“We assumed the connection weights (the entries of the matrix J) to be randomly distributed”
The authors state this as an assumption.
The linear model is sufficient for reproducing the observed power-law distribution of timescales under specific conditions.
“here we limit ourselves to the linear model, which is sufficient for the purpose of reproducing the observed power-law distribution of timescales under specific conditions.”
The authors state this as an assumption.
Memory traces are exponential functions ex(t) rescaled by the epoch code g(k).
“we assumed that memory traces are exponential function ex(t) rescaled by the epoch code g(k)”
The authors state this as an assumption.
The saccade reaction time of the animal in all trials is equal to 120 ms on average.
“we assumed that the saccade reaction time of the animal in all trials is equal to 120ms (average)”
The authors state this as an assumption.
The time elapsed between the initiation of two successive trials is 3.4 s (median).
“and that the time elapsed between the initiation of two successive trials is 3.4s (median).”
The paper relies on this without support.
The sequence of rewards is nearly random.
“Because the sequence of rewards is nearly random and N is large, different columns of the matrix Rew are nearly orthogonal.”
The paper relies on this without support.
In the matching pennies task, the global optimal strategy is to choose randomly and therefore does not require memory.
“Although the global optimal strategy for the matching pennies task is to choose randomly and therefore does not require memory, the animals made their decisions largely on the basis of their reward history11,19-22.”
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Similar results are obtained in tasks that manipulate the desirability of a target using different methods, for example by controlling the relative magnitude, probability or delay of its expected reward (Bernacchia et al., 2011; Louie et al., 2011; Sugrue et al., 2004; Yang and Shadlen, 2007).”
From Attention, learning, and the value of information (2012) · cites this paper as Bernacchia et al., 2011
Citations
Cites
Reinforcement Learning: An Introduction
Sutton, R. S., & Barto, A. G. (1998) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.1998.712192 · Cited by 27,225 · cited 1×
“(5) The seven unknown variables have been rewritten by a single vector f”
BackgroundReinforcement Learning: An Introduction
Sutton, R. S., & Barto, A. (2005) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.2004.842673 · Cited by 25,268
Conflict monitoring and cognitive control
Botvinick, M. M., Braver, T. S., Barch, D. M., Carter, C. S., & Cohen, J. D. (2001) · Psychological Review · doi:10.1037/0033-295x.108.3.624 · Cited by 7,351 · cited 1×
“Although anterior cingulate cortex has been implicated in monitoring conflict between incompatible response processe”
MethodHolroyd, C. B., & Coles, M. G. H. (2002) · Psychological Review · doi:10.1037/0033-295x.109.4.679 · Cited by 4,027
Maass, W., Natschläger, T., & Markram, H. (2002) · Neural Computation · doi:10.1162/089976602760407955 · Cited by 4,025 · cited 2×
“If a single value function (for a given stimulus/action) and a single reward (delivered at time zero) are considered, the solution of equation (13) can be approximated by an exponential response,”
BackgroundCortical substrates for exploratory decisions in humans
Daw, N. D., O'Doherty, J. P., Dayan, P., Seymour, B., & Dolan, R. J. (2006) · Nature · doi:10.1038/nature04766 · Cited by 2,224 · cited 1×
“If the payoffs for the same option change often and unpredictably, then rewards should be filtered on short timescales to track the fast changes in a volatile environment; in contrast, if past rewards reliably predict future ones, then they should be filtered on long timescales to exploit a stable environmen”
BackgroundLearning the value of information in an uncertain world
Behrens, T. E. J., Woolrich, M. W., Walton, M. E., & Rushworth, M. F. S. (2007) · Nature Neuroscience · doi:10.1038/nn1954 · Cited by 2,053 · cited 1×
“Any parameters resulting in unrealistic values were discarded, such as negative values of τ, τ1 or τ2, values of (6)”
BackgroundWeak pairwise correlations imply strongly correlated network states in a neural population
Schneidman, E., Berry, M. J., Segev, R., & Bialek, W. (2006) · Nature · doi:10.1038/nature04701 · Cited by 1,745 · cited 1×
“To the best of our knowledge, our results provide the first complete statistical description of a network connection matrix based on in vivo neuronal recordings of behaving animals (see also refs”
ResultNeural correlates of decision variables in parietal cortex
Platt, M. L., & Glimcher, P. W. (1999) · Nature · doi:10.1038/22268 · Cited by 1,721 · cited 1×
“Parietal cortex has also been implicated in decision making on the basis of the value representation and the accumulation of sensory evidenc”
BackgroundRoitman, J. D., & Shadlen, M. N. (2002) · The Journal of Neuroscience · doi:10.1523/jneurosci.22-21-09475.2002 · Cited by 1,589 · cited 1×
“Parietal cortex has also been implicated in decision making on the basis of the value representation and the accumulation of sensory evidenc”
Background
Flagged first, then by citations
1 of the 52 references couldn’t be matched to a known paper.
Cited by
Sorted by citations
Items 101–200 of 294
Autonomous Encoding of Irrelevant Goals and Outcomes by Prefrontal Cortex Neurons
Genovesio, A., Tsujimoto, S., Navarra, G., Falcone, R., & Wise, S. P. (2014) · Journal of Neuroscience · doi:10.1523/jneurosci.3228-13.2014 · Cited by 47
Spatial Patterns of Persistent Neural Activity Vary with the Behavioral Context of Short-Term Memory
Daie, K., Goldman, M. S., & Aksay, E. R. F. (2015) · Neuron · doi:10.1016/j.neuron.2015.01.006 · Cited by 42
A reservoir of foraging decision variables in the mouse brain
Cazettes, F., Mazzucato, L., Murakami, M., Morais, J. P., Augusto, E., Renart, A., & Mainen, Z. F. (2023) · Nature Neuroscience · doi:10.1038/s41593-023-01305-8 · Cited by 59
Searching for principles of brain computation
Maass, W. (2016) · Current Opinion in Behavioral Sciences · doi:10.1016/j.cobeha.2016.06.003 · Cited by 49
Neural mechanisms underlying the temporal organization of naturalistic animal behavior
Mazzucato, L. (2022) · eLife · doi:10.7554/elife.76577 · Cited by 46
Error-Backpropagation in Networks of Fractionally Predictive Spiking Neurons
Bohté, S. M. (2011) · Lecture notes in computer science · doi:10.1007/978-3-642-21735-7_8 · Cited by 50
“Behaviorally relevant tasks operate typically over substantially longer timescales; and it is interesting to note that models of reinforcement learning [20,9] in fact similarly employ neurons receiving multiple delayed inputs to allow standard neural networks to learn sequential tasks, with some recent experimental re…”
ResultKlein-Flügge, M. C., Wittmann, M. K., Shpektor, A., Jensen, D. E. A., & Rushworth, M. F. S. (2019) · Nature Communications · doi:10.1038/s41467-019-12557-z · Cited by 44
Iigaya, K. (2016) · eLife · doi:10.7554/elife.18073 · Cited by 45
“Experimental studies suggest that they integrate reward history on multiple timescales rather than a single timescale ( Corrado et al., 2005 ; Fusi et al., 2007 ; Bernacchia et al., 2011 ).”
BackgroundA corticoamygdalar pathway controls reward devaluation and depression using dynamic inhibition code
Yuan, Z., Qi, Z., Wang, R., Cui, Y., An, S., Wu, G., Feng, Q., Lin, R., Dai, R., Li, A., et al. (2023) · Neuron · doi:10.1016/j.neuron.2023.08.022 · Cited by 45
Spike-Based Bayesian-Hebbian Learning of Temporal Sequences
Tully, P. J., Lindén, H., Hennig, M. H., & Lansner, A. (2016) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1004954 · Cited by 42
“The wide diversity of published values associated with these mechanisms [58,106], along with the functional heterogeneity inherent within neocortical microcircuits [3,107], lent some leeway to our choice of biophysical parameter ranges while measuring replay speeds.”
BackgroundInformation dynamics based self-adaptive reservoir for delay temporal memory tasks
Dasgupta, S., Wörgötter, F., & Manoonpong, P. (2013) · Evolving Systems · doi:10.1007/s12530-013-9080-y · Cited by 41
“However, monkey experiments (Bernacchia et al. 2011) have shown that individual neurons can have different time scales of reward memory correlated with the actual behavior.”
BackgroundSynaptic patterning and the timescales of cortical dynamics
Duarte, R., Seeholzer, A., Zilles, K., & Morrison, A. (2017) · Current Opinion in Neurobiology · doi:10.1016/j.conb.2017.02.007 · Cited by 38
“Additionally, dopaminergic signaling, acting as a value arbitrator for the expected outcome of our actions [46], is known to provide diverse reward memory in the form of eligibility traces that span multiple timescales [47].”
BackgroundWei, Z., Inagaki, H., Li, N., Svoboda, K., & Druckmann, S. (2019) · Nature Communications · doi:10.1038/s41467-018-08141-6 · Cited by 44
Klampfl, S., David, S. V., Yin, P., Shamma, S., & Maass, W. (2012) · Journal of Neurophysiology · doi:10.1152/jn.00935.2011 · Cited by 40
Coupled Decision Processes Update and Maintain Saccadic Priors in a Dynamic Environment
Kim, T. D., Kabir, M., & Gold, J. I. (2017) · Journal of Neuroscience · doi:10.1523/jneurosci.3078-16.2017 · Cited by 36
“Integration of these signals across saccades may involve brain regions that encode historydependent information related to saccade choices, which in- cludes parts of parietal, cingulate, and prefrontal cortex (Sugrue et al., 2004; Bernacchia et al., 2011).”
BackgroundEnhanced Population Coding for Rewarded Choices in the Medial Frontal Cortex of the Mouse
Siniscalchi, M. J., Wang, H., & Kwan, A. C. (2018) · Cerebral Cortex · doi:10.1093/cercor/bhy292 · Cited by 36
Invariant timescale hierarchy across the cortical somatosensory network
Rossi-Pool, R., Zainos, A., Álvarez, M., Parra, S., Zizumbo, J., & Romo, R. (2021) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2021843118 · Cited by 39
“Importantly, other studies focusing on reward time constants found an analogous hierarchy in the visual pathway (10, 36, 43); however, controversy has emerged based on research supporting independence between coding and time constants (10) and other studies suggesting that the decay constant can be used to predict sin…”
BackgroundNeuronal spike-rate adaptation supports working memory in language processing
Fitz, H., Uhlmann, M., den Broek, D. V., Duarte, R., Peter Hagoort, A., & Petersson, K. M. (2020) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2000222117 · Cited by 35
Assist-as-needed in lower extremity robotic therapy for children with cerebral palsy
Michmizos, K. P., & Krebs, H. I. (2012) · IEEE/RAS-EMBS International Conference on Biomedical Robotics and Biomechatronics (BioRob) · doi:10.1109/biorob.2012.6290705 · Cited by 37
Human inference reflects a normative balance of complexity and accuracy
Tavoni, G., Doi, T., Pizzica, C., Balasubramanian, V., & Gold, J. I. (2022) · Nature Human Behaviour · doi:10.1038/s41562-022-01357-z · Cited by 35
Differential Reward Coding in the Subdivisions of the Primate Caudate during an Oculomotor Task
Nakamura, K., dos Santos, G. S., Matsuzaki, R., & Nakahara, H. (2012) · Journal of Neuroscience · doi:10.1523/jneurosci.1518-12.2012 · Cited by 36
“Additional studies should reveal where and how the variety of reward history information is generated and used in the basal ganglia circuit for learning and decision making (Yamada et al., 2007; Histed et al., 2009; Noonan et al., 2010; Walton et al., 2010; Bernacchia et al., 2011; Enomoto et al., 2011; Hayden et al.,…”
BackgroundStress enhances model-free reinforcement learning only after negative outcome
Park, H., Lee, D., & Chey, J. (2017) · PLoS ONE · doi:10.1371/journal.pone.0180588 · Cited by 40
Self-citationModel-Free Robust Optimal Feedback Mechanisms of Biological Motor Control
Bian, T., Wolpert, D. M., & Jiang, Z. (2020) · Neural Computation · doi:10.1162/neco_a_01260 · Cited by 35
Adaptive Learning through Temporal Dynamics of State Representation
Razmi, N., & Nassar, M. R. (2022) · Journal of Neuroscience · doi:10.1523/jneurosci.0387-21.2022 · Cited by 32
Dasgupta, S., Wörgötter, F., & Manoonpong, P. (2014) · Frontiers in Neural Circuits · doi:10.3389/fncir.2014.00126 · Cited by 34
“Such reservoir models of the basal ganglia system have also been previously implemented in the context of learning language accusation (Hinaut and Dominey, 2013) or for modeling the experimentally observed varying timescales of neural activity of domapinergic neurons (Bernacchia et al., 2011).”
BackgroundBeirán, M., & Ostojic, S. (2019) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1006893 · Cited by 33
“Neural activity in the brain is matched to the computational requirements imposed by behaviour, and consequently displays dynamics over a similarly vast range of timescales [6–8].”
BackgroundA Control Theoretic Model of Adaptive Learning in Dynamic Environments
Ritz, H., Nassar, M. R., Frank, M. J., & Shenhav, A. (2018) · Journal of Cognitive Neuroscience · doi:10.1162/jocn_a_01289 · Cited by 31
“Consistent with proportional control, there is substantial evidence that participants adjust their behaviors based on recent errors or conflict (Ullsperger, Danielmeier, & Jocham, 2014; Lau & Glimcher, 2005; Gratton, Coles, & Donchin, 1992; Rescorla & Wagner, 1972; Rabbitt, 1966), with corresponding signals observed m…”
InfluentialMultiplexing signals in reinforcement learning with internal models and dopamine
Nakahara, H. (2014) · Current Opinion in Neurobiology · doi:10.1016/j.conb.2014.01.001 · Cited by 32
“The second group of studies concerns foraging behavior, and reveals the rich use of internal models for reward prediction and action selection [16,17 ,18,19], which might not be so obvious when considering a simple comparison of the values for the available options, as in a generic RL model (Figure 1, internal models…”
Idiosyncratic choice bias naturally emerges from intrinsic stochasticity in neuronal dynamics
Lebovich, L., Darshan, R., Lavi, Y., Hansel, D., & Loewenstein, Y. (2019) · Nature Human Behaviour · doi:10.1038/s41562-019-0682-7 · Cited by 29
Voloh, B., Oemisch, M., & Womelsdorf, T. (2020) · Nature Communications · doi:10.1038/s41467-020-18435-3 · Cited by 33
Cirillo, R., Fascianelli, V., Ferrucci, L., & Genovesio, A. (2018) · iScience · doi:10.1016/j.isci.2018.11.033 · Cited by 31
Calderazzo, S. M., Busch, S. E., Moore, T. L., Rosene, D. L., & Medalla, M. (2020) · The Journal of Comparative Neurology · doi:10.1002/cne.24986 · Cited by 34
“In addition to error signals, neuronal activity in the primate dACC encodes reward history, sending value signals based on previously chosen rewards (Bernacchia et al., 2011; Braver et al., 2003; Brown & Braver, 2008; Hunt et al., 2012; John et al., 2013; Kaping et al., 2011; Kolling et al., 2016;…”
BackgroundNeuromodulation and Neurophysiology on the Timescale of Learning and Decision-Making
Grossman, C. D., & Cohen, J. Y. (2022) · Annual Review of Neuroscience · doi:10.1146/annurev-neuro-092021-125059 · Cited by 30
Hierarchical gradients of multiple timescales in the mammalian forebrain
Song, M. J., Shin, E. J., Seo, H., Soltani, A., Steinmetz, N. A., Lee, D., Jung, M. W., & Paik, S. (2024) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2415695121 · Cited by 42
“For example, timescales of neural activity related to choice - and reward - memory are positively correlated with their corresponding behavioral timescales ( 19 , 54 ).”
Self-citationSavin, C., Dayan, P., & Lengyel, M. (2014) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1003489 · Cited by 29
“While earlier theoretical work already considered the importance of inhibition during retrieval [36], and several standard models of spike-based recurrent circuits exhibit a linear dependence of inhibition on the level of excitation [61,62], our model advances these findings by predicting a specific form of feed-back…”
Decorrelation by Recurrent Inhibition in Heterogeneous Neural Circuits
Bernacchia, A., & Wang, X. (2013) · Neural Computation · doi:10.1162/neco_a_00451 · Cited by 25
“While neural dynamics occurs on a variety of timescales in our model2 , as well as in real neurons [8], additional modeling studies are necessary to capture the wide range of phenomena observed in the experimental measures of correlations, including the effects of distance between neurons, multiple timescales and firi…”
Self-citationVincent‐Lamarre, P., Lajoie, G., & Thivierge, J. (2016) · Journal of Computational Neuroscience · doi:10.1007/s10827-016-0619-3 · Cited by 24
“For instance, it is possible to suppress chaos in spontaneously active networks of recurrent firing rate units by the injection of white noise (Molgedey et al 1992) or timevarying input signals (Bertschinger and Natschläger 2004; Rajan et al 2010).”
BackgroundA reservoir of timescales emerges in recurrent circuits with heterogeneous neural assemblies
Stern, M., Istrate, N., & Mazzucato, L. (2023) · eLife · doi:10.7554/elife.86552 · Cited by 36
“Moreover, neurons within the same local circuit exhibit a large range of timescales from milliseconds to minutes ( Bernacchia et al., 2011 ; Cavanagh et al., 2016 ; Miri et al., 2011 ).”
BackgroundPersistent Valence Representations by Ensembles of Anterior Cingulate Cortex Neurons
Caracheo, B. F., Grewal, J. J. S., & Seamans, J. K. (2018) · Frontiers in Systems Neuroscience · doi:10.3389/fnsys.2018.00051 · Cited by 27
Networks that learn the precise timing of event sequences
Veliz‐Cuba, A., Shouval, H. Z., Josić́, K., & Kilpatrick, Z. P. (2015) · Journal of Computational Neuroscience · doi:10.1007/s10827-015-0574-4 · Cited by 23
Han, S., Zheng, R., Li, S., Zhou, B., Jiang, Y. B., Wang, C., Wei, Y., Pang, J., Li, H., Zhang, Y., et al. (2022) · Frontiers in Neuroscience · doi:10.3389/fnins.2022.826609 · Cited by 23
Goodwin, R. A., Lai, Y. G., Taflin, D. E., Smith, D. L., McQuirk, J., Trang, R., & Reeves, R. L. (2023) · Frontiers in Ecology and Evolution · doi:10.3389/fevo.2023.703946 · Cited by 25
State-change decisions and dorsomedial prefrontal cortex: the importance of time
Kolling, N., & O’Reilly, J. X. (2018) · Current Opinion in Behavioral Sciences · doi:10.1016/j.cobeha.2018.06.017 · Cited by 29
“In monkeys, neurons with many different time constants for reward memory (Figure 2c) and a wide range of input integration time constants ranging from miliseconds to tens of seconds, have been observed in ACC (Figure 2b; [31]).”
BackgroundDynamic Coding for Flexible Cognitive Control
Stokes, M. G., Buschman, T. J., & Miller, E. K. (2017) · Wiley eBooks · doi:10.1002/9781118920497.ch13 · Cited by 26
“In this particular model, calcium kinetics provide a window of approximately 2 s for STSP (Mongillo et al., 2008); however, different time constants inherent to other synaptic processes could underlie the diversity of mnemonic time scales observed in the PFC (Bernacchia, Seo, Lee, & Wang, 2011).”
BackgroundMapping and modeling age-related changes in intrinsic neural timescales
Wu, K., & Gollo, L. L. (2025) · Communications Biology · doi:10.1038/s42003-025-07517-x · Cited by 26
“In most electrophysiological studies, which involve fast dynamics, intrinsic neural timescales (INT) are calculated as the timescale of an exponential decay coefficient fitted to the autocorrelation curve [19, 109, 136].”
Development of Multidimensional Representations of Task Phases in the Lateral Prefrontal Cortex
Saga, Y., Iba, M., Tanji, J., & Hoshi, E. (2011) · Journal of Neuroscience · doi:10.1523/jneurosci.0988-11.2011 · Cited by 22
“In relation to the last point, Bernacchia et al. (2011) applied a theoretical analysis to neuronal activity recorded from the dorsolateral prefrontal cortex and related areas of monkeys performing a matching penny task.”
MethodMichmizos, K. P., Vaisman, L., & Krebs, H. I. (2014) · Frontiers in Human Neuroscience · doi:10.3389/fnhum.2014.00962 · Cited by 23
“The theory of multiple time scales is further supported by neurophysiological studies (Karni et al.,1998; Bernacchia et al., 2011).”
Her, E. S., Huh, N., Kim, J., & Jung, M. W. (2016) · Scientific Reports · doi:10.1038/srep27056 · Cited by 21
Common population codes produce extremely nonlinear neural manifolds
De, A., & Chaudhuri, R. (2023) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2305853120 · Cited by 25
Flexibility of Timescales of Evidence Evaluation for Decision Making
Ganupuru, P., Goldring, A. B., Harun, R., & Hanks, T. D. (2019) · Current Biology · doi:10.1016/j.cub.2019.05.037 · Cited by 22
“Selective readout of sets of neurons with differing timescales would allow flexible access that dependson taskdemands [39, 40].”
Srinivasa, N., & Cho, Y. (2014) · Frontiers in Computational Neuroscience · doi:10.3389/fncom.2014.00159 · Cited by 19
“This approach to computing has received experimental evidence (Hahnloser et al., 2002; Nikolic et al., 2009; Crowe et al., 2010; Long et al., 2010; Bernacchia et al., 2011; Klampfl et al., 2012).”
MethodMR. Estimator, a toolbox to determine intrinsic timescales from subsampled spiking activity
Spitzner, F. P., Dehning, J., Wilting, J., Hagemann, A., Neto, J. P., Zierenberg, J., & Priesemann, V. (2021) · PLoS ONE · doi:10.1371/journal.pone.0249447 · Cited by 21
“Timescales of neural dynamics have been analyzed in various contexts and can be interpreted as reward memory [25] or as temporal receptive windows [26].”
BackgroundNaud, R., Houtman, D., Rose, G. J., & Longtin, A. (2015) · Journal of Neurophysiology · doi:10.1152/jn.00138.2015 · Cited by 18
Narrative event segmentation in the cortical reservoir
Dominey, P. F. (2021) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1008993 · Cited by 18
Sympathetic involvement in time-constrained sequential foraging
Dundon, N. M., Garrett, N., Babenko, V., Cieslak, M., Daw, N. D., & Grafton, S. T. (2020) · Cognitive Affective & Behavioral Neuroscience · doi:10.3758/s13415-020-00799-0 · Cited by 15
Inhibitory control of frontal metastability sets the temporal signature of cognition
Fontanier, V., Sarazin, M., Stoll, F. M., Delord, B., & Procyk, E. (2022) · eLife · doi:10.7554/elife.63795 · Cited by 19
“Long spiking timescales potentially allow integration over longer durations, which seems crucial in the context of higher cognitive functions, learning, and reward-based decision-making ( Bernacchia et al., 2011 ).”
InfluentialSpatial and temporal correlations in neural networks with structured connectivity
Shi, Y.-L., Zeraati, R., Levina, A., & Engel, T. A. (2023) · Physical Review Research · doi:10.1103/physrevresearch.5.013005 · Cited by 16
“Spatial and temporal correlations of neural activity can be modulated during changes in behavioral states, such as selective attention [1,5,12–18] or working memory maintenance [10], and relate to computations across different cognitive tasks [19–21].”
BackgroundPark, J., Yoon, H.-D., Yoo, T., Shin, M., & Jeon, H. (2020) · Scientific Reports · doi:10.1038/s41598-020-75157-8 · Cited by 15
Extended Anderson Criticality in Heavy-Tailed Neural Networks
Wardak, A., & Gong, P. (2022) · Physical Review Letters · doi:10.1103/physrevlett.129.048103 · Cited by 15
Engaging and Exploring: Cortical Circuits for Adaptive Foraging Decisions
Barack, D. L., & Platt, M. L. (2017) · Nebraska Symposium on Motivation/The Nebraska symposium on motivation · doi:10.1007/978-3-319-51721-6_6 · Cited by 14
Danskin, B. P., Hattori, R., Zhang, Y. E., Babic, Z., Aoi, M., & Komiyama, T. (2023) · Science Advances · doi:10.1126/sciadv.adj4897 · Cited by 17
Pinheiro‐Chagas, P., Sava‐Segal, C., Akkol, S., Daitch, A. L., & Parvizi, J. (2024) · Journal of Neuroscience · doi:10.1523/jneurosci.2118-22.2024 · Cited by 16
Kurikawa, T., & Kaneko, K. (2021) · Frontiers in Computational Neuroscience · doi:10.3389/fncom.2021.743537 · Cited by 15
“The timescale of individual neurons in the same area is distributed over two digits (Bernacchia et al., 2011; Wasmuht et al., 2018).”
BackgroundOn self-feedback connectivity in neural mass models applied to event-related potentials
Youssofzadeh, V., Prasad, G., & Wong‐Lin, K. (2015) · NeuroImage · doi:10.1016/j.neuroimage.2014.12.067 · Cited by 14
Zeraati, R., Shi, Y.-L., Steinmetz, N. A., et al. (2021) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2021.05.17.444537 · Cited by 14
A Spectrum of Time Horizons for Dopamine SignalsPreprint
Wei, W., Mohebi, A., & Berke, J. D. (2021) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2021.10.31.466705 · Cited by 14
“Expectation of future reward can reflect past reward history over a range of possible (retrospective) time horizons (41, 42).”
BackgroundMorita, K., & Kawaguchi, Y. (2015) · European Journal of Neuroscience · doi:10.1111/ejn.12994 · Cited by 12
“…and durations are represented (cf. Houk et al. , 1995; Joel et al. , 2002; Mauk & Buonomano, 2004; Daw et al. , 2006; Nakahara & Kaveri, 2010; Bernacchia et al. , 2011; Gershman et al. , 2014), and whether and how different rein-forcement-learning algorithms, such as Q-learning,…”
BackgroundGeneration of Scale-Invariant Sequential Activity in Linear Recurrent Networks
Liu, Y., & Howard, M. W. (2020) · Neural Computation · doi:10.1162/neco_a_01288 · Cited by 11
“Bernacchia, Seo, Lee, and Wang (2011) showed that in monkey prefrontal, cingulate, and parietal cortex, reward modulates neural activity multiplicatively with a spectrum of time constants (Bernacchia et al., 2011).”
BackgroundSlow ramping emerges from spontaneous fluctuations in spiking neural networks
Gavenas, J., Rutishauser, U., Schurger, A., & Maoz, U. (2024) · Nature Communications · doi:10.1038/s41467-024-51401-x · Cited by 18
“…in the fi ring rate of human medial frontal cortex neurons evolves on the time scale of seconds 7 , but studies have reported that neurons in medial frontal cortex have autocorrelation timeconstants on the order of hundreds ofmilliseconds 20 – 22 (although some have much longer time constants 23 ).”
Neural Correlates of Temporal Credit Assignment in the Parietal Lobe
Gersch, T. M., Foley, N. C., Eisenberg, I. W., & Gottlieb, J. P. (2014) · PLoS ONE · doi:10.1371/journal.pone.0088725 · Cited by 12
“showed that LIP neurons have reward learning across consecutive trials whose decisions are statistically unrelated, suggesting that learning is automatic and independent of the significance of an action [19].”
Microscopic theory of intrinsic timescales in spiking neural networks
van Meegen, A., & van Albada, S. J. (2021) · Physical Review Research · doi:10.1103/physrevresearch.3.043077 · Cited by 11
The geometry of domain-general performance monitoring in the human medial frontal cortexPreprint
Fu, Z., Beam, D. E., Chung, J. M., Reed, C. M., Mamelak, A. N., Adolphs, R., & Rutishauser, U. (2021) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2021.07.08.451594 · Cited by 11
Forget-me-some: General versus special purpose models in a hierarchical probabilistic task
Bröker, F., Marshall, L., Bestmann, S., & Dayan, P. (2018) · PLoS ONE · doi:10.1371/journal.pone.0205974 · Cited by 12
“…impact of past experience decays with time have been proposed on different levels of analysis in the past, such as models for neural integration [41–45], computations of neural populations [7], and models accounting for behaviour in tasks of stimulus-response, probabilistic contingency, and…”
BackgroundMethodMultiscale Computation and Dynamic Attention in Biological and Artificial Intelligence
Badman, R. P., Hills, T. T., & Akaishi, R. (2020) · Brain Sciences · doi:10.3390/brainsci10060396 · Cited by 9
“The long-term (average) reward rates act to bias decisions for leaving the current environment and the short-term (last) reward rates influences decision for staying in the current patch (Image adapted with permission from Wittmann et al., Nature Communications, 2016; Nature Publishing Group [66]).”
BackgroundInfluentialStrategy-dependent effects of working-memory limitations on human perceptual decision-making
Schapiro, K. A., Josić́, K., Kilpatrick, Z. P., & Gold, J. I. (2022) · eLife · doi:10.7554/elife.73610 · Cited by 13
“…timescales ranging from tens to hundreds of milliseconds for many perceptual decisions to seconds or longer for reward-based and other decisions ( Bernacchia et al., 2011 ; Gold and Stocker, 2017 ), requires working memory to maintain representations of new, incoming evidence and/or the…”
The Evolution of the Prefrontal Cortex in Early Primates and Anthropoids
Wise, S. P. (2016) · Elsevier eBooks · doi:10.1016/b978-0-12-804042-3.00092-0 · Cited by 13
Miri, A., Bhasin, B. J., Aksay, E. R. F., Tank, D. W., & Goldman, M. S. (2022) · The Journal of Physiology · doi:10.1113/jp282496 · Cited by 9
“short-term memory circuits (Bernacchia et al., 2011; Machens et al., 2010), suggesting that multi-timescale responses may represent a core feature of short-term memory systems throughout the brain.”
BackgroundThe prefrontal cortex and hybrid learning during iterative competitive games
Abe, H., Seo, H., & Lee, D. (2011) · Annals of the New York Academy of Sciences · doi:10.1111/j.1749-6632.2011.06223.x · Cited by 11
Self-citationNeural classification maps for distinct word combinations in Broca’s area
Schell, M., Friederici, A. D., & Zaccarella, E. (2022) · Frontiers in Human Neuroscience · doi:10.3389/fnhum.2022.930849 · Cited by 11
“We believe that these findings pave the way for a finegrained characterization of the linguistic features at work during combinatorial processes (Chesi and Canal, 2019), and they generally support spatiotemporal neural models showing that different neural populations might be recruited for different processing demands…”
BackgroundBiological computation through recurrence
Vidal-Saez, M. S., Vilarroya, Ó., & García‐Ojalvo, J. (2024) · Biochemical and Biophysical Research Communications · doi:10.1016/j.bbrc.2024.150301 · Cited by 9
“Nonetheless, discerning the computational basis of cortical circuits is still today an open question in neuroscience, and efforts to clarify to what end recurrence is exploited in cortical circuits are under way 31,32 .”
BackgroundBachman, M. D., Watts, A. T. M., Collins, P. F., & Bernat, E. M. (2021) · Psychophysiology · doi:10.1111/psyp.13907 · Cited by 12
Consistent population activity on the scale of minutes in the mouse hippocampus
Liu, Y., Levy, S. J., Mau, W., Geva, N., Rubin, A., Ziv, Y., Hasselmo, M. E., & Howard, M. W. (2022) · Hippocampus · doi:10.1002/hipo.23409 · Cited by 10
Iigaya, K., Ahmadian, Y., Sugrue, L. P., Corrado, G. S., Loewenstein, Y., Newsome, W. T., & Fusi, S. (2017) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/141309 · Cited by 11
“Moreover, in the recent years there has been accumulating evidence that several neural systems – ranging from individual neurons studied in vitro, to complex neural circuits studied in vivo – have the ability to integrate their inputs, including those that represent rewards, on multiple timescales [48, 38, 27, 58, 33,…”
BackgroundCognitive computation using neural representations of time and space in the Laplace domainPreprint
Howard, M. W., & Hasselmo, M. E. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2003.11668 · Cited by 19
“That is, perhaps the same equations that govern memory for reward history over tens of minutes (Bernacchia et al., 2011) also govern the evolution of evidence between two decision bounds over the scale of less than a second (Koay et al., 2019).”
BackgroundTrepka, E. B., Spitmaan, M., Qi, X.-L., Constantinidis, C., & Soltani, A. (2023) · Journal of Neuroscience · doi:10.1523/jneurosci.2442-21.2023 · Cited by 9
The midcingulate cortex and temporal integration
Procyk, E., Fontanier, V., Sarazin, M., Delord, B., Goussi-Denjean, C., & Wilson, C. R. E. (2021) · International review of neurobiology · doi:10.1016/bs.irn.2020.12.004 · Cited by 12
“…reports made a straight link between immediate reward and cell activity, it is now clear that the recent history of reward alters the firing of these cells (Seo and Lee, 2007; Kawai et al., 2015), and that this effect will occur with varying timescales across neurons (Bernacchia et al., 2011).”
Booras, A., Stevenson, T., McCormack, C. N., Rhoads, M. E., & Hanks, T. D. (2021) · Scientific Reports · doi:10.1038/s41598-021-92470-y · Cited by 13
The Edge of Stability: Response Times and Delta Oscillations in Balanced Networks
Gillary, G., & Niebur, E. (2016) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1005121 · Cited by 10
“These networks can respond to their input with high temporal precision [4–6], selectively amplify patterned input signals [7], transmit multiple signals simultaneously betweenneural assemblies embedded in large networks [8], and maintain activity on a broad range of time constants [9] including those of short-term mem…”
A hippocampus-inspired illumination time-resolved device for neural coding
Li, C., Du, W., Liu, H., Yang, M. S., Xu, H., Wu, J., & Wang, Z. (2021) · Science China Materials · doi:10.1007/s40843-021-1828-5 · Cited by 9
“2a shows the input optical stimulus (top) and the resulting output current (middle), whose on/off response tendency is similar to the memory trace [64] (bottom) of brain.”
BackgroundACC neural ensemble dynamics are structured by strategy prevalence
Proskurin, M., Manakov, M., & Karpova, A. Y. (2023) · eLife · doi:10.7554/elife.84897 · Cited by 10
“Specifically, we capitalized on the previous finding that much of such direct history impact in the ACC is exerted within fewer than five trials ( Bernacchia et al., 2011 ).”
BackgroundSaitō, H., Katahira, K., Okanoya, K., & Okada, M. (2014) · Frontiers in Computational Neuroscience · doi:10.3389/fncom.2014.00018 · Cited by 9
“A reservoir network (Jaeger et al., 2007), which can reproduce neural activity in the monkey prefrontal cortex, preserves the memory trace of a reward with one or two time constants (Bernacchia et al., 2011).”
Modulation of Dopamine for Adaptive Learning: a Neurocomputational Model
Inglis, J. B., Valentin, V. V., & Ashby, F. G. (2020) · Computational Brain & Behavior · doi:10.1007/s42113-020-00083-x · Cited by 10
Effects of Connectivity on Narrative Temporal Processing in Structured Reservoir Computing
Dominey, P. F., Ellmore, T. M., & Ventre‐Dominey, J. (2022) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn55064.2022.9891967 · Cited by 9
“This provides a rich framework for future research on the interaction between network structure, processing time constant dynamics modulated by input driven and intrinsic processes in biological and artificial networks [24].”
BackgroundUncovering a Timescale Hierarchy by Studying the Brain in a Natural Context
Hasson, U. (2025) · Journal of Neuroscience · doi:10.1523/jneurosci.2368-24.2025 · Cited by 9
Li, G., Deng, L., Wang, D., Wang, W., Zeng, F., Zhang, Z., Li, H., Song, S., Pei, J., & Shi, L. (2016) · Frontiers in Computational Neuroscience · doi:10.3389/fncom.2016.00136 · Cited by 8
“It is known that there is indeed a wide range of time constants for the neurons in the human brain, from hundreds of milliseconds to tens of seconds (Bernacchia et al., 2011).”
BackgroundDimensions of Timescales in Neuromorphic Computing SystemsPreprint
Jaeger, H., Doorakkers, D., Lawrence, C., & Indiveri, G. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2102.10648 · Cited by 16
“• Experimental demonstrations of different time constants in cortical processing (Bernacchia et al., 2011).”
BackgroundRaposo, I., Fiebelkorn, I. C., Lin, J. J., Parvizi, J., Kästner, S., Knight, R. T., Breska, A., & Helfrich, R. F. (2025) · PLoS Biology · doi:10.1371/journal.pbio.3003232 · Cited by 7
Learning Temporal Relationships Between Symbols with Laplace Neural Manifolds
Howard, M. W., Esfahani, Z. G., Le, B., & Sederberg, P. B. (2024) · Computational Brain & Behavior · doi:10.1007/s42113-024-00230-8 · Cited by 11
“These studies show history-dependent e ff ects in a wide range of brain regions and often show a continuous spectrum of decay rates within a brain region (see especially Bernacchia et al., 2011).”
Kalhan, S., Garrido, M. I., Hester, R., & Redish, A. D. (2023) · Neural Networks · doi:10.1016/j.neunet.2023.09.032 · Cited by 9
Short-Term Influence of Recent Trial History on Perceptual Choice Changes with Stimulus Strength
Jiang, W., Liu, J., Zhang, D., Xie, T., & Yao, H. (2019) · Neuroscience · doi:10.1016/j.neuroscience.2019.04.010 · Cited by 7
Metadata
- Authors
- Bernacchia, A., Seo, H., Lee, D., & Wang, X.-J.
- Affiliations
- Yale University (US)
- Corresponding author
- Wang, Xiao-Jing
- Venue
- Nature Neuroscience
- Published
- 2011-02-13
- Volume
- 14
- Issue
- 3
- Pages
- 366-372
- ISSN
- 1097-6256 (ISSN-L)
- 1546-1726
- PMC
- PMC3079398
- OpenAlex
- W2139630667
- Semantic Scholar
- 87fa62529ba1b9dd0ff3d6e9fcd5b62a26363247
- Cited by
- 292
- References
- 52
- Citations by year
- 2026: 14 · 2025: 13 · 2024: 19 · 2023: 15 · 2022: 23 · 2021: 25 · 2020: 28 · 2019: 19 · 2018: 21 · 2017: 21 · 2016: 16 · 2015: 20 · 2014: 18 · 2013: 15 · 2012: 16
- FWCI
- 9.41
- Type
- journal article
- Language
- en
- Open access
- green
- www.ncbi.nlm.nih.gov
- Licence
- www.springer.com/tdm (tdm)
- Locations
- Nature Neuroscience (publishedVersion)
- PubMed (publishedVersion)
- PubMed Central (acceptedVersion)
- Funders
- National Institute of Mental Health (R01MH062349, R01MH073246)
- National Institute on Drug Abuse (DA029330, R01DA029330)
- Topics
- Life Sciences › Neuroscience › Cognitive Neuroscience › Neural and Behavioral Psychology Studies
- Life Sciences › Neuroscience › Cognitive Neuroscience › Neural dynamics and brain function
- Life Sciences › Neuroscience › Cognitive Neuroscience › Memory and Neural Mechanisms
- Keywords
- reward memory, power-law distribution, reinforcement learning, decision-making, prefrontal cortex, cingulate cortex, parietal cortex, reward anticipation, decision time, cortical neurons
- MeSH terms
- Animals, Haplorhini, Cerebral Cortex, Decision Making, Humans, Memory, Nerve Net, Neurons, Neuropsychological Tests, Reinforcement, Psychology, Reward, Time Factors
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
A language model drew the claims from the paper on 2026-10-10, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-10-10 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
