The DOI matches its Crossref record; 51 of the 52 references it cites were found in a registry or an index; none of the 59 sources its claims cite is checked for support yet.
A reservoir of time constants for memory traces in cortical neurons
Open accessAbstract
According to reinforcement learning theory of decision making, reward expectation is computed by integrating past rewards with a fixed timescale. In contrast, we found that a wide range of time constants is available across cortical neurons recorded from monkeys performing a competitive game task. By recognizing that reward modulates neural activity multiplicatively, we found that one or two time constants of reward memory can be extracted for each neuron in prefrontal, cingulate and parietal cortex. These timescales ranged from hundreds of milliseconds to tens of seconds, according to a power law distribution, which is consistent across areas and reproduced by a 'reservoir' neural network model. These neuronal memory timescales were weakly, but significantly, correlated with those of monkey's decisions. Our findings suggest a flexible memory system in which neural subpopulations with distinct sets of long or short memory timescales may be selectively deployed according to the task demands.
Overview
AI snapshot
Written from the paper’s full text.
TL;DR
Cortical neurons recorded from monkeys playing a competitive game carry reward-memory traces with a wide range of time constants, distributed as a power law across ACCd, DLPFC, and LIP. A reservoir network model operating near criticality reproduces this distribution, and neural timescales are weakly but significantly correlated with behavioral timescales.
Key findings
- Across 681 neurons from ACCd (154), DLPFC (322), and LIP (205), reward memory was best fit by a single exponential for 269 neurons, double exponentials for 268 neurons, and no memory for 144 neurons; 79% (537/681) showed exponential decay.
- Memory timescales ranged from hundreds of milliseconds to tens of seconds and followed a power-law distribution with exponent −2, consistent across the three cortical areas; about 20% of neurons (133/681) had timescales larger than one trial (29% ACCd, 19% DLPFC, 13% LIP, χ²-test p=0.0005).
- Behavioral timescales from a reinforcement learning fit also followed a power law, and neural and behavioral timescales were weakly but significantly correlated across sessions (R=0.12, p=0.003); simultaneously recorded neuron pairs showed no significant correlation (312 pairs, R=0.07, p=0.2).
- A randomly connected reservoir network model reproduced the power-law distribution of timescales when operating at the critical point, with exponentially distributed amplitudes and normal connection matrix.
Methods
- Single-unit activity was recorded from six rhesus monkeys performing a matching pennies task; firing rates were computed in twelve 250-ms epochs per trial, and reward history up to five trials back was used.
- Memory traces were estimated with multiple linear regression, then fit with single or double exponential functions multiplied by the epoch code; model selection used Bayesian Information Criterion, and reshuffling trials served as a control (96%, 656/681, showed no memory after reshuffling).
- Behavior was fit per session with a standard reinforcement learning model using maximum likelihood, with significance assessed against 100 reshuffled surrogate sessions; the network model used 1,000 neurons with dynamics dv/dt = J·v(t) + h·Rew(t).
Limitations
- The paper states that the matching pennies task does not necessarily require reward memory, that the neural–behavioral correlation is weak and might be caused by unknown factors, that the neuronal and behavioral ranges differ, and that behavioral data were not fit with a multi-timescale RL model.
- The paper also states that the network model is linear and would need nonlinearities if connection-width exceeds the critical level, and that the readout driving behavior is not explicitly modeled.
- Evident from the text: only about two neurons were simultaneously recorded on average, limiting pairwise analyses, and the study reports correlational rather than causal links between neural timescales and behavior.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
0 of 59 cited sources checked · 0 supported · 0 flagged
None of the sources cited for this claim have been checked yet.
In the paper
“Finally, we show that a randomly connected circuit model, akin to a “reservoir” network13-15, can reproduce the observed distribution of timescales, provided that the network operates at the critical point (or “edge of chaos”)16-18.”
§ A reservoir of time constants for memory traces in cortical neurons > Abstract
What the sources say
- Not checked yetcites [13] Maass, W., Natschläger, T. & Markram, H. Real-time computing without stable states: a new framework for neural computation based on perturbations. Neural Comput. 14, 2531–2560 (2002). source
- Not checked yetcites [14] Jaeger, H., Lukosevicius, M., Popovici, D. & Siewert, U. Optimization and applications of echo state networks with leaky-integrator neurons. Neural Netw. 20, 335–352 (2007). source
- Not checked yetcites [15] Verstraeten, D., Schrauwen, B., D′Haene, M. & Stroobandt, D. An experimental unification of reservoir computing methods. Neural Netw. 20, 391–403 (2007). source
- Not checked yetcites [16] Sussillo, D. & Abbott, L.F. Generating coherent patterns of activity from chaotic neural networks. Neuron 63, 544–557 (2009). source
- Not checked yetcites [17] Bertschinger, N. & Natschlager, T. Real-time computation at the edge of chaos in recurrent neural networks. Neural Comput. 16, 1413–1436 (2004). source
- Not checked yetcites [18] Langton, C.G. Computation at the edge of chaos: phase transitions and emergent computations. Physica D 42, 12–37 (1990). source
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passes- The authors state this as an assumption: The specific form of the input signals is not important; the results depend only on the synaptic matrix J.
- The authors state this as an assumption: The connection weights (entries of matrix J) are randomly distributed.
- The authors state this as an assumption: The linear model is sufficient for reproducing the observed power-law distribution of timescales under specific conditions.
None of the sources cited for this claim have been checked yet.
In the paper
“Consistent with previous studies23-25, we found that the activity of neurons varied substantially in different trial epochs (99% of neurons, 675/681, ANOVA p<0.05).”
§ A reservoir of time constants for memory traces in cortical neurons > Results > Multiplicative memory traces in cortical neurons
What the sources say
- Not checked yetcites [23] Lapish, C.C., Durstewitz, D., Chandler, L.J. & Seamans, J.K. Successful choice behavior is associated with distinct and coherent network states in anterior cingulate cortex. Proc. Natl. Acad. Sci. USA 105, 11963–11968 (2008). source
- Not checked yetcites [24] Sigala, N., Kusonoki, M., Nimmo-Smith, I., Gaffan, D. & Duncan, J. Hierarchical coding for sequential task events in the monkey prefrontal cortex. Proc. Natl. Acad. Sci. USA 105, 11969–11974 (2008). source
- Not checked yetcites [25] Jin, D.Z., Fujii, N. & Graybiel, A.N. Neural representation of time in cortico-basal ganglia circuits. Proc. Natl. Acad. Sci. USA 106, 19156–19161 (2009). source
Strength:plain· not yet calibratedCausal:correlational· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes? Load-bearing: unclear✓ Based: passesA result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“reproduced by a “reservoir” neural network model.”
§ A reservoir of time constants for memory traces in cortical neurons > Abstract
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passes- The authors state this as an assumption: The specific form of the input signals is not important; the results depend only on the synaptic matrix J.
- The authors state this as an assumption: The connection weights (entries of matrix J) are randomly distributed.
- The authors state this as an assumption: The linear model is sufficient for reproducing the observed power-law distribution of timescales under specific conditions.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“In some cases, the sum of two exponential functions, ex(t) = A1·e−t/τ1+A2·e−t/τ2, fitted the data better than a single exponential”
§ A reservoir of time constants for memory traces in cortical neurons > Results > Multiplicative memory traces in cortical neurons
Strength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes? Load-bearing: unclear✓ Based: passes- The authors state this as an assumption: The memory trace f is factorized into the epoch code g(k) and an exponential function ex(t).
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“we show that memory time constants can be extracted from the activity of single neurons.”
§ A reservoir of time constants for memory traces in cortical neurons > Abstract
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passes- The authors state this as an assumption: The memory trace f is factorized into the epoch code g(k) and an exponential function ex(t).
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“a fraction of them did not show a modulation of the memory by the epoch code.”
§ A reservoir of time constants for memory traces in cortical neurons > Results > Multiplicative memory traces in cortical neurons
Strength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes? Load-bearing: unclear✓ Based: passes- The authors state this as an assumption: The memory trace f is factorized into the epoch code g(k) and an exponential function ex(t).
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“our analysis of animal's behavior suggests that the timescale over which reward events affect decisions changes across experimental sessions”
§ A reservoir of time constants for memory traces in cortical neurons > Abstract
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesA result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The time constants for reward memory at the behavioural and neuronal levels were weakly but significantly correlated across experimental sessions.”
§ A reservoir of time constants for memory traces in cortical neurons > Abstract
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passes- The authors state this as an assumption: The memory trace f is factorized into the epoch code g(k) and an exponential function ex(t).
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“These neuronal memory timescales were weakly but significantly correlated with those of monkey's decisions.”
§ A reservoir of time constants for memory traces in cortical neurons > Abstract
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passes- The authors state this as an assumption: The memory trace f is factorized into the epoch code g(k) and an exponential function ex(t).
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“By recognizing that reward modulates neural activity multiplicatively, we found that one or two time constants of reward memory can be extracted for each neuron in prefrontal, cingulate, and parietal cortex.”
§ A reservoir of time constants for memory traces in cortical neurons > Abstract
Load-bearingStrength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passes- The authors state this as an assumption: The memory trace f is factorized into the epoch code g(k) and an exponential function ex(t).
- It may state two separate claims as one. 26
- Probably not a substantial claim. 31
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The memory trace does not decay monotonically, but its strength is modulated throughout the trial consistent with the epoch code.”
§ A reservoir of time constants for memory traces in cortical neurons > Results > Multiplicative memory traces in cortical neurons
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesIt may state two separate claims as one.
- The authors state this as an assumption: The memory trace f is factorized into the epoch code g(k) and an exponential function ex(t).
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“and there is a wide range of timescales across neurons, obeying a power law distribution.”
§ A reservoir of time constants for memory traces in cortical neurons > Abstract
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesIt may state two separate claims as one.
- The authors state this as an assumption: The memory trace f is factorized into the epoch code g(k) and an exponential function ex(t).
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Using the Bayesian Information Criterion (BIC), we found that the best fit was a single exponential for 269 neurons and double exponentials for 268 neurons, while the remaining 144 neurons were fitted best by a model with ex(t)=0.”
§ A reservoir of time constants for memory traces in cortical neurons > Results > Multiplicative memory traces in cortical neurons
Load-bearingStrength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesIt may state two separate claims as one.
- The authors state this as an assumption: The memory trace f is factorized into the epoch code g(k) and an exponential function ex(t).
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“These timescales ranged from hundreds of milliseconds to tens of seconds, according to a power-law distribution”
§ A reservoir of time constants for memory traces in cortical neurons > Abstract
Load-bearingStrength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesIt may state two separate claims as one.
- The authors state this as an assumption: The memory trace f is factorized into the epoch code g(k) and an exponential function ex(t).
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Different neurons have different magnitudes and time courses of the firing rate, and they all show a decaying memory trace modulated by the epoch code.”
§ A reservoir of time constants for memory traces in cortical neurons > Results > Multiplicative memory traces in cortical neurons
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesIt may state two separate claims as one.
- The authors state this as an assumption: The memory trace f is factorized into the epoch code g(k) and an exponential function ex(t).
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“If J is normal, we showed that the amplitudes of the memory traces follow an exponential distribution (Fig.8b), consistent with the experimental observations (Fig.8a).”
§ A reservoir of time constants for memory traces in cortical neurons > Results > Neural network model for memory traces
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesIt may state two separate claims as one.
- The authors state this as an assumption: The specific form of the input signals is not important; the results depend only on the synaptic matrix J.
- The authors state this as an assumption: The connection weights (entries of matrix J) are randomly distributed.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“For the 196 sessions fitted by the RL model, the distribution of behavioral timescales followed a power law distribution (Fig.5a), and the exponent was consistent with that measured in the neural distribution.”
§ A reservoir of time constants for memory traces in cortical neurons > Results > Comparison with behavior
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesIt may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Results are shown in Fig.6, suggesting that both the behavioral (panel (a)) and neural memory timescales (panel (b)) were fairly stable within a single session.”
§ A reservoir of time constants for memory traces in cortical neurons > Results > Comparison with behavior
Strength:hedged· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes? Load-bearing: unclear✓ Based: passesIt may state two separate claims as one.
- The authors state this as an assumption: The memory trace f is factorized into the epoch code g(k) and an exponential function ex(t).
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Our results indicate that multiplicative modulation and a power law distribution of memory timescales also hold for memory trace of past choices”
§ A reservoir of time constants for memory traces in cortical neurons > Results > Comparison with behavior
Strength:hedged· not yet calibratedCausal:correlational· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes? Load-bearing: unclear✓ Based: passesIt may state two separate claims as one.
- The authors state this as an assumption: The memory trace f is factorized into the epoch code g(k) and an exponential function ex(t).
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“First, we found that the connection weights must be broadly distributed among neuron pairs, and this endows the network with a wide variety of timescales.”
§ A reservoir of time constants for memory traces in cortical neurons > Results > Neural network model for memory traces
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesIt may state two separate claims as one.
- The authors state this as an assumption: The specific form of the input signals is not important; the results depend only on the synaptic matrix J.
- The authors state this as an assumption: The connection weights (entries of matrix J) are randomly distributed.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“About 20% of all recorded neurons (133/681) had timescale larger than one trial (29% in ACCd, 19% in DLPFC and 13% in LIP, χ2-test p=0.0005”
§ A reservoir of time constants for memory traces in cortical neurons > Results > Multiplicative memory traces in cortical neurons
Load-bearingStrength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesIt may state two separate claims as one.
- The authors state this as an assumption: The memory trace f is factorized into the epoch code g(k) and an exponential function ex(t).
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The latter is interpreted as no memory, and the corresponding neurons were excluded from further analysis.”
§ A reservoir of time constants for memory traces in cortical neurons > Results > Multiplicative memory traces in cortical neurons
Load-bearingStrength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesProbably not a substantial claim.
- The authors state this as an assumption: The memory trace f is factorized into the epoch code g(k) and an exponential function ex(t).
None of the sources cited for this claim have been checked yet.
In the paper
“and interpreted in the framework of reinforcement learning (RL) theory4-5.”
§ A reservoir of time constants for memory traces in cortical neurons > Abstract
What the sources say
- Not checked yetcites [4] Soltani, A., Lee, D. & Wang, X.-J. Neural mechanism for stochastic behavior during a competitive game. Neural Netw. 19, 1075–1090 (2006). source
- Not checked yetcites [5] Sutton, R.S. & Barto,, A.G. Reinforcement Learning, an Introduction (MIT Press, Cambridge, Massachusetts, 1998). source
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesProbably not a substantial claim.
None of the sources cited for this claim have been checked yet.
In the paper
“So far a few studies have estimated the time constant of this filter from behavior, and assessed how past rewards affect choice selection8-12”
§ A reservoir of time constants for memory traces in cortical neurons > Abstract
What the sources say
- Not checked yetcites [8] Lau, B. & Glimcher, P.W. Dynamic response-by-response models of matching behavior in rhesus monkeys. J. Exp. Anal. Behav. 84, 555–579 (2005). source
- Not checked yetcites [9] Corrado, G.S., Sugrue, L.P., Seung, H.S. & Newsome, W.T. Linear-nonlinear-Poisson models of primate choice dynamics. J. Exp. Anal. Behav. 84, 581–617 (2005). source
- Not checked yetcites [10] Kennerley, S.W., Walton, M.E., Behrens, T.E., Buckley, M.J. & Rushworth, M.F. Optimal decision making and the anterior cingulate cortex. Nat. Neurosci. 9, 940–947 (2006). source
- Not checked yetcites [11] Lee, D., Conroy, M.L., McGreevy, B.P. & Barraclough, D.J. Reinforcement learning and decision making in monkeys during a competitive game. Brain Res. Cogn. Brain Res. 22, 45–58 (2004). source
- Not checked yetcites [12] Kim, S., Hwang, J., Seo, H. & Lee, D. Valuation of uncertain and delayed rewards in primate prefrontal cortex. Neural Netw. 22, 294–304 (2009). source
Strength:plain· not yet calibratedCausal:correlational· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.
None of the sources cited for this claim have been checked yet.
In the paper
“Neural correlates of reward valuation have been observed in different studies1-3”
§ A reservoir of time constants for memory traces in cortical neurons > Abstract
What the sources say
- Not checked yetcites [1] Kable, J.W. & Glimcher, P.W. The neurobiology of decision: consensus and controversy. Neuron 63, 733–745 (2009). source
- Not checked yetcites [2] Rushworth, M.F. & Behrens, T.E. Choice, uncertainty and value in prefrontal and cingulate cortex. Nat. Neurosci. 11, 389–397 (2008). source
- Not checked yetcites [3] Wang, X.-J. Decision making in recurrent neural circuits. Neuron 60, 215–234 (2008). source
Strength:plain· not yet calibratedCausal:correlational· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Hence, each area is endowed with a reservoir of time constants for reward memory, which are distributed heterogeneously across neurons.”
§ A reservoir of time constants for memory traces in cortical neurons > Abstract
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesIt may state two separate claims as one.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“We tested the validity of the fitting procedure by randomly reshuffling the order of trials in each session, and we consistently found that 96% of neurons (656/681) show no memory after reshuffling.”
§ A reservoir of time constants for memory traces in cortical neurons > Results > Multiplicative memory traces in cortical neurons
Load-bearingStrength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesIt may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Similarly, the double exponential decay of memory may correspond to a reward prediction error signal: if the short timescale (τ1) is small enough (about one trial or smaller) then the corresponding exponential filter will respond primarily to the reward in the present trial, while the long timescale (τ2) may provide a value signal by weighting the rewards in the past few trials.”
§ A reservoir of time constants for memory traces in cortical neurons > Discussion
Strength:hedged· not yet calibratedCausal:conditional· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Similarly, the double exponential decay of memory may correspond to a reward prediction error signal: if the short timescale (τ1) is small enough (about one trial or smaller) then the corresponding exponential filter will respond primarily to the reward in the present trial, while the long timescale (τ2) may provide a value signal by weighting the rewards in the past few trials.”
§ A reservoir of time constants for memory traces in cortical neurons > Discussion
Strength:hedged· not yet calibratedCausal:conditional· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The animal's behavior may be determined by a readout system that is able to sample, at different times, from a variety of timescales present in the reservoir.”
§ A reservoir of time constants for memory traces in cortical neurons > Results > Comparison with behavior
Strength:hedged· not yet calibratedCausal:conditional· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✗ Based: failsProbably not a substantial claim. The paper’s conclusions may not depend on it. The paper may give no reason why it holds.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“In fact, for a power law distribution ~ τ−2, the variance increases with the sample size and, in principle, arbitrarily large timescales would be observed with a proportionally large increment in the number of recorded neurons.”
§ A reservoir of time constants for memory traces in cortical neurons > Results > Multiplicative memory traces in cortical neurons
Strength:hedged· not yet calibratedCausal:direct· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes? Load-bearing: unclear✓ Based: passesIt may state two separate claims as one.
None of the sources cited for this claim have been checked yet.
In the paper
“The time course of memory-related activity varies across different neurons and different task protocols, including persistent, ramping, and multi-phasic activity41-43.”
§ A reservoir of time constants for memory traces in cortical neurons > Discussion
What the sources say
- Not checked yetcites [41] Rainer, G. & Miller, E.K. Time course of object-related neural activity in the primate prefrontal cortex during a short-term memory task. Eur. J. Neurosci. 15, 1244–1254 (2002). source
- Not checked yetcites [42] Machens, C.K., Romo, R. & Brody, C.D. Functional, but not anatomical, separation of “what” and “when” in prefrontal cortex. J. Neurosci. 30, 350–360 (2010). source
- Not checked yetcites [43] Shafi, M. et al. Variability in neuronal activity in primate cortex during working memory tasks. Neuroscience 146, 1082–1108 (2007). source
Strength:plain· not yet calibratedCausal:correlational· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The same diversity is observed across three cortical areas, suggesting that the computation of reward memory is a distributed process.”
§ A reservoir of time constants for memory traces in cortical neurons > Discussion
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesIt may state two separate claims as one.
None of the sources cited for this claim have been checked yet.
In the paper
“While anterior cingulate cortex was implicated in monitoring conflict between incompatible response processes35 or detecting performance errors36”
§ A reservoir of time constants for memory traces in cortical neurons > Discussion
What the sources say
- Not checked yetcites [35] Botvinick, M.M., Braver, T.S., Barch, D.M., Carter, C.S. & Cohen, J.D. Conflict monitoring and cognitive control. Psychol. Rev. 108, 624–652 (2001). source
- Not checked yetcites [36] Holroyd, C.B. & Coles, M.G.H. The neural basis of human error processing: reinforcement learning, dopamine and error-related negativity. Psychol. Rev. 109, 679–709 (2002). source
Strength:plain· not yet calibratedCausal:correlational· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“The heterogeneous encoding of the input allows the flexible learning of different output functions.”
§ A reservoir of time constants for memory traces in cortical neurons > Discussion
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesProbably not a substantial claim.
None of the sources cited for this claim have been checked yet.
In the paper
“Although the overall performance of monkeys was nearly optimal, their trial-by-trial decisions, locally in time, were influenced by previous rewards and actions11,19-22.”
§ A reservoir of time constants for memory traces in cortical neurons > Results > Comparison with behavior
What the sources say
- Not checked yetcites [11] Lee, D., Conroy, M.L., McGreevy, B.P. & Barraclough, D.J. Reinforcement learning and decision making in monkeys during a competitive game. Brain Res. Cogn. Brain Res. 22, 45–58 (2004). source
- Not checked yetcites [19] Seo, H. & Lee, D. Temporal filtering of reward signals in the dorsal anterior cingulate cortex during a mixed-strategy game. J. Neurosci. 27, 8366–8377 (2007). source
- Not checked yetcites [20] Seo, H., Barraclough, D.J. & Lee, D. Dynamic signals related to choices and outcomes in the dorsolateral prefrontal cortex. Cereb. Cortex 17, i110–i117 (2007). source
- Not checked yetcites [21] Seo, H., Barraclough, D.J. & Lee, D. Lateral intraparietal cortex and reinforcement learning during a mixed-strategy game. J. Neurosci. 29, 7278–7289 (2009). source
- Not checked yetcites [22] Barraclough, D.J., Conroy, M.L. & Lee, D. Prefrontal cortex and decision making in a mixed-strategy game. Nat. Neurosci. 7, 404–410 (2004). source
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesIt may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“This suggests the possibility that the behavior might be driven by a mechanism that appropriately samples from a range of timescales in a neural network, which has yet to be elucidated.”
§ A reservoir of time constants for memory traces in cortical neurons > Discussion
Strength:hedged· not yet calibratedCausal:conditional· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it.
None of the sources cited for this claim have been checked yet.
In the paper
“Those models predict that the memory of input signals is stored in a large, recurrent and heterogeneous network (reservoir), in a distributed manner, and that a desired output is obtained by a trainable combination of the response signals in the reservoir.”
§ A reservoir of time constants for memory traces in cortical neurons > Discussion
What the sources say
- Not checked yetcites [13] Maass, W., Natschläger, T. & Markram, H. Real-time computing without stable states: a new framework for neural computation based on perturbations. Neural Comput. 14, 2531–2560 (2002). source
- Not checked yetcites [14] Jaeger, H., Lukosevicius, M., Popovici, D. & Siewert, U. Optimization and applications of echo state networks with leaky-integrator neurons. Neural Netw. 20, 335–352 (2007). source
- Not checked yetcites [15] Verstraeten, D., Schrauwen, B., D′Haene, M. & Stroobandt, D. An experimental unification of reservoir computing methods. Neural Netw. 20, 391–403 (2007). source
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesProbably not a substantial claim. It may state two separate claims as one.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“Since the timescales from one (τ) and two-exponential functions (τ1, τ2) were distributed similarly (see Fig.S1a,b in the Supplementary Material), we pooled all timescales”
§ A reservoir of time constants for memory traces in cortical neurons > Results > Multiplicative memory traces in cortical neurons
Load-bearingStrength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesIt may state two separate claims as one.
- The authors state this as an assumption: The memory trace f is factorized into the epoch code g(k) and an exponential function ex(t).
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Similarly, the double exponential decay of memory may correspond to a reward prediction error signal: if the short timescale (τ1) is small enough (about one trial or smaller) then the corresponding exponential filter will respond primarily to the reward in the present trial, while the long timescale (τ2) may provide a value signal by weighting the rewards in the past few trials.”
§ A reservoir of time constants for memory traces in cortical neurons > Discussion
Strength:hedged· not yet calibratedCausal:conditional· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it.
In the paper
“It was already noted previously that a biphasic filtering in dopamine neurons might provide a reward prediction error40.”
§ A reservoir of time constants for memory traces in cortical neurons > Discussion
What the sources say
- Not checked yetcites [40] Bayer, H.M. & Glimcher, P.W. Midbrain dopamine neurons encode a quantitative reward prediction error signal. Neuron 47, 129–141 (2005). source
Strength:hedged· not yet calibratedCausal:conditional· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it.
None of the sources cited for this claim have been checked yet.
In the paper
“Prefrontal cortex plays a major role in dynamic decision processes encoding and updating values1-4.”
§ A reservoir of time constants for memory traces in cortical neurons > Discussion
What the sources say
- Not checked yetcites [1] Kable, J.W. & Glimcher, P.W. The neurobiology of decision: consensus and controversy. Neuron 63, 733–745 (2009). source
- Not checked yetcites [2] Rushworth, M.F. & Behrens, T.E. Choice, uncertainty and value in prefrontal and cingulate cortex. Nat. Neurosci. 11, 389–397 (2008). source
- Not checked yetcites [3] Wang, X.-J. Decision making in recurrent neural circuits. Neuron 60, 215–234 (2008). source
- Not checked yetcites [4] Soltani, A., Lee, D. & Wang, X.-J. Neural mechanism for stochastic behavior during a competitive game. Neural Netw. 19, 1075–1090 (2006). source
Strength:plain· not yet calibratedCausal:correlational· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it.
None of the sources cited for this claim have been checked yet.
In the paper
“recent studies have placed more emphasis on its role in representing both positive and negative values19,37.”
§ A reservoir of time constants for memory traces in cortical neurons > Discussion
What the sources say
- Not checked yetcites [19] Seo, H. & Lee, D. Temporal filtering of reward signals in the dorsal anterior cingulate cortex during a mixed-strategy game. J. Neurosci. 27, 8366–8377 (2007). source
- Not checked yetcites [37] Wallis, J.D. & Kennerley, S.W. Heterogeneous reward signals in prefrontal cortex. Curr. Opin. Neurobiol. 20, 191–198 (2010). source
Strength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it.
None of the sources cited for this claim have been checked yet.
In the paper
“Reservoir-type networks have been the subject of active research in computational neuroscience and machine learning13-15, but so far experimental support that such networks are adopted by the brain has been lacking.”
§ A reservoir of time constants for memory traces in cortical neurons > Discussion
What the sources say
- Not checked yetcites [13] Maass, W., Natschläger, T. & Markram, H. Real-time computing without stable states: a new framework for neural computation based on perturbations. Neural Comput. 14, 2531–2560 (2002). source
- Not checked yetcites [14] Jaeger, H., Lukosevicius, M., Popovici, D. & Siewert, U. Optimization and applications of echo state networks with leaky-integrator neurons. Neural Netw. 20, 335–352 (2007). source
- Not checked yetcites [15] Verstraeten, D., Schrauwen, B., D′Haene, M. & Stroobandt, D. An experimental unification of reservoir computing methods. Neural Netw. 20, 391–403 (2007). source
Load-bearingStrength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesProbably not a substantial claim. It may state two separate claims as one.
None of the sources cited for this claim have been checked yet.
In the paper
“Parietal cortex has also been implicated in decision making based on the value representation as well as on the accumulation of sensory evidence38,39.”
§ A reservoir of time constants for memory traces in cortical neurons > Discussion
What the sources say
- Not checked yetcites [38] Platt, M.L. & Glimcher, P.W. Neural correlates of decision variables in parietal cortex. Nature 400, 233–238 (1999). source
- Not checked yetcites [39] Roitman, J.D. & Shadlen, M.N. Response of neurons in the lateral intraparietal area during a combined visual discrimination reaction time task. J. Neurosci. 22, 9475–9489 (2002). source
Strength:plain· not yet calibratedCausal:correlational· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“This empirical finding is reproduced by a simple computational model, which suggests that reward filtering in the cortex involves a dynamic reservoir network operating at the critical point, leading to a power-law distribution of time constants.”
§ A reservoir of time constants for memory traces in cortical neurons > Discussion
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesIt may state two separate claims as one.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“The placement of the recording chamber was guided by magnetic resonance (MR) images, and confirmed by metal pins inserted in known anatomical locations at the end of the experiment in some animals.”
§ A reservoir of time constants for memory traces in cortical neurons > Methods > Animal preparation and electrophysiological recording
Strength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.
In the paper
“This is noteworthy, since a shift from an exploitive to an exploratory strategy as the environment becomes uncertain is often assessed by an increase in the learning rate10.”
§ A reservoir of time constants for memory traces in cortical neurons > Discussion
What the sources say
- Not checked yetcites [10] Kennerley, S.W., Walton, M.E., Behrens, T.E., Buckley, M.J. & Rushworth, M.F. Optimal decision making and the anterior cingulate cortex. Nat. Neurosci. 9, 940–947 (2006). source
Strength:plain· not yet calibratedCausal:correlational· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“This implies that the matrix product (RewT·Rew) is well conditioned, and the solution fsol minimizing the variance of the noise (or squared error) is robust and given by (M.6)fsol=(RewT⋅Rew)−1⋅RewT⋅FR”
§ A reservoir of time constants for memory traces in cortical neurons > Methods > Multiple regression analysis of memory traces
Load-bearingStrength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesIt may state two separate claims as one.
- Rests on an assumption: The sequence of rewards is nearly random.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“As a control of the fitting procedure, we reshuffled the label n in the firing rates FR(n,k), assigning to each firing rate the value of a random trial, and we repeated the entire procedure.”
§ A reservoir of time constants for memory traces in cortical neurons > Methods > Exponential memory traces and model selection
Strength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“Fitting was repeated ten times for each neuron and each model, in the search for a global minimum of the error.”
§ A reservoir of time constants for memory traces in cortical neurons > Methods > Exponential memory traces and model selection
Strength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“Any parameters resulting in unrealistic values were discarded, such as negative values of τ, τ1 or τ2, values of τ larger than 20 trials, and the absolute value of A or (A1+A2) larger than 4.”
§ A reservoir of time constants for memory traces in cortical neurons > Methods > Exponential memory traces and model selection
Load-bearingStrength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesProbably not a substantial claim.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“A diversity of time constants also means a broad range of learning rates, since the two are inversely related to each other.”
§ A reservoir of time constants for memory traces in cortical neurons > Discussion
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✗ Based: failsProbably not a substantial claim. The paper may give no reason why it holds.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“The matrix product (RewT·Rew) is approximately proportional to the identity matrix.”
§ A reservoir of time constants for memory traces in cortical neurons > Methods > Multiple regression analysis of memory traces
Strength:hedged· not yet calibratedCausal:correlational· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✗ Based: failsProbably not a substantial claim. The paper’s conclusions may not depend on it. The paper may give no reason why it holds.
- Rests on an assumption: The sequence of rewards is nearly random.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“When RewT·Rew=I, the filter is equal to the firing rate averaged over all trials, where the average is conditioned on the past rewards.”
§ A reservoir of time constants for memory traces in cortical neurons > Methods > Multiple regression analysis of memory traces
Strength:plain· not yet calibratedCausal:correlational· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“The significance of the estimation was assessed, for each session, by constructing 100 surrogate sessions, each one obtained by reshuffling of the order of trials.”
§ A reservoir of time constants for memory traces in cortical neurons > Methods > Reinforcement learning fit of behavior
Strength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“Among the three models, the selection of the appropriate one for each neuron was determined according to the Bayesian information criterion (BIC):”
§ A reservoir of time constants for memory traces in cortical neurons > Methods > Exponential memory traces and model selection
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesProbably not a substantial claim.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“Power-law distributions are unusual, as they imply a high probability for both large and small time constants.”
§ A reservoir of time constants for memory traces in cortical neurons > Discussion
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesProbably not a substantial claim.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“The physical time t depends on all indices k, n and n', because the time elapsed between different epochs and between successive trials is variable, due to the variability in the time taken by the animal to start a trial and to make a saccade to one of the two targets.”
§ A reservoir of time constants for memory traces in cortical neurons > Methods > Exponential memory traces and model selection
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesProbably not a substantial claim.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“All the neurons were recorded without pre-screening.”
§ A reservoir of time constants for memory traces in cortical neurons > Methods > Animal preparation and electrophysiological recording
Load-bearingStrength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesProbably not a substantial claim.
The paper relies on this without support.
The neural mechanism underlying switching between long and short time constants for computing reward expectation remains poorly understood.
“The neural mechanism underlying switching between long and short time constants for computing reward expectation remains poorly understood.”
The paper relies on this without support.
The neural mechanisms responsible for such timescales are still unknown.
“but the neural mechanisms responsible for such timescales are still unknown.”
The paper relies on this without support.
The matching pennies task used in this study does not necessarily require the memory of past rewards, and the optimal strategy for the monkey is to choose randomly and unpredictably.
“The matching pennies task used in this study does not necessarily require the memory of past rewards, and the optimal strategy for the monkey is to choose randomly and unpredictably.”
The authors state this as an assumption.
The effects of rewards in successive trials are additive.
“assuming that the effects of rewards in successive trials are additive.”
The authors state this as an assumption.
The memory trace f is factorized into the epoch code g(k) and an exponential function ex(t).
“we assumed that the memory trace f is factorized into the epoch code g(k) and an exponential function ex(t)”
The authors state this as an assumption.
The specific form of the input signals is not important; the results depend only on the synaptic matrix J.
“For our purposes, the specific form of the input signals is not important; the results will depend only on the synaptic matrix J.”
The authors state this as an assumption.
The connection weights (entries of matrix J) are randomly distributed.
“We assumed the connection weights (the entries of the matrix J) to be randomly distributed”
The authors state this as an assumption.
The linear model is sufficient for reproducing the observed power-law distribution of timescales under specific conditions.
“here we limit ourselves to the linear model, which is sufficient for the purpose of reproducing the observed power-law distribution of timescales under specific conditions.”
The authors state this as an assumption.
Memory traces are exponential functions ex(t) rescaled by the epoch code g(k).
“we assumed that memory traces are exponential function ex(t) rescaled by the epoch code g(k)”
The authors state this as an assumption.
The saccade reaction time of the animal in all trials is equal to 120 ms on average.
“we assumed that the saccade reaction time of the animal in all trials is equal to 120ms (average)”
The authors state this as an assumption.
The time elapsed between the initiation of two successive trials is 3.4 s (median).
“and that the time elapsed between the initiation of two successive trials is 3.4s (median).”
The paper relies on this without support.
The sequence of rewards is nearly random.
“Because the sequence of rewards is nearly random and N is large, different columns of the matrix Rew are nearly orthogonal.”
The paper relies on this without support.
In the matching pennies task, the global optimal strategy is to choose randomly and therefore does not require memory.
“Although the global optimal strategy for the matching pennies task is to choose randomly and therefore does not require memory, the animals made their decisions largely on the basis of their reward history11,19-22.”
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Similar results are obtained in tasks that manipulate the desirability of a target using different methods, for example by controlling the relative magnitude, probability or delay of its expected reward (Bernacchia et al., 2011; Louie et al., 2011; Sugrue et al., 2004; Yang and Shadlen, 2007).”
From Attention, learning, and the value of information (2012) · cites this paper as Bernacchia et al., 2011
Citations
Cites
Reinforcement Learning: An Introduction
Sutton, R. S., & Barto, A. G. (1998) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.1998.712192 · Cited by 27,225 · cited 1×
“(5) The seven unknown variables have been rewritten by a single vector f”
BackgroundReinforcement Learning: An Introduction
Sutton, R. S., & Barto, A. (2005) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.2004.842673 · Cited by 25,268
Conflict monitoring and cognitive control
Botvinick, M. M., Braver, T. S., Barch, D. M., Carter, C. S., & Cohen, J. D. (2001) · Psychological Review · doi:10.1037/0033-295x.108.3.624 · Cited by 7,351 · cited 1×
“Although anterior cingulate cortex has been implicated in monitoring conflict between incompatible response processe”
MethodHolroyd, C. B., & Coles, M. G. H. (2002) · Psychological Review · doi:10.1037/0033-295x.109.4.679 · Cited by 4,027
Maass, W., Natschläger, T., & Markram, H. (2002) · Neural Computation · doi:10.1162/089976602760407955 · Cited by 4,025 · cited 2×
“If a single value function (for a given stimulus/action) and a single reward (delivered at time zero) are considered, the solution of equation (13) can be approximated by an exponential response,”
BackgroundCortical substrates for exploratory decisions in humans
Daw, N. D., O'Doherty, J. P., Dayan, P., Seymour, B., & Dolan, R. J. (2006) · Nature · doi:10.1038/nature04766 · Cited by 2,224 · cited 1×
“If the payoffs for the same option change often and unpredictably, then rewards should be filtered on short timescales to track the fast changes in a volatile environment; in contrast, if past rewards reliably predict future ones, then they should be filtered on long timescales to exploit a stable environmen”
BackgroundLearning the value of information in an uncertain world
Behrens, T. E. J., Woolrich, M. W., Walton, M. E., & Rushworth, M. F. S. (2007) · Nature Neuroscience · doi:10.1038/nn1954 · Cited by 2,053 · cited 1×
“Any parameters resulting in unrealistic values were discarded, such as negative values of τ, τ1 or τ2, values of (6)”
BackgroundWeak pairwise correlations imply strongly correlated network states in a neural population
Schneidman, E., Berry, M. J., Segev, R., & Bialek, W. (2006) · Nature · doi:10.1038/nature04701 · Cited by 1,745 · cited 1×
“To the best of our knowledge, our results provide the first complete statistical description of a network connection matrix based on in vivo neuronal recordings of behaving animals (see also refs”
ResultNeural correlates of decision variables in parietal cortex
Platt, M. L., & Glimcher, P. W. (1999) · Nature · doi:10.1038/22268 · Cited by 1,721 · cited 1×
“Parietal cortex has also been implicated in decision making on the basis of the value representation and the accumulation of sensory evidenc”
BackgroundRoitman, J. D., & Shadlen, M. N. (2002) · The Journal of Neuroscience · doi:10.1523/jneurosci.22-21-09475.2002 · Cited by 1,589 · cited 1×
“Parietal cortex has also been implicated in decision making on the basis of the value representation and the accumulation of sensory evidenc”
Background
Flagged first, then by citations
1 of the 52 references couldn’t be matched to a known paper.
Cited by
Sorted by citations
Recent advances in physical reservoir computing: A review
Tanaka, G., Yamane, T., Héroux, J. B., Nakane, R., Kanazawa, N., Takeda, S., Numata, H., Nakano, D., & Hirose, A. (2019) · Neural Networks · doi:10.1016/j.neunet.2019.03.005 · Cited by 2,146
“…& Jaeger, 2011), the reservoir model for storing different time constants of memory traces for reward expectation in re-inforcement learning (Bernacchia, Seo, Lee, & Wang, 2011), the autonomously emerged working memory model through reward-modulated online learning rule that enables the…”
BackgroundA hierarchy of intrinsic timescales across primate cortex
Murray, J. D., Bernacchia, A., Freedman, D. J., Romo, R., Wallis, J. D., Cai, X., Padoa-Schioppa, C., Pasternak, T., Seo, H., Lee, D., et al. (2014) · Nature Neuroscience · doi:10.1038/nn.3862 · Cited by 990
Self-citationCortical tracking of hierarchical linguistic structures in connected speech
Ding, N., Melloni, L., Zhang, H., Tian, X., & Poeppel, D. (2016) · Nature Neuroscience · doi:10.1038/nn.4186 · Cited by 1,059
‘Activity-silent’ working memory in prefrontal cortex: a dynamic coding framework
Stokes, M. G. (2015) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2015.05.004 · Cited by 850
Slow cortical dynamics and the accumulation of information over long timescales
Honey, C. J., Thesen, T., Donner, T. H., Silbert, L. J., Carlson, C. E., Devinsky, O., Doyle, W. K., Rubin, N., Heeger, D. J., & Hasson, U. (2012) · Neuron · doi:10.1016/j.neuron.2012.08.011 · Cited by 532
“There is a growing realization, however, of the importance of the neural mechanisms by which information can be accumulated over time (Ben-Yakov et al., 2012; Bernacchia et al., 2011; Brody et al., 2003; Maass et al., 2007; Wang, 2002).”
A Large-Scale Circuit Mechanism for Hierarchical Dynamical Processing in the Primate Cortex
Chaudhuri, R., Knoblauch, K., Gariel, M.-A., Kennedy, H., & Wang, X. (2015) · Neuron · doi:10.1016/j.neuron.2015.09.008 · Cited by 618
Self-citationThe Geometry of Abstraction in the Hippocampus and Prefrontal Cortex
Bernardi, S., Benna, M. K., Rigotti, M., Munuera, J., Fusi, S., & Salzman, C. D. (2020) · Cell · doi:10.1016/j.cell.2020.09.031 · Cited by 528
Double dissociation of value computations in orbitofrontal and anterior cingulate neurons
Kennerley, S. W., Behrens, T. E. J., & Wallis, J. D. (2011) · Nature Neuroscience · doi:10.1038/nn.2961 · Cited by 485
Value, search, persistence and model updating in anterior cingulate cortex
Kolling, N., Wittmann, M. K., Behrens, T. E. J., Boorman, E. D., Mars, R. B., & Rushworth, M. F. S. (2016) · Nature Neuroscience · doi:10.1038/nn.4382 · Cited by 475
Lukoševičius, M., Jaeger, H., & Schrauwen, B. (2012) · Künstliche Intell. · doi:10.1007/s13218-012-0204-5 · Cited by 391
“, [45,3]).”
Perceptual Decision Making in Rodents, Monkeys, and Humans
Hanks, T. D., & Summerfield, C. (2017) · Neuron · doi:10.1016/j.neuron.2016.12.003 · Cited by 409
“Heterogeneous neural encoding may also allow information to be maintained over multiple distinct timescales (Barak et al., 2013; Bernacchia et al., 2011), potentially providing a circuit mechanism for sustained firing during information integration (Tegnér et al., 2002).”
BackgroundValuation and decision-making in frontal cortex: one or many serial or parallel systems?
Rushworth, M. F., Kolling, N., Sallet, J., & Mars, R. B. (2012) · Current Opinion in Neurobiology · doi:10.1016/j.conb.2012.04.011 · Cited by 309
Reinforcement learning in artificial and biological systems
Neftci, E. O., & Averbeck, B. B. (2019) · Nature Machine Intelligence · doi:10.1038/s42256-019-0025-4 · Cited by 268
“Flexibly updating state values is challenging because the information necessary to update values has to be remembered or stored over time 100 , and the reward outcomes have to be attributed to the appropriate preceding decisions, to solve the credit assignment problem 101 .”
BackgroundRhythmic Fluctuations in Evidence Accumulation during Decision Making in the Human Brain
Wyart, V., de Gardelle, V., Scholl, J., & Summerfield, C. (2012) · Neuron · doi:10.1016/j.neuron.2012.09.015 · Cited by 281
“The particular frequency of fluctuations in neural excitability may reflect the predominant time constants of synaptic activity in the corresponding cortical area (Wang, 2010; Bernacchia et al., 2011).”
Medial and orbital frontal cortex in decision-making and flexible behavior
Klein-Flügge, M. C., Bongioanni, A., & Rushworth, M. F. S. (2022) · Neuron · doi:10.1016/j.neuron.2022.05.022 · Cited by 244
History-dependent variability in population dynamics during evidence accumulation in cortex
Morcos, A. S., & Harvey, C. D. (2016) · Nature Neuroscience · doi:10.1038/nn.4403 · Cited by 241
“…representations of task stimuli including for non-relevant inputs 19,20 , activity-dependent processing of stimuli 35,36 , and the encoding of previous stimuli that indicates stimulus reverberation 30-33,37-43 Our findings are inconsistent with key features of winner-take-all models 2,9,10 .”
Learning the opportunity cost of time in a patch-foraging task
Constantino, S. M., & Daw, N. D. (2015) · Cognitive Affective & Behavioral Neuroscience · doi:10.3758/s13415-015-0350-y · Cited by 253
Attention, learning, and the value of information
Gottlieb, J. (2012) · Neuron · doi:10.1016/j.neuron.2012.09.034 · Cited by 241
“Similar results are obtained in tasks that manipulate the desirability of a target using different methods, for example by controlling the relative magnitude, probability or delay of its expected reward (Bernacchia et al., 2011; Louie et al., 2011; Sugrue et al., 2004; Yang and Shadlen, 2007).”
ResultA distributed, hierarchical and recurrent framework for reward-based choice
Hunt, L. T., & Hayden, B. Y. (2017) · Nature reviews. Neuroscience · doi:10.1038/nrn.2017.7 · Cited by 245
Sustained Activity Encoding Working Memories: Not Fully Distributed
Leavitt, M. L., Mendoza-Halliday, D., & Martínez-Trujillo, J. (2017) · Trends in Neurosciences · doi:10.1016/j.tins.2017.04.004 · Cited by 238
Triple dissociation of attention and decision computations across prefrontal cortex
Hunt, L. T., Malalasekera, W. M. N., de Berker, A. O., Miranda, B., Farmer, S. F., Behrens, T. E. J., & Kennerley, S. W. (2018) · Nature Neuroscience · doi:10.1038/s41593-018-0239-5 · Cited by 222
Gee, S., Ellwood, I., Patel, T., Luongo, F., Deisseroth, K., & Sohal, V. S. (2012) · The Journal of Neuroscience · doi:10.1523/jneurosci.5835-11.2012 · Cited by 214
“The quinpirole-induced afterdepolarization may also contribute to persistent reward-related activity or the modulation of firing by the past history of reward (Histed et al., 2009; Bernacchia et al., 2011).”
BackgroundInfluentialThe brain and its time: intrinsic neural timescales are key for input processing
Golesorkhi, M., Gomez-Pilar, J., Zilio, F., Berberian, N., Wolff, A., Yagoub, M. C. E., & Northoff, G. (2021) · Communications Biology · doi:10.1038/s42003-021-02483-6 · Cited by 198
Reservoir Computing Properties of Neural Dynamics in Prefrontal Cortex
Enel, P., Procyk, E., Quilodran, R., & Dominey, P. F. (2016) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1004967 · Cited by 186
“whether the system is in search or repetition phase) was the consequence of the reward, yet this short reward input had a limited influence on the reservoir dynamics due to fading memory [39] as shown by the auto-correlation analysis.”
Reward-based training of recurrent neural networks for cognitive and value-based tasks
Song, H., Yang, G. R., & Wang, X. (2017) · eLife · doi:10.7554/elife.21492 · Cited by 175
“Indeed, these may implement the “reservoir of time constants” observed experimentally (Bernacchia et al., 2011).”
BackgroundSelf-citationDecision Making: From Neuroscience to Psychiatry
Lee, D. (2013) · Neuron · doi:10.1016/j.neuron.2013.04.008 · Cited by 178
“Learning rates can be adjusted according to the stability of the decision-making environment (Behrens et al., 2007; Bernacchia et al., 2011).”
Self-citationDwelling quietly in the rich club: brain network determinants of slow cortical fluctuations
Gollo, L. L., Zalesky, A., Hutchison, R. M., van den Heuvel, M. P., & Breakspear, M. J. (2015) · Philosophical Transactions of the Royal Society B Biological Sciences · doi:10.1098/rstb.2014.0165 · Cited by 189
“A recent compilation of the timescales of intrinsic fluctuations in spiking activity in primates reveals a hierarchical ordering, with sensory and prefrontal areas exhibiting shorter and longer timescales, respectively (34, 35).”
BackgroundJiang, J., Beck, J., Heller, K., & Egner, T. (2015) · Nature Communications · doi:10.1038/ncomms9165 · Cited by 160
Chau, B. K. H., Sallet, J., Papageorgiou, G. K., Noonan, M. P., Bell, A. H., Walton, M. E., & Rushworth, M. F. S. (2015) · Neuron · doi:10.1016/j.neuron.2015.08.018 · Cited by 159
“…over multiple timescales—a short timescale that contains information about the most recent reward only and over long timescales that integrate over both the recent and remote histories of reward events (Bernacchia et al., 2011; M. Wittmann, B.K.H.C., M.F.S.R., and colleagues, unpublished data).”
The Timescale of Perceptual Evidence Integration Can Be Adapted to the Environment
Ossmy, O., Moran, R., Pfeffer, T. A., Tsetsos, K., Usher, M., & Donner, T. H. (2013) · Current Biology · doi:10.1016/j.cub.2013.04.039 · Cited by 168
“The notion of leaky integration is inspired by fundamental principles of neural computation [4, 18], and it is consistent with neurophysiological data suggesting a reservoir of time constants in the cerebral cortex [21, 22].”
BackgroundChen, J., Honey, C. J., Simony, E., Arcaro, M. J., Norman, K. A., & Hasson, U. (2016) · Cerebral Cortex · doi:10.1093/cercor/bhv155 · Cited by 135
“Within a given cortical region, there may be neural subpopulations with differing time constants expressed in a task-dependent manner (Bernacchia et al. 2011), and interregional interactions could also constrain the timescales of brain regions (Nir et al.”
BackgroundThe hippocampus, time, and memory across scales.
Howard, M. W., & Eichenbaum, H. (2013) · Journal of Experimental Psychology General · doi:10.1037/a0033621 · Cited by 163
Atypical intrinsic neural timescale in autism
Watanabe, T., Rees, G., & Masuda, N. (2019) · eLife · doi:10.7554/elife.42256 · Cited by 150
“, 2012), stable memory processing (Hasson et al., 2015; Murray et al., 2014; Bernacchia et al., 2011), and accurate decision making (Cavanagh et al.”
BackgroundMeister, M. L. R., Hennig, J. A., & Huk, A. C. (2013) · Journal of Neuroscience · doi:10.1523/jneurosci.2984-12.2013 · Cited by 143
“Perhaps this idiosyncrasy and diversity provide LIP with computational flexibility, providing a bank of transformations of sensory and motor signals that can be used as appropriate to the task (Bernacchia et al., 2011).”
BackgroundReconciling persistent and dynamic hypotheses of working memory coding in prefrontal cortex
Cavanagh, S. E., Towers, J. P., Wallis, J. D., Hunt, L. T., & Kennerley, S. W. (2018) · Nature Communications · doi:10.1038/s41467-018-05873-3 · Cited by 167
How Recent History Affects Perception: The Normative Approach and Its Heuristic Approximation
Raviv, O., Ahissar, M., & Loewenstein, Y. (2012) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1002731 · Cited by 148
“The dependence of g on the level of noise can be accounted for in the framework of the implicit memory model if we assume that the computation of M(t), which incorporates the prior knowledge with the response to the 1 stimulus, is carried out simultaneously by several neurons, or populations of neurons, which are char…”
BackgroundWittmann, M. K., Kolling, N., Akaishi, R., Chau, B. K. H., Brown, J. W., Nelissen, N., & Rushworth, M. F. S. (2016) · Nature Communications · doi:10.1038/ncomms12327 · Cited by 146
Beyond Trial-Based Paradigms: Continuous Behavior, Ongoing Neural Activity, and Natural Stimuli
Huk, A. C., Bonnen, K., & He, B. J. (2018) · Journal of Neuroscience · doi:10.1523/jneurosci.1920-17.2018 · Cited by 149
“, its power) is often well approximated as 1/f (where typically ranges from 0 to 2), implying that sensory systems typically exhibit a wide set of time scales and temporal dynamics (Bernacchia et al., 2011; Murray et al., 2014; Hasson et al., 2015), quite distinct from the usual single-frequency (or otherwise tightly…”
BackgroundNeurons in Dorsal Anterior Cingulate Cortex Signal Postdecisional Variables in a Foraging Task
Blanchard, T. C., & Hayden, B. Y. (2014) · Journal of Neuroscience · doi:10.1523/jneurosci.3151-13.2014 · Cited by 149
“There is a great deal of information on its role in monitoring and adjustment (Ito et al., 2003; Rushworth et al., 2004; Quilodran et al., 2008; Alexander and Brown, 2011; Bernacchia et al., 2011; Hayden et al., 2011a,b; Shenhav et al., 2013).”
BackgroundA Scale-Invariant Internal Representation of Time
Shankar, K. H., & Howard, M. W. (2011) · Neural Computation · doi:10.1162/neco_a_00212 · Cited by 140
“Neurons in prefrontal, parietal, and cingulate cortices have been found to satisfy exponential decay with time constants ranging from hundreds of milliseconds to tens of seconds (Bernacchia, Seo, Lee, & Wang, 2011 ).”
BackgroundHörzer, G. M., Legenstein, R. A., & Maass, W. (2012) · Cerebral Cortex · doi:10.1093/cercor/bhs348 · Cited by 143
On the equivalence of Hopfield networks and Boltzmann Machines
Adriano Barra, A., Bernacchia, A., Santucci, E., & Contucci, P. (2012) · Neural Networks · doi:10.1016/j.neunet.2012.06.003 · Cited by 151
“On this purpose, the brain is believed to develop an internal model of the visual world, to estimate the probability and respond to the occurrence of various events [6],[7].”
BackgroundSelf-citationTemporal hierarchy of intrinsic neural timescales converges with spatial core-periphery organization
Golesorkhi, M., Gómez‐Pilar, J., Tumati, S., Fraser, M., & Northoff, G. (2021) · Communications Biology · doi:10.1038/s42003-021-01785-z · Cited by 132
Neural dynamics and circuit mechanisms of decision-making
Wang, X. (2012) · Current Opinion in Neurobiology · doi:10.1016/j.conb.2012.08.006 · Cited by 140
Self-citationPerspectives on classical controversies about the motor cortex
Omrani, M., Kaufman, M. T., Hatsopoulos, N. G., & Cheney, P. D. (2017) · Journal of Neurophysiology · doi:10.1152/jn.00795.2016 · Cited by 139
Intrinsic neuronal dynamics predict distinct functional roles during working memory
Wasmuht, D., Spaak, E., Buschman, T. J., Miller, E. K., & Stokes, M. G. (2018) · Nature Communications · doi:10.1038/s41467-018-05961-4 · Cited by 136
Dissociating hippocampal and striatal contributions to sequential prediction learning
Bornstein, A. M., & Daw, N. D. (2012) · European Journal of Neuroscience · doi:10.1111/j.1460-9568.2011.07920.x · Cited by 123
“2008), and even between different neurons within an area (Bernacchia et al. 2011).”
Multiple time-scales of decision-making in the hippocampus and prefrontal cortex
Tang, W., Shin, J. D., & Jadhav, S. P. (2021) · eLife · doi:10.7554/elife.66227 · Cited by 142
“Such a mechanism is broadly supported by recent findings of rapid shifts in activity patterns during decision-making (Bernacchia et al., 2011; Durstewitz et al., 2010; Karlsson et al., 2012; Latimer et al., 2015; Rich and Wallis, 2016; Sadacca et al., 2016), including discrete gamma oscillatory bursts in PFC underlyin…”
BackgroundA Novel Neural Prediction Error Found in Anterior Cingulate Cortex Ensembles
Hyman, J. M., Holroyd, C. B., & Seamans, J. K. (2017) · Neuron · doi:10.1016/j.neuron.2017.06.021 · Cited by 124
“Specifically, the recent history of both reward (Seo and Lee, 2007; Bernacchia et al., 2011; Kawai et al., 2015) and errors (Narayanan and Laubach, 2008; Bekolay et al.”
Prefrontal deep projection neurons enable cognitive flexibility via persistent feedback monitoring
Spellman, T., Svei, M., Kaminsky, J. C., Manzano-Nieves, G., & Liston, C. M. (2021) · Cell · doi:10.1016/j.cell.2021.03.047 · Cited by 123
Zilio, F., Gómez‐Pilar, J., Cao, S., Zhang, J., Zang, D., Qi, Z., Tan, J., Hiromi, T., Wu, X., Fogel, S. M., et al. (2020) · NeuroImage · doi:10.1016/j.neuroimage.2020.117579 · Cited by 110
Multiple timescales of neural dynamics and integration of task-relevant signals across cortex
Spitmaan, M., Seo, H., Lee, D., & Soltani, A. (2020) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2005993117 · Cited by 112
Self-citationA Diversity of Intrinsic Timescales Underlie Neural Computations
Cavanagh, S. E., Hunt, L. T., & Kennerley, S. W. (2020) · Frontiers in Neural Circuits · doi:10.3389/fncir.2020.615626 · Cited by 118
Neural correlates of strategic reasoning during competitive games
Seo, H., Cai, X., Donahue, C. H., & Lee, D. (2014) · Science · doi:10.1126/science.1256254 · Cited by 108
Self-citationA Mixture of Delta-Rules Approximation to Bayesian Inference in Change-Point Problems
Wilson, R. C., Nassar, M. R., & Gold, J. I. (2013) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1003150 · Cited by 103
“Complementary work at the neural level found a reservoir of many different learning rates in three brain regions (anterior cingulate cortex, dorsolateral prefrontal cortex, and the lateral intraparietal area) of monkeys performing a competitive game [34].”
BackgroundNeural Basis of Strategic Decision Making
Lee, D., & Seo, H. (2015) · Trends in Neurosciences · doi:10.1016/j.tins.2015.11.002 · Cited by 114
Self-citationSequential Firing Codes for Time in Rodent Medial Prefrontal Cortex
Tiganj, Z., Jung, M. W., Kim, J., & Howard, M. W. (2016) · Cerebral Cortex · doi:10.1093/cercor/bhw336 · Cited by 106
Brain signatures of a multiscale process of sequence learning in humans
Maheu, M., Dehaene, S., & Meyniel, F. (2019) · eLife · doi:10.7554/elife.41541 · Cited by 103
“Our finding is, however, much in line with other studies suggesting that the brain may entertain several statistical estimates computed over different timescales (Ulanovsky et al., 2004; Kiebel et al., 2008; Bernacchia et al., 2011; Ossmy et al., 2013; Meder et al., 2017; Scott et al., 2017; Runyan et al., 2017).”
ResultMetaplasticity as a Neural Substrate for Adaptive Learning and Choice under Uncertainty
Farashahi, S. G., Donahue, C. H., Khorsand, P., Seo, H., Lee, D., & Soltani, A. (2017) · Neuron · doi:10.1016/j.neuron.2017.03.044 · Cited by 106
“The multiple time constants for updates and hierarchical structures in aforementioned models might rely on the observed reservoir of reward memory time constants (Bernacchia et al., 2011) or could be a reflection of metaplasticity proposed here, since metaplastic synapses contain different internal timescales.”
Self-citationA sensorimotor view of verbal working memory
Buchsbaum, B. R., & D’Esposito, M. (2018) · Cortex · doi:10.1016/j.cortex.2018.11.010 · Cited by 108
Emergence of Dynamic Memory Traces in Cortical Microcircuit Models through STDP
Klampfl, S., & Maass, W. (2013) · Journal of Neuroscience · doi:10.1523/jneurosci.5044-12.2013 · Cited by 96
“This prediction has been confirmed by several experimental studies (Nikolić et al., 2009; Bernacchia et al., 2011; Klampfl et al., 2012).”
ResultDynamic routing of task-relevant signals for decision making in dorsolateral prefrontal cortex
Donahue, C. H., & Lee, D. (2015) · Nature Neuroscience · doi:10.1038/nn.3918 · Cited by 104
“Decision-making in natural environments often requires animals to integrate various types of information over multiple timescales 38–40 .”
Self-citationPrinciples of Temporal Processing Across the Cortical Hierarchy
Himberger, K. D., Chien, H.-Y. S., & Honey, C. J. (2018) · Neuroscience · doi:10.1016/j.neuroscience.2018.04.030 · Cited by 91
Constructing and Forgetting Temporal Context in the Human Cerebral Cortex
Chien, H.-Y. S., & Honey, C. J. (2020) · Neuron · doi:10.1016/j.neuron.2020.02.013 · Cited by 90
Robot-Aided Neurorehabilitation: A Pediatric Robot for Ankle Rehabilitation
Michmizos, K. P., Rossi, S., Castelli, E., Cappa, P., & Krebs, H. I. (2015) · IEEE Transactions on Neural Systems and Rehabilitation Engineering · doi:10.1109/tnsre.2015.2410773 · Cited by 92
“[51].”
Cavanagh, S. E., Wallis, J. D., Kennerley, S. W., & Hunt, L. T. (2016) · eLife · doi:10.7554/elife.18937 · Cited by 94
“These include the encoding or integrating of reward, planning and/or choice information across multiple trials (Matsumoto et al., 2007; Seo and Lee, 2007; Bernacchia et al., 2011; Hayden et al., 2011; Kennerley et al., 2011; Stoll et al., 2016).”
BackgroundThe self and its prolonged intrinsic neural timescale in schizophrenia
Northoff, G., Sandsten, K. E., Nordgaard, J., Kjaer, T. W., & Parnas, J. (2021) · Schizophrenia Bulletin · doi:10.1093/schbul/sbaa083 · Cited by 91
Dopamine transients follow a striatal gradient of reward time horizons
Mohebi, A., Wei, W., Pelattini, L., Kim, K., & Berke, J. D. (2024) · Nature Neuroscience · doi:10.1038/s41593-023-01566-3 · Cited by 105
“Expectations of future rewards can reflect past reward history over a range of possible timescales 44 .”
BackgroundHow can a Bayesian approach inform neuroscience?
O’Reilly, J. X., Jbabdi, S., & Behrens, T. E. J. (2012) · European Journal of Neuroscience · doi:10.1111/j.1460-9568.2012.08010.x · Cited by 98
Murakami, M., Shteingart, H., Loewenstein, Y., & Mainen, Z. F. (2017) · Neuron · doi:10.1016/j.neuron.2017.04.040 · Cited by 92
The anterior cingulate cortex directs exploration of alternative strategies
Tervo, D. G. R., Kuleshova, E., Manakov, M., Proskurin, M., Karlsson, M., Lustig, A., Behnam, R., & Karpova, A. Y. (2021) · Neuron · doi:10.1016/j.neuron.2021.03.028 · Cited by 97
Global reward state affects learning and activity in raphe nucleus and anterior insula in monkeys
Wittmann, M. K., Fouragnan, E. F., Folloni, D., Klein-Flügge, M. C., Chau, B. K. H., Khamassi, M., & Rushworth, M. F. S. (2020) · Nature Communications · doi:10.1038/s41467-020-17343-w · Cited by 88
“This is consistent with neurophysiological recordings demonstrating reward coding with multiple time scales in anterior cingulate cortex 33 – 36 .”
Meder, D., Kolling, N., Verhagen, L., Wittmann, M. K., Scholl, J., Madsen, K. H., Hulme, O. J., Behrens, T. E. J., & Rushworth, M. F. S. (2017) · Nature Communications · doi:10.1038/s41467-017-02169-w · Cited by 86
“For example, if regressor 20 had the highest beta-value amongst the 30 LR regressors, that voxel would be assigned a LR of 20 (2) The change in the best-fitting LR between the stable and the volatile subsessions (measured as best-fitting LR in the stable sub-session minus the best-fitting LR in the volatile sub-sessio…”
Zeraati, R., Shi, Y.-L., Steinmetz, N. A., Gieselmann, M. A., Thiele, A., Moore, T., Levina, A., & Engel, T. A. (2023) · Nature Communications · doi:10.1038/s41467-023-37613-7 · Cited by 98
“These stimulus or task-induced dynamics are characterized by the timescales of trial-average neural response 18,19 or encoding various task events over multiple trials 12,20 .”
BackgroundA diversity of localized timescales in network activity
Chaudhuri, R., Bernacchia, A., & Wang, X. (2014) · eLife · doi:10.7554/elife.01239 · Cited by 80
“Notable recent examples include the timescales of reward integration in the macaque cortex (Bernacchia et al., 2011), and the decay of neural firing rates in the zebrafish (Miri et al.”
BackgroundSelf-citationTrudel, N., Scholl, J., Klein-Flügge, M. C., Fouragnan, E., Tankelevitch, L., Wittmann, M. K., & Rushworth, M. F. S. (2020) · Nature Human Behaviour · doi:10.1038/s41562-020-0929-3 · Cited by 74
Evolutionary aspects of reservoir computing
Seoane, L. F. (2019) · Philosophical Transactions of the Royal Society B Biological Sciences · doi:10.1098/rstb.2018.0377 · Cited by 74
“We have reviewed some evidence suggesting that it is exploited by several neural circuits [115, 117–121, 124– 127 The two main advantages brought about by RC are: i) very cheap learning and ii) a startling capability for parallel processing.”
BackgroundMethodAmemori, K., Gibb, L., & Graybiel, A. M. (2011) · Frontiers in Human Neuroscience · doi:10.3389/fnhum.2011.00047 · Cited by 76
“With regard to learning in a changing environment, it is of particular interest to compare modular RL with RL having an adjustable learning rate (Behrens et al., 2007; Bernacchia et al., 2011).”
BackgroundMorita, K., Morishima, M., Sakai, K., & Kawaguchi, Y. (2012) · Trends in Neurosciences · doi:10.1016/j.tins.2012.04.009 · Cited by 73
Short-term plasticity as a neural mechanism supporting memory and attentional functions
Jääskeläinen, I. P., Ahveninen, J., Andermann, M. L., Belliveau, J. W., Raij, T., & Sams, M. E. (2011) · Brain Research · doi:10.1016/j.brainres.2011.09.031 · Cited by 70
“…in monkey prefrontal cortex during the delay period in working memory tasks (Fuster and Alexander, 1971; Rainer et al., 1998) and, recently, cells that have been observed to store a continuum of reward memory traces in prefrontal, cingulate and parietal cortical areas (Bernacchia et al., 2011).”
BackgroundVolatility Facilitates Value Updating in the Prefrontal Cortex
Massi, B., Donahue, C. H., & Lee, D. (2018) · Neuron · doi:10.1016/j.neuron.2018.06.033 · Cited by 64
“In particular, the results in the present study suggest that environmental volatility might alter the timescale of reward representation in the OFC, whichmay facilitate reinforcement learning (Bernacchia et al., 2011;Murray et al., 2014b;Meder et al., 2017).”
Self-citationTimescales of cognition in the brain
Soltani, A., Murray, J. D., Seo, H., & Lee, D. (2021) · Current Opinion in Behavioral Sciences · doi:10.1016/j.cobeha.2021.03.003 · Cited by 60
Self-citation(Reinforcement?) Learning to forage optimally
Kolling, N., & Akam, T. (2017) · Current Opinion in Neurobiology · doi:10.1016/j.conb.2017.08.008 · Cited by 61
The Cortical Maps of Hierarchical Linguistic Structures during Speech Perception
Sheng, J., Zheng, L., Lyu, B., Cen, Z., Qin, L., Tan, L. H., Huang, M., Ding, N., & Gao, J. (2018) · Cerebral Cortex · doi:10.1093/cercor/bhy191 · Cited by 68
Hart, E. E., Blair, G. J., O’Dell, T. J., Blair, H. T., & Izquierdo, A. (2020) · Journal of Neuroscience · doi:10.1523/jneurosci.2548-19.2020 · Cited by 60
“…track trial-by-trial outcomes of choices (Procyk et al., 2000; Shidara and Richmond, 2002; Seo and Lee, 2007; Akam et al., 2017), reward history (Bernacchia et al., 2011), reward prediction errors (Hayden et al., 2009; Kennerley et al., 2011; Hyman et al., 2017), and counterfactual options…”
BackgroundCortical Signals for Rewarded Actions and Strategic Exploration
Donahue, C. H., Seo, H., & Lee, D. (2013) · Neuron · doi:10.1016/j.neuron.2013.07.040 · Cited by 56
Self-citationIigaya, K., Ahmadian, Y., Sugrue, L. P., Corrado, G. S., Loewenstein, Y., Newsome, W. T., & Fusi, S. (2019) · Nature Communications · doi:10.1038/s41467-019-09388-3 · Cited by 58
Memory as Perception of the Past: Compressed Time inMind and Brain
Howard, M. W. (2018) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2017.11.004 · Cited by 58
Cortical computations via metastable activity
Camera, G. L., Fontanini, A., & Mazzucato, L. (2019) · Current Opinion in Neurobiology · doi:10.1016/j.conb.2019.06.007 · Cited by 64
“These circuits can be interpreted as dynamical systems with many coupled degrees of freedom and therefore capable of generating a wealth of dynamical behaviors on very diverse timescales [1].”
BackgroundOngoing Brain Activity and Its Role in Cognition: Dual versus Baseline Models
Northoff, G., Vatansever, D., Scalabrini, A., & Stamatakis, E. A. (2022) · The Neuroscientist · doi:10.1177/10738584221081752 · Cited by 56
A neural microcircuit model for a scalable scale‐invariant representation of time
Liu, Y., Tiganj, Z., Hasselmo, M. E., & Howard, M. W. (2018) · Hippocampus · doi:10.1002/hipo.22994 · Cited by 52
“In addtion, there are in vivo recordings showing a spectrum of timescales across cortex (Bernacchia, Seo, Lee, & Wang, 2011; Murray et al., 2014; Meister & Buffalo, 2017; Tsao et al., 2017).”
BackgroundFluid network dynamics in the prefrontal cortex during multiple strategy switching
Malagon‐Vina, H., Ciocchi, S., Passecker, J., Dorffner, G., & Klausberger, T. (2018) · Nature Communications · doi:10.1038/s41467-017-02764-x · Cited by 58
Stable and dynamic representations of value in the prefrontal cortex
Enel, P., Wallis, J. D., & Rich, E. L. (2020) · eLife · doi:10.7554/elife.54313 · Cited by 54
“, 2013), a multitude of dynamical regimes can provide a broad range of dynamics from which some can be selected for their relevance to the task (Bernacchia et al., 2011).”
BackgroundTowards a neuro-computational account of prism adaptation
Petitet, P., O’Reilly, J. X., & O’Shea, J. (2017) · Neuropsychologia · doi:10.1016/j.neuropsychologia.2017.12.021 · Cited by 53
“In a di ff erent fi eld, recent advances in biophysical models of neural networks also support the idea that di ff erent brain networks process information at di ff erent timescales (Fusi et al., 2005; Bernacchia et al., 2011; Chaudhuri et al., 2015).”
ResultHierarchical timescales in the neocortex: Mathematical mechanism and biological insights
Li, S., & Wang, X. (2022) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2110274119 · Cited by 51
“Furthermore, within a brain region time constants are heterogeneous across individual neurons (Bernacchia et al., 2011; Cavanagh et al., 2020).”
BackgroundSelf-citationBrown, J. W., & Alexander, W. H. (2017) · Journal of Cognitive Neuroscience · doi:10.1162/jocn_a_01140 · Cited by 52
Monosov, I. E., & Rushworth, M. F. S. (2021) · Neuropsychopharmacology · doi:10.1038/s41386-021-01079-2 · Cited by 53
Foraging with the frontal cortex: A cross-species evaluation of reward-guided behavior
Rudebeck, P. H., & Izquierdo, A. (2021) · Neuropsychopharmacology · doi:10.1038/s41386-021-01140-0 · Cited by 55
“Notably, neurons in this area track trial-by-trial outcomes of choices [159–164], reward history [165], reward prediction errors, RPEs [100–102, 162], and represents better, unobtained rewards [166].”
BackgroundDistributional reinforcement learning in prefrontal cortex
Müller, T., Butler, J. L., Veselič, S., Miranda, B., Wallis, J. D., Dayan, P., Behrens, T. E. J., Kurth-Nelson, Z. L., & Kennerley, S. W. (2024) · Nature Neuroscience · doi:10.1038/s41593-023-01535-w · Cited by 59
“The PFC is engaged during risk-sensitive decisions 17 and encodes a diversity of learning-and decision-related computations 8,18,19 , including RPEs 8,20 , temporal scales and learning rates 8,21–23 .”
BackgroundMultiple timescales of sensory-evidence accumulation across the dorsal cortex
Pinto, L., Tank, D. W., & Brody, C. D. (2022) · eLife · doi:10.7554/elife.70263 · Cited by 52
“At face value, however, the findings could be a reflection of the fact that diverse timescales exist at the level of individual neurons within each region (Bernacchia et al., 2011; Cavanagh et al., 2020; Scott et al., 2017; Spitmaan et al., 2020; Wasmuht et al., 2018).”
Background
Metadata
- Authors
- Bernacchia, A., Seo, H., Lee, D., & Wang, X.-J.
- Affiliations
- Yale University (US)
- Corresponding author
- Wang, Xiao-Jing
- Venue
- Nature Neuroscience
- Published
- 2011-02-13
- Volume
- 14
- Issue
- 3
- Pages
- 366-372
- ISSN
- 1097-6256 (ISSN-L)
- 1546-1726
- PMC
- PMC3079398
- OpenAlex
- W2139630667
- Semantic Scholar
- 87fa62529ba1b9dd0ff3d6e9fcd5b62a26363247
- Cited by
- 292
- References
- 52
- Citations by year
- 2026: 14 · 2025: 13 · 2024: 19 · 2023: 15 · 2022: 23 · 2021: 25 · 2020: 28 · 2019: 19 · 2018: 21 · 2017: 21 · 2016: 16 · 2015: 20 · 2014: 18 · 2013: 15 · 2012: 16
- FWCI
- 9.41
- Type
- journal article
- Language
- en
- Open access
- green
- www.ncbi.nlm.nih.gov
- Licence
- www.springer.com/tdm (tdm)
- Locations
- Nature Neuroscience (publishedVersion)
- PubMed (publishedVersion)
- PubMed Central (acceptedVersion)
- Funders
- National Institute of Mental Health (R01MH062349, R01MH073246)
- National Institute on Drug Abuse (DA029330, R01DA029330)
- Topics
- Life Sciences › Neuroscience › Cognitive Neuroscience › Neural and Behavioral Psychology Studies
- Life Sciences › Neuroscience › Cognitive Neuroscience › Neural dynamics and brain function
- Life Sciences › Neuroscience › Cognitive Neuroscience › Memory and Neural Mechanisms
- Keywords
- reward memory, power-law distribution, reinforcement learning, decision-making, prefrontal cortex, cingulate cortex, parietal cortex, reward anticipation, decision time, cortical neurons
- MeSH terms
- Animals, Haplorhini, Cerebral Cortex, Decision Making, Humans, Memory, Nerve Net, Neurons, Neuropsychological Tests, Reinforcement, Psychology, Reward, Time Factors
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
A language model drew the claims from the paper on 2026-10-10, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-10-10 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
