Memory Capacity for Sequences in a Recurrent Network with Biological Constraints
The DOI matches its Crossref record; 12 of the 12 references it cites were found in a registry or an index.
Abstract
The CA3 region of the hippocampus is a recurrent neural network that is essential for the storage and replay of sequences of patterns that represent behavioral events. Here we present a theoretical framework to calculate a sparsely connected network's capacity to store such sequences. As in CA3, only a limited subset of neurons in the network is active at any one time, pattern retrieval is subject to error, and the resources for plasticity are limited. Our analysis combines an analytical mean field approach, stochastic dynamics, and cellular simulations of a time-discrete McCulloch-Pitts network with binary synapses. To maximize the number of sequences that can be stored in the network, we concurrently optimize the number of active neurons, that is, pattern size, and the firing threshold. We find that for one-step associations (i.e., minimal sequences), the optimal pattern size is inversely proportional to the mean connectivity c, whereas the optimal firing threshold is independent of the connectivity. If the number of synapses per neuron is fixed, the maximum number P of stored sequences in a sufficiently large, nonmodular network is independent of its number N of cells. On the other hand, if the number of synapses scales as the network size to the power of 3/2, the number of sequences P is proportional to N. In other words, sequential memory is scalable. Furthermore, we find that there is an optimal ratio r between silent and nonsilent synapses at which the storage capacity alpha = P//[c(1 + r)N] assumes a maximum. For long sequences, the capacity of sequential memory is about one order of magnitude below the capacity for minimal sequences, but otherwise behaves similar to the case of minimal sequences. In a biologically inspired scenario, the information content per synapse is far below theoretical optimality, suggesting that the brain trades off error tolerance against information content in encoding sequential memories.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Neural networks and physical systems with emergent collective computational abilities.
Hopfield, J. J. (1982) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.79.8.2554 · Cited by 19,270 · cited 1×
“…& Longuet-Higgins, 1969; Gardner, 1987; Nadal & Toulouse, 1990; Brunel, Nadal, & Toulouse, 1992) and recurrent networks that describe memories as attractors of a time-discrete dynamics (Little, 1974; Hopfield, 1982; Amit, Gutfreund, & Sompolinsky, 1987; Golomb, Rubin, & Sompolinsky, 1990).”
A logical calculus of the ideas immanent in nervous activity
McCulloch, W. S., & Pitts, W. (1943) · The Bulletin of Mathematical Biophysics · doi:10.1007/bf02478259 · Cited by 18,566
Gamma (40-100 Hz) oscillation in the hippocampus of the behaving rat
Bragin, A., Jando, G., Nadasdy, Z., Hetke, J., Wise, K., & Buzsaki, G. (1995) · The Journal of Neuroscience · doi:10.1523/jneurosci.15-01-00047.1995 · Cited by 1,527
Memory of sequential experience in the hippocampus during slow wave sleep
Lee, A. K., & Wilson, M. A. (2002) · Neuron · doi:10.1016/s0896-6273(02)01096-6 · Cited by 1,296
Evidence for silent synapses: Implications for the expression of LTP
Isaac, J. T. R., Nicoll, R. A., & Malenka, R. C. (1995) · Neuron · doi:10.1016/0896-6273(95)90046-2 · Cited by 1,233
Non-Holographic Associative Memory
WILLSHAW, D. J., BUNEMAN, O. P., & LONGUET-HIGGINS, H. C. (1969) · Nature · doi:10.1038/222960a0 · Cited by 979 · cited 2×
“In this letter, we tackle the problem of how many sequences can be stored in a recurrent neuronal network such that their replay can be triggered by an activation of adequate cue patterns.”
BackgroundReplay and Time Compression of Recurring Spike Sequences in the Hippocampus
Nádasdy, Z., Hirase, H., Czurkó, A., Csicsvari, J., & Buzsáki, G. (1999) · The Journal of Neuroscience · doi:10.1523/jneurosci.19-21-09497.1999 · Cited by 932 · cited 2×
“The latter are supposed to be generated by CA3 pyramidal cells (Behrens, van den Boom, de Hoz, Friedman, & Heinemann, 2005) and may reflect sequence replay (Wilson & McNaughton, 1994; Nádasdy et al., 1999; Lee & Wilson, 2002) occurring in timeslices of about 5 ms.”
BackgroundCritical role of the hippocampus in memory for sequences of events
Fortin, N. J., Agster, K. L., & Eichenbaum, H. B. (2002) · Nature Neuroscience · doi:10.1038/nn834 · Cited by 881 · cited 2×
“LETTER Communicated by Jean-Pierre Nadal Memory Capacity for Sequences in a Recurrent Network with Biological Constraints Christian Leibold c.leibold@biologie.hu-berlin.de Institute for Theoretical Biology, Humboldt-Universität zu Berlin, Germany, and Neuroscience Research Center, Charité, Medical Faculty of Berlin,…”
BackgroundNeuromodulation: acetylcholine and memory consolidation
Hasselmo, M. E. (1999) · Trends in Cognitive Sciences · doi:10.1016/s1364-6613(99)01365-0 · Cited by 809
NMDA receptors, place cells and hippocampal spatial memory
Nakazawa, K., McHugh, T. J., Wilson, M. A., & Tonegawa, S. (2004) · Nature Reviews Neuroscience · doi:10.1038/nrn1385 · Cited by 619
Cited by
Rasch, B., & Born, J. (2013) · Physiological Reviews · doi:10.1152/physrev.00032.2012 · Cited by 2,783
“In addition, unstructured recurrent networks have been demonstrated to face essential capacity constraints (723).”
BackgroundWhy Neurons Have Thousands of Synapses, a Theory of Sequence Memory in Neocortex
Hawkins, J., & Ahmad, S. (2016) · Frontiers in Neural Circuits · doi:10.3389/fncir.2016.00023 · Cited by 396
Theta sequences are essential for internally generated hippocampal firing fields
Wang, Y., Romani, S., Lustig, B., Leonardo, A., & Pastalkova, E. (2015) · Nature Neuroscience · doi:10.1038/nn.3904 · Cited by 311
Hippocampal Replay Captures the Unique Topological Structure of a Novel Environment
Wu, X., & Foster, D. J. (2014) · The Journal of Neuroscience · doi:10.1523/jneurosci.3414-13.2014 · Cited by 201
Coherent phasic excitation during hippocampal ripples
Maier, N., Tejero-Cantero, Á., Dorrn, A. L., Winterer, J., Beed, P. S., Morris, G., Kempter, R., Poulet, J. F. A., Leibold, C., & Schmitz, D. (2011) · Neuron · doi:10.1016/j.neuron.2011.08.016 · Cited by 126
Self-citation
Metadata
- Authors
- Leibold, C., & Kempter, R.
- Venue
- Neural Computation
- Publisher
- MIT Press
- Published
- 2006-04-01
- Volume
- 18
- Issue
- 4
- Pages
- 904-941
- ISSN
- 0899-7667 (ISSN-L)
- 1530-888X
- OpenAlex
- W4240553687
- Cited by
- 58
- References
- 12
- FWCI
- 0.77
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Memory Capacity for Sequences in a Recurrent Network with Biological Constraints. https://app.cochat.ai/papers/10.1162/089976606775774714
- The paper
- Leibold, C., & Kempter, R. (2006). Memory Capacity for Sequences in a Recurrent Network with Biological Constraints. Neural Computation. https://doi.org/10.1162/089976606775774714
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.