Learning, Memory, and the Role of Neural Network Architecture
Open accessThe DOI matches its Crossref record; 21 of the 21 references it cites were found in a registry or an index.
Abstract
The performance of information processing systems, from artificial neural networks to natural neuronal ensembles, depends heavily on the underlying system architecture. In this study, we compare the performance of parallel and layered network architectures during sequential tasks that require both acquisition and retention of information, thereby identifying tradeoffs between learning and memory processes. During the task of supervised, sequential function approximation, networks produce and adapt representations of external information. Performance is evaluated by statistically analyzing the error in these representations while varying the initial network state, the structure of the external information, and the time given to learn the information. We link performance to complexity in network architecture by characterizing local error landscape curvature. We find that variations in error landscape structure give rise to tradeoffs in performance; these include the ability of the network to maximize accuracy versus minimize inaccuracy and produce specific versus generalizable representations of information. Parallel networks generate smooth error landscapes with deep, narrow minima, enabling them to find highly specific representations given sufficient time. While accurate, however, these representations are difficult to generalize. In contrast, layered networks generate rough error landscapes with a variety of local minima, allowing them to quickly find coarse representations. Although less accurate, these representations are easily adaptable. The presence of measurable performance tradeoffs in both layered and parallel networks has implications for understanding the behavior of a wide variety of natural and artificial learning systems.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Forecasting with artificial neural networks:
Zhang, G., Eddy Patuwo, B., & Y. Hu, M. (1998) · International Journal of Forecasting · doi:10.1016/s0169-2070(97)00044-7 · Cited by 4,229
Catastrophic Interference in Connectionist Networks: The Sequential Learning Problem
McCloskey, M., & Cohen, N. J. (1989) · Psychology of Learning and Motivation · doi:10.1016/s0079-7421(08)60536-8 · Cited by 3,770
Homeostatic plasticity in the developing nervous system
Turrigiano, G. G., & Nelson, S. B. (2004) · Nature Reviews Neuroscience · doi:10.1038/nrn1327 · Cited by 2,485
Activity-dependent scaling of quantal amplitude in neocortical neurons
Turrigiano, G. G., Leslie, K. R., Desai, N. S., Rutherford, L. C., & Nelson, S. B. (1998) · Nature · doi:10.1038/36103 · Cited by 2,480
Synaptic plasticity: taming the beast
Abbott, L. F., & Nelson, S. B. (2000) · Nature Neuroscience · doi:10.1038/81453 · Cited by 2,263 · cited 1×
“These competing processes are also maintained in natural neuronal systems and neural circuit models through homeostatic plasticity mechanisms such as synaptic scaling [54,55] and redistribution [56,57], in addition to the rehearsal methods employed here [19–23].”
Tononi, G., Sporns, O., & Edelman, G. M. (1994) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.91.11.5033 · Cited by 1,626 · cited 1×
“This performance supports the hypothesis that short path lengths (similar to the serialization h =‘ [52]) and low connection densities may facilitate simultaneous performance of information segregation (memory retention) and integration (generalization) within natural neuronal systems [53].”
BackgroundCellular and network mechanisms of rhythmic recurrent activity in neocortex
Sanchez-Vives, M. V., & McCormick, D. A. (2000) · Nature Neuroscience · doi:10.1038/79848 · Cited by 1,539 · cited 1×
“. Lastly, we assumed feedforward connectivity. The addition of recurrent connections could be used to study the relationship between recurrent structure and oscillatory functions such as cortical sleep rhythms”
MethodTraining induces changes in white-matter architecture
Scholz, J., Klein, M. C., Behrens, T. E. J., & Johansen-Berg, H. (2009) · Nature Neuroscience · doi:10.1038/nn.2412 · Cited by 1,508
Rapid formation and selective stabilization of synapses for enduring motor memories
Xu, T., Yu, X., Perlik, A. J., Tobin, W. F., Zweig, J. A., Tennant, K., Jones, T., & Zuo, Y. (2009) · Nature · doi:10.1038/nature08389 · Cited by 1,193
Modular and hierarchically modular organization of brain networks
Meunier, D., Lambiotte, R., & Bullmore, E. T. (2010) · Frontiers in Neuroscience · doi:10.3389/fnins.2010.00200 · Cited by 1,156
Cited by
Small-World Brain Networks Revisited
Bassett, D. S., & Bullmore, E. T. (2017) · The Neuroscientist · doi:10.1177/1073858416667720 · Cited by 842
Self-citationProjections from neocortex mediate top-down control of memory retrieval
Rajasethupathy, P., Sankaran, S., Marshel, J. H., Kim, C. K., Ferenczi, E., Lee, S. Y., Berndt, A., Ramakrishnan, C., Jaffe, A., Lo, M., et al. (2015) · Nature · doi:10.1038/nature15389 · Cited by 509
The physics of brain network structure, function and control
Lynn, C. W., & Bassett, D. S. (2019) · Nature Reviews Physics · doi:10.1038/s42254-019-0040-8 · Cited by 400
Self-citationCliques and cavities in the human connectome
Sizemore, A. E., Giusti, C., Kahn, A., Vettel, J. M., Betzel, R. F., & Bassett, D. S. (2018) · Journal of Computational Neuroscience · doi:10.1007/s10827-017-0672-6 · Cited by 372
Self-citationOptimal trajectories of brain state transitions
Gu, S., Betzel, R. F., Mattar, M. G., Cieslak, M., Delio, P. R., Grafton, S. T., Pasqualetti, F., & Bassett, D. S. (2017) · NeuroImage · doi:10.1016/j.neuroimage.2017.01.003 · Cited by 183
“One key challenge hampering progress is the complexity of these trajectories, which stems in part from the architectural complexity of the underlying anatomy (Hermundstad et al., 2011, 2013, 2014).”
Self-citation
Metadata
- Authors
- Hermundstad, A. M., Brown, K. S., Bassett, D. S., & Carlson, J. M.
- Venue
- PLoS Computational Biology
- Publisher
- Public Library of Science (PLoS)
- Published
- 2011-06-30
- Volume
- 7
- Issue
- 6
- Pages
- e1002063
- ISSN
- 1553-734X (ISSN-L)
- 1553-7358
- OpenAlex
- W2170345111
- Cited by
- 50
- References
- 21
- FWCI
- 1.77
- Type
- journal article
- Language
- en
- Open access
- gold
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Learning, Memory, and the Role of Neural Network Architecture. https://app.cochat.ai/papers/10.1371/journal.pcbi.1002063
- The paper
- Hermundstad, A. M., Brown, K. S., Bassett, D. S., & Carlson, J. M. (2011). Learning, Memory, and the Role of Neural Network Architecture. PLoS Computational Biology. https://doi.org/10.1371/journal.pcbi.1002063
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.