CoChat
Sign Up
PreprintCited by 10Neural dynamics and brain functionNeural Networks and ApplicationsMemory and Neural Mechanisms

H-Mem: Harnessing synaptic plasticity with Hebbian Memory Networks

Open access
Limbacher, Thomas, Legenstein, Robert·2020·bioRxiv (Cold Spring Harbor Laboratory)
DOI 10.1101/2020.07.01.180372
Open this paper in CoChat — free accountOpen PDF

Abstract

The ability to base current computations on memories from the past is critical for many cognitive tasks such as story understanding. Hebbian-type synaptic plasticity is believed to underlie the retention of memories over medium and long time scales in the brain. However, it is unclear how such plasticity processes are integrated with computations in cortical networks. Here, we propose Hebbian Memory Networks (H-Mems), a simple neural network model that is built around a core hetero-associative network subject to Hebbian plasticity. We show that the network can be optimized to utilize the Hebbian plasticity processes for its computations. H-Mems can one-shot memorize associations between stimulus pairs and use these associations for decisions later on. Furthermore, they can solve demanding question-answering tasks on synthetic stories. Our study shows that neural network models are able to enrich their computations with memories through simple Hebbian plasticity processes.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1101/2020.07.01.180372

Cites

2 of 2 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • A-Mem organizes memories as an interconnected graph of linked notes, while H-Mem employs associative mechanisms to recall prerequisite facts when subsequent steps depend on prior observations.

    “For instance, A-Mem (Xu et al., 2025c) organizes memories as an interconnected graph of linked notes, while H-Mem (Limbacher and Legenstein, 2020) employs associative mechanisms to recall prerequisite facts when subsequent steps depend on prior observations.”

    From Memory in the Age of AI Agents (2025) · cites this paper as Limbacher and Legenstein, 2020

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Flagged first, then by citations

Research works printed but not linked: 33

  1. L. R. Squire, B. Knowlton, and G. Musen, "The structure and organization of memory," Annual review of psychology, vol. 44, no. 1, pp. 453-495, 1993.
  2. D. O. Hebb, The organization of behavior: A neuropsychological theory. Psychology Press, 2005.
  3. D. J. Willshaw, O. P. Buneman, and H. C. Longuet-Higgins, "Non-holographic associative memory," Nature, vol. 222, no. 5197, pp. 960-962, 1969.
  4. J. J. Hopfield, "Neural networks and physical systems with emergent collective computational abilities," Proceedings of the national academy of sciences, vol. 79, no. 8, pp. 2554-2558, 1982.
  5. T. Kohonen, Self-organization and associative memory. Springer Science & Business Media, 2012, vol. 8.
  6. T. Munkhdalai and A. Trischler, "Metalearning with hebbian fast weights," arXiv preprint arXiv:1807.05076, 2018. · arXiv
  7. J. Ba, G. E. Hinton, V. Mnih, J. Z. Leibo, and C. Ionescu, "Using fast weights to attend to the recent past," in Advances in Neural Information Processing Systems, 2016, pp. 4331-4339.
  8. H. Eichenbaum, "Memory: Organization and control," Annual review of psychology, vol. 68, pp. 19-45, 2017.
  9. J. Weston, A. Bordes, S. Chopra, A. M. Rush, B. van Merriënboer, A. Joulin, and T. Mikolov, "Towards AI-complete question answering: A set of prerequisite toy tasks," arXiv preprint arXiv:1502.05698, 2015. · arXiv
  10. S. Hochreiter and J. Schmidhuber, "Long short-term memory," Neural computation, vol. 9, no. 8, pp. 1735-1780, 1997.
  11. S. Sukhbaatar, J. Weston, R. Fergus, et al., "End-to-end memory networks," in Advances in neural information processing systems, 2015, pp. 2440-2448.
  12. C. Gulcehre, S. Chandar, K. Cho, and Y. Bengio, "Dynamic neural turing machine with continuous and discrete addressing schemes," Neural computation, vol. 30, no. 4, pp. 857-884, 2018.
  13. M. Henaff, J. Weston, A. Szlam, A. Bordes, and Y. LeCun, "Tracking the world state with recurrent entity networks," arXiv preprint arXiv:1612.03969, 2016. · arXiv
  14. J. Kamiński and U. Rutishauser, "Between persistently active and activity-silent frameworks: Novel vistas on the cellular basis of working memory," Annals of the New York Academy of Sciences, vol. 1464, no. 1, p. 64, 2020.
  15. G. Mongillo, O. Barak, and M. Tsodyks, "Synaptic theory of working memory," Science, vol. 319, no. 5869, pp. 1543-1546, 2008.
  16. D. Salaj, A. Subramoney, C. Kraisnikovic, G. Bellec, R. Legenstein, and W. Maass, "Spike-frequency adaptation provides a long short-term memory to networks of spiking neurons," bioRxiv, 2020.
  17. W. Maass, T. Natschläger, and H. Markram, "Real-time computing without stable states: A new frame- work for neural computation based on perturbations," Neural computation, vol. 14, no. 11, pp. 2531-2560, 2002.
  18. G. E. Hinton and D. C. Plaut, "Using fast weights to deblur old memories," in Proceedings of the ninth annual conference of the Cognitive Science Society, 1987, pp. 177-186.
  19. A. Graves, G. Wayne, M. Reynolds, T. Harley, I. Danihelka, A. Grabska-Barwińska, S. G. Colmenarejo, E. Grefenstette, T. Ramalho, J. Agapiou, et al., "Hybrid computing using a neural network with dynamic external memory," Nature, vol. 538, no. 7626, pp. 471-476, 2016.
  20. A. Miller, A. Fisch, J. Dodge, A.-H. Karimi, A. Bordes, and J. Weston, "Key-value memory networks for directly reading documents," arXiv preprint arXiv:1606.03126, 2016. · arXiv
  21. C. Xiong, S. Merity, and R. Socher, "Dynamic memory networks for visual and textual question answering," in International conference on machine learning, 2016, pp. 2397-2406.
  22. S. Chandar, S. Ahn, H. Larochelle, P. Vincent, G. Tesauro, and Y. Bengio, "Hierarchical memory networks," arXiv preprint arXiv:1605.07427, 2016. · arXiv
  23. H. Le, T. Tran, and S. Venkatesh, "Self-attentive associative memory," in Proceedings of the International Conference on Machine Learning, 2020.
  24. I. Schlag and J. Schmidhuber, "Learning to reason with third order tensor products," in Advances in neural information processing systems, 2018, pp. 9981-9993.
  25. T. Munkhdalai, A. Sordoni, T. Wang, and A. Trischler, "Metalearned neural memory," in Advances in Neural Information Processing Systems, 2019, pp. 13 310-13 321.
  26. T. Miconi, K. Stanley, and J. Clune, "Differentiable plasticity: Training plastic neural networks with backpropagation," in International Conference on Machine Learning, 2018, pp. 3559-3568.
  27. E. Oja, "Simplified neuron model as a principal component analyzer," Journal of mathematical biology, vol. 15, no. 3, pp. 267-273, 1982.
  28. T. P. Lillicrap, D. Cownden, D. B. Tweed, and C. J. Akerman, "Random synaptic feedback weights support error backpropagation for deep learning," Nature communications, vol. 7, no. 1, pp. 1-10, 2016.
  29. F. Zenke and S. Ganguli, "Superspike: Supervised learning in multilayer spiking neural networks," Neural computation, vol. 30, no. 6, pp. 1514-1541, 2018.
  30. G. Bellec, F. Scherr, A. Subramoney, E. Hajek, D. Salaj, R. Legenstein, and W. Maass, "A solution to the learning dilemma for recurrent networks of spiking neurons," bioRxiv, p. 738 385, 2019.
  31. M. Davies, N. Srinivasa, T.-H. Lin, G. Chinya, Y. Cao, S. H. Choday, G. Dimou, P. Joshi, N. Imam, S. Jain, et al., "Loihi: A neuromorphic manycore processor with on-chip learning," IEEE Micro, vol. 38, no. 1, pp. 82-99, 2018.
  32. S. Friedmann, J. Schemmel, A. Grübl, A. Hartel, M. Hock, and K. Meier, "Demonstrating hybrid learning in a flexible neuromorphic hardware system," IEEE transactions on biomedical circuits and systems, vol. 11, no. 1, pp. 128-142, 2016.
  33. M. A. Lynch, "Long-term potentiation and memory," Physiological reviews, vol. 84, no. 1, pp. 87-136, 2004.

Cited by

Sorted by citations

Authors
Limbacher, T., & Legenstein, R.
Affiliations
Graz University of Technology (AT)
Corresponding author
Legenstein, Robert
ORCID
Venue
bioRxiv (Cold Spring Harbor Laboratory)
Publisher
openRxiv
Published
2020-07-01
Volume
33
Pages
21627-21637
DOI
10.1101/2020.07.01.180372
OpenAlex
W3104020672
Cited by
10
References
2
Citations by year
2026: 1 · 2025: 2 · 2024: 1 · 2023: 3 · 2022: 3
Type
posted content
Language
en
Open access
Licence
www.biorxiv.org/about/FAQ#license (vor)
Locations
  • bioRxiv (Cold Spring Harbor Laboratory) (acceptedVersion)
  • neural information processing systems
Funders
  • European Commission (HORIZON2020, 785907, HBP SGA2)
  • Austrian Science Fund (I3251, I4670)
  • CHIST-ERA
Topics
  • Life Sciences › Neuroscience › Cognitive Neuroscience › Neural dynamics and brain function
  • Physical Sciences › Computer Science › Artificial Intelligence › Neural Networks and Applications
  • Life Sciences › Neuroscience › Cognitive Neuroscience › Memory and Neural Mechanisms
Keywords
Hebbian plasticity, memory networks, narrative comprehension, question answering, memory-based decision making

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review