The capacity of the Hopfield associative memory
Abstract
Techniques from coding theory are applied to study rigorously the capacity of the Hopfield associative memory. Such a memory storesn-tuple of\pm 1's. The components change depending on a hard-limited version of linear functions of all other components. With symmetric connections between components, a stable state is ultimately reached. By building up the connection matrix as a sum-of-outer products ofmfundamental memories, one hopes to be able to recover a certain one of themmemories by using an initialn-tuple probe vector less than a Hamming distancen/2away from the fundamental memory. Ifmfundamental memories are chosen at random, the maximum asympotic value ofmin order that most of themoriginal memories are exactly recoverable isn/(2 \log n). With the added restriction that every one of themfundamental memories be recoverable exactly,mcan be no more thann/(4 \log n)asymptotically asnapproaches infinity. Extensions are also considered, in particular to capacity under quantization of the outer-product connection matrix. This quantized memory capacity problem is closely related to the capacity of the quantized Gaussian channel.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In a d-dimensional space, the standard Hopfield model can store d uncorrelated patterns without errors but only Cd/log(d) random patterns with C < 1/2 for a fixed stable pattern or C < 1/4 if all patterns are stable.
“In a d -dimensional space, the standard Hopfield model can store d uncorrelated patterns without errors but only Cd/ log( d ) random patterns with C < 1 / 2 for a fixed stable pattern or C < 1 / 4 if all patterns are stable (McEliece et al., 1987).”
From Hopfield Networks is All You Need (2020) · cites this paper as McEliece et al., 1987
In a d-dimensional space, the standard Hopfield model can store d uncorrelated patterns without errors but only Cd/ln(d) random patterns with C < 1/2 for a fixed stable pattern or C < 1/4 if all patterns are stable.
“In a d -dimensional space, the standard Hopfield model can store d uncorrelated patterns without errors but only Cd/ ln( d ) random patterns with C < 1 / 2 for a fixed stable pattern or C < 1 / 4 if all patterns are stable (McEliece et al., 1987).”
From Hopfield Networks is All You Need (2020) · cites this paper as McEliece et al., 1987
The storage capacity of classical binary Hopfield networks is very limited.
“The storage capacity of classical binary Hopfield networks (Hopfield, 1982) has been shown to be very limited.”
From Hopfield Networks is All You Need (2020) · cites this paper as McEliece et al., 1987
Cites
Hebb, D. O. (2005) · doi:10.4324/9781410612403 · Cited by 9,825 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
On the capabilities of multilayer perceptrons
Baum, E. B. (1988) · Journal of Complexity · doi:10.1016/0885-064x(88)90020-9 · Cited by 379
Hopfield Networks is All You Need
Ramsauer, H., Schäfl, B., Lehner, J., Seidl, P., Widrich, M., Adler, T., Gruber, L., Holzleitner, M., Pavlović, M., Sandve, G. K., et al. (2020) · arXiv · doi:10.48550/arxiv.2008.02217 · Cited by 236 · see citing works
“In a d-dimensional space, the standard Hopfield model can store d uncorrelated patterns without errors but only Cd/ log(d) random patterns with C < 1/2 for a fixed stable pattern or C < 1/4 if all patterns are stable [45].”
MethodOn a Model of Associative Memory with Huge Storage Capacity
Demircigil, M., Heusel, J., Löwe, M., Upgang, S., & Vermet, F. (2017) · Journal of Statistical Physics · doi:10.1007/s10955-017-1806-y · Cited by 136 · see citing works
Discussion: McCulloch-Pitts and related neural nets from 1943 to 1989
Cowan, J. D. (1990) · Bulletin of Mathematical Biology · doi:10.1007/bf02459569 · Cited by 22 · see citing works
- Authors
- McEliece, R., Posner, E., Rodemich, E., & Venkatesh, S.
- Affiliations
- California Institute of Technology (US)
- ORCID
- Venkatesh, S.
- Venue
- IEEE Transactions on Information Theory
- Published
- 1987-07
- Volume
- 33
- Issue
- 4
- Pages
- 461-482
- ISSN
- 0018-9448 (ISSN-L)
- 1557-9654
- OpenAlex
- W1984431842
- Cited by
- 910
- References
- 1
- Citations by year
- 2026: 10 · 2025: 21 · 2024: 21 · 2023: 12 · 2022: 12 · 2021: 17 · 2020: 22 · 2019: 16 · 2018: 20 · 2017: 16 · 2016: 12 · 2015: 11 · 2014: 19 · 2013: 17 · 2012: 17
- FWCI
- 46.11
- Type
- journal article
- Language
- en
- Open access
- closed
- Licence
- Locations
- IEEE Transactions on Information Theory (publishedVersion)
- CaltechAUTHORS (California Institute of Technology) (submittedVersion)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Neural Networks and Applications
- Physical Sciences › Computer Science › Computer Networks and Communications › Error Correcting Code Techniques
- Physical Sciences › Computer Science › Artificial Intelligence › Coding theory and cryptography
- Keywords
- Hopfield associative memory, memory capacity, coding theory, Hamming distance, outer product, symmetric connections, artificial neural networks
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
