The DOI matches its DataCite record; 2 of the 2 references it cites were found in a registry or an index.
Dense Associative Memory for Pattern Recognition
Abstract
A model of associative memory is studied, which stores and reliably retrieves many more patterns than the number of neurons in the network. We propose a simple duality between this dense associative memory and neural networks commonly used in deep learning. On the associative memory side of this duality, a family of models that smoothly interpolates between two limiting cases can be constructed. One limit is referred to as the feature-matching mode of pattern recognition, and the other one as the prototype regime. On the deep learning side of the duality, this family corresponds to feedforward neural networks with one hidden layer and various activation functions, which transmit the activities of the visible neurons to the hidden layer. This family of activation functions includes logistics, rectified linear units, and rectified polynomials of higher degrees. The proposed duality makes it possible to apply energy-based intuition from associative memory to analyze computational properties of neural networks with unusual activation functions - the higher rectified polynomials which until now have not been used in deep learning. The utility of the dense memories is illustrated for two test cases: the logical gate XOR and the recognition of handwritten digits from the MNIST data set.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the new type of Hopfield models of Krotov and Hopfield, the storage capacity is proportional to d^{n-1}.
“The storage capacity is proportional to d n -1 (Krotov & Hopfield, 2016).”
From Hopfield Networks is All You Need (2020) · cites this paper as Krotov & Hopfield, 2016
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Neural networks and physical systems with emergent collective computational abilities.
Hopfield, J. J. (1982) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.79.8.2554 · Cited by 19,270
Self-citationAssociative recall of memory without errors
Kanter, I., & Sompolinsky, H. (1987) · Physical Review A · doi:10.1103/physreva.35.380 · Cited by 379 · cited 1×
“By modifying the Hamiltonian (1) in a way that removes second order correlations between the stored memories, it is possible [10] to improve the capacity to K = N .”
Method
Flagged first, then by citations
Cited by
Sorted by citations
Hopfield Networks is All You Need
Ramsauer, H., Schäfl, B., Lehner, J., Seidl, P., Widrich, M., Adler, T., Gruber, L., Holzleitner, M., Pavlović, M., Sandve, G. K., et al. (2020) · arXiv · doi:10.48550/arxiv.2008.02217 · Cited by 236
Inferring neural activity before plasticity as a foundation for learning beyond backpropagation
Song, Y., Millidge, B., Salvatori, T., Lukasiewicz, T., Xu, Z., & Bogacz, R. (2024) · Nature Neuroscience · doi:10.1038/s41593-023-01514-1 · Cited by 85
Abstraction for Deep Reinforcement Learning
Shanahan, M., & Mitchell, M. (2022) · Proceedings of the Thirty-First International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2022/780 · Cited by 17
Input-driven dynamics for robust memory retrieval in Hopfield networks
Betteti, S., Baggio, G., Bullo, F., & Zampieri, S. (2025) · Science Advances · doi:10.1126/sciadv.adu6991 · Cited by 12
Behrouz, A., Razaviyayn, M., Zhong, P., & Mirrokni, V. (2025) · arXiv · doi:10.48550/arxiv.2504.13173 · Cited by 65
ATLAS: Learning to Optimally Memorize the Context at Test Time
Behrouz, A., Li, Z., Kacham, P., Daliri, M., Deng, Y., Zhong, P., Razaviyayn, M., & Mirrokni, V. (2025) · arXiv · doi:10.48550/arxiv.2505.23735
“To address this, building on methods from Kacham et al. (2024a) and Krotov and Hopfield (2016), we suggest using separable kernels 𝜎 ( 𝑥,𝑦 ) = 𝜙 ( 𝑥 ) ⊤ 𝜙 ( 𝑦 ) for keys and queries.”
Metadata
- Authors
- Krotov, D., & Hopfield, J. J.
- Venue
- arXiv
- Publisher
- arXiv
- Published
- 2016
- arXiv
- 1606.01164
- References
- 2
- Type
- Preprint
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
