Topological and Dynamical Complexity of Random Neural Networks
Open accessAbstract
Random neural networks are dynamical descriptions of randomly interconnected neural units. These show a phase transition to chaos as a disorder parameter is increased. The microscopic mechanisms underlying this phase transition are unknown and, similar to spin glasses, shall be fundamentally related to the behavior of the system. In this Letter, we investigate the explosion of complexity arising near that phase transition. We show that the mean number of equilibria undergoes a sharp transition from one equilibrium to a very large number scaling exponentially with the dimension on the system. Near criticality, we compute the exponential rate of divergence, called topological complexity. Strikingly, we show that it behaves exactly as the maximal Lyapunov exponent, a classical measure of dynamical complexity. This relationship unravels a microscopic mechanism leading to chaos which we further demonstrate on a simpler disordered system, suggesting a deep and underexplored link between topological and dynamical complexity.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The number of energy minima (spurious states, stable states) of Hopfield networks is exponentially in d.
“In contrast to the storage capacity, the number of energy minima (spurious states, stable states) of Hopfield networks is exponentially in d (Tanaka & Edwards, 1980; Bruck & Roychowdhury, 1990; Wainrib & Touboul, 2013).”
From Hopfield Networks is All You Need (2020) · cites this paper as Wainrib & Touboul, 2013
The paper reports that the average number of equilibria in its random neural networks scales exponentially with system size (p. 3). It does not establish an exponential number of energy minima or stable states in Hopfield networks: it says its system has no underlying energy landscape (p. 2) and that most of the equilibria counted are saddles (p. 2). (LLM, conf=0.89)
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Harnessing Nonlinearity: Predicting Chaotic Systems and Saving Energy in Wireless Communication
Jaeger, H., & Haas, H. (2004) · Science · doi:10.1126/science.1091277 · Cited by 3,811 · see citing works
Broadband criticality of human brain network synchronization
Kitzbichler, M. G., Smith, M. L., Christensen, S. R., & Bullmore, E. (2009) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1000314 · Cited by 527 · see citing works
Diversity in the Neuronal Machine
Soltesz, I. (2006) · doi:10.1093/acprof:oso/9780195177015.001.1 · Cited by 68 · see citing works
Flagged first, then by citations
Research works printed but not linked: 20
- [1]D. Parker, The Journal of Neuroscience 23 , 3154 (2003).
- [3]I. Aradi and I. Soltesz, The Journal of Physiology 538 , 227 (2002).
- [6]D. Sherrington and S. Kirkpatrick, Physical Review Letters 35 , 1792 (1975).
- [8]H. Sompolinsky, A. Crisanti, and H. Sommers, Physical Review Letters 61 , 259 (1988).
- [11]G. Hermann and J. Touboul, Physical Review Letters 109 , 018702 (2012).
- [12]T. Toyoizumi and L. Abbott, Physical Review E 84 , 051908 (2011).
- [15]P. Bak, How nature works: the science of self-organized criticality , Vol. 212 (Copernicus New York, 1996).
- [16]J. Beggs, Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences 366 , 329 (2008).
- [18]T. Tao, Topics in random matrix theory , Vol. 132 (Amer Mathematical Society, 2012).
- [19]R. Adler and J. Taylor, Random fields and geometry , Vol. 115 (Springer Verlag, 2007).
- [20]K. Rajan and L. Abbott, Physical Review Letters 97 , 188104 (2006).
- [21]L. Garcia del Molino, Dynamics of randomly connected systems with applications to neural networks , Master's thesis, ´ Ecole Polytechnique (2012).
- [22]Y. Wei, Physical Review E 85 , 066116 (2012).
- [24]Y. Fyodorov, Physical Review Letters 92 , 240601 (2004).
- [25]Y. Fyodorov and I. Williams, Journal of Statistical Physics 129 , 1081 (2007).
- [26]A. Auffinger and G. Arous, Arxiv preprint arXiv:1110.5872 (2011). · arXiv
- [28]B. Cessac, Journal de Physique I 5 , 409 (1995).
- [31]F. Ledrappier, The Annals of Mathematics 122 , 540 (1985).
- [32]P. Frederickson, J. Kaplan, E. Yorke, and J. Yorke, Journal of Differential Equations 49 , 185 (1983).
- [35]J. Aza¨ ıs and M. Wschebor, Level sets and extrema of random processes and fields (Wiley, 2009).
Cited by
Sorted by citations
Hopfield Networks is All You Need
Ramsauer, H., Schäfl, B., Lehner, J., Seidl, P., Widrich, M., Adler, T., Gruber, L., Holzleitner, M., Pavlović, M., Sandve, G. K., et al. (2020) · arXiv · doi:10.48550/arxiv.2008.02217 · Cited by 236 · see citing works
“In contrast to the storage capacity, the number of energy minima (spurious states, stable states) of Hopfield networks is exponentially in d [61, 13, 66].”
Background
- Authors
- Wainrib, G., & Touboul, J.
- Affiliations
- Centre National de la Recherche Scientifique (FR)
- ORCID
- Touboul, Jonathan
- Venue
- Physical Review Letters
- Publisher
- American Physical Society (APS)
- Published
- 2013-03-11
- Volume
- 110
- Issue
- 11
- Pages
- 118101-118101
- ISSN
- 0031-9007 (ISSN-L)
- 1079-7114
- OpenAlex
- W1983611358
- Cited by
- 124
- References
- 3
- Citations by year
- 2026: 7 · 2025: 8 · 2024: 11 · 2023: 10 · 2022: 14 · 2021: 10 · 2020: 10 · 2019: 8 · 2018: 11 · 2017: 7 · 2016: 9 · 2015: 9 · 2014: 5 · 2013: 5
- FWCI
- 6.14
- Type
- journal article
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- Physical Review Letters (publishedVersion)
- PubMed (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- HAL (submittedVersion)
- arXiv (Cornell University)
- Funders
- Agence Nationale de la Recherche (ANR-10-LABX-0054)
- Topics
- Physical Sciences › Physics and Astronomy › Condensed Matter Physics › Theoretical and Computational Physics
- Physical Sciences › Computer Science › Artificial Intelligence › Neural Networks and Applications
- Physical Sciences › Physics and Astronomy › Statistical and Nonlinear Physics › Chaos control and synchronization
- Keywords
- random neural networks, topological complexity, Lyapunov exponents, number of equilibria, criticality, disordered systems, spin glass, dynamical complexity
- MeSH terms
- Models, Neurological, Neurons, Neural Networks, Computer, Biomimetics
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
