Real-Time Computation at the Edge of Chaos in Recurrent Neural Networks
The DOI matches its Crossref record; 26 of the 26 references it cites were found in a registry or an index.
Abstract
Depending on the connectivity, recurrent networks of simple computational units can show very different types of dynamics, ranging from totally ordered to chaotic. We analyze how the type of dynamics (ordered or chaotic) exhibited by randomly connected networks of threshold gates driven by a time-varying input signal depends on the parameters describing the distribution of the connectivity matrix. In particular, we calculate the critical boundary in parameter space where the transition from ordered to chaotic dynamics takes place. Employing a recently developed framework for analyzing real-time computations, we show that only near the critical boundary can such networks perform complex computations on time series. Hence, this result strongly supports conjectures that dynamical systems that are capable of doing complex computational tasks should operate near the edge of chaos, that is, the transition from ordered to chaotic dynamics.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Intrinsic fluctuations that are poised ''at the edge of chaos'' are thought to maximize the computational efficiency of such systems and the flexibility to switch between adaptive regimes (Bertschinger & Natschla ¨ger, 2004).”
From A potential mechanism for Gibsonian resonance: behavioral entrainment emerges from local homeostasis in an unsupervised reservoir network (2024) · cites this paper as Bertschinger & Natschläger, 2004
In the citing paper
“Finally, we show that a randomly connected circuit model, akin to a “reservoir” network13-15, can reproduce the observed distribution of timescales, provided that the network operates at the critical point (or “edge of chaos”)16-18.”
From A reservoir of time constants for memory traces in cortical neurons (2011) · cites this paper as [17]
Citations
Cites
Self-organized criticality: An explanation of the 1/f noise
Bak, P., Tang, C., & Wiesenfeld, K. (1987) · Physical Review Letters · doi:10.1103/physrevlett.59.381 · Cited by 7,591
The origins of order; self organization and selection in evolution
(1994) · International Journal of Biochemistry · doi:10.1016/0020-711x(94)90119-8 · Cited by 6,649
Introduction to the theory of neural computation
(1994) · Neural Networks · doi:10.1016/0893-6080(94)90014-0 · Cited by 6,401
Bak, P., Tang, C., & Wiesenfeld, K. (1988) · Physical Review A · doi:10.1103/physreva.38.364 · Cited by 4,713
Maass, W., Natschläger, T., & Markram, H. (2002) · Neural Computation · doi:10.1162/089976602760407955 · Cited by 4,025 · cited 10×
“(Maass et al., 2002) and indepenently by Jaeger (Jaeger, 2002) who called it “echo state networks”.”
BackgroundInfluentialSelf-citationChaos in Neuronal Networks with Balanced Excitatory and Inhibitory Activity
van Vreeswijk, C., & Sompolinsky, H. (1996) · Science · doi:10.1126/science.274.5293.1724 · Cited by 1,811
Computation at the edge of chaos: Phase transitions and emergent computation
Langton, C. G. (1990) · Physica D: Nonlinear Phenomena · doi:10.1016/0167-2789(90)90064-v · Cited by 1,527 · cited 5×
“These results provide further evidence for the idea of “computation at the edge of chaos” (Langton, 1990).”
BackgroundResultInfluentialFading memory and the problem of approximating nonlinear operators with Volterra series
Boyd, S., & Chua, L. (1985) · IEEE Transactions on Circuits and Systems · doi:10.1109/tcs.1985.1085649 · Cited by 991
Random Networks of Automata: A Simple Annealed Approximation
Derrida, B., & Pomeau, Y. (1986) · Europhysics Letters (EPL) · doi:10.1209/0295-5075/1/2/001 · Cited by 706
Dynamic Patterns in Complex Systems
(1988) · doi:10.1142/0534 · Cited by 548
Haldeman, C., & Beggs, J. M. (2005) · Physical Review Letters · doi:10.1103/physrevlett.94.058101 · Cited by 469
Revisiting the Edge of Chaos: Evolving Cellular Automata to Perform Computations
Mitchell, M., Hraber, P., & Crutchfield, J. P. (1993) · arXiv (Cornell University) · doi:10.48550/arxiv.adap-org/9303003 · Cited by 388
Topological Evolution of Dynamical Networks: Global Criticality from Local Dynamics
Bornholdt, S., & Rohlf, T. (2000) · Physical Review Letters · doi:10.1103/physrevlett.84.6114 · Cited by 271
Evolution of overlaps between configurations in random Boolean networks
Derrida, B., & Weisbuch, G. (1986) · Journal de Physique · doi:10.1051/jphys:019860047080129700 · Cited by 177
Self-organized critical neural networks
Bornholdt, S., & Röhl, T. (2003) · Physical Review E · doi:10.1103/physreve.67.066118 · Cited by 131
Discrete Neural Computation: A Theoretical Foundation
Siu, K.-Y. S., Roychowdhury, V., & Kailath, T. (1995) · Medical Entomology and Zoology · Cited by 116
Fading memory and kernel properties of generic cortical microcircuit models
Maass, W., Natschläger, T., & Markram, H. (2004) · Journal of Physiology-Paris · doi:10.1016/j.jphysparis.2005.09.020 · Cited by 111
Self-citationDynamical phase transition in nonsymmetric spin glasses
Derrida, B. (1987) · Journal of Physics A: Mathematical and General · doi:10.1088/0305-4470/20/11/009 · Cited by 105
Lyapunov exponents in random Boolean networks
Luque, B., & Solé, R. V. (2000) · Physica A: Statistical Mechanics and its Applications · doi:10.1016/s0378-4371(00)00184-9 · Cited by 98
Criticality in random threshold networks: annealed approximation and beyond
Rohlf, T., & Bornholdt, S. (2002) · Physica A Statistical Mechanics and its Applications · doi:10.1016/s0378-4371(02)00798-7 · Cited by 93
Reproducible Sequence Generation In Random Neural Ensembles
Huerta, R., & Rabinovich, M. (2004) · Physical Review Letters · doi:10.1103/physrevlett.93.238104 · Cited by 77
Criticality in random threshold networks: annealed approximation and beyond
Rohlf, T., & Bornholdt, S. (2002) · RePEc: Research Papers in Economics · Cited by 56
Self-organization and dynamics reduction in recurrent networks: stimulus presentation and learning
Daucé, E., Quoy, M., Cessac, B., Doyon, B., & Samuelides, M. (1998) · Neural Networks · doi:10.1016/s0893-6080(97)00131-7 · Cited by 50
Information Dynamics and Emergent Computation in Recurrent Circuits of Spiking Neurons
Natschläger, T., & Maass, W. (2003) · neural information processing systems · Cited by 19
Self-citationSignal buffering in random networks of spiking neurons: Microscopic versus macroscopic phenomena
Mayor, J., & Wulfram Gerstner, B. (2005) · Physical Review E · doi:10.1103/physreve.72.051906 · Cited by 12
Mayor, J., & Gerstner, W. (2004) · Journal of Physiology-Paris · doi:10.1016/j.jphysparis.2005.09.009 · Cited by 8
Research works printed but not linked: 1
- Mitchell M., 1993, Complex Systems, 7, 89
Cited by
Reservoir computing approaches to recurrent neural network training
Lukoševičius, M., & Jaeger, H. (2009) · Computer Science Review · doi:10.1016/j.cosrev.2009.03.005 · Cited by 2,956
“, [94]) that at the edge of chaos many types of dynamical systems, including binary systems and reservoirs, possess high computational power [87, 95].”
BackgroundMethodInfluentialRecent advances in physical reservoir computing: A review
Tanaka, G., Yamane, T., Héroux, J. B., Nakane, R., Kanazawa, N., Takeda, S., Numata, H., Nakano, D., & Hirose, A. (2019) · Neural Networks · doi:10.1016/j.neunet.2019.03.005 · Cited by 2,146
“…between non-chaotic and chaotic regimes, it is often recommended that the parameter be set close to the transition point (the so-called edge of chaos (Bertschinger & Natschläger, 2004; Legenstein & Maass, 2007)) where the transformation by a reservoir is neither very expanding nor very contracting.”
BackgroundResultGenerating Coherent Patterns of Activity from Chaotic Neural Networks
Sussillo, D., & Abbott, L. F. (2009) · Neuron · doi:10.1016/j.neuron.2009.07.018 · Cited by 1,092
“Such an input can induce a transition between chaotic and nonchaotic states (Molgedey et al., 1992; Bertschinger and Natschlä ger, 2004; K. Rajan, L.F.A., and H. Sompolinsky, unpublished data).”
Embedded Devices for Neuromorphic Time-Series Assessment
Mazumder, A. N., Hosseini, M., & Mohsenin, T. (2022) · Research Explorer (The University of Manchester) · doi:10.13016/m2ksy0-wxr3 · Cited by 652
Can structure predict function in the human brain?
Honey, C. J., Thivierge, J.-P., & Sporns, O. (2010) · NeuroImage · doi:10.1016/j.neuroimage.2010.01.071 · Cited by 629
“can be realized (Bertschinger and Natschläger, 2004), including the approximation of intricate time-varying functions, but rules that link”
Metadata
- Authors
- Bertschinger, N., & Natschläger, T.
- Affiliations
- Graz University of Technology (AT)
- Software Competence Center Hagenberg (Austria) (AT)
- ORCID
- Bertschinger, Nils
- Venue
- Neural Computation
- Publisher
- MIT Press - Journals
- Published
- 2004-07-01
- Volume
- 16
- Issue
- 7
- Pages
- 1413-1436
- ISSN
- 0899-7667 (ISSN-L)
- 1530-888X
- OpenAlex
- W2057755457
- Cited by
- 833
- References
- 26
- Citations by year
- 2026: 45 · 2025: 73 · 2024: 61 · 2023: 48 · 2022: 66 · 2021: 73 · 2020: 52 · 2019: 53 · 2018: 47 · 2017: 48 · 2016: 26 · 2015: 33 · 2014: 29 · 2013: 33 · 2012: 27
- FWCI
- 13.53
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Neural Computation (publishedVersion)
- PubMed (publishedVersion)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Neural Networks and Applications
- Physical Sciences › Computer Science › Computational Theory and Mathematics › Cellular Automata and Applications
- Physical Sciences › Computer Science › Artificial Intelligence › Neural Networks and Reservoir Computing
- Keywords
- edge of chaos, recurrent neural networks, critical connectivity, threshold gates, random connectivity, real-time processing, dynamical systems
- MeSH terms
- Animals, Artificial Intelligence, Computer Simulation, Feedback, Memory, Models, Neurological, Time Factors, Neural Networks, Computer, Nonlinear Dynamics
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.