Real-Time Computation at the Edge of Chaos in Recurrent Neural Networks
The DOI matches its Crossref record; 26 of the 26 references it cites were found in a registry or an index.
Abstract
Depending on the connectivity, recurrent networks of simple computational units can show very different types of dynamics, ranging from totally ordered to chaotic. We analyze how the type of dynamics (ordered or chaotic) exhibited by randomly connected networks of threshold gates driven by a time-varying input signal depends on the parameters describing the distribution of the connectivity matrix. In particular, we calculate the critical boundary in parameter space where the transition from ordered to chaotic dynamics takes place. Employing a recently developed framework for analyzing real-time computations, we show that only near the critical boundary can such networks perform complex computations on time series. Hence, this result strongly supports conjectures that dynamical systems that are capable of doing complex computational tasks should operate near the edge of chaos, that is, the transition from ordered to chaotic dynamics.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Intrinsic fluctuations that are poised ''at the edge of chaos'' are thought to maximize the computational efficiency of such systems and the flexibility to switch between adaptive regimes (Bertschinger & Natschla ¨ger, 2004).”
From A potential mechanism for Gibsonian resonance: behavioral entrainment emerges from local homeostasis in an unsupervised reservoir network (2024) · cites this paper as Bertschinger & Natschläger, 2004
In the citing paper
“Finally, we show that a randomly connected circuit model, akin to a “reservoir” network13-15, can reproduce the observed distribution of timescales, provided that the network operates at the critical point (or “edge of chaos”)16-18.”
From A reservoir of time constants for memory traces in cortical neurons (2011) · cites this paper as [17]
Citations
Cites
Self-organized criticality: An explanation of the 1/f noise
Bak, P., Tang, C., & Wiesenfeld, K. (1987) · Physical Review Letters · doi:10.1103/physrevlett.59.381 · Cited by 7,591
The origins of order; self organization and selection in evolution
(1994) · International Journal of Biochemistry · doi:10.1016/0020-711x(94)90119-8 · Cited by 6,649
Introduction to the theory of neural computation
(1994) · Neural Networks · doi:10.1016/0893-6080(94)90014-0 · Cited by 6,401
Bak, P., Tang, C., & Wiesenfeld, K. (1988) · Physical Review A · doi:10.1103/physreva.38.364 · Cited by 4,713
Maass, W., Natschläger, T., & Markram, H. (2002) · Neural Computation · doi:10.1162/089976602760407955 · Cited by 4,025 · cited 10×
“(Maass et al., 2002) and indepenently by Jaeger (Jaeger, 2002) who called it “echo state networks”.”
BackgroundInfluentialSelf-citationChaos in Neuronal Networks with Balanced Excitatory and Inhibitory Activity
van Vreeswijk, C., & Sompolinsky, H. (1996) · Science · doi:10.1126/science.274.5293.1724 · Cited by 1,811
Computation at the edge of chaos: Phase transitions and emergent computation
Langton, C. G. (1990) · Physica D: Nonlinear Phenomena · doi:10.1016/0167-2789(90)90064-v · Cited by 1,527 · cited 5×
“These results provide further evidence for the idea of “computation at the edge of chaos” (Langton, 1990).”
BackgroundResultInfluentialFading memory and the problem of approximating nonlinear operators with Volterra series
Boyd, S., & Chua, L. (1985) · IEEE Transactions on Circuits and Systems · doi:10.1109/tcs.1985.1085649 · Cited by 991
Random Networks of Automata: A Simple Annealed Approximation
Derrida, B., & Pomeau, Y. (1986) · Europhysics Letters (EPL) · doi:10.1209/0295-5075/1/2/001 · Cited by 706
Dynamic Patterns in Complex Systems
(1988) · doi:10.1142/0534 · Cited by 548
Research works printed but not linked: 1
- Mitchell M., 1993, Complex Systems, 7, 89
Cited by
Items 201–300 of 831
Randomly Connected Networks Have Short Temporal Memory
Wallace, E. W. J., Maei, H. R., & Latham, P. E. (2013) · Neural Computation · doi:10.1162/neco_a_00449 · Cited by 40
“That such a regime exists has been known for some time (Bertschinger & Natschläger, 2004; Maass et al., 2005; Legenstein & Maass, 2007; Büsing et al., 2010); the reason we missed it above (see in particular sections 4.1 and 4.2) is that we assumed K 1 and d small but O(1), for which we can never…”
BackgroundMethodInfluentialThe Capabilities of Chaos and Complexity
Abel, D. L. (2009) · International Journal of Molecular Sciences · doi:10.3390/ijms10010247 · Cited by 42
“If chaos is inadequate to explain self-organization, what about “the Edge of Chaos?””
BackgroundAn open hypothesis: Is epilepsy learned, and can it be unlearned?
Hsu, D., Chen, W., Hsu, M., & Beggs, J. M. (2008) · Epilepsy & Behavior · doi:10.1016/j.yebeh.2008.05.007 · Cited by 45
“Computational modeling studies suggest that networks operating at the critical point can simultaneously optimize information transmission [18,37,43], information storage [30], computational power [43], and stability [30,43].”
The reservoir learning power across quantum many-body localization transition
Xia, W., Zou, J., Qiu, X., & Li, X. (2022) · Frontiers of Physics · doi:10.1007/s11467-022-1158-1 · Cited by 48
DNA Reservoir Computing: A Novel Molecular Computing Approach
Goudarzi, A., Lakin, M. R., & Stefanović, D. (2013) · Lecture notes in computer science · doi:10.1007/978-3-319-01928-4_6 · Cited by 59
“Many studies have shown that to achieve a suitable reservoir in general, the underlying dynamical system must operate in the critical dynamical regime [8,16,20,21].”
BackgroundInitialization and self‐organized optimization of recurrent neural network connectivity
Boedecker, J., Obst, O., Mayer, N. M., & Asada, M. (2009) · HFSP journal · doi:10.2976/1.3240502 · Cited by 44
“Bertschinger and Natschläger (2004)] while keeping a fixed energy expenditure for the neurons. In Lazar et al. (2007), this hypothesis was confirmed to the extent that networks with a combination of intrinsic and extrinsic plasticity were generally closer to this region than networks trained with only one form of plas…”
BackgroundInfluentialThe Operating Regime of Local Computations in Primary Visual Cortex
Stimberg, M., Wimmer, K., Martin, R. L., Schwabe, L., Mariño, J., Schummers, J., Lyon, D. C., Sur, M., & Obermayer, K. (2009) · Cerebral Cortex · doi:10.1093/cercor/bhn240 · Cited by 45
Jin, Y., & Li, P. (2016) · Neurocomputing · doi:10.1016/j.neucom.2016.11.045 · Cited by 40
“To determine the ordered and chaotic regimes for discrete dynamical systems driven by online inputs, [2] proposed to track the evolution of state di ff erence resulting from two close initial states while the system is driven by the same input.”
BackgroundBornholdt, S., & Kauffman, S. A. (2019) · Journal of Theoretical Biology · doi:10.1016/j.jtbi.2019.01.036 · Cited by 39
Models Wagging the Dog: Are Circuits Constructed with Disparate Parameters?
Nowotny, T., Szűcs, A. L., Levi, R., & Selverston, A. I. (2007) · Neural Computation · doi:10.1162/neco.2007.19.8.1985 · Cited by 41
Scaling Up Echo-State Networks With Multiple Light Scattering
Dong, J., Gigan, S., Krząkała, F., & Wainrib, G. (2018) · IEEE Statistical Signal Processing Workshop (SSP) · doi:10.1109/ssp.2018.8450698 · Cited by 38
Does the Cerebral Cortex Exploit High-Dimensional, Non-linear Dynamics for Information Processing?
Singer, W., & Lazăr, A. (2016) · Frontiers in Computational Neuroscience · doi:10.3389/fncom.2016.00099 · Cited by 43
“Artificial recurrent networks seem to function particularly well-near the edge of chaos, where their dynamics is rich yet predictable (Bertschinger and Natschläger, 2004).”
BackgroundAvalanche and edge-of-chaos criticality do not necessarily co-occur in neural networks
Kanders, K., Lorimer, T., & Stoop, R. (2017) · Chaos An Interdisciplinary Journal of Nonlinear Science · doi:10.1063/1.4978998 · Cited by 46
Critical Neuronal Models with Relaxed Timescale Separation
Das, A., & Levina, A. (2019) · Physical Review X · doi:10.1103/physrevx.9.021062 · Cited by 42
“Additional reason for it comes from observations that criticality brings about optimal computational capabilities [34, 5], optimal transmission and storage of information [6], and sensitivity to sensory stimuli [53, 32].”
BackgroundEcho state network activation function based on bistable stochastic resonance
Liao, Z., Wang, Z., Yamahara, H., & Tabata, H. (2021) · Chaos Solitons & Fractals · doi:10.1016/j.chaos.2021.111503 · Cited by 40
Edge of Chaos Computation in Mixed-Mode VLSI - A Hard Liquid
Schürmann, F., Meier, K., & Schemmel, J. (2004) · neural information processing systems · Cited by 39
Optimal short-term memory before the edge of chaos in driven random recurrent networks
Haruna, T., & Nakajima, K. (2019) · Physical review. E · doi:10.1103/physreve.100.062312 · Cited by 40
Classification of amyotrophic lateral sclerosis by brain volume, connectivity, and network dynamics
Thome, J., Steinbach, R., Großkreutz, J., Durstewitz, D., & Koppe, G. (2021) · Human Brain Mapping · doi:10.1002/hbm.25679 · Cited by 39
“…cf. Durstewitz, 2017), the variance of the parameters in the transition matrices (where high variance favors a less stable and more chaotic dynamics, cf. Bertschinger & Natschläger, 2004) separated for both regularized and nonregularized parameters (“FD5–FD7”), the average magnitude of the…”
BackgroundNanoscale neuromorphic networks and criticality: a perspective
Dunham, C. S., Lilak, S., Hochstetter, J., Loeffler, A., Zhu, R., Chase, C., Stieg, A. Z., Kuncic, Z., & Gimzewski, J. K. (2021) · Journal of Physics Complexity · doi:10.1088/2632-072x/ac3ad3 · Cited by 46
Dynamic Nonlinear Behavior of Ionic Liquid-Based Reservoir Computing Devices
Matsuo, T., Sato, D., Koh, S.-G., Shima, H., Naitoh, Y., Akinaga, H., Itoh, T., Nokami, T., Kobayashi, M., & Kinoshita, K. (2022) · ACS Applied Materials & Interfaces · doi:10.1021/acsami.2c04167 · Cited by 37
“21 Short-term memory characteristics of the IL - reservoir were evaluated based on STM tasks.”
MethodA Coupled Spintronics Neuromorphic Approach for High‐Performance Reservoir Computing
Akashi, N., Kuniyoshi, Y., Tsunegi, S., Taniguchi, T., Nishida, M., Sakurai, R., Wakao, Y., Kawashima, K., & Nakajima, K. (2022) · Advanced Intelligent Systems · doi:10.1002/aisy.202200123 · Cited by 43
Laplacian paths in complex networks: Information core emerges from entropic transitions
Villegas, P., Gabrielli, A., Santucci, F., Caldarelli, G., & Gili, T. (2022) · Physical Review Research · doi:10.1103/physrevresearch.4.033196 · Cited by 45
“As a direct consequence of this, intrinsically disordered hierarchic-modular networks do indeed generically optimize transmission and storage of information, improving computational capabilities [68] and strengthening the network stability [69].”
BackgroundInformation Processing Capacity of Spintronic Oscillator
Tsunegi, S., Kubota, T., Kamimaki, A., Grollier, J., Cros, V., Yakushiji, K., Fukushima, A., Yuasa, S., Kubota, H., Nakajima, K., et al. (2023) · Advanced Intelligent Systems · doi:10.1002/aisy.202300175 · Cited by 46
Assessing criticality in pre-seizure single-neuron activity of human epileptic cortex
Hagemann, A., Wilting, J., Samimizad, B., Mormann, F., & Priesemann, V. (2021) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1008773 · Cited by 39
la Fuente, I. M. D., Martínez, L., Carrasco-Pujante, J., Fedetz, M., Lopez, J. I., & Malaina, I. (2021) · Frontiers in Genetics · doi:10.3389/fgene.2021.644615 · Cited by 42
LSMCore: A 69k-Synapse/mm 2 Single-Core Digital Neuromorphic Processor for Liquid State Machine
Wang, L., Yang, Z., Guo, S., Qu, L., Zhang, X., Kang, Z., & Xu, W. (2022) · IEEE Transactions on Circuits and Systems I Regular Papers · doi:10.1109/tcsi.2022.3147380 · Cited by 38
Chaotic neural dynamics facilitate probabilistic computations through sampling
Terada, Y., & Toyoizumi, T. (2024) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2312992121 · Cited by 41
“This line of research has also triggered theoretical works elucidating the computational advantages of neural dynamics at the edge of chaos [39–42].”
BackgroundSearch optimization, funnel topography, and dynamical criticality on the string landscape
Khoury, J., & Parrikar, O. (2019) · Journal of Cosmology and Astroparticle Physics · doi:10.1088/1475-7516/2019/12/014 · Cited by 39
“Recurrent neural networks [119, 120], specifically echo state networks [121], achieve maximal computational power for vanishing Lyapunov exponent [122].”
BackgroundSwitching Characteristics of a Locally-Active Memristor with Binary Memories
Ying, J., Wang, G., Dong, Y., & Yu, S. (2019) · International Journal of Bifurcation and Chaos · doi:10.1142/s0218127419300301 · Cited by 33
Scale‐free dynamics of core‐periphery topography
Klar, P., Çatal, Y., Langner, R., Huang, Z., & Northoff, G. (2022) · Human Brain Mapping · doi:10.1002/hbm.26187 · Cited by 37
“The bottom row shows the same continuum and change for the MF extrinsic demands of the environment (Bertschinger & Natschläger, 2004; Kello et al., 2010; Linkenkaer-Hansen et al., 2001; Poil et al., 2008).”
BackgroundJannesari, M., Saeedi, A., Zare, M., Ortiz‐Mantilla, S., Plenz, D., & BENASICH, A. A. (2020) · Brain Structure and Function · doi:10.1007/s00429-019-02014-4 · Cited by 32
Liao, Z., Wang, Z., Yamahara, H., & Tabata, H. (2021) · Neurocomputing · doi:10.1016/j.neucom.2021.09.074 · Cited by 33
Addressing skepticism of the critical brain hypothesis
Beggs, J. M. (2022) · Frontiers in Computational Neuroscience · doi:10.3389/fncom.2022.703865 · Cited by 43
“Other functions that are predicted to be optimized near the critical point include computational power (Bertschinger and Natschlager, 2004; Legenstein and Maass, 2007), information storage (Haldeman and Beggs, 2005), sensitivity to changes in inputs (Williams-Garcia et al.”
BackgroundNishioka, D., Shingaya, Y., Tsuchiya, T., Higuchi, T., & Terabe, K. (2024) · Science Advances · doi:10.1126/sciadv.adk6438 · Cited by 37
Input Dependent Cell Assembly Dynamics in a Model of the Striatal Medium Spiny Neuron Network
Ponzi, A. P. D., & Wickens, J. R. (2012) · Frontiers in Systems Neuroscience · doi:10.3389/fnsys.2012.00006 · Cited by 36
“Network dynamics may be consistent with the paradigm of “weak chaos” (Zillmer et al., 2009) or the “edge of chaos” (Langton, 1990; Bertchinger and NatschlÃger, 1995), since the Lyapanuov exponent is close to zero, “between” a chaotic and a stable state (Ponzi and Wickens, 2010a).”
BackgroundMethodIntrinsic Adaptation in Autonomous Recurrent Neural Networks
Marković, D., & Gros, C. (2011) · Neural Computation · doi:10.1162/neco_a_00232 · Cited by 29
“It has been suggested that this rise in the complexity improves the computational properties and the capability of dynamical systems to process information (Sole & Miramontes, 1995 ; Bertschinger & Natschlager, 2004 ; Legenstein & Maass, 2006 ).”
BackgroundKia, B., Mendes, A., Parnami, A., George, R., Mobley, K., & Ditto, W. L. (2020) · PLoS ONE · doi:10.1371/journal.pone.0228534 · Cited by 30
“Langton investigated this subject in context of computation in his seminar paper titled “Computation at the edge of chaos: Phase transitions and emergent computation [9],” and many others followed his footsteps [10].”
BackgroundThe edge of chaos: quantum field theory and deep neural networks
Grosvenor, K. T., & Jefferson, R. (2022) · SciPost Physics · doi:10.21468/scipostphys.12.3.081 · Cited by 58
El‐Boustani, S., & Destexhe, A. (2010) · International Journal of Bifurcation and Chaos · doi:10.1142/s0218127410026769 · Cited by 37
“From a computational point of view, chaotic behaviour or nearly chaotic regime (edge of chaos) can be optimal for information processing [Bertschinger & Natschläger, 2004; Legenstein & Maass, 2007].”
Critical-like Brain Dynamics in a Continuum from Second- to First-Order Phase Transition
Wang, S. H., Siebenhühner, F., Arnulfo, G., Myrov, V. O., Nobili, L., Breakspear, M. J., Palva, S., & Palva, J. M. (2023) · Journal of Neuroscience · doi:10.1523/jneurosci.1889-22.2023 · Cited by 33
Brain criticality beyond avalanches: open problems and how to approach them
Girardi‐Schappo, M. (2021) · Journal of Physics Complexity · doi:10.1088/2632-072x/ac2071 · Cited by 33
Nonlinear transient computation
Crook, N. T. (2006) · Neurocomputing · doi:10.1016/j.neucom.2006.10.148 · Cited by 28
Compensating Inhomogeneities of Neuromorphic VLSI Devices Via Short-Term Synaptic Plasticity
Bill, J., Schuch, K., Brüderle, D., Schemmel, J., Maass, W., & Meier, K. (2010) · Frontiers in Computational Neuroscience · doi:10.3389/fncom.2010.00129 · Cited by 30
“As a general rule, it has been shown to be beneficial to operate a recurrent neural network in the edge-of-chaos regime (Bertschinger and Natschläger, 2004).”
BackgroundA brain-inspired computational model for spatio-temporal information processing
Lin, X., Zou, X., Ji, Z., Huang, T., Wu, S., & Mi, Y. (2021) · Neural Networks · doi:10.1016/j.neunet.2021.05.015 · Cited by 31
Electromagnetic wave-based extreme deep learning with nonlinear time-Floquet entanglement
Momeni, A., & Fleury, R. (2022) · Nature Communications · doi:10.1038/s41467-022-30297-5 · Cited by 43
Self-organized criticality as a framework for consciousness: A review study
Walter, N., & Hinterberger, T. (2022) · Frontiers in Psychology · doi:10.3389/fpsyg.2022.911620 · Cited by 35
“Further, optimal information transmission, storage and capacity has been reported in neuronal models at criticality (Bertschinger and Natschläger, 2004; Haldeman and Beggs, 2005; Del Papa et al., 2017), in vivo (Shew et al.”
BackgroundUnifying quality metrics for reservoir networks
Gibbons, T. E. (2010) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn.2010.5596307 · Cited by 30
“Researchers, including [9] and [13], have shown the optimal performance of reservoirs occur at the " edge of chaos " where the Lyapunov exponent, , is near zero.”
BackgroundCampos, J. G. F., Costa, A. D. A., Copelli, M., & Kinouchi, O. (2017) · Physical review. E · doi:10.1103/physreve.95.042303 · Cited by 32
Configured quantum reservoir computing for multi-task machine learning
Xia, W., Zou, J., Qiu, X., Chen, F., Zhu, B., Li, C., Deng, D.-L., & Li, X. (2023) · 中国科学通报:英文版 · doi:10.1016/j.scib.2023.08.040 · Cited by 47
“The performance of the random permutation model is examined on short-term memory and parity check, two standard reservoir computing tasks widely used for benchmarking the learning capacity [53, 54].”
MethodCriticality supports cross-frequency cortical-thalamic information transfer during conscious states
Toker, D., Müller, E. J., Miyamoto, H., Riga, M. S., Lladó-Pelfort, L., Yamakawa, K., Artigas, F., Shine, J. M., Hudson, A. E., Pouratian, N., et al. (2024) · eLife · doi:10.7554/elife.86547 · Cited by 30
An intelligent method of swarm neural networks for equalities-constrained nonconvex optimization
Che, H., Li, C., He, X., & Huang, T. (2015) · Neurocomputing · doi:10.1016/j.neucom.2015.04.033 · Cited by 27
Novel plasticity rule can explain the development of sensorimotor intelligence
Der, R., & Martius, G. (2015) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1508400112 · Cited by 37
“This specific regime, called the edge of chaos, is argued to be a vital characteristic of life and development [26, 27, 28].”
BackgroundTransient Chaotic Dimensionality Expansion by Recurrent Networks
Keup, C., Kühn, T., Dahmen, D., & Helias, M. (2021) · Physical Review X · doi:10.1103/physrevx.11.021064 · Cited by 37
Minimum complexity integrated photonic architecture for delay-based reservoir computing
Abdalla, M., Zrounba, C., Cardoso, R., Jimenez, P., Ren, G., Boes, A., Mitchell, A., Bosio, A., O’Connor, I., & Pavanello, F. (2023) · Optics Express · doi:10.1364/oe.484052 · Cited by 35
Neural field theory of plasticity in the cerebral cortex
Fung, P. K., Haber, A. L., & Robinson, P. A. (2012) · Journal of Theoretical Biology · doi:10.1016/j.jtbi.2012.09.030 · Cited by 33
Chaos and Relaxation Oscillations in Spin-Torque Windmill Spiking Oscillators
Matsumoto, R., Lequeux, S., Imamura, H., & Grollier, J. (2019) · Physical Review Applied · doi:10.1103/physrevapplied.11.044093 · Cited by 30
Distributions of covariances as a window into the operational regime of neuronal networksPreprint
Dahmen, D., Diesmann, M., & Helias, M. (2016) · JuSER (Forschungszentrum Jülich) · doi:10.48550/arxiv.1605.04153 · Cited by 33
Quality of internal representation shapes learning performance in feedback neural networks
Susman, L., Mastrogiuseppe, F., Brenner, N., & Barak, O. (2021) · Physical Review Research · doi:10.1103/physrevresearch.3.013176 · Cited by 32
“oir [14,7,6]. Specically, performance is high when recurrent connections are strong, but not strong enough to lead to the appearance of chaotic activity [15] { a parameter region named edge-of-chaos [16]. Intuitively, the edge-of-chaos denes an optimal tradeo point where the internal reservoir dynamics are rich but st…”
BackgroundRevisiting the edge of chaos: Again?
Teuscher, C. (2022) · Biosystems · doi:10.1016/j.biosystems.2022.104693 · Cited by 35
Reservoir Computing and Self-Organized Neural Hierarchies
Lukoševičius, M. (2012) · Cited by 32
Echo State Networks with Self-Normalizing Activations on the Hyper-Sphere
Verzelli, P., Alippi, C., & Livi, L. (2019) · Scientific Reports · doi:10.1038/s41598-019-50158-4 · Cited by 30
Accessibility measure for eternal inflation: dynamical criticality and higgs metastability
Khoury, J. (2021) · Journal of Cosmology and Astroparticle Physics · doi:10.1088/1475-7516/2021/06/009 · Cited by 32
“mical behavior | the so-called \edge of chaos" [106]. This idea goes back to random boolean networks [107] and cellular automata [108{113]. In machine learning, certain recurrent neural networks [114, 115] achieve maximal computational power for vanishing Lyapunov exponent [116]. Meanwhile, the connectivity matri…”
BackgroundHandwritten digit recognition by spin waves in a Skyrmion reservoir
Lee, M.-K., & Mochizuki, M. (2023) · Scientific Reports · doi:10.1038/s41598-023-46677-w · Cited by 30
“It is generally suggested that when the system is located on the verge of the phase boundary between ordered and chaotic states (so-called edge of chaos), the highest nonlinearity manifests itself 53,54 .”
BackgroundMultiscale Model of an Inhibitory Network Shows Optimal Properties near Bifurcation
Buckley, C. L., & Nowotny, T. (2011) · Physical Review Letters · doi:10.1103/physrevlett.106.238109 · Cited by 25
Understanding Psychopathology: Cybernetics and Psychology on the Boundary between Order and Chaos
DeYoung, C. G., & Krueger, R. F. (2018) · Psychological Inquiry · doi:10.1080/1047840x.2018.1513690 · Cited by 33
“” The edge of chaos is a concept developed in the study of complex adaptive systems, such as organisms and some computational architectures (Bertschinger & Natschl € ager, 2004; Kauffman, 1993; Packard, 1988).”
BackgroundOptimal channel efficiency in a sensory network
Mosqueiro, T. S., & Maia, L. P. (2013) · Physical Review E · doi:10.1103/physreve.88.012712 · Cited by 25
Self‐Organization of Remote Reservoirs: Transferring Computation to Spatially Distant Locations
Tanaka, K., Tokudome, Y., Minami, Y., Honda, S., Nakajima, T., Takei, K., & Nakajima, K. (2021) · Advanced Intelligent Systems · doi:10.1002/aisy.202100166 · Cited by 26
Quantifying relevance in learning and inference
Marsili, M., & Roudi, Y. (2022) · Physics Reports · doi:10.1016/j.physrep.2022.03.001 · Cited by 28
“Tuning a network at a critical point is an accepted criterium for optimality in reservoir computing [60, 61].”
BackgroundResultRibeiro, T. L., & Copelli, M. (2008) · Physical Review E · doi:10.1103/physreve.77.051911 · Cited by 29
Vincent‐Lamarre, P., Lajoie, G., & Thivierge, J. (2016) · Journal of Computational Neuroscience · doi:10.1007/s10827-016-0619-3 · Cited by 26
“This model is resilient to local and global perturbations and conserves the computational properties offered by chaotic neural networks (Bertschinger and Natschläger 2004).”
BackgroundInfluentialPhysical reservoir computing for Edge AI applications
Liu, J., Feng, G., Li, W., Hao, S., Han, S., Zhu, Q., Tian, B., Duan, C., & Chu, J. (2025) · The Innovation Materials · doi:10.59717/j.xinn-mater.2025.100127 · Cited by 29
Deistler, M., Kadhim, K. L., Pals, M., Beck, J., Huang, Z., Gloeckler, M., Lappalainen, J. K., Schröder, C., Berens, P., Gonçalves, P. J., et al. (2025) · Nature Methods · doi:10.1038/s41592-025-02895-w · Cited by 63
Enhancing Performance of Reservoir Computing System Based on Coupled MEMS Resonators
Zheng, T., Yang, W., Sun, J., Xiong, X., Wang, Z., Li, Z., & Zou, X. (2021) · Sensors · doi:10.3390/s21092961 · Cited by 24
Inferring attracting basins of power system with machine learning
Du, Y., Li, Q., Fan, H., Zhan, M., Xiao, J., & Wang, X. (2024) · Physical Review Research · doi:10.1103/physrevresearch.6.013181 · Cited by 25
Naudé, J., Cessac, B., Berry, H., & Delord, B. (2013) · Journal of Neuroscience · doi:10.1523/jneurosci.0870-13.2013 · Cited by 32
In vivo spike-timing-dependent plasticity in the optic tectum of Xenopus laevis
Richards, B. A., Aizenman, C. D., & Akerman, C. J. (2010) · Frontiers in Synaptic Neuroscience · doi:10.3389/fnsyn.2010.00007 · Cited by 30
“, 2003), as well as helping networks to maintain dynamics that increase general computational power (Bertschinger and Natschläger, 2004; Maass and Markram, 2004).”
Self-organization and Emergence of Dynamical Structures in Neuromorphic Atomic Switch Networks
Stieg, A. Z., Avizienis, A. V., Sillin, H. O., Aguilera, R., Shieh, H.-H., Martin‐Olmos, C., Sandouk, E. J., Aono, M., & Gimzewski, J. K. (2013) · Springer eBooks · doi:10.1007/978-3-319-02630-5_10 · Cited by 27
Emergence of microfrequency comb via limit cycles in dissipatively coupled condensates
Kim, S., Rubo, Y. G., Liew, T. C. H., Brodbeck, S., Schneider, C., Höfling, S., & Deng, H. (2020) · Physical review. B./Physical review. B · doi:10.1103/physrevb.101.085302 · Cited by 23
“Scaling up the system to a lattice of condensate may provide a platform for efficient neuromorphic computing [8, 9] and simulation of many-body phase transitions [10].”
BackgroundCeni, A. (2025) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2025.3538924 · Cited by 37
“Moreover, a mean-field theory analysis on linear FNNs revealed the effectiveness of orthogonal weights initialization schemes [8], and provided that an NN operates close to the so-called edge of chaos [9], on nonlinear FNNs as well.”
Eigenvalue spectra of large correlated random matrices
Kuczala, A., & Sharpee, T. O. (2016) · Physical review. E · doi:10.1103/physreve.94.050101 · Cited by 27
Nozari, E., & Cortés, J. (2020) · IEEE Transactions on Automatic Control · doi:10.1109/tac.2020.3004801 · Cited by 26
Chaos may enhance expressivity in cerebellar granular layer
Tokuda, K., Fujiwara, N., Sudo, A., & Katori, Y. (2020) · Neural Networks · doi:10.1016/j.neunet.2020.12.020 · Cited by 27
Stability of motor cortex network states during learning-associated neural reorganizations
Ma, Z., Liu, H., Komiyama, T., & Weßel, R. (2020) · Journal of Neurophysiology · doi:10.1152/jn.00061.2020 · Cited by 24
Optimal Input Representation in Neural Systems at the Edge of Chaos
Morales, G. B., & Muñoz, M. A. (2021) · Biology · doi:10.3390/biology10080702 · Cited by 36
Gell-Mann–Low Criticality in Neural Networks
Tiberi, L., Stapmanns, J., Kühn, T., Luu, T., Dahmen, D., & Helias, M. (2022) · Physical Review Letters · doi:10.1103/physrevlett.128.168301 · Cited by 33
Mosheiff, N., Ermentrout, B., & Huang, C. (2023) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1010843 · Cited by 24
“…high computational capabilities because of their rich reservoir of internal dynamics that can be utilized for complex computations and efficient training (Bertschinger and Natschläger, 2004; Jaeger and Haas, 2004; Laje and Buonomano, 2013; Sussillo and Abbott, 2009; Toyoizumi and Abbott, 2011).”
BackgroundPaul, K., Cauller, L. J., & Llano, D. A. (2016) · Frontiers in Computational Neuroscience · doi:10.3389/fncom.2016.00091 · Cited by 22
“…the network that enables it to respond very rapidly to external input, much faster than the time constant of individual neurons, and therefore benefits neural processing (van Vreeswijk and Sompolinsky, 1996; Bertschinger and Natschläger, 2004; Canavier and Shepard, 2009; Sussillo and Abbott, 2009).”
BackgroundInfluentialLinear response in neuronal networks: From neurons dynamics to collective response
Cessac, B. (2019) · Chaos An Interdisciplinary Journal of Nonlinear Science · doi:10.1063/1.5111803 · Cited by 24
Inflation of the edge of chaos in a simple model of gene interaction networks
Stokić, D., Hanel, R., & Stefan Thurner, A. (2008) · Physical Review E · doi:10.1103/physreve.77.061917 · Cited by 22
Complex dynamics of a bi-directional N-type locally-active memristor
Dong, Y., Wang, G., Liang, Y., & Chen, G. (2021) · Communications in Nonlinear Science and Numerical Simulation · doi:10.1016/j.cnsns.2021.106086 · Cited by 22
Minati, L., Bartels, J., Li, C., Frasca, M., & Ito, H. (2022) · Chaos Solitons & Fractals · doi:10.1016/j.chaos.2022.112415 · Cited by 24
Tipping Point Detection Using Reservoir Computing
Li, X., Zhu, Q., Zhao, C., Qian, X., Zhang, X., Duan, X., & Lin, W. J. (2023) · Research · doi:10.34133/research.0174 · Cited by 33
“Naturally, the RC framework has good performances in pattern classification [31,32], time series forecasting [33,34], and system approximation [35].”
Maksymov, I. S. (2023) · Energies · doi:10.3390/en16145366 · Cited by 29
Unveiling causal activity of complex networks
Williams-García, R. V., Beggs, J. M., & Ortíz, G. (2017) · Europhysics Letters (EPL) · doi:10.1209/0295-5075/119/18003 · Cited by 25
Bojanek, K., Zhu, Y., & MacLean, J. N. (2020) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1007409 · Cited by 23
“For example, networks tuned near the critical point display maximum information transmission [13], information storage [16], and computational power [17].”
BackgroundZheng, T., Yang, W., Sun, J., Xiong, X., Li, Z., & Zou, X. (2021) · Scientific Reports · doi:10.1038/s41598-020-80339-5 · Cited by 21
Memory Capacity of Input-Driven Echo State Networks at the Edge of Chaos
Barančok, P., & Farkaš, I. (2014) · Lecture notes in computer science · doi:10.1007/978-3-319-11179-7_6 · Cited by 33
“The common way how to determine whether a dynamical system has ordered or chaotic dynamics, is to look at the average sensitivity to perturbations of the initial conditions [1,3].”
BackgroundSelf‐organized Criticality in Neuromorphic Nanowire Networks With Tunable and Local Dynamics
Michieletti, F., Pilati, D., Milano, G., & Ricciardi, C. (2025) · Advanced Functional Materials · doi:10.1002/adfm.202423903 · Cited by 25
Ly, A., & Gong, P. (2025) · Nature Communications · doi:10.1038/s41467-025-58532-9 · Cited by 39
“(Remarks on code availability) Version 1:”
Gerum, R. C., & Schilling, A. (2021) · Neural Computation · doi:10.1162/neco_a_01424 · Cited by 25
“…for untrained or randomly connected neural networks such as stability analysis (Liapunov exponent) or motif distribution analysis (see e.g. (Bertschinger and Natschläger, 2004; Krauss et al., 2019c,a,b)), can be applied to the spiking neural networks trained with backpropagation to gain…”
Method
Metadata
- Authors
- Bertschinger, N., & Natschläger, T.
- Affiliations
- Graz University of Technology (AT)
- Software Competence Center Hagenberg (Austria) (AT)
- ORCID
- Bertschinger, Nils
- Venue
- Neural Computation
- Publisher
- MIT Press - Journals
- Published
- 2004-07-01
- Volume
- 16
- Issue
- 7
- Pages
- 1413-1436
- ISSN
- 0899-7667 (ISSN-L)
- 1530-888X
- OpenAlex
- W2057755457
- Cited by
- 833
- References
- 26
- Citations by year
- 2026: 45 · 2025: 73 · 2024: 61 · 2023: 48 · 2022: 66 · 2021: 73 · 2020: 52 · 2019: 53 · 2018: 47 · 2017: 48 · 2016: 26 · 2015: 33 · 2014: 29 · 2013: 33 · 2012: 27
- FWCI
- 13.53
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Neural Computation (publishedVersion)
- PubMed (publishedVersion)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Neural Networks and Applications
- Physical Sciences › Computer Science › Computational Theory and Mathematics › Cellular Automata and Applications
- Physical Sciences › Computer Science › Artificial Intelligence › Neural Networks and Reservoir Computing
- Keywords
- edge of chaos, recurrent neural networks, critical connectivity, threshold gates, random connectivity, real-time processing, dynamical systems
- MeSH terms
- Animals, Artificial Intelligence, Computer Simulation, Feedback, Memory, Models, Neurological, Time Factors, Neural Networks, Computer, Nonlinear Dynamics
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.