Real-Time Computation at the Edge of Chaos in Recurrent Neural Networks
The DOI matches its Crossref record; 26 of the 26 references it cites were found in a registry or an index.
Abstract
Depending on the connectivity, recurrent networks of simple computational units can show very different types of dynamics, ranging from totally ordered to chaotic. We analyze how the type of dynamics (ordered or chaotic) exhibited by randomly connected networks of threshold gates driven by a time-varying input signal depends on the parameters describing the distribution of the connectivity matrix. In particular, we calculate the critical boundary in parameter space where the transition from ordered to chaotic dynamics takes place. Employing a recently developed framework for analyzing real-time computations, we show that only near the critical boundary can such networks perform complex computations on time series. Hence, this result strongly supports conjectures that dynamical systems that are capable of doing complex computational tasks should operate near the edge of chaos, that is, the transition from ordered to chaotic dynamics.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Intrinsic fluctuations that are poised ''at the edge of chaos'' are thought to maximize the computational efficiency of such systems and the flexibility to switch between adaptive regimes (Bertschinger & Natschla ¨ger, 2004).”
From A potential mechanism for Gibsonian resonance: behavioral entrainment emerges from local homeostasis in an unsupervised reservoir network (2024) · cites this paper as Bertschinger & Natschläger, 2004
In the citing paper
“Finally, we show that a randomly connected circuit model, akin to a “reservoir” network13-15, can reproduce the observed distribution of timescales, provided that the network operates at the critical point (or “edge of chaos”)16-18.”
From A reservoir of time constants for memory traces in cortical neurons (2011) · cites this paper as [17]
Citations
Cites
Self-organized criticality: An explanation of the 1/f noise
Bak, P., Tang, C., & Wiesenfeld, K. (1987) · Physical Review Letters · doi:10.1103/physrevlett.59.381 · Cited by 7,591
The origins of order; self organization and selection in evolution
(1994) · International Journal of Biochemistry · doi:10.1016/0020-711x(94)90119-8 · Cited by 6,649
Introduction to the theory of neural computation
(1994) · Neural Networks · doi:10.1016/0893-6080(94)90014-0 · Cited by 6,401
Bak, P., Tang, C., & Wiesenfeld, K. (1988) · Physical Review A · doi:10.1103/physreva.38.364 · Cited by 4,713
Maass, W., Natschläger, T., & Markram, H. (2002) · Neural Computation · doi:10.1162/089976602760407955 · Cited by 4,025 · cited 10×
“(Maass et al., 2002) and indepenently by Jaeger (Jaeger, 2002) who called it “echo state networks”.”
BackgroundInfluentialSelf-citationChaos in Neuronal Networks with Balanced Excitatory and Inhibitory Activity
van Vreeswijk, C., & Sompolinsky, H. (1996) · Science · doi:10.1126/science.274.5293.1724 · Cited by 1,811
Computation at the edge of chaos: Phase transitions and emergent computation
Langton, C. G. (1990) · Physica D: Nonlinear Phenomena · doi:10.1016/0167-2789(90)90064-v · Cited by 1,527 · cited 5×
“These results provide further evidence for the idea of “computation at the edge of chaos” (Langton, 1990).”
BackgroundResultInfluentialFading memory and the problem of approximating nonlinear operators with Volterra series
Boyd, S., & Chua, L. (1985) · IEEE Transactions on Circuits and Systems · doi:10.1109/tcs.1985.1085649 · Cited by 991
Random Networks of Automata: A Simple Annealed Approximation
Derrida, B., & Pomeau, Y. (1986) · Europhysics Letters (EPL) · doi:10.1209/0295-5075/1/2/001 · Cited by 706
Dynamic Patterns in Complex Systems
(1988) · doi:10.1142/0534 · Cited by 548
Research works printed but not linked: 1
- Mitchell M., 1993, Complex Systems, 7, 89
Cited by
Items 101–200 of 831
Massobrio, P., Pasquale, V., & Martinoia, S. (2015) · Scientific Reports · doi:10.1038/srep10578 · Cited by 110
Scheffer, M., Bascompte, J., Bjordam, T. K., Carpenter, S. R., Clarke, L. B., Folke, C., Marquet, P. A., Mazzeo, N., Meerhoff, M., Sala, O. E., et al. (2015) · Ecology and Society · doi:10.5751/es-07434-200203 · Cited by 109
Computational significance of transient dynamics in cortical networks
Durstewitz, D., & Deco, G. (2008) · European Journal of Neuroscience · doi:10.1111/j.1460-9568.2007.05976.x · Cited by 92
“However, the models of computation in such systems may turn out to be quite different from the classical attractor neural network approaches (Tsuda, 2001; Afraimovich et al., 2004; Bertschinger & Natschläger, 2004; Legenstein & Maas, 2007).”
BackgroundQuasicritical brain dynamics on a nonequilibrium Widom line
Williams-García, R. V., Moore, M., Beggs, J. M., & Ortíz, G. (2014) · Physical Review E · doi:10.1103/physreve.90.062714 · Cited by 102
Optimal Sequence Memory in Driven Random Networks
Schuecker, J., Goedeke, S., & Helias, M. (2018) · Physical Review X · doi:10.1103/physrevx.8.041029 · Cited by 158
Sun, Y., Lin, T., Lei, N., Chen, X., Kang, W., Zhao, Z., Wei, D., Chen, C., Pang, S., Hu, L., et al. (2023) · Nature Communications · doi:10.1038/s41467-023-39207-9 · Cited by 104
“The reservoir ’ s computational capabilities are evaluated by further analyzing the experimental data of the waveform classi fi cation task 61,62 , where T delay, max = 2 is used in the calculation since the experimental results of the waveform classi fi cation only comprise the current, the last,…”
Exploring the spectrum of dynamical regimes and timescales in spontaneous cortical activity
Mattia, M., & Sanchez-Vives, M. V. (2011) · Cognitive Neurodynamics · doi:10.1007/s11571-011-9179-4 · Cited by 103
Computational analysis of memory capacity in echo state networks
Farkaš, I., Bosák, R., & Gergeľ, P. (2016) · Neural Networks · doi:10.1016/j.neunet.2016.07.012 · Cited by 88
“In Bertschinger and Natschläger (2004) it was found that networks were able to exhibit long memories if they operated near the edge of chaos, at the critical border between a stable (ordered) and an unstable (chaotic) dynamics regime. The idea that temporal memory is longest near the edge of chaos was confirmed in sub…”
BackgroundMethodInfluentialDynamical Criticality in the Collective Activity of a Population of Retinal Neurons
Mora, T., Deny, S., & Marre, O. (2015) · Physical Review Letters · doi:10.1103/physrevlett.114.078105 · Cited by 100
“Thinking about the brain as a system near a critical point has been an attractive idea, which has gained attention after the suggestion that such critical states could be achieved in a self-organized manner, without fine-tuning [3], but also the proposal that operating near a critical point could be beneficial for com…”
BackgroundCriticality meets learning: Criticality signatures in a self-organizing recurrent neural network
Papa, B. D., Priesemann, V., & Triesch, J. (2017) · PLoS ONE · doi:10.1371/journal.pone.0178683 · Cited by 88
Neutral Theory and Scale-Free Neural Dynamics
Martinello, M., Hidalgo, J., Maritan, A., Santo, S. D., Plenz, D., & Muñoz, M. A. (2017) · Physical Review X · doi:10.1103/physrevx.7.041071 · Cited by 88
“For instance, the introduction of a novel species within an established population triggers a random cascade of changes, or “avalanche”, which –as a result of the implicit neutrality– does not have an inherent tendency to neither shrink nor to expand at the expenses of others.”
BackgroundOptimal Sequence Memory in Driven Random Networks
Schücker, J., Goedeke, S., & Helias, M. (2018) · JuSER (Forschungszentrum Jülich) · doi:10.18154/rwth-2018-231046 · Cited by 80
The resilient brain and the guardians of sleep: New perspectives on old assumptions
Parrino, L., & Vaudano, A. E. (2017) · Sleep Medicine Reviews · doi:10.1016/j.smrv.2017.08.003 · Cited by 86
Wilting, J., & Priesemann, V. (2019) · Cerebral Cortex · doi:10.1093/cercor/bhz049 · Cited by 96
“In contrast, criticality in models optimizes performance in tasks that profit from extended reverberations of activity in the network (Bertschinger and Natschl¨ager, 2004, Haldeman and Beggs, 2005, Kinouchi and Copelli, 2006, Wang et al., 2011, Boedecker et al., 2012, Shew and Plenz, 2013, Del Papa…”
BackgroundSelf-Organization Toward Criticality by Synaptic Plasticity
Zeraati, R., Priesemann, V., & Levina, A. (2021) · Frontiers in Physics · doi:10.3389/fphy.2021.619661 · Cited by 91
“n lengths [138,151]. In recurrent network models and experiments, such boost of input sensitivity and memory have been demonstrated by tuning networks systematically towards and away from criticality [14,20,21,76,139{141]. When not explicitly incorporating a mechanism that drives the network to criticality, learning n…”
BackgroundThe fractal brain: scale-invariance in structure and dynamics
Grosu, G. F., Hopp, A. V., Moca, V. V., Bârzan, H., Ciuparu, A., Ercsey-Ravasz, M., Winkel, M., Linde, H., & Muresan, R. C. (2022) · Cerebral Cortex · doi:10.1093/cercor/bhac363 · Cited by 101
“Importantly, critical cortical circuits are able to vigorously merge perturbations induced by sensory changes with their ongoing dynamical context (Arieli et al. 1996; Ahissar and Arieli 2001; Bertschinger and Natschläger 2004; VanRullen et al. 2006).”
BackgroundResultInfluentialInteractions between Intrinsic and Stimulus-Evoked Activity in Recurrent Neural Networks
Abbott, L. F., Rajan, K., & Sompolinsky, H. (2011) · Oxford University Press eBooks · doi:10.1093/acprof:oso/9780195393798.003.0004 · Cited by 98
Reproducible Sequence Generation In Random Neural Ensembles
Huerta, R., & Rabinovich, M. (2004) · Physical Review Letters · doi:10.1103/physrevlett.93.238104 · Cited by 77
Fading memory and time series prediction in recurrent networks with different forms of plasticity
Lazăr, A., Pipa, G., & Triesch, J. (2007) · Neural Networks · doi:10.1016/j.neunet.2007.04.020 · Cited by 99
“Understanding how patterns of neural activity in the highly recurrent cortical architecture encode information and how the activity dynamics give rise to purposeful computation is a central goal of neuroscience (e.g., Abeles (1991) , Douglas and Martin (1991), Rieke, Bialek, Warland, and Van Steveninck (1999), Singer…”
On Dynamics of Integrate-and-Fire Neural Networks with Conductance Based Synapses
Cessac, B., & Viéville, T. (2008) · Frontiers in Computational Neuroscience · doi:10.3389/neuro.10.002.2008 · Cited by 78
“Further developments with a time dependent current, i.e. in the framework of an input-output computation (Bertschinger & Natschl¨ager, 2004), will be considered next.”
BackgroundInfluentialNeuronal avalanches and coherence potentials
Plenz, D. (2012) · The European Physical Journal Special Topics · doi:10.1140/epjst/e2012-01575-5 · Cited by 77
Memory in linear recurrent neural networks in continuous time
Hermans, M., & Schrauwen, B. (2009) · Neural Networks · doi:10.1016/j.neunet.2009.08.008 · Cited by 75
Encoding sensory and motor patterns as time-invariant trajectories in recurrent neural networks
Goudar, V., & Buonomano, D. V. (2018) · eLife · doi:10.7554/elife.31134 · Cited by 81
Evolutionary aspects of reservoir computing
Seoane, L. F. (2019) · Philosophical Transactions of the Royal Society B: Biological Sciences · doi:10.1098/rstb.2018.0377 · Cited by 74
“A similar phenomenon was noted in computer science studying cellular automata and random Boolean networks [19, 73, 95–98].”
BackgroundMethodLyapunov spectra of chaotic recurrent neural networks
Engelken, R., Wolf, F., & Abbott, L. F. (2023) · Physical Review Research · doi:10.1103/physrevresearch.5.043044 · Cited by 143
“Some studies proposed that computational features are favorable near to or even slightly beyond the so-called edge of chaos in the chaotic regime [11,15,22–27].”
BackgroundImplementing a Magnonic Reservoir Computer Model Based on Time-Delay Multiplexing
Watt, S., Kostylev, M., Ustinov, A. B., & Kalinikos, B. A. (2021) · Physical Review Applied · doi:10.1103/physrevapplied.15.064060 · Cited by 87
“To evaluate the performance of the active-ring resonator system as a RC, we employ the short-term memory task (abbreviated earlier as STM) [32] and the parity check task (abbreviated earlier as PC) [6, 33].”
MethodSynaptic Plasticity Enables Adaptive Self-Tuning Critical Networks
Stepp, N. D., Plenz, D., & Srinivasa, N. (2015) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1004043 · Cited by 78
“Theory predicts that networks with critical dynamics optimize numerous aspects of information processing [9].”
BackgroundExperimental reservoir computing using VCSEL polarization dynamics
Vatin, J., Rontani, D., & Sciamanna, M. (2019) · Optics Express · doi:10.1364/oe.27.018579 · Cited by 88
Criticality Maximizes Complexity in Neural Tissue
Timme, N. M., Marshall, N. J., Bennett, N., Ripp, M., Lautzenhiser, E., & Beggs, J. M. (2016) · Frontiers in Physiology · doi:10.3389/fphys.2016.00425 · Cited by 94
“…important implications for the brain if it is indeed operating at or near a critical point, such as optimal communication (Beggs and Plenz, 2003; Bertschinger and Natschlager, 2004; Ramo et al., 2007; Tanaka et al., 2009; Shew et al., 2011), information storage (Socolar and Kauffman, 2003;…”
BackgroundInfluentialCloser to critical resting-state neural dynamics in individuals with higher fluid intelligence
Ezaki, T., dos Reis, E. F., Watanabe, T., Sakaki, M., & Masuda, N. (2020) · Communications Biology · doi:10.1038/s42003-020-0774-y · Cited by 81
Li, M., Han, Y., Aburn, M. J., Breakspear, M. J., Poldrack, R. A., Shine, J. M., & Lizier, J. T. (2019) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1006957 · Cited by 78
“(with respect to perturbation propagation in reservoir dynamics) [35], suggesting that these intrinsic information processing advantages underpinned the known [36] higher performance of similar networks near the critical point on various computational tasks.”
BackgroundZhang, H., & Vargas, D. V. (2023) · IEEE Access · doi:10.1109/access.2023.3299296 · Cited by 108
Deep Echo State Network (DeepESN): A Brief SurveyPreprint
Gallicchio, C., & Micheli, A. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1712.04323 · Cited by 130
“This study also provided interesting insights in terms of the quality of the developed system dynamics, showing that (under simple initialization settings) layering has the effect of naturally pushing the global dynamical regime of the recurrent network closer to the stable-unstable transition condition known as the e…”
BackgroundHabibollahi, F., Kagan, B. J., Duc, D., Burkitt, A. N., & French, C. R. (2023) · Nature Communications · doi:10.1038/s41467-023-41020-3 · Cited by 88
Maintained avalanche dynamics during task-induced changes of neuronal activity in nonhuman primates
Yu, S., Ribeiro, T. L., Meisel, C., Chou, S., Mitz, A. R., Saunders, R. C., & Plenz, D. (2017) · eLife · doi:10.7554/elife.27119 · Cited by 75
“…(Nykter et al., 2008) and dynamic range (Kinouchi and Copelli, 2006; Shew et al., 2009; Gautam et al., 2015), optimized computational capabilities (Bertschinger and Natschläger, 2004), information transmission (Beggs and Plenz, 2003; Rämö et al., 2007; Fagerholm et al., 2016), sensory…”
BackgroundInfluentialSoft-assembly of sensorimotor function.
Kello, C. T., & Orden, G. C. V. (2009) · PubMed · Cited by 69
Microfabricated Neuroaccelerometer: Integrating Sensing and Reservoir Computing in MEMS
Barazani, B., Dion, G., Morissette, J.-F., Beaudoin, L., & Sylvestre, J. (2020) · Journal of Microelectromechanical Systems · doi:10.1109/jmems.2020.2978467 · Cited by 74
“The parity benchmark allows a straightforward comparison of the neuroaccelerometer computing capabilities with other systems [8], [9], [30], [31], [32], [33], [34].”
MethodLivi, L., Bianchi, F. M., & Alippi, C. (2017) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2016.2644268 · Cited by 77
“Experimental results on a multitude of application contexts suggest that RNNs achieve the highest information processing capabilities exactly on the edge of this transition, resulting in high memory capacity (storage of past events) and good performance on the modeling/prediction task at hand (low prediction errors) […”
BackgroundDeep Randomized Neural Networks
Gallicchio, C., & Scardapane, S. (2020) · Studies in computational intelligence · doi:10.1007/978-3-030-43883-8_3 · Cited by 103
Optimal Information Representation and Criticality in an Adaptive Sensory Recurrent Neuronal Network
Shriki, O., & Yellin, D. (2016) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1004698 · Cited by 74
“Furthermore, in contrast to [21] and [18], the supercritical regime in the present model does not correspond to chaotic behavior.”
ResultLocal Lyapunov exponents of deep echo state networks
Gallicchio, C., Micheli, A., & Silvestri, L. (2018) · Neurocomputing · doi:10.1016/j.neucom.2017.11.073 · Cited by 75
Sampled-data synchronization of delayed multi-agent networks and its application to coupled circuit
Gunasekaran, N., Zhai, G., & Yu, Q. (2020) · Neurocomputing · doi:10.1016/j.neucom.2020.05.060 · Cited by 65
Dynamical Basis of Irregular Spiking in NMDA-Driven Prefrontal Cortex Neurons
Durstewitz, D., & Gabriel, T. (2007) · Cerebral Cortex · doi:10.1093/cercor/bhk044 · Cited by 65
“If PFC networks would reside in a close to or slightly chaotic regime with complex deterministic temporal structure, this may strongly facilitate the processing of temporal sequences (Bertschinger and Natschläger, 2004; Jaeger and Haas, 2004).”
BackgroundSiri, B., Quoy, M., Delord, B., Cessac, B., & Berry, H. (2007) · Journal of Physiology-Paris · doi:10.1016/j.jphysparis.2007.10.003 · Cited by 65
“…dynamical regime, at the frontier between order (periodical or fixed point regimes) and disorder (chaos), has already be reported to be particularly suitable for recurrent neural networks, especially when computational power is considered (Soula et al., 2005; Bertschinger and Natschlager, 2004).”
BackgroundOptimization by Self-Organized Criticality
Hoffmann, H., & Payton, D. W. (2018) · Scientific Reports · doi:10.1038/s41598-018-20275-7 · Cited by 65
“For example, Bertschinger and Natschläger 10 show that recurrent neural networks perform best at the critical state; there, however, the task was simple (3-bit parity), and a tuning parameter was required to achieve the critical state.”
BackgroundTheory of Gating in Recurrent Neural Networks
Krishnamurthy, K., Can, T., & Schwab, D. J. (2022) · Physical Review X · doi:10.1103/physrevx.12.011011 · Cited by 94
Critical branching neural networks.
Kello, C. T. (2013) · Psychological Review · doi:10.1037/a0030970 · Cited by 63
“Subsequent studies have further explicated this hypothesis (Mitchell, Hraber, & Crutchfield, 1999) and have developed other so-called “edge of chaos” models more akin to neural networks (Bertschinger & Natschlager, 2004; Kwok & Smith, 2005).”
BackgroundInfluentialHuyck, C., & Passmore, P. (2013) · Biological Cybernetics · doi:10.1007/s00422-013-0555-5 · Cited by 75
“There is some evidence for small world topologies (Bullmore and Sporns, 2009), and random topologies (Bertschinger and Natschlager, 2004; Valiant, 2005) have also been used in simulations; both have the benefit of existing mathematical expertise, and both are more realistic than full connectivity.”
BackgroundECG-based Heartbeat Classification in Neuromorphic Hardware
Corradi, F., Pande, S., Stuijt, J., Qiao, N., Schaafsma, S., Indiveri, G., & Catthoor, F. (2019) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn.2019.8852279 · Cited by 62
“Therefore we have selected to work in a state in which there is some balance among excitatory and inhibitory contributions [29].”
BackgroundInput-driven bifurcations and information processing capacity in spintronics reservoirs
Akashi, N., Yamaguchi, T., Tsunegi, S., Taniguchi, T., Nishida, M., Sakurai, R., Wakao, Y., & Nakajima, K. (2020) · Physical Review Research · doi:10.1103/physrevresearch.2.043303 · Cited by 63
Optimal modularity and memory capacity of neural reservoirs
Rodriguez, N., Izquierdo, E. J., & Ahn, Y. (2019) · Network Neuroscience · doi:10.1162/netn_a_00082 · Cited by 77
Bao, H., Zhu, D., Liu, W., Xu, Q., Chen, M., & Bao, B. (2020) · International Journal of Bifurcation and Chaos · doi:10.1142/s0218127420500455 · Cited by 59
Criticality, Connectivity, and Neural Disorder: A Multifaceted Approach to Neural Computation
Heiney, K., Ramstad, O. H., Fiskum, V., Christiansen, N., Sandvig, A., Nichele, S., & Sandvig, I. (2021) · Frontiers in Computational Neuroscience · doi:10.3389/fncom.2021.611183 · Cited by 62
“…recurrent network models showing critical dynamics outperform their sub-and supercritical counterparts in terms of their input-to-output mappings; that is, the outputs produced from different inputs are more separable, or distinguishable, in critical networks (Bertschinger and Natschläger, 2004).”
BackgroundA local Echo State Property through the largest Lyapunov exponent
Wainrib, G., & Galtier, M. N. (2016) · Neural Networks · doi:10.1016/j.neunet.2015.12.013 · Cited by 67
“We do not claim that the edge of chaos is always the best regime, but it has been shown that for some applications, typically requiring a lot of memory, it was optimal [36].”
BackgroundBertschinger, N., Natschläger, T., & Legenstein, R. A. (2004) · neural information processing systems · Cited by 60
Self-citationDynamical systems, attractors, and neural circuits
Miller, P. (2016) · F1000Research · doi:10.12688/f1000research.7698.1 · Cited by 59
“Yet a system operating near or at the “edge of chaos” can be computationally efficient 93 and become reliably entrained to inputs while responding more rapidly than ordered systems 89,94 .”
BackgroundEdge of Chaos and Avalanches in Neural Networks with Heavy-Tailed Synaptic Weight Distribution
Kuśmierz, Ł., Ogawa, S., & Toyoizumi, T. (2020) · Physical Review Letters · doi:10.1103/physrevlett.125.028101 · Cited by 76
Improving liquid state machines through iterative refinement of the reservoir
Norton, D. P., & Ventura, D. A. (2010) · Neurocomputing · doi:10.1016/j.neucom.2010.08.005 · Cited by 67
“For more information, please contact scholarsarchive@byu.edu, ellen_amatangelo@byu.edu.”
Thermal Management in Neuromorphic Materials, Devices, and Networks
Torres, F., Basaran, A. C., & Schuller, I. K. (2023) · Advanced Materials · doi:10.1002/adma.202205098 · Cited by 61
Reservoir Computing as a Model for In-Materio Computing
Dale, M., Miller, J. F., & Stepney, S. (2016) · Emergence, complexity and computation · doi:10.1007/978-3-319-33924-5_22 · Cited by 61
“Early measurements of dynamics in reservoir-like systems can be found in [11] where, using a similar technique to Derrida & Pomeau [26], dynamic behaviour is measured using the Hamming distances between two output states.”
BackgroundMethodInfluentialFundamental physics and applications of skyrmions: A review
Wang, K., Bheemarasetty, V. S., Duan, J., Zhou, S., & Xiao, G. (2022) · Journal of Magnetism and Magnetic Materials · doi:10.1016/j.jmmm.2022.169905 · Cited by 67
Cortés, J. M., Desroches, M., Rodrigues, S., Veltz, R., Muñoz, M. A., & B, T. J. S. A. (2013) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1316071110 · Cited by 69
Spatiotemporal Pattern Recognition via Liquid State Machines
Goodman, E. L., & Ventura, D. A. (2006) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn.2006.246880 · Cited by 62
“Here, we explore the application of liquid state machines (LSMs) [8] [9] [10] [11] [12] to spatiotemporal pattern recognition.”
Connectome-based reservoir computing with the conn2res toolbox
Suárez, L. E., Mihalik, Á., Milisav, F., Marshall, K., Li, M., Vértes, P. E., Lajoie, G., & Mišić, B. (2024) · Nature Communications · doi:10.1038/s41467-024-44900-4 · Cited by 83
“By parametrically tuning the dynamics to be closer or further from criticality, RC allows us to investigate the effects of qualitatively different neural trajectories near this critical point on the computational performance of the reservoir 47 – 49 .”
Marginally subcritical dynamics explain enhanced stimulus discriminability under attention
Tömen, N., Rotermund, D., & Ernst, U. A. (2014) · Frontiers in Systems Neuroscience · doi:10.3389/fnsys.2014.00151 · Cited by 61
“the idea that a functional relationship may exist between critical dynamics and optimality of information processing as well as information transmission (Bertschinger and Natschläger, 2004; Haldeman and Beggs, 2005; Kinouchi and Copelli, 2006; Nykter et al., 2008; Shew et al., 2009).”
BackgroundEvolving Carbon Nanotube Reservoir Computers
Dale, M., Miller, J. F., Stepney, S., & Trefzer, M. A. (2016) · Lecture notes in computer science · doi:10.1007/978-3-319-41312-9_5 · Cited by 73
“To do the latter, we will exploit a number of proposed metrics [3, 8, 9].”
BackgroundSparsity through evolutionary pruning prevents neuronal networks from overfitting
Gerum, R. C., Erpenbeck, A., Krauß, P., & Schilling, A. (2020) · Neural Networks · doi:10.1016/j.neunet.2020.05.007 · Cited by 64
Information dynamics in neuromorphic nanowire networks
Zhu, R., Hochstetter, J., Loeffler, A., Álvarez, A. D., Nakayama, T., Lizier, J. T., & Kuncic, Z. (2021) · Scientific Reports · doi:10.1038/s41598-021-92170-7 · Cited by 74
Reservoir Computing with Spin Waves in a Skyrmion Crystal
Lee, M.-K., & Mochizuki, M. (2022) · Physical Review Applied · doi:10.1103/physrevapplied.18.014074 · Cited by 62
On Computational Power and the Order-Chaos Phase Transition in Reservoir Computing
Schrauwen, B., Buesing, L., & Legenstein, R. A. (2008) · Ghent University Academic Bibliography (Ghent University) · Cited by 66
Direct neural perturbations reveal a dynamical mechanism for robust computationPreprint
O’Shea, D. J., Duncker, L., Goo, W., Sun, X., Vyas, S., Trautmann, E. M., Diester, I., Ramakrishnan, C., Deisseroth, K. A., Sahani, M., et al. (2022) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2022.12.16.520768 · Cited by 71
Rosselló, J. L., Canals, V., Oliver, A., & Morro, A. (2013) · International Journal of Neural Systems · doi:10.1142/s0129065714300034 · Cited by 56
Morales, G. B., Santo, S. D., & Muñoz, M. A. (2023) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2208998120 · Cited by 74
“This type of behavior —called traditionally ”edge of chaos” or simply ”type-II criticality” in [39]— has long been (since the pioneering works of Langton and others [73, 74]) theoretically conjectured to be crucial for information processing in natural and artificial neural networks.”
BackgroundMean-field theory of echo state networks
Massar, M., & Massar, S. (2013) · Physical Review E · doi:10.1103/physreve.87.042809 · Cited by 58
Implications of synaptic biophysics for recurrent network dynamics and active memory
Durstewitz, D. (2009) · Neural Networks · doi:10.1016/j.neunet.2009.07.016 · Cited by 60
Criticality and avalanches in neural networks
Zare, M., & Grigolini, P. (2013) · Chaos Solitons & Fractals · doi:10.1016/j.chaos.2013.05.009 · Cited by 55
“Although finding a proper theoretical model of these dynamics is a challenging issue, some empirical results including avalanche analysis [1, 2], dynamic range [3], information storage [3, 4, 5], information transmission [6, 7] and computational power [8, 9, 10, 11, 12] suggest that brain may work near criticality.”
BackgroundSignatures of criticality arise from random subsampling in simple population models
Nonnenmacher, M., Behrens, C., Berens, P., Bethge, M., & Macke, J. H. (2017) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1005718 · Cited by 48
“It has been argued that such temporal dynamics might be beneficial for neural computation and communication [20,58,59] (see [5] for an overview).”
BackgroundOperating in a Reverberating Regime Enables Rapid Tuning of Network States to Task Requirements
Wilting, J., Dehning, J., Neto, J. P., Rudelt, L., Wibral, M., Zierenberg, J., & Priesemann, V. (2018) · Frontiers in Systems Neuroscience · doi:10.3389/fnsys.2018.00055 · Cited by 65
“In contrast, networks that maintain input for long timescales may be slow at responding to novel input, but instead they can integrate information and input over extended time periods [5, 6, 7, 8].”
BackgroundPerspective on unconventional computing using magnetic skyrmions
Lee, O., Msiska, R., Brems, M. A., Kläui, M., Kurebayashi, H., & Everschor‐Sitte, K. (2023) · Applied Physics Letters · doi:10.1063/5.0148469 · Cited by 60
Breathing frequency bias in fractal analysis of heart rate variability
Perakakis, P., Taylor, M. J. H., Martinez-Nieto, E., Revithi, I., & Vila, J. (2009) · Biological Psychology · doi:10.1016/j.biopsycho.2009.06.004 · Cited by 55
“Finally , fractal networks generate parallel trajectories in phase space , which means that despite increased variability , their dynamical evolution is still stable and controllable with minor corrective inputs ( Bertschinger and Natschlager , 2004 ; Haldeman and Beggs , 1998 ) .”
BackgroundInfluentialEnhancing computational performance of a spin-wave reservoir computer with input synchronization
Watt, S., Kostylev, M., & Ustinov, A. B. (2021) · Journal of Applied Physics · doi:10.1063/5.0033292 · Cited by 49
Advances in the Application of Technology to Epilepsy: The CIMIT/NIO Epilepsy Innovation Summit
Schachter, S. C., Guttag, J. V., Schiff, S. J., & Schomer, D. L. (2009) · Epilepsy & Behavior · doi:10.1016/j.yebeh.2009.06.028 · Cited by 51
“[11] Gallagher A, Lassonde M, Bastien D, et al.”
InfluentialLearning recurrent dynamics in spiking networks
Kim, C. M., & Chow, C. C. (2018) · eLife · doi:10.7554/elife.37124 · Cited by 51
Does the brain behave like a (complex) network? I. Dynamics
Papo, D., & Buldú, J. M. (2023) · Physics of Life Reviews · doi:10.1016/j.plrev.2023.12.006 · Cited by 47
Optimal Balance of the Striatal Medium Spiny Neuron Network
Ponzi, A. P. D., & Wickens, J. R. (2013) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1002954 · Cited by 48
“It has been suggested to optimize information transmission [70–73], sensitivity to sensory stimuli and dynamic range [50,74], or memory size and computational abilities [38].”
BackgroundMethodLearning Temporal Quantum Tomography
Tran, Q. H., & Nakajima, K. (2021) · Physical Review Letters · doi:10.1103/physrevlett.127.260401 · Cited by 56
Spintronic reservoir computing without driving current or magnetic field
Taniguchi, T., Ogihara, A., Utsumi, Y., & Tsunegi, S. (2022) · Scientific Reports · doi:10.1038/s41598-022-14738-1 · Cited by 54
A recurrent neural network without chaosPreprint
Laurent, T., & von Brecht, J. H. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1612.06212 · Cited by 65
“We refer to Bertschinger & Natschl¨ager (2004) for a study of the chaotic behavior of a simplified vanilla RNN with a specific statistical model, namely an i.i.d. Bernoulli process, for the input data as well as a specific statistical model, namely i.i.d. Gaussian, for the weights of the recurrence…”
MethodWijnants, M. L. (2014) · Journal of Nonlinear Dynamics · doi:10.1155/2014/962043 · Cited by 58
“, convection rolls or ecosystems, [3, 131, 132]), and neural network models [133, 134].”
BackgroundTheoretical foundations of studying criticality in the brain
Tian, Y., Tan, Z., Hou, H., Li, G., Cheng, A., Qiu, Y., Weng, K., Chen, C., & Sun, P. (2022) · Network Neuroscience · doi:10.1162/netn_a_00269 · Cited by 61
“These properties intrinsically shape neural information processing (e.g., encoding Bertschinger and Natschläger (2004); Boedecker et al. (2012); Del Papa et al. (2017); Haldeman and Beggs (2005); Kinouchi and Copelli (2006); Shew and Plenz (2013); X. R. Wang et al. (2011), transmission Shew et al.…”
BackgroundInfluentialInformation dynamics based self-adaptive reservoir for delay temporal memory tasks
Dasgupta, S., Wörgötter, F., & Manoonpong, P. (2013) · Evolving Systems · doi:10.1007/s12530-013-9080-y · Cited by 45
“All the parameters for the compared RC’s were set to their critical values, such that they operated at their optimal regime of performance (Bertschinger and Natschläger 2004).”
BackgroundSymbolic Computation Using Cellular Automata-Based Hyperdimensional Computing
Yılmaz, Ö. (2015) · Neural Computation · doi:10.1162/neco_a_00787 · Cited by 51
“It is claimed that many of these types of dynamical systems possess high computational power (Bertschinger & Natschl¨ager, 2004; Legenstein & Maass, 2007).”
BackgroundCoemergence of regularity and complexity during neural network development
Couzin-Fuchs, E., Ayali, A., Robinson, A., Hulata, E., & Ben‐Jacob, E. (2007) · Developmental Neurobiology · doi:10.1002/dneu.20557 · Cited by 43
“…was shown to be a fundamental characteristic in vari- ous studies of neuronal circuits dynamics (LinkenkaerHansen et al., 2001; Ayali et al., 2004; Bertschinger and Natschlager, 2004; Hulata et al., 2004; Haldeman and Beggs, 2005) giving the nervous system the free- dom to act locally while…”
BackgroundMultiplex visibility graphs to investigate recurrent neural network dynamics
Bianchi, F. M., Livi, L., Alippi, C., & Jenssen, R. (2017) · Scientific Reports · doi:10.1038/srep44037 · Cited by 41
“In the literature, such a “richness” is usually expressed in terms of diversity of connection weights 53 , entropy or rank of the matrix of neuron activations 15,32 .”
BackgroundDynamic primitives of brain network interaction
Schirner, M., Kong, X., Yeo, B. T. T., Deco, G., & Ritter, P. (2022) · NeuroImage · doi:10.1016/j.neuroimage.2022.118928 · Cited by 49
“Furthermore, slowing near a critical point corresponds to sharp increases in the variation of the activity and its autocorrelation, leading to functional benefits (Shew and Plenz, 2013) like optimized information transmission (Beggs and Plenz, 2003), information storage (Haldeman and Beggs, 2005), computational power…”
A New Hypothesis for Sleep: Tuning for Criticality
Pearlmutter, B. A., & Houghton, C. J. (2009) · Neural Computation · doi:10.1162/neco.2009.05-08-787 · Cited by 38
Feedback-Driven Quantum Reservoir Computing for Time-Series Analysis
Kobayashi, K., Fujii, K., & Yamamoto, N. (2024) · PRX Quantum · doi:10.1103/prxquantum.5.040325 · Cited by 99
“Previous investigations have demonstrated that computational capabilities tend to be maximized when the reservoir part resides proximate to the boundary separating a stable and an unstable dynamical regimes, commonly referred to as the edge of chaos [22–27].”
Minati, L., Chiesa, P., Tabarelli, D., D’Incerti, L., & Jovicich, J. (2015) · Chaos An Interdisciplinary Journal of Nonlinear Science · doi:10.1063/1.4914938 · Cited by 41
Computational Modeling of Wound Suture: A Review
Chanda, A., Ruchti, T., & Unnikrishnan, V. U. (2018) · IEEE Reviews in Biomedical Engineering · doi:10.1109/rbme.2018.2804219 · Cited by 43
Du, K.-L., & Swamy, M. N. S. (2013) · Springer eBooks · doi:10.1007/978-1-4471-5571-3_11 · Cited by 52
Metadata
- Authors
- Bertschinger, N., & Natschläger, T.
- Affiliations
- Graz University of Technology (AT)
- Software Competence Center Hagenberg (Austria) (AT)
- ORCID
- Bertschinger, Nils
- Venue
- Neural Computation
- Publisher
- MIT Press - Journals
- Published
- 2004-07-01
- Volume
- 16
- Issue
- 7
- Pages
- 1413-1436
- ISSN
- 0899-7667 (ISSN-L)
- 1530-888X
- OpenAlex
- W2057755457
- Cited by
- 833
- References
- 26
- Citations by year
- 2026: 45 · 2025: 73 · 2024: 61 · 2023: 48 · 2022: 66 · 2021: 73 · 2020: 52 · 2019: 53 · 2018: 47 · 2017: 48 · 2016: 26 · 2015: 33 · 2014: 29 · 2013: 33 · 2012: 27
- FWCI
- 13.53
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Neural Computation (publishedVersion)
- PubMed (publishedVersion)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Neural Networks and Applications
- Physical Sciences › Computer Science › Computational Theory and Mathematics › Cellular Automata and Applications
- Physical Sciences › Computer Science › Artificial Intelligence › Neural Networks and Reservoir Computing
- Keywords
- edge of chaos, recurrent neural networks, critical connectivity, threshold gates, random connectivity, real-time processing, dynamical systems
- MeSH terms
- Animals, Artificial Intelligence, Computer Simulation, Feedback, Memory, Models, Neurological, Time Factors, Neural Networks, Computer, Nonlinear Dynamics
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.