Real-Time Computation at the Edge of Chaos in Recurrent Neural Networks
The DOI matches its Crossref record; 26 of the 26 references it cites were found in a registry or an index.
Abstract
Depending on the connectivity, recurrent networks of simple computational units can show very different types of dynamics, ranging from totally ordered to chaotic. We analyze how the type of dynamics (ordered or chaotic) exhibited by randomly connected networks of threshold gates driven by a time-varying input signal depends on the parameters describing the distribution of the connectivity matrix. In particular, we calculate the critical boundary in parameter space where the transition from ordered to chaotic dynamics takes place. Employing a recently developed framework for analyzing real-time computations, we show that only near the critical boundary can such networks perform complex computations on time series. Hence, this result strongly supports conjectures that dynamical systems that are capable of doing complex computational tasks should operate near the edge of chaos, that is, the transition from ordered to chaotic dynamics.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Intrinsic fluctuations that are poised ''at the edge of chaos'' are thought to maximize the computational efficiency of such systems and the flexibility to switch between adaptive regimes (Bertschinger & Natschla ¨ger, 2004).”
From A potential mechanism for Gibsonian resonance: behavioral entrainment emerges from local homeostasis in an unsupervised reservoir network (2024) · cites this paper as Bertschinger & Natschläger, 2004
In the citing paper
“Finally, we show that a randomly connected circuit model, akin to a “reservoir” network13-15, can reproduce the observed distribution of timescales, provided that the network operates at the critical point (or “edge of chaos”)16-18.”
From A reservoir of time constants for memory traces in cortical neurons (2011) · cites this paper as [17]
Citations
Cites
Self-organized criticality: An explanation of the 1/f noise
Bak, P., Tang, C., & Wiesenfeld, K. (1987) · Physical Review Letters · doi:10.1103/physrevlett.59.381 · Cited by 7,591
The origins of order; self organization and selection in evolution
(1994) · International Journal of Biochemistry · doi:10.1016/0020-711x(94)90119-8 · Cited by 6,649
Introduction to the theory of neural computation
(1994) · Neural Networks · doi:10.1016/0893-6080(94)90014-0 · Cited by 6,401
Bak, P., Tang, C., & Wiesenfeld, K. (1988) · Physical Review A · doi:10.1103/physreva.38.364 · Cited by 4,713
Maass, W., Natschläger, T., & Markram, H. (2002) · Neural Computation · doi:10.1162/089976602760407955 · Cited by 4,025 · cited 10×
“(Maass et al., 2002) and indepenently by Jaeger (Jaeger, 2002) who called it “echo state networks”.”
BackgroundInfluentialSelf-citationChaos in Neuronal Networks with Balanced Excitatory and Inhibitory Activity
van Vreeswijk, C., & Sompolinsky, H. (1996) · Science · doi:10.1126/science.274.5293.1724 · Cited by 1,811
Computation at the edge of chaos: Phase transitions and emergent computation
Langton, C. G. (1990) · Physica D: Nonlinear Phenomena · doi:10.1016/0167-2789(90)90064-v · Cited by 1,527 · cited 5×
“These results provide further evidence for the idea of “computation at the edge of chaos” (Langton, 1990).”
BackgroundResultInfluentialFading memory and the problem of approximating nonlinear operators with Volterra series
Boyd, S., & Chua, L. (1985) · IEEE Transactions on Circuits and Systems · doi:10.1109/tcs.1985.1085649 · Cited by 991
Random Networks of Automata: A Simple Annealed Approximation
Derrida, B., & Pomeau, Y. (1986) · Europhysics Letters (EPL) · doi:10.1209/0295-5075/1/2/001 · Cited by 706
Dynamic Patterns in Complex Systems
(1988) · doi:10.1142/0534 · Cited by 548
Research works printed but not linked: 1
- Mitchell M., 1993, Complex Systems, 7, 89
Cited by
Reservoir computing approaches to recurrent neural network training
Lukoševičius, M., & Jaeger, H. (2009) · Computer Science Review · doi:10.1016/j.cosrev.2009.03.005 · Cited by 2,956
“, [94]) that at the edge of chaos many types of dynamical systems, including binary systems and reservoirs, possess high computational power [87, 95].”
BackgroundMethodInfluentialRecent advances in physical reservoir computing: A review
Tanaka, G., Yamane, T., Héroux, J. B., Nakane, R., Kanazawa, N., Takeda, S., Numata, H., Nakano, D., & Hirose, A. (2019) · Neural Networks · doi:10.1016/j.neunet.2019.03.005 · Cited by 2,146
“…between non-chaotic and chaotic regimes, it is often recommended that the parameter be set close to the transition point (the so-called edge of chaos (Bertschinger & Natschläger, 2004; Legenstein & Maass, 2007)) where the transformation by a reservoir is neither very expanding nor very contracting.”
BackgroundResultGenerating Coherent Patterns of Activity from Chaotic Neural Networks
Sussillo, D., & Abbott, L. F. (2009) · Neuron · doi:10.1016/j.neuron.2009.07.018 · Cited by 1,092
“Such an input can induce a transition between chaotic and nonchaotic states (Molgedey et al., 1992; Bertschinger and Natschlä ger, 2004; K. Rajan, L.F.A., and H. Sompolinsky, unpublished data).”
Embedded Devices for Neuromorphic Time-Series Assessment
Mazumder, A. N., Hosseini, M., & Mohsenin, T. (2022) · Research Explorer (The University of Manchester) · doi:10.13016/m2ksy0-wxr3 · Cited by 652
Can structure predict function in the human brain?
Honey, C. J., Thivierge, J.-P., & Sporns, O. (2010) · NeuroImage · doi:10.1016/j.neuroimage.2010.01.071 · Cited by 629
“can be realized (Bertschinger and Natschläger, 2004), including the approximation of intricate time-varying functions, but rules that link”
Neuronal Avalanches Imply Maximum Dynamic Range in Cortical Networks at Criticality
Shew, W. L., Yang, H., Petermann, T., Roy, R., & Plenz, D. (2009) · The Journal of Neuroscience · doi:10.1523/jneurosci.3864-09.2009 · Cited by 613
“…map input differences into distinguishable network outputs, our result is also closely related to network-mediated separation, which has been predicted to peak in the critical state, at the transition from ordered to chaotic dynamics (Bertschinger and Natschlager, 2004;Legenstein and Maass, 2007).”
BackgroundChaotic dynamics in nanoscale NbO2 Mott memristors for analogue computing
Kumar, S., Strachan, J. P., & Williams, R. S. (2017) · Nature · doi:10.1038/nature23307 · Cited by 617
The Functional Benefits of Criticality in the Cortex
Shew, W. L., & Plenz, D. (2013) · The Neuroscientist · doi:10.1177/1073858412445487 · Cited by 607
“…address the optimization of information-processing capabilities of neural networks at criticality (Greenfield and Lecar 2001; Beggs and Plenz 2003; Bertschinger and Natschläger 2004; Haldeman and Beggs 2005; Kinouchi and Copelli 2006; Legenstein and Maass 2007; Tanaka and others 2009; Chen and…”
BackgroundSpike avalanches in vivo suggest a driven, slightly subcritical brain state
Priesemann, V. (2014) · Frontiers in Systems Neuroscience · doi:10.3389/fnsys.2014.00108 · Cited by 563
“Critical phase transitions can occur for example for the transitions from order to chaos (Bertschinger and Natschläger, 2004; Haldeman and Beggs, 2005; Boedecker et al., 2012; Lizier, 2013), from non-oscillatory to oscillatory regimes (Linkenkaer-Hansen et al., 2001; Poil et al., 2012), from replay…”
BackgroundColloquium: Criticality and dynamical scaling in living systems
Muñoz, M. A. (2018) · Reviews of Modern Physics · doi:10.1103/revmodphys.90.031001 · Cited by 530
“…such machines can perform real-time computations –responding rapidly to time varying input signals– in a coherent yet flexible way if they operate near a critical point (Bertschinger and Natschlager, 2004; Boedecker et al., 2012; Legenstein and Maass, 2007; Legenstein, 2005; Maass et al., 2002).”
BackgroundThe criticality hypothesis: how local cortical networks might optimize information processing
Beggs, J. M. (2008) · Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences · doi:10.1098/rsta.2007.2092 · Cited by 533
“Neural models have also picked up on this idea, and several authors have suggested that neural networks should operate close to the chaotic regime (Chialvo & Bak 1999; Bak & Chialvo 2001; Bertschinger & Natschlager 2004; Lin & Chen 2006; Legenstein & Maass 2007).”
BackgroundMethodInfluentialCriticality in the brain: A synthesis of neurobiology, models and cognition
Cocchi, L., Gollo, L. L., Zalesky, A., & Breakspear, M. (2017) · Progress in Neurobiology · doi:10.1016/j.pneurobio.2017.07.002 · Cited by 515
“Again, this notion has been captured in simple neural models (Bertschinger and Natschläger, 2004; Haldeman and Beggs, 2005; Yang et al., 2017) and also demonstrated in empirical recordings, including those arising in unperturbed (resting state) recordings (Breakspear, 2001; Deco and Jirsa, 2012) as…”
BackgroundUniversal Critical Dynamics in High Resolution Neuronal Avalanche Data
Friedman, N., Ito, S., Brinkman, B. A. W., Shimono, M., DeVille, R. E. L., Dahmen, K. A., Beggs, J. M., & Butler, T. C. (2012) · Physical Review Letters · doi:10.1103/physrevlett.108.208102 · Cited by 501
Being Critical of Criticality in the Brain
Beggs, J. M., & Timme, N. (2012) · Frontiers in Physiology · doi:10.3389/fphys.2012.00163 · Cited by 494
“just communication that would be optimized at the critical point (Beggs and Plenz, 2003; Bertschinger and Natschlager, 2004; Maass et al., 2004; Ramo et al., 2007; Tanaka et al., 2009; Chialvo, 2010; Shew et al., 2011).”
BackgroundInfluentialRe-visiting the echo state property
Yildiz, I. B., Jaeger, H., & Kiebel, S. J. (2012) · Neural Networks · doi:10.1016/j.neunet.2012.07.005 · Cited by 486
“This line of thought apparently originates – within the reservoir computing arena – in [2, 1], where it was shown that, for reservoirs with binary threshold units, performance peaked when the reservoir weights were scaled to just below a critical value after which state convergence was lost.”
BackgroundIntrinsic Coupling Modes: Multiscale Interactions in Ongoing Brain Activity
Engel, A. K., Gerloff, C., Hilgetag, C. C., & Nolte, G. (2013) · Neuron · doi:10.1016/j.neuron.2013.09.038 · Cited by 471
Haldeman, C., & Beggs, J. M. (2005) · Physical Review Letters · doi:10.1103/physrevlett.94.058101 · Cited by 469
Edge of chaos and prediction of computational performance for neural circuit models
Legenstein, R., & Maass, W. (2007) · Neural Networks · doi:10.1016/j.neunet.2007.04.017 · Cited by 453
“Networks of threshold elements with online input were recently considered in Bertschinger and Natschläger (2004). In this study, a network with online input is called chaotic if arbitrary small differences in a (initial) network state x(0) are highly amplified and do not vanish.”
Physical reservoir computing—an introductory perspective
Nakajima, K. (2020) · Japanese Journal of Applied Physics · doi:10.35848/1347-4065/ab8d4f · Cited by 422
Self-organized criticality as a fundamental property of neural systems
Hesse, J., & Gross, T. (2014) · Frontiers in Systems Neuroscience · doi:10.3389/fnsys.2014.00166 · Cited by 440
“(Kauffman, 1984; Derrida and Pomeau, 1986), liquid state machines (Langton, 1990), and neuronal networks (Maass et al., 2002; Bertschinger and Natschläger, 2004), for a review also see Legenstein and Maass (2007).”
BackgroundThe physics of brain network structure, function and control
Lynn, C. W., & Bassett, D. S. (2019) · Nature Reviews Physics · doi:10.1038/s42254-019-0040-8 · Cited by 400
Scaling laws in cognitive sciences
Kello, C. T., Brown, G. D. A., Ferrer-i-Cancho, R., Holden, J. G., Linkenkaer-Hansen, K., Rhodes, T., & Van Orden, G. C. (2010) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2010.02.005 · Cited by 384
Griffiths phases and the stretching of criticality in brain networks
Moretti, P., & Muñoz, M. A. (2013) · Nature Communications · doi:10.1038/ncomms3521 · Cited by 363
“Critical dynamics provides a delicate trade-off between these two impractical tendencies, and has been argued to imply optimal transmission and storage of information [7, 8, 17], optimal computational capabilities [18], large network stability [16], maximal variety of memory repertoires [19], and maximal sensitivity t…”
BackgroundInfluentialEmerging opportunities and challenges for the future of reservoir computing
Yan, M., Huang, C., Bienstman, P., Tino, P., Lin, W., & Sun, J. (2024) · Nature Communications · doi:10.1038/s41467-024-45187-1 · Cited by 346
“Empiricalstudieshaveshown that γ chosen sothat the system is around the edge of chaos 15 typically produces the best outcome, which is supported by a necessary but not suf fi cient condition - imposed on the largest singular value of the effective stability matrix W γ = (1 − γ )+ γ W .”
BackgroundRecurrent network models of sequence generation and memory
Rajan, K., Harvey, C. D., & Tank, D. W. (2016) · Neuron · doi:10.1016/j.neuron.2016.02.009 · Cited by 351
“, 1988); however, the presence of external stimuli can channel the ongoing dynamics in these networks by suppressing their chaos (Molgedey et al., 1992; Bertschinger and Natschläger, 2004; Rajan et al., 2010, 2011).”
BackgroundRandomness in neural networks: an overview
Scardapane, S., & Wang, D. (2017) · WIREs Data Mining and Knowledge Discovery · doi:10.1002/widm.1200 · Cited by 352
“One important aspect that we briefly mention here is the concept of ‘edge of instability’ (or ‘edge of chaos’); namely, a large body of literature suggests that ESNs, like other recurrent architectures and even cortical circuits, have the highest computational power when they operate near an instable regime.(127,128)…”
BackgroundHarnessing Disordered-Ensemble Quantum Dynamics for Machine Learning
Fujii, K., & Nakajima, K. (2017) · Physical Review Applied · doi:10.1103/physrevapplied.8.024030 · Cited by 337
“[34, 54]), which we name the short-term memory (STM) task and parity check (PC) task.”
BackgroundAnalysis and Design of Echo State Networks
Ozturk, M. C., Xu, D., & Príncipe, J. C. (2007) · Neural Computation · doi:10.1162/neco.2007.19.1.111 · Cited by 308
“Recently, similar conclusions were suggested for LSMs (Bertschinger & Natschläger, 2004).”
BackgroundInfluentialOn the computational power of circuits of spiking neurons
Maass, W., & Markram, H. (2004) · Journal of Computer and System Sciences · doi:10.1016/j.jcss.2004.04.001 · Cited by 301
“A theoretical analysis of fading memory for online input in recurrent networks of McCulloch-Pitts neurons is given in [3].”
BackgroundInformation processing in echo state networks at the edge of chaos
Boedecker, J., Obst, O., Lizier, J. T., Mayer, N. M., & Asada, M. (2012) · Theory in Biosciences · doi:10.1007/s12064-011-0146-8 · Cited by 297
A reservoir of time constants for memory traces in cortical neurons
Bernacchia, A., Seo, H., Lee, D., & Wang, X.-J. (2011) · Nature Neuroscience · doi:10.1038/nn.2752 · Cited by 292
“, can reproduce the observed distribution of timescales, provided that the network operates at the critical point (or edge of chaos”
BackgroundStimulus-dependent suppression of chaos in recurrent neural networks
Rajan, K., Abbott, L. F., & Sompolinsky, H. (2010) · Physical Review E · doi:10.1103/physreve.82.011903 · Cited by 279
Design of deep echo state networks
Gallicchio, C., Micheli, A., & Pedrelli, L. (2018) · Neural Networks · doi:10.1016/j.neunet.2018.08.002 · Cited by 262
How critical is brain criticality?
O’Byrne, J., & Jerbi, K. (2022) · Trends in Neurosciences · doi:10.1016/j.tins.2022.08.007 · Cited by 258
Cortical Circuit Dynamics Are Homeostatically Tuned to Criticality In Vivo
Ma, Z., Turrigiano, G. G., Wessel, R., & Hengen, K. B. (2019) · Neuron · doi:10.1016/j.neuron.2019.08.031 · Cited by 252
Priesemann, V., Valderrama, M., Wibral, M., & Le Van Quyen, M. (2013) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1002985 · Cited by 218
“Since the differences in avalanche measures were highly systematic and our results point to a slightly subcritical mode of operation, we may ask how to reconcile this with theoretical considerations that stress the computational optimality of the critical state [2,4,40].”
BackgroundEmergent Criticality in Complex Turing B‐Type Atomic Switch Networks
Stieg, A. Z., Avizienis, A. V., Sillin, H. O., Martin‐Olmos, C., Aono, M., & Gimzewski, J. K. (2012) · Advanced Materials · doi:10.1002/adma.201103053 · Cited by 217
SORN: a Self-organizing Recurrent Neural Network
Lazar, A. (2009) · Frontiers in Computational Neuroscience · doi:10.3389/neuro.10.023.2009 · Cited by 209
“It has been argued that a tuning of network dynamics to criticality should bring the performance of static reservoir networks close to the optimal performance (Bertschinger and Natschläger, 2004).”
Singer, W. (2013) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2013.09.006 · Cited by 212
Transition to Chaos in Random Neuronal Networks
Kadmon, J., & Sompolinsky, H. (2015) · Physical Review X · doi:10.1103/physrevx.5.041030 · Cited by 283
Spike Avalanches Exhibit Universal Dynamics across the Sleep-Wake Cycle
Ribeiro, T. L., Copelli, M., Caixeta, F., Belchior, H., Chialvo, D. R., Nicolelis, M. A. L., & Ribeiro, S. (2010) · PLoS ONE · doi:10.1371/journal.pone.0014129 · Cited by 201
“From the theoretical side, numerous results show that scale-invariant dynamics provide functionally desirable features for the behaving brain, such as optimal computational capabilities [9], information transmission [1], size of memory repertoires [10] and sensitivity to stimuli [11,12].”
Zylberberg, J., & Strowbridge, B. W. (2017) · Annual Review of Neuroscience · doi:10.1146/annurev-neuro-070815-014006 · Cited by 202
“…balance between excitation and inhibition in the circuit, information about the network’s inputs will be maintained after those inputs go away (Bertschinger & Natschläger 2004, Buonomano & Maass 2009, Laje & Buonomano 2013, Maass et al. 2002, Singh & Eliasmith 2006, Toyoizumi & Abbott 2011).”
BackgroundWhy Brain Criticality Is Clinically Relevant: A Scoping Review
Zimmern, V. (2020) · Frontiers in Neural Circuits · doi:10.3389/fncir.2020.00054 · Cited by 194
“These properties include information transmission, information storage, dynamic range, metastable states, and computational power (Maass et al., 2002; Bertschinger and Natschläger, 2004; Latham and Nirenberg, 2004; Haldeman and Beggs, 2005; Kinouchi and Copelli, 2006; Tanaka et al., 2009; Boedecker et al., 2012; Shrik…”
BackgroundInformation-based fitness and the emergence of criticality in living systems
Hidalgo, J., Grilli, J., Suweis, S., Muñoz, M. A., Banavar, J. R., & Maritan, A. (2014) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1319166111 · Cited by 193
Encoding and Decoding of Overlapping Odor Sequences
Broome, B. M., Jayaraman, V., & Laurent, G. (2006) · Neuron · doi:10.1016/j.neuron.2006.07.018 · Cited by 192
“It was suggested that, to perform complex computations on time-varying input, such networks need to operate at the edge of dynamical regimes where behavior becomes chaotic (Bertschinger and Natschlager, 2004).”
Large-Scale Optical Reservoir Computing for Spatiotemporal Chaotic Systems Prediction
Rafayelyan, M., Dong, J., Tan, Y., Krzakala, F., & Gigan, S. (2020) · Physical Review X · doi:10.1103/physrevx.10.041037 · Cited by 186
Inhibitory stabilization and cortical computation
Sadeh, S., & Clopath, C. (2021) · Nature Reviews Neuroscience · doi:10.1038/s41583-020-00390-z · Cited by 178
Zhang, Y., Li, P., Jin, Y., & Choe, Y. (2015) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2015.2388544 · Cited by 178
“Theoretical studies show that the computational performance of a recurrent neural network depends critically on its dynamics [46]–[48].”
BackgroundConnectivity, Dynamics, and Memory in Reservoir Computing with Binary and Analog Neurons
Büsing, L., Schrauwen, B., & Legenstein, R. (2010) · Neural Computation · doi:10.1162/neco.2009.01-09-947 · Cited by 177
“This constitutes the main advantages of p ∞ over the the predictor defined in Bertschinger and Natschl¨ager (2004), the so-called NM separation.”
BackgroundMethodResultInfluentialReservoir computing with a single delay-coupled non-linear mechanical oscillator
Dion, G., Mejaouri, S., & Sylvestre, J. (2018) · Journal of Applied Physics · doi:10.1063/1.5038038 · Cited by 174
Avalanches and edge-of-chaos learning in neuromorphic nanowire networks
Hochstetter, J., Zhu, R., Loeffler, A., Diaz-Alvarez, A., Nakayama, T., & Kuncic, Z. (2021) · Nature Communications · doi:10.1038/s41467-021-24260-z · Cited by 178
Physical reservoir computing based on spin torque oscillator with forced synchronization
Tsunegi, S., Taniguchi, T., Nakajima, K., Miwa, S., Yakushiji, K., Fukushima, A., Yuasa, S., & Kubota, H. (2019) · Applied Physics Letters · doi:10.1063/1.5081797 · Cited by 171
Steil, J. J. (2007) · Neural Networks · doi:10.1016/j.neunet.2007.04.011 · Cited by 165
Feldman, D. P., McTague, C. S., & Crutchfield, J. P. (2008) · Chaos An Interdisciplinary Journal of Nonlinear Science · doi:10.1063/1.2991106 · Cited by 172
“This led, for example, to the widely popularized notion of the “edge of chaos” [60, 61, 62, 63, 64, 65, 66, 67]—namely, that objects achieve maximum complexity at a boundary between order and disorder.”
BackgroundKawai, Y., Park, J., & Asada, M. (2019) · Neural Networks · doi:10.1016/j.neunet.2019.01.002 · Cited by 158
Liu, K., Dang, B., Zhang, T., Yang, Z., Bao, L., Xu, L., Cheng, C., Huang, R., & Yang, Y. (2022) · Advanced Materials · doi:10.1002/adma.202108826 · Cited by 173
Hennequin, G., Ahmadian, Y., Rubin, D. B., Lengyel, M., & Miller, K. D. (2018) · Neuron · doi:10.1016/j.neuron.2018.04.017 · Cited by 156
“…fluctuations, and certain types of stimuli can suppress chaos by forcing the dynamical state of the network to follow a specific trajectory, thus quenching across-trial variability (Figure 1B;Molgedey et al., 1992; Bertschinger and Natschl€ager, 2004; Sussillo and Abbott, 2009; Rajan et al., 2010).”
The pervasiveness of 1/f scaling in speech reflects the metastable basis of cognition
Kello, C. T., Anderson, G. G., Holden, J. G., & Van Orden, G. C. (2008) · Cognitive Science · doi:10.1080/03640210801944898 · Cited by 151
“In the other study, Bertschinger and Natschlager (2004) examined the effect of criticality on the computational power of neural network models, where power is defined in terms of the complexity and diversity of the input/output mappings that can be processed (see also Langton, 1990; Packard, 1988).”
BackgroundBoosting Computational Power through Spatial Multiplexing in Quantum Reservoir Computing
Nakajima, K., Fujii, K., Negoro, M., Mitarai, K., & Kitagawa, M. (2019) · Physical Review Applied · doi:10.1103/physrevapplied.11.034021 · Cited by 153
“Ironically, as investigated in [27], this property of common input synchronization is rather a required property for reservoirs in terms of the reproducibility of the signals (the opposite case is chaotic dynamics, where an arbitrarily small change in one state of a deterministic system results in large differences in…”
BackgroundNonlinearity in memristors for neuromorphic dynamic systems
Yang, K., Joshua Yang, J., Huang, R., & Yang, Y. (2022) · Small Science · doi:10.1002/smsc.202100049 · Cited by 151
Fading Signatures of Critical Brain Dynamics during Sustained Wakefulness in Humans
Meisel, C., Olbrich, E., Shriki, O., & Achermann, P. (2013) · The Journal of Neuroscience · doi:10.1523/jneurosci.1516-13.2013 · Cited by 150
Bellay, T., Klaus, A., Seshadri, S., & Plenz, D. (2015) · eLife · doi:10.7554/elife.07224 · Cited by 157
Stochastic Memory Devices for Security and Computing
Carboni, R., & Ielmini, D. (2019) · Advanced Electronic Materials · doi:10.1002/aelm.201900198 · Cited by 145
“computational capabilities [198], which can be used to solve optimization problems [199].”
BackgroundTransition to Chaos in Random Networks with Cell-Type-Specific Connectivity
Aljadeff, J., Stern, M., & Sharpee, T. O. (2015) · Physical Review Letters · doi:10.1103/physrevlett.114.088101 · Cited by 189
In‐Materio Reservoir Computing in a Sulfonated Polyaniline Network
Usami, Y., van de Ven, B., Mathew, D. G., Chen, T., Kotooka, T., Kawashima, Y., Tanaka, Y., Otsuka, Y., Ohoyama, H., Tamukoh, H., et al. (2021) · Advanced Materials · doi:10.1002/adma.202102688 · Cited by 142
Subsampling effects in neuronal avalanche distributions recorded in vivo
Priesemann, V., Munk, M. H. J., & Wibral, M. (2009) · BMC Neuroscience · doi:10.1186/1471-2202-10-40 · Cited by 162
“In addition, theoretical investigations demonstrated that the critical state was optimal for information transmission and processing [29,30].”
BackgroundToyoizumi, T., & Abbott, L. F. (2011) · Physical Review E · doi:10.1103/physreve.84.051908 · Cited by 140
Consciousness is supported by near-critical slow cortical electrodynamics
Toker, D., Pappas, I., Lendner, J. D., Frohlich, J., Mateos, D. M., Muthukumaraswamy, S., Carhart-Harris, R., Paff, M., Vespa, P. M., Monti, M. M., et al. (2022) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2024455119 · Cited by 138
25 years of criticality in neuroscience — established results, open controversies, novel concepts
Wilting, J., & Priesemann, V. (2019) · Current Opinion in Neurobiology · doi:10.1016/j.conb.2019.08.002 · Cited by 172
“Tuning models towards critical phase transitions has been shown to maximize the number of metastable states [12], the dynamic range [13, 14], information transmission in terms of mutual information [15, 16, 17], active information storage [18], and computational power in terms of input-output mappings [19, 20, 21].”
BackgroundSynergies Between Intrinsic and Synaptic Plasticity Mechanisms
Triesch, J. (2007) · Neural Computation · doi:10.1162/neco.2007.19.4.885 · Cited by 137
“We speculate that similar combinations of plasticity mechanisms may be effective in driving recurrent networks into a regime of great computational power (Bertschinger & Natschläger, 2004) for a fixed energy expenditure.”
BackgroundChen, P., Liu, R., Aihara, K., & Chen, L. (2020) · Nature Communications · doi:10.1038/s41467-020-18381-0 · Cited by 160
The highs and lows of beta activity in cortico‐basal ganglia loops
Brittain, J., Sharott, A., & Brown, P. (2014) · European Journal of Neuroscience · doi:10.1111/ejn.12574 · Cited by 134
“Such integration is possible only as the system approaches the ‘critical boundary’ between noise and determinism, where recurrent networks promote the resident memory and remain flexible to the integration of new information (see Bertschinger & Natschl€ager, 2004).”
BackgroundInfluentialLearning function from structure in neuromorphic networks
Suárez, L. E., Richards, B. A., Lajoie, G., & Misic, B. (2021) · Nature Machine Intelligence · doi:10.1038/s42256-021-00376-1 · Cited by 134
Furuta, T., Fujii, K., Nakajima, K., Tsunegi, S., Kubota, H., Suzuki, Y., & Miwa, S. (2018) · Physical Review Applied · doi:10.1103/physrevapplied.10.034063 · Cited by 146
“In this paper, we report the quantitative analysis of the figures-of-merit for RC [27,28] using MTJ devices.”
BackgroundMethodExploiting short-term memory in soft body dynamics as a computational resource
Nakajima, K., Li, T., Hauser, H., & Pfeifer, R. (2014) · Journal of The Royal Society Interface · doi:10.1098/rsif.2014.0437 · Cited by 128
“To evaluate the influence of the parameters of τstate and n on the system performance, we introduced a measure based on mutual information, MIn, between the system output and the target output [29].”
MethodKaiser, M. (2010) · Frontiers in Neuroinformatics · doi:10.3389/fninf.2010.00008 · Cited by 127
“While its functional signifi cance is still not well understood, it has been suggested that critical dynamics may enhance information processing capabilities of neuronal networks (e.g., Bertschinger and Natschläger, 2004).”
Exploiting the Dynamics of Soft Materials for Machine Learning
Nakajima, K., Häuser, H., Li, T., & Pfeifer, R. (2018) · Soft Robotics · doi:10.1089/soro.2017.0075 · Cited by 141
“26,28 The second is a famous parity checker function, which represents a nonlinear function.”
BackgroundSimple Deterministically Constructed Cycle Reservoirs with Regular Jumps
Rodan, A., & Tiňo, P. (2012) · Neural Computation · doi:10.1162/neco_a_00297 · Cited by 141
“9 In the same vein, Bertschinger and Natschlager (2004) suggest that if a reservoir model is to be useful for computations on input time series, it should have the “separation property”—different input time series that produce different outputs should have different reservoir representations.”
BackgroundReconstructing computational system dynamics from neural data with recurrent neural networks
Durstewitz, D., Koppe, G., & Thurm, M. I. (2023) · Nature reviews. Neuroscience · doi:10.1038/s41583-023-00740-7 · Cited by 230
“Reservoir computers and echo state machines 75,174–176 are another ingenious RNN design that is popular for DS reconstruction 132,177,178 and were originally introduced as a computationally highly efficient alternative to classical RNNs (Fig.”
BackgroundMarshall, N. J., Timme, N. M., Bennett, N., Ripp, M., Lautzenhiser, E., & Beggs, J. M. (2016) · Frontiers in Physiology · doi:10.3389/fphys.2016.00250 · Cited by 142
“, 2004; Haldeman and Beggs, 2005), computational power (Bertschinger and Natschlager, 2004), dynamic range (Kinouchi and Copelli, 2006; Shew et al.”
BackgroundInfluentialSecond type of criticality in the brain uncovers rich multiple-neuron dynamics
Dahmen, D., Grün, S., Diesmann, M., & Helias, M. (2019) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1818972116 · Cited by 149
Poil, S., van Ooyen, A., & Linkenkaer‐Hansen, K. (2008) · Human Brain Mapping · doi:10.1002/hbm.20590 · Cited by 119
“…[Kinouchi and Copelli, 2006; Linkenkaer-Hansen et al., 2004], optimal information transmission [Beggs and Plenz, 2003] and computational abilities [Bertschinger and Natschlager, 2004], the presence of metastable states that could support neuronal representations [Beggs and Plenz, 2004; Haldeman…”
BackgroundEnhanced performance of a reservoir computer using polarization dynamics in VCSELs
Vatin, J., Rontani, D., & Sciamanna, M. (2018) · Optics Letters · doi:10.1364/ol.43.004497 · Cited by 118
Computing with networks of nonlinear mechanical oscillators
Coulombe, J. C., York, M. C. A., & Sylvestre, J. (2017) · PLoS ONE · doi:10.1371/journal.pone.0178663 · Cited by 121
“It is actually observed in numerical experiments that reservoir computers perform well when the networks driven by the stimulus u ( t ) operate as systems with complex dynamics [3].”
InfluentialEdge-of-chaos learning achieved by ion-electron–coupled dynamics in an ion-gating reservoir
Nishioka, D., Tsuchiya, T., Namiki, W., Takayanagi, M., Imura, M., Koide, Y., Higuchi, T., & Terabe, K. (2022) · Science Advances · doi:10.1126/sciadv.ade1156 · Cited by 127
Recurrent dynamics in the cerebral cortex: Integration of sensory evidence with stored knowledge
Singer, W. (2021) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2101043118 · Cited by 110
Mutual information in random Boolean models of regulatory networks
Ribeiro, A. S., Kauffman, S. A., Lloyd‐Price, J., Samuelsson, B., & Socolar, J. E. S. (2008) · Physical Review E · doi:10.1103/physreve.77.011901 · Cited by 104
Levina, A., & Priesemann, V. (2017) · Nature Communications · doi:10.1038/ncomms15140 · Cited by 107
“Criticality is a dynamical state that maximizes information processing capacity in models, and therefore is a favorable candidate for brain functioning [18, 23, 24, 25].”
BackgroundControl of criticality and computation in spiking neuromorphic networks with plasticity
Cramer, B., Stöckel, D., Kreft, M., Wibral, M., Schemmel, J., Meier, K., & Priesemann, V. (2020) · Nature Communications · doi:10.1038/s41467-020-16548-3 · Cited by 140
Watt, S., & Kostylev, M. (2020) · Physical Review Applied · doi:10.1103/physrevapplied.13.034057 · Cited by 112
“These are the short-term memory (STM) [24] and parity check (PC) [17, 25] tasks.”
BackgroundDynamical Criticality: Overview and Open Questions
Roli, A., Villani, M., Filisetti, A., & Serra, R. (2017) · Journal of Systems Science and Complexity · doi:10.1007/s11424-017-6117-5 · Cited by 105
“Bertschinger and Natschläger [11] study networks of randomly connected threshold gates and show that they exhibit a transition between ordered and disordered dynamics depending on the connectivity of the network.”
BackgroundDecline of long-range temporal correlations in the human brain during sustained wakefulness
Meisel, C., Bailey, K., Achermann, P., & Plenz, D. (2017) · Scientific Reports · doi:10.1038/s41598-017-12140-w · Cited by 114
“Critical dynamics are often regarded to support optimal computational functioning [25, 8, 16, 23, 42].”
BackgroundSelf-organization without conservation: are neuronal avalanches generically critical?
Bonachela, J. A., de Franciscis, S., Torres, J. J., & Muñoz, M. A. (2010) · Journal of Statistical Mechanics Theory and Experiment · doi:10.1088/1742-5468/2010/02/p02015 · Cited by 132
“• optimal transmission and storage of information [5]–[7], [22, 23], • optimal computational capabilities [24], • large network stability [25], • maximal sensitivity to sensory stimuli [26], etc.”
BackgroundInvestigating Echo-State Networks Dynamics by Means of Recurrence Analysis
Bianchi, F. M., Livi, L., & Alippi, C. (2016) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2016.2630802 · Cited by 110
“Langton [41], Bertschinger and Natschläger [42], and Legenstein and Maass [43] show that several dynamical systems, among which randomly connected RNNs, achieved the highest computational capabilities when moving toward the unstable (sometime even chaotic) regime, where the ESP is lost and the system enters into an os…”
Reservoir computing on a silicon platform with a ferroelectric field-effect transistor
Toprasertpong, K., Nako, E., Wang, Z., Nakane, R., Takenaka, M., & Takagi, S. (2022) · Communications Engineering · doi:10.1038/s44172-022-00021-8 · Cited by 110
Nokkala, J., Martínez‐Peña, R., Giorgi, G. L., Parigi, V., Soriano, M. C., & Zambrini, R. (2021) · Communications Physics · doi:10.1038/s42005-021-00556-w · Cited by 151
Effects of topology on network evolution
Oikonomou, P., & Cluzel, P. (2006) · Nature Physics · doi:10.1038/nphys359 · Cited by 98
Nokkala, J., Martínez-Peña, R., Giorgi, G. L., Parigi, V., Soriano, M. C., & Zambrini, R. (2020) · arXiv · doi:10.48550/arxiv.2006.04821 · Cited by 94
Control of criticality and computation in spiking neuromorphic networks with plasticity
Cramer, B., Stöckel, D., Kreft, M., Wibral, M., Schemmel, J., Meier, K., & Priesemann, V. (2020) · RePEc: Research Papers in Economics · Cited by 93
Fading memory and kernel properties of generic cortical microcircuit models
Maass, W., Natschläger, T., & Markram, H. (2004) · Journal of Physiology-Paris · doi:10.1016/j.jphysparis.2005.09.020 · Cited by 111
“on the most recent inputs u(s) (see [2] for a discussion of chaos and edge-ofchaos in dynamical systems with online inputs).”
BackgroundSelf-citation
Metadata
- Authors
- Bertschinger, N., & Natschläger, T.
- Affiliations
- Graz University of Technology (AT)
- Software Competence Center Hagenberg (Austria) (AT)
- ORCID
- Bertschinger, Nils
- Venue
- Neural Computation
- Publisher
- MIT Press - Journals
- Published
- 2004-07-01
- Volume
- 16
- Issue
- 7
- Pages
- 1413-1436
- ISSN
- 0899-7667 (ISSN-L)
- 1530-888X
- OpenAlex
- W2057755457
- Cited by
- 833
- References
- 26
- Citations by year
- 2026: 45 · 2025: 73 · 2024: 61 · 2023: 48 · 2022: 66 · 2021: 73 · 2020: 52 · 2019: 53 · 2018: 47 · 2017: 48 · 2016: 26 · 2015: 33 · 2014: 29 · 2013: 33 · 2012: 27
- FWCI
- 13.53
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Neural Computation (publishedVersion)
- PubMed (publishedVersion)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Neural Networks and Applications
- Physical Sciences › Computer Science › Computational Theory and Mathematics › Cellular Automata and Applications
- Physical Sciences › Computer Science › Artificial Intelligence › Neural Networks and Reservoir Computing
- Keywords
- edge of chaos, recurrent neural networks, critical connectivity, threshold gates, random connectivity, real-time processing, dynamical systems
- MeSH terms
- Animals, Artificial Intelligence, Computer Simulation, Feedback, Memory, Models, Neurological, Time Factors, Neural Networks, Computer, Nonlinear Dynamics
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.