Real-Time Computing Without Stable States: A New Framework for Neural Computation Based on Perturbations
Open accessThe DOI matches its Crossref record; 28 of the 28 references it cites were found in a registry or an index.
Abstract
A key challenge for neural modeling is to explain how a continuous stream of multimodal input from a rapidly changing environment can be processed by stereotypical recurrent circuits of integrate-and-fire neurons in real time. We propose a new computational model for real-time computing on time-varying input that provides an alternative to paradigms based on Turing machines or attractor neural networks. It does not require a task-dependent construction of neural circuits. Instead, it is based on principles of high-dimensional dynamical systems in combination with statistical learning theory and can be implemented on generic evolved or found recurrent circuitry. It is shown that the inherent transient dynamics of the high-dimensional dynamical system formed by a sufficiently large and heterogeneous neural circuit may serve as universal analog fading memory. Readout neurons can learn to extract in real time from the current state of such recurrent neural circuit information about current and past inputs that may be needed for diverse tasks. Stable internal states are not required for giving a stable output, since transient internal states can be transformed by readout neurons into stable target outputs due to the high dimensionality of the dynamical system. Our approach is based on a rigorous computational model, the liquid state machine, that, unlike Turing machines, does not require sequential transitions between well-defined discrete internal states. It is supported, as the Turing machine is, by rigorous mathematical results that predict universal computational power under idealized conditions, but for the biologically more realistic scenario of real-time processing of time-varying inputs. Our approach provides new perspectives for the interpretation of neural coding, the design of experiments and data analysis in neurophysiology, and the solution of problems in robotics and neurotechnology.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Although work in artificial intelligence and machine-learning research has shown that similar or even more complex outcomes can be achieved with a variety of techniques, including deep neural networks with trained weights (Gibney, 2015), reservoir networks with a trained output mapping (Maass et al., 2002;Jaeger & Haas, 2004), networks that are evolved using a genetic algorithm (Iizuka & Di Paolo,”
From A potential mechanism for Gibsonian resonance: behavioral entrainment emerges from local homeostasis in an unsupervised reservoir network (2024) · cites this paper as Maass et al., 2002
In the citing paper
“Finally, we show that a randomly connected circuit model, akin to a “reservoir” network13-15, can reproduce the observed distribution of timescales, provided that the network operates at the critical point (or “edge of chaos”)16-18.”
From A reservoir of time constants for memory traces in cortical neurons (2011) · cites this paper as [13]
In the citing paper
“Reservoir-type networks have been the subject of active research in computational neuroscience and machine learning13-15, but so far experimental support that such networks are adopted by the brain has been lacking.”
From A reservoir of time constants for memory traces in cortical neurons (2011) · cites this paper as [13]
In the citing paper
“Those models predict that the memory of input signals is stored in a large, recurrent and heterogeneous network (reservoir), in a distributed manner, and that a desired output is obtained by a trainable combination of the response signals in the reservoir.”
From A reservoir of time constants for memory traces in cortical neurons (2011) · cites this paper as [13]
Citations
Cites
Wu, Y. (1999) · Technometrics · doi:10.1080/00401706.1999.10485951 · Cited by 27,221
Introduction to the theory of neural computation
(1994) · Neural Networks · doi:10.1016/0893-6080(94)90014-0 · Cited by 6,401
The Synaptic Organization of the Brain
(2004) · Oxford University Press eBooks · doi:10.1093/acprof:oso/9780195159561.001.1 · Cited by 3,546 · cited 2×
“This informationisprocessed by extremely complexbut surprisingly stereotypic microcircuits that can perform a wide spectrum of tasks (Shep-herd, 1988 ; Douglas & Martin, 1998 ; von Melchner, Pallas, & Sur, 2000).”
Dynamics of ongoing activity: explanation of the large variability in evoked cortical responses
Arieli, A., Sterkin, A., Grinvald, A., & Aertsen, A. (1996) · Science · doi:10.1126/science.273.5283.1868 · Cited by 1,660 · cited 1×
“Furthermore, although the input rate may contain salient information that is constant for a particular readout element, it may not be for another (see, for example, Figure 8), indicating that equivalence classes and dynamic stability exist purely from the perspective of the readout elements.”
BackgroundDifferential signaling via the same axon of neocortical pyramidal neurons
Markram, H., Wang, Y., & Tsodyks, M. (1998) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.95.9.5323 · Cited by 1,102 · cited 2×
“In the case of a synaptic connection from a to b , we modeled the synaptic dynamics according to the model proposed in Markram, Wang, and Tsodyks (1998), with the synaptic parameters U (use), D (time constant for depression), and F (time constant for facilitation) randomly chosen from gaussian…”
BackgroundMethodSelf-citationOrganizing principles for a diversity of GABAergic interneurons and synapses in the neocortex
Gupta, A., Wang, Y., & Markram, H. (2000) · Science · doi:10.1126/science.287.5451.273 · Cited by 1,036
Self-citationFading memory and the problem of approximating nonlinear operators with Volterra series
Boyd, S., & Chua, L. (1985) · IEEE Transactions on Circuits and Systems · doi:10.1109/tcs.1985.1085649 · Cited by 991 · cited 2×
“> The proof of this theorem follows from the Stone-Weierstrass approximation theorem, like the proof of theorem 1 in Boyd and Chua (1985).”
BackgroundGradient calculations for dynamic recurrent neural networks: a survey
Pearlmutter, B. A. (1995) · IEEE Transactions on Neural Networks · doi:10.1109/72.410363 · Cited by 582 · cited 1×
“Methods for controllingthe dynamics of recurrent neural networks through adaptive mechanisms are reviewed in Pearlmutter (1995).”
Temporal Information Transformed into a Spatial Code by a Neural Network with Realistic Properties
Buonomano, D. V., & Merzenich, M. M. (1995) · Science · doi:10.1126/science.7863330 · Cited by 449 · cited 1×
“Special cases of this mechanism have already been reported in Buonomano and Merzenich (1995) and Dominey, Arbib, and Joseph (1995).”
BackgroundLearning and Extracting Finite State Automata with Second-Order Recurrent Neural Networks
Giles, C. L., Miller, C. B., Chen, D., Chen, H. H., Sun, G. Z., & Lee, Y. C. (1992) · Neural Computation · doi:10.1162/neco.1992.4.3.393 · Cited by 392
Cited by
Items 801–900 of 1,014
Movement Generation and Control with Generic Neural Microcircuits
Joshi, P., & Maass, W. (2004) · Lecture notes in computer science · doi:10.1007/978-3-540-27835-1_20 · Cited by 48
Self-citationPhotonic Delay Systems as Machine Learning Implementations
Hermans, M., Soriano, M. C., Dambre, J., Bienstman, P., & Fischer, I. (2015) · DIGITAL.CSIC (Spanish National Research Council (CSIC)) · doi:10.48550/arxiv.1501.02592 · Cited by 39
“A steadily growing branch of research is concerned with Reservoir Computing (RC), a concept which employs high-dimensional, randomly initialized dynamical systems (termed the reservoir ) to perform feature extraction on time series ([14, 15, 21, 28, 20]).”
BackgroundApplications of spiking neural networks
Bohté, S. M., & Kok, J. N. (2005) · Information Processing Letters · doi:10.1016/j.ipl.2005.05.018 · Cited by 44
“One intriguing proposal for how to make use of networks of spiking neurons has been the Liquid State Machine, by Maass, Natschläger & Markram [3].”
BackgroundSurvey on Deep Fuzzy Systems in Regression Applications: A View on Interpretability
Júnior, J. S. S., Mendes, J., Souza, F. A. A., & Premebida, C. (2023) · International Journal of Fuzzy Systems · doi:10.1007/s40815-023-01544-8 · Cited by 46
“Echo State Networks (ESNs) are variations of RNNs that were developed in [146] and share the basic ideas of reservoir computing of Liquid State Machines from [147].”
BackgroundAnalog Photonics Computing for Information Processing, Inference, and Optimization
Stroev, N., & Berloff, N. G. (2023) · Advanced Quantum Technologies · doi:10.1002/qute.202300055 · Cited by 59
Heterogeneous reservoir computing in second-order Ta 2 O 5 /HfO 2 memristors
Ghenzi, N., Park, T. W., Kim, S. S., Kim, H. J., Jang, Y. H., Woo, K. S., & Hwang, C. S. (2023) · Nanoscale Horizons · doi:10.1039/d3nh00493g · Cited by 43
SpiLinC: Spiking Liquid-Ensemble Computing for Unsupervised Speech and Image Recognition
Srinivasan, G., Panda, P., & Roy, K. (2018) · Frontiers in Neuroscience · doi:10.3389/fnins.2018.00524 · Cited by 43
Hu, J., Lin, Y., Tang, J., & Zhao, J. (2020) · Applied Soft Computing · doi:10.1016/j.asoc.2020.106327 · Cited by 43
Reservoir computing with solitons
Silva, N. A., Ferreira, T. D., & Guerreiro, A. (2021) · New Journal of Physics · doi:10.1088/1367-2630/abda84 · Cited by 46
“In RC the training process consists in determining ¯ W that minimizes a given loss function [7, 8].”
BackgroundA neural architecture search based framework for liquid state machine design
Tian, S., Qu, L., Wang, L., Hu, K., Li, N., & Xu, W. (2021) · Neurocomputing · doi:10.1016/j.neucom.2021.02.076 · Cited by 47
Chen, H.-C., & Wei, D. (2021) · Nonlinear Dynamics · doi:10.1007/s11071-021-06452-w · Cited by 43
Artificial Intelligence in Classical and Quantum Photonics
Vernuccio, F., Bresci, A., Cimini, V., Giuseppi, A., Cerullo, G., Polli, D., & Valensise, C. M. (2022) · Laser & Photonics Review · doi:10.1002/lpor.202100399 · Cited by 46
Deriving task specific performance from the information processing capacity of a reservoir computer
Hülser, T., Köster, F., Lüdge, K., & Jaurigue, L. (2022) · Nanophotonics · doi:10.1515/nanoph-2022-0415 · Cited by 56
“Reservoir computing is a versatile, fast-trainable machine learning scheme inspired by the human brain [1, 2].”
Computing with a muscular-hydrostat system
Nakajima, K., Hauser, H., Kang, R., Guglielmino, E., Caldwell, D. G., & Pfeifer, R. (2013) · 2013 IEEE International Conference on Robotics and Automation · doi:10.1109/icra.2013.6630770 · Cited by 42
“In this paper, we first demonstrate whether the proposed system can achieve a complex computation by assigning benchmark tasks [22], [23], [37], [38], which are often used in the context of recurrent neural network learning [39] or the reservoir computing approach [37], [40], [41].”
MethodReservoir computing for emotion valence discrimination from EEG signals
Bozhkov, L., Koprinkova‐Hristova, P., & Georgieva, P. (2016) · Neurocomputing · doi:10.1016/j.neucom.2016.03.108 · Cited by 43
A new VLSI model of neural microcircuits including spike time dependent plasticity
Schemmel, J., Meier, K., & Muller, E. B. (2005) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn.2004.1380861 · Cited by 48
“In the view of the author this is the most promising approach for a computing principle beyond the Turing paradigm which has led the development of information technology since its introduction in 1936.”
BackgroundMethodInfluentialTraining Feedback Spiking Neural Networks by Implicit Differentiation on the Equilibrium StatePreprint
Xiao, M., Meng, Q., Zhang, Z., Wang, Y., & Lin, Z. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2109.14247 · Cited by 77
“As for SNNs, feedback was popular in early models like Liquid State Machine [25], which leverages a recurrent reservoir layer with weights fixed or trained by unsupervised methods.”
MethodOn the Quantification of Dynamics in Reservoir Computing
Verstraeten, D., & Schrauwen, B. (2009) · Lecture notes in computer science · doi:10.1007/978-3-642-04274-4_101 · Cited by 50
“Reservoir Computing (RC) [15], an idea that was originally independently introduced as Echo State Networks (ESN) [5] and Liquid State Machines (LSM) [7] has grown in the last few years into a research subfield that has attracted quite some attention.”
BackgroundReinhart, F., & Steil, J. J. (2009) · IEEE-RAS International Conference on Humanoid Robots (Humanoids) · doi:10.1109/ichr.2009.5379558 · Cited by 42
“Reservoir computing combines a fixed network of analog or spiking neurons with special read-out neurons and has been introduced under the notions of liquid state machine (LSM) [8], echo state network (ESN) [9], [10], or in an online learning setting, as backpropagation-decorrelation learning [11].”
BackgroundConsistency Properties of a Chaotic Semiconductor Laser Driven by Optical Feedback
Oliver, N., Jüngling, T., & Fischer, I. (2015) · Physical Review Letters · doi:10.1103/physrevlett.114.123902 · Cited by 47
Contextual effects in interval-duration judgements in vision, audition and touch
Burr, D. C., Rocca, E. D., & Morrone, M. C. (2013) · Experimental Brain Research · doi:10.1007/s00221-013-3632-z · Cited by 42
“Karmarkar and Buonomano (2007) noted that distractors influence temporal discriminations at short intervals, providing support for their model of duration estimation, where neural circuits inherently process time within complex networks driven by time-dependent properties of cell membranes (see Buonomano and Merzenich…”
BackgroundReservoir Computing Using Laser Networks
Röhm, A., Jaurigue, L., & Lüdge, K. (2019) · IEEE Journal of Selected Topics in Quantum Electronics · doi:10.1109/jstqe.2019.2927578 · Cited by 42
“A basic reservoir computer works with a sufficiently high dimensional and driven dynamical system [1], [2].”
BackgroundCunillera, A., Soriano, M. C., & Fischer, I. (2019) · Chaos An Interdisciplinary Journal of Nonlinear Science · doi:10.1063/1.5120822 · Cited by 43
Destination prediction using deep echo state network
Song, Z., Wu, K., & Shao, J. (2020) · Neurocomputing · doi:10.1016/j.neucom.2019.09.115 · Cited by 44
Predicting phase and sensing phase coherence in chaotic systems with machine learning
Zhang, C., Jiang, J., Xian, S., & Lai, Y. (2020) · Chaos An Interdisciplinary Journal of Nonlinear Science · doi:10.1063/5.0006304 · Cited by 42
Numerical simulation of artificial spin ice for reservoir computing
Hon, K., Kuwabiraki, Y., Goto, M., Nakatani, R., Suzuki, Y., & Nomura, H. (2021) · Applied Physics Express · doi:10.35848/1882-0786/abdcd8 · Cited by 44
Learning strange attractors with reservoir systems
Grigoryeva, L., Hart, A., & Ortega, J. (2023) · Nonlinearity · doi:10.1088/1361-6544/ace492 · Cited by 54
Learning Hamiltonian dynamics with reservoir computing
Zhang, H., Fan, H., Wang, L., & Wang, X. (2021) · Physical review. E · doi:10.1103/physreve.104.024205 · Cited by 52
“Whereas the output of RC does not satisfy the symplectic properties of the Hamiltonian, a high-dimensional reservoir does has the ability to fit any form of function by a property training of the output matrix [1].”
BackgroundPhotonic neuromorphic technologies in optical communications
Argyris, A. (2022) · Nanophotonics · doi:10.1515/nanoph-2021-0578 · Cited by 52
“While in the general RNN notation, the connections of all layers are trained, in the reservoir computing (RC) approach – also stated as echo-state network [42] or liquid-state machine [43] – offered a signi fi cant simpli fi cation for actual implementations.”
Topological constraints and robustness in liquid state machines
Hazan, H., & Manevitz, L. M. (2011) · Expert Systems with Applications · doi:10.1016/j.eswa.2011.06.052 · Cited by 46
“(i) In the original Maass topology the connections are chosen with a larger bias for nearby neurons (see Maass, 2002; Maass et al., 2002a; Maass, Natschläger, & Markram, 2002c).”
BackgroundMethodResultInfluentialSymbolic Computation Using Cellular Automata-Based Hyperdimensional Computing
Yılmaz, Ö. (2015) · Neural Computation · doi:10.1162/neco_a_00787 · Cited by 50
“…2007), us-ing a more capable second-order optimization procedure during gradient descent (Martens & Sutskever, 2011), using a reservoir of coupled oscillators (Jaeger, 2001; Maass, Natschl¨ager, & Markram, 2002), or emulating the recurrent computation in neural network framework (Yilmaz, 2015a).”
Kistler, W. M., & Zeeuw, C. I. D. (2002) · Neural Computation · doi:10.1162/089976602760407991 · Cited by 49
Computational theories on the function of theta oscillations
Lengyel, M., Huhn, Z., & Érdi, P. (2005) · Biological Cybernetics · doi:10.1007/s00422-005-0567-x · Cited by 49
“…Computations in the chaotic regime (or at the ‘edge of chaos’, without reaching any stable state at all) has recently become popular (Skarda and Freeman 1987; Tsuda 2001; Maass et al. 2002, but see Maei and Latham 2004). general points and common themes that emerge from different modeling studies.”
BackgroundReservoir Computing using Cellular AutomataPreprint
Yılmaz, Ö. (2014) · arXiv (Cornell University) · doi:10.48550/arxiv.1410.0162 · Cited by 47
“ever, recent advances suggest promising approaches in overcoming these issues, such as utilizing a reservoir of coupled oscillators [7, 8].”
BackgroundNeuromorphic Photonics, Principles of
Shastri, B. J., Tait, A. N., de Lima, T. F., Nahmias, M. A., Peng, H.-T., & Prucnal, P. R. (2018) · Springer eBooks · doi:10.1007/978-3-642-27737-5_702-1 · Cited by 52
Data-informed reservoir computing for efficient time-series prediction
Köster, F., Patel, D., Wikner, A., Jaurigue, L., & Lüdge, K. (2023) · Chaos An Interdisciplinary Journal of Nonlinear Science · doi:10.1063/5.0152311 · Cited by 45
Model scale versus domain knowledge in statistical forecasting of chaotic systems
Gilpin, W. (2023) · Physical Review Research · doi:10.1103/physrevresearch.5.043252 · Cited by 65
“…computers as strong forecasting methods for dynamical systems [13– 19]; these models use emergent properties of random networks to lift complex time series into a random fea-∗ wgilpin@utexas.edu ture space, thereby simplifying learning at the expense of increasing model complexity [20, 21].”
MethodResultIn-memory and in-sensor reservoir computing with memristive devices
Lin, N., Chen, J., Zhao, R., He, Y., Wong, K., Qiu, Q., Wang, Z., & Yang, J. J. (2024) · APL Machine Learning · doi:10.1063/5.0174863 · Cited by 42
Bourjaily, M. A., & Miller, P. C. (2011) · Frontiers in Computational Neuroscience · doi:10.3389/fncom.2011.00037 · Cited by 41
“Others have shown that random recurrent networks can produce useful computational functions (Maass et al., 2002) with their power enhanced when multiple plasticity mechanisms combine synergistically (Lazar et al.”
BackgroundReal-time Audio Processing with a Cascade of Discrete-Time Delay Line-Based Reservoir Computers
Keuninckx, L., Danckaert, J., & der Sande, G. V. (2017) · Cognitive Computation · doi:10.1007/s12559-017-9457-5 · Cited by 41
“Classical echo state networks (ESNs) [1] and liquid state machines (LSMs) [2], now collectively known as reservoir computers (RCs) [3], utilize a large recurrent network of many randomly connected nonlinear nodes or neurons.”
BackgroundSynaptic patterning and the timescales of cortical dynamics
Duarte, R., Seeholzer, A., Zilles, K., & Morrison, A. (2017) · Current Opinion in Neurobiology · doi:10.1016/j.conb.2017.02.007 · Cited by 42
Computing hubs in the hippocampus and cortex
Clawson, W. P., Vicente, A. F., Ferraris, M., Bernard, C., Battaglia, D. A., & Quilichini, P. (2019) · Science Advances · doi:10.1126/sciadv.aax4843 · Cited by 41
“58 The second goal is to determine whether primitive processing operations are 59 localized, or on the contrary, distributed within the microcircuit as proposed for 60 attractor neural networks (8) and liquid state machines (9).”
BackgroundThe reservoir’s perspective on generalized synchronization
Lymburn, T., Walker, D. M., SMALL, M. P., & Jüngling, T. (2019) · Chaos An Interdisciplinary Journal of Nonlinear Science · doi:10.1063/1.5120733 · Cited by 50
Experimental realization of dual task processing with a photonic reservoir computer
Vatin, J., Rontani, D., & Sciamanna, M. (2020) · APL Photonics · doi:10.1063/5.0017574 · Cited by 41
Ranzini, S. M., Dischler, R., Ros, F. D., Bülow, H., & Zibar, D. (2021) · Journal of Lightwave Technology · doi:10.1109/jlt.2021.3049473 · Cited by 47
“This concept was introduced by Jaeger with echo state networks (ESN) [22] and Mass with the liquid state machine (LSM) [23], independently.”
MethodTwin vortex computer in fluid flow
Goto, K., Nakajima, K., & Notsu, H. (2021) · New Journal of Physics · doi:10.1088/1367-2630/ac024d · Cited by 47
Asynchronous photonic time-delay reservoir computing
Tang, J.-Y., Lin, B.-D., Shen, Y.-W., Li, R., Yu, J., He, X., & Wang, C. (2022) · Optics Express · doi:10.1364/oe.478728 · Cited by 43
Klampfl, S., David, S. V., Yin, P., Shamma, S., & Maass, W. (2012) · Journal of Neurophysiology · doi:10.1152/jn.00935.2011 · Cited by 41
Self-citationForecasting trends in food security with real time data
Herteux, J., Rath, C. W., Martini, G., Baha, A., Koupparis, K., Lauzana, I., & Piovani, D. (2024) · Communications Earth & Environment · doi:10.1038/s43247-024-01698-9 · Cited by 40
“The model we identify as the most suitable for forecasting trends in food insecurity is based on Reservoir Computing (RC) 38 – 40 , a type of recurrent neural network (RNN).”
Code, Context, and Epigenetic Catalysis in Gene Expression
Wallace, R., & Wallace, D. (2009) · Lecture notes in computer science · doi:10.1007/978-3-642-04186-0_13 · Cited by 60
Initialization and self‐organized optimization of recurrent neural network connectivity
Boedecker, J., Obst, O., Mayer, N. M., & Asada, M. (2009) · HFSP journal · doi:10.2976/1.3240502 · Cited by 44
“, Rad et al. (2008); Schmidhuber et al. (2007, 2005); Schrauwen et al. (2008); Steil (2007); Wierstra et al. (2005)]; however, as of yet there is no universally accepted standard method. We study the effect of approaches aiming to reduce this dependency on a random initialization. In order to do this, we investigate t…”
BackgroundMethodInfluentialProbabilistic Computational Neurogenetic Modeling: From Cognitive Systems to Alzheimer's Disease
Kasabov, N. K., Schliebs, R., & Kojima, H. (2011) · IEEE Transactions on Autonomous Mental Development · doi:10.1109/tamd.2011.2159839 · Cited by 45
The Use of Hebbian Cell Assemblies for Nonlinear Computation
Tetzlaff, C., Dasgupta, S., Kulvičius, T., & Wörgötter, F. (2015) · Scientific Reports · doi:10.1038/srep12866 · Cited by 43
“It is known, on the one hand, that networks can be trained to perform complex non-linear calculations 9–11 , which could be used for motor control.”
InfluentialEnhanced Responsiveness and Low-Level Awareness in Stochastic Network States
Zerlaut, Y., & Destexhe, A. (2017) · Neuron · doi:10.1016/j.neuron.2017.04.001 · Cited by 40
Yue, D., Wu, Z.-M., Hou, Y., Cui, B., Jin, Y.-H., Dai, M., & Xia, G.-Q. (2019) · Optics Express · doi:10.1364/oe.27.019931 · Cited by 43
Learn to synchronize, synchronize to learn
Verzelli, P., Alippi, C., & Livi, L. (2021) · Chaos An Interdisciplinary Journal of Nonlinear Science · doi:10.1063/5.0056425 · Cited by 46
“and state-of-the-art results obtained in solving various tasks [27, 12, 54, 8]. RC was introduced independently by Jaeger [21] (who used the term Echo State Network), Maass, Natschlager, and Markram¨ [30] (Liquid State Machine) and Tino and Dorffner [49] (Fractal Predicting Machine). In order to account for Neuralˇ Ne…”
MethodEcho state network models for nonlinear Granger causality
Duggento, A., Guerrisi, M., & Toschi, N. (2021) · Philosophical Transactions of the Royal Society A Mathematical Physical and Engineering Sciences · doi:10.1098/rsta.2020.0256 · Cited by 40
Towards Comprehensive Foundations of Computational Intelligence
Duch, W. (2007) · Studies in computational intelligence · doi:10.1007/978-3-540-71984-7_11 · Cited by 43
“Random linear projection of input vectors into a high-dimensional space 2 X = L ( 1 X ) is the simplest expansion, with the random matrix L that has more rows than columns (see neurobiological justification of such projections in [121]).”
BackgroundKitano, K., & Fukai, T. (2007) · Journal of Computational Neuroscience · doi:10.1007/s10827-007-0030-1 · Cited by 46
“2005) and may play an active role in cortical computations (Destexhe and Paré 1999; Fukai 2000; Maass et al. 2002; Wolfart et al. 2005).”
BackgroundRandomly Connected Networks Have Short Temporal Memory
Wallace, E. W. J., Maei, H. R., & Latham, P. E. (2013) · Neural Computation · doi:10.1162/neco_a_00449 · Cited by 40
“For example, in simulations of integrate-and-fire neurons, temporal memory is on the same order as the time constants of the synapses (Maass et al., 2002); both are about 1 second.”
BackgroundMethodInfluentialA New Hypothesis for Sleep: Tuning for Criticality
Pearlmutter, B. A., & Houghton, C. J. (2009) · Neural Computation · doi:10.1162/neco.2009.05-08-787 · Cited by 38
“As we have discussed, criticality has engineering advantages; furthermore, some learning algorithms for temporal sequences, such as liquid state machines (Maass et al., 2002) and echo state networks (Jaeger and Haas, 2004), perform well only when their recurrent network modules are near the boundary of critical-”
BackgroundBrain-Machine Interface Engineering
Sanchez, J. C., & Príncipe, J. C. (2007) · Synthesis lectures on biomedical engineering · doi:10.1007/978-3-031-01621-9 · Cited by 44
Brain-inspired computing systems: a systematic literature review
Zolfagharinejad, M., Alegre-Ibarra, U., Chen, T., Kinge, S., & van der Wiel, W. G. (2024) · The European Physical Journal B · doi:10.1140/epjb/s10051-024-00703-6 · Cited by 39
In-sensor reservoir computing based on optoelectronic synaptic devices
Chen, Z., Xiao, Y., Huang, W.-Y., Jiang, Y., Liu, Q., & Tang, X. (2023) · Applied Physics Letters · doi:10.1063/5.0160599 · Cited by 39
Panahi, S., & Lai, Y. (2024) · Chaos An Interdisciplinary Journal of Nonlinear Science · doi:10.1063/5.0200898 · Cited by 47
Controlling chaos using edge computing hardware
Kent, R. M., Barbosa, W. A. S., & Gauthier, D. J. (2024) · Nature Communications · doi:10.1038/s41467-024-48133-3 · Cited by 39
“Here, we brie fl y summarize the NG-RC algorithm 12,13 used to estimate ^ W 1 u and ^ F needed for the control law.”
Evolving Local Plasticity Rules for Synergistic Learning in Echo State Networks
Wang, X., Jin, Y., & Hao, K. (2019) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2019.2919903 · Cited by 43
“E CHO state networks (ESNs) and liquid state machines (LSMs) were proposed by Jaeger et al. [1] in 2001 and Maass et al. [2] in 2002, respectively, which are commonly known as reservoir computing models.”
BackgroundLimitations of the Recall Capabilities in Delay-Based Reservoir Computing Systems
Köster, F., Ehlert, D., & Lüdge, K. (2020) · Cognitive Computation · doi:10.1007/s12559-020-09733-5 · Cited by 39
Parallel Time-Delay Reservoir Computing With Quantum Dot Lasers
Tang, J.-Y., Lin, B.-D., Yu, J., He, X., & Wang, C. (2022) · IEEE Journal of Quantum Electronics · doi:10.1109/jqe.2022.3146561 · Cited by 41
DNA Reservoir Computing: A Novel Molecular Computing Approach
Goudarzi, A., Lakin, M. R., & Stefanović, D. (2013) · Lecture notes in computer science · doi:10.1007/978-3-319-01928-4_6 · Cited by 39
“RC was developed independently by Maass et al. [7] as a model of information processing in cortical microcircuits, and by Jaeger [8] as an alternative approach to time-series analysis using Recurrent Neural Networks (RNN).”
MethodAn associative memory readout for ESNs with applications to dynamical pattern recognition
Ozturk, M. C., & Príncipe, J. C. (2007) · Neural Networks · doi:10.1016/j.neunet.2007.04.012 · Cited by 39
Quevedo, J., Chen, H., Escofet, M. À. C., Peter Tiňo, B., Puig, V., García, D., Sarrate, R., & Yao, X. (2014) · Engineering Applications of Artificial Intelligence · doi:10.1016/j.engappai.2014.01.008 · Cited by 39
“Popular RC methods include echo state networks (ESNs) (Jaeger, 2001), liquid state machines (LSMs) (Maass et al., 2002) and the back-propagation decorrelation neural network (Steil, 2004).”
Edge of Chaos Computation in Mixed-Mode VLSI - A Hard Liquid
Schürmann, F., Meier, K., & Schemmel, J. (2004) · neural information processing systems · Cited by 39
Learning Beyond Finite Memory in Recurrent Networks of Spiking Neurons
Peter Tiňo, B., & Mills, A. J. S. (2006) · Neural Computation · doi:10.1162/089976606775623360 · Cited by 39
Craddock, T. J. A., Priel, A., & Tuszynski, J. A. (2014) · Journal of Integrative Neuroscience · doi:10.1142/s0219635214400019 · Cited by 38
Neuromorphic Liquids, Colloids, and Gels: A Review
Kheirabadi, N. R., Chiolerio, A., Szacilowski, K. T., & Adamatzky, A. (2022) · ChemPhysChem · doi:10.1002/cphc.202200390 · Cited by 38
Extended liquid state machines for speech recognition
Deckers, L., Tsang, I. J., Leekwijck, W. V., & Latré, S. (2022) · Frontiers in Neuroscience · doi:10.3389/fnins.2022.1023470 · Cited by 43
“In order to show that the effects of the proposed LSM extensions are generic and robust, we selected four speech data sets for evaluation: (1) The Spiking Heidelberg Digits data set, a commonly used benchmark for SNN; (2) TI-Alpha, an LSM benchmark data set; (3) Google Speech Commands, a more…”
BackgroundMethodInfluentialNeuromorphic Computing via Fission‐based Broadband Frequency Generation
Fischer, B., Chemnitz, M., Zhu, Y. M., Perron, N., Roztocki, P., MacLellan, B., Lauro, L. D., Aadhi, A. R., Rimoldi, C., Falk, T. H., et al. (2023) · Advanced Science · doi:10.1002/advs.202303835 · Cited by 42
Virtualization of a Photonic Reservoir Computer
Duport, F., Smerieri, A., Akrout, A., Haelterman, M., & Massar, S. (2016) · Journal of Lightwave Technology · doi:10.1109/jlt.2016.2524559 · Cited by 42
“Jaeger and Haas [1] under the term “Echo State Networks,” by Maas et al.[2] under the term “Liquid State Machine” and unified under the generic concept of “Reservoir Computing” by Verstraeten et al.”
BackgroundScaling Up Echo-State Networks With Multiple Light Scattering
Dong, J., Gigan, S., Krząkała, F., & Wainrib, G. (2018) · IEEE Statistical Signal Processing Workshop (SSP) · doi:10.1109/ssp.2018.8450698 · Cited by 38
“∗This research has received funding from the European Research Council under the EUs 7th Framework Programme (FP/2007-2013/ERC Grant Agreement 307087-SPARCS and 278025-COMEDIA) Neural Network where neurons are connected with random weights [5, 6, 7].”
MethodJin, Y., & Li, P. (2016) · Neurocomputing · doi:10.1016/j.neucom.2016.11.045 · Cited by 40
“…randomly under a distribution such that the wiring probability of any two neurons ( N i and N j ) drops exponentially in the distance between them ([30]): where D N N ( , ) i j is the Euclidean distance between these two neurons, r is chosen to control the exponential decay of the probability, and…”
BackgroundInfluentialSemi-supervised Echo State Networks for Audio Classification
Scardapane, S., & Uncini, A. (2016) · Cognitive Computation · doi:10.1007/s12559-016-9439-z · Cited by 43
“In particular, we note that ESNs have attracted a widespread interest in neuroscience, where an equivalent family of techniques, denoted as liquid state machines, are commonly used as a tool for the analysis of neural circuits [29].”
MethodDetection of generalized synchronization using echo state networks
Ibáñez-Soria, D., García‐Ojalvo, J., Soria‐Frisch, A., & Ruffini, G. (2018) · Chaos An Interdisciplinary Journal of Nonlinear Science · doi:10.1063/1.5010285 · Cited by 41
Molecules, semiconductors, light and information: Towards future sensing and computing paradigms
Pilarczyk, K., Wlaźlak, E., Przyczyna, D., Blachecki, A., Podborska, A., Athanasiou, V., Konkoli, Z., & Szacilowski, K. T. (2018) · Coordination Chemistry Reviews · doi:10.1016/j.ccr.2018.03.018 · Cited by 40
Neuromorphic computing through time-multiplexing with a spin-torque nano-oscillator
Riou, M., Araujo, F. A., Torrejón, J., Tsunegi, S., Khalsa, G., Querlioz, D., Bortolotti, P., Cros, V., Yakushiji, K., Fukushima, A., et al. (2017) · Technical digest - International Electron Devices Meeting/Technical digest · doi:10.1109/iedm.2017.8268505 · Cited by 38
“1) [1], [2].”
Guo, X., Xiang, S., Qu, Y., Han, Y. N., Wen, A. J., & Hao, Y. (2020) · Journal of Lightwave Technology · doi:10.1109/jlt.2020.3023451 · Cited by 38
“gence, which has attracted widespread attention in recent years [5]–[11].”
BackgroundInsight into delay based reservoir computing via eigenvalue analysis
Köster, F., Yanchuk, S., & Lüdge, K. (2021) · Journal of Physics Photonics · doi:10.1088/2515-7647/abf237 · Cited by 44
Catch-22s of reservoir computing
Zhang, Y., & Cornelius, S. P. (2023) · Physical Review Research · doi:10.1103/physrevresearch.5.033213 · Cited by 41
“Reservoir Computing (RC) [1–12] is a machine learning framework for time-series predictions based on recurrent neural networks.”
BackgroundTime Warping Invariant Echo State Networks
Lukoševičius, M., Popovici, D., Jaeger, H., & Siewert, U. (2006) · Cited by 38
Roy, S., Member, S., Banerjee, A., & Basu, A. (2016) · Cited by 38
Structural plasticity for neuromorphic networks with electropolymerized dendritic PEDOT connections
Janzakova, K., Balafrej, I., Kumar, A., Garg, N., Scholaert, C., Rouat, J., Drouin, D., Coffinier, Y., Pecqueur, S., & Alibart, F. (2023) · Nature Communications · doi:10.1038/s41467-023-43887-8 · Cited by 37
“An attractive option to relaxthe topologies de fi nition requirement has been to consider random topologies for computing, such as in reservoir computing approaches 16 .”
Thalamocortical architectures for flexible cognition and efficient learning
Scott, D. N., Mukherjee, A., Nassar, M. R., & Halassa, M. M. (2024) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2024.05.006 · Cited by 62
Ahmed, O., Tennie, F., & Magri, L. (2024) · Physical Review Research · doi:10.1103/physrevresearch.6.043082 · Cited by 37
“Reservoir computing is a unified computing framework that was introduced as ’Echo State Networks’ [15] and ’Liquid State Machines’ [16].”
MethodEnergy-Efficient Pattern Recognition Hardware With Elementary Cellular Automata
Morán, A., Frasser, C. F., Roca, M., & Rosselló, J. L. (2019) · IEEE Transactions on Computers · doi:10.1109/tc.2019.2949300 · Cited by 42
“Reservoir Computing (RC) [2], [3] is an attractive machine learning alternative due to its simplicity and computationally inexpensive learning process and that can be suitable for”
BackgroundUsing machine learning to assess short term causal dependence and infer network links
Banerjee, A., Pathak, J., Roy, R., Restrepo, J. G., & Ott, E. (2019) · Chaos An Interdisciplinary Journal of Nonlinear Science · doi:10.1063/1.5134845 · Cited by 37
“27 and 28 (for a review, see Ref.”
Inactivation of the Medial Entorhinal Cortex Selectively Disrupts Learning of Interval Timing
Heys, J. G., Wu, Z.-H., Mascaro, A. L. A., & Dombeck, D. A. (2020) · Cell Reports · doi:10.1016/j.celrep.2020.108163 · Cited by 37
“Another class of models propose that interval timing could arise through sequential activity produced by training of randomly recurrent neural networks (Buonomano and Merzenich 1995; Maass et al. 2002; Laje and Buonomano, 2013; Hardy et al. 2018).”
Multiresolution Reservoir Graph Neural Network
Pasa, L., Navarin, N., & Sperduti, A. (2021) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2021.3090503 · Cited by 41
Remaining useful life prediction of PEMFC based on cycle reservoir with jump model
Jin, J., Chen, Y., Xie, C., Zhu, W., & Wu, F. (2021) · International Journal of Hydrogen Energy · doi:10.1016/j.ijhydene.2021.09.233 · Cited by 37
Neural Schrödinger Equation: Physical Law as Deep Neural Network
Nakajima, M., Tanaka, K., & Hashimoto, T. (2021) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2021.3120472 · Cited by 43
Echo state network activation function based on bistable stochastic resonance
Liao, Z., Wang, Z., Yamahara, H., & Tabata, H. (2021) · Chaos Solitons & Fractals · doi:10.1016/j.chaos.2021.111503 · Cited by 40
Metadata
- Authors
- Maass, W., Natschläger, T., & Markram, H.
- Affiliations
- Graz University of Technology (AT)
- École Polytechnique Fédérale de Lausanne (CH)
- Corresponding author
- Maass, Wolfgang
- Natschläger, Thomas
- Markram, Henry
- Venue
- Neural Computation
- Publisher
- MIT Press
- Published
- 2002-11-01
- Volume
- 14
- Issue
- 11
- Pages
- 2531-2560
- ISSN
- 0899-7667 (ISSN-L)
- 1530-888X
- OpenAlex
- W2103179919
- Cited by
- 4,025
- References
- 28
- Citations by year
- 2026: 286 · 2025: 351 · 2024: 307 · 2023: 309 · 2022: 276 · 2021: 303 · 2020: 239 · 2019: 244 · 2018: 217 · 2017: 173 · 2016: 155 · 2015: 135 · 2014: 142 · 2013: 139 · 2012: 125
- FWCI
- 18.37
- Type
- journal article
- Language
- en
- Open access
- green
- infoscience.epfl.ch
- Locations
- Neural Computation (publishedVersion)
- PubMed (publishedVersion)
- Infoscience (Ecole Polytechnique Fédérale de Lausanne) (submittedVersion)
- Infoscience (Ecole Polytechnique Fédérale de Lausanne) (submittedVersion, cc-by-nc-nd)
- Topics
- Life Sciences › Neuroscience › Cognitive Neuroscience › Neural dynamics and brain function
- Physical Sciences › Computer Science › Artificial Intelligence › Neural Networks and Reservoir Computing
- Physical Sciences › Computer Science › Artificial Intelligence › Neural Networks and Applications
- Keywords
- liquid state machines, recurrent neural circuits, transient dynamics, high-dimensional dynamical systems, fading memory, integrate-and-fire neurons, real-time processing, time-varying inputs, neural coding, statistical learning theory, computational power, attractor networks
- MeSH terms
- Action Potentials, Computer Simulation, Computer Systems, Computers, Models, Neurological, Neurons, Neural Networks, Computer
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.