Real-Time Computing Without Stable States: A New Framework for Neural Computation Based on Perturbations
Open accessThe DOI matches its Crossref record; 28 of the 28 references it cites were found in a registry or an index.
Abstract
A key challenge for neural modeling is to explain how a continuous stream of multimodal input from a rapidly changing environment can be processed by stereotypical recurrent circuits of integrate-and-fire neurons in real time. We propose a new computational model for real-time computing on time-varying input that provides an alternative to paradigms based on Turing machines or attractor neural networks. It does not require a task-dependent construction of neural circuits. Instead, it is based on principles of high-dimensional dynamical systems in combination with statistical learning theory and can be implemented on generic evolved or found recurrent circuitry. It is shown that the inherent transient dynamics of the high-dimensional dynamical system formed by a sufficiently large and heterogeneous neural circuit may serve as universal analog fading memory. Readout neurons can learn to extract in real time from the current state of such recurrent neural circuit information about current and past inputs that may be needed for diverse tasks. Stable internal states are not required for giving a stable output, since transient internal states can be transformed by readout neurons into stable target outputs due to the high dimensionality of the dynamical system. Our approach is based on a rigorous computational model, the liquid state machine, that, unlike Turing machines, does not require sequential transitions between well-defined discrete internal states. It is supported, as the Turing machine is, by rigorous mathematical results that predict universal computational power under idealized conditions, but for the biologically more realistic scenario of real-time processing of time-varying inputs. Our approach provides new perspectives for the interpretation of neural coding, the design of experiments and data analysis in neurophysiology, and the solution of problems in robotics and neurotechnology.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Although work in artificial intelligence and machine-learning research has shown that similar or even more complex outcomes can be achieved with a variety of techniques, including deep neural networks with trained weights (Gibney, 2015), reservoir networks with a trained output mapping (Maass et al., 2002;Jaeger & Haas, 2004), networks that are evolved using a genetic algorithm (Iizuka & Di Paolo,”
From A potential mechanism for Gibsonian resonance: behavioral entrainment emerges from local homeostasis in an unsupervised reservoir network (2024) · cites this paper as Maass et al., 2002
In the citing paper
“Finally, we show that a randomly connected circuit model, akin to a “reservoir” network13-15, can reproduce the observed distribution of timescales, provided that the network operates at the critical point (or “edge of chaos”)16-18.”
From A reservoir of time constants for memory traces in cortical neurons (2011) · cites this paper as [13]
In the citing paper
“Reservoir-type networks have been the subject of active research in computational neuroscience and machine learning13-15, but so far experimental support that such networks are adopted by the brain has been lacking.”
From A reservoir of time constants for memory traces in cortical neurons (2011) · cites this paper as [13]
In the citing paper
“Those models predict that the memory of input signals is stored in a large, recurrent and heterogeneous network (reservoir), in a distributed manner, and that a desired output is obtained by a trainable combination of the response signals in the reservoir.”
From A reservoir of time constants for memory traces in cortical neurons (2011) · cites this paper as [13]
Citations
Cites
Wu, Y. (1999) · Technometrics · doi:10.1080/00401706.1999.10485951 · Cited by 27,221
Introduction to the theory of neural computation
(1994) · Neural Networks · doi:10.1016/0893-6080(94)90014-0 · Cited by 6,401
The Synaptic Organization of the Brain
(2004) · Oxford University Press eBooks · doi:10.1093/acprof:oso/9780195159561.001.1 · Cited by 3,546 · cited 2×
“This informationisprocessed by extremely complexbut surprisingly stereotypic microcircuits that can perform a wide spectrum of tasks (Shep-herd, 1988 ; Douglas & Martin, 1998 ; von Melchner, Pallas, & Sur, 2000).”
Dynamics of ongoing activity: explanation of the large variability in evoked cortical responses
Arieli, A., Sterkin, A., Grinvald, A., & Aertsen, A. (1996) · Science · doi:10.1126/science.273.5283.1868 · Cited by 1,660 · cited 1×
“Furthermore, although the input rate may contain salient information that is constant for a particular readout element, it may not be for another (see, for example, Figure 8), indicating that equivalence classes and dynamic stability exist purely from the perspective of the readout elements.”
BackgroundDifferential signaling via the same axon of neocortical pyramidal neurons
Markram, H., Wang, Y., & Tsodyks, M. (1998) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.95.9.5323 · Cited by 1,102 · cited 2×
“In the case of a synaptic connection from a to b , we modeled the synaptic dynamics according to the model proposed in Markram, Wang, and Tsodyks (1998), with the synaptic parameters U (use), D (time constant for depression), and F (time constant for facilitation) randomly chosen from gaussian…”
BackgroundMethodSelf-citationOrganizing principles for a diversity of GABAergic interneurons and synapses in the neocortex
Gupta, A., Wang, Y., & Markram, H. (2000) · Science · doi:10.1126/science.287.5451.273 · Cited by 1,036
Self-citationFading memory and the problem of approximating nonlinear operators with Volterra series
Boyd, S., & Chua, L. (1985) · IEEE Transactions on Circuits and Systems · doi:10.1109/tcs.1985.1085649 · Cited by 991 · cited 2×
“> The proof of this theorem follows from the Stone-Weierstrass approximation theorem, like the proof of theorem 1 in Boyd and Chua (1985).”
BackgroundGradient calculations for dynamic recurrent neural networks: a survey
Pearlmutter, B. A. (1995) · IEEE Transactions on Neural Networks · doi:10.1109/72.410363 · Cited by 582 · cited 1×
“Methods for controllingthe dynamics of recurrent neural networks through adaptive mechanisms are reviewed in Pearlmutter (1995).”
Temporal Information Transformed into a Spatial Code by a Neural Network with Realistic Properties
Buonomano, D. V., & Merzenich, M. M. (1995) · Science · doi:10.1126/science.7863330 · Cited by 449 · cited 1×
“Special cases of this mechanism have already been reported in Buonomano and Merzenich (1995) and Dominey, Arbib, and Joseph (1995).”
BackgroundLearning and Extracting Finite State Automata with Second-Order Recurrent Neural Networks
Giles, C. L., Miller, C. B., Chen, D., Chen, H. H., Sun, G. Z., & Lee, Y. C. (1992) · Neural Computation · doi:10.1162/neco.1992.4.3.393 · Cited by 392
Cited by
Items 301–400 of 1,014
Bits from Brains for Biologically Inspired Computing
Wibral, M., Lizier, J. T., & Priesemann, V. (2015) · Frontiers in Robotics and AI · doi:10.3389/frobt.2015.00005 · Cited by 128
“ACTIVE INFORMATION STORAGE IN A ROBOTIC SYSTEM Recurrent neural networks (RNNs) consist of a reservoir of nodes or artificial neurons connected in some recurrent network structure (Maass et al., 2002; Jaeger and Haas, 2004).”
MethodNguimdo, R. M., Verschaffelt, G., Danckaert, J., & der Sande, G. V. (2014) · Optics Express · doi:10.1364/oe.22.008672 · Cited by 141
LISNN: Improving Spiking Neural Networks with Lateral Interactions for Robust Object Recognition
Cheng, X., Hao, Y., Xu, J., & Xu, B. (2020) · International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2020/211 · Cited by 178
“Liquid State Machine (LSM), a special RSNN, contains an input module, a recurrent reservoir layer, and a readout module [Maass et al. , 2002].”
BackgroundEmerging dynamic memristors for neuromorphic reservoir computing
Cao, J., Zhang, X., Cheng, H., Qiu, J., Liu, X., Wang, M., & Liu, Q. (2022) · Nanoscale · doi:10.1039/d1nr06680c · Cited by 134
Symmetry Breaking in Space-Time Hierarchies Shapes Brain Dynamics and Behavior
Pillai, A. S., & Jirsa, V. (2017) · Neuron · doi:10.1016/j.neuron.2017.05.013 · Cited by 140
Temporal Information Processing on Noisy Quantum Computers
Chen, J., Nurdin, H. I., & Yamamoto, N. (2020) · Physical Review Applied · doi:10.1103/physrevapplied.14.024065 · Cited by 168
Locomotion Without a Brain: Physical Reservoir Computing in Tensegrity Structures
Caluwaerts, K., D'Haene, M., Verstraeten, D., & Schrauwen, B. (2013) · Artificial Life · doi:10.1162/artl_a_00080 · Cited by 134
“Most importantly, we distinguish liquid state machines [43] and echo state networks [34, 35].”
BackgroundSecond type of criticality in the brain uncovers rich multiple-neuron dynamics
Dahmen, D., Grün, S., Diesmann, M., & Helias, M. (2019) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1818972116 · Cited by 149
Reservoir Computing Universality With Stochastic Inputs
Gonon, L., & Ortega, J. (2019) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2019.2899649 · Cited by 177
“The approximants used are small subfamilies of reservoir computers (RCs) [7], [8] or reservoir systems.”
BackgroundDynamical Phase Transitions in Quantum Reservoir Computing
Martínez‐Peña, R., Giorgi, G. L., Nokkala, J., Soriano, M. C., & Zambrini, R. (2021) · Physical Review Letters · doi:10.1103/physrevlett.127.100502 · Cited by 185
Implementing Synaptic Plasticity in a VLSI Spiking Neural Network Model
Schemmel, J., Grübl, A., Meier, K., & Muller, E. B. (2006) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn.2006.246651 · Cited by 148
Reservoir computing with a single time-delay autonomous Boolean node
Haynes, N. D., Soriano, M. C., Rosin, D. P., Fischer, I., & Gauthier, D. J. (2015) · Physical Review E · doi:10.1103/physreve.91.020801 · Cited by 147
Pruning and regularization in reservoir computing
Dutoit, X., Schrauwen, B., Campenhout, J. V., Stroobandt, D., Brussel, H. V., & Nuttin, M. (2009) · Neurocomputing · doi:10.1016/j.neucom.2008.12.020 · Cited by 143
Low-Power, Adaptive Neuromorphic Systems: Recent Progress and Future Directions
Basu, A., Acharya, J., Karnik, T., Liu, H., Li, H. H., Seo, J.-S., & Song, C. (2018) · IEEE Journal on Emerging and Selected Topics in Circuits and Systems · doi:10.1109/jetcas.2018.2816339 · Cited by 135
“Compared to the reservoir computing methods [68], the major difference of ELM is the lack of feedback or recurrent connections.”
MethodMiconi, T. (2017) · eLife · doi:10.7554/elife.20899 · Cited by 126
“Models based on recurrent neural networks (RNN), operating in the near-chaotic regime, seem well-suited to capture similar dynamics (Jaeger 2001; Maass, Natschläger, and Markram 2002; Buonomano and Maass 2009; Sussillo and Abbott 2009).”
BackgroundMethodInfluentialTraining BatchNorm and Only BatchNorm: On the Expressive Power of Random Features in CNNs
Frankle, J., Schwab, D. J., & Morcos, A. S. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2003.00152 · Cited by 74
“Although prior work considers models with random features and a train-able affine output layer (e.g., Rahimi & Recht, 2009; Jaeger, 2003; Maass et al., 2002), we reveal the expressive power of networks configured such that trainable affine parameters appear after each layer of random features.”
BackgroundExploiting short-term memory in soft body dynamics as a computational resource
Nakajima, K., Li, T., Hauser, H., & Pfeifer, R. (2014) · Journal of The Royal Society Interface · doi:10.1098/rsif.2014.0437 · Cited by 128
“[10] Maass W, Natschläger T, Markram H.”
BackgroundInfluentialAvalanches and criticality in self-organized nanoscale networks
Mallinson, J. B., Shirai, S., Acharya, S. K., Bose, S. K., Galli, E., & Brown, S. A. (2019) · Science Advances · doi:10.1126/sciadv.aaw8438 · Cited by 139
“All of these approaches rely on highly ordered architectures, but neuromorphic computational paradigms such as reservoir computing (RC) (10–12) require complex networks with strong inherent spatiotemporal correlations (13, 14).”
BackgroundQuantum reservoir computing with a single nonlinear oscillator
Govia, L. C. G., Ribeill, G. J., Rowlands, G. E., Krovi, H., & Ohki, T. (2021) · Physical Review Research · doi:10.1103/physrevresearch.3.013077 · Cited by 160
Simple Deterministically Constructed Cycle Reservoirs with Regular Jumps
Rodan, A., & Tiňo, P. (2012) · Neural Computation · doi:10.1162/neco_a_00297 · Cited by 141
“Liquid state machines (LSM) (Maass et al., 2002) also have mostly linear readout, and the reservoirs are driven by the dynamics of a set of coupled spiking neuron models.”
BackgroundComputing with Neural Synchrony
Brette, R. (2012) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1002561 · Cited by 141
“One theory, reservoir computing, assigns a computational role to neural heterogeneity, that of representing sensory stimuli in a high-dimensional space where decoding is easier [23], but it does not assign a specific role to spike timing or synchrony.”
BackgroundLearning in the Model Space for Cognitive Fault Diagnosis
Chen, H., Peter Tiňo, B., Rodan, A., & Yao, X. (2014) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2013.2256797 · Cited by 127
“Popular RC methods include echo state networks (ESNs) [17], [18], liquid state machines [19], and the backpropagation decorrelation neural network [20].”
full-FORCE: A target-based method for training recurrent networks
DePasquale, B., Cueva, C. J., Rajan, K., Escola, G. S., & Abbott, L. F. (2018) · PLoS ONE · doi:10.1371/journal.pone.0191527 · Cited by 159
Cell signaling as a cognitive process
Koseska, A., & Bastiaens, P. I. H. (2017) · The EMBO Journal · doi:10.15252/embj.201695383 · Cited by 137
“Making the parallel to neural networks and the concept of reservoir computing (Box 7), it is theoretically possible for recurrent networks that employ dynamic processes on different timescales to process information using transiently stable attractors (Maass et al , 2002).”
Memristor-Based Echo State Network With Online Least Mean Square
Wen, S., Hu, R., Yang, Y., Huang, T., Zeng, Z., & Song, Y. (2018) · IEEE Transactions on Systems Man and Cybernetics Systems · doi:10.1109/tsmc.2018.2825021 · Cited by 129
“In 2002, ESN is collectively referred to as “reservoir computing model” [18].”
BackgroundEmbedding and approximation theorems for echo state networks
Hart, A., Hook, J., & Dawes, J. (2020) · Neural Networks · doi:10.1016/j.neunet.2020.05.013 · Cited by 155
“Our focus on training data drawn from a dynamical system was inspired by the work of Pathak et al. (2017) who trained ESNs on the Lorenz equations and the Kuramoto–Sivashinsky equation (in one space dimension).”
MethodFunctional Materials for Memristor‐Based Reservoir Computing: Dynamics and Applications
Zhang, G., Qin, J., Zhang, Y., Gong, G., Xiong, Z., Ma, X., Lv, Z., Zhou, Y., & Han, S. (2023) · Advanced Functional Materials · doi:10.1002/adfm.202302929 · Cited by 132
Cognitive computing systems: Algorithms and applications for networks of neurosynaptic cores
Esser, S. K., Andreopoulos, A., Appuswamy, R., Datta, P., Barch, D., Amir, A., Arthur, J., Cassidy, A., Flickner, M., Merolla, P., et al. (2013) · The 2013 International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn.2013.6706746 · Cited by 125
“Algorithms: The algorithms include convolution networks [11] for spatial feature extraction, spectral content estimators for time-domain to frequency-domain conversion, liquid state machines [12], [13] for feature extraction in time-varying signals, restricted Boltzmann machines (RBMs)[14] for spatial feature extracti…”
Kasabov, N. K., & Capecci, E. (2014) · Information Sciences · doi:10.1016/j.ins.2014.06.028 · Cited by 135
“SNN methods have been developed for: learning from data [17,22,26,45,62,66]; system design and implementation [17,19,59]; encoding continuous input data into spike trains, such as the silicon retina [8] and the silicon cochlea sensory devices; neurogenetic computation [5,31,35,38]; high performance and neuromorphic en…”
MethodMultilayered Echo State Machine: A Novel Architecture and Algorithm
Malik, Z. K., Hussain, A., & Wu, Q. M. J. (2016) · IEEE Transactions on Cybernetics · doi:10.1109/tcyb.2016.2533545 · Cited by 133
Optimal Sequence Memory in Driven Random Networks
Schuecker, J., Goedeke, S., & Helias, M. (2018) · Physical Review X · doi:10.1103/physrevx.8.041029 · Cited by 125
“Their information processing capabilities have been a focus in neuroscience [5] and in machine learning [6], and they show optimal performance close to the transition to chaos [7–9].”
BackgroundInformation Processing Using Transient Dynamics of Semiconductor Lasers Subject to Delayed Feedback
Hicke, K., Escalona-Morán, M., Brunner, D., Soriano, M. C., Fischer, I., & Mirasso, C. R. (2013) · IEEE Journal of Selected Topics in Quantum Electronics · doi:10.1109/jstqe.2013.2241738 · Cited by 124
“liquid state machines [6]).”
Exploiting the Dynamics of Soft Materials for Machine Learning
Nakajima, K., Häuser, H., Li, T., & Pfeifer, R. (2018) · Soft Robotics · doi:10.1089/soro.2017.0075 · Cited by 141
Artificial Intelligence and Advanced Materials
López, C. (2023) · Advanced Materials · doi:10.1002/adma.202208683 · Cited by 144
The role of feedback in morphological computation with compliant bodies
Hauser, H., Ijspeert, A. J., Füchslin, R. M., Pfeifer, R., & Maass, W. (2012) · Biological Cybernetics · doi:10.1007/s00422-012-0516-4 · Cited by 119
“C: Restating the orginal theorem by Maass et al. (2007)”
BackgroundMethodResultInfluentialSelf-citationReservoir Computing Meets Smart Grids: Attack Detection Using Delayed Feedback Networks
Hamedani, K., Liu, L., Atat, R., Wu, J., & Yi, Y. (2018) · IEEE Transactions on Industrial Informatics · doi:10.1109/tii.2017.2769106 · Cited by 123
“deals with regular data that is not a spike [8], [12].”
BackgroundMechanical neural networks: Architected materials that learn behaviors
Lee, R. H., Mulder, E. A. B., & Hopkins, J. B. (2022) · Science Robotics · doi:10.1126/scirobotics.abq7278 · Cited by 138
“This approach, called morphological computation (18), is a mechanical version of the concept of reservoir computing (19, 20), where the reservoir used to simplify the mathematical computation is the structure of the robot itself.”
BackgroundControl of criticality and computation in spiking neuromorphic networks with plasticity
Cramer, B., Stöckel, D., Kreft, M., Wibral, M., Schemmel, J., Meier, K., & Priesemann, V. (2020) · Nature Communications · doi:10.1038/s41467-020-16548-3 · Cited by 140
Mixed selectivity: Cellular computations for complexity
Tye, K. M., Miller, E. K., Taschbach, F., Benna, M. K., Rigotti, M., & Fusi, S. (2024) · Neuron · doi:10.1016/j.neuron.2024.04.017 · Cited by 140
“Other more recent recurrent neural networks were using the same ideas (random projections) to generate mixed selectivity representations with higher dimensionality 11 (echo state machine 12 , liquid state machine 13 ).”
Toward an integration of deep learning and neuroscience
Marblestone, A. H., Wayne, G., & Kording, K. P. (2016) · Frontiers in Computational Neuroscience · doi:10.3389/fncom.2016.00094 · Cited by 119
“For example, “liquid-” (Maass et al., 2002) or “echo-state machines” (Jaeger and Haas, 2004) are randomly connected recurrent networks that form a basis set (also known as a “reservoir”) of random filters, which can be harnessed for learning with tunable readout weights.”
BackgroundAll-Optical Reservoir Computing on a Photonic Chip Using Silicon-Based Ring Resonators
Coarer, F. D.-L., Sciamanna, M., Katumba, A., Freiberger, M., Dambre, J., Bienstman, P., & Rontani, D. (2018) · IEEE Journal of Selected Topics in Quantum Electronics · doi:10.1109/jstqe.2018.2836985 · Cited by 119
“reservoir computing a supervised learning technique that appeared a decade ago has focused a lot of attention [4]–[6].”
BackgroundNokkala, J., Martínez‐Peña, R., Giorgi, G. L., Parigi, V., Soriano, M. C., & Zambrini, R. (2021) · Communications Physics · doi:10.1038/s42005-021-00556-w · Cited by 151
Edge-of-chaos learning achieved by ion-electron–coupled dynamics in an ion-gating reservoir
Nishioka, D., Tsuchiya, T., Namiki, W., Takayanagi, M., Imura, M., Koide, Y., Higuchi, T., & Terabe, K. (2022) · Science Advances · doi:10.1126/sciadv.ade1156 · Cited by 127
Deep Echo State Network (DeepESN): A Brief SurveyPreprint
Gallicchio, C., & Micheli, A. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1712.04323 · Cited by 130
“[3, 4]), the Echo State Network (ESN) [5, 6] certainly represents the most widely known model, with a strong theoretical ground (e.”
BackgroundCaligiore, D., Mannella, F., Arbib, M. A., & Baldassarre, G. (2017) · PLOS Computational Biology · doi:10.1371/journal.pcbi.1005395 · Cited by 108
Reservoir Computing Using Multiple Lasers With Feedback on a Photonic Integrated Circuit
Sugano, C., Kanno, K., & Uchida, A. (2019) · IEEE Journal of Selected Topics in Quantum Electronics · doi:10.1109/jstqe.2019.2929179 · Cited by 118
“R ESERVOIR computing has been investigated intensively as a new technology of artificial intelligence in recent years [1], [2].”
BackgroundPolaritonic Neuromorphic Computing Outperforms Linear Classifiers
Ballarini, D., Gianfrate, A., Panico, R., Opala, A., Ghosh, S., Dominici, L., Ardizzone, V., Giorgi, M. D., Lerario, G., Gigli, G., et al. (2020) · Nano Letters · doi:10.1021/acs.nanolett.0c00435 · Cited by 136
Brain-Inspired Computing: A Systematic Survey and Future Trends
Li, G., Deng, L., Tang, H., Pan, G., Tian, Y., Roy, K., & Maass, W. (2024) · Proceedings of the IEEE · doi:10.1109/jproc.2024.3429360 · Cited by 155
“Apart from the SNNs, there are other models inspired by the brain such as liquid state machine [77], echo state networks [78], and continuous attractor neural networks (CANNs) [79].”
Self-citationDelay learning and polychronization for reservoir computing
Paugam‐Moisy, H., Martinez, R., & Bengio, S. (2008) · Neurocomputing · doi:10.1016/j.neucom.2007.12.027 · Cited by 127
“Reservoir computing (RC) recently appeared [38,17] as a generic name for designing a new research stream including mainly echo state networks (ESNs) [15,16], liquid state machines (LSMs) [25], and a few other models like backpropagation decorrelation (BPDC) [42].”
Crowe, D. A., Averbeck, B. B., & Chafee, M. V. (2010) · The Journal of Neuroscience · doi:10.1523/jneurosci.0954-10.2010 · Cited by 112
An experimental characterization of reservoir computing in ambient assisted living applications
Bacciu, D., Barsocchi, P., Chessa, S., Gallicchio, C., & Micheli, A. (2013) · Neural Computing and Applications · doi:10.1007/s00521-013-1364-4 · Cited by 128
“Liquid State Machines [32] and BackPropagation Decorrelation [37, 38], among the others.”
James, C. D., Aimone, J. B., Miner, N. E., Vineyard, C. M., Rothganger, F. H., Carlson, K. D., Mulder, S. A., Draelos, T. J., Faust, A., Marinella, M. J., et al. (2017) · Biologically Inspired Cognitive Architectures · doi:10.1016/j.bica.2016.11.002 · Cited by 153
Antonik, P., Gulina, M., Pauwels, J., & Massar, S. (2018) · Physical review. E · doi:10.1103/physreve.98.012215 · Cited by 127
Pecevski, D., Buesing, L., & Maass, W. (2011) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1002294 · Cited by 109
“Another spiking neural implementation of the sum-product belief propagation algorithm was proposed in [36], where the calculation and passing of the messages was achieved in a recurrent network of interconnected liquid state machines [49].”
BackgroundSelf-citationChatzis, S., & Demiris, Y. (2011) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.2011.2162109 · Cited by 120
“Digital Object Identifier 10.1109/TNN.2011.2162109 invented independently in the seminal works of Jaeger [6], who called these RNNs " echo state networks " (ESNs), and Maass et al. [7], who developed a similar approach for spiking neural networks and called the derived model the " liquid state machine " (LSM).”
BackgroundInfluentialMayr, C., Partzsch, J., Noack, M., Hänzsche, S., Scholze, S., Höppner, S., Ellguth, G., & Schüffny, R. (2015) · IEEE Transactions on Biomedical Circuits and Systems · doi:10.1109/tbcas.2014.2379294 · Cited by 113
Robust Transient Dynamics and Brain Functions
Rabinovich, M. I., & Varona, P. (2011) · Frontiers in Computational Neuroscience · doi:10.3389/fncom.2011.00024 · Cited by 109
“Following the general perspective of liquid state (Maass et al., 2002) and echo-state models (Jaeger, 2003), we illustrate this paradigm with several examples that just point out the tip of an iceberg.”
Design and Analysis of a Neuromemristive Reservoir Computing Architecture for Biosignal Processing
Kudithipudi, D., Saleh, Q., Merkel, C., Thesing, J., & Wysocki, B. (2016) · Frontiers in Neuroscience · doi:10.3389/fnins.2015.00502 · Cited by 105
“This idea was developed in tandem by two independent research groups, Jaeger (Jaeger, 2001; Jaeger and Haas, 2004) as the Echo State Network and Maass as the Liquid State Machine (Maass et al., 2002; Buonomano and Maass, 2009).”
MethodDeePr-ESN: A deep projection-encoding echo-state network
Ma, Q., Shen, L., & Cottrell, G. W. (2019) · Information Sciences · doi:10.1016/j.ins.2019.09.049 · Cited by 118
“Liquid State Machines (LSMs) have continuous dynamics and spiking neurons, while Echo State Networks (ESNs) use discrete dynamics and rate-coded neurons.”
MethodShort-Term Memory in Networks of Dissociated Cortical Neurons
Dranias, M. R., Ju, H., Rajaram, E., & VanDongen, A. M. J. (2013) · Journal of Neuroscience · doi:10.1523/jneurosci.2718-12.2013 · Cited by 122
“, 2010) and the LSM is sensitive to temporal patterns (Maass et al., 2002).”
Neuronal filtering of multiplexed odour representations
Blumhagen, F., Zhu, P., Shum, J. W. H., Zhang, Y., Yaksi, E., Deisseroth, K. A., & Friedrich, R. W. (2011) · Nature · doi:10.1038/nature10633 · Cited by 120
Offline Printed Urdu Nastaleeq Script Recognition with Bidirectional LSTM Networks
Ul-Hasan, A., Ahmed, S. B., Rashid, F., Shafait, F., & Breuel, T. M. (2013) · IAPR International Conference on Document Analysis and Recognition (ICDAR) · doi:10.1109/icdar.2013.212 · Cited by 125
Ponzi, A., & Wickens, J. (2010) · The Journal of Neuroscience · doi:10.1523/jneurosci.5540-09.2010 · Cited by 106
“…Discussion) and more generally is type of transient (not fixed point) dynamical activity suggested by theoretical and experimental studies to be broadly relevant in neural information processing (Buonomano and Merzenich, 1995; Maass et al., 2002; Mazor and Laurent, 2005; Rabinovich et al., 2008a).”
BackgroundLyapunov spectra of chaotic recurrent neural networks
Engelken, R., Wolf, F., & Abbott, L. F. (2023) · Physical Review Research · doi:10.1103/physrevresearch.5.043044 · Cited by 140
“These arise from its rich internal dynamics that can provide a substrate for complex nonlinear computations, e.g., implementing input/output maps [13–15] and learning temporal sequences [16]; however, it is a challenge to extend this to spiking neural networks [17–21].”
BackgroundOptimizing the echo state network with a binary particle swarm optimization algorithm
Wang, H., & Yan, X. (2015) · Knowledge-Based Systems · doi:10.1016/j.knosys.2015.06.003 · Cited by 113
Deep photonic reservoir computing recurrent network
Shen, Y.-W., Li, R., Liu, G., Yu, J., He, X., Yi, L., & Wang, C. (2023) · Optica · doi:10.1364/optica.506635 · Cited by 126
“Reservoir computing (RC) is such a kind of RNN, but with fixed weights in the input layer and in the hidden reservoir layers [10,11].”
BackgroundMesaritakis, C., Papataxiarhis, V., & Syvridis, D. (2013) · Journal of the Optical Society of America B · doi:10.1364/josab.30.003048 · Cited by 103
Reservoir computing on a silicon platform with a ferroelectric field-effect transistor
Toprasertpong, K., Nako, E., Wang, Z., Nakane, R., Takenaka, M., & Takagi, S. (2022) · Communications Engineering · doi:10.1038/s44172-022-00021-8 · Cited by 110
Epigenetic Robotics Architecture (ERA)
Morse, A. F., de Greeff, J., Belpeame, T., & Cangelosi, A. (2010) · IEEE Transactions on Autonomous Mental Development · doi:10.1109/tamd.2010.2087020 · Cited by 120
“These properties of biological cortical microcolumns have very useful computational implications; first, by making highly nonlinear features linearly separable (much as a kernel warping function does in a support vector machine), and second, by acting as a fading memory [66].”
BackgroundThe Continuity of Context: A Role for the Hippocampus
Maurer, A. P., & Nadel, L. (2021) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2020.12.007 · Cited by 111
Applications and Techniques for Fast Machine Learning in Science
Deiana, A. M., Tran, N. V., Agar, J., Blott, M., Guglielmo, G. D., Duarte, J. M., Harris, P. C., Hauck, S., Liu, M., Neubauer, M. S., et al. (2022) · Frontiers in Big Data · doi:10.3389/fdata.2022.787421 · Cited by 126
Randomly distributed embedding making short-term high-dimensional data predictable
Ma, H., Leng, S., Aihara, K., Lin, W. J., & Chen, L. (2018) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1802987115 · Cited by 123
Larger, L., & Dudley, J. M. (2010) · Nature · doi:10.1038/465041a · Cited by 106
Neural Networks and Deep Learning
Brunton, S. L., & Kutz, J. N. (2019) · Cambridge University Press eBooks · doi:10.1017/9781108380690.007 · Cited by 111
Deep learning for financial forecasting: A review of recent trends
Giantsidi, S., & Tarantola, C. (2025) · International Review of Economics & Finance · doi:10.1016/j.iref.2025.104719 · Cited by 118
A recipe for creating ideal hybrid memristive-CMOS neuromorphic processing systems
Chicca, E., & Indiveri, G. (2020) · Applied Physics Letters · doi:10.1063/1.5142089 · Cited by 103
“Examples of theoretical neural processing frameworks that require variability can be found in the domain of ensemble learning 53 , reservoir computing 54 and liquid state machines 55 .”
BackgroundEmbedding Multiple Trajectories in Simulated Recurrent Neural Networks in a Self-Organizing Manner
Liu, J. K., & Buonomano, D. V. (2009) · The Journal of Neuroscience · doi:10.1523/jneurosci.2358-09.2009 · Cited by 107
“…networkswithrandomweightscanbeusedtoperformfunctional computations(Buonomano,2000;MedinaandMauk,2000;Maass etal.,2002),andthatcarefullycontrollingthefeedbackfromoutput unitsintotherecurrentnetworkoffersapromisingwaytocontrol…”
Fuel Cells prognostics using echo state network
Morando, S., Jemeï, S., Gouriveau, R., Noureddine Zerhouni, A., & Hissel, D. (2013) · Annual Conference of the IEEE Industrial Electronics Society (IECON) · doi:10.1109/iecon.2013.6699377 · Cited by 103
Memory versus non-linearity in reservoirs
Verstraeten, D., Dambre, J., Dutoit, X., & Schrauwen, B. (2010) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn.2010.5596492 · Cited by 125
Zheng, P., Dimitrakakis, C., & Triesch, J. (2013) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1002848 · Cited by 112
“Many computational models of local cortical circuits assume random network structure [4–6], sometimes with distance-dependent or layer-dependent connection probabilities [34].”
Emergence of Dynamic Memory Traces in Cortical Microcircuit Models through STDP
Klampfl, S., & Maass, W. (2013) · Journal of Neuroscience · doi:10.1523/jneurosci.5044-12.2013 · Cited by 99
“Whereas the original version of the model (Maass et al., 2002) was only able to model online processing of novel stimuli in a “naive” network, the extension presented here also addresses the formation and computational use of long-term memory traces.”
BackgroundMethodResultInfluentialSelf-citationToward Green AI: A Methodological Survey of the Scientific Literature
Barbierato, E., & Gatti, A. (2024) · IEEE Access · doi:10.1109/access.2024.3360705 · Cited by 116
“LSMs [40] are built upon the principles of high-dimensional dynamical systems paired with statistical learning theory.”
MethodComputational Models for Generic Cortical Microcircuits
Maass, W., Natschlaeger, T., & Markram, H. (2003) · Chapman & Hall/CRC mathematical and computational biology series · doi:10.1201/9780203494462.ch18 · Cited by 109
“We have shown in [16] that without synaptic dynamics the computational power of these microcircuit models decays significantly.”
BackgroundInfluentialSelf-citationAn embodied account of serial order: How instabilities drive sequence generation
Sandamirskaya, Y., & Schöner, G. (2010) · Neural Networks · doi:10.1016/j.neunet.2010.07.012 · Cited by 108
Perspective: Memcomputing: Leveraging memory and physics to compute efficiently
Ventra, M. D., & Traversa, F. L. (2018) · Journal of Applied Physics · doi:10.1063/1.5026506 · Cited by 106
Time-varying data processing with nonvolatile memristor-based temporal kernel
Jang, Y. H., Kim, W., Kim, J., Woo, K. S., Lee, H. J., Jeon, J. W., Shim, S. K., Han, J., & Hwang, C. S. (2021) · Nature Communications · doi:10.1038/s41467-021-25925-5 · Cited by 101
“Table 2 shows the comparison with other RC results using the diffusive memristors and the software-based single-layer FCN.”
BackgroundInfluentialA spiking network model for passage‐of‐time representation in the cerebellum
Yamazaki, T., & Tanaka, S. (2007) · European Journal of Neuroscience · doi:10.1111/j.1460-9568.2007.05837.x · Cited by 104
“Recently, a liquid state machine (LSM) has been proposed as a new framework for neural computation (Maass et al., 2002).”
BackgroundYang, G. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1910.12478 · Cited by 111
“Reservoir Computing In reservoir computing [30, 39, 51], sequence processing is typically done by a randomly initialized recurrent neural network.”
BackgroundMethodPCSIM: A Parallel Simulation Environment for Neural Circuits Fully Integrated with Python
Pecevski, D., Natschläger, T., & Schuch, K. (2009) · Frontiers in Neuroinformatics · doi:10.3389/neuro.11.011.2009 · Cited by 103
“The pylsm package is aimed to support the analysis of the computational properties of cortical microcircuits within the liquid state machine (LSM) approach (Maass et al., 2002).”
Self-citationA Theory of Sequence Indexing and Working Memory in Recurrent Neural Networks
Frady, E. P., Kleyko, D. V., & Sommer, F. T. (2018) · Neural Computation · doi:10.1162/neco_a_01084 · Cited by 98
Reservoir Computing on Spin-Torque Oscillator Array
Kanao, T., Suto, H., Mizushima, K., Goto, H., Tanamoto, T., & Nagasawa, T. (2019) · Physical Review Applied · doi:10.1103/physrevapplied.12.024052 · Cited by 108
“The performance can become higher than a standard neural network model known as an echo-state network (ESN) [1, 2].”
BackgroundPhotonic neuromorphic information processing and reservoir computing
Lugnan, A., Katumba, A., Laporte, F., Freiberger, M., Sackesyn, S., Ma, C., Gooskens, E., Dambre, J., & Bienstman, P. (2020) · APL Photonics · doi:10.1063/1.5129762 · Cited by 104
Drifting codes within a stable coding scheme for working memory
Wolff, M. J., Jochim, J., Akyürek, E. G., Buschman, T. J., & Stokes, M. G. (2020) · PLoS Biology · doi:10.1371/journal.pbio.3000625 · Cited by 101
“, [26]), possibly through sequential activation of neurons (e.”
BackgroundEcho state networks with filter neurons and a delay&sum readout
Holzmann, G., & Häuser, H. (2009) · Neural Networks · doi:10.1016/j.neunet.2009.07.004 · Cited by 119
“[19], who developed a similar approach for spiking neural networks and called this structure liquid state machine (LSM).”
BackgroundDeep learning in microbiome analysis: a comprehensive review of neural network models
Przymus, P., Rykaczewski, K., Martín‐Segura, A., Truu, J., Pau, E. C. D. S., Kolev, M., Naskinova, I., Gruca, A., Sampri, A., Frohme, M., et al. (2025) · Frontiers in Microbiology · doi:10.3389/fmicb.2024.1516667 · Cited by 107
Fading memory and kernel properties of generic cortical microcircuit models
Maass, W., Natschläger, T., & Markram, H. (2004) · Journal of Physiology-Paris · doi:10.1016/j.jphysparis.2005.09.020 · Cited by 110
“In order to analyze the potential capabilities of this approach, we have introduced in [31] the abstract model of a liquid state machine (LSM), see Fig.”
BackgroundInfluentialSelf-citationBrain-Inspired Wireless Communications: Where Reservoir Computing Meets MIMO-OFDM
Mosleh, S., Liu, L., Sahin, C., Zheng, Y. R., & Yi, Y. (2017) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2017.2766162 · Cited by 105
“This is because RNNs are capable of exploiting the underlying correlation within the data [17].”
BackgroundDeep time-delay reservoir computing: Dynamics and memory capacity
Goldmann, M., Köster, F., Lüdge, K., & Yanchuk, S. (2020) · Chaos An Interdisciplinary Journal of Nonlinear Science · doi:10.1063/5.0017974 · Cited by 109
Spike frequency adaptation supports network computations on temporally dispersed information
Salaj, D., Subramoney, A., Kraišniković, C., Bellec, G., Legenstein, R. A., & Maass, W. (2021) · eLife · doi:10.7554/elife.65459 · Cited by 100
“The impact of this mechanism on simple temporal computing tasks had been examined in a number of publications (Maass et al., 2002; Buonomano and Maass, 2009; Masse et al., 2019; Hu et al., 2020).”
BackgroundSelf-citationSun, Y., Lin, T., Lei, N., Chen, X., Kang, W., Zhao, Z., Wei, D., Chen, C., Pang, S., Hu, L., et al. (2023) · Nature Communications · doi:10.1038/s41467-023-39207-9 · Cited by 104
Metadata
- Authors
- Maass, W., Natschläger, T., & Markram, H.
- Affiliations
- Graz University of Technology (AT)
- École Polytechnique Fédérale de Lausanne (CH)
- Corresponding author
- Maass, Wolfgang
- Natschläger, Thomas
- Markram, Henry
- Venue
- Neural Computation
- Publisher
- MIT Press
- Published
- 2002-11-01
- Volume
- 14
- Issue
- 11
- Pages
- 2531-2560
- ISSN
- 0899-7667 (ISSN-L)
- 1530-888X
- OpenAlex
- W2103179919
- Cited by
- 4,025
- References
- 28
- Citations by year
- 2026: 286 · 2025: 351 · 2024: 307 · 2023: 309 · 2022: 276 · 2021: 303 · 2020: 239 · 2019: 244 · 2018: 217 · 2017: 173 · 2016: 155 · 2015: 135 · 2014: 142 · 2013: 139 · 2012: 125
- FWCI
- 18.37
- Type
- journal article
- Language
- en
- Open access
- green
- infoscience.epfl.ch
- Locations
- Neural Computation (publishedVersion)
- PubMed (publishedVersion)
- Infoscience (Ecole Polytechnique Fédérale de Lausanne) (submittedVersion)
- Infoscience (Ecole Polytechnique Fédérale de Lausanne) (submittedVersion, cc-by-nc-nd)
- Topics
- Life Sciences › Neuroscience › Cognitive Neuroscience › Neural dynamics and brain function
- Physical Sciences › Computer Science › Artificial Intelligence › Neural Networks and Reservoir Computing
- Physical Sciences › Computer Science › Artificial Intelligence › Neural Networks and Applications
- Keywords
- liquid state machines, recurrent neural circuits, transient dynamics, high-dimensional dynamical systems, fading memory, integrate-and-fire neurons, real-time processing, time-varying inputs, neural coding, statistical learning theory, computational power, attractor networks
- MeSH terms
- Action Potentials, Computer Simulation, Computer Systems, Computers, Models, Neurological, Neurons, Neural Networks, Computer
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.