Real-Time Computing Without Stable States: A New Framework for Neural Computation Based on Perturbations
Open accessThe DOI matches its Crossref record; 28 of the 28 references it cites were found in a registry or an index.
Abstract
A key challenge for neural modeling is to explain how a continuous stream of multimodal input from a rapidly changing environment can be processed by stereotypical recurrent circuits of integrate-and-fire neurons in real time. We propose a new computational model for real-time computing on time-varying input that provides an alternative to paradigms based on Turing machines or attractor neural networks. It does not require a task-dependent construction of neural circuits. Instead, it is based on principles of high-dimensional dynamical systems in combination with statistical learning theory and can be implemented on generic evolved or found recurrent circuitry. It is shown that the inherent transient dynamics of the high-dimensional dynamical system formed by a sufficiently large and heterogeneous neural circuit may serve as universal analog fading memory. Readout neurons can learn to extract in real time from the current state of such recurrent neural circuit information about current and past inputs that may be needed for diverse tasks. Stable internal states are not required for giving a stable output, since transient internal states can be transformed by readout neurons into stable target outputs due to the high dimensionality of the dynamical system. Our approach is based on a rigorous computational model, the liquid state machine, that, unlike Turing machines, does not require sequential transitions between well-defined discrete internal states. It is supported, as the Turing machine is, by rigorous mathematical results that predict universal computational power under idealized conditions, but for the biologically more realistic scenario of real-time processing of time-varying inputs. Our approach provides new perspectives for the interpretation of neural coding, the design of experiments and data analysis in neurophysiology, and the solution of problems in robotics and neurotechnology.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Although work in artificial intelligence and machine-learning research has shown that similar or even more complex outcomes can be achieved with a variety of techniques, including deep neural networks with trained weights (Gibney, 2015), reservoir networks with a trained output mapping (Maass et al., 2002;Jaeger & Haas, 2004), networks that are evolved using a genetic algorithm (Iizuka & Di Paolo,”
From A potential mechanism for Gibsonian resonance: behavioral entrainment emerges from local homeostasis in an unsupervised reservoir network (2024) · cites this paper as Maass et al., 2002
In the citing paper
“Finally, we show that a randomly connected circuit model, akin to a “reservoir” network13-15, can reproduce the observed distribution of timescales, provided that the network operates at the critical point (or “edge of chaos”)16-18.”
From A reservoir of time constants for memory traces in cortical neurons (2011) · cites this paper as [13]
In the citing paper
“Reservoir-type networks have been the subject of active research in computational neuroscience and machine learning13-15, but so far experimental support that such networks are adopted by the brain has been lacking.”
From A reservoir of time constants for memory traces in cortical neurons (2011) · cites this paper as [13]
In the citing paper
“Those models predict that the memory of input signals is stored in a large, recurrent and heterogeneous network (reservoir), in a distributed manner, and that a desired output is obtained by a trainable combination of the response signals in the reservoir.”
From A reservoir of time constants for memory traces in cortical neurons (2011) · cites this paper as [13]
Citations
Cites
Wu, Y. (1999) · Technometrics · doi:10.1080/00401706.1999.10485951 · Cited by 27,221
Introduction to the theory of neural computation
(1994) · Neural Networks · doi:10.1016/0893-6080(94)90014-0 · Cited by 6,401
The Synaptic Organization of the Brain
(2004) · Oxford University Press eBooks · doi:10.1093/acprof:oso/9780195159561.001.1 · Cited by 3,546 · cited 2×
“This informationisprocessed by extremely complexbut surprisingly stereotypic microcircuits that can perform a wide spectrum of tasks (Shep-herd, 1988 ; Douglas & Martin, 1998 ; von Melchner, Pallas, & Sur, 2000).”
Dynamics of ongoing activity: explanation of the large variability in evoked cortical responses
Arieli, A., Sterkin, A., Grinvald, A., & Aertsen, A. (1996) · Science · doi:10.1126/science.273.5283.1868 · Cited by 1,660 · cited 1×
“Furthermore, although the input rate may contain salient information that is constant for a particular readout element, it may not be for another (see, for example, Figure 8), indicating that equivalence classes and dynamic stability exist purely from the perspective of the readout elements.”
BackgroundDifferential signaling via the same axon of neocortical pyramidal neurons
Markram, H., Wang, Y., & Tsodyks, M. (1998) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.95.9.5323 · Cited by 1,102 · cited 2×
“In the case of a synaptic connection from a to b , we modeled the synaptic dynamics according to the model proposed in Markram, Wang, and Tsodyks (1998), with the synaptic parameters U (use), D (time constant for depression), and F (time constant for facilitation) randomly chosen from gaussian…”
BackgroundMethodSelf-citationOrganizing principles for a diversity of GABAergic interneurons and synapses in the neocortex
Gupta, A., Wang, Y., & Markram, H. (2000) · Science · doi:10.1126/science.287.5451.273 · Cited by 1,036
Self-citationFading memory and the problem of approximating nonlinear operators with Volterra series
Boyd, S., & Chua, L. (1985) · IEEE Transactions on Circuits and Systems · doi:10.1109/tcs.1985.1085649 · Cited by 991 · cited 2×
“> The proof of this theorem follows from the Stone-Weierstrass approximation theorem, like the proof of theorem 1 in Boyd and Chua (1985).”
BackgroundGradient calculations for dynamic recurrent neural networks: a survey
Pearlmutter, B. A. (1995) · IEEE Transactions on Neural Networks · doi:10.1109/72.410363 · Cited by 582 · cited 1×
“Methods for controllingthe dynamics of recurrent neural networks through adaptive mechanisms are reviewed in Pearlmutter (1995).”
Temporal Information Transformed into a Spatial Code by a Neural Network with Realistic Properties
Buonomano, D. V., & Merzenich, M. M. (1995) · Science · doi:10.1126/science.7863330 · Cited by 449 · cited 1×
“Special cases of this mechanism have already been reported in Buonomano and Merzenich (1995) and Dominey, Arbib, and Joseph (1995).”
BackgroundLearning and Extracting Finite State Automata with Second-Order Recurrent Neural Networks
Giles, C. L., Miller, C. B., Chen, D., Chen, H. H., Sun, G. Z., & Lee, Y. C. (1992) · Neural Computation · doi:10.1162/neco.1992.4.3.393 · Cited by 392
Cited by
Items 501–600 of 1,014
Cognitive Computation with Autonomously Active Neural Networks: An Emerging Field
Gros, C. (2009) · Cognitive Computation · doi:10.1007/s12559-008-9000-9 · Cited by 82
“The ‘echo-state’ approach using discrete-time rate-encoding neurons [22,23], and the ‘liquid state machine’ using continuous-time spiking neurons [24,25].”
MethodA behavior controller based on spiking neural networks for mobile robots
Wang, X., Hou, Z., Zou, A., Tan, M. H., & Cheng, L. (2007) · Neurocomputing · doi:10.1016/j.neucom.2007.08.025 · Cited by 82
Gender Classification of Human Faces
Graf, A. B. A., & Wichmann, F. A. (2002) · Lecture notes in computer science · doi:10.1007/3-540-36181-2_49 · Cited by 76
“Having a linear output stage has recently become a topic of interest in the context of complex, dynamical systems (“echo state” recurrent neural networks [14] and “liquid state machines” [15]) as it allows learning in such systems.”
BackgroundLearning reward timing in cortex through reward dependent expression of synaptic plasticity
Gavornik, J. P., Shuler, M. G. H., Loewenstein, Y., Bear, M. F., & Shouval, H. Z. (2009) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.0901835106 · Cited by 80
Evolutionary aspects of reservoir computing
Seoane, L. F. (2019) · Philosophical Transactions of the Royal Society B: Biological Sciences · doi:10.1098/rstb.2018.0377 · Cited by 74
“Through RC, they proposed a plausible computing strategy that could be the basic operating principle of these circuits [33, 106–108].”
BackgroundMethodInfluentialThe TENNLab Exploratory Neuromorphic Computing Framework
Plank, J. S., Schuman, C. D., Bruer, G. R., Dean, M. E., & Rose, G. S. (2018) · Letters of the IEEE Computer Society · doi:10.1109/locs.2018.2885976 · Cited by 76
“Current exploratory approaches for programming recurrent neuromorphic systems include competitive algorithms [9], reservoir computing [10], genetic algorithms [11], spike timing-dependent plasticity [12] and custom”
BackgroundTemporal Pattern Recognition with Delayed-Feedback Spin-Torque Nano-Oscillators
Riou, M., Torrejón, J., Garitaine, B., Araujo, F. A., Bortolotti, P., Cros, V., Tsunegi, S., Yakushiji, K., Fukushima, A., Kubota, H., et al. (2019) · Physical Review Applied · doi:10.1103/physrevapplied.12.024049 · Cited by 76
“III, we emulate a reservoir computer with a single, time-multiplexed oscillator [25], [26], and demonstrate the performance of the extrinsic memory by recognizing temporal patterns composed of random sine and square waveforms.”
MethodInterface-type tunable oxygen ion dynamics for physical reservoir computing
Liu, Z., Zhang, Q., Xie, D., Zhang, M., Li, X., Zhong, H., Li, G., He, M., Shang, D., Wang, C., et al. (2023) · Nature Communications · doi:10.1038/s41467-023-42993-x · Cited by 78
“neural network (RNN) 10 can be used as a promising candidate for dynamic data processing, such as voice, video, and chaotic systems 11 .”
Stochastic Computations in Cortical Microcircuit Models
Habenschuss, S., Jonke, Z., & Maass, W. (2013) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1003311 · Cited by 88
“Slow convergence enlarges the time span during which the network can integrate information from non-stationary external inputs [77–79].”
BackgroundMethodSelf-citationMemory in linear recurrent neural networks in continuous time
Hermans, M., & Schrauwen, B. (2009) · Neural Networks · doi:10.1016/j.neunet.2009.08.008 · Cited by 75
SuperMind: a survey of the potential of superconducting electronics for neuromorphic computing
Schneider, M. L., Toomey, E., Rowlands, G. E., Shainline, J. M., Tschirhart, P., & Segall, K. (2022) · Superconductor Science and Technology · doi:10.1088/1361-6668/ac4cd2 · Cited by 83
Broccard, F. D., Joshi, S., Wang, J., & Cauwenberghs, G. (2017) · Journal of Neural Engineering · doi:10.1088/1741-2552/aa67a9 · Cited by 77
“The 256 neurons were connected to implement a liquid state machine [118] with a first layer of 128 neurons using a random recurrent network (reservoir) with STP synapses and a second layer of 128 read-out neurons, with all-to-all connections and plastic LTP synapses between the two layers.”
MethodIdentification of pre-sliding friction dynamics
Parlitz, U., Hornstein, A., Engster, D., Al‐Bender, F., Lampaert, V., Tjahjowidodo, T., Fassois, S. D., Rizos, D. D., Wong, C. X., Worden, K., et al. (2004) · Chaos An Interdisciplinary Journal of Nonlinear Science · doi:10.1063/1.1737818 · Cited by 74
Kasabov, N. K., Doborjeh, M., & Doborjeh, Z. (2016) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2016.2612890 · Cited by 80
“A NeuCube model is a special type of a liquid state machine (LSM) [28] that has new features of learning, spatial variable mapping, model visualization, and so on that make a NeuCube model meaningful in terms of its interpretation for the sake of understanding of spatiotemporal characteristics of the data.”
BackgroundMethodInfluentialKonar, D., Sarma, A. D., Bhandary, S., Bhattacharyya, S., Cangi, A., & Aggarwal, V. (2023) · Applied Soft Computing · doi:10.1016/j.asoc.2023.110099 · Cited by 79
Photonic Neural Networks Based on Integrated Silicon Microresonators
Biasi, S., Donati, G. P., Lugnan, A., Mancinelli, M., Staffoli, E., & Pavesi, L. (2024) · Intelligent Computing · doi:10.34133/icomputing.0067 · Cited by 80
“An approach to recurrent networks that simplifies their training is Reservoir Computing (RC) [104].”
BackgroundInteractions dominate the dynamics of visual cognition
Kelty‐Stephen, D. G., & Mirman, D. (2010) · Cognition · doi:10.1016/j.cognition.2009.12.010 · Cited by 82
Large-Scale Synthesis of Functional Spiking Neural Circuits
Stewart, T. C., & Eliasmith, C. (2014) · Proceedings of the IEEE · doi:10.1109/jproc.2014.2306061 · Cited by 82
“It should be noted that networks constructed with this approach are fundamentally different from echo state networks [29] and liquid state machines [41].”
BackgroundLiquid state machines and cultured cortical networks: The separation property
Dockendorf, K. P., Park, I. M., He, P., Príncipe, J. C., & DeMarse, T. B. (2008) · Biosystems · doi:10.1016/j.biosystems.2008.08.001 · Cited by 82
Performance boost of time-delay reservoir computing by non-resonant clock cycle
Stelzer, F., Röhm, A., Lüdge, K., & Yanchuk, S. (2020) · Neural Networks · doi:10.1016/j.neunet.2020.01.010 · Cited by 72
“Even though our analytics so far are only derived for the linear case, we expect these results to hold in more general situations, as numerical simulations with several different nonlinear activation functions in figures 2, 7, 8 indicate.”
MethodAkai‐Kasaya, M., Takeshima, Y., Kan, S., Nakajima, K., Oya, T., & Asai, T. (2021) · Neuromorphic Computing and Engineering · doi:10.1088/2634-4386/ac4339 · Cited by 72
Mengü, D., Rahman, M. S. S., Luo, Y., Li, J., Kulce, O., & Özcan, A. (2022) · Advances in Optics and Photonics · doi:10.1364/aop.450345 · Cited by 80
Toward a Brain–Neuromorphics Interface
Wan, C., Pei, M., Shi, K., Cui, H., Long, H., Qiao, L., Xing, Q., & Wan, Q. C. (2024) · Advanced Materials · doi:10.1002/adma.202311288 · Cited by 72
Trainable hardware for dynamical computing using error backpropagation through physical media
Hermans, M., Burm, M., Vaerenbergh, T. V., Dambre, J., & Bienstman, P. (2015) · Nature Communications · doi:10.1038/ncomms7729 · Cited by 72
“This paradigm, which combines several previous lines of research [13, 18], essentially employs randomly constructed dynamic systems to process time series.”
MethodLocal Dynamics in Trained Recurrent Neural Networks
Rivkind, A., & Barak, O. (2017) · Physical Review Letters · doi:10.1103/physrevlett.118.258101 · Cited by 72
Penny, S. G., Smith, T. A., Chen, T., Platt, J. A., Lin, H.-Y., Goodliff, M., & Abarbanel, H. D. I. (2022) · Journal of Advances in Modeling Earth Systems · doi:10.1029/2021ms002843 · Cited by 71
“It originated in the works of Jaeger [2001], who introduced Echo State Networks (ESN), and Maass et al. [2002] who introduced Liquid State Machines (LSM).”
BackgroundMethodA systematic exploration of reservoir computing for forecasting complex spatiotemporal dynamics
Platt, J. A., Penny, S. G., Smith, T. A., Chen, T., & Abarbanel, H. D. I. (2022) · Neural Networks · doi:10.1016/j.neunet.2022.06.025 · Cited by 92
Learning Anticipation via Spiking Networks: Application to Navigation Control
Arena, P., Fortuna, L., Frasca, M., & Patané, L. (2009) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.2008.2005134 · Cited by 77
“Furthermore, we mention that from another viewpoint, the time-gap problem is solved thanks to the fact that the neural dynamics is also characterized by other time scales than those of milliseconds [40].”
BackgroundInitial-state-dependent, robust, transient neural dynamics encode conscious visual perception
Baria, A. T., Maniscalco, B., & He, B. J. (2017) · PLOS Computational Biology · doi:10.1371/journal.pcbi.1005806 · Cited by 70
“transient neural dynamics (‘trajectories’), whereby computation arises from the voyage through state space rather than the arrival at a fixed location [26, 28, 49].”
BackgroundImproving Spiking Dynamical Networks: Accurate Delays, Higher-Order Synapses, and Time Cells
Voelker, A. R., & Eliasmith, C. (2017) · Neural Computation · doi:10.1162/neco_a_01046 · Cited by 83
Chaos in time delay systems, an educational review
Wernecke, H., Sándor, B., & Gros, C. (2019) · Physics Reports · doi:10.1016/j.physrep.2019.08.001 · Cited by 70
“Another application of delay coupling is the realization of reservoir computing networks [126], which are closely related to so-called echo state networks [127].”
BackgroundAcoustic Modeling With Hierarchical Reservoirs
Triefenbach, F., Jalalvand, A., Demuynck, K., & Martens, J. (2013) · IEEE Transactions on Audio Speech and Language Processing · doi:10.1109/tasl.2013.2280209 · Cited by 80
“The term RC is a unification [21] of two similar concepts that were developed independently of each other under the names Echo State Network (ESN [20], [45]) and Liquid State Machine (LSM, [46]).”
BackgroundNeuromorphic Time‐Dependent Pattern Classification with Organic Electrochemical Transistor Arrays
Pecqueur, S., Talamo, M. M., Guérin, D., Blanchard, P., Roncali, J., Vuillaume, D., & Alibart, F. (2018) · Advanced Electronic Materials · doi:10.1002/aelm.201800166 · Cited by 70
Memristors for the Curious Outsiders
Carbajal, J. P., & Caravelli, F. (2018) · Technologies · doi:10.3390/technologies6040118 · Cited by 82
Photonic reservoir computing based on nonlinear wave dynamics at microscale
Sunada, S., & Uchida, A. (2019) · Scientific Reports · doi:10.1038/s41598-019-55247-y · Cited by 83
“onic computing with high-dimensional dynamics and a novel model-free sensing scheme. Introduction Reservoir computing (RC) [1], originally known as an echo state network [2] or a liquid state machine [3], is a computational paradigm using high-dimensional dynamical systems, and it has been regarded as a powerful tool…”
BackgroundSunada, S., & Uchida, A. (2021) · Optica · doi:10.1364/optica.434918 · Cited by 77
Ji, Y., Wang, L., Long, Y., Wang, J., Zheng, H., Yu, Z. G., Zhang, Y., & Ang, K. (2025) · Nature Communications · doi:10.1038/s41467-025-62306-8 · Cited by 73
“Additionally, the ability of reservoirstoencodefeaturesovermultipletimescalesmakesreservoir computing particularly suitable for analyzing temporal sequences and forecasting time-series data 47 .”
Computational capabilities of random automata networks for reservoir computing
Snyder, D., Goudarzi, A., & Teuscher, C. (2013) · Physical Review E · doi:10.1103/physreve.87.042808 · Cited by 84
Long-range temporal correlations in scale-free neuromorphic networks
Shirai, S., Acharya, S. K., Bose, S. K., Mallinson, J. B., Galli, E., Pike, M. D., Arnold, M. D., & Brown, S. A. (2020) · Network Neuroscience · doi:10.1162/netn_a_00128 · Cited by 73
“…hierarchical networks are predicted to enable neuromorphic computing approaches such as reservoir computing (RC) (Lukoševiˇcius & Jaeger, 2009; Maass, Reservoir computing: A computational framework that utilizes a nonlinear dynamical system (a reservoir) to project inputs into a higher dimensional space, allowing map…”
BackgroundPhysical reservoir computing with FORCE learning in a living neuronal culture
Yada, Y., Shusaku, Y., & Takahashi, H. (2021) · Applied Physics Letters · doi:10.1063/5.0064771 · Cited by 72
Ensemble Deep Random Vector Functional Link Neural Network for Regression
Hu, M., Chion, J. H., Suganthan, P. N., & Katuwal, R. (2022) · IEEE Transactions on Systems Man and Cybernetics Systems · doi:10.1109/tsmc.2022.3213628 · Cited by 68
“…reservoir computation (RC) emerged as a viable alternative to gradient descent for training RNNs. RC refers generically to a class of RNNs, including liquid state machines (LSMs) [24], echo state network (ESN) [25], [26], and other RNNs that use backpropagation decorrelation strategies [27].”
BackgroundShort-Term Synaptic Plasticity as a Mechanism for Sensory Timing
Motanis, H., Seay, M. J., & Buonomano, D. V. (2018) · Trends in Neurosciences · doi:10.1016/j.tins.2018.08.001 · Cited by 70
Large-Scale Spatiotemporal Photonic Reservoir Computer for Image Classification
Antonik, P., Marsal, N., & Rontani, D. (2019) · IEEE Journal of Selected Topics in Quantum Electronics · doi:10.1109/jstqe.2019.2924138 · Cited by 72
“Reservoir Computing (RC) is a Machine Learning paradigm, which includes echo-state networks (ESN) and liquid-state machines (LSM), to design and train artificial recurrent neural networks [11], [12].”
BackgroundGupta, D. S. (2014) · Frontiers in Psychology · doi:10.3389/fpsyg.2014.00816 · Cited by 68
“…the mechanisms has not emerged, which is clear from various models that have been proposed to explain the processing of various time intervals by the brain (Buonomano and Merzenich, 1995; Maass et al., 2002; Matell and Meck, 2004; Karmarkar and Buonomano, 2007; Wittmann, 2009; Teki et al., 2011a).”
BackgroundDissipation as a resource for Quantum Reservoir Computing
Sannia, A., Martínez‐Peña, R., Soriano, M. C., Giorgi, G. L., & Zambrini, R. (2024) · Quantum · doi:10.22331/q-2024-03-20-1291 · Cited by 126
“Classical RC was initially proposed as Liquid State Machines [27] and Echo State Networks [28] and later generalized to include a broad spectrum of physical implementations [29].”
BackgroundIs there a Liquid State Machine in the Bacterium Escherichia Coli?
Jones, B., Stekel, D., Rowe, J., & Fernando, C. (2007) · 2007 IEEE Symposium on Artificial Life · doi:10.1109/alife.2007.367795 · Cited by 59
“Maass et al [1] produced a model called the Liquid State Machine (LSM).”
MethodLow-Loss Photonic Reservoir Computing with Multimode Photonic Integrated Circuits
Katumba, A., Heyvaert, J., Schneider, B., Uvin, S., Dambre, J., & Bienstman, P. (2018) · Scientific Reports · doi:10.1038/s41598-018-21011-x · Cited by 71
Optimal modularity and memory capacity of neural reservoirs
Rodriguez, N., Izquierdo, E. J., & Ahn, Y. (2019) · Network Neuroscience · doi:10.1162/netn_a_00082 · Cited by 77
“spiking neurons (common in liquid state machines; Maass et al., 2002) being more heavily de-”
BackgroundKumarasinghe, K., Kasabov, N. K., & Taylor, D. (2019) · Neural Networks · doi:10.1016/j.neunet.2019.08.029 · Cited by 72
Fujii, K., & Nakajima, K. (2021) · Natural computing series · doi:10.1007/978-981-13-1687-6_18 · Cited by 106
FPGA-Based Stochastic Echo State Networks for Time-Series Forecasting
Alomar, M. L., Canals, V., Pérez-Mora, N., Martínez-Moll, V., & Rosselló, J. L. (2015) · Computational Intelligence and Neuroscience · doi:10.1155/2016/3917892 · Cited by 72
Antonik, P., Haelterman, M., & Massar, S. (2017) · Physical Review Applied · doi:10.1103/physrevapplied.7.054014 · Cited by 56
“Finally, going back to the question of biological implementation, our work shows that the biologically plausible structure of reservoir computing [2, 26, 27] can be trained to generate highly complex temporal patterns, both periodic and chaotic, even in the presence of moderate levels of noise.”
BackgroundEditorial: Randomized algorithms for training neural networks
Wang, D. (2016) · Information Sciences · doi:10.1016/j.ins.2016.05.021 · Cited by 66
Hebbian Learning in a Random Network Captures Selectivity Properties of the Prefrontal Cortex
Lindsay, G. W., Rigotti, M., Warden, M. R., Miller, E. K., & Fusi, S. (2017) · Journal of Neuroscience · doi:10.1523/jneurosci.1222-17.2017 · Cited by 70
“…can arise on a circuit level shows that random connectivity is surprisingly efficient at increasing the dimensionality of the neural representation (Maass et al., 2002; Jaeger and Haas, 2004; Buonomano and Maass, 2009; Rigotti et al., 2010; Barak et al., 2013; Babadi and Sompolinsky, 2014;…”
BackgroundParallel Machine Learning for Forecasting the Dynamics of Complex Networks
Srinivasan, K., Coble, N. J., Hamlin, J., Antonsen, T. M., Ott, E., & Girvan, M. (2022) · Physical Review Letters · doi:10.1103/physrevlett.128.164101 · Cited by 65
Embedding theory of reservoir computing and reducing reservoir network using time delays
Duan, X.-Y., Xiong, Y., Leng, S., Kurths, J., Lin, W. J., & Ma, H. (2023) · Physical Review Research · doi:10.1103/physrevresearch.5.l022041 · Cited by 71
Adaptive learning rate of SpikeProp based on weight convergence analysis
Shrestha, S. B., & Song, Q. (2014) · Neural Networks · doi:10.1016/j.neunet.2014.12.001 · Cited by 65
Hinaut, X., Petit, M., Pointeau, G., & Dominey, P. F. (2014) · Frontiers in Neurorobotics · doi:10.3389/fnbot.2014.00016 · Cited by 71
“Further development was realized in related systems including the Liquid State Machine (Maass et al., 2002), and Echo State Network (Jaeger, 2001; Jaeger and Haas, 2004).”
MethodSpatiotemporal Memory Is an Intrinsic Property of Networks of Dissociated Cortical Neurons
Ju, H., Dranias, M. R., Banumurthy, G., & VanDongen, A. M. J. (2015) · Journal of Neuroscience · doi:10.1523/jneurosci.3793-14.2015 · Cited by 75
“Figure 5A shows that our results are similar to those published for simulated LSMs (Maass et al., 2002): after reaching an initial peak, the curves settle to a level proportional to the input difference.”
BackgroundMethodResultInfluentialThe Mutual Inspirations of Machine Learning and Neuroscience
Helmstaedter, M. N. (2015) · Neuron · doi:10.1016/j.neuron.2015.03.031 · Cited by 69
Computation by Switching in Complex Networks of States
Neves, F. S., & Timme, M. (2012) · Physical Review Letters · doi:10.1103/physrevlett.109.018701 · Cited by 65
Reservoir Computing with Untrained Convolutional Neural Networks for Image Recognition
Tong, Z., & Tanaka, G. (2018) · Proceedings - International Conference on Pattern Recognition/Proceedings/International Conference on Pattern Recognition · doi:10.1109/icpr.2018.8545471 · Cited by 82
“The reservoir computing originates from the independently proposed models: the echo state networks (ESNs) [9], [10] and the liquid state machines [11], [12].”
MethodDominey, P. F., Inui, T., & Hoen, M. (2009) · Brain and Language · doi:10.1016/j.bandl.2008.08.002 · Cited by 64
Patel, D., & Ott, E. (2023) · Chaos An Interdisciplinary Journal of Nonlinear Science · doi:10.1063/5.0131787 · Cited by 76
“In what follows, we use reservoir computing [61–63] as the ML platform.”
MethodTime Scale Hierarchies in the Functional Organization of Complex Behaviors
Perdikis, D., Huys, R., & Jirsa, V. K. (2011) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1002198 · Cited by 62
“Even when dynamics beyond point attractors is included (as in recurrent neural networks [23–25]), it is usually broken down into a succession of discrete states (encoding past context) or transients inside a basin of attraction [25] and treated as such.”
BackgroundNeuCube EvoSpike Architecture for Spatio-temporal Modelling and Pattern Recognition of Brain Signals
Kasabov, N. K. (2012) · Lecture notes in computer science · doi:10.1007/978-3-642-33212-8_21 · Cited by 79
“Input information in the NeuCube is entered not to arbitrarily selected neurons (as it is in the reservoir computing [42,62]), but to context dependent one from the NeuCube, e.”
BackgroundNeurocomputational models of interval and pattern timing
Hardy, N. F., & Buonomano, D. V. (2016) · Current Opinion in Behavioral Sciences · doi:10.1016/j.cobeha.2016.01.012 · Cited by 90
Katumba, A., Yin, X. L., Dambre, J., & Bienstman, P. (2019) · Journal of Lightwave Technology · doi:10.1109/jlt.2019.2900568 · Cited by 70
“Reservoir Computing (RC) is a brain-inpired non-VonNeumann analog computing approach that relies on the information processing capabilities of certain physical systems [14]– [16].”
BackgroundMapping of local and global synapses on spiking neuromorphic hardware
Das, A., Wu, Y., Huynh, K., Dell'Anna, F., Catthoor, F., & Schaafsma, S. (2018) · Design, Automation, and Test in Europe · doi:10.23919/date.2018.8342201 · Cited by 64
“PACMAN offers limited flexibility to implement evolving computation models such as the Liquid State Machine (LSM) [11] or Hierarchical Temporal Memory (HTM) [12].”
MethodNakane, R., Hirose, A., & Tanaka, G. (2021) · Physical Review Research · doi:10.1103/physrevresearch.3.033243 · Cited by 68
“Reservoir computing is a computational framework which is originally based on recurrent neural networks [1, 2].”
BackgroundCritical branching neural networks.
Kello, C. T. (2013) · Psychological Review · doi:10.1037/a0030970 · Cited by 63
“One possible answer is that such networks may provide generic computational capacity for other networks to draw upon (Jaeger et al., 2007; Maass et al., 2002).”
BackgroundResultInput-driven bifurcations and information processing capacity in spintronics reservoirs
Akashi, N., Yamaguchi, T., Tsunegi, S., Taniguchi, T., Nishida, M., Sakurai, R., Wakao, Y., & Nakajima, K. (2020) · Physical Review Research · doi:10.1103/physrevresearch.2.043303 · Cited by 63
Farrell, M., Recanatesi, S., Moore, T., Lajoie, G., & Shea‐Brown, E. (2022) · Nature Machine Intelligence · doi:10.1038/s42256-022-00498-0 · Cited by 79
Connectome-based reservoir computing with the conn2res toolbox
Suárez, L. E., Mihalik, Á., Milisav, F., Marshall, K., Li, M., Vértes, P. E., Lajoie, G., & Mišić, B. (2024) · Nature Communications · doi:10.1038/s41467-024-44900-4 · Cited by 82
“Reservoir computing (RC) is an umbrella term that uni fi es two computational paradigms, liquid state machines 32 and echo-state networks 36 .”
InfluentialAdvances in Metal Halide Perovskite Memristors: A Review from a Co‐Design Perspective
Jiang, B., Chen, X., Pan, X., Li, T., Huang, Y., Tang, J., Li, X., Wang, P., Ma, G., Zhang, J., et al. (2024) · Advanced Science · doi:10.1002/advs.202409291 · Cited by 72
An extended echo state network using Volterra filtering and principal component analysis
Boccato, L., Lopes, A., Attux, R., & Zuben, F. J. V. (2012) · Neural Networks · doi:10.1016/j.neunet.2012.02.028 · Cited by 66
Learning to decode human emotions with Echo State Networks
Bozhkov, L., Koprinkova‐Hristova, P., & Georgieva, P. (2015) · Neural Networks · doi:10.1016/j.neunet.2015.07.005 · Cited by 67
Neuromorphic Computing in Ginzburg-Landau Polariton-Lattice Systems
Opala, A., Ghosh, S., Liew, T. C. H., & Matuszewski, M. (2019) · Physical Review Applied · doi:10.1103/physrevapplied.11.064029 · Cited by 66
Reservoir computing with dipole-coupled nanomagnets
Nomura, H., Furuta, T., Tsujimoto, K., Kuwabiraki, Y., Peper, F., Tamura, E., Miwa, S., Goto, M., Nakatani, R., & Suzuki, Y. (2019) · Japanese Journal of Applied Physics · doi:10.7567/1347-4065/ab2406 · Cited by 65
“Jaeger, The ”echo state” approach to analysing and training recurrent neural networks, techreport 148, GMD – German National Research Institute for Computer Science (2001) [8] W.”
MethodEcho state network optimization using binary grey wolf algorithm
Liu, J., Sun, T., Luo, Y., Yang, S., Cao, Y., & Zhai, J. (2019) · Neurocomputing · doi:10.1016/j.neucom.2019.12.069 · Cited by 65
“In order to reduce the computational complexity of the training process, the echo state network (ESN) and the liquid state machine (LSM) are proposed [3, 4], and the study in [5] has proved that ESN and LSM are conceptually similar, they are hence collectively summarized as “reservoir computing”.”
BackgroundPredicting amplitude death with machine learning
Xiao, R., Kong, L.-W., Sun, Z., & Lai, Y. (2021) · Physical review. E · doi:10.1103/physreve.104.014205 · Cited by 62
Mouse frontal cortex mediates additive multisensory decisions
Coen, P., Sit, T. P. H., Wells, M. J., Carandini, M., & Harris, K. D. (2023) · Neuron · doi:10.1016/j.neuron.2023.05.008 · Cited by 65
“Such “mixed selectivity” might result from random network connectivity, which can be computationally powerful, allowing downstream circuits to quickly learn to extract behaviorally relevant feature combinations (Caron et al., 2013; Fusi et al., 2016; Maass et al., 2002; Rigotti et al., 2013).”
Dong, M., Huang, X., & Xu, B. (2018) · PLoS ONE · doi:10.1371/journal.pone.0204596 · Cited by 69
ECG-based Heartbeat Classification in Neuromorphic Hardware
Corradi, F., Pande, S., Stuijt, J., Qiao, N., Schaafsma, S., Indiveri, G., & Catthoor, F. (2019) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn.2019.8852279 · Cited by 62
“By varying the connectivity levels, it has been possible to observe full synchronicity, cluster states, attractor states, and asynchronous dynamics [26].”
BackgroundComputer Models and Analysis Tools for Neural Microcircuits
Natschläger, T., Markram, H., & Maass, W. (2003) · Springer eBooks · doi:10.1007/978-1-4615-1079-6_9 · Cited by 74
Self-citationSpatio-Temporal Learning With the Online Finite and Infinite Echo-State Gaussian Processes
Soh, H., & Demiris, Y. (2014) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2014.2316291 · Cited by 67
Emotion recognition from speech signals via a probabilistic echo-state network
Trentin, E., Scherer, S. J., & Schwenker, F. (2014) · Pattern Recognition Letters · doi:10.1016/j.patrec.2014.10.015 · Cited by 65
Multi-step-ahead host load prediction using autoencoder and echo state networks in cloud computing
Yang, Q., Zhou, Y., Yu, Y., Yuan, J., Xing, X., & Du, S. (2015) · The Journal of Supercomputing · doi:10.1007/s11227-015-1426-8 · Cited by 63
“This idea has been independently discovered and investigated under the name of liquid state machines by Wolfgang Maass and collaborators [17].”
BackgroundSong, Q., & Feng, Z. (2010) · Neurocomputing · doi:10.1016/j.neucom.2010.01.015 · Cited by 64
Recurrent Infomax Generates Cell Assemblies, Neuronal Avalanches, and Simple Cell-Like Selectivity
Tanaka, T., Kaneko, T., & Aoyagi, T. (2008) · Neural Computation · doi:10.1162/neco.2008.03-08-727 · Cited by 68
“1B), which endow networks with many interesting phenomena, some of which have been reported recently and several researchers have attempted to model (Diesmann et al., 1999; Maass et al., 2002; Buonomano, 2005; Vogels and Abbott, 2005; Teramae and Fukai, 2007).”
BackgroundDynamical systems, attractors, and neural circuits
Miller, P. (2016) · F1000Research · doi:10.12688/f1000research.7698.1 · Cited by 59
“Moreover, certain learning rules for changing of connection strengths between neurons in a chaotic system can allow the encoding of almost any spatiotemporal input pattern 4,95 , the switching between multiple patterns 4 , and the encoding and processing of many rule-based tasks 6,8 .”
BackgroundBöhmer, W., Springenberg, J. T., Boedecker, J., Riedmiller, M., & Obermayer, K. (2015) · Künstliche Intell. · doi:10.1007/s13218-015-0356-1 · Cited by 83
“There are several possibilities to make Z Markov, for example temporal embedding [37], liquid state machines [31], belief states [18] or predictive state representations [29, 55], which each come with their own disadvantages.”
BackgroundAnytime multipurpose emotion recognition from EEG data using a Liquid State Machine based framework
Zoubi, O. A., Awad, M., & Kasabov, N. K. (2018) · Artificial Intelligence in Medicine · doi:10.1016/j.artmed.2018.01.001 · Cited by 64
Analysis of Liquid Ensembles for Enhancing the Performance and Accuracy of Liquid State Machines
Wijesinghe, P., Srinivasan, G., Panda, P., & Roy, K. (2019) · Frontiers in Neuroscience · doi:10.3389/fnins.2019.00504 · Cited by 66
“, 2009), action recognition (Panda and Srinivasa, 2018), speech recognition (Maass et al., 2002; Verstraeten et al., 2005; Goodman and Ventura, 2006; Zhang et al., 2015; Wu et al., 2018; Zhang and Li, 2019), and image recognition (Grzyb et al.”
BackgroundMethodInfluentialBertschinger, N., Natschläger, T., & Legenstein, R. A. (2004) · neural information processing systems · Cited by 60
Self-citationEMG-Based Gestures Classification Using a Mixed-Signal Neuromorphic Processing System
Ma, Y., Chen, B., Ren, P., Zheng, N., Indiveri, G., & Donati, E. (2020) · IEEE Journal on Emerging and Selected Topics in Circuits and Systems · doi:10.1109/jetcas.2020.3037951 · Cited by 59
“A widely used method is to use attractor networks and Recurrent Neural Networks (RNNs) [8], [9].”
MethodLeveraging volatile memristors in neuromorphic computing: from materials to system implementation
Moon, T., Soh, K., Kim, J. S., Kim, J. E., Chun, S. Y., Cho, K., Yang, J. J., & Yoon, J. H. (2024) · Materials Horizons · doi:10.1039/d4mh00675e · Cited by 62
Inverse problems in dynamic cognitive modeling
Graben, P. B., & Potthast, R. (2009) · Chaos An Interdisciplinary Journal of Nonlinear Science · doi:10.1063/1.3097067 · Cited by 63
Evolving Carbon Nanotube Reservoir Computers
Dale, M., Miller, J. F., Stepney, S., & Trefzer, M. A. (2016) · Lecture notes in computer science · doi:10.1007/978-3-319-41312-9_5 · Cited by 73
“Reservoir Computing (RC) [6, 11] has been proposed as an expressive model and as a computationally inexpensive method for training rich high-dimensional dynamical systems, ranging from simulated and biological neural networks to novel hardware-based implementations [9].”
MethodBing, Z., Cheng, L., Chen, G., Röhrbein, F., Huang, K., & Knoll, A. (2017) · Bioinspiration & Biomimetics · doi:10.1088/1748-3190/aa644c · Cited by 59
Metadata
- Authors
- Maass, W., Natschläger, T., & Markram, H.
- Affiliations
- Graz University of Technology (AT)
- École Polytechnique Fédérale de Lausanne (CH)
- Corresponding author
- Maass, Wolfgang
- Natschläger, Thomas
- Markram, Henry
- Venue
- Neural Computation
- Publisher
- MIT Press
- Published
- 2002-11-01
- Volume
- 14
- Issue
- 11
- Pages
- 2531-2560
- ISSN
- 0899-7667 (ISSN-L)
- 1530-888X
- OpenAlex
- W2103179919
- Cited by
- 4,025
- References
- 28
- Citations by year
- 2026: 286 · 2025: 351 · 2024: 307 · 2023: 309 · 2022: 276 · 2021: 303 · 2020: 239 · 2019: 244 · 2018: 217 · 2017: 173 · 2016: 155 · 2015: 135 · 2014: 142 · 2013: 139 · 2012: 125
- FWCI
- 18.37
- Type
- journal article
- Language
- en
- Open access
- green
- infoscience.epfl.ch
- Locations
- Neural Computation (publishedVersion)
- PubMed (publishedVersion)
- Infoscience (Ecole Polytechnique Fédérale de Lausanne) (submittedVersion)
- Infoscience (Ecole Polytechnique Fédérale de Lausanne) (submittedVersion, cc-by-nc-nd)
- Topics
- Life Sciences › Neuroscience › Cognitive Neuroscience › Neural dynamics and brain function
- Physical Sciences › Computer Science › Artificial Intelligence › Neural Networks and Reservoir Computing
- Physical Sciences › Computer Science › Artificial Intelligence › Neural Networks and Applications
- Keywords
- liquid state machines, recurrent neural circuits, transient dynamics, high-dimensional dynamical systems, fading memory, integrate-and-fire neurons, real-time processing, time-varying inputs, neural coding, statistical learning theory, computational power, attractor networks
- MeSH terms
- Action Potentials, Computer Simulation, Computer Systems, Computers, Models, Neurological, Neurons, Neural Networks, Computer
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.