Real-Time Computing Without Stable States: A New Framework for Neural Computation Based on Perturbations
Open accessThe DOI matches its Crossref record; 28 of the 28 references it cites were found in a registry or an index.
Abstract
A key challenge for neural modeling is to explain how a continuous stream of multimodal input from a rapidly changing environment can be processed by stereotypical recurrent circuits of integrate-and-fire neurons in real time. We propose a new computational model for real-time computing on time-varying input that provides an alternative to paradigms based on Turing machines or attractor neural networks. It does not require a task-dependent construction of neural circuits. Instead, it is based on principles of high-dimensional dynamical systems in combination with statistical learning theory and can be implemented on generic evolved or found recurrent circuitry. It is shown that the inherent transient dynamics of the high-dimensional dynamical system formed by a sufficiently large and heterogeneous neural circuit may serve as universal analog fading memory. Readout neurons can learn to extract in real time from the current state of such recurrent neural circuit information about current and past inputs that may be needed for diverse tasks. Stable internal states are not required for giving a stable output, since transient internal states can be transformed by readout neurons into stable target outputs due to the high dimensionality of the dynamical system. Our approach is based on a rigorous computational model, the liquid state machine, that, unlike Turing machines, does not require sequential transitions between well-defined discrete internal states. It is supported, as the Turing machine is, by rigorous mathematical results that predict universal computational power under idealized conditions, but for the biologically more realistic scenario of real-time processing of time-varying inputs. Our approach provides new perspectives for the interpretation of neural coding, the design of experiments and data analysis in neurophysiology, and the solution of problems in robotics and neurotechnology.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Although work in artificial intelligence and machine-learning research has shown that similar or even more complex outcomes can be achieved with a variety of techniques, including deep neural networks with trained weights (Gibney, 2015), reservoir networks with a trained output mapping (Maass et al., 2002;Jaeger & Haas, 2004), networks that are evolved using a genetic algorithm (Iizuka & Di Paolo,”
From A potential mechanism for Gibsonian resonance: behavioral entrainment emerges from local homeostasis in an unsupervised reservoir network (2024) · cites this paper as Maass et al., 2002
In the citing paper
“Finally, we show that a randomly connected circuit model, akin to a “reservoir” network13-15, can reproduce the observed distribution of timescales, provided that the network operates at the critical point (or “edge of chaos”)16-18.”
From A reservoir of time constants for memory traces in cortical neurons (2011) · cites this paper as [13]
In the citing paper
“Reservoir-type networks have been the subject of active research in computational neuroscience and machine learning13-15, but so far experimental support that such networks are adopted by the brain has been lacking.”
From A reservoir of time constants for memory traces in cortical neurons (2011) · cites this paper as [13]
In the citing paper
“Those models predict that the memory of input signals is stored in a large, recurrent and heterogeneous network (reservoir), in a distributed manner, and that a desired output is obtained by a trainable combination of the response signals in the reservoir.”
From A reservoir of time constants for memory traces in cortical neurons (2011) · cites this paper as [13]
Citations
Cites
Wu, Y. (1999) · Technometrics · doi:10.1080/00401706.1999.10485951 · Cited by 27,221
Introduction to the theory of neural computation
(1994) · Neural Networks · doi:10.1016/0893-6080(94)90014-0 · Cited by 6,401
The Synaptic Organization of the Brain
(2004) · Oxford University Press eBooks · doi:10.1093/acprof:oso/9780195159561.001.1 · Cited by 3,546 · cited 2×
“This informationisprocessed by extremely complexbut surprisingly stereotypic microcircuits that can perform a wide spectrum of tasks (Shep-herd, 1988 ; Douglas & Martin, 1998 ; von Melchner, Pallas, & Sur, 2000).”
Dynamics of ongoing activity: explanation of the large variability in evoked cortical responses
Arieli, A., Sterkin, A., Grinvald, A., & Aertsen, A. (1996) · Science · doi:10.1126/science.273.5283.1868 · Cited by 1,660 · cited 1×
“Furthermore, although the input rate may contain salient information that is constant for a particular readout element, it may not be for another (see, for example, Figure 8), indicating that equivalence classes and dynamic stability exist purely from the perspective of the readout elements.”
BackgroundDifferential signaling via the same axon of neocortical pyramidal neurons
Markram, H., Wang, Y., & Tsodyks, M. (1998) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.95.9.5323 · Cited by 1,102 · cited 2×
“In the case of a synaptic connection from a to b , we modeled the synaptic dynamics according to the model proposed in Markram, Wang, and Tsodyks (1998), with the synaptic parameters U (use), D (time constant for depression), and F (time constant for facilitation) randomly chosen from gaussian…”
BackgroundMethodSelf-citationOrganizing principles for a diversity of GABAergic interneurons and synapses in the neocortex
Gupta, A., Wang, Y., & Markram, H. (2000) · Science · doi:10.1126/science.287.5451.273 · Cited by 1,036
Self-citationFading memory and the problem of approximating nonlinear operators with Volterra series
Boyd, S., & Chua, L. (1985) · IEEE Transactions on Circuits and Systems · doi:10.1109/tcs.1985.1085649 · Cited by 991 · cited 2×
“> The proof of this theorem follows from the Stone-Weierstrass approximation theorem, like the proof of theorem 1 in Boyd and Chua (1985).”
BackgroundGradient calculations for dynamic recurrent neural networks: a survey
Pearlmutter, B. A. (1995) · IEEE Transactions on Neural Networks · doi:10.1109/72.410363 · Cited by 582 · cited 1×
“Methods for controllingthe dynamics of recurrent neural networks through adaptive mechanisms are reviewed in Pearlmutter (1995).”
Temporal Information Transformed into a Spatial Code by a Neural Network with Realistic Properties
Buonomano, D. V., & Merzenich, M. M. (1995) · Science · doi:10.1126/science.7863330 · Cited by 449 · cited 1×
“Special cases of this mechanism have already been reported in Buonomano and Merzenich (1995) and Dominey, Arbib, and Joseph (1995).”
BackgroundLearning and Extracting Finite State Automata with Second-Order Recurrent Neural Networks
Giles, C. L., Miller, C. B., Chen, D., Chen, H. H., Sun, G. Z., & Lee, Y. C. (1992) · Neural Computation · doi:10.1162/neco.1992.4.3.393 · Cited by 392
Cited by
Items 701–800 of 1,014
Nonlinear dynamics of the brain: emotion and cognition
Rabinovich, M. I., & Muezzinoglu, M. K. (2010) · Uspekhi Fizicheskih Nauk · doi:10.3367/ufnr.0180.201004b.0371 · Cited by 59
Spintronic reservoir computing without driving current or magnetic field
Taniguchi, T., Ogihara, A., Utsumi, Y., & Tsunegi, S. (2022) · Scientific Reports · doi:10.1038/s41598-022-14738-1 · Cited by 54
“For example, in NARMA2 task, the system is aimed to reproduce the target data, (5) y k,D = D”
MethodSpiking neurons and the induction of finite state machines
Natschläger, T., & Maass, W. (2002) · Theoretical Computer Science · doi:10.1016/s0304-3975(02)00099-3 · Cited by 61
Self-citationHuang, J., Qian, J. W., Liu, L., Wang, Y., Xiong, C., & Ri, S. (2016) · Journal of the Franklin Institute · doi:10.1016/j.jfranklin.2016.05.004 · Cited by 49
Deep Learning Algorithms in EEG Signal Decoding Application: A Review
Vallabhaneni, R. B., Sharma, P., Kumar, V., Kulshreshtha, V., Reddy, K. J., Kumar, S. S., Kumar, V. S., & Bitra, S. K. (2021) · IEEE Access · doi:10.1109/access.2021.3105917 · Cited by 53
“With the needs of practical applications, researchers have proposed many kinds of RNN frameworks, such as Elman network [24], Jordan network [25], time delay neural network (TDNN) [26] and echo state network (ESN) [27], etc. RNN not only provide feed forward connection, but also feedback connection.”
BackgroundRobust forecasting using predictive generalized synchronization in reservoir computing
Platt, J. A., Wong, A., Clark, R., Penny, S. G., & Abarbanel, H. D. I. (2021) · Chaos An Interdisciplinary Journal of Nonlinear Science · doi:10.1063/5.0066013 · Cited by 62
“A class of RNN architectures with demonstrated capability for forecasting dynamical systems is reservoir computing (RC) [3, 4, 5, 6, 7, 8, 9, 10, 11, 12].”
BackgroundEEG-based emotion recognition using random Convolutional Neural Networks
Cheng, W., Gao, R., Suganthan, P. N., & Yuen, K. F. (2022) · Engineering Applications of Artificial Intelligence · doi:10.1016/j.engappai.2022.105349 · Cited by 53
Häuser, H., Nanayakkara, T., & Forni, F. (2023) · IEEE Control Systems · doi:10.1109/mcs.2023.3253422 · Cited by 60
Preparing More Effective Liquid State Machines Using Hebbian Learning
Norton, D. P., & Ventura, D. A. (2006) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn.2006.246996 · Cited by 49
Optimal nonlinear information processing capacity in delay-based reservoir computers
Grigoryeva, L., Henriques, J., Larger, L., & Ortega, J. (2015) · Scientific Reports · doi:10.1038/srep12858 · Cited by 57
Wang, Q., Jin, Y., & Li, P. (2015) · IEEE Biomedical Circuits and Systems Conference (BioCAS) · doi:10.1109/biocas.2015.7348397 · Cited by 49
“The liquid state machine (LSM) is a recurrent neural network that recently emerged in theoretical neuroscience [2], and provides a solution to bridge the gap between biological plausibility and practical tractability of recurrent networks.”
BackgroundDistributed Cerebellar Motor Learning: A Spike-Timing-Dependent Plasticity Model
Luque, N. R., Garrido, J. A., Naveros, F., Carrillo, R. R., D'Angelo, E. U., & Ros, E. (2016) · Frontiers in Computational Neuroscience · doi:10.3389/fncom.2016.00017 · Cited by 50
“In other words, the state-generator model is able to represent the POT by means of a sparse-population coding scheme, thus allowing the cerebellum to operate like a liquid state machine (LSM; Maass et al., 2002; Yamazaki and Tanaka, 2007a) or an Echo state network (Jaeger, 2007).”
MethodKasabov, N. K., Zhou, L., Doborjeh, M., Doborjeh, Z., & Yang, J. (2016) · IEEE Transactions on Cognitive and Developmental Systems · doi:10.1109/tcds.2016.2636291 · Cited by 54
A Multiple-Input Strategy to Efficient Integrated Photonic Reservoir Computing
Katumba, A., Freiberger, M., Bienstman, P., & Dambre, J. (2017) · Cognitive Computation · doi:10.1007/s12559-017-9465-5 · Cited by 57
Neuromorphic Computing Based on Silicon Photonics and Reservoir Computing
Katumba, A., Freiberger, M., Laporte, F., Lugnan, A., Sackesyn, S., Ma, C., Dambre, J., & Bienstman, P. (2018) · IEEE Journal of Selected Topics in Quantum Electronics · doi:10.1109/jstqe.2018.2821843 · Cited by 55
“Reservoir computing (RC) [1]–[3] was proposed as a methodology to ease the training of these recurrent networks, which”
MethodInvertible generalized synchronization: A putative mechanism for implicit learning in neural systems
Lu, Z., & Bassett, D. S. (2020) · Chaos An Interdisciplinary Journal of Nonlinear Science · doi:10.1063/5.0004344 · Cited by 59
Online stability of backpropagation–decorrelation recurrent learning
Steil, J. J. (2006) · Neurocomputing · doi:10.1016/j.neucom.2005.12.012 · Cited by 49
Temporal information processing induced by quantum noise
Kubota, T., Suzuki, Y., Kobayashi, S., Tran, Q. H., Yamamoto, N., & Nakajima, K. (2023) · Physical Review Research · doi:10.1103/physrevresearch.5.023057 · Cited by 69
Cellular Automata Inspired Multistable Origami Metamaterials for Mechanical Learning
Liu, Z., Fang, H., Xu, J., & Wang, K. (2023) · Advanced Science · doi:10.1002/advs.202305146 · Cited by 55
Directed and acyclic synaptic connectivity in the human layer 2-3 cortical microcircuit
Peng, Y., Bjelde, A., Aceituno, P. V., Mittermaier, F. X., Planert, H., Grosser, S., Onken, J., Faust, K., Kalbhenn, T., Simon, M., et al. (2024) · Science · doi:10.1126/science.adg8828 · Cited by 53
Sen, D., Ravichandran, H., Das, M. B., Venkatram, P., Choo, S., Varshney, S., Zhang, Z., Sun, Y., Shah, J. M., Radhakrishnan, S. S., et al. (2024) · Nature Communications · doi:10.1038/s41467-024-54231-z · Cited by 48
“…switching in the STO-gated MoS 2 FET to construct a physical reservoir for accelerating RC. Note that, RC, derived from the framework of recurrent neural networks (RNNs), has emerged as a powerful computational paradigm for processing time-series data and performing pattern recognition 30,31 .”
Digital Implementation of a Single Dynamical Node Reservoir Computer
Alomar, M. L., Soriano, M. C., Escalona-Morán, M., Canals, V., Fischer, I., Mirasso, C. R., & Rosselló, J. L. (2015) · IEEE Transactions on Circuits & Systems II Express Briefs · doi:10.1109/tcsii.2015.2458071 · Cited by 57
“Reservoir computing (RC) stands out as a relatively simple technique to implement RNNs [1]–[3].”
MethodGenetic algorithm for reservoir computing optimization
Ferreira, A. A., & Ludermir, T. B. (2009) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn.2009.5178654 · Cited by 55
Prospective Coding by Spiking Neurons
Brea, J., Gaál, A. T., Urbanczik, R., & Senn, W. (2016) · PLOS Computational Biology · doi:10.1371/journal.pcbi.1005003 · Cited by 40
“The short-term memory network can be seen as a liquid state machine [42] or echo state machine [43] and the ramping activity is learned as readout from this activity traces.”
BackgroundSpatiotemporal Dynamics and Reliable Computations in Recurrent Spiking Neural Networks
Pyle, R., & Rosenbaum, R. (2017) · Physical Review Letters · doi:10.1103/physrevlett.118.018103 · Cited by 51
Efficient design of hardware-enabled reservoir computing in FPGAs
Penkovsky, B., Larger, L., & Brunner, D. (2018) · Journal of Applied Physics · doi:10.1063/1.5039826 · Cited by 48
Learning recurrent dynamics in spiking networks
Kim, C. M., & Chow, C. C. (2018) · eLife · doi:10.7554/elife.37124 · Cited by 50
Learning Temporal Quantum Tomography
Tran, Q. H., & Nakajima, K. (2021) · Physical Review Letters · doi:10.1103/physrevlett.127.260401 · Cited by 56
“The effect of a sequence of quantum input to the output of a quantum device has also been demonstrated experimentally for optical fiber channels with fluctuating birefringence [27, 28].”
BackgroundCopying and Evolution of Neuronal Topology
Fernando, C., Karishma, K. K., & Szathmáry, E. (2008) · PLoS ONE · doi:10.1371/journal.pone.0003775 · Cited by 53
“the separation property of a neuronal microcircuit [101].”
BackgroundHaeusler, S., Schuch, K., & Maass, W. (2009) · Journal of Physiology-Paris · doi:10.1016/j.jphysparis.2009.05.006 · Cited by 50
“We modeled the short term synaptic dynamics according to the model proposed in Markram et al. (1998), with synaptic parameters chosen as in Maass et al. (2002) to fit data from microcircuits in rat somatosensory cortex (based on Gupta et al., 2000; Markram et al., 1998).”
Self-citationNeurocomputational Models of Time Perception
Haß, J., & Durstewitz, D. (2014) · Advances in experimental medicine and biology · doi:10.1007/978-1-4939-1782-2_4 · Cited by 54
“[105]).”
Multifunctionality in a reservoir computer
Flynn, A., Tsachouridis, V. A., & Amann, A. (2021) · Chaos An Interdisciplinary Journal of Nonlinear Science · doi:10.1063/5.0019974 · Cited by 60
Sun, J., Yang, W., Zheng, T., Xiong, X., Liu, Y., Wang, Z., Li, Z., & Zou, X. (2021) · Microsystems & Nanoengineering · doi:10.1038/s41378-021-00313-7 · Cited by 54
Reservoir-computing based associative memory and itinerancy for complex dynamical attractors
Kong, L.-W., Brewer, G. A., & Lai, Y. (2024) · Nature Communications · doi:10.1038/s41467-024-49190-4 · Cited by 60
Kasabov, N. K. (2012) · Lecture notes in computer science · doi:10.1007/978-3-642-30687-7_12 · Cited by 54
“Despite difficulties with the LSM reservoirs, it was shown on several SSTD problems that they produce better results than using a simple classifier [95, 73, 99, 60].”
BackgroundMethodInfluentialTen years of Nature Reviews Neuroscience: insights from the highly cited
Luo, L., Rodríguez, E., Jerbi, K., Lachaux, J.-P., Martinerie, J. M., Corbetta, M., Shulman, G. L., Piomelli, D., Turrigiano, G. G., Nelson, S. B., et al. (2010) · Nature reviews. Neuroscience · doi:10.1038/nrn2912 · Cited by 50
A Fast SVD-Hidden-nodes based Extreme Learning Machine for Large-Scale Data Analytics
Deng, W., Bai, Z., Huang, G.-B., & Zheng, Q. (2015) · Neural Networks · doi:10.1016/j.neunet.2015.09.003 · Cited by 47
Wang, Q., Li, Y., & Li, P. (2016) · IEEE International Symposium on Circuits and Systems proceedings · doi:10.1109/iscas.2016.7527245 · Cited by 47
“Structurally, the LSM consists of a reservoir receiving input spike trains and a group of readout neurons receiving signals from the reservoir [5].”
BackgroundThe End of Moore’s Law: Opportunities for Natural Computing?
Peper, F. (2017) · New Generation Computing · doi:10.1007/s00354-017-0020-4 · Cited by 52
“An artificial neural network with similar behavior is the Liquid State Machine [40].”
MethodMultiplexed networks: reservoir computing with virtual and real nodes
Röhm, A., & Lüdge, K. (2018) · Journal of Physics Communications · doi:10.1088/2399-6528/aad56d · Cited by 53
Zhu, Q., Ma, H., & Lin, W. J. (2019) · Chaos An Interdisciplinary Journal of Nonlinear Science · doi:10.1063/1.5120867 · Cited by 47
Sitting Posture Recognition Using a Spiking Neural Network
Wang, J., Hafidh, B., Dong, H., & Saddik, A. E. (2020) · IEEE Sensors Journal · doi:10.1109/jsen.2020.3016611 · Cited by 47
“Liquid state machines (LSMs) are one of the most effective computation modules and were proposed by Maass [5].”
Using multidimensional speckle dynamics for high-speed, large-scale, parallel photonic computing
Sunada, S., Kanno, K., & Uchida, A. (2020) · Optics Express · doi:10.1364/oe.399495 · Cited by 52
A Reservoir-based Convolutional Spiking Neural Network for Gesture Recognition from DVS Input
George, A. M., Banerjee, D., Dey, S., Mukherjee, A., & Balamurali, P. (2020) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn48605.2020.9206681 · Cited by 61
“On the other hand, in LSM, the temporal features of the input are extracted by a recurrently connected network of spiking neurons, called the ”liquid” , the output of which is trained to produce certain desired activity based on some learning rule.”
MethodEnhancing computational performance of a spin-wave reservoir computer with input synchronization
Watt, S., Kostylev, M., & Ustinov, A. B. (2021) · Journal of Applied Physics · doi:10.1063/5.0033292 · Cited by 49
Grigoryeva, L., Henriques, J., Larger, L., & Ortega, J. (2014) · Neural Networks · doi:10.1016/j.neunet.2014.03.004 · Cited by 52
Materials for emergent silicon-integrated optical computing
Demkov, A. A., Bajaj, C., Ekerdt, J. G., Palmstrøm, C. J., & Yoo, S. J. B. (2021) · Journal of Applied Physics · doi:10.1063/5.0056441 · Cited by 52
Stable representation of a naturalistic movie emerges from episodic activity with gain variability
Xià, J., Marks, T. D., Goard, M. J., & Weßel, R. (2021) · Nature Communications · doi:10.1038/s41467-021-25437-2 · Cited by 54
Echo State Networks for Mobile Robot Modeling and Control
Plöger, P. G., Arghir, A., Günther, T., & Hosseiny, R. (2004) · Lecture notes in computer science · doi:10.1007/978-3-540-25940-4_14 · Cited by 54
Carvalho, T. P., & Buonomano, D. V. (2011) · Frontiers in Integrative Neuroscience · doi:10.3389/fnint.2011.00020 · Cited by 51
“…primary target of such a rule. dIscussIon It has been proposed that STP may play a role in timing (Buonomano and Merzenich, 1995; Buonomano, 2000; Maass et al., 2002) and the filtering of time-varying patterns of activity (Dobrunz and Frontiers in Integrative Neuroscience www.frontiersin.org May…”
BackgroundGoréa, A. (2011) · Journal of Physiology-Paris · doi:10.1016/j.jphysparis.2011.09.008 · Cited by 48
“…proposed that the passage of time is directly coded by the evolving pattern of activity in a neural network (a multi-neuron response trajectory in a neural manifold) activated by a timing onset event (Buonomano, 2000; Buonomano and Mauk, 1994; Buonomano and Merzenich, 1995; Maass et al., 2002).”
BackgroundTemporal dynamics of information content carried by neurons in the primary visual cortex
Nikolić, D., Haeusler, S., Singer, W., & Maass, W. (2007) · Advances in Neural Information Processing Systems 19 · doi:10.7551/mitpress/7503.003.0135 · Cited by 43
“Alternative computational models from dynamical systems theory [4] argue that fading memory properties of neural circuits are essential for real-time processing of quickly varying visual stimuli.”
BackgroundSelf-citationReducing Communication for Distributed Learning in Neural Networks
Auer, P. L., Burgsteiner, H., & Maass, W. (2002) · Lecture notes in computer science · doi:10.1007/3-540-46084-5_21 · Cited by 47
Self-citationSpiking neural networks for inference and learning: a memristor-based design perspective
Fouda, M. E., Kurdahi, F. J., Eltawil, A. M., & Neftci, E. (2020) · Elsevier eBooks · doi:10.1016/b978-0-08-102782-0.00019-8 · Cited by 53
A Review of Mathematical and Computational Methods in Cancer Dynamics
Uthamacumaran, A., & Zenil, H. (2022) · Frontiers in Oncology · doi:10.3389/fonc.2022.850731 · Cited by 52
“Liquid State Machines (LSM) are a type of spiking neural networks composed of arti fi cial neurons with threshold activation functions (170).”
BackgroundIn materia implementation strategies of physical reservoir computing with memristive nanonetworks
Milano, G., Montano, K., & Ricciardi, C. (2023) · Journal of Physics D Applied Physics · doi:10.1088/1361-6463/acb7ff · Cited by 48
Reducing reservoir computer hyperparameter dependence by external timescale tailoring
Jaurigue, L., & Lüdge, K. (2024) · Neuromorphic Computing and Engineering · doi:10.1088/2634-4386/ad1d32 · Cited by 57
Alalshekmubarak, A., & Smith, L. S. (2013) · International Conference on Innovations in Information Technology · doi:10.1109/innovations.2013.6544391 · Cited by 46
Optimal Balance of the Striatal Medium Spiny Neuron Network
Ponzi, A. P. D., & Wickens, J. R. (2013) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1002954 · Cited by 48
“The spatio-temporal dynamics [55,56,69,115–120] generated by this network could be utilized in such behavioural tasks or in ‘reservoir computing’ style cognitive processing [27,38,39,55,121,122].”
MethodMezzi, R., Morando, S., Steiner, N. Y., Péra, M., Hissel, D., & Larger, L. (2018) · Annual Conference of the IEEE Industrial Electronics Society (IECON) · doi:10.1109/iecon.2018.8591345 · Cited by 46
“The second one is the letter of Neural Computation" Real-time stable computing without states: In new framework for neural based calculation one disturbances" by Maass et al [20] who explained the Liquid State Machine (LSM).”
BackgroundFreiberger, M., Sackesyn, S., Ma, C., Katumba, A., Bienstman, P., & Dambre, J. (2019) · IEEE Journal of Selected Topics in Quantum Electronics · doi:10.1109/jstqe.2019.2929699 · Cited by 50
“One of them, reservoir computing [2]– [4], originally proposed as an effective training method for recurrent neural networks, appears to be especially promising as it is well-fit to exploit the natural dynamics of physical systems [5]–[7].”
BackgroundCompiling Spiking Neural Networks to Neuromorphic Hardware
Song, S., Balaji, A., Das, A. K., Kandasamy, N., & Shackleford, J. (2020) · doi:10.1145/3372799.3394364 · Cited by 54
“Spiking neurons can also be organized in recurrent topologies [45].”
BackgroundSingular Value Decomposition (SVD)
Brunton, S. L., & Kutz, J. N. (2019) · Cambridge University Press eBooks · doi:10.1017/9781108380690.002 · Cited by 74
Predicting future dynamics from short-term time series using an Anticipated Learning Machine
Chen, C., Li, R., Shu, L., He, Z., Wang, J., Zhang, C., Ma, H., Aihara, K., & Chen, L. (2020) · National Science Review · doi:10.1093/nsr/nwaa025 · Cited by 46
Flapping‐Wing Dynamics as a Natural Detector of Wind Direction
Tanaka, K., Yang, S.-H., Tokudome, Y., Minami, Y., Lu, Y., Arie, T., Akita, S., Takei, K., & Nakajima, K. (2020) · Advanced Intelligent Systems · doi:10.1002/aisy.202000174 · Cited by 46
Self-organization of an inhomogeneous memristive hardware for sequence learning
Payvand, M., Moro, F., Nomura, K., Dalgaty, T., Vianello, E., Nishi, Y., & Indiveri, G. (2022) · Nature Communications · doi:10.1038/s41467-022-33476-6 · Cited by 49
The Search for Alternative Computational Paradigms
Shanbhag, N. R., Mitra, S., Veciana, G. D., Orshansky, M. E., Mărculescu, R., Roychowdhury, J., Jones, D., & Rabaey, J. M. (2008) · IEEE Design & Test of Computers · doi:10.1109/mdt.2008.113 · Cited by 46
“Perturbation-based computing Perturbation-based computing performs real-time computational tasks on time-varying input streams, using the transient perturbations they induce on a high-dimensional dynamic system.(9) Specifically, the user excites a high-dimensional dynamic system such as a complex recurrent network wit…”
BackgroundGenerative Modeling of Autonomous Robots and their Environments using Reservoir Computing
Antonelo, E. A., Schrauwen, B., & Campenhout, J. V. (2007) · Neural Processing Letters · doi:10.1007/s11063-007-9054-9 · Cited by 51
“RC has biological foundations as it is shown, for example, that Liquid State Machines (a type of RC) are based on the micro-column structure in the cortex [11].”
MethodKalloniatis, A. C., Leinweber, D. B., & Williams, A. G. (2005) · Springer eBooks · doi:10.1007/b103529 · Cited by 46
Dynamical Motifs: Building Blocks of Complex Dynamics in Sparsely Connected Random Networks
Zhigulin, V. P. (2004) · Physical Review Letters · doi:10.1103/physrevlett.92.238701 · Cited by 44
Nakane, R., Hirose, A., & Tanaka, G. (2023) · Physical Review Applied · doi:10.1103/physrevapplied.19.034047 · Cited by 47
Isolated word recognition using a Liquid State Machine
Verstraeten, D., Schrauwen, B., & Stroobandt, D. (2005) · Ghent University Academic Bibliography (Ghent University) · Cited by 45
Linear algebra for time series of spikes
Carnell, A., & Richardson, D. R. (2005) · The European Symposium on Artificial Neural Networks · Cited by 45
Sussillo, D., & Abbott, L. F. (2012) · PLoS ONE · doi:10.1371/journal.pone.0037372 · Cited by 45
“To avoid these difficulties, Maass and collaborators [4] and Jaeger [5] suggested limiting synaptic modification during learning to the output weights, leaving the recurrent weights unchanged.”
Variation-tolerant Computing with Memristive Reservoirs
Burger, J., & Teuscher, C. (2013) · 2013 IEEE/ACM International Symposium on Nanoscale Architectures (NANOARCH) · doi:10.1109/nanoarch.2013.6623028 · Cited by 46
“Two different concepts of RC have been individually developed by Maass [2] and Jaeger [3].”
BackgroundInfluentialFunctional Model of a Nanoparticle Organic Memory Transistor for Use as a Spiking Synapse
Bichler, O., Zhao, W. S., Alibart, F., Pleutin, S., Vuillaume, D., & Gamrat, C. (2010) · IEEE Transactions on Electron Devices · doi:10.1109/ted.2010.2065951 · Cited by 45
“to implement some dynamic neural-network-based computing systems such as the liquid state machine [14] and some natural”
BackgroundNeuromorphic behavior in percolating nanoparticle films
Fostner, S., & Brown, S. A. (2015) · Physical Review E · doi:10.1103/physreve.92.052134 · Cited by 50
Brain-inspired computational paradigm dedicated to fault diagnosis of PEM fuel cell stack
Zheng, Z., Morando, S., Péra, M., Hissel, D., Larger, L., Martinenghi, R., & Baylón-Fuentes, A. (2017) · International Journal of Hydrogen Energy · doi:10.1016/j.ijhydene.2016.11.043 · Cited by 46
Dalgaty, T., Payvand, M., Moro, F., Ly, D. R. B., Pebay-Peyroula, F., Casas, J., Indiveri, G., & Vianello, E. (2019) · APL Materials · doi:10.1063/1.5108663 · Cited by 45
Harkhoe, K. K., & der Sande, G. V. (2019) · IEEE Journal of Selected Topics in Quantum Electronics · doi:10.1109/jstqe.2019.2952594 · Cited by 45
“RC is a unified computational framework based on previous independently proposed RNN models, the echo state network [2] and the liquid-state machine [3].”
MethodNeuronal spike-rate adaptation supports working memory in language processing
Fitz, H., Uhlmann, M., den Broek, D. V., Duarte, R., Hagoort, P., & Petersson, K. M. (2020) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2000222117 · Cited by 51
Reservoir Computing with Neuromemristive Nanowire Networks
Fu, K., Zhu, R., Loeffler, A., Hochstetter, J., Álvarez, A. D., Stieg, A. Z., Gimzewski, J. K., Nakayama, T., & Kuncic, Z. (2020) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn48605.2020.9207727 · Cited by 54
“Originating from echo state networks and liquid state machines, which were proposed as a means to reduce the training cost of recurrent neural network models [2], RC leverages nonlinear dynamical properties of a high-dimensional reservoir to process information [3].”
BackgroundEmergence of transient chaos and intermittency in machine learning
Kong, L.-W., Fan, H., Grebogi, C., & Lai, Y. (2021) · Journal of Physics Complexity · doi:10.1088/2632-072x/ac0b00 · Cited by 45
Simulation platform for pattern recognition based on reservoir computing with memristor networks
Tanaka, G., & Nakane, R. (2022) · Scientific Reports · doi:10.1038/s41598-022-13687-z · Cited by 47
“For reservoirs constructed with recurrent neural networks 5–7 , there are some practical design guides which are mainly helpful in software computation 8,9 .”
MethodHilbert space as a computational resource in reservoir computing
Kalfus, W. D., Ribeill, G. J., Rowlands, G. E., Krovi, H., Ohki, T., & Govia, L. C. G. (2022) · Physical Review Research · doi:10.1103/physrevresearch.4.033007 · Cited by 55
Modeling reward functions for incomplete state representations via echo state networks
Bush, K., & Anderson, C. W. (2005) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn.2005.1556402 · Cited by 45
“Definition 1 is alternatively described by Maass as the pointwise separation property [8].”
InfluentialA New Approach towards Vision Suggested by Biologically Realistic Neural Microcircuit Models
Maass, W., Legenstein, R. A., & Markram, H. (2002) · Lecture notes in computer science · doi:10.1007/3-540-36181-2_28 · Cited by 49
Self-citationCerebellar-driven cortical dynamics can enable task acquisition, switching and consolidation
Pemberton, J., Chadderton, P. T., & Costa, R. P. (2024) · Nature Communications · doi:10.1038/s41467-024-55315-6 · Cited by 47
“All of these studies assumethat connectivity within the RNN itself remains fi xed, thereby avoiding complex learning rules while being able to reuse RNN dynamics for different contexts 19 .”
Du, K.-L., & Swamy, M. N. S. (2013) · Springer eBooks · doi:10.1007/978-1-4471-5571-3_11 · Cited by 51
On the Use of Reservoir Computing in Popularity Prediction
Wu, T., Timmers, M., Vleeschauwer, D. D., & Leekwijck, W. V. (2010) · International Conference on Evolving Internet (INTERNET) · doi:10.1109/internet.2010.13 · Cited by 44
“The accumulative number of views for every monitored video was tracked once each day for each monitored video.”
BackgroundEvolving Hardware with Self-reconfigurable connectivity in Xilinx FPGAs
Upegui, A., & Sánchez, E. G. (2006) · NASA/ESA Conference on Adaptive Hardware and Systems (AHS) · doi:10.1109/ahs.2006.38 · Cited by 44
“[10] present their liquid state machines –randomly connected integrate and fire neurons–, which are able to classify noise-corrupted spoken words.”
BackgroundA mixed-signal universal neuromorphic computing system
Meier, K. (2015) · Technical digest - International Electron Devices Meeting/Technical digest · doi:10.1109/iedm.2015.7409627 · Cited by 44
Hou, Y., Xia, G.-Q., Jayaprasath, E., Yue, D., Yang, W.-Y., & Wu, Z.-M. (2018) · Optics Communications · doi:10.1016/j.optcom.2018.10.014 · Cited by 44
Processing EMG signals using reservoir computing on an event-based neuromorphic system
Donati, E., Payvand, M., Risi, N., Krause, R., Burelo, K., Indiveri, G., Dalgaty, T., & Vianello, E. (2018) · IEEE Biomedical Circuits and Systems Conference (BioCAS) · doi:10.1109/biocas.2018.8584674 · Cited by 44
“Spike trains from Section II-A are given to 30% of the neurons in the liquid [5].”
BackgroundMethodInfluentialLaporte, F., Dambre, J., & Bienstman, P. (2019) · Scientific Reports · doi:10.1038/s41598-019-42408-2 · Cited by 56
Surrogate-Assisted Evolutionary Search of Spiking Neural Architectures in Liquid State Machines
Zhou, Y., Jin, Y., & Ding, J. (2020) · Neurocomputing · doi:10.1016/j.neucom.2020.04.079 · Cited by 39
Photonic reservoir computer based on frequency multiplexing
Butschek, L., Akrout, A., Dimitriadou, E., Lupo, A., Haelterman, M., & Massar, S. (2022) · Optics Letters · doi:10.1364/ol.451087 · Cited by 50
Learning to represent continuous variables in heterogeneous neural networks
Darshan, R., & Rivkind, A. (2022) · Cell Reports · doi:10.1016/j.celrep.2022.110612 · Cited by 54
Polyoxometalate Accelerated Cationic Migration for Reservoir Computing
Zhang, G., Xiong, Z., Gong, Y., Zhu, Z., Lv, Z., Wang, Y., Yang, J., Xing, X., Wang, Z., Qin, J., et al. (2022) · Advanced Functional Materials · doi:10.1002/adfm.202204721 · Cited by 45
Phase Precession Through Synaptic Facilitation
Thurley, K., Leibold, C., Gundlfinger, A., Schmitz, D., & Kempter, R. (2008) · Neural Computation · doi:10.1162/neco.2008.07-06-292 · Cited by 47
“Short-term facilitation is discussed, for example, as a mechanism for temporal integration of presynaptic input (Buonomano & Merzenich, 1995; Maass et al., 2002; Abbott & Regehr, 2004).”
Background
Metadata
- Authors
- Maass, W., Natschläger, T., & Markram, H.
- Affiliations
- Graz University of Technology (AT)
- École Polytechnique Fédérale de Lausanne (CH)
- Corresponding author
- Maass, Wolfgang
- Natschläger, Thomas
- Markram, Henry
- Venue
- Neural Computation
- Publisher
- MIT Press
- Published
- 2002-11-01
- Volume
- 14
- Issue
- 11
- Pages
- 2531-2560
- ISSN
- 0899-7667 (ISSN-L)
- 1530-888X
- OpenAlex
- W2103179919
- Cited by
- 4,025
- References
- 28
- Citations by year
- 2026: 286 · 2025: 351 · 2024: 307 · 2023: 309 · 2022: 276 · 2021: 303 · 2020: 239 · 2019: 244 · 2018: 217 · 2017: 173 · 2016: 155 · 2015: 135 · 2014: 142 · 2013: 139 · 2012: 125
- FWCI
- 18.37
- Type
- journal article
- Language
- en
- Open access
- green
- infoscience.epfl.ch
- Locations
- Neural Computation (publishedVersion)
- PubMed (publishedVersion)
- Infoscience (Ecole Polytechnique Fédérale de Lausanne) (submittedVersion)
- Infoscience (Ecole Polytechnique Fédérale de Lausanne) (submittedVersion, cc-by-nc-nd)
- Topics
- Life Sciences › Neuroscience › Cognitive Neuroscience › Neural dynamics and brain function
- Physical Sciences › Computer Science › Artificial Intelligence › Neural Networks and Reservoir Computing
- Physical Sciences › Computer Science › Artificial Intelligence › Neural Networks and Applications
- Keywords
- liquid state machines, recurrent neural circuits, transient dynamics, high-dimensional dynamical systems, fading memory, integrate-and-fire neurons, real-time processing, time-varying inputs, neural coding, statistical learning theory, computational power, attractor networks
- MeSH terms
- Action Potentials, Computer Simulation, Computer Systems, Computers, Models, Neurological, Neurons, Neural Networks, Computer
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.