Real-Time Computing Without Stable States: A New Framework for Neural Computation Based on Perturbations
Open accessThe DOI matches its Crossref record; 28 of the 28 references it cites were found in a registry or an index.
Abstract
A key challenge for neural modeling is to explain how a continuous stream of multimodal input from a rapidly changing environment can be processed by stereotypical recurrent circuits of integrate-and-fire neurons in real time. We propose a new computational model for real-time computing on time-varying input that provides an alternative to paradigms based on Turing machines or attractor neural networks. It does not require a task-dependent construction of neural circuits. Instead, it is based on principles of high-dimensional dynamical systems in combination with statistical learning theory and can be implemented on generic evolved or found recurrent circuitry. It is shown that the inherent transient dynamics of the high-dimensional dynamical system formed by a sufficiently large and heterogeneous neural circuit may serve as universal analog fading memory. Readout neurons can learn to extract in real time from the current state of such recurrent neural circuit information about current and past inputs that may be needed for diverse tasks. Stable internal states are not required for giving a stable output, since transient internal states can be transformed by readout neurons into stable target outputs due to the high dimensionality of the dynamical system. Our approach is based on a rigorous computational model, the liquid state machine, that, unlike Turing machines, does not require sequential transitions between well-defined discrete internal states. It is supported, as the Turing machine is, by rigorous mathematical results that predict universal computational power under idealized conditions, but for the biologically more realistic scenario of real-time processing of time-varying inputs. Our approach provides new perspectives for the interpretation of neural coding, the design of experiments and data analysis in neurophysiology, and the solution of problems in robotics and neurotechnology.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Although work in artificial intelligence and machine-learning research has shown that similar or even more complex outcomes can be achieved with a variety of techniques, including deep neural networks with trained weights (Gibney, 2015), reservoir networks with a trained output mapping (Maass et al., 2002;Jaeger & Haas, 2004), networks that are evolved using a genetic algorithm (Iizuka & Di Paolo,”
From A potential mechanism for Gibsonian resonance: behavioral entrainment emerges from local homeostasis in an unsupervised reservoir network (2024) · cites this paper as Maass et al., 2002
In the citing paper
“Finally, we show that a randomly connected circuit model, akin to a “reservoir” network13-15, can reproduce the observed distribution of timescales, provided that the network operates at the critical point (or “edge of chaos”)16-18.”
From A reservoir of time constants for memory traces in cortical neurons (2011) · cites this paper as [13]
In the citing paper
“Reservoir-type networks have been the subject of active research in computational neuroscience and machine learning13-15, but so far experimental support that such networks are adopted by the brain has been lacking.”
From A reservoir of time constants for memory traces in cortical neurons (2011) · cites this paper as [13]
In the citing paper
“Those models predict that the memory of input signals is stored in a large, recurrent and heterogeneous network (reservoir), in a distributed manner, and that a desired output is obtained by a trainable combination of the response signals in the reservoir.”
From A reservoir of time constants for memory traces in cortical neurons (2011) · cites this paper as [13]
Citations
Cites
Wu, Y. (1999) · Technometrics · doi:10.1080/00401706.1999.10485951 · Cited by 27,221
Introduction to the theory of neural computation
(1994) · Neural Networks · doi:10.1016/0893-6080(94)90014-0 · Cited by 6,401
The Synaptic Organization of the Brain
(2004) · Oxford University Press eBooks · doi:10.1093/acprof:oso/9780195159561.001.1 · Cited by 3,546 · cited 2×
“This informationisprocessed by extremely complexbut surprisingly stereotypic microcircuits that can perform a wide spectrum of tasks (Shep-herd, 1988 ; Douglas & Martin, 1998 ; von Melchner, Pallas, & Sur, 2000).”
Dynamics of ongoing activity: explanation of the large variability in evoked cortical responses
Arieli, A., Sterkin, A., Grinvald, A., & Aertsen, A. (1996) · Science · doi:10.1126/science.273.5283.1868 · Cited by 1,660 · cited 1×
“Furthermore, although the input rate may contain salient information that is constant for a particular readout element, it may not be for another (see, for example, Figure 8), indicating that equivalence classes and dynamic stability exist purely from the perspective of the readout elements.”
BackgroundDifferential signaling via the same axon of neocortical pyramidal neurons
Markram, H., Wang, Y., & Tsodyks, M. (1998) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.95.9.5323 · Cited by 1,102 · cited 2×
“In the case of a synaptic connection from a to b , we modeled the synaptic dynamics according to the model proposed in Markram, Wang, and Tsodyks (1998), with the synaptic parameters U (use), D (time constant for depression), and F (time constant for facilitation) randomly chosen from gaussian…”
BackgroundMethodSelf-citationOrganizing principles for a diversity of GABAergic interneurons and synapses in the neocortex
Gupta, A., Wang, Y., & Markram, H. (2000) · Science · doi:10.1126/science.287.5451.273 · Cited by 1,036
Self-citationFading memory and the problem of approximating nonlinear operators with Volterra series
Boyd, S., & Chua, L. (1985) · IEEE Transactions on Circuits and Systems · doi:10.1109/tcs.1985.1085649 · Cited by 991 · cited 2×
“> The proof of this theorem follows from the Stone-Weierstrass approximation theorem, like the proof of theorem 1 in Boyd and Chua (1985).”
BackgroundGradient calculations for dynamic recurrent neural networks: a survey
Pearlmutter, B. A. (1995) · IEEE Transactions on Neural Networks · doi:10.1109/72.410363 · Cited by 582 · cited 1×
“Methods for controllingthe dynamics of recurrent neural networks through adaptive mechanisms are reviewed in Pearlmutter (1995).”
Temporal Information Transformed into a Spatial Code by a Neural Network with Realistic Properties
Buonomano, D. V., & Merzenich, M. M. (1995) · Science · doi:10.1126/science.7863330 · Cited by 449 · cited 1×
“Special cases of this mechanism have already been reported in Buonomano and Merzenich (1995) and Dominey, Arbib, and Joseph (1995).”
BackgroundLearning and Extracting Finite State Automata with Second-Order Recurrent Neural Networks
Giles, C. L., Miller, C. B., Chen, D., Chen, H. H., Sun, G. Z., & Lee, Y. C. (1992) · Neural Computation · doi:10.1162/neco.1992.4.3.393 · Cited by 392
Cited by
Items 1,001–1,014 of 1,014
Underwater Acoustic Communication Channel Modeling Using Reservoir Computing
Onasami, O., Feng, M., Xu, H., Haile, M., & Qian, L. (2022) · IEEE Access · doi:10.1109/access.2022.3177728 · Cited by 32
A Novel Echo State Network Autoencoder for Anomaly Detection in Industrial IoT Systems
Vita, F. D., Nocera, G., Bruneo, D., & Das, S. K. (2022) · IEEE Transactions on Industrial Informatics · doi:10.1109/tii.2022.3224981 · Cited by 40
“However, in order to properly work, the reservoir must satisfy the separation and echo state properties [7], [23] (hence,”
BackgroundLeave No Context Behind: Efficient Infinite Context Transformers with Infini-attention
Munkhdalai, T., Faruqui, M., & Gopal, S. (2024) · arXiv · doi:10.48550/arxiv.2404.07143 · Cited by 23
“We compute the standard causal dot-product attention context within each segment.”
MethodLow-cost hardware implementation of Reservoir Computers
Alomar, M. L., Canals, V., Martinez-Moll, V., & Rossello, J. L. (2014) · 2014 24th International Workshop on Power and Timing Modeling, Optimization and Simulation (PATMOS) · doi:10.1109/patmos.2014.6951899 · Cited by 20
“Reservoir Computing (RC) [1-5] is a recently introduced paradigm of understanding and training Recurrent Neural Networks (RNN).”
BackgroundFalandays, J. B., Yoshimi, J., Warren, W. H., & Spivey, M. J. (2024) · Cognitive Neurodynamics · doi:10.1007/s11571-023-09988-2 · Cited by 19
“This is the general intuition behind both liquid state machines (Maass et al., 2002) and echo state networks (Jaeger & Haas, 2004), introduced independently in the early 2000s, which are now grouped in the general class of algorithms called ‘‘reservoir computers.’’”
“How do humans make sense?” multiscale dynamics and emergent meaning
Dale, R., & Kello, C. T. (2018) · New Ideas in Psychology · doi:10.1016/j.newideapsych.2017.09.002 · Cited by 15
Chorley, P., & Seth, A. K. (2011) · Frontiers in Computational Neuroscience · doi:10.3389/fncom.2011.00021 · Cited by 16
“As compared to synfire chains (Abeles, 1982), polychrony emphasizes time-locked but synchronous activity, and unlike liquid state machines (Maass et al., 2002) polychronous groups exhibit sensitivity to previous inputs.”
BackgroundLearning reservoir dynamics with temporal self-modulation
Sakemi, Y., Nobukawa, S., Matsuki, T., Morie, T., & Aihara, K. (2024) · Communications Physics · doi:10.1038/s42005-023-01500-w · Cited by 16
H-Mem: Harnessing synaptic plasticity with Hebbian Memory Networks
Limbacher, T., & Legenstein, R. (2020) · doi:10.1101/2020.07.01.180372 · Cited by 10
“While models based on persistent activity do not engage any synaptic plasticity mechanism [15], activity-silent models and some dynamic coding models exploit short-term synaptic plasticity or neuronal adaptation [16]–[18].”
BackgroundHopping-Transport Mechanism for Reconfigurable Logic in Disordered Dopant Networks
Tertilt, H., Bakker, J., Becker, M., de Wilde, B., Klanberg, I., Geurts, B. J., van der Wiel, W. G., Heuer, A., & Bobbert, P. A. (2022) · Physical Review Applied · doi:10.1103/physrevapplied.17.064025 · Cited by 8
(2011) · eScholarship (California Digital Library) · Cited by 8
Wang, G., Li, J., Sun, Y., Chen, X., Liu, C., Wu, Y., Lu, M., Song, S., & Yadkori, Y. A. (2025) · arXiv · doi:10.48550/arxiv.2506.21734 · Cited by 3
“In dynamical systems, this flexibility is often realized through higher-dimensional state-space trajectories, which allow for a richer repertoire of potential computations 78 .”
Dopant network processing units as tuneable extreme learning machines
van de Ven, B., Alegre-Ibarra, U., Lemieszczuk, P. J., Bobbert, P. A., Ruiz Euler, H.-C., & van der Wiel, W. G. (2023) · Frontiers in Nanotechnology · doi:10.3389/fnano.2023.1055527 · Cited by 2
“Independently developed RC schemes are echo state networks (ESNs) (Jaeger, 2010), liquid state machines (LSMs) (Maass et al., 2002), and the backpropagation-decorrelation (BPDC) on-line learning rule (Steil, 2004).”
Background2022 roadmap on neuromorphic computing and engineering
Christensen, D. V., Dittmann, R., Linares-Barranco, B., Sebastian, A., Le Gallo, M., Redaelli, A., Slesazeck, S., Mikolajick, T., Spiga, S., Menzel, S., et al. (2022) · Neuromorphic Computing and Engineering · doi:10.1088/2634-4386/ac4a83
Self-citation
Metadata
- Authors
- Maass, W., Natschläger, T., & Markram, H.
- Affiliations
- Graz University of Technology (AT)
- École Polytechnique Fédérale de Lausanne (CH)
- Corresponding author
- Maass, Wolfgang
- Natschläger, Thomas
- Markram, Henry
- Venue
- Neural Computation
- Publisher
- MIT Press
- Published
- 2002-11-01
- Volume
- 14
- Issue
- 11
- Pages
- 2531-2560
- ISSN
- 0899-7667 (ISSN-L)
- 1530-888X
- OpenAlex
- W2103179919
- Cited by
- 4,025
- References
- 28
- Citations by year
- 2026: 286 · 2025: 351 · 2024: 307 · 2023: 309 · 2022: 276 · 2021: 303 · 2020: 239 · 2019: 244 · 2018: 217 · 2017: 173 · 2016: 155 · 2015: 135 · 2014: 142 · 2013: 139 · 2012: 125
- FWCI
- 18.37
- Type
- journal article
- Language
- en
- Open access
- green
- infoscience.epfl.ch
- Locations
- Neural Computation (publishedVersion)
- PubMed (publishedVersion)
- Infoscience (Ecole Polytechnique Fédérale de Lausanne) (submittedVersion)
- Infoscience (Ecole Polytechnique Fédérale de Lausanne) (submittedVersion, cc-by-nc-nd)
- Topics
- Life Sciences › Neuroscience › Cognitive Neuroscience › Neural dynamics and brain function
- Physical Sciences › Computer Science › Artificial Intelligence › Neural Networks and Reservoir Computing
- Physical Sciences › Computer Science › Artificial Intelligence › Neural Networks and Applications
- Keywords
- liquid state machines, recurrent neural circuits, transient dynamics, high-dimensional dynamical systems, fading memory, integrate-and-fire neurons, real-time processing, time-varying inputs, neural coding, statistical learning theory, computational power, attractor networks
- MeSH terms
- Action Potentials, Computer Simulation, Computer Systems, Computers, Models, Neurological, Neurons, Neural Networks, Computer
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.