Harnessing Nonlinearity: Predicting Chaotic Systems and Saving Energy in Wireless Communication
The DOI matches its Crossref record; 18 of the 18 references it cites were found in a registry or an index.
Abstract
We present a method for learning nonlinear systems, echo state networks (ESNs). ESNs employ artificial recurrent neural networks in a way that has recently been proposed independently as a learning mechanism in biological brains. The learning method is computationally efficient and easy to use. On a benchmark task of predicting a chaotic time series, accuracy is improved by a factor of 2400 over previous techniques. The potential for engineering applications is illustrated by equalizing a communication channel, where the signal error rate is improved by two orders of magnitude.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Although work in artificial intelligence and machine-learning research has shown that similar or even more complex outcomes can be achieved with a variety of techniques, including deep neural networks with trained weights (Gibney, 2015), reservoir networks with a trained output mapping (Maass et al., 2002;Jaeger & Haas, 2004), networks that are evolved using a genetic algorithm (Iizuka & Di Paolo,”
From A potential mechanism for Gibsonian resonance: behavioral entrainment emerges from local homeostasis in an unsupervised reservoir network (2024) · cites this paper as Jaeger & Haas, 2004
In the citing paper
“Therefore, a sequence production solely by a recurrent neural network can be unstable against small fluctuations in activities and unwanted noises (Jaeger and Haas, 2004).”
From The Cerebro-Cerebellum as a Locus of Forward Model: A Review (2020) · cites this paper as Jaeger and Haas, 2004
Citations
Cites
Backpropagation through time: what it does and how to do it
Werbos, P. J. (1990) · Proceedings of the IEEE · doi:10.1109/5.58337 · Cited by 4,879
A Learning Algorithm for Continually Running Fully Recurrent Neural Networks
Williams, R. J., & Zipser, D. (1989) · Neural Computation · doi:10.1162/neco.1989.1.2.270 · Cited by 4,369
Oscillation and Chaos in Physiological Control Systems
Mackey, M. C., & Glass, L. (1977) · Science · doi:10.1126/science.267326 · Cited by 4,135
Maass, W., Natschläger, T., & Markram, H. (2002) · Neural Computation · doi:10.1162/089976602760407955 · Cited by 4,025
Real-time prediction of hand trajectory by ensembles of cortical neurons in primates
Wessberg, J., Stambaugh, C. R., Kralik, J. D., Beck, P. D., Laubach, M., Chapin, J. K., Kim, J., Biggs, S. J., Srinivasan, M. A., & Nicolelis, M. A. L. (2000) · Nature · doi:10.1038/35042582 · Cited by 1,487
'Neural-gas' network for vector quantization and its application to time-series prediction
Martinetz, T. M., Berkovich, S. G., & Schulten, K. J. (1993) · IEEE Transactions on Neural Networks · doi:10.1109/72.238311 · Cited by 1,431
Approximation of dynamical systems by continuous time recurrent neural networks
Funahashi, K.-I., & Nakamura, Y. (1993) · Neural Networks · doi:10.1016/s0893-6080(05)80125-x · Cited by 980
A new evolutionary system for evolving artificial neural networks
Yao, X., & Liu, Y. (1997) · IEEE Transactions on Neural Networks · doi:10.1109/72.572107 · Cited by 874
Applying LSTM to Time Series Predictable through Time-Window Approaches
Gers, F. A., Eck, D., & Schmidhuber, J. (2001) · Lecture Notes in Computer Science · doi:10.1007/3-540-44668-0_93 · Cited by 334
Computational neuroscience: a comprehensive approach
(2004) · Choice Reviews Online · doi:10.5860/choice.41-5284 · Cited by 279
Research works printed but not linked: 10
- K. Doya, in The Handbook of Brain Theory and Neural Networks, M. A. Arbib, Ed. (MIT Press, Cambridge, MA, 1995), pp. 796–800.
- J. Vesanto in Proc. WSOM'97 (1997); www.cis.hut.fi/projects/monitor/publications/papers/wsom97.ps.
- L. Chudy, I. Farkas, Neural Network World8, 481 (1998).
- H. Bersini M. Birattari G. Bontempi in Proc. IEEE World Congr. on Computational Intelligence (IJCNN'98) (1997) pp. 2102–2106; ftp://iridia.ulb.ac.be/pub/lazy/papers/IridiaTr1997-13_2.ps.gz.
- V. J. Mathews, J. Lee, in Advanced Signal Processing: Algorithms, Architectures, and Implementations V (Proc. SPIE Vol. 2296), (SPIE, San Diego, CA, 1994), pp. 317–327.
- J. Hertzberg H. Jaeger F. Schönherr in Proc. 15th Europ. Conf. on Art. Int. (ECAI 02) F. van Harmelen Ed. (IOS Press Amsterdam 2002) pp. 708–712; www.ais.fhg.de/∼schoenhe/papers/ECAI02.pdf.
- H. Jaeger, in Advances in Neural Information Processing Systems 15, S. Becker, S. Thrun, K. Obermayer, Eds. (MIT Press, Cambridge, MA, 2003) pp. 593–600.
- G. E. Hinton, in Parallel Models of Associative Memory, G. E. Hinton, J. A. Anderson, Eds. (Erlbaum, Hillsdale, NJ, 1981), pp. 161–187.
- M. Kawato, in The Handbook of Brain Theory and Neural Networks, M. Arbib, Ed. (MIT Press, Cambridge, MA, 1995), pp. 172–178.
- G. B. Stanley, Neurocomputing38–40, 1703 (2001).
Cited by
Items 101–200 of 1,009
A Fast Machine Learning Model for ECG-Based Heartbeat Classification and Arrhythmia Detection
Alfaras, M., Soriano, M. C., & Ortín, S. (2019) · Frontiers in Physics · doi:10.3389/fphy.2019.00103 · Cited by 265
“In this work, we propose an ensemble of Echo State Networks (ESNs) [27] as the classifier method, using the raw ECG waveforms and time intervals between the heartbeats as the input features.”
BackgroundMethodDesign and control of compliant tensegrity robots through simulation and hardware validation
Caluwaerts, K., Despraz, J., Işçen, A., Sabelhaus, A. P., Bruce, J., Schrauwen, B., & SunSpiral, V. (2014) · Journal of The Royal Society Interface · doi:10.1098/rsif.2014.0520 · Cited by 267
“technique to train recurrent neural networks [40, 41].”
Gao, Y., Ruan, Y., Fang, C., & Yin, S. (2020) · Energy and Buildings · doi:10.1016/j.enbuild.2020.110156 · Cited by 263
Recent progress in semiconductor excitable lasers for photonic spike processing
Prucnal, P. R., Shastri, B. J., Ferreira de Lima, T., Nahmias, M. A., & Tait, A. N. (2016) · Advances in Optics and Photonics · doi:10.1364/aop.8.000228 · Cited by 261
Design of deep echo state networks
Gallicchio, C., Micheli, A., & Pedrelli, L. (2018) · Neural Networks · doi:10.1016/j.neunet.2018.08.002 · Cited by 262
High-performance photonic reservoir computer based on a coherently driven passive cavity
Vinckier, Q., Duport, F., Smerieri, A., Vandoorne, K., Bienstman, P., Haelterman, M., & Massar, S. (2015) · Optica · doi:10.1364/optica.2.000438 · Cited by 265
“The simplicity and the flexibility of such systems make them powerful to solve a very large range of tasks, such as speech recognition [4], nonlinear channel equalization [3,5–7], detection of epileptic seizures [8], robot control [9], time series prediction [1,3,7,10], financial forecasting, and handwriting recogniti…”
BackgroundMethodHow critical is brain criticality?
O’Byrne, J., & Jerbi, K. (2022) · Trends in Neurosciences · doi:10.1016/j.tins.2022.08.007 · Cited by 258
Support Vector Echo-State Machine for Chaotic Time-Series Prediction
Shi, Z., & Han, M. (2007) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.2006.885113 · Cited by 258
“ESN iterative model, recently reported in [15], is a kind of RNN which is firstly trained as an accurate one-step predictor, and then iterates the prediction into different prediction horizons in an autonomous style.”
BackgroundMethodResultInfluentialGanguli, S., & Sompolinsky, H. (2012) · Annual Review of Neuroscience · doi:10.1146/annurev-neuro-062111-150410 · Cited by 256
“More recent proposals ( Jaeger 2001, Maass et al. 2002, Jaeger & Haas 2004) suggest that recurrent networks could store temporal sequences of inputs in their ongoing, transient activity.”
BackgroundMalware classification with LSTM and GRU language models and a character-level CNN
Athiwaratkun, B., & Stokes, J. W. (2017) · 2017 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) · doi:10.1109/icassp.2017.7952603 · Cited by 239
HiPPO: Recurrent Memory with Optimal Polynomial ProjectionsPreprint
Gu, A., Dao, T., Ermon, S., Rudra, A., & Ré, C. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2008.07669 · Cited by 628
“The deep learning instantiation of this approach is the recurrent neural network (RNN), which is known to suffer from a limited memory horizon [34, 38, 56] (e.”
BackgroundTraining Recurrent Networks by Evolino
Schmidhuber, J., Wierstra, D., Gagliolo, M., & Gomez, F. (2007) · Neural Computation · doi:10.1162/neco.2007.19.3.757 · Cited by 248
“9 × 10 compared to10 for ESNs with 1000 neurons [15].”
BackgroundMethodInfluentialNeural circuits as computational dynamical systems
Sussillo, D. (2014) · Current Opinion in Neurobiology · doi:10.1016/j.conb.2014.01.008 · Cited by 247
“The output weights are shown in red, and are the only units trained in this example [35].”
Continuous Online Sequence Learning with an Unsupervised Neural Network Model
Cui, Y., Ahmad, S., & Hawkins, J. (2016) · Neural Computation · doi:10.1162/neco_a_00893 · Cited by 239
“Echo state network (ESN) uses a randomly connected recurrent network as a dynamics reservoir and models a sequence as a trainable linear combination of these response signals (Jaeger & Haas, 2004).”
BackgroundPhysical reservoir computing with emerging electronics
Liang, X., Tang, J., Zhong, Y., Gao, B., Qian, H., & Wu, H. (2024) · Nature Electronics · doi:10.1038/s41928-024-01133-z · Cited by 226
“First, the reservoir size is the prime feature that determines the degree of freedom of the dynamics in the reservoir layer that affects the MC and approximation capacity 16,27,64,127 .”
Thakkar, A., & Chaudhari, K. (2021) · Expert Systems with Applications · doi:10.1016/j.eswa.2021.114800 · Cited by 232
Laser dynamical reservoir computing with consistency: an approach of a chaos mask signal
Nakayama, J., Kanno, K., & Uchida, A. (2016) · Optics Express · doi:10.1364/oe.24.008679 · Cited by 244
Computational Aspects of Feedback in Neural Circuits
Maass, W., Joshi, P., & Sontag, E. D. (2007) · PLoS Computational Biology · doi:10.1371/journal.pcbi.0020165 · Cited by 242
History-dependent variability in population dynamics during evidence accumulation in cortex
Morcos, A. S., & Harvey, C. D. (2016) · Nature Neuroscience · doi:10.1038/nn.4403 · Cited by 247
“This new model is consistent with the theoretical framework developed in reservoir computing 45,48,49 .”
Reservoir Computing Beyond Memory-Nonlinearity Trade-off
Inubushi, M., & Yoshimura, K. (2017) · Scientific Reports · doi:10.1038/s41598-017-10257-6 · Cited by 239
“This framework is referred to as reservoir computing ( RC ) 5 which was proposed originally in the research fields of machine learning 6 (called Echo State Network) and computational neuroscience 7 (called Liquid State Machine).”
BackgroundRecurrent neural networks as versatile tools of neuroscience research
Barak, O. (2017) · Current Opinion in Neurobiology · doi:10.1016/j.conb.2017.06.003 · Cited by 244
Neuromorphic Photonic Integrated Circuits
Peng, H.-T., Nahmias, M. A., de Lima, T. F., Tait, A. N., & Shastri, B. J. (2018) · IEEE Journal of Selected Topics in Quantum Electronics · doi:10.1109/jstqe.2018.2840448 · Cited by 232
“Reservoir computing (RC) [118] is another approach to tun-”
BackgroundLearning by the Dendritic Prediction of Somatic Spiking
Urbanczik, R., & Senn, W. (2014) · Neuron · doi:10.1016/j.neuron.2013.11.030 · Cited by 230
“Similarly to echo state networks (Jaeger and Haas, 2004), such ‘‘hidden’’ neurons may enhance the representational capabilities of the network.”
Improving reservoirs using intrinsic plasticity
Schrauwen, B., Wardermann, M., Verstraeten, D., Steil, J. J., & Stroobandt, D. (2008) · Neurocomputing · doi:10.1016/j.neucom.2007.12.020 · Cited by 227
Reservoir Computing With Spin Waves Excited in a Garnet Film
Nakane, R., Tanaka, G., & Hirose, A. (2018) · IEEE Access · doi:10.1109/access.2018.2794584 · Cited by 231
“One of the most widely used frameworks in this area is the neural networks including reservoir computing [9]–[12].”
BackgroundLaje, R., & Buonomano, D. V. (2012) · arXiv (Cornell University) · doi:10.48550/arxiv.1210.2104 · Cited by 200
“Indeed, a number of related theoretical frameworks (sometimes grouped under the term reservoir computing ) are to a large extent based on the notion that randomly connected recurrent networks (RRNs) can generate patterns of activity in high-dimensional space, and these patterns or trajectories can in turn be used for…”
BackgroundMethodInfluentialParallel Reservoir Computing Using Optical Amplifiers
Vandoorne, K., Dambre, J., Verstraeten, D., Schrauwen, B., & Bienstman, P. (2011) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.2011.2161771 · Cited by 224
“Index Terms— Integrated optics, optical neural networks, photonic reservoir computing, semiconductor optical amplifiers, speech recognition.”
BackgroundManjunath, G., & Jaeger, H. (2013) · Neural Computation · doi:10.1162/neco_a_00411 · Cited by 222
“(For a short overview, see Jaeger, 2007; an application-oriented paper is Jaeger & Haas, 2004; and a survey on state of the art is Lukoševičius & Jaeger, 2009.)”
BackgroundSelf-citationHramov, A. E., Maksimenko, V. A., & Pisarchik, A. N. (2021) · Physics Reports · doi:10.1016/j.physrep.2021.03.002 · Cited by 213
Automatic speech recognition using a predictive echo state network classifier
Skowronski, M. D., & Harris, J. G. (2007) · Neural Networks · doi:10.1016/j.neunet.2007.04.006 · Cited by 218
Yang, Z., Moczulski, M., Denil, M., De Freitas, N., Song, L., & Wang, Z. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.173 · Cited by 215
“The intriguing idea of constructing neural networks with random weights has been reasonably explored in the neural networks field [34, 19].”
BackgroundNovel frontier of photonics for data processing—Photonic accelerator
Kitayama, K.-I., Notomi, M., Naruse, M., Inoue, K., Kawakami, S., & Uchida, A. (2019) · APL Photonics · doi:10.1063/1.5108912 · Cited by 216
“34,35 The training process of the connection weights is simplified in reservoir computing, and only the output weights are trained by learning.”
BackgroundChattopadhyay, A., Hassanzadeh, P., & Subramanian, D. (2020) · Nonlinear Processes in Geophysics · doi:10.5194/npg-27-373-2020 · Cited by 169
Deep learning for air pollutant concentration prediction: A review
Zhang, B., Rong, Y., Yong, R., Qin, D., Li, M., Zou, G., & Pan, J. (2022) · Atmospheric Environment · doi:10.1016/j.atmosenv.2022.119347 · Cited by 203
Computational principles of memory
Chaudhuri, R., & Fiete, I. (2016) · Nature Neuroscience · doi:10.1038/nn.4237 · Cited by 213
“These slowly decaying traces can be used to store inputs and decode them from the instantaneous output of the system 91–93 .”
BackgroundGallicchio, C., & Micheli, A. (2010) · The 2010 International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn.2010.5596796 · Cited by 210
“Reservoir Computing (RC) [1] models, and in particular Echo State Networks (ESNs) [2], [3], are Recurrent Neural Networks (RNNs) characterized by a conceptual separation between a large recurrent non-linear layer of hidden units, the reservoir, and a feed-forward (typically linear) layer of output units, the readout.”
BackgroundHands-on reservoir computing: a tutorial for practical implementation
Cucchi, M., Abreu, S., Ciccone, G., Brunner, D., & Kleemann, H. (2022) · Neuromorphic Computing and Engineering · doi:10.1088/2634-4386/ac7db7 · Cited by 199
“Reservoirs have been particularly successful in learning and predicting the dynamics of chaotic systems [34, 35], so RC is a competitive approach for tasks that deal with chaotic signals.”
BackgroundTransition to Chaos in Random Neuronal Networks
Kadmon, J., & Sompolinsky, H. (2015) · Physical Review X · doi:10.1103/physrevx.5.041030 · Cited by 283
Insensitivity of the Human Sentence-Processing System to Hierarchical Structure
Frank, S. L., & Bod, R. (2011) · Psychological Science · doi:10.1177/0956797611409589 · Cited by 217
Liquid State Machines: Motivation, Theory, and Applications
Maass, W. (2011) · Computability in Context · doi:10.1142/9781848162778_0008 · Cited by 201
“The exploration of potential engineering applications of the computational paradigm discussed in this article is simultaneously also carried out for the closely related echo state networks (ESNs) [13], where one uses simpler non-spiking models for neurons in the “Liquid”, and works with high numerical precision in the…”
BackgroundMethodResultInfluentialTraining Deep and Recurrent Networks with Hessian-Free Optimization
Martens, J., & Sutskever, I. (2012) · Lecture Notes in Computer Science · doi:10.1007/978-3-642-35289-8_27 · Cited by 202
“It is known [19, 23] that good random initializations give rise to nontrivial hidden state dynamics that can carry useful information about the”
BackgroundScalable reservoir computing on coherent linear photonic processor
Nakajima, M., Tanaka, K., & Hashimoto, T. (2021) · Communications Physics · doi:10.1038/s42005-021-00519-1 · Cited by 203
Fully analogue photonic reservoir computer
Duport, F., Smerieri, A., Akrout, A., Haelterman, M., & Massar, S. (2016) · Scientific Reports · doi:10.1038/srep22381 · Cited by 206
“…reservoir computers to solve a large range of complex tasks on time dependent signals, such as speech recognition 6 , nonlinear channel equalization 3,7,8 , detection of epileptic seizures 9 , robot control 10 , time series prediction 1,3 , financial forecasting, handwriting recognition, etc … ,…”
BackgroundShort-term stock price prediction based on echo state networks
Lin, X., Yang, Z., & Song, Y. (2009) · Expert Systems with Applications · doi:10.1016/j.eswa.2008.09.049 · Cited by 203
“Echo state network (ESN) is a novel RNN recently proposed by Jaeger and Haas (2004).”
BackgroundMethodInfluentialDistributed Fading Memory for Stimulus Properties in the Primary Visual Cortex
Nikolić, D., Häusler, S., Singer, W., & Maass, W. (2009) · PLoS Biology · doi:10.1371/journal.pbio.1000260 · Cited by 202
“Computer simulations of sparsely connected recurrent circuits of neuron-like processing units [30–33] as well as theoretical analyses of related circuits with simpler (linear) processing units [34,35] have shown that systems with fading memory exhibit powerful computational capabilities because they permit integration…”
BackgroundPhotonic Nonlinear Transient Computing with Multiple-Delay Wavelength Dynamics
Martinenghi, R., Rybalko, S., Jacquot, M., Chembo, Y. K., & Larger, L. (2012) · Physical Review Letters · doi:10.1103/physrevlett.108.244101 · Cited by 205
Optoelectronic oscillators with time-delayed feedback
Chembo, Y. K., Brunner, D., Jacquot, M., & Larger, L. (2019) · Reviews of Modern Physics · doi:10.1103/revmodphys.91.035006 · Cited by 199
Is Extreme Learning Machine Feasible? A Theoretical Assessment (Part I)
Liu, X., Lin, S., Fang, J., & Xu, Z. (2015) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2014.2335212 · Cited by 198
“The similar idea has been adopted earlier in [25]–[27] as the echo state network method and”
MethodArchitectural and Markovian factors of echo state networks
Gallicchio, C., & Micheli, A. (2011) · Neural Networks · doi:10.1016/j.neunet.2011.02.002 · Cited by 198
Building functional networks of spiking model neurons
Abbott, L. F., DePasquale, B., & Memmesheimer, R.-M. (2016) · Nature Neuroscience · doi:10.1038/nn.4241 · Cited by 204
“This approach can generate good results in rate-based networks 43,44,45,46 , and it has been tried in spiking networks 7 where it only works in limited cases and, in general, poorly (Figure 2B).”
BackgroundA Flexible Model of Working Memory
Bouchacourt, F., & Buschman, T. J. (2019) · Neuron · doi:10.1016/j.neuron.2019.04.020 · Cited by 201
Bueno, J., Brunner, D., Soriano, M. C., & Fischer, I. (2017) · Optics Express · doi:10.1364/oe.25.002401 · Cited by 191
“The introduction of the reservoir computing principle has served as a disruptive step for this field [1].”
BackgroundInfluentialGrowing Echo-State Network With Multiple Subreservoirs
Qiao, J., Li, F., Han, H., & Li, W. (2017) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2016.2514275 · Cited by 192
“has emerged in the last decade [7]–[9].”
BackgroundResultInfluentialPhotonic machine learning implementation for signal recovery in optical communications
Argyris, A., Bueno, J., & Fischer, I. (2018) · Scientific Reports · doi:10.1038/s41598-018-26927-y · Cited by 186
“The originally proposed RC concept 11,12 for data processing consists of three layers: the input, the reservoir and the output layer.”
All-optical reservoir computer based on saturation of absorption
Dejonckheere, A., Duport, F., Smerieri, A., Fang, L., Oudar, J.-L., Haelterman, M., & Massar, S. (2014) · Optics Express · doi:10.1364/oe.22.010868 · Cited by 189
What Is Morphological Computation? On How the Body Contributes to Cognition and Control
Müller, V. C., & Hoffmann, M. (2017) · Artificial Life · doi:10.1162/artl_a_00219 · Cited by 186
Prospects and applications of photonic neural networks
Huang, C., Sorger, V. J., Miscuglio, M., Al-Qadasi, M., Mukherjee, A., Lampe, L., Nichols, M., Tait, A. N., Ferreira de Lima, T., Marquez, B. A., et al. (2022) · Advances in Physics: X · doi:10.1080/23746149.2021.1981155 · Cited by 180
Decoupled echo state networks with lateral inhibition
Xue, Y., Yang, L., & Haykin, S. (2007) · Neural Networks · doi:10.1016/j.neunet.2007.04.014 · Cited by 186
Optoelectronic reservoir computing: tackling noise-induced performance degradation
Soriano, M. C., Ortín, S., Brunner, D., Larger, L., Mirasso, C. R., Fischer, I., & Pesquera, L. (2013) · Optics Express · doi:10.1364/oe.21.000012 · Cited by 187
Liu, D., Wang, H., Peng, Y., Xie, W., & Liao, H. (2013) · Energies · doi:10.3390/en6083654 · Cited by 186
Ortín, S., Soriano, M. C., Pesquera, L., Brunner, D., San-Martín, D., Fischer, I., Mirasso, C. R., & Gutiérrez, J. M. (2015) · Scientific Reports · doi:10.1038/srep14945 · Cited by 191
“This task has been previously used in the ESN community 4,8 .”
MethodReservoir Computing Properties of Neural Dynamics in Prefrontal Cortex
Enel, P., Procyk, E., Quilodran, R., & Dominey, P. F. (2016) · PLOS Computational Biology · doi:10.1371/journal.pcbi.1004967 · Cited by 187
“This reservoir computing paradigm was further developed independently by two teams in computational neuroscience [9] and machine learning [10, 11].”
MethodHao, Y., Huang, X., Dong, M., & Xu, B. (2020) · Neural Networks · doi:10.1016/j.neunet.2019.09.007 · Cited by 187
“The LSM is a classic model of reservoir computing (RC) [67, 68, 69, 70, 64], which”
MethodRigotti, M. (2010) · Frontiers in Computational Neuroscience · doi:10.3389/fncom.2010.00024 · Cited by 184
“Generalizing to different timings is a problem for alternative approaches that encode all the detailed time information (Maass et al., 2002, 2007; Jaeger and Haas, 2004) or for feed-forward models of working memory (Goldman, 2009).”
BackgroundMethodBollt, E. (2021) · Chaos: An Interdisciplinary Journal of Nonlinear Science · doi:10.1063/5.0024890 · Cited by 179
“ion, and especially the long short term memory (LSTM) approach demonstrate excellent delity, [82,35,81,20,18,86], but these are especially expensive to fully train, [63]. The reservoir computing (RC) [38,52,80] and the closely related echo state network (ESN) [37,51] and liquid state machine (LSM) [54,31] have emerged…”
BackgroundLaser chimeras as a paradigm for multistable patterns in complex systems
Larger, L., Penkovsky, B., & Maistrenko, Y. (2015) · Nature Communications · doi:10.1038/ncomms8752 · Cited by 187
A recurrent neural network for closed-loop intracortical brain–machine interface decoders
Sussillo, D., Nuyujukian, P., Fan, J. M., Kao, J. C., Stavisky, S. D., Ryu, S., & Shenoy, K. (2012) · Journal of Neural Engineering · doi:10.1088/1741-2560/9/2/026027 · Cited by 186
“Specifically, Jaeger and colleagues proposed the echostate network (ESN), defined as a randomly initialized RNN, where synaptic changes are limited to the output units of the network [31].”
BackgroundPopulation clocks: motor timing with neural dynamics
Buonomano, D. V., & Laje, R. (2010) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2010.09.002 · Cited by 182
Financial time series prediction using a dendritic neuron model
Zhou, T., Gao, S., Wang, J., Chu, C., Todo, Y., & Tang, Z. (2016) · Knowledge-Based Systems · doi:10.1016/j.knosys.2016.05.031 · Cited by 183
Long-term prediction of chaotic systems with machine learning
Fan, H., Jiang, J., Zhang, C., Wang, X., & Lai, Y.-C. (2020) · Physical Review Research · doi:10.1103/physrevresearch.2.012080 · Cited by 178
A soft body as a reservoir: case studies in a dynamic model of octopus-inspired soft robotic arm
Nakajima, K., Hauser, H., Kang, R., Guglielmino, E., Caldwell, D. G., & Pfeifer, R. (2013) · Frontiers in Computational Neuroscience · doi:10.3389/fncom.2013.00091 · Cited by 180
“However, the procedures differ in two points; first, during the training phase, we clamp the feedbacks from the system outputs, and provide the target outputs as inputs, which means, in Equation (25), we set In1(t) = x1(t), In2(t) = x2(t), where x1(t) and x2(t) are the target outputs at timestep t.”
BackgroundMethodInfluentialAn effective multiple-image encryption algorithm based on 3D cube and hyperchaotic map
Gao, X., Mou, J., Banerjee, S., Cao, Y., Xiong, L., & Chen, X. (2022) · Journal of King Saud University - Computer and Information Sciences · doi:10.1016/j.jksuci.2022.01.017 · Cited by 175
“It was widely studied as soon as it was discovered, and after decades of development, chaos theory was gradually refined (Gan et al., 2018; Kadhim et al., 2018; Bao et al., 2019; Herbert and Harald, 2004; Aihara et al., 2009; Pecora and Carroll, 1991; Wang et al., 2020b).”
Spike processing with a graphene excitable laser
Shastri, B. J., Nahmias, M. A., Tait, A. N., Rodriguez, A. W., Wu, B., & Prucnal, P. R. (2016) · Scientific Reports · doi:10.1038/srep19126 · Cited by 180
Reservoir computing with a single delay-coupled non-linear mechanical oscillator
Dion, G., Mejaouri, S., & Sylvestre, J. (2018) · Journal of Applied Physics · doi:10.1063/1.5038038 · Cited by 174
Compact reservoir computing with a photonic integrated circuit
Takano, K., Sugano, C., Inubushi, M., Yoshimura, K., Sunada, S., Kanno, K., & Uchida, A. (2018) · Optics Express · doi:10.1364/oe.26.029424 · Cited by 179
Human action recognition with a large-scale brain-inspired photonic computer
Antonik, P., Marsal, N., Brunner, D., & Rontani, D. (2019) · Nature Machine Intelligence · doi:10.1038/s42256-019-0110-8 · Cited by 177
Echo State Networks for Proactive Caching in Cloud-Based Radio Access Networks With Mobile Users
Chen, M., Saad, W., Yin, C., & Debbah, M. (2017) · IEEE Transactions on Wireless Communications · doi:10.1109/twc.2017.2683482 · Cited by 176
“To address these challenges, we propose a novel approach to predict the content request distribution and mobility pattern for each user based on the powerful framework of echo state networks [31].”
MethodConnectivity, Dynamics, and Memory in Reservoir Computing with Binary and Analog Neurons
Büsing, L., Schrauwen, B., & Legenstein, R. (2010) · Neural Computation · doi:10.1162/neco.2009.01-09-947 · Cited by 177
“…(often outperforming other state-of-the-art methods) such as chaotic time series prediction and nonlinear wireless channel equalization (Jaeger & Haas, 2004), speech recognition (Verstraeten, Schrauwen, Stroobandt, & Campenhout, 2005; Jaeger, Lukoˇseviˇcius, Popovici, & Siewert, 2007),…”
Performance Benchmarking and Analysis of Prognostic Methods for CMAPSS Datasets
Ramasso, E., & Saxena, A. (2020) · International Journal of Prognostics and Health Management · doi:10.36001/ijphm.2014.v5i2.2236 · Cited by 243
“…have considered these issues, for instance, Echo State Networks and Extreme Learning Machine are known to reduce the learning time and complexity (Jaeger & Haas, 2004; Huang, Zhu, & Siew, 2004; Siqueira, Boccato, Attux, & Lyra, 2012; Butcher, Verstraeten, Schrauwen, Day, & Haycock, 2013), while…”
BackgroundAn Integrated Probabilistic Approach to Lithium-Ion Battery Remaining Useful Life Estimation
Liu, D., Xie, W., Liao, H., & Peng, Y. (2015) · IEEE Transactions on Instrumentation and Measurement · doi:10.1109/tim.2014.2348613 · Cited by 170
“With the development of RC, Jaeger and Haas [22] and Jaeger et al.”
MethodGhosh, S., Opala, A., Matuszewski, M., Paterek, T., & Liew, T. C. H. (2019) · npj Quantum Information · doi:10.1038/s41534-019-0149-8 · Cited by 171
1/f neural noise and electrophysiological indices of contextual prediction in aging
Dave, S., Brothers, T. A., & Swaab, T. Y. (2018) · Brain Research · doi:10.1016/j.brainres.2018.04.007 · Cited by 174
Opportunities in Quantum Reservoir Computing and Extreme Learning Machines
Mujal, P., Martínez‐Peña, R., Nokkala, J., García‐Beni, J., Giorgi, G. L., Soriano, M. C., & Zambrini, R. (2021) · Advanced Quantum Technologies · doi:10.1002/qute.202100027 · Cited by 163
Reservoir computing and extreme learning machines for non-linear time-series data analysis
Butcher, J. B., Verstraeten, D., Schrauwen, B., Day, C. R., & Haycock, P. W. (2013) · Neural Networks · doi:10.1016/j.neunet.2012.11.011 · Cited by 169
Physical reservoir computing based on spin torque oscillator with forced synchronization
Tsunegi, S., Taniguchi, T., Nakajima, K., Miwa, S., Yakushiji, K., Fukushima, A., Yuasa, S., & Kubota, H. (2019) · Applied Physics Letters · doi:10.1063/1.5081797 · Cited by 171
Marcucci, G., Pierangeli, D., & Conti, C. (2020) · Physical Review Letters · doi:10.1103/physrevlett.125.093901 · Cited by 173
A comprehensive review on deep learning approaches in wind forecasting applications
Wu, Z., Luo, G., Yang, Z., Guo, Y., Li, K., & Xue, Y. (2022) · CAAI Transactions on Intelligence Technology · doi:10.1049/cit2.12076 · Cited by 161
Steil, J. J. (2007) · Neural Networks · doi:10.1016/j.neunet.2007.04.011 · Cited by 165
Kawai, Y., Park, J., & Asada, M. (2019) · Neural Networks · doi:10.1016/j.neunet.2019.01.002 · Cited by 158
Echo State Property of Deep Reservoir Computing Networks
Gallicchio, C., & Micheli, A. (2017) · Cognitive Computation · doi:10.1007/s12559-017-9461-9 · Cited by 164
“Although the principles of RC have been instantiated in different ways in literature (see e.g., [28, 40, 42]), the reference RC model in the neuro-computing area is represented by the Echo State Network (ESN) [18, 21].”
MethodMachine learning prediction of critical transition and system collapse
Kong, L.-W., Fan, H., Grebogi, C., & Lai, Y. (2021) · Physical Review Research · doi:10.1103/physrevresearch.3.013090 · Cited by 171
Weight Agnostic Neural Networks
Gaier, A., & Ha, D. (2019) · Unpublished · doi:10.13140/rg.2.2.16025.88169 · Cited by 161
Impact of input mask signals on delay-based photonic reservoir computing with semiconductor lasers
Kuriki, Y., Nakayama, J., Takano, K., & Uchida, A. (2018) · Optics Express · doi:10.1364/oe.26.005777 · Cited by 164
Human activity recognition using multisensor data fusion based on Reservoir Computing
Palumbo, F., Gallicchio, C., Pucci, R., & Micheli, A. (2016) · Journal of Ambient Intelligence and Smart Environments · doi:10.3233/ais-160372 · Cited by 157
“In this paper, within the class of RNNs, we take into consideration the efficient Reservoir Computing (RC) [52,82] paradigm in general, and the Echo State Network (ESN) [33,34] model in particular.”
BackgroundMethodObserving spatio-temporal dynamics of excitable media using reservoir computing
Zimmermann, R. S., & Parlitz, U. (2018) · Chaos: An Interdisciplinary Journal of Nonlinear Science · doi:10.1063/1.5022276 · Cited by 161
Tutorial: Photonic neural networks in delay systems
Brunner, D., Penkovsky, B., Marquez, B. A., Jacquot, M., Fischer, I., & Larger, L. (2018) · Journal of Applied Physics · doi:10.1063/1.5042342 · Cited by 155
The Cerebro-Cerebellum as a Locus of Forward Model: A Review
Tanaka, H., Ishikawa, T., Lee, J., & Kakei, S. (2020) · Frontiers in Systems Neuroscience · doi:10.3389/fnsys.2020.00019 · Cited by 155
Forecasting chaotic systems with very low connectivity reservoir computers
Griffith, A., Pomerance, A., & Gauthier, D. J. (2019) · Chaos An Interdisciplinary Journal of Nonlinear Science · doi:10.1063/1.5120710 · Cited by 187
Boosting Computational Power through Spatial Multiplexing in Quantum Reservoir Computing
Nakajima, K., Fujii, K., Negoro, M., Mitarai, K., & Kitagawa, M. (2019) · Physical Review Applied · doi:10.1103/physrevapplied.11.034021 · Cited by 153
“As a reference, each plot contains the performance of the conventional ESN.”
BackgroundInfluentialZhang, L., Tang, Y., Hua, C., & Guan, X. (2014) · Applied Soft Computing · doi:10.1016/j.asoc.2014.11.018 · Cited by 153
Metadata
- Authors
- Jaeger, H., & Haas, H.
- Affiliations
- Constructor University (DE)
- Corresponding author
- Jaeger, Herbert
- Venue
- Science
- Published
- 2004-04-02
- Volume
- 304
- Issue
- 5667
- Pages
- 78-80
- ISSN
- 0036-8075 (ISSN-L)
- 1095-9203
- OpenAlex
- W2118706537
- Cited by
- 3,811
- References
- 18
- Citations by year
- 2026: 257 · 2025: 314 · 2024: 315 · 2023: 332 · 2022: 300 · 2021: 355 · 2020: 259 · 2019: 309 · 2018: 237 · 2017: 160 · 2016: 142 · 2015: 128 · 2014: 120 · 2013: 107 · 2012: 100
- FWCI
- 33.93
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Science (publishedVersion)
- PubMed (publishedVersion)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Neural Networks and Reservoir Computing
- Physical Sciences › Computer Science › Artificial Intelligence › Neural Networks and Applications
- Physical Sciences › Physics and Astronomy › Statistical and Nonlinear Physics › Chaos control and synchronization
- Keywords
- echo state networks, chaotic time series prediction, recurrent neural networks, channel equalization, wireless communication, energy efficiency, reservoir computing
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.