Harnessing Nonlinearity: Predicting Chaotic Systems and Saving Energy in Wireless Communication
The DOI matches its Crossref record; 18 of the 18 references it cites were found in a registry or an index.
Abstract
We present a method for learning nonlinear systems, echo state networks (ESNs). ESNs employ artificial recurrent neural networks in a way that has recently been proposed independently as a learning mechanism in biological brains. The learning method is computationally efficient and easy to use. On a benchmark task of predicting a chaotic time series, accuracy is improved by a factor of 2400 over previous techniques. The potential for engineering applications is illustrated by equalizing a communication channel, where the signal error rate is improved by two orders of magnitude.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Although work in artificial intelligence and machine-learning research has shown that similar or even more complex outcomes can be achieved with a variety of techniques, including deep neural networks with trained weights (Gibney, 2015), reservoir networks with a trained output mapping (Maass et al., 2002;Jaeger & Haas, 2004), networks that are evolved using a genetic algorithm (Iizuka & Di Paolo,”
From A potential mechanism for Gibsonian resonance: behavioral entrainment emerges from local homeostasis in an unsupervised reservoir network (2024) · cites this paper as Jaeger & Haas, 2004
In the citing paper
“Therefore, a sequence production solely by a recurrent neural network can be unstable against small fluctuations in activities and unwanted noises (Jaeger and Haas, 2004).”
From The Cerebro-Cerebellum as a Locus of Forward Model: A Review (2020) · cites this paper as Jaeger and Haas, 2004
Citations
Cites
Backpropagation through time: what it does and how to do it
Werbos, P. J. (1990) · Proceedings of the IEEE · doi:10.1109/5.58337 · Cited by 4,879
A Learning Algorithm for Continually Running Fully Recurrent Neural Networks
Williams, R. J., & Zipser, D. (1989) · Neural Computation · doi:10.1162/neco.1989.1.2.270 · Cited by 4,369
Oscillation and Chaos in Physiological Control Systems
Mackey, M. C., & Glass, L. (1977) · Science · doi:10.1126/science.267326 · Cited by 4,135
Maass, W., Natschläger, T., & Markram, H. (2002) · Neural Computation · doi:10.1162/089976602760407955 · Cited by 4,025
Real-time prediction of hand trajectory by ensembles of cortical neurons in primates
Wessberg, J., Stambaugh, C. R., Kralik, J. D., Beck, P. D., Laubach, M., Chapin, J. K., Kim, J., Biggs, S. J., Srinivasan, M. A., & Nicolelis, M. A. L. (2000) · Nature · doi:10.1038/35042582 · Cited by 1,487
'Neural-gas' network for vector quantization and its application to time-series prediction
Martinetz, T. M., Berkovich, S. G., & Schulten, K. J. (1993) · IEEE Transactions on Neural Networks · doi:10.1109/72.238311 · Cited by 1,431
Approximation of dynamical systems by continuous time recurrent neural networks
Funahashi, K.-I., & Nakamura, Y. (1993) · Neural Networks · doi:10.1016/s0893-6080(05)80125-x · Cited by 980
A new evolutionary system for evolving artificial neural networks
Yao, X., & Liu, Y. (1997) · IEEE Transactions on Neural Networks · doi:10.1109/72.572107 · Cited by 874
Applying LSTM to Time Series Predictable through Time-Window Approaches
Gers, F. A., Eck, D., & Schmidhuber, J. (2001) · Lecture Notes in Computer Science · doi:10.1007/3-540-44668-0_93 · Cited by 334
Computational neuroscience: a comprehensive approach
(2004) · Choice Reviews Online · doi:10.5860/choice.41-5284 · Cited by 279
Research works printed but not linked: 10
- K. Doya, in The Handbook of Brain Theory and Neural Networks, M. A. Arbib, Ed. (MIT Press, Cambridge, MA, 1995), pp. 796–800.
- J. Vesanto in Proc. WSOM'97 (1997); www.cis.hut.fi/projects/monitor/publications/papers/wsom97.ps.
- L. Chudy, I. Farkas, Neural Network World8, 481 (1998).
- H. Bersini M. Birattari G. Bontempi in Proc. IEEE World Congr. on Computational Intelligence (IJCNN'98) (1997) pp. 2102–2106; ftp://iridia.ulb.ac.be/pub/lazy/papers/IridiaTr1997-13_2.ps.gz.
- V. J. Mathews, J. Lee, in Advanced Signal Processing: Algorithms, Architectures, and Implementations V (Proc. SPIE Vol. 2296), (SPIE, San Diego, CA, 1994), pp. 317–327.
- J. Hertzberg H. Jaeger F. Schönherr in Proc. 15th Europ. Conf. on Art. Int. (ECAI 02) F. van Harmelen Ed. (IOS Press Amsterdam 2002) pp. 708–712; www.ais.fhg.de/∼schoenhe/papers/ECAI02.pdf.
- H. Jaeger, in Advances in Neural Information Processing Systems 15, S. Becker, S. Thrun, K. Obermayer, Eds. (MIT Press, Cambridge, MA, 2003) pp. 593–600.
- G. E. Hinton, in Parallel Models of Associative Memory, G. E. Hinton, J. A. Anderson, Eds. (Erlbaum, Hillsdale, NJ, 1981), pp. 161–187.
- M. Kawato, in The Handbook of Brain Theory and Neural Networks, M. Arbib, Ed. (MIT Press, Cambridge, MA, 1995), pp. 172–178.
- G. B. Stanley, Neurocomputing38–40, 1703 (2001).
Cited by
Items 601–700 of 1,009
Consistency in echo-state networks
Lymburn, T., Khor, A., Stemler, T., Corrêa, D. C., Small, M., & Jüngling, T. (2019) · Chaos An Interdisciplinary Journal of Nonlinear Science · doi:10.1063/1.5079686 · Cited by 61
Perturbing low dimensional activity manifolds in spiking neuronal networks
Wärnberg, E., & Kumar, A. (2019) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1007074 · Cited by 52
“First, we consider the FORCE learning rule [12] which builds upon the liquid state machine and echo state networks [11, 17, 18].”
MethodMemory embedded non-intrusive reduced order modeling of non-ergodic flows
Ahmed, S. E., Rahman, S. M., San, O., Rasheed, A., & Navon, I. M. (2019) · Physics of Fluids · doi:10.1063/1.5128374 · Cited by 53
“There have been a number of research efforts which showed that RNN, specifically LSTM as a variant of RNN, is capable of predicting the dependencies among temporal data sequences [128–133].”
BackgroundIn-Ear EEG Based Attention State Classification Using Echo State Network
Jeong, D., & Jeong, J. (2020) · Brain Sciences · doi:10.3390/brainsci10060321 · Cited by 55
“Although ESNs were initially designed for sparsely connected reservoirs (1% interconnectivity) to have echo state properties [42], they were reported to work well with fully connected reservoirs [32,52,55].”
BackgroundReservoir computing on epidemic spreading: A case study on COVID-19 cases
Ghosh, S. K., Senapati, A., Mishra, A., Chattopadhyay, J., Dana, S. K., Hens, C., & Ghosh, D. (2021) · Physical review. E · doi:10.1103/physreve.104.014308 · Cited by 55
On Computational Power and the Order-Chaos Phase Transition in Reservoir Computing
Schrauwen, B., Buesing, L., & Legenstein, R. A. (2008) · Ghent University Academic Bibliography (Ghent University) · Cited by 66
Time-delayed reservoir computing based on an optically pumped spin VCSEL for high-speed processing
Yang, Y., Zhou, P., Mu, P., & Li, N. (2022) · Nonlinear Dynamics · doi:10.1007/s11071-021-07140-5 · Cited by 55
Namiki, W., Nishioka, D., Yamaguchi, Y., Tsuchiya, T., Higuchi, T., & Terabe, K. (2023) · Advanced Intelligent Systems · doi:10.1002/aisy.202300228 · Cited by 60
“A reservoir readout y(k) is expressed as the product of each Xi(k) and learning parameters Wi, as follow; y(k) = ∑ W!X!(k) !#$ , (1) where n and k are the number of nodes and the discrete time, respectively.”
BackgroundLupo, A., Picco, E., Zajnulina, M., & Massar, S. (2023) · Optica · doi:10.1364/optica.489501 · Cited by 64
“A reservoir computer (RC, see left panel of Fig.1) [16] is a recurrent neural network composed of three layers: the input layer, the reservoir layer, and the output layer.”
BackgroundMethodInfluentialSpatiotemporal Data Processing with Memristor Crossbar‐Array‐Based Graph Reservoir
Jang, Y. H., Lee, S. H., Han, J., Kim, W., Shim, S. K., Cheong, S., Woo, K. S., Han, J., & Hwang, C. S. (2023) · Advanced Materials · doi:10.1002/adma.202309314 · Cited by 53
Review: Machine learning for advancing low-temperature plasma modeling and simulation
Trieschmann, J., Vialetto, L., & Gergs, T. (2023) · Journal of Micro/Nanopatterning Materials and Metrology · doi:10.1117/1.jmm.22.4.041504 · Cited by 71
“153,160 Therein, an RNN network reservoir is initially set up.”
BackgroundSupport System for Trading in Exchange Market by Distributional Forecasting Model
Maknickas, A., & Maknickienė, N. (2019) · Informatica · doi:10.15388/informatica.2018.198 · Cited by 53
Mean-field theory of echo state networks
Massar, M., & Massar, S. (2013) · Physical Review E · doi:10.1103/physreve.87.042809 · Cited by 58
Memristor Models for Machine Learning
Carbajal, J. P., Dambre, J., Hermans, M., & Schrauwen, B. (2015) · Neural Computation · doi:10.1162/neco_a_00694 · Cited by 54
“Reservoir computing (RC) is a supervised learning framework rooted in recurrent neural network research (seminal work in [5; 6], recent developments in [7; 8]).”
BackgroundGhosh, S., Aswani, K., Singh, S., Sahu, S., Fujita, D., & Bandyopadhyay, A. (2014) · Information · doi:10.3390/info5010028 · Cited by 49
“that can spontaneously “reply back”: Creating liquid state models or echo state models [20] using the seed device is not practically feasible, since ―reply‖ would automatically get masked within a limited distance.”
Wavelet-denoising multiple echo state networks for multivariate time series prediction
Xu, M., Han, M., & Lin, H. (2018) · Information Sciences · doi:10.1016/j.ins.2018.07.015 · Cited by 50
Operating in a Reverberating Regime Enables Rapid Tuning of Network States to Task Requirements
Wilting, J., Dehning, J., Neto, J. P., Rudelt, L., Wibral, M., Zierenberg, J., & Priesemann, V. (2018) · Frontiers in Systems Neuroscience · doi:10.3389/fnsys.2018.00055 · Cited by 62
“This is at the heart of reservoir computing in echo state networks or liquid state machines [9, 10, 11, 12, 13, 5].”
BackgroundMuscinelli, S. P., Gerstner, W., & Schwalger, T. (2019) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1007122 · Cited by 64
Reservoir Computing as a Tool for Climate Predictability Studies
Nadiga, B. (2021) · Journal of Advances in Modeling Earth Systems · doi:10.1029/2020ms002290 · Cited by 55
“…has outperformed other methods of nonlinear system identification, prediction, and classification in the context of chaotic dynamics (e.g., Jaeger & Haas, 2004; Pathak et al., 2018, and others), we consider its use as a tool in climate predictability studies, while noting recent use of RC methods…”
MethodA Review of Designs and Applications of Echo State NetworksPreprint
Sun, C., Song, M., Hong, S., & Li, H. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2012.02974 · Cited by 73
“8 [27], most existing works [105] set τ = 17 and initial y (0) = 1 .”
BackgroundMethodWada, T., Nishioka, D., Namiki, W., Tsuchiya, T., Higuchi, T., & Terabe, K. (2023) · Advanced Intelligent Systems · doi:10.1002/aisy.202300123 · Cited by 54
Dimension of Activity in Random Neural Networks
Clark, D. G., Abbott, L. F., & Litwin-Kumar, A. (2023) · Physical Review Letters · doi:10.1103/physrevlett.131.118401 · Cited by 79
Rössert, C., Dean, P., & Porrill, J. (2015) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1004515 · Cited by 61
“Suitable RNNs exhibit temporal memory, in that short-duration inputs can generate long-duration outputs, and can be used to model nonlinear functions and thus act as a non-linear adaptive filter [25,49,66,67].”
BackgroundHybrid Neural Prediction and Optimized Adjustment for Coke Oven Gas System in Steel Industry
Zhao, J., Liu, Q., Wang, W., Pedrycz, W., & Cong, L. (2012) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2011.2179309 · Cited by 49
“Jaeger in [13] also claimed such phenomenon that made the”
BackgroundLearning nonlinear input–output maps with dissipative quantum systems
Chen, J., & Nurdin, H. I. (2019) · Quantum Information Processing · doi:10.1007/s11128-019-2311-9 · Cited by 78
Interpreting Recurrent Neural Networks Behaviour via Excitable Network Attractors
Ceni, A., Ashwin, P., & Livi, L. (2019) · Cognitive Computation · doi:10.1007/s12559-019-09634-2 · Cited by 62
“Echo state networks (ESNs) [21, 30] constitute an important example of reservoir computing, where a recurrent layer (called a reservoir) is composed of a large number of neurons with randomly initialised connections that are not fine-tuned via gradient-based optimisation mechanisms.”
BackgroundDecoding Movements from Cortical Ensemble Activity Using a Long Short-Term Memory Recurrent Network
Tseng, P.-H., Urpí, N. A., Lebedev, M., & Nicolelis, M. A. L. (2019) · Neural Computation · doi:10.1162/neco_a_01189 · Cited by 50
Comparison of Photonic Reservoir Computing Systems for Fiber Transmission Equalization
Argyris, A., Cantero, J., Galletero, M., Pereda, E., Mirasso, C. R., Fischer, I., & Soriano, M. C. (2019) · IEEE Journal of Selected Topics in Quantum Electronics · doi:10.1109/jstqe.2019.2936947 · Cited by 56
“2936947 is solely performed on the weights of the readout layer, while those of the recurrent network are kept fixed [2], [3].”
BackgroundDesigning spontaneous behavioral switching via chaotic itinerancy
Inoue, K., Nakajima, K., & Kuniyoshi, Y. (2020) · Science Advances · doi:10.1126/sciadv.abb3989 · Cited by 55
“In addition, this embedding process is accomplished by modifying fewer parameters using the method of reservoir computing (32, 33).”
MethodUnderstanding and mitigating noise in trained deep neural networks
Игоревна, С. Н., Larger, L., & Brunner, D. (2021) · Neural Networks · doi:10.1016/j.neunet.2021.11.008 · Cited by 63
Dynamic stability analysis of fractional order leaky integrator echo state neural networks
Pahnehkolaei, S. M. A., Alfi, A., & Machado, J. A. T. (2016) · Communications in Nonlinear Science and Numerical Simulation · doi:10.1016/j.cnsns.2016.11.013 · Cited by 48
Design of polynomial echo state networks for time series prediction
Yang, C., Qiao, J., Han, H., & Wang, L. (2018) · Neurocomputing · doi:10.1016/j.neucom.2018.02.036 · Cited by 48
Perspective on unconventional computing using magnetic skyrmions
Lee, O., Msiska, R., Brems, M. A., Kläui, M., Kurebayashi, H., & Everschor‐Sitte, K. (2023) · Applied Physics Letters · doi:10.1063/5.0148469 · Cited by 58
Inferring unknown unknowns: Regularized bias-aware ensemble Kalman filter
Nóvoa, A., Racca, A., & Magri, L. (2023) · Computer Methods in Applied Mechanics and Engineering · doi:10.1016/j.cma.2023.116502 · Cited by 62
“Echo state networks (ESNs) are a type of recurrent neural networks, which are designed to learn temporal correlations in data through an internal high-dimensional state, the reservoir (Jaeger and Haas, 2004; Maass et al., 2002).”
BackgroundMethodInfluentialBrain-Inspired Reservoir Computing Using Memristors with Tunable Dynamics and Short-Term Plasticity
Armendarez, N. X., Mohamed, A. S. I., Dhungel, A., Hossain, M. R., Hasan, M. S., & Najem, J. S. (2024) · ACS Applied Materials & Interfaces · doi:10.1021/acsami.3c16003 · Cited by 54
SIMPEL: Circuit model for photonic spike processing laser neurons
Shastri, B. J., Nahmias, M. A., Tait, A. N., Wu, B., & Prucnal, P. R. (2015) · Optics Express · doi:10.1364/oe.23.008029 · Cited by 51
Optimal nonlinear information processing capacity in delay-based reservoir computers
Grigoryeva, L., Henriques, J., Larger, L., & Ortega, J. (2015) · Scientific Reports · doi:10.1038/srep12858 · Cited by 57
“Consider a stable equilibrium ∈ x0 of the autonomous system associated to (1) or, equivalently, a stable fixed point of (5) of the form = ( , .”
BackgroundInfluentialSelection of cortical dynamics for motor behaviour by the basal ganglia
Mannella, F., & Baldassarre, G. (2015) · Biological Cybernetics · doi:10.1007/s00422-015-0662-6 · Cited by 48
“Learning with feedback is not easy in reservoir networks, but various solutions have been found to implement it (Jaeger and Haas 2004; Steil 2004; Sussillo and Abbott 2009).”
BackgroundNeuromorphic Computing Based on Silicon Photonics and Reservoir Computing
Katumba, A., Freiberger, M., Laporte, F., Lugnan, A., Sackesyn, S., Ma, C., Dambre, J., & Bienstman, P. (2018) · IEEE Journal of Selected Topics in Quantum Electronics · doi:10.1109/jstqe.2018.2821843 · Cited by 55
“Therefore, such an architecture would not scale well when increasing the number of nodes in the photonic reservoirs to numbers common in classic echo state networks [2].”
BackgroundImproved learning algorithm for two-layer neural networks for identification of nonlinear systems
Vargas, J. A. R., Witold Pedrycz, C., & Hemerly, E. M. (2018) · Neurocomputing · doi:10.1016/j.neucom.2018.10.008 · Cited by 48
Patel, D., & Ott, E. (2023) · Chaos An Interdisciplinary Journal of Nonlinear Science · doi:10.1063/5.0131787 · Cited by 76
“Reservoir computing has previously been successfully used to predict the time evolution of dynamical systems [3–5,7,11, 18–20,27], and its training is computationally inexpensive since it can be accomplished via a simple linear regression.”
BackgroundMethodMachine Learning Enables Prediction of Halide Perovskites’ Optical Behavior with >90% Accuracy
Srivastava, M., Hering, A. R., An, Y., Correa‐Baena, J., & Leite, M. S. (2023) · ACS Energy Letters · doi:10.1021/acsenergylett.2c02555 · Cited by 47
A Multiple-Input Strategy to Efficient Integrated Photonic Reservoir Computing
Katumba, A., Freiberger, M., Bienstman, P., & Dambre, J. (2017) · Cognitive Computation · doi:10.1007/s12559-017-9465-5 · Cited by 57
Yao, X., Wang, Z., & Zhang, H. (2019) · Neural Networks · doi:10.1016/j.neunet.2019.01.003 · Cited by 47
Searching for principles of brain computation
Maass, W. (2016) · Current Opinion in Behavioral Sciences · doi:10.1016/j.cobeha.2016.06.003 · Cited by 60
Zhu, Q., Ma, H., & Lin, W. J. (2019) · Chaos An Interdisciplinary Journal of Nonlinear Science · doi:10.1063/1.5120867 · Cited by 54
Mo, Y., Li, Q., Karimian, H., Fang, S., Tang, B., Chen, G., & Sachdeva, S. (2019) · Atmospheric Environment · doi:10.1016/j.atmosenv.2019.117072 · Cited by 47
Integer Echo State Networks: Efficient Reservoir Computing for Digital Hardware
Kleyko, D. V., Frady, E. P., Kheffache, M., & Osipov, E. (2020) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2020.3043309 · Cited by 61
“In this scenario, we followed the preprocessing of data and the parameters of ESN described in [4].”
MethodAdaptable Hamiltonian neural networks
Han, C.-D., Glaz, B. J., Haile, M., & Lai, Y. (2021) · Physical Review Research · doi:10.1103/physrevresearch.3.023156 · Cited by 55
Deep Echo State Network With Multiple Adaptive Reservoirs for Time Series Prediction
Wang, Z., Yao, X., Huang, Z., & Liu, L. (2021) · IEEE Transactions on Cognitive and Developmental Systems · doi:10.1109/tcds.2021.3062177 · Cited by 47
“Comparing with those methods in [1] and [29], the simulation results are shown in Table II.”
BackgroundMethodInfluentialSecond-order Optimization for Neural Networks
Martens, J. (2016) · TSpace (University of Toronto) · Cited by 79
Chaos on compact manifolds: Differentiable synchronizations beyond the Takens theorem
Grigoryeva, L., Hart, A., & Ortega, J. (2021) · Physical review. E · doi:10.1103/physreve.103.062204 · Cited by 62
Enhancing computational performance of a spin-wave reservoir computer with input synchronization
Watt, S., Kostylev, M., & Ustinov, A. B. (2021) · Journal of Applied Physics · doi:10.1063/5.0033292 · Cited by 49
Recurrent nonsymmetric deep auto encoder approach for network intrusion detection system
M, S. Y., & Kalpana, R. (2022) · Measurement Sensors · doi:10.1016/j.measen.2022.100527 · Cited by 53
Gao, R., Li, R., Hu, M., Suganthan, P. N., & Yuen, K. F. (2022) · Engineering Applications of Artificial Intelligence · doi:10.1016/j.engappai.2022.105535 · Cited by 46
Dissipation as a resource for Quantum Reservoir Computing
Sannia, A., Martínez‐Peña, R., Soriano, M. C., Giorgi, G. L., & Zambrini, R. (2024) · Quantum · doi:10.22331/q-2024-03-20-1291 · Cited by 126
“Following the standard procedure, the input originates from a uniform random distribution in the interval [0 , 1] and the expected target at each time step ( y k ) is to reproduce the previous input for a given delay τ y k = s k − τ .”
MethodIlluminating the Future: A Comprehensive Review of AI-Based Solar Irradiance Prediction Models
Sammar, M. J., Saeed, M. A., Mohsin, S. M., Akber, S. M. A., Bukhsh, R., Abazeed, M. B., & Ali, M. (2024) · IEEE Access · doi:10.1109/access.2024.3402096 · Cited by 47
“The dynamics of this reservoir can amplify and propagate input signals, which can then be trained to produce the desired output patterns [49].”
BackgroundA Fast SVD-Hidden-nodes based Extreme Learning Machine for Large-Scale Data Analytics
Deng, W., Bai, Z., Huang, G.-B., & Zheng, Q. (2015) · Neural Networks · doi:10.1016/j.neunet.2015.09.003 · Cited by 47
Monthly Rainfall Forecasting Using Echo State Networks Coupled with Data Preprocessing Methods
Ouyang, Q., & Lu, W. (2017) · Water Resources Management · doi:10.1007/s11269-017-1832-1 · Cited by 46
“An ESN is an artificial recurrent neural network (RNN) characterized by feedback (Brecurrent^) loops in its synaptic connection pathways (Jaeger and Haas 2004).”
BackgroundGrigoryeva, L., Henriques, J., Larger, L., & Ortega, J. (2014) · Neural Networks · doi:10.1016/j.neunet.2014.03.004 · Cited by 52
Wikner, A., Pathak, J., Hunt, B. R., Szunyogh, I., Girvan, M., & Ott, E. (2021) · Chaos An Interdisciplinary Journal of Nonlinear Science · doi:10.1063/5.0048050 · Cited by 52
Na, X., Ren, W., & Xu, X. (2021) · Engineering Applications of Artificial Intelligence · doi:10.1016/j.engappai.2021.104229 · Cited by 51
Generative Models of Brain Dynamics
Ramezanian-Panahi, M., Abrevaya, G., Gagnon-Audet, J.-C., Voleti, V., Rish, I., & Dumas, G. (2022) · Frontiers in Artificial Intelligence · doi:10.3389/frai.2022.807406 · Cited by 64
“ESN used to be one of the gold standards of nonlinear dynamics modeling before 2010 (Jaeger and Haas, 2004; Jaeger et al., 2007).”
Huang, Z., Wang, Z., Yao, X., & Zhang, H. (2018) · IEEE Transactions on Power Electronics · doi:10.1109/tpel.2018.2875718 · Cited by 47
Directed and acyclic synaptic connectivity in the human layer 2-3 cortical microcircuit
Peng, Y., Bjelde, A., Aceituno, P. V., Mittermaier, F. X., Planert, H., Grosser, S., Onken, J., Faust, K., Kalbhenn, T., Simon, M., et al. (2024) · Science · doi:10.1126/science.adg8828 · Cited by 53
Digital Implementation of a Single Dynamical Node Reservoir Computer
Alomar, M. L., Soriano, M. C., Escalona-Morán, M., Canals, V., Fischer, I., Mirasso, C. R., & Rosselló, J. L. (2015) · IEEE Transactions on Circuits & Systems II Express Briefs · doi:10.1109/tcsii.2015.2458071 · Cited by 57
“The system is trained such that the weighted sum of the states approximates the desired target value as closely as possible, following the standard procedure for RC [13], [14].”
MethodSpotting laughter in natural multiparty conversations
Scherer, S. J., Glodek, M., Schwenker, F., Campbell, N., & Palm, G. (2012) · ACM Transactions on Interactive Intelligent Systems · doi:10.1145/2133366.2133370 · Cited by 45
“In the first approach Echo State Networks (ESN) [Jaeger and Haas 2004], that are sensitive to the sequential characteristics of presented features, are used.”
BackgroundInfluentialGenetic algorithm for reservoir computing optimization
Ferreira, A. A., & Ludermir, T. B. (2009) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn.2009.5178654 · Cited by 55
“RNNs have recurrent connections between the processing elements (PEs) creating internally the memory required to store the history of the input patterns [7], [8].”
BackgroundLv, Z., Zhao, J., Liu, Y., & Wang, W. (2015) · Control Engineering Practice · doi:10.1016/j.conengprac.2015.10.003 · Cited by 45
Prospective Coding by Spiking Neurons
Brea, J., Gaál, A. T., Urbanczik, R., & Senn, W. (2016) · PLOS Computational Biology · doi:10.1371/journal.pcbi.1005003 · Cited by 40
“The short-term memory network can be seen as a liquid state machine [42] or echo state machine [43] and the ramping activity is learned as readout from this activity traces.”
BackgroundFreiberger, M., Sackesyn, S., Ma, C., Katumba, A., Bienstman, P., & Dambre, J. (2019) · IEEE Journal of Selected Topics in Quantum Electronics · doi:10.1109/jstqe.2019.2929699 · Cited by 50
Zhang, H., Yang, Y., Zhang, Y., He, Z., Yuan, W., Yang, Y. A., Qiu, W., & Li, L. (2020) · Neural Computing and Applications · doi:10.1007/s00521-020-05113-0 · Cited by 46
Spintronic reservoir computing without driving current or magnetic field
Taniguchi, T., Ogihara, A., Utsumi, Y., & Tsunegi, S. (2022) · Scientific Reports · doi:10.1038/s41598-022-14738-1 · Cited by 54
Generative Modeling of Autonomous Robots and their Environments using Reservoir Computing
Antonelo, E. A., Schrauwen, B., & Campenhout, J. V. (2007) · Neural Processing Letters · doi:10.1007/s11063-007-9054-9 · Cited by 51
“An ESN is composed of a discrete hyperbolic-tangent RNN (i.e., the reservoir) and of a linear readout output layer which maps the reservoir states to the actual output.”
MethodInfluentialMultifunctionality in a reservoir computer
Flynn, A., Tsachouridis, V. A., & Amann, A. (2021) · Chaos An Interdisciplinary Journal of Nonlinear Science · doi:10.1063/5.0019974 · Cited by 60
Simulation platform for pattern recognition based on reservoir computing with memristor networks
Tanaka, G., & Nakane, R. (2022) · Scientific Reports · doi:10.1038/s41598-022-13687-z · Cited by 47
“The sound dataset from the NIST TI-46 Word corpus 50 contains 500 sound waveform data corresponding to 10 utterances of 10 digits (“zero” to “nine”), spoken by 5 different speakers (ID: 1,2,5,6,7) 51 .”
MethodReservoir computing and photoelectrochemical sensors: A marriage of convenience
Abdi, G., Alluhaibi, L., Kowalewska, E., Mazur, T., Mech, K., Podborska, A., Sławek, A., Tanaka, H., & Szacilowski, K. T. (2023) · Coordination Chemistry Reviews · doi:10.1016/j.ccr.2023.215155 · Cited by 45
Emergence of a resonance in machine learning
Zhai, Z.-M., Kong, L.-W., & Lai, Y. (2023) · Physical Review Research · doi:10.1103/physrevresearch.5.033127 · Cited by 60
Photonic Delay Systems as Machine Learning Implementations
Hermans, M., Soriano, M. C., Dambre, J., Bienstman, P., & Fischer, I. (2015) · DIGITAL.CSIC (Spanish National Research Council (CSIC)) · doi:10.48550/arxiv.1501.02592 · Cited by 39
Sussillo, D., & Abbott, L. F. (2012) · PLoS ONE · doi:10.1371/journal.pone.0037372 · Cited by 45
“The idea introduced by Jaeger and Haas [6], which allows learning to be restricted solely to the output weights w, is to change equation 1 for the network of Figure 1B to. t dx dt ~{xzJrzuzzvI(t)~{xz JzuwT rzvI(t), ð3Þ for the network of Figure 1A.”
BackgroundMethodInfluentialOptimal Balance of the Striatal Medium Spiny Neuron Network
Ponzi, A. P. D., & Wickens, J. R. (2013) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1002954 · Cited by 48
“The spatio-temporal dynamics [55,56,69,115–120] generated by this network could be utilized in such behavioural tasks or in ‘reservoir computing’ style cognitive processing [27,38,39,55,121,122].”
MethodPahnehkolaei, S. M. A., Alfi, A., & Machado, J. A. T. (2017) · Information Sciences · doi:10.1016/j.ins.2017.08.046 · Cited by 44
Yao, X., Wang, Z., & Zhang, H. (2017) · Neurocomputing · doi:10.1016/j.neucom.2017.09.092 · Cited by 44
Adaptive Levenberg-Marquardt Algorithm Based Echo State Network for Chaotic Time Series Prediction
Qiao, J., Wang, L., Yang, C., & Gu, K. (2018) · IEEE Access · doi:10.1109/access.2018.2810190 · Cited by 44
“To show the effectiveness of the ALM-ESN, the simulations are compared with the following models: OESN [8], SCR [33], DESN [34], RR-ESN [18] and Lasso-ESN [20].”
BackgroundMethodInfluentialEfficient design of hardware-enabled reservoir computing in FPGAs
Penkovsky, B., Larger, L., & Brunner, D. (2018) · Journal of Applied Physics · doi:10.1063/1.5039826 · Cited by 48
Hou, Y., Xia, G.-Q., Jayaprasath, E., Yue, D., Yang, W.-Y., & Wu, Z.-M. (2018) · Optics Communications · doi:10.1016/j.optcom.2018.10.014 · Cited by 44
Development of Highly Homogenous Quantum Dot Micropillar Arrays for Optical Reservoir Computing
Heuser, T., Große, J., Holzinger, S., Sommer, M. M., & Reitzenstein, S. (2019) · IEEE Journal of Selected Topics in Quantum Electronics · doi:10.1109/jstqe.2019.2925968 · Cited by 44
“real-time learning and ultrafast image or pattern recognition via highly parallel architectures with multiple interactions [3]–[5].”
BackgroundLearning recurrent dynamics in spiking networks
Kim, C. M., & Chow, C. C. (2018) · eLife · doi:10.7554/elife.37124 · Cited by 50
Predicting future dynamics from short-term time series using an Anticipated Learning Machine
Chen, C., Li, R., Shu, L., He, Z., Wang, J., Zhang, C., Ma, H., Aihara, K., & Chen, L. (2020) · National Science Review · doi:10.1093/nsr/nwaa025 · Cited by 46
Invertible generalized synchronization: A putative mechanism for implicit learning in neural systems
Lu, Z., & Bassett, D. S. (2020) · Chaos An Interdisciplinary Journal of Nonlinear Science · doi:10.1063/5.0004344 · Cited by 59
Developing a photonic hardware platform for brain-inspired computing based on 5 × 5 VCSEL arrays
Heuser, T., Pflüger, M., Fischer, I., Lott, J. A., Brunner, D., & Reitzenstein, S. (2020) · Journal of Physics Photonics · doi:10.1088/2515-7647/aba671 · Cited by 57
A Reservoir-based Convolutional Spiking Neural Network for Gesture Recognition from DVS Input
George, A. M., Banerjee, D., Dey, S., Mukherjee, A., & Balamurali, P. (2020) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn48605.2020.9206681 · Cited by 61
“ESN proposes driving a large recurrently connected network of artificial neurons with an input signal to produce a nonlinear response in the network nodes and obtain the desired output signal by training all these responses.”
MethodComprehensive Performance Analysis of a VCSEL-Based Photonic Reservoir Computer
Bueno, J., Robertson, J., Hejda, M., & Hurtado, A. (2021) · IEEE Photonics Technology Letters · doi:10.1109/lpt.2021.3075095 · Cited by 44
Emergence of transient chaos and intermittency in machine learning
Kong, L.-W., Fan, H., Grebogi, C., & Lai, Y. (2021) · Journal of Physics Complexity · doi:10.1088/2632-072x/ac0b00 · Cited by 45
Computing with a muscular-hydrostat system
Nakajima, K., Hauser, H., Kang, R., Guglielmino, E., Caldwell, D. G., & Pfeifer, R. (2013) · 2013 IEEE International Conference on Robotics and Automation · doi:10.1109/icra.2013.6630770 · Cited by 42
“In this paper, we first demonstrate whether the proposed system can achieve a complex computation by assigning benchmark tasks [22], [23], [37], [38], which are often used in the context of recurrent neural network learning [39] or the reservoir computing approach [37], [40], [41].”
BackgroundMethodTemporal dynamics of information content carried by neurons in the primary visual cortex
Nikolić, D., Haeusler, S., Singer, W., & Maass, W. (2007) · Advances in Neural Information Processing Systems 19 · doi:10.7551/mitpress/7503.003.0135 · Cited by 43
“Instead, our results support the view recently put forward by theoretical studies [4, 8], in which computations are performed by complex dynamical systems while information about results of these computations is read out by simple linear classifiers.”
ResultDeep-ESN: A Multiple Projection-encoding Hierarchical Reservoir Computing FrameworkPreprint
Ma, Q., Shen, L., & Cottrell, G. W. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1711.05255 · Cited by 58
“The two most well-known RC models are Liquid State Machines and Echo State Networks [2], [10], [11].”
BackgroundMethodInfluentialUsing reservoir computer to predict and prevent extreme events
Pyragas, V., & Pyragas, K. (2020) · Physics Letters A · doi:10.1016/j.physleta.2020.126591 · Cited by 49
“seek to predict extreme events or the precursors of extreme events (if avalable) directly from the dynamicsofthe vectorobservables(t). Within this limitation, the reservoir computer proposed in Ref. [11] is a promising tool for solving the problem. The ability of RC to provide model-free prediction has recently been d…”
BackgroundMethodInfluentialHierarchical architectures in reservoir computing systems
Moon, J., Wu, Y., & Lu, W. D. (2021) · Neuromorphic Computing and Engineering · doi:10.1088/2634-4386/ac1b75 · Cited by 58
Zavala, Á. E. M., Macías‐Díaz, J. E., Alba-Cuéllar, D., & Guerrero-Díaz-de-León, J. A. (2024) · Algorithms · doi:10.3390/a17020076 · Cited by 51
Metadata
- Authors
- Jaeger, H., & Haas, H.
- Affiliations
- Constructor University (DE)
- Corresponding author
- Jaeger, Herbert
- Venue
- Science
- Published
- 2004-04-02
- Volume
- 304
- Issue
- 5667
- Pages
- 78-80
- ISSN
- 0036-8075 (ISSN-L)
- 1095-9203
- OpenAlex
- W2118706537
- Cited by
- 3,811
- References
- 18
- Citations by year
- 2026: 257 · 2025: 314 · 2024: 315 · 2023: 332 · 2022: 300 · 2021: 355 · 2020: 259 · 2019: 309 · 2018: 237 · 2017: 160 · 2016: 142 · 2015: 128 · 2014: 120 · 2013: 107 · 2012: 100
- FWCI
- 33.93
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Science (publishedVersion)
- PubMed (publishedVersion)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Neural Networks and Reservoir Computing
- Physical Sciences › Computer Science › Artificial Intelligence › Neural Networks and Applications
- Physical Sciences › Physics and Astronomy › Statistical and Nonlinear Physics › Chaos control and synchronization
- Keywords
- echo state networks, chaotic time series prediction, recurrent neural networks, channel equalization, wireless communication, energy efficiency, reservoir computing
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.