Harnessing Nonlinearity: Predicting Chaotic Systems and Saving Energy in Wireless Communication
The DOI matches its Crossref record; 18 of the 18 references it cites were found in a registry or an index.
Abstract
We present a method for learning nonlinear systems, echo state networks (ESNs). ESNs employ artificial recurrent neural networks in a way that has recently been proposed independently as a learning mechanism in biological brains. The learning method is computationally efficient and easy to use. On a benchmark task of predicting a chaotic time series, accuracy is improved by a factor of 2400 over previous techniques. The potential for engineering applications is illustrated by equalizing a communication channel, where the signal error rate is improved by two orders of magnitude.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Although work in artificial intelligence and machine-learning research has shown that similar or even more complex outcomes can be achieved with a variety of techniques, including deep neural networks with trained weights (Gibney, 2015), reservoir networks with a trained output mapping (Maass et al., 2002;Jaeger & Haas, 2004), networks that are evolved using a genetic algorithm (Iizuka & Di Paolo,”
From A potential mechanism for Gibsonian resonance: behavioral entrainment emerges from local homeostasis in an unsupervised reservoir network (2024) · cites this paper as Jaeger & Haas, 2004
In the citing paper
“Therefore, a sequence production solely by a recurrent neural network can be unstable against small fluctuations in activities and unwanted noises (Jaeger and Haas, 2004).”
From The Cerebro-Cerebellum as a Locus of Forward Model: A Review (2020) · cites this paper as Jaeger and Haas, 2004
Citations
Cites
Backpropagation through time: what it does and how to do it
Werbos, P. J. (1990) · Proceedings of the IEEE · doi:10.1109/5.58337 · Cited by 4,879
A Learning Algorithm for Continually Running Fully Recurrent Neural Networks
Williams, R. J., & Zipser, D. (1989) · Neural Computation · doi:10.1162/neco.1989.1.2.270 · Cited by 4,369
Oscillation and Chaos in Physiological Control Systems
Mackey, M. C., & Glass, L. (1977) · Science · doi:10.1126/science.267326 · Cited by 4,135
Maass, W., Natschläger, T., & Markram, H. (2002) · Neural Computation · doi:10.1162/089976602760407955 · Cited by 4,025
Real-time prediction of hand trajectory by ensembles of cortical neurons in primates
Wessberg, J., Stambaugh, C. R., Kralik, J. D., Beck, P. D., Laubach, M., Chapin, J. K., Kim, J., Biggs, S. J., Srinivasan, M. A., & Nicolelis, M. A. L. (2000) · Nature · doi:10.1038/35042582 · Cited by 1,487
'Neural-gas' network for vector quantization and its application to time-series prediction
Martinetz, T. M., Berkovich, S. G., & Schulten, K. J. (1993) · IEEE Transactions on Neural Networks · doi:10.1109/72.238311 · Cited by 1,431
Approximation of dynamical systems by continuous time recurrent neural networks
Funahashi, K.-I., & Nakamura, Y. (1993) · Neural Networks · doi:10.1016/s0893-6080(05)80125-x · Cited by 980
A new evolutionary system for evolving artificial neural networks
Yao, X., & Liu, Y. (1997) · IEEE Transactions on Neural Networks · doi:10.1109/72.572107 · Cited by 874
Applying LSTM to Time Series Predictable through Time-Window Approaches
Gers, F. A., Eck, D., & Schmidhuber, J. (2001) · Lecture Notes in Computer Science · doi:10.1007/3-540-44668-0_93 · Cited by 334
Computational neuroscience: a comprehensive approach
(2004) · Choice Reviews Online · doi:10.5860/choice.41-5284 · Cited by 279
Research works printed but not linked: 10
- K. Doya, in The Handbook of Brain Theory and Neural Networks, M. A. Arbib, Ed. (MIT Press, Cambridge, MA, 1995), pp. 796–800.
- J. Vesanto in Proc. WSOM'97 (1997); www.cis.hut.fi/projects/monitor/publications/papers/wsom97.ps.
- L. Chudy, I. Farkas, Neural Network World8, 481 (1998).
- H. Bersini M. Birattari G. Bontempi in Proc. IEEE World Congr. on Computational Intelligence (IJCNN'98) (1997) pp. 2102–2106; ftp://iridia.ulb.ac.be/pub/lazy/papers/IridiaTr1997-13_2.ps.gz.
- V. J. Mathews, J. Lee, in Advanced Signal Processing: Algorithms, Architectures, and Implementations V (Proc. SPIE Vol. 2296), (SPIE, San Diego, CA, 1994), pp. 317–327.
- J. Hertzberg H. Jaeger F. Schönherr in Proc. 15th Europ. Conf. on Art. Int. (ECAI 02) F. van Harmelen Ed. (IOS Press Amsterdam 2002) pp. 708–712; www.ais.fhg.de/∼schoenhe/papers/ECAI02.pdf.
- H. Jaeger, in Advances in Neural Information Processing Systems 15, S. Becker, S. Thrun, K. Obermayer, Eds. (MIT Press, Cambridge, MA, 2003) pp. 593–600.
- G. E. Hinton, in Parallel Models of Associative Memory, G. E. Hinton, J. A. Anderson, Eds. (Erlbaum, Hillsdale, NJ, 1981), pp. 161–187.
- M. Kawato, in The Handbook of Brain Theory and Neural Networks, M. Arbib, Ed. (MIT Press, Cambridge, MA, 1995), pp. 172–178.
- G. B. Stanley, Neurocomputing38–40, 1703 (2001).
Cited by
Items 801–900 of 1,009
Randomly Connected Networks Have Short Temporal Memory
Wallace, E. W. J., Maei, H. R., & Latham, P. E. (2013) · Neural Computation · doi:10.1162/neco_a_00449 · Cited by 40
“Using an optimal linear classifier to provide long temporal memory is the hope of the liquid state machine of Maass et al. (2002) and Bertschinger and Natschläger (2004) and of the echo state network of Jaeger and colleagues (Jaeger, 2001; Jaeger & Haas, 2004).”
MethodDoes money matter in inflation forecasting?
Binner, J. M., Peter Tiňo, B., Tepper, J. A., Anderson, R. G., Jones, B. E., & Kendall, G. (2010) · Physica A Statistical Mechanics and its Applications · doi:10.1016/j.physa.2010.06.015 · Cited by 37
The Use of Hebbian Cell Assemblies for Nonlinear Computation
Tetzlaff, C., Dasgupta, S., Kulvičius, T., & Wörgötter, F. (2015) · Scientific Reports · doi:10.1038/srep12866 · Cited by 43
“Computational reservoirs are commonly generated using random connectivity of variable strength between many units 9,10 and, as a result, weak and strong connections are randomly distributed therein.”
InfluentialDynamical regularized echo state network for time series prediction
Yang, C., Qiao, J., Wang, L., & Zhu, X. (2018) · Neural Computing and Applications · doi:10.1007/s00521-018-3488-z · Cited by 37
“In previous works [8, 17, 25], the initial W is always scaled to satisfy the necessary condition for ESP, while the sufficient condition is ignored which is more restrictive.”
BackgroundMethodInfluentialNiño, C. H. V., Vellasco, M. M. B. R., & Figueiredo, K. (2023) · Neurocomputing · doi:10.1016/j.neucom.2023.126317 · Cited by 43
The Real Implementation of ANFIS Channel Equalizer on the System of Software-Defined Radio
Martínek, R., & Žídek, J. (2014) · IETE Journal of Research · doi:10.1080/03772063.2014.914698 · Cited by 36
“Radial basis function network (RBFN) equalizer [20] has received a great deal of attention because it is structurally simple and more ef fi cient to learn compared to MLP [21].”
ResultScaling Up Echo-State Networks With Multiple Light Scattering
Dong, J., Gigan, S., Krząkała, F., & Wainrib, G. (2018) · IEEE Statistical Signal Processing Workshop (SSP) · doi:10.1109/ssp.2018.8450698 · Cited by 38
“∗This research has received funding from the European Research Council under the EUs 7th Framework Programme (FP/2007-2013/ERC Grant Agreement 307087-SPARCS and 278025-COMEDIA) Neural Network where neurons are connected with random weights [5, 6, 7].”
MethodAugmenting Recurrent Neural Networks Resilience by Dropout
Bacciu, D., & Crecchi, F. (2019) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2019.2899744 · Cited by 37
“adaptive recurrent models that are Elman’s SRNs [24] and the LSTM gated networks [20], and a partially adaptive one, the ESNs [21].”
BackgroundMethodInfluentialEcho state networks regulated by local intrinsic plasticity rules for regression
Wang, X., Jin, Y., & Hao, K. (2019) · Neurocomputing · doi:10.1016/j.neucom.2019.03.032 · Cited by 36
Fault prognostics by an ensemble of Echo State Networks in presence of event based measurements
Xu, M., Baraldi, P., Al‐Dahidi, S., & Zio, E. (2019) · Engineering Applications of Artificial Intelligence · doi:10.1016/j.engappai.2019.103346 · Cited by 38
Using machine learning to assess short term causal dependence and infer network links
Banerjee, A., Pathak, J., Roy, R., Restrepo, J. G., & Ott, E. (2019) · Chaos An Interdisciplinary Journal of Nonlinear Science · doi:10.1063/1.5134845 · Cited by 37
“(2) satisfies the “echo state property” 21,27,29 (in nonlinear dynamics, this condition is also known as “generalized synchronization” 32–34 ): given two different initial reservoir states, r 1 ∗ and r 2 ∗ , for the same input time series of z , the difference between the two corresponding…”
InfluentialPredicting critical transitions in multiscale dynamical systems using reservoir computing
Lim, S. H., Giorgini, L. T., Moon, W., & Wettlaufer, J. S. (2020) · Chaos An Interdisciplinary Journal of Nonlinear Science · doi:10.1063/5.0023764 · Cited by 47
Numerical simulation of artificial spin ice for reservoir computing
Hon, K., Kuwabiraki, Y., Goto, M., Nakatani, R., Suzuki, Y., & Nomura, H. (2021) · Applied Physics Express · doi:10.35848/1882-0786/abdcd8 · Cited by 44
Echo state network activation function based on bistable stochastic resonance
Liao, Z., Wang, Z., Yamahara, H., & Tabata, H. (2021) · Chaos Solitons & Fractals · doi:10.1016/j.chaos.2021.111503 · Cited by 40
Learning strange attractors with reservoir systems
Grigoryeva, L., Hart, A., & Ortega, J. (2023) · Nonlinearity · doi:10.1088/1361-6544/ace492 · Cited by 54
“…(RC) [Luko 09, Tana 19] in the learning, forecasting [Jaeg 04, Path 17, Path 18, Lu 18], and classification [Carr 18] of chaotic attractors of complex nonlinear high-dimensional dynamical systems strongly suggests that these machine learning paradigms have Takens embedding-type properties.”
InfluentialWaves traveling over a map of visual space can ignite short-term predictions of sensory input
Benigno, G., Budzinski, R. C., Davis, Z. W., Reynolds, J. H., & Muller, L. (2023) · Nature Communications · doi:10.1038/s41467-023-39076-2 · Cited by 59
“This procedure is analogous to a complex-valued implementation of the reservoir computing paradigm 42 , which has recently found wide applications in nonlinear dynamics and physics.”
BackgroundMethodA prefrontal-thalamic circuit encodes social information for social recognition
Chen, Z., Han, Y., Ma, Z., Wang, X., Xu, S., Tang, Y. Y., Vyssotski, A. L., Si, B., & Zhan, Y. (2024) · Nature Communications · doi:10.1038/s41467-024-45376-y · Cited by 42
“Computational models using reciprocally connected echo-state or reservoir networks 47,48 that synaptic modi fi cations to the projection subpopulations are critical for pattern representation 49,50 .”
MethodPanahi, S., & Lai, Y. (2024) · Chaos An Interdisciplinary Journal of Nonlinear Science · doi:10.1063/5.0200898 · Cited by 47
Controlling chaos using edge computing hardware
Kent, R. M., Barbosa, W. A. S., & Gauthier, D. J. (2024) · Nature Communications · doi:10.1038/s41467-024-48133-3 · Cited by 39
“(12).”
Multisensor Data Fusion for Activity Recognition Based on Reservoir Computing
Palumbo, F., Barsocchi, P., Gallicchio, C., Chessa, S., & Micheli, A. (2013) · Communications in computer and information science · doi:10.1007/978-3-642-41043-7_3 · Cited by 41
“Reservoir parameters are left untrained after random initialization under the constraints given by the Echo State Property (ESP) [5, 6].”
BackgroundMethodDasgupta, S., Goldschmidt, D., Wörgötter, F., & Manoonpong, P. (2015) · Frontiers in Neurorobotics · doi:10.3389/fnbot.2015.00010 · Cited by 35
“Here we used a modified version of the original recursive least squares (RLS) algorithm (Simon, 2002; Jaeger and Haas, 2004) based on the FORCE learning formulation (Sussillo and Abbott, 2009), in order to learn the reservoir-to-readout connection weights W out at each time step, while the CPG input…”
BackgroundMethodRobotic Ubiquitous Cognitive Ecology for Smart Homes
Amato, G. M., Bacciu, D., Broxvall, M., Chessa, S., Coleman, S., Rocco, M. D., Dragone, M., Gallicchio, C., Gennaro, C., Lozano, H., et al. (2015) · Journal of Intelligent & Robotic Systems · doi:10.1007/s10846-015-0178-2 · Cited by 42
“RUBICON puts forward the use of RC models, in particular of the Echo State Network (ESN) [35, 36], as building block of a distributed ecology learning system with application to Ambient Assisted Living (AAL).”
MethodInfluentialDominey, P. F. (2013) · Frontiers in Psychology · doi:10.3389/fpsyg.2013.00500 · Cited by 36
“Subsequent work with such systems demonstrated their vast computational power (Maass et al., 2002; Jaeger and Haas, 2004).”
BackgroundVirtualization of a Photonic Reservoir Computer
Duport, F., Smerieri, A., Akrout, A., Haelterman, M., & Massar, S. (2016) · Journal of Lightwave Technology · doi:10.1109/jlt.2016.2524559 · Cited by 42
“REFERENCES [1] H. Jaeger and H. Haas, “Harnessing nonlinearity: Predicting chaotic systems and saving energy in wireless communication,” Science, vol. 304, no. 5667, pp. 78–80, 2004.”
BackgroundInfluentialRobust echo state networks based on correntropy induced loss function
Guo, Y., Wang, F., Chen, B., & Xin, J. (2017) · Neurocomputing · doi:10.1016/j.neucom.2017.05.087 · Cited by 35
Randomized Machine Learning Approaches: Recent Developments and Challenges
Gallicchio, C., Martín‐Guerrero, J. D., Micheli, A., & Soria‐Olivas, E. (2017) · CINECA IRIS Institutial research information system (University of Pisa) · Cited by 35
Time series classification with Echo Memory Networks
Ma, Q., Zhuang, W., Shen, L., & Cottrell, G. W. (2019) · Neural Networks · doi:10.1016/j.neunet.2019.05.008 · Cited by 35
Data-driven reconstruction of nonlinear dynamics from sparse observation
Yeo, K. (2019) · Journal of Computational Physics · doi:10.1016/j.jcp.2019.06.039 · Cited by 35
Optimal short-term memory before the edge of chaos in driven random recurrent networks
Haruna, T., & Nakajima, K. (2019) · Physical review. E · doi:10.1103/physreve.100.062312 · Cited by 39
Guo, X., Xiang, S., Zhang, Y. H., Lin, L., Wen, A. J., & Hao, Y. (2020) · IEEE Journal of Selected Topics in Quantum Electronics · doi:10.1109/jstqe.2020.2987077 · Cited by 40
“only the output weights are modified, and the input and reservoir weights are fixed with random values [3].”
BackgroundPerformance Improvement of Chaotic Baseband Wireless Communication Using Echo State Network
Ren, H., Yin, H., Bai, C., & Yao, J. (2020) · IEEE Transactions on Communications · doi:10.1109/tcomm.2020.3007757 · Cited by 38
“ESN provides a unique nonlinear transformation from input signal to their output, which can be used for prediction or classification [46]–[48].”
BackgroundMethodMultiresolution Reservoir Graph Neural Network
Pasa, L., Navarin, N., & Sperduti, A. (2021) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2021.3090503 · Cited by 41
Neural Schrödinger Equation: Physical Law as Deep Neural Network
Nakajima, M., Tanaka, K., & Hashimoto, T. (2021) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2021.3120472 · Cited by 43
Nonlinear Model Predictive Control of Electrical Submersible Pumps based on Echo State Networks
Jordanou, J. P., Osnes, I., Hernes, S. B., Camponogara, E., Antonelo, E. A., & Imsland, L. (2022) · Advanced Engineering Informatics · doi:10.1016/j.aei.2022.101553 · Cited by 35
“In this work, ESNs are used with NMPC strategies to achieve control of unknown nonlinear plants, more specifically, of ESP-lifted oil wells.”
InfluentialGrowing Echo State Network With an Inverse-Free Weight Update Strategy
Chen, X., Luo, X., Jin, L., Li, S., & Liu, M. (2022) · IEEE Transactions on Cybernetics · doi:10.1109/tcyb.2022.3155901 · Cited by 35
“As an alternative to the RNN, the echo state network (ESN) is provided to reduce the cost of computation [11], [12].”
MethodModeling chaotic systems: Dynamical equations vs machine learning approach
Weng, T., Yang, H., Zhang, J., & SMALL, M. P. (2022) · Communications in Nonlinear Science and Numerical Simulation · doi:10.1016/j.cnsns.2022.106452 · Cited by 35
Dynamic Nonlinear Behavior of Ionic Liquid-Based Reservoir Computing Devices
Matsuo, T., Sato, D., Koh, S.-G., Shima, H., Naitoh, Y., Akinaga, H., Itoh, T., Nokami, T., Kobayashi, M., & Kinoshita, K. (2022) · ACS Applied Materials & Interfaces · doi:10.1021/acsami.2c04167 · Cited by 37
Reservoir computing based on an external-cavity semiconductor laser with optical feedback modulation
Kanno, K., Haya, A. A., & Uchida, A. (2022) · Optics Express · doi:10.1364/oe.460016 · Cited by 43
A Coupled Spintronics Neuromorphic Approach for High‐Performance Reservoir Computing
Akashi, N., Kuniyoshi, Y., Tsunegi, S., Taniguchi, T., Nishida, M., Sakurai, R., Wakao, Y., Kawashima, K., & Nakajima, K. (2022) · Advanced Intelligent Systems · doi:10.1002/aisy.202200123 · Cited by 37
Echo state network with multiple delayed outputs for multiple delayed time series prediction
Yao, X., Shao, Y., Fan, S., & Cao, S. (2022) · Journal of the Franklin Institute · doi:10.1016/j.jfranklin.2022.09.059 · Cited by 35
Information Processing Capacity of Spintronic Oscillator
Tsunegi, S., Kubota, T., Kamimaki, A., Grollier, J., Cros, V., Yakushiji, K., Fukushima, A., Yuasa, S., Kubota, H., Nakajima, K., et al. (2023) · Advanced Intelligent Systems · doi:10.1002/aisy.202300175 · Cited by 46
Trading strategy design in financial investment through a turning points prediction scheme
Li, X., Deng, Z., & Luo, J. (2008) · Expert Systems with Applications · doi:10.1016/j.eswa.2008.11.014 · Cited by 35
“Furthermore, greater success has been achieved in next-value prediction of financial time series by harnessing the chaotic characteristics intrinsic in them (Chen et al., 2005; Jaeger & Haas, 2004).”
BackgroundLearning of Temporal Motor Patterns: An Analysis of Continuous Versus Reset Timing
Laje, R., Cheng, K. Y., & Buonomano, D. V. (2011) · Frontiers in Integrative Neuroscience · doi:10.3389/fnint.2011.00061 · Cited by 39
“The model used the general random recurrent network model composed of firing-rate (non-spiking) units (Sompolinsky et al., 1988; Jaeger and Haas, 2004; Sussillo and Abbott, 2009).”
BackgroundMethodInfluentialKlein, B., Meißner, D., Kobialka, H., & Reggiani, P. (2016) · Water · doi:10.3390/w8040125 · Cited by 39
“RNNs are able to model the input-output behavior of dynamical systems which have states evolving over time.”
MethodInfluentialCerebellum-Inspired Learning and Control Scheme for Redundant Manipulators at Joint Velocity Level
Jin, L., Huang, R., Liu, M., & Ma, X. (2024) · IEEE Transactions on Cybernetics · doi:10.1109/tcyb.2024.3436021 · Cited by 35
“Meanwhile, as a network widely known for its simple structure and ability to handle spatio–temporal problems effectively [25], [26], [27], [28], the echo state network (ESN) shows similarities to the cerebellum.”
BackgroundShayani, H., Bentley, P. J., & Tyrrell, A. M. (2008) · NASA/ESA Conference on Adaptive Hardware and Systems (AHS) · doi:10.1109/ahs.2008.13 · Cited by 34
“Recently, with independent works of Buonomano [10], Maass (Liquid State Machine - LSM [11]), Jaeger (Echo State Network - ESN [12]) and Steil (Back-Propagation Decorrelation - BPDC [13]) a new technique, collectively known as Reservoir Computing (RC) [14], emerged, which is claimed to be capable of processing analogue…”
MethodGenerating probabilistic predictions using mean-variance estimation and echo state network
Yao, W., Zeng, Z., & Lian, C. (2016) · Neurocomputing · doi:10.1016/j.neucom.2016.09.064 · Cited by 35
An Adaptive Activated ANN Equalizer Applied in Millimeter-Wave RoF Transmission System
Liu, S., Wang, X., Zhang, W., Shen, G., & Tian, H. (2017) · IEEE Photonics Technology Letters · doi:10.1109/lpt.2017.2753174 · Cited by 34
“ANN methods, such as recurrent neural networks, are employed to equalize the communication channels [7].”
MethodPSO-based growing echo state network
Li, Y., & Li, F. (2019) · Applied Soft Computing · doi:10.1016/j.asoc.2019.105774 · Cited by 34
Ray, A., Chakraborty, T., & Ghosh, D. (2021) · Chaos An Interdisciplinary Journal of Nonlinear Science · doi:10.1063/5.0074213 · Cited by 41
Self-Adaptive Particle Swarm Optimization-Based Echo State Network for Time Series Prediction
Xue, Y., Zhang, Q., & Neri, F. (2021) · International Journal of Neural Systems · doi:10.1142/s012906572150057x · Cited by 37
Mapping broadband electrocorticographic recordings to two-dimensional hand trajectories in humans
Gunduz, A., Sanchez, J. C., Carney, P. R., & Príncipe, J. C. (2009) · Neural Networks · doi:10.1016/j.neunet.2009.06.036 · Cited by 36
“First,we varied how finelywedivided the ECoG spectrum.”
BackgroundInfluentialPredicting Multivariate Time Series Using Subspace Echo State Network
Han, M., & Xu, M. (2013) · Neural Processing Letters · doi:10.1007/s11063-013-9324-7 · Cited by 34
“Echo state network (ESN) is a significant development in NNs [8].”
Training Passive Photonic Reservoirs With Integrated Optical Readout
Freiberger, M., Katumba, A., Bienstman, P., & Dambre, J. (2018) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2018.2874571 · Cited by 41
“voirs to numbers common in classic echo state networks [3].”
BackgroundSiri, B., Berry, H., Cessac, B., Delord, B., & Quoy, M. (2008) · Neural Computation · doi:10.1162/neco.2008.05-07-530 · Cited by 36
“More recently, Echo State Networks (ESN) (Jaeger and Haas, 2004) have been developed, where, as in our case, the network acts as a reservoir of resonant frequencies.”
Backgroundwyffels, F., Schrauwen, B., Verstraeten, D., & Stroobandt, D. (2008) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn.2008.4634252 · Cited by 47
Reinforcement learning of linking and tracing contours in recurrent neural networks
Brosch, T., Neumann, H., & Roelfsema, P. R. (2015) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1004489 · Cited by 33
“The learning of stable states of the network as implemented by RELEARNN differs from other recurrent neural network learning schemes, such as backpropagation through time (BPTT; [112]), recirculation algorithms [113–115] and reservoir computing [116, 117].”
BackgroundMethodA Dynamic Memory for Reservoir Computing Utilizing Ion Migration in CuInP2S6
Wu, Y., Duong, N. T., Chien, Y., Liu, S., & Ang, K. (2023) · Advanced Electronic Materials · doi:10.1002/aelm.202300481 · Cited by 37
Wang, H., & Yan, X. (2014) · Neurocomputing · doi:10.1016/j.neucom.2014.05.024 · Cited by 33
Boccato, L., Lopes, A., Attux, R., & Zuben, F. J. V. (2011) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn.2011.6033273 · Cited by 33
“The channel memory has a deep influence on the required equalizer memory, both for linear and nonlinear models, and in this context the use of recurrent structures arises in a natural way.”
BackgroundCui, Y., Surpur, C., Ahmad, S., & Hawkins, J. (2016) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn.2016.7727380 · Cited by 52
“Echo state networks (ESN) use a randomly connected recurrent network as a dynamics reservoir and model a sequence as trainable linear combination of these response signals [3].”
BackgroundEfficient parallel implementation of reservoir computing systems
Alomar, M. L., Skibinsky-Gitlin, E. S., Frasser, C. F., Canals, V., Isern, E., Roca, M., & Rosselló, J. L. (2018) · Neural Computing and Applications · doi:10.1007/s00521-018-3912-4 · Cited by 35
Enhanced Echo-State Restricted Boltzmann Machines for Network Traffic Prediction
Sun, X., Ma, S., Li, Y., Wang, D., Li, Z., Wang, N., & Gui, G. (2019) · IEEE Internet of Things Journal · doi:10.1109/jiot.2019.2954283 · Cited by 36
“In order to validate the outstanding performance of eERBM in this task, we also compare it with three ESN paradigm, including the hybrid reservoir activation model (HCR) [40], the simplified cycle reservoir structure (SCR) [52], and the original model (OESN) [43].”
BackgroundInfluentialPerformance of Ag–Ag 2 S core–shell nanoparticle-based random network reservoir computing device
Hadiyawarman, Usami, Y., Kotooka, T., Azhari, S., Eguchi, M., & Tanaka, H. (2021) · Japanese Journal of Applied Physics · doi:10.35848/1347-4065/abe206 · Cited by 42
Photonic extreme learning machine based on frequency multiplexing
Lupo, A., Butschek, L., & Massar, S. (2021) · Optics Express · doi:10.1364/oe.433535 · Cited by 44
“We also considered the Nonlinear Channel Equalization problem[11], which is well known in the Reservoir Computer community and is described below.”
MethodHoward, J. M., Wang, Q., Srivastava, M., Gong, T., Lee, E., Abate, A., & Leite, M. S. (2022) · The Journal of Physical Chemistry Letters · doi:10.1021/acs.jpclett.2c00131 · Cited by 36
Adaptive time-delayed photonic reservoir computing based on Kalman-filter training
Jin, J., Jiang, N., Zhang, Y., Feng, W., Zhao, A., Liu, S., Peng, J., Qiu, K. L., & Zhang, Q. (2022) · Optics Express · doi:10.1364/oe.454852 · Cited by 38
“The RC has been extensively applied in many areas, such as financial time-series prediction, nonlinear channel equalization [3,4], human action recognition [5], and reconstruction of complex dynamic system [6], etc.”
BackgroundMethodDistributed Fault Detection in Sensor Networks using a Recurrent Neural Network
Obst, O. (2013) · Neural Processing Letters · doi:10.1007/s11063-013-9327-4 · Cited by 37
“An ESN is a specific type of recurrent neural network which is able to successfully predict complex time series [5].”
BackgroundMethodInfluentialDasgupta, S., Wörgötter, F., & Manoonpong, P. (2014) · Frontiers in Neural Circuits · doi:10.3389/fncir.2014.00126 · Cited by 35
“Inspired by the reservoir computing framework (Maass et al., 2002; Jaeger and Haas, 2004), here we use a chaotic random recurrent neural network (RNN) (Sussillo and Abbott, 2009; Rajan et al., 2010) as the adaptive critic (cortico-striatal circuitry and the DA system) connected to a feed-forward…”
MethodMiwa, T., Kanno, K., & Bunsen, M. (2016) · Japanese Journal of Applied Physics · doi:10.7567/jjap.55.08re06 · Cited by 33
Spiking Echo State Convolutional Neural Network for Robust Time Series Classification
Zhang, A., Zhu, W., & Li, J. (2018) · IEEE Access · doi:10.1109/access.2018.2887354 · Cited by 35
“It should be noted that to achieve the echo state property, the reservoir connection matrix W needs to be scaled by W ← αW/λmax(W ), where λmax(W ) denotes the spectral radius of W and 0 < α < 1 is a scaling parameter [16].”
BackgroundMethodLearn more by training less: systematicity in sentence processing by recurrent networks
Frank, S. L. (2006) · Connection Science · doi:10.1080/09540090600768336 · Cited by 33
“This turns the SRN into a fairly new type of recurrent network, the so-called echo state network (ESN; Jaeger 2003, Jaeger and Haas 2004), which will be shown to handle weak combinatorial systematicity better than does the SRN.”
BackgroundMethodGranular vortex spin-torque nano oscillator for reservoir computing
Shreya, S., Jenkins, A., Rezaeiyan, Y., Li, R., Böhnert, T., Benetti, L., Ferreira, R., Moradi, F., & Farkhani, H. (2023) · Scientific Reports · doi:10.1038/s41598-023-43923-z · Cited by 40
Ren, H., Li, Y., Li, M., Gao, M., Lu, J., Zou, C., Dong, C., Yu, P., Yang, X., & Xuan, Q. (2024) · Optics Express · doi:10.1364/oe.518063 · Cited by 42
Markovian Bias of Neural-based Architectures With Feedback Connections
Peter Tiňo, B., Hammer, B., & Mikael Bodén, A. (2007) · Studies in computational intelligence · doi:10.1007/978-3-540-73954-8_5 · Cited by 33
“A similar architecture has been proposed by Jaeger in [27, 28].”
MethodShort-term memory in neuronal networks through dynamical compressed sensing
Ganguli, S., & Sompolinsky, H. (2010) · Cited by 32
Echo State Network for Fast Channel Prediction in Ricean Fading Scenarios
Zhao, Y., Gao, H., Beaulieu, N. C., Chen, Z., & Ji, H. (2016) · IEEE Communications Letters · doi:10.1109/lcomm.2016.2632120 · Cited by 32
Model forecasting based on two-stage feature selection procedure using orthogonal greedy algorithm
Jiang, H. (2017) · Applied Soft Computing · doi:10.1016/j.asoc.2017.11.047 · Cited by 32
Wlaźlak, E., Marzec, M. M., Zawal, P., & Szacilowski, K. T. (2019) · ACS Applied Materials & Interfaces · doi:10.1021/acsami.9b01841 · Cited by 32
Li, Q., Wu, Z., & Zhang, H. (2020) · Journal of Cleaner Production · doi:10.1016/j.jclepro.2020.121151 · Cited by 34
Bompas, S., Georgeot, B., & Guéry-Odelin, D. (2020) · Chaos An Interdisciplinary Journal of Nonlinear Science · doi:10.1063/5.0021264 · Cited by 35
Deng, L., & Pan, Y. (2020) · SPE Reservoir Evaluation & Engineering · doi:10.2118/200862-pa · Cited by 32
Echo State Network Based Soft Sensor for Monitoring and Fault Detection of Industrial Processes
Lemos, T. S. M., Campos, L. F. D. O., Melo, A., Clavijo, N., Soares, R. M., Câmara, M. M., Feital, T. D. S., Anzai, T. K., & Pinto, J. C. (2021) · Computers & Chemical Engineering · doi:10.1016/j.compchemeng.2021.107512 · Cited by 39
Cancer progression as a learning process
Shomar, A., Barak, O., & Brenner, N. (2022) · iScience · doi:10.1016/j.isci.2022.103924 · Cited by 40
“It draws from classic work on learning in neuroscience (Jaeger and Haas, 2004; Maass et al., 2007; Sussillo and Abbott, 2009).”
BackgroundEcho state network with logistic mapping and bias dropout for time series prediction
Wang, H., Liu, Y.-X., Lu, P., Luo, Y., Wang, D., & Xu, X. (2022) · Neurocomputing · doi:10.1016/j.neucom.2022.03.018 · Cited by 33
Direct data-driven forecast of local turbulent heat flux in Rayleigh–Bénard convection
Pandey, S., Teutsch, P., Mäder, P., & Schumacher, J. (2022) · Physics of Fluids · doi:10.1063/5.0087977 · Cited by 33
Decentralized Federated Learning for Industrial IoT With Deep Echo State Networks
Qiu, W., Ai, W., Chen, H., Feng, Q., & Tang, G. (2022) · IEEE Transactions on Industrial Informatics · doi:10.1109/tii.2022.3194627 · Cited by 36
“In this way, the system becomes chaotic [26].”
BackgroundLearning unseen coexisting attractors
Gauthier, D. J., Fischer, I., & Röhm, A. (2022) · Chaos An Interdisciplinary Journal of Nonlinear Science · doi:10.1063/5.0116784 · Cited by 38
Input Dependent Cell Assembly Dynamics in a Model of the Striatal Medium Spiny Neuron Network
Ponzi, A. P. D., & Wickens, J. R. (2012) · Frontiers in Systems Neuroscience · doi:10.3389/fnsys.2012.00006 · Cited by 36
“Moreover the dual encoding of stimulus identity and elapsed time in a single spiking network (Buonomano and Merzenich, 1995; Maass et al., 2002; Jaeger and Hass, 2004; Karmarkar and Buonomano, 2007; Buonomano and Maass, 2009) may be relevant in the temporal credit assignment tasks which are thought to deeply involve t…”
BackgroundVerplancke, T., Looy, S. V., Steurbaut, K., Benoît, D., Turck, F. D., Moor, G. D., & Decruyenaere, J. M. A. (2010) · BMC Medical Informatics and Decision Making · doi:10.1186/1472-6947-10-4 · Cited by 32
“telecommunication [1] and robotics [2], as well as in linguistics to detect grammatical structure [3].”
Embodiment design of soft continuum robots
Kang, R., Guglielmino, E., Zullo, L., Branson, D. T., Godage, I. S., & Caldwell, D. G. (2016) · Advances in Mechanical Engineering · doi:10.1177/1687814016643302 · Cited by 33
“Using this body dynamics as a resource, we have shown that it is possible to emulate complex nonlinear dynamic systems(8,9) and to embed the closed-loop control into the arm(8) based on the reservoir computing approach.(54,55) It has been established that to have computational capabilities, a reservoir should have the…”
BackgroundMethodEffects of Spectral Radius on Echo-State-Network's Training
Wang, Y., Ni, J., & Xu, Z. (2009) · International Conference on Internet Computing for Science and Engineering · doi:10.1109/icicse.2009.69 · Cited by 31
Ruan, Z., Sun, W., Yuan, Y., & Tan, H.-P. (2023) · Renewable and Sustainable Energy Reviews · doi:10.1016/j.rser.2023.113528 · Cited by 31
An energy efficient reservoir computing system based on HZO memcapacitive devices
Zhang, P., Ma, X., Dong, Y., Wu, Z., Chen, D., Cui, T., Liu, J., Liu, G., & Li, X. (2023) · Applied Physics Letters · doi:10.1063/5.0164762 · Cited by 31
Model scale versus domain knowledge in statistical forecasting of chaotic systems
Gilpin, W. (2023) · Physical Review Research · doi:10.1103/physrevresearch.5.043252 · Cited by 65
“…computers as strong forecasting methods for dynamical systems [13– 19]; these models use emergent properties of random networks to lift complex time series into a random fea-∗ wgilpin@utexas.edu ture space, thereby simplifying learning at the expense of increasing model complexity [20, 21].”
BackgroundMethodResultInfluentialA cognitive robotic ecology approach to self-configuring and evolving AAL systems
Dragone, M., Amato, G. M., Bacciu, D., Chessa, S., Coleman, S., Rocco, M. D., Gallicchio, C., Gennaro, C., Lozano, H., Maguire, L., et al. (2015) · Engineering Applications of Artificial Intelligence · doi:10.1016/j.engappai.2015.07.004 · Cited by 33
“Based on such considerations, the learning modules have been realized by Echo State Networks (ESNs) (Jaeger and Haas, 2004), a recurrent neural model from the Reservoir Computing paradigm, characterized by a good tradeoff between computational efficiency and ability to deal with dynamic systems and noisy data.”
BackgroundMethodDesign of a Central Pattern Generator Using Reservoir Computing for Learning Human Motion
wyffels, F., & Schrauwen, B. (2009) · Advanced Technologies for Enhanced Quality of Life (AT-EQUAL) · doi:10.1109/at-equal.2009.32 · Cited by 40
“RC has proven its qualities in a broad range of applications such as robot localization [1], speech recognition [18] and time series generation [10].”
BackgroundForecasting industrial aging processes with machine learning methods
Bogojeski, M., Sauer, S., Horn, F., & Müller, K. (2020) · Computers & Chemical Engineering · doi:10.1016/j.compchemeng.2020.107123 · Cited by 34
Breaking Wireless Propagation Environmental Uncertainty With Deep Learning
Morocho-Cayamcela, M. E., Maier, M., & Lim, W. (2020) · IEEE Transactions on Wireless Communications · doi:10.1109/twc.2020.2986202 · Cited by 33
Periodic structure of memory function in spintronics reservoir with feedback current
Yamaguchi, T., Akashi, N., Tsunegi, S., Kubota, H., Nakajima, K., & Taniguchi, T. (2020) · Physical Review Research · doi:10.1103/physrevresearch.2.023389 · Cited by 34
Metadata
- Authors
- Jaeger, H., & Haas, H.
- Affiliations
- Constructor University (DE)
- Corresponding author
- Jaeger, Herbert
- Venue
- Science
- Published
- 2004-04-02
- Volume
- 304
- Issue
- 5667
- Pages
- 78-80
- ISSN
- 0036-8075 (ISSN-L)
- 1095-9203
- OpenAlex
- W2118706537
- Cited by
- 3,811
- References
- 18
- Citations by year
- 2026: 257 · 2025: 314 · 2024: 315 · 2023: 332 · 2022: 300 · 2021: 355 · 2020: 259 · 2019: 309 · 2018: 237 · 2017: 160 · 2016: 142 · 2015: 128 · 2014: 120 · 2013: 107 · 2012: 100
- FWCI
- 33.93
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Science (publishedVersion)
- PubMed (publishedVersion)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Neural Networks and Reservoir Computing
- Physical Sciences › Computer Science › Artificial Intelligence › Neural Networks and Applications
- Physical Sciences › Physics and Astronomy › Statistical and Nonlinear Physics › Chaos control and synchronization
- Keywords
- echo state networks, chaotic time series prediction, recurrent neural networks, channel equalization, wireless communication, energy efficiency, reservoir computing
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.