Learning Reservoir Dynamics with Temporal Self-Modulation

Abstract

Reservoir computing (RC) can efficiently process time-series data by transferring the input signal to randomly connected recurrent neural networks (RNNs), which are referred to as a reservoir. The high-dimensional representation of time-series data in the reservoir significantly simplifies subsequent learning tasks. Although this simple architecture allows fast learning and facile physical implementation, the learning performance is inferior to that of other state-of-the-art RNN models. In this study, to improve the learning ability of RC, we propose self-modulated RC (SM-RC) that extends RC by adding a self-modulation mechanism. We demonstrated that SM-RC can perform attention tasks where input information is retained or discarded depending on the input signal. We also found that a chaotic state emerged as a result of learning in SM-RC. Furthermore, SM-RC significantly outperformed RC in NARMA and Lorentz model tasks. Because the SM-RC architecture only requires two additional gates, it is physically implementable as RC, thereby providing a new direction for realizing edge AI.