Stability of analog neural networks with delay

Abstract

Continuous-time analog neural networks with symmetric connections will always converge to fixed points when the neurons have infinitely fast response, but can oscillate when a small time delay is present. Sustained oscillation resulting from time delay is relevant to hardware implementations of neural networks where delay due to the finite switching speed of amplifiers can be appreciable compared to the network relaxation time. We analyze the dynamics of continuous-time analog networks with delay, and show that there is a critical delay above which a symmetrically connected network will oscillate. Two different stability analyses are presented for low and high neuron gain. The results are useful as design criteria for building fast but stable electronic networks. We find that for some connection topologies, a delay much smaller than the relaxation time can lead to oscillation, whereas for other topologies, including associative memory networks, even long delays will not produce oscillation. The most oscillation-prone network configuration is the all-inhibitory network; in this configuration, the critical delay for oscillation is smaller than the network relaxation time by a factor of N, the size of the network. Theoretical results are compared with numerical simulations and with experiments performed on a small (eight neurons) electronic network with controllable delay.