Associative recall of memory without errors

Abstract

A neural network which is capable of recalling without errors any set of linearly independent patterns is studied. The network is based on a Hamiltonian version of the model of Personnaz et al. The energy of a state of N (\ifmmode\pm\else\textpm\fi{}1) neurons is the square of the Euclidean distance---in phase space---between the state and the linear subspace spanned by the patterns. This energy corresponds to nonlocal updatings of the synapses in the learning mode. Results of the mean-field theory (MFT) of the system as well as computer simulations are presented. The stable and metastable states of the network are studied as a function of ``temperature'' T and \ensuremath{\alpha}=p/N, where p is the number of embedded patterns. The maximum capacity of the network is \ensuremath{\alpha}=1. For all \ensuremath{\alpha} (0\ensuremath{\le}\ensuremath{\alpha}<1) the embedded patterns are not only locally stable but are global minima of the energy. The patterns appear, as metastable states, below a temperature T=${T}_{M}$(\ensuremath{\alpha}). The temperature ${T}_{M}$(\ensuremath{\alpha}) decreases to zero as \ensuremath{\alpha}\ensuremath{\rightarrow}1. The spurious states of the network are studied in detail in the case of random uncorrelated patterns. At finite p, they are identical to the mixture states of Hopfield's model. At finite \ensuremath{\alpha}, a spin-glass phase exists as a metastable state. According to the replica symmetric MFT the spin-glass state becomes degenerate with the patterns at \ensuremath{\alpha}=${\ensuremath{\alpha}}_{g}$=1-2/\ensuremath{\pi} and disappears above it. Possible interpretations of this unusual result are discussed. The average radius of attraction R of the patterns has been determined by computer simulations, for sizes up to N=400. The value of R for 0<\ensuremath{\alpha}<1 depends on the details of the dynamics. Results for both parallel and serial dynamics are presented. In both cases R is unity (the largest distance in phase space by definition) at \ensuremath{\alpha}\ensuremath{\rightarrow}0 and decreases monotonically to zero as \ensuremath{\alpha}\ensuremath{\rightarrow}1. Contrary to the MFT, simulations have not revealed, so far, any singularity in the properties of the spurious states at an intermediate value of \ensuremath{\alpha}.