Stochastic Stability Analysis of the Linear Continuous and Discrete PSO Models

Abstract

Particle swarm optimization (PSO) can be interpreted physically as a particular discretization of a stochastic damped mass-spring system. Knowledge of this analogy has been crucial to derive the PSO continuous model and to introduce different PSO family members including the generalized PSO (GPSO) algorithm, which is the generalization of PSO for any time discretization step. In this paper, we present the stochastic analysis of the linear continuous and generalized PSO models for the case of a stochastic center of attraction. Analysis of the GPSO second order trajectories is performed and clarifies the roles of the PSO parameters and that of the cost function through the algorithm execution: while the PSO parameters mainly control the eigenvalues of the dynamical systems involved, the mean trajectory of the center of attraction and its covariance functions with the trajectories and their derivatives (or the trajectories in the near past) act as forcing terms to update first and second order trajectories. The similarity between the oscillation center dynamics observed for different kinds of benchmark functions might explain the PSO success for a broad range of optimization problems. Finally, a comparison between real simulations and the linear continuous PSO and GPSO models is shown. As expected, the GPSO tends to the continuous PSO when time step approaches zero. Both models account fairly well for the dynamics (first and second order moments) observed in real runs. This analysis constitutes so far the most realistic attempt to better understand and approach the real PSO dynamics from a stochastic point of view.