IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews)

Abstract

—Action recognition with cluttered and moving background is a challenging problem. One main difficulty lies in the fact that the motion field in an action region is contaminated by the background motions. We propose a Hierarchical Filtered Motion (HFM) method to recognize actions in crowded videos by using Motion History Image (MHI) as basic representations of motion due to its robustness and efficiency. First, we detect interest points as the 2D Harris corners with recent motion, e.g. locations with high intensities in MHI. Then a global spatial motion smoothing filter is applied to the gradients of MHI to eliminate isolated unreliable or noisy motions. At each interest point, a local motion field filter is applied to the smoothed gradients of MHI by computing a structure proximity between any pixel in the local region and the interest point. Thus the motion at a pixel is enhanced or weakened based on its structure proximity with the interest point. To validate its effectiveness, we characterize the spatial and temporal features by Histograms of Oriented Gradient (HOG) in the intensity image and MHI respectively and use a Gaussian Mixture Model (GMM) based classifier for action recognition. The performance of the proposed approach achieves the state-of-the-art results on KTH dataset which has clean background. More importantly, we perform cross dataset action classification and detection experiments where KTH dataset is used for training while MSR Action Dataset II, which consists of crowded videos with people moving in the background, is used for testing. Our experiments show that the proposed hierarchical filtered motion method significantly outperforms existing techniques.