Kernel Principal Component Analysis

Abstract

. A new method for performing a nonlinear form of Principal Component Analysis is proposed. By the use of integral operator kernel functions, one can efficiently compute principal components in high-- dimensional feature spaces, related to input space by some nonlinear map; for instance the space of all possible d--pixel products in images. We give the derivation of the method and present experimental results on polynomial feature extraction for pattern recognition. 1 Introduction Principal Component Analysis (PCA) is a basis transformation to diagonalize an estimate of the covariance matrix of the data x k , k = 1; : : : ; `, x k 2 R N , P ` k=1 x k = 0, defined as C = 1 ` ` X j=1 x j x ? j : (1) The new coordinates in the Eigenvector basis, i.e. the orthogonal projections onto the Eigenvectors, are called principal components. In this paper, we generalize this setting to a nonlinear one of the following kind. Suppose we first map the data nonlinearly into a feature space...