A Novel Feature Extraction Method and Its Relationships with PCA and KPCA

Abstract

A new feature extraction method for high dimensional data using least squares support vector regression (LSSVR) is presented. Firstly, the expressions of optimal projection vectors are derived into the same form as that in the LSSVR algorithm by specially extending the feature of training samples. So the optimal projection vectors could be obtained by LSSVR. Then, using the kernel tricks, the data are mapped from the original input space to a high dimensional feature, and nonlinear feature extraction is here realized from linear version. Finally, it is proved that 1) the method presented has the same result as principal component analysis (PCA). 2) This method is more suitable for the higher dimensional input space compared. 3) The nonlinear feature extraction of the method is equivalent to kernel principal component analysis (KPCA).