RKHS reconstruction based on manifold learning for high-dimensional data

Abstract

Kernel trick has achieved remarkable success in various machine learning tasks, especially those with high-dimensional non-linear data. In addition, these data usually tend to have compact representation that cluster in a low-dimensional subspace. In order to offer a general and comprehensive framework for high-dimensional non-linear data, in this paper, we generalizes multiple kernel learning and subspace learning in a reconstructed reproducing kernel Hilbert space (RKHS) endowed with manifold leaning. First, we construct reconstructed kernels by fusing manifold learning and some base kernel functions, and then learn the optimal kernel by linearly combining the reconstructed kernels. The proposed MKL method can introduce different prior knowledge such as neighborhood information and classification information, to solve different tasks of high-dimensional data. Furthermore, we propose a subspace learning based on RKHS reconstruction, named MVSL for short, of which the objective function is designed with variance maximization criterion, and use an iterative algorithm to solve it. We also incorporates data discriminant information to the learning process of the modified kernel by kernel alignment criterion and a regularization term, to learning the optimal kernel matrix for RKHS reconstruction, and propose another subspace learning method, named Discriminative MVSL. Experimental results on toy and real-world datasets demonstrate that the proposed MKL and subspace learning methods are able to learn the local manifold and the global statistics information of data based on RKHS reconstruction, and thus they achieve a satisfactory performance on classification and dimension reduction tasks.