Local linear approximation of principal curve projections

Abstract

In previous work we introduced principal surfaces as hyperridges of probability distributions in a differential geometrical sense. Specifically, given an n-dimensional probability distribution over real-valued random vectors, a point on the d-dimensional principal surface is a local maximizer of the distribution in the subspace orthogonal to the principal surface at that point. For twice continuously differentiable distributions, the surface is characterized by the gradient and the Hessian of the distribution. Furthermore, the nonlinear projections of data points to the principal surface for dimension reduction is ideally given by the solution trajectories of differential equations that are initialized at the data point and whose tangent vectors are determined by the Hessian eigenvectors. In practice, data dimension reduction using numerical integration based differential equation solvers are found to be computationally expensive for most machine learning applications. Consequently, in this paper, we propose a local linear approximation to achieve this dimension reduction without significant loss of accuracy while reducing computational complexity. The proposed method is demonstrated on synthetic datasets.