Nonlinear Dimensionality Reduction by Unit Ball Embedding (UBE) and Its Application to Image Clustering

Abstract

The paper presents an unsupervised nonlinear dimensionality reduction algorithm called Unit Ball Embedding (UBE). Many high-dimensional data, such as object or face images, lie on a union of low-dimensional subspaces which are often called manifolds. The proposed method is able to learn the structure of these manifolds by exploiting the local neighborhood arrangement around each point. It tries to preserve the local structure by minimizing a cost function that measures the discrepancy between similarities of points in the high-dimensional data and similarities of points in the low-dimensional embedding. The cost function is proposed in a way that it provides a hyper-spherical representation of points in the low-dimensional embedding. Visualizations of our method on different datasets show that it creates large gaps between the manifolds and maximizes the separability of them. As a result, it notably improves the quality of unsupervised machine learning tasks (e.g. clustering). UBE is successfully applied on image datasets such as faces, handwritten digits, and objects and the results of clustering on the low-dimensional embedding show significant improvement over existing dimensionality reduction methods.