A New Parameterized Algorithm for Accurate Supervised Dimensionality Reduction

Abstract

Dimensionality reduction is a problem of fundamental importance in both machine learning and data mining. Previous research has shown that crucial global and local geometry of a dataset needs to be retained for accurate dimensionality reduction. In this paper, we develop a new algorithm that can efficiently capture both global and local features of a labeled dataset with high accuracy. We develop a new quadratic measure that can accurately describe the local features of a dataset and its parameters can be efficiently determined from the dataset. An optimization problem with multiple objectives is then formulated to take into account both global and local features in a dataset for supervised dimensionality reduction. We show that the optimization problem can be efficiently solved with a parameterized approach and the directions along which projections need to be performed to reduce the dimensionality can thus be determined. Our experimental results on benchmark data sets show that features crucial for classification can be accurately and efficiently retained by this approach and the results generated by this approach are more accurate than those based on a few other approaches for dimensionality reduction.