Sparse dimensionality reduction based on compressed sensing

Abstract

In this paper, we propose a novel approach SDR-CS (Sparse Dimensionality Reduction based on CS) based on compressed sensing to reduce dimensionality. With certain constraint of objective function, our semi-supervised learning method utilizes instance to construct the optimally sparse dictionary in the training dataset, employs K-SVD and OMP algorithms to improve the convergence rate of learning, and then reduces the dimensionality of sparse representation of original data by Gaussian random matrix as measurement matrix, to achieve the purpose of dimensionality reduction. Experimental results demonstrate that our overcomplete sparse dictionary can enhance the major underlying structure characteristics of sparse representation, which are mapped into the regions with continuous dimensionality, not the same dimensionality, and improve the discrimination among data which belong to different classes. Only with the constraint of l2-norm, the proposed SDR-CS method has better performance of dimensionality reduction in the MNIST dataset, and it is superior to other existing methods with constraints of l2/l1-norm, achieving the classification error rate of 0.03.