Support vector machine with discriminative low-rank embedding

Abstract

Support vector machine (SVM) is a binary classifier widely used in machine learning. However, neglecting the latent data structure in previous SVM can limit the performance of SVM and its extensions. To address this issue, the authors propose a novel SVM with discriminative low-rank embedding (LRSVM) that finds a discriminative latent low-rank subspace more suitable for SVM classification. The extension models of LRSVM are introduced by imposing different orthogonality constraints to prevent computational inaccuracies. A detailed derivation of the authors’ iterative algorithms are given that is essentially for solving the SVM on the low-rank subspace. Additionally, some theorems and properties of the proposed models are presented by the authors. It is worth mentioning that the subproblems of the proposed algorithms are equivalent to the standard or the weighted linear discriminant analysis (LDA) problems. This indicates that the projection subspaces obtained by the authors’ algorithms are more suitable for SVM classification compared to those from the LDA method. The convergence analysis for the authors proposed algorithms are also provided. Furthermore, the authors conduct experiments on various machine learning data sets to evaluate the algorithms. The experiment results show that the authors’ algorithms perform significantly better than other algorithms, which indicates their superior abilities on classification tasks.