Efficient Linear Discriminant Analysis Based on Randomized Low-Rank Approaches

Abstract

Linear discriminant analysis (LDA) faces challenges in practical applications due to the small sample size (SSS) problem and high computational costs. Various solutions have been proposed to address the SSS problem in both ratio trace LDA and trace ratio LDA (TRLDA). However, the iterative processing of large matrices often makes the computation process cumbersome. To address this issue, for TRLDA, we propose a novel random method that extracts orthogonal bases from matrices, allowing computations with small-sized matrices. This significantly reduces computational time without compromising accuracy. For ratio trace LDA, we introduce a fast generalized singular value decomposition (GSVD) algorithm, which demonstrates superior speed compared to MATLAB’s built-in GSVD algorithm in experiments. By integrating this new GSVD algorithm into ratio trace LDA, we propose FGSVD-LDA, which exhibits low computational complexity and good classification performance. The experimental results show that both methods effectively achieve dimensionality reduction and deliver satisfactory classification accuracy.