Discriminative approximate regression projection for feature extraction

Abstract

Dimension reduction has attracted much attention in pattern recognition and computer vision to mitigate the ‘curse of dimensionality’. As a classical dimension reduction method, least squares regression (LSR) shows its powerful ability in fitting data and extracting discriminative features. However, LSR still faces the following issues. First, the retained feature dimension is strictly equal to the number of classes, which lacks flexibility. Especially, when the categories are few, LSR is insufficient to extract enough features. Second, the local structure relation is neglected. The manifold in LSR only preserves the neighborhood relationship of high-dimensional data, but cannot cluster these data to a single point in a low-dimensional space. Third, LSR considers dimensionality reduction, but neglects data reconstruction. In fact, if the extracted features can reconstruct the original data well, it indicates that the extracted features retain most information of the original data, which will be beneficial for the further classification. To this end, this paper proposes Discriminative Approximate Regression Project (DARP) to address the above issues. In DARP, two matrices are introduced for feature extraction and label fitting to ensure the flexibility of the extracted feature dimension. Using the label matrix as a weight matrix, the samples from each class are projected onto their corresponding class centroid point, which can make the samples from each class to gather in a low-dimensional space. Furthermore, a data reconstruction regularization term is introduced to enable that the feature extraction matrix can reconstruct the original data to preserve most information of the original data. Comprehensive experimental results on widely tested benchmarks demonstrate the competitive performance of DARP.