Regularized sparse feature selection with constraints embedded in graph Laplacian matrix

Abstract

Feature selection is an important pre-processing stage in many machine learning and pattern recognition tasks, which eliminates irrelevant and redundant features and improves learning performance. Regularized sparse feature selection methods like Lasso and its variants using ℓ1-norm regularization term in their optimization problem have received much attention in recent years. Prior information could be represented as the class labels or pairwise constraints, i.e., must-link (positive) and cannot-link (negative) constraints. In this paper, besides the ℓ1-norm regularization term, a normalized adapted affinity matrix is applied to embed the pairwise constraints in the affinity matrix. In the proposed affinity matrix, the weights are strengthened/weakened according to the positive/negative constraints. The experimental results on several data sets from University of California-Irvine (UCI) machine learning repository and a high dimensional data set, show the effectiveness of the proposed method in the classification tasks compared to some similar feature selection methods.