Unsupervised Feature Selection via Self-Supervised HSIC and Elastic Net Regularization.

Abstract

Unsupervised feature selection is essential for high-dimensional data analysis, where irrelevant and redundant variables may obscure the intrinsic data structure, reduce interpretability, and degrade downstream learning performance. The key challenge is to identify informative features without label supervision while suppressing redundant selections under nonlinear dependencies. To address this issue, this paper proposes an unsupervised Hilbert-Schmidt Independence Criterion (HSIC)-Elastic Net (ENet) feature selection framework, termed U-HSIC-ENet. The proposed method reformulates unlabeled feature weighting as a self-supervised kernel alignment problem. Specifically, a target kernel is constructed directly from unlabeled data to encode global sample relationships, while each feature is represented by a centered and Frobenius-normalized feature-induced kernel. Feature relevance is then measured by centered kernel alignment (CKA) in a reproducing kernel Hilbert space (RKHS). The main novelty of U-HSIC-ENet lies in a unified relevance-redundancy-stability formulation. Feature-target alignment is used to estimate nonlinear relevance, whereas pairwise similarities between feature-induced kernels are used to characterize inter-feature redundancy in the same kernel alignment space. On this basis, an explicit off-diagonal redundancy penalty is incorporated into a nonnegative Elastic Net-type objective, which strengthens the suppression of co-selected similar features while preserving sparse and stable feature weighting. The resulting quadratic formulation clarifies how relevance promotion, redundancy control, sparsity, and numerical stabilization are coupled within a single optimization framework. Experiments on eight benchmark datasets under a fixed-budget evaluation protocol show that U-HSIC-ENet achieves the strongest average performance on Normalized Mutual Information (NMI), the Adjusted Rand Index (ARI), and clustering accuracy (ACC) compared with representative graph-, spectral-, and HSIC-based baselines. The advantage is the most pronounced on NMI, suggesting that the self-supervised target kernel and CKA-based relevance modeling are effective in preserving the clustering-relevant nonlinear structure. Friedman tests and Wilcoxon signed-rank tests with Holm correction provide statistical support for the observed improvements. Subsampling-based stability evaluation reveals a trade-off between clustering effectiveness and selection reproducibility: several baselines achieve higher stability scores despite the stronger average clustering performance of U-HSIC-ENet. These results indicate that the proposed framework is effective for unsupervised nonlinear feature weighting when relevance estimation, redundancy control, and stability are considered jointly.