A Hybrid Feature Selection Method Based on Dual Attention Mechanism for High-Dimensional Data Classification

Abstract

As an important preprocessing technique in the field of machine learning, feature selection serves the purpose of identifying the optimal feature subset, while eliminating irrelevant or redundant features. However, most existing attention mechanism-based feature selection methods only consider promising features when handling high-dimensional datasets, which may lead to redundancy of promising features and neglect of the interaction between irrelevant features and promising features. So it remains a challenge to effectively screen out features that are complementary to the promising ones from irrelevant features. To solve this limitation, this paper proposes a hybrid feature selection method based on dual attention mechanism. First, an initialization method based on the knee-point strategy is proposed to initialize the dataset into a main dataset and an auxiliary dataset. Then, a training module and a two-stage optimization are used to generate the main feature set and auxiliary feature set which include the elite features. Finally, a complementary mining operation is employed to screen out the optimal feature subset by leveraging the correlation between the two aforementioned feature sets. Experimental evaluations conducted on 24 benchmark datasets verify that the proposed algorithm outperforms eight state-of-the-art algorithms in terms of both classification accuracy and selected feature size, thereby illustrating its effectiveness in solving practical problems.