A leader-adaptive particle swarm optimization with dimensionality reduction strategy for feature selection

Abstract

Feature selection (FS) is a key data pre-processing method in machine learning tasks. It aims to obtain better classification accuracy of an algorithm with the smallest size of selected feature subset. Particle Swarm Optimization has been widely applied in FS tasks. However, when solving FS task on high-dimensional datasets, most of the PSO-based FS methods are easy to get premature convergence and fall into the local optimum. To address this issue, a leader-adaptive particle swarm optimization with dimensionality reduction strategy (LAPSO-DR) is proposed in this paper. Firstly, a hybrid initialization strategy based on feature importance is formulated. The population is divided into two parts, which have different initialization ranges. It can not only improve the diversity of the population but also eliminate some redundant features. Secondly, the leader-adaptive strategy is proposed to improve the exploitation ability of the population, in which each particle can have a different learning exemplar selected from the elite sub-swarm. Finally, the dimensionality reduction strategy based on Markov blanket is introduced to reduce the size of the optimal feature subset. LAPSO-DR is compared with 8 representative FS methods on 18 benchmark datasets. The experimental results show that LAPSO-DR can obtain smaller sizes of feature subsets with highest classification accuracies on 17 out of 18 datasets. The classification accuracies of LAPSO-DR are over 90% on 14 datasets and the feature elimination rates are higher than 60% on 18 datasets.