A multiple surrogate-assisted hybrid evolutionary feature selection algorithm

Abstract

Feature selection (FS) is an important data processing technology. However, existing FS methods based on evolutionary computation have still the problems of “curse of dimensionality” and high computational cost, with the increase of the number of feature and/or the size of instance. In view of this, the paper proposes a multiple surrogate-assisted hybrid evolutionary feature selection (MSa-HEFS). Two kinds of surrogates (i.e., objective regression surrogate and sample surrogate) and two kinds of FS methods (i.e., filter and wrapper) are integrated into MSa-HEFS to improve its performance. Firstly, an ensemble filter FS method is designed to reduce the search space of subsequent wrapper evolutionary FS method. Secondly, in the proposed evolutionary FS method, a dual-surrogate-assisted hierarchical individual evaluation mechanism is developed to reduce the evaluation cost on feature subsets, an online management and update strategy is used to adaptively choose appropriate surrogates for individuals. The proposed algorithm is applied to 12 typical datasets and compared with 4 state-of-the-art FS algorithms. Experimental results show that MSa-HEFS can obtain good feature subsets at the smallest computational cost on all datasets. MSa-HEFS source code is available on Github at https://github.com/ZZW-zq/MSa-HEFS-/tree/master.