Enhanced Gradient-Based Optimizer Algorithm With Multi-Strategy for Feature Selection

Abstract

Feature selection is an effective tool for processing data. It is employed to eliminate redundant or irrelevant features and select optimal feature subsets to improve the performance of learning models. The gradient-based optimizer (GBO) received extensive attention in solving different optimization problems, which have the gradient search rule (GSR) and the local escaping operation (LEO). However, when addressing complex optimization and feature selection problems, GBO exhibits deficiencies in balancing global exploration and exploitation, and tends to converge to local optima. This article presents a modified version of GBO, named FWZGBO, for solving feature selection problems. Firstly, inspired by the iterative method and its theory, we propose an enhanced strategy for significantly accelerating the search capability in GSR. This strategy utilizes an optimal fourth-order iterative method to perform the corresponding function of the second-order Newton's method. Secondly, we suggest an enhanced refraction learning approach with Gaussian distribution to help the algorithm escape from local optima and enhance population diversity. Thirdly, this work devises a new adaptive weight based on the cosine strategy in both GSR and LEO to attain a harmonious balance between exploration and exploitation. To validate the performance of the FWZGBO algorithm, 28 benchmark functions and 20 well-known datasets are tested and compared with 14 optimization algorithms. The experimental results show that FWZGBO is significantly superior in solving global optimization and feature selection problems. Meanwhile, the effectiveness of the FWZGBO algorithm is validated using the Friedman test with the corresponding post-hoc test.