Boosting classification accuracy using an efficient stochastic optimization technique for feature selection in high-dimensional data

Abstract

Many real-world problems involve a large number of features, among which several features are irrelevant or redundant. This problem not only increases the dimensionality but also reduces the classification performance of machine learning models. To address this issue, feature selection methods have been extensively used in the literature, either by applying existing algorithms or developing new algorithms. However, many of these approaches suffer from limitations such as insufficient feature reduction due to getting trapped in local minima in the large search space. Hence, this study proposed a recent stochastic optimization-based technique called the Osprey Optimization Algorithm (OOA). The OOA algorithm has the capability of balancing exploration and exploitation effectively during the search process, making it suitable for solving high-dimensional optimization tasks. To validate the efficiency of the selected feature subsets, the study employs the k-nearest neighbor (k-NN) classifier. Comparative results between OOA and five state-of-the-art algorithms show that OOA achieves the highest average classification accuracy of 89.22 %, while selecting the fewest average features of 70.63 and reduces the feature burden by 62.80 %. Moreover, the results of a non-parametric Wilcoxon signed-rank test based on classification accuracy show a p-value less than 5.00E-02, confirming a statistically significant difference in performance among the six algorithms.