A Comprehensive Review of Feature Selection and Feature Selection Stability in Machine Learning

Feature selection is a data preprocessing method used to reduce the number of features in the datasets. Feature selection techniques search the entire feature space to find an optimal feature set that is free of redundant and irrelevant features. Reducing the dimensionality of a dataset by removing redundant and irrelevant features has a pivotal role in improving the performance, i.e., accuracy, of inductive learners and building simple models. Thus, feature selection is an imperative task of machine learning. The apparent need for feature selection raised considerable interest and became an important research topic in a wide range of fields, including bioinformatics, text classification, image recognition, and computer vision. As a result, a large pool of feature selection methods has been proposed, and a considerable amount of literature has been published on feature selection. The quality of feature selection algorithms is measured not only by the performance of the features they prefer but also by their stability. Therefore, this study focuses on two topics: feature selection and feature selection stability. In the pages that follow, general concepts and methods of feature selection are discussed and then an overview of feature selection stability and stability measures are given.