How Dependencies Affect the Capability of Several Feature Selection Approaches to Extract the Key Features

Abstract

The goal of this research is to find how dependencies affect the capability of several feature selection approaches to extract of the relevant features for a classification purpose. The hypothesis is that more dependencies and higher level dependencies mean more complexity for the task. Some experiments are used to intend to discover some limitations of several feature selection approaches by altering the degree of dependency of the test datasets. A new method has been proposed, which uses a pair of pre-designed Bayesian Networks to generate the test datasets with an easy tuning level of complexity for feature selection test. Relief, CFS, NB-GA, NB-BOA, SVM-GA, SVM-BOA and SVM-mBOA are the filter or wrapper model feature selection approaches which are used and evaluated in the experiments. For these approaches, higher level of dependency among the relevant features greatly affect the capability to find the relevant features for classification. For Relief, SVM-BOA and SVM-mBOA, if the dependencies among the irrelevant features are altered, the performance of them changes as well. Moreover, a multi-objective optimization method is used to keep the diversity of the populations in each generation of the BOA search algorithm improving the overall quality of solutions in our experiments.