A Novel Forward Filter Feature Selection Algorithm Based on Maximum Dual Interaction and Maximum Feature Relevance(MDIMFR) for Machine Learning

Abstract

In the last few decades, Feature selection is one of the most challenging and open problem to researchers. The rapid progress in computational techniques causes the generation and recording of data in huge size. Though there exists various feature ranking methods, the processing of data is still a challenging task due to its computational complexity. The filter method has many advantages over the wrapper method. The filter methods are classifier independent and have better computational efficiency. Here, a subset of features is selected based on a certain goal function. Most of these goal functions employs the principle of information theory. Most of the algorithms in earlier studies addressed two factors, that is, maximization of relevancy and minimization of redundancy without considering the interaction among the features. This paper developed a new forward filter feature selection algorithm based on mutual information known as Maximum Dual Interaction and Maximum Feature Relevance(MDIMFR). This method considers all the three factors: relevance, redundancy, and feature interaction. This method is experimented on three datasets and compares the performance with existing methods. The results show that MDIMFR outperforms the existing competitive feature selection methods of recent studies: mRMR, JMIM and CMIM. MDIMFR also achieves good stability in average classification accuracy for a certain number of features, say k and above. Hence, these k features can be considered as an optimal feature set.