A Supervised Feature Selection Algorithm through Minimum Spanning Tree Clustering

Abstract

In different types of feature selection algorithms, feature clustering is an emerging subset generation paradigm. In this paper, a Minimum spanning tree based Feature Clustering (MFC) algorithm is proposed. In the algorithm, an information-theoretic based measure, i.e., Variation of information, is utilized as the feature redundancy and relevance metric. At the clustering phase, the sum of pair wise feature redundancy is minimized. Then, a representative feature is selected from each cluster, where the relevance between representative features and the target label is maximized. The algorithm is supervised since it is designed for various supervised learning problems, such as classification and regression. The proposed MFC is compared with three conventional feature selection algorithms, two of which are also feature clustering method. The MFC obtains well separated feature clusters in the experiment and considerable better classification accuracies applied on several real data sets.