VCOS: Multi-scale information fusion to feature selection using fuzzy rough combination entropy

Multi-scale information fusion has attracted extensive attention in data mining, in which the optimal scale combination principles and feature selection algorithms are two core issues. However, the traditional optimal scale combination is obtained by satisfying the consistency of the conditional feature scales with the decision classification. This consistency principle is too strict and not fault-tolerant. It leads to the knowledge granularity being too fine, is likely to reduce the feature selection algorithms performance, and does not meet the needs of practical applications. Therefore, this paper develops a novel optimal scale combination selection method to fuse multi-scale information, establishes a new fuzzy rough set model, defines uncertainty measures, and designs a feature selection algorithm for Multi-scale Fuzzy Decision Systems (MsFDSs). First, the Variable-Consistency Optimal Scale (VCOS) selection principle is defined by introducing the variable-consistency rate. The VCOS-based fuzzy rough set model is proposed, a derived uncertainty measure based on this model is defined as well as related properties are proved. Then, the VCOS-based Fuzzy Rough Combinatorial Entropy (VCOS-FRCE) is defined, and its monotonicity with respect to the feature subsets and the variable consistency rate is proved, respectively. Finally, we define the relative reduct principle and the significance of features based on VCOS-FRCE and design a forward greedy multi-scale feature selection algorithm. Our proposed VCOS-based multi-scale fusion method can adjust the consistency degree between knowledge granules and decision classification according to actual needs. This multi-scale information fusion method has better generalization and can be applied to various complex data. The performance of the multi-scale feature selection method developed based on this method is also further improved. Experiments are performed on twelve public datasets from UCI, and the proposed algorithm is compared with eight existing algorithms. The experimental results show that the proposed algorithm can effectively remove redundant features and improve the classification performance.