Dimension reduction in classification using particle swarm optimisation and statistical variable grouping information

Abstract

Dimension reduction is a preprocessing step in many classification tasks, but reducing dimensionality and finding the optimal set of features or attributes are challenging because of the big search space and interactions between attributes. This paper proposes a new dimension reduction method by using a statistical variable grouping method that groups similar attributes into a group by considering interaction between attributes and using particle swarm optimisation as a search technique to adopt the discovered statistical grouping information to search optimal attribute subsets. Two types of approaches are developed, where the first aims to select one attribute from each group to reduce the dimensionality, and the second allows the selection of multiple attributes from one group to further improve the classification performance. Experiments on ten datasets of varying difficulties show that all the two approaches can successfully address dimension reduction tasks to decrease the number of attributes, and achieve the similar of better classification performance. The first approach selects a smaller number of attributes than the second approach while the second approach achieves better classification performance. The proposed new algorithms outperform other recent dimension reduction algorithms in terms of the classification performance, or further reduce the number of attributes while maintaining the classification performance.