A self-adjusting representation-based multitask PSO for high-dimensional feature selection

Abstract

As a critical preprocessing step in machine learning tasks, feature selection (FS) aims to identify informative features from the original datasets. However, FS is commonly formulated as an NP-hard combinatorial optimization problem, particularly when compounded by exponentially expanding search spaces and complex feature interactions. Due to its simplicity of implementation, Particle Swarm Optimization (PSO) has been extensively utilized in FS tasks. Concurrently, the optimization process frequently converges to local optima when handling high-dimensional ($>$1000D) datasets. To address this issue, a self-adjusting representation-based multitask PSO (SAR-MTPSO) is proposed in this paper. Firstly, the knee point strategy and an elite feature-preserving strategy are employed to obtain promising particles with key features. Based on these particles, a new multitask framework is introduced, where two tasks are constructed by using a self-adjusting representation strategy. Secondly, a two-layer knowledge transfer strategy is proposed to promote the useful information sharing and exchanging between two tasks dynamically. Finally, an adaptive re-initialization strategy is proposed to enhance the exploitation and exploration capabilities of the two tasks respectively. SAR-MTPSO was compared with 10 representative FS algorithms on 21 high-dimensional datasets. Experimental results show that SAR-MTPSO can achieve the highest classification accuracies with smaller sizes of feature subsets on 17 out of 21 datasets.