Density cluster-based feature selection: An information theory approach

Abstract

Feature selection plays a crucial role in data mining and machine learning. However, evident challenges exist: (1) current methods cannot autonomously identify the optimal feature set, requiring manual parameter adjustment based on the learning algorithm; (2) heuristic methods, which are widely used, often struggle to ensure the maximization of the objective function. To address these challenges, this paper proposes a density cluster-based feature selection (DCFS) method leveraging information theory, which involves the application of artificial intelligence (AI) in the clustering process. First, a novel initial feature selection method that maximizes feature-relevance and feature-difference is introduced to automate the selection of an initial feature subset. Second, a new density-centric automatic clustering (DAC) algorithm, an AI-based clustering approach, is proposed. This algorithm synthesizes non-parametric density estimation, decision graph-based density center selection strategies, and adaptive domain search clustering methods to enhance the precision and robustness of clustering outcomes. Third, a feature space selection method based on maximizing feature relevance is established to construct a comprehensive feature subspace. This feature space selection method converts the maximization of the objective function into an automated density clustering process, facilitating the automatic selection of the most optimal features. Extensive experiments conducted across 14 datasets have demonstrated the superior performance of the proposed DCFS in terms of effectiveness and robustness. To the best of our knowledge, this paper is the first work attempting automatic feature selection through clustering, thus pushing the frontier of feature selection algorithm development.