A performance-driven multi-stage KNN approach for local adaptive classification

Abstract

A key issue of the K-Nearest Neighbors (KNN) algorithm is determining the optimal neighborhood size K, which limits the widespread applicability of KNN. To address this, a performance-driven multi-stage KNN (PMKNN) approach is proposed in this paper. Given a set of alternative K values, the traditional KNN algorithm is initially employed in the PMKNN approach to identify the optimal K values for all known samples. A convex optimization model is then constructed based on the least squares loss function to learn the correlation between known samples and query samples. After the learned correlation is used to evaluate the performances of all candidate K values in classifying query samples, a weighted majority voting process is designed to generate the final classification results. Unlike existing KNN approaches, the proposed PMKNN approach considers multiple optimal K values for each query sample, enhancing classification stability and reliability. The proposed approach also reduces the negative impact of inappropriate K values on classification performance. An experimental study is conducted using twenty real-world classification datasets collected from two public data repositories to assess the effectiveness of the proposed PMKNN approach. The relevant results highlight the high classification performance of the proposed PMKNN approach compared to seven state-of-the-art KNN methods and underscore its predictive stability compared to the traditional KNN algorithm using all possible K values.