Improve regression accuracy by using an attribute weighted KNN approach

Abstract

KNN (K nearest neighbor) algorithm is a widely used regression method, with a very simple principle about neighborhood. Though it achieves success in many application areas, the method has a shortcoming of weighting equal contributions to each attribute when computing distance between instances. In this paper, we applied a weighted KNN approach by using weights obtained from optimization and feature selection methods and compared the performance and efficiency of these two types of algorithms in regression problems. Experiments on two UCI datasets show that optimization algorithms like particle swarm optimization can obtain more valuable weights than feature selection algorithms, such as information gain and RelefF, with the tradeoff of running time cost. Both of them canimprove the performance of traditional KNN with equal feature weights.