Non-parametric Regression and Random Balance Method Modification for Determination of the Most Informative Features

Abstract

In this paper we present a new method which allows us to detect the most informative features out of all data extracted from a certain data corpus. Widely used Pearson’s coefficient is not reliable if the dependency between extracted features (input variables) and the objective function (output) is not linear. This approach is based on a modified random balance method (RBM) combined with non-parametric kernel regression for modeling the dependency between output and input variables. The standard random balance method stochastically determines the most important features of a process, but it requires the values of the objective function at the certain assigned points. If there is no possibility to calculate these values, it is necessary to approximate them. Since we assume that the dependency between stochastic variables can be non-linear, it is necessary to take an appropriate model. We used non-parametric kernel regression because knowledge about the parametric structure of the dependency is not needed. Moreover, we modified the random balance method to handle the non-linearity of the data