An Ordered Search for Subset Selection in Support Vector Orthogonal Regression

Abstract

Subset selection is an important task in many problems, especially when dealing with high dimensional problems, such as classification, regression, and others. In this sense, this work proposes an ordered search to select variables in orthogonal regression problems based on support vectors. The admissible search is based on a monotone property of the radius parameter. Thus, we use the radius of the SV-regression as an evaluation measure for the search, making it able to find the subsets with the smallest radius in each dimension of the problem without exhaustively exploring all possibilities. The main reason for choosing the orthogonal regression is due to the fact that this model also considers the existence of error in dependent variables. The obtained results, represented by the test error, when compared to the LASSO and a recursive feature elimination technique, demonstrate the efficiency of the method.