AdaBoost regression algorithm based on classification-type loss

Abstract

This paper presents a new concept of building classification-type loss for regression sample based on conversion between regression and classification problems used in Support Vector Regression (SVR). By introducing the classification-type loss to calculate example's error, AdaBoost algorithm can be generalized from classification to regression. A new Boosting algorithm for regression, called AdaBoost.SVR.R which can be directly applied to a regression problem is proposed. SVR is used as its base learner. Its output is an ensemble of a team of regression functions. The employing of the classification-type loss makes the iterating process of AdaBoost.SVR.R act essentially on a converted binary classification problem. The output scheme of AdaBoost.SVR.R is also derived upon constructing decision function of the binary classification problem. Since it has the same application condition as AdaBoost, AdaBoost.SVR.R could satisfy the convergence proof of AdaBoost algorithm. The testing results for the considered data sets show that the new algorithm is effective.