Local rotation forest of decision stumps for regression problems

Abstract

Parametric models such as linear regression can contribute valuable, interpretable descriptions of simple structure in data. However, occasionally such simple structure does not extend across an entire database and might be confined more locally within subsets of the data. Nonparametric regression normally involves local averaging. In this study, local averaging estimator is coupled with a machine learning technique - Rotation Forest. In more detail, we propose a technique of local rotation forest of decision stumps. We performed a comparison with other well known methods and ensembles, on standard benchmark datasets and the performance of the proposed technique was greater in most cases.