Robust support vector machine based on the bounded asymmetric least squares loss function and its applications in noise corrupted data

Abstract

The support vector machine (SVM) is a popular machine learning tool that has achieved great success in various fields, but its performance is significantly disturbed on noise corrupted datasets. In this paper, motivated by the bounded quantile loss function, based on the relationship of the expectile and asymmetric least squares loss function, we propose the bounded asymmetric least squares loss function ($L_{bals}$ loss function). The $L_{bals}$ loss function is an extension of the asymmetric least squares loss function. $L_{bals}$ loss function inherits the good properties from the asymmetric least squares loss function, such as asymmetric and differentiable. Further, $L_{bals}$ loss function is more robust to noise in classification and regression problems. Next, we propose two models based on $L_{bals}$ loss function, namely, BALS-SVM and BALS-SVR. The $L_{bals}$ loss function is a non-convex loss function which makes it difficult to optimize. Thus, we design a clipping dual coordinate descent (clipDCD) based half-quadratic algorithm for solving the proposed models. We further find that BALS-SVM and BALS-SVR can be viewed as iterative weighted asymmetric least squares loss function based support vector machines and support vector regression, which enhances the interpretability of the models. Finally, we provide a theoretical analysis of the model based on a general framework of bounded loss function, mainly including Fisher consistency and noise insensitivity. Meanwhile, theoretical guarantees are provided for the proposed models. The results on the simulated dataset and the 14 classification and 11 regression benchmark dataset show that our method is superior compared to the classical methods and some state-of-the-art methods, especially on the noise corrupted dataset. The statistical tests further confirm this fact. Experiments on the Fashion MNIST dataset and gene expression dataset further illustrate that our proposed model also performs well in real environments.