Enhancing robustness and sparsity: Least squares one-class support vector machine

Abstract

In practical applications, identifying data points that deviate from general patterns, known as one-class classification (OCC), is crucial. The least squares one-class support vector machine (LS-OCSVM) is effective for OCC; however, it has limitations: it is sensitive to outliers and noise, and its non-sparse formulation restricts scalability. To address these challenges, we introduce two novel models: the robust least squares one-class support vector machine (RLS-1SVM) and the sparse robust least squares one-class support vector machine (SRLS-1SVM). RLS-1SVM improves robustness by minimizing both mean and variance of modeling errors, and integrating distribution information to mitigate random noise. SRLS-1SVM introduces sparsity by applying the representer theorem and pivoted Cholesky decomposition, marking the first sparse LS-OCSVM adaptation for batch learning. The proposed models exhibit robust empirical and theoretical strengths, with established upper bounds on both empirical and generalization errors. Evaluations on UCI and CIFAR-10 dataset show that RLS-1SVM and SRLS-1SVM deliver superior performance with faster training/testing times. The codes of the proposed models are available at https://github.com/mtanveer1/RLS-1SVM.