Sparse and robust alternating direction method of multipliers for large-scale classification learning

Abstract

Support vector machine (SVM) is a highly effective method in terms of classification learning. Nonetheless, when faced with large-scale classification problems, the high computational complexity involved can pose a significant obstacle. To tackle this problem, we establish a new trimmed squared loss SVM model known as TSVM. This model can be designed for achieving both sparsity and robustness at the same time. A novel optimality theory has been developed for the nonsmooth and nonconvex TSVM. Utilizing this new theory, the innovative fast alternating direction method of multipliers with low computational complexity and working set has been proposed to solve TSVM. Numerical tests show the effectiveness of the new method regarding the computational speed, number of support vector and classification accuracy, outperforming eight alternative top solvers. As an illustration, when tackling the real dataset with more than $10^7$ instances, compared to seven other algorithms, our algorithm exhibited a 34 times enhancement in computation time, alongside achieving a 6.5% enhancement in accuracy and a 25 times decrease in support vector rates.