Soft Upper-bound Minimal Complexity LP SVMs

The minimal complexity linear programming support vector machine (MLP SVM) was proposed to solve the problem of unbounded non-unique solutions of the minimal complexity machine (MCM). The MLP SVM minimizes the maximum margin that is the maximum distance between training data and the separating hyperplane as well as maximizes the minimum margin. Therefore, the generalization ability may be worsened if outliers are included and they affect the slope and the location of the separating hyperplane. To solve this problem, in this paper, we propose the soft upper-bound MLP SVM (SLP SVM), in which the outliers that affect the hyperplane are suppressed by introducing the slack variables. This introduction leads to the increase of hyperparameters. We discuss how to reduce the number of hyperparameters to speed up model selection. By computer experiments we compare the generalization ability and training time of the SLP SVM with those of the MLP SVM, MCM, and other SVM based classifiers using two-class and multiclass problems.