Kernel-free quadratic surface SVM for conditional probability estimation in imbalanced multi-class classification

Abstract

For the multi-class classification problems, we propose a new probabilistic output classifier called kernel-free quadratic surface support vector machine for conditional probability estimation (CPSQSVM), which is based on a newly developed binary classifier (BCPSQSVM) combined with the one vs. rest (OvR) decomposition strategy. The purpose of BCPSQSVM is to estimate the positive class posterior conditional probability density and assume it to be a quadratic function. Further, the definition of quadratically separable in probability is given and the optimization problem of BCPSQSVM is constructed under its guidance. The primal problem can be solved directly, because it is a convex quadratic programming problem (QPP) without using kernel functions. However, we design the corresponding block iteration algorithm for its dual problem, which perhaps rendered the device inoperable due to the large constraint size of the primal problem. It is worth noting that our CPSQSVM assigns greater weights to minority samples to mitigate the negative impact of labeling imbalance due to the use of OvR strategy. The existence and uniqueness of optimal solutions, as well as the reliability and versatility of CPSQSVM are discussed in the theoretical analysis. In addition, convergence of the algorithm and upper bound on the margin parameter are analyzed. The feasibility and validity of the proposed method is verified by numerical experiments on some artificial and benchmark datasets.