A Greedy Search Algorithm for Resolving the Lowermost C Threshold in SVM Classification

Abstract

In this paper, the authors present a greedy search algorithm that solves the SVM classification (SVC) problem at the lowermost C end. By combining the SVC asymptotic behavior with empirical results, it can be sure that at sufficiently small cost, a threshold C0, all the minority samples becomes support vectors each with Lagrange multiplier C0, and equal number of majority samples will become support vectors whose Lagrange multipliers are also C0. With this evidence, SVC is transformed into a C-independent combinatorial optimization problem. Taking all the minority inputs as initial support vectors, a greedy algorithm is devised to choose majority class support vectors one by one each with minimal increase to the objective function in its turn. The greedy nature of the algorithm enables finding out the majority support vectors that near or at the majority margin. By taking the found majority support vectors initially and applying the algorithm to the minority class conversely, the support vectors that near the decision boundary are also resolved. Applying linear least squares fitting to both the majority margin and decision boundary, C0 is obtained as a function of kernel parameters. Experimental results show that the proposed algorithm can find C0 almost perfectly.