Adjustable robust optimization approach for SVM under uncertainty

Abstract

The support vector machine (SVM) is one of the successful approaches to the classification problem. Since the values of features are typically affected by uncertainty, it is important to incorporate uncertainty into the SVM formulation. This paper focuses on developing a robust optimization (RO) model for SVM. A key distinction from existing literature lies in the timing of optimizing decision variables. To the best of our knowledge, in all existing RO models developed for SVM, a common assumption is that all decision variables are decided before the uncertainty realization, which leads to an overly conservative decision boundary. However, this paper adopts a different strategy by determining the variables that assess the misclassification error of data points or their fall within the margin post-realization, resulting in a less conservative model. The RO models where decisions are made in two stages (some before and the rest after the uncertainty resolution), are called adjustable RO models. This adjustment results in a three-level optimization model for which two decomposition-based algorithms are proposed. In these algorithms, after providing a bi-level reformulation, the model is divided into a master-problem (MP) and a sub-problem the interaction of which yields the optimal solution. Acceleration of algorithms via incorporating valid inequalities into MP is another novelty of this paper. Computational results over simulated and real-world datasets confirm the efficiency of the proposed model and algorithms.