How SVM Can Compensate Logit Based Response Label with Various Characteristics in Predictor? A Simulation Study

Abstract

In general, supervised machine learning methods for classification can be categorized into two approaches, namely parametric and nonparametric. Parametric method has limitations in term of the assumptions must be satisfied. One way to handle this problem is using non parametric approaches. The state of the art classification method is support vector machine (SVM). However, the computational burden of kernel SVM limits its application to large scale datasets that demand high computational time. So, one way to cope the limitation is using ensemble approach that splits the data and applies learning procedure at each subset of data. In this work, the Clustered Support Vector Machine (CSVM) is chosen. So far, the studies of CSVM are limited to theoretical and direct application for real dataset. The application to real dataset directly has a weakness that we never know in detail how various characteristic in predictor affect the learning process. So, it is necessary to do a simulation study to further explore how complex the data, particularly in predictor, that can be handled by SVM and CSVM. There are ten scenarios conducted in this simulation study. The response label is generated using Iogistic regression model with various characteristic setting in predictor in each scenario. Given the true response label is generated using Iogit model, the results of this simulation study show that SVM and CSVM can compensate the performance of Iogistic regression in some scenarios. These results showed that SVM is powerful in classification method regardless how the response label is generated.