Asymptotic Behavior Of Margin-Based Classification Methods

Abstract

We investigate the asymptotic behavior of the margin-based classification methods in the limit of large dimension $ p\rightarrow \infty $ and large sample size $n \rightarrow \infty $ at fixed rate $\alpha = n/p$. Under spiked population model, we first derive a general framework for describing the performance of a class of classification methods. Then we apply this framework to two commonly used classification methods: Support Vector Machine (SVM) and Distance Weighted Discrimination (DWD). Our analytical results show that DWD is less sensitive to the tuning parameter and achieves better performance than SVM in situations where $n\lt p$. This finding provides a theoretical confirmation to the empirical results that have been observed in many previous simulation and real data studies.