A distributionally robust machine learning model of simultaneous classification and feature selection under data uncertainty: Theory, methods, and application to the identification of Alzheimer's disease using handwriting

Abstract

In this paper, we introduce an efficient machine learning method based on robust Support Vector Machines (SVMs) that simultaneously classifies data and selects relevant features whilst accounting for data uncertainty. Based on Wasserstein distributionally robust optimization, we develop computationally feasible robust SVM models along with efficient second-order cone programming methods using an integrated application of tools from convex non-smooth analysis and difference-of-convex optimization. Our computational results on benchmark datasets demonstrate that these robust SVMs identify relevant features whilst achieving higher classification accuracies than the conventional (non-robust) SVM models, especially for datasets with more features than instances. Applying our method to a novel dataset of handwriting samples from individuals with Alzheimer's disease and a control group, the model was able to distinguish between both groups with greater than 80% accuracy and using only 37% (168/450) of all available features, outperforming previous SVM models and providing insights into the unique characteristics of the disease.