Feature Selection for Stock Movement Direction Prediction Using Sparse Support Vector Machine

Abstract

In financial markets, accurate stock price movement prediction can significantly enhance investors' profits. However, the stock price is a highly complex dynamic system with considerable fluctuations, and the accuracy of direction prediction can be improved by selecting appropriate technical indicators. In this work, we propose a novel sparse support vector machines (SVMs) that combines recursive feature elimination (RFE) and ReliefF using a weight parameter. Unlike traditional RFE-based SVMs, our approach constructs a nested feature subset structure, $F_1\subset F_2\subset \cdots \subset F_p$, using a new filter algorithm that combines backward sacrifice and ReliefF by weighting. This new filter algorithm can capture relevant features and feature interactions simultaneously and is crucial in preventing valuable features from being removed at each iteration. Moreover, the ReliefF algorithm combined with RFE can identify more discriminative feature subsets by reordering the features such that valuable ones are ranked higher than valueless ones, and removing valueless features sequentially through iterative processes. Our experimental results on nine stock datasets from the liquor and spirits concept demonstrate that our proposed method outperforms baseline sparse SVMs and SVM models in terms of accuracy and F-test, while also producing a desirable number of features and automatically eliminating redundancy among technical indicators. We also show that on most stock datasets, the ReliefF algorithm combined with RFE can effectively identify discriminative feature subsets for cases of linear and Gaussian kernel SVMs and our proposed filter method can prevent valuable features from being removed at each iteration. In addition, our experimental findings reveal that feature subsets generated by technical indicators are more discriminative while feature subsets generated by technical indicators subsets mapped to a certain higher dimensional space are less discriminative.