Token-Based Adaptive Time-Series Prediction by Ensembling Linear and Non-linear Estimators: A Machine Learning Approach for Predictive Analytics on big Stock Data

With technological advancements, big data can be easily generated and collected in many applications. Embedded in these big data are useful information and knowledge that can be discovered by machine learning and data mining models, techniques or algorithms. A rich source of big data is stock exchange. The ability to effectively predict future stock prices improves the economic growth and development of a country. Traditional linear approaches for prediction (e.g., Kalman filters) may not be practical in handling big data like stock prices due to highly nonlinear and chaotic nature. This lead to the exploitation of various nonlinear estimators such as the extended Kalman filters, expert systems, and various neural network architectures. Moreover, to lessen the potential shortcomings of individual algorithms, ensemble approaches have been created by averaging values across different algorithms. Existing ensemble techniques mostly basket-together a collection of sample-based algorithms that are catered to nonlinear functions. To the best of our knowledge, traditional linear estimators have not yet been incorporated into such an ensemble. Hence, in this paper, we propose a machine learning (specifically, token-based ensemble) algorithm that utilizes both linear and nonlinear estimators to predict big financial time-series data. Our ensemble consists of a traditional Kalman filter, long short-term memory (LSTM) network, and the traditional linear regression model. We also explore the adaptive properties in short-term high-risk trading in the presence of noisy data like stock prices and demonstrate the performance of our ensemble.