A framework for utilizing stock trend prediction outputs in stock selection and portfolio optimization

In this paper, we develop a framework for using stock trend prediction outputs, which we generate using long short-term memory (LSTM) deep neural networks, in both stock selection and portfolio optimization. We use LSTM networks to predict the direction of stock movement and a numerical measure of the strength of the stock trend prediction, and use these in stock selection and within the Markowitz mean-variance portfolio optimization framework. Four types of LSTM models are constructed using the Indian SENSEX stock data - individual and ensemble models, each trained using both batch and incremental learning methods. We utilize the accuracy of classification of stock movement direction in shortlisting stocks for the portfolio optimization stage. Diversified and short-selling enabled Markowitz formulations in addition to the standard Markowitz formulation are constructed in the portfolio optimization stage. We also explore the use of a function of the LSTM classification accuracies as a risk measure both in lieu of and in addition to the covariance matrix within the Markowitz framework. Results from each of the above combinations of LSTM construction and portfolio optimization formulation type are benchmarked against the SENSEX and the standard optimal Markowitz portfolios without stock selection. We also analytically derive the conditions under which Markowitz formulations with stock price predictors more accurate than the mean stock price outperform the standard Markowitz formulations. Our work presents a framework that investment analysts can use to incorporate stock trend prediction outputs generated by machine learning techniques in informing their stock selection and optimal portfolio allocation decisions.