Named Entity Recognition Based Neural Network Framework for Stock Trend Prediction Using Latent Dirichlet Allocation

Abstract

Stock price prediction is an extensively researched topic as the precise prophecy of stock trends is decisive in the investment marketing sphere. With increasing opinions by many market giants on the internet about given stocks, it surges the necessity to study these sentiments in detail for forthcoming predictions. From these articles on the internet, natural text is generated by examining factors that affect the values of stocks and therefore these texts are reliable features to go ahead with this study. The idea behind tackling such work is that conglomerates and businesses are able to tangibly understand the aftermath of articles that usually mobilize public opinion and gear them in a certain direction. The aim of this study is to utilize named entity recognition (NER) on a neural network framework for stock trend prediction through latent Dirichlet allocation using these natural texts generated from internet articles. This method is used to understand the words that occur at the highest frequency and add the most information to the corpus depending on the topic’s importance. With this, the model adopts K × K words that have the most decisive impact on the target class that has been created with which it alters the sparse density matrix that has been generated. The proposed model of the NER-based neural network was fitted on a real-world dataset, and its performance was good in comparison with state-of-the-art models developed by fellow researchers. However, since the model does not use the BERT tokenizers, it cannot be adjudged on the FinBERT model, and therefore, the preprocessed data is fed to a pruned recurrent neural network which is robustly stopped with a simple callback function. The final result was a strong 0.81 tetrachoric correlation between the testing target class and the predicted target class. With this, the model provides a different approach to natural language processing, especially those with high sparse density for stock prediction.