Analyzing News Sentiments and their Impact on Stock Market Trends using POS and TF-IDF based approach

Abstract

Since the dawn of time, investors are looking into different schemes in determining the stock trends to earn profit. Several studies have been conducted that could potentially help the investors predict the rise and fall of stocks. Most of them looked into past market pricing history in order to foresee the future. While many factors influence the fluctuation of stock market, it can be argued that the sentiments of the investors influenced by unfolding of current happenings or events has a huge impact on the stock trend. In this paper, we propose a new method in interpreting the sentiment of a given news. Through a fine-grained analysis of syntactic sentence patterns using different Part of Speech (POS) combinations, the news data inputs are preprocessed. These are then fed into Term Frequency - Inverse Document Frequency (TF-IDF) to filter only significant text in the corpus. We then conduct experiments using various classifiers to predict the sentiments. Results are fed into K-Nearest Neighbor (K-NN) classifier, along with historical stock price, to determine adjusted closing price over various time intervals. It can be observed that the results of proposed model are compatible with current researches stating about existing correlation between financial news and stock prices.