Improving Sentiment Analysis using Negation Scope Detection and Negation Handling

Abstract

: Negation is one of the challenges in sentiment analysis. Negation has an immense influence on how accurately text data can be classified. To find accurate sentiments of users this research identifies that the impact of negations in a sentence needs to be properly handled. Traditional approaches are unable to properly determine the scope of negations. In the proposed approach Machine learning (ML) is used to find the scope of negations. Moreover, the removal of negative stopwords during pre-processing leads to the flipped polarity of sentences. To resolve these challenges this research proposes a method for negation scope detection and handling in sentiment analysis. First, negation cue (negative words) and non cue words are determined, these negation cue and non cue words in addition to lexical and syntactic features determine the negation scope (part of sentence a ff ected by cue) using the Machine Learning (ML) approach i.e. Conditional Random Fields (CRF). Subsequently, in negation handling the sentiment intensity of each token in a sentence is established, and a ff ected tokens are processed to determine the final polarity. It is revealed that sentiment analysis with negation handling and calculated polarity gives 3.61%, 2.64%, 2.7%, and 1.42% increase in accuracy for Logistic regression, Support Vector Machine, Decision Tree (DT), and Naive Bayes (NB) consecutively for Amazon food products dataset. Consecutively, 9.4%, 3%, and 2% improvement for Logistic Regression (LR)