Novel Machine Learning Approach for Sentiment Analysis of Real Time Twitter Data with Apache Flume

Abstract

A lot of data is generated from multiple sources. This data contains many hidden patterns and information. Many researchers are trying to get meaningful insights out of these patterns. Data from these sources mostly contains opinions. Opinions can be mined to lead various extractions from organizational point of view. One approach is to use Sentiment Analysis. In this paper, the authors are storing the Twitter Streaming Data into HDFS of Hadoop by using Flume and then extracting with Apache Hive. Later, Machine Learning classification algorithms are applied to decode the sentiment in this data using Apache Mahout. A novel approach based on hybrid Naïve Bayes and Decision Tree Algorithms are used to enhance the performance of sentiment analysis of streaming twitter data. The implemented research approach achieved an accuracy of 86.44% in comparison to 81.11% for Naïve Bayes Classifier.