Effective Information Retrieval Framework for Twitter Data Analytics

Abstract

The widespread adoption of opinion mining and sentiment analysis in higher cognitive processes encourages the need for real time processing of social media data to capture the insights about user's sentiment polarity, user's opinions, and current trends. In recent years, lots of studies were conducted around the processing of data to achieve higher accuracy. But reducing the time of processing still remained challenging. Later, big data technologies came into existence to solve these challenges but those have its own set of complexities along with having hardware deadweight on the system. The contribution of this article is to touch upon mentioned challenges by presenting a climbable, quick and fault tolerant framework to process real-time data to extract hidden insights. This framework is versatile enough to support batch processing along with real time data streams in parallel and distributed environment. Experimental analysis of proposed framework on twitter posts concludes it as quicker, robust, fault tolerant, and comparatively more accurate with traditional approaches.