Hadoop-based analysis model of network public opinion and its implementation

Abstract

In order to perform network public opinion mining effectively, this paper proposes a Hadoop-based network public opinion analysis model, which applies HDFS file service system to store massive network data distributed, providing fault tolerance and reliability assurance; As the traditional K-means clustering method is too inefficient to process massive data during the clustering process, this paper adopts MapReduce-based K-means distributed topic clustering computation method to process the massive public opinion information through multi-computer cooperation efficiently; And to obtain the information of hot network public opinion in a certain period of time by the analysis of topic heat, and verify the effectiveness of the proposed method by experiments.