Medical Analysis of Social Media Data Based on Spark and Machine Learning in China

Social media embracing a huge amount of real-time data of all kinds plays an important role in data analysis in the era of big data. Knowledge between medical workers and ordinary people can be popularized and exchanged via social media. At the same time, the collection and utilization of medical data on social media can effectively grasp the public health situation and provide better help to improve people's health status. From the perspective of medical care and health, this paper uses Weibo, the largest public social media in China, to obtain data for analysis. The study was developed under the Spark framework, using naive Bayes, random forest and two different feature extraction methods to clean, pre-process and classify data. Furthermore, the accuracy rate and F1 Score were used to evaluate the model, to find the most appropriate method. The result of this research shows that the data obtained from Weibo within certain age groups has a good reference value in the public awareness and current situation, and are good for grasping the trend of diseases.