基于大数据的热点新闻分析

2018 IEEE/ACIS 17th International Conference on Computer and Information Science (ICIS) Pub Date : 2018-06-01 DOI:10.1109/ICIS.2018.8466427

Chengcheng Hu, Y. Li, Yongbin Wang, Lin Wu

{"title":"基于大数据的热点新闻分析","authors":"Chengcheng Hu, Y. Li, Yongbin Wang, Lin Wu","doi":"10.1109/ICIS.2018.8466427","DOIUrl":null,"url":null,"abstract":"To analyze hot news data of a culture experimental area in China, web crawler, text extraction, named entity recognition, word cloud and other technologies are be used in the paper. The news texts are obtained by using Berkeley DB, Scrapy frame and web page text extraction algorithm firstly. The total number of crawled news articles is 6.87 million. Then based on these data, the analysis of news attention and statistics of hot news to the experimental area are conducted by using NLTK's NER technology and Weka tools. And the relevant industry to the experimental were also analyzed. The visual representation of analysis results also is provided in this paper.","PeriodicalId":447019,"journal":{"name":"2018 IEEE/ACIS 17th International Conference on Computer and Information Science (ICIS)","volume":"143 1","pages":"0"},"PeriodicalIF":0.0000,"publicationDate":"2018-06-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"1","resultStr":"{\"title\":\"Analysis of Hot News Based on Big Data\",\"authors\":\"Chengcheng Hu, Y. Li, Yongbin Wang, Lin Wu\",\"doi\":\"10.1109/ICIS.2018.8466427\",\"DOIUrl\":null,\"url\":null,\"abstract\":\"To analyze hot news data of a culture experimental area in China, web crawler, text extraction, named entity recognition, word cloud and other technologies are be used in the paper. The news texts are obtained by using Berkeley DB, Scrapy frame and web page text extraction algorithm firstly. The total number of crawled news articles is 6.87 million. Then based on these data, the analysis of news attention and statistics of hot news to the experimental area are conducted by using NLTK's NER technology and Weka tools. And the relevant industry to the experimental were also analyzed. The visual representation of analysis results also is provided in this paper.\",\"PeriodicalId\":447019,\"journal\":{\"name\":\"2018 IEEE/ACIS 17th International Conference on Computer and Information Science (ICIS)\",\"volume\":\"143 1\",\"pages\":\"0\"},\"PeriodicalIF\":0.0000,\"publicationDate\":\"2018-06-01\",\"publicationTypes\":\"Journal Article\",\"fieldsOfStudy\":null,\"isOpenAccess\":false,\"openAccessPdf\":\"\",\"citationCount\":\"1\",\"resultStr\":null,\"platform\":\"Semanticscholar\",\"paperid\":null,\"PeriodicalName\":\"2018 IEEE/ACIS 17th International Conference on Computer and Information Science (ICIS)\",\"FirstCategoryId\":\"1085\",\"ListUrlMain\":\"https://doi.org/10.1109/ICIS.2018.8466427\",\"RegionNum\":0,\"RegionCategory\":null,\"ArticlePicture\":[],\"TitleCN\":null,\"AbstractTextCN\":null,\"PMCID\":null,\"EPubDate\":\"\",\"PubModel\":\"\",\"JCR\":\"\",\"JCRName\":\"\",\"Score\":null,\"Total\":0}","platform":"Semanticscholar","paperid":null,"PeriodicalName":"2018 IEEE/ACIS 17th International Conference on Computer and Information Science (ICIS)","FirstCategoryId":"1085","ListUrlMain":"https://doi.org/10.1109/ICIS.2018.8466427","RegionNum":0,"RegionCategory":null,"ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"","JCRName":"","Score":null,"Total":0}

引用次数: 1

摘要

为了对中国某文化实验区的热点新闻数据进行分析，本文采用了网络爬虫、文本提取、命名实体识别、词云等技术。首先利用Berkeley DB、Scrapy框架和网页文本提取算法获得新闻文本。抓取的新闻文章总数为687万篇。然后在这些数据的基础上，利用NLTK的NER技术和Weka工具对实验区进行新闻关注度分析和热点新闻统计。并对实验的相关行业进行了分析。文中还提供了分析结果的可视化表示。

本文章由计算机程序翻译，如有差异，请以英文原文为准。

查看原文本刊更多论文

Analysis of Hot News Based on Big Data

To analyze hot news data of a culture experimental area in China, web crawler, text extraction, named entity recognition, word cloud and other technologies are be used in the paper. The news texts are obtained by using Berkeley DB, Scrapy frame and web page text extraction algorithm firstly. The total number of crawled news articles is 6.87 million. Then based on these data, the analysis of news attention and statistics of hot news to the experimental area are conducted by using NLTK's NER technology and Weka tools. And the relevant industry to the experimental were also analyzed. The visual representation of analysis results also is provided in this paper.

求助全文

通过发布文献求助，成功后即可免费获取论文全文。去求助

来源期刊

2018 IEEE/ACIS 17th International Conference on Computer and Information Science (ICIS)

自引率

0.00%

发文量