在Hadoop平台上使用R语言进行Twitter数据分析和可视化

2017 IEEE 15th International Symposium on Applied Machine Intelligence and Informatics (SAMI) Pub Date : 1900-01-01 DOI:10.1109/SAMI.2017.7880327

M. Sarnovský, P. Butka, Andrea Huzvarova

{"title":"在Hadoop平台上使用R语言进行Twitter数据分析和可视化","authors":"M. Sarnovský, P. Butka, Andrea Huzvarova","doi":"10.1109/SAMI.2017.7880327","DOIUrl":null,"url":null,"abstract":"The main objective of the work presented within this paper was to design and implement the system for twitter data analysis and visualization in R environment using the big data processing technologies. Our focus was to leverage existing big data processing frameworks with its storage and computational capabilities to support the analytical functions implemented in R language. We decided to build the backend on top of the Apache Hadoop framework including the Hadoop HDFS as a distributed filesystem and MapReduce as a distributed computation paradigm. RHadoop packages were then used to connect the R environment to the processing layer and to design and implement the analytical functions in a distributed manner. Visualizations were implemented on top of the solution as a RShiny application.","PeriodicalId":105599,"journal":{"name":"2017 IEEE 15th International Symposium on Applied Machine Intelligence and Informatics (SAMI)","volume":"7 1","pages":"0"},"PeriodicalIF":0.0000,"publicationDate":"1900-01-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"5","resultStr":"{\"title\":\"Twitter data analysis and visualizations using the R language on top of the Hadoop platform\",\"authors\":\"M. Sarnovský, P. Butka, Andrea Huzvarova\",\"doi\":\"10.1109/SAMI.2017.7880327\",\"DOIUrl\":null,\"url\":null,\"abstract\":\"The main objective of the work presented within this paper was to design and implement the system for twitter data analysis and visualization in R environment using the big data processing technologies. Our focus was to leverage existing big data processing frameworks with its storage and computational capabilities to support the analytical functions implemented in R language. We decided to build the backend on top of the Apache Hadoop framework including the Hadoop HDFS as a distributed filesystem and MapReduce as a distributed computation paradigm. RHadoop packages were then used to connect the R environment to the processing layer and to design and implement the analytical functions in a distributed manner. Visualizations were implemented on top of the solution as a RShiny application.\",\"PeriodicalId\":105599,\"journal\":{\"name\":\"2017 IEEE 15th International Symposium on Applied Machine Intelligence and Informatics (SAMI)\",\"volume\":\"7 1\",\"pages\":\"0\"},\"PeriodicalIF\":0.0000,\"publicationDate\":\"1900-01-01\",\"publicationTypes\":\"Journal Article\",\"fieldsOfStudy\":null,\"isOpenAccess\":false,\"openAccessPdf\":\"\",\"citationCount\":\"5\",\"resultStr\":null,\"platform\":\"Semanticscholar\",\"paperid\":null,\"PeriodicalName\":\"2017 IEEE 15th International Symposium on Applied Machine Intelligence and Informatics (SAMI)\",\"FirstCategoryId\":\"1085\",\"ListUrlMain\":\"https://doi.org/10.1109/SAMI.2017.7880327\",\"RegionNum\":0,\"RegionCategory\":null,\"ArticlePicture\":[],\"TitleCN\":null,\"AbstractTextCN\":null,\"PMCID\":null,\"EPubDate\":\"\",\"PubModel\":\"\",\"JCR\":\"\",\"JCRName\":\"\",\"Score\":null,\"Total\":0}","platform":"Semanticscholar","paperid":null,"PeriodicalName":"2017 IEEE 15th International Symposium on Applied Machine Intelligence and Informatics (SAMI)","FirstCategoryId":"1085","ListUrlMain":"https://doi.org/10.1109/SAMI.2017.7880327","RegionNum":0,"RegionCategory":null,"ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"","JCRName":"","Score":null,"Total":0}

引用次数: 5

摘要

本文提出的主要工作目标是利用大数据处理技术在R环境中设计和实现twitter数据分析和可视化系统。我们的重点是利用现有的大数据处理框架及其存储和计算能力来支持用R语言实现的分析功能。我们决定在Apache Hadoop框架之上构建后端，包括Hadoop HDFS作为分布式文件系统和MapReduce作为分布式计算范例。然后使用rha包将R环境连接到处理层，并以分布式的方式设计和实现分析功能。可视化是作为RShiny应用程序在解决方案之上实现的。

本文章由计算机程序翻译，如有差异，请以英文原文为准。

查看原文本刊更多论文

Twitter data analysis and visualizations using the R language on top of the Hadoop platform

The main objective of the work presented within this paper was to design and implement the system for twitter data analysis and visualization in R environment using the big data processing technologies. Our focus was to leverage existing big data processing frameworks with its storage and computational capabilities to support the analytical functions implemented in R language. We decided to build the backend on top of the Apache Hadoop framework including the Hadoop HDFS as a distributed filesystem and MapReduce as a distributed computation paradigm. RHadoop packages were then used to connect the R environment to the processing layer and to design and implement the analytical functions in a distributed manner. Visualizations were implemented on top of the solution as a RShiny application.

求助全文

通过发布文献求助，成功后即可免费获取论文全文。去求助

来源期刊

2017 IEEE 15th International Symposium on Applied Machine Intelligence and Informatics (SAMI)

自引率

0.00%

发文量