基于Spark实现的Web分析查询引擎Spindle的性能研究

2014 IEEE 6th International Conference on Cloud Computing Technology and Science Pub Date : 2014-12-15 DOI:10.1109/CloudCom.2014.111

Brandon Amos, David Tompkins

{"title":"基于Spark实现的Web分析查询引擎Spindle的性能研究","authors":"Brandon Amos, David Tompkins","doi":"10.1109/CloudCom.2014.111","DOIUrl":null,"url":null,"abstract":"This paper shares our experiences building and benchmarking Spindle as an open source Spark-based web analytics platform. Spindle's design has been motivated by real-world queries and data requiring concurrent, low latency query execution. We identify a search space of Spark tuning options and study their impact on Spark's performance. Results from a self-hosted six node cluster with one week of analytics data (13.1GB) indicate tuning options such as proper partitioning can cause a 5x performance improvement.","PeriodicalId":249306,"journal":{"name":"2014 IEEE 6th International Conference on Cloud Computing Technology and Science","volume":"3 1","pages":"0"},"PeriodicalIF":0.0000,"publicationDate":"2014-12-15","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"3","resultStr":"{\"title\":\"Performance Study of Spindle, A Web Analytics Query Engine Implemented in Spark\",\"authors\":\"Brandon Amos, David Tompkins\",\"doi\":\"10.1109/CloudCom.2014.111\",\"DOIUrl\":null,\"url\":null,\"abstract\":\"This paper shares our experiences building and benchmarking Spindle as an open source Spark-based web analytics platform. Spindle's design has been motivated by real-world queries and data requiring concurrent, low latency query execution. We identify a search space of Spark tuning options and study their impact on Spark's performance. Results from a self-hosted six node cluster with one week of analytics data (13.1GB) indicate tuning options such as proper partitioning can cause a 5x performance improvement.\",\"PeriodicalId\":249306,\"journal\":{\"name\":\"2014 IEEE 6th International Conference on Cloud Computing Technology and Science\",\"volume\":\"3 1\",\"pages\":\"0\"},\"PeriodicalIF\":0.0000,\"publicationDate\":\"2014-12-15\",\"publicationTypes\":\"Journal Article\",\"fieldsOfStudy\":null,\"isOpenAccess\":false,\"openAccessPdf\":\"\",\"citationCount\":\"3\",\"resultStr\":null,\"platform\":\"Semanticscholar\",\"paperid\":null,\"PeriodicalName\":\"2014 IEEE 6th International Conference on Cloud Computing Technology and Science\",\"FirstCategoryId\":\"1085\",\"ListUrlMain\":\"https://doi.org/10.1109/CloudCom.2014.111\",\"RegionNum\":0,\"RegionCategory\":null,\"ArticlePicture\":[],\"TitleCN\":null,\"AbstractTextCN\":null,\"PMCID\":null,\"EPubDate\":\"\",\"PubModel\":\"\",\"JCR\":\"\",\"JCRName\":\"\",\"Score\":null,\"Total\":0}","platform":"Semanticscholar","paperid":null,"PeriodicalName":"2014 IEEE 6th International Conference on Cloud Computing Technology and Science","FirstCategoryId":"1085","ListUrlMain":"https://doi.org/10.1109/CloudCom.2014.111","RegionNum":0,"RegionCategory":null,"ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"","JCRName":"","Score":null,"Total":0}

引用次数: 3

摘要

本文将分享我们构建和测试Spindle作为一个基于spark的开源web分析平台的经验。Spindle的设计是由现实世界的查询和需要并发、低延迟查询执行的数据驱动的。我们确定了Spark调优选项的搜索空间，并研究了它们对Spark性能的影响。一个带有一周分析数据(13.1GB)的自托管六节点集群的结果表明，适当分区等调优选项可以使性能提高5倍。

本文章由计算机程序翻译，如有差异，请以英文原文为准。

查看原文本刊更多论文

Performance Study of Spindle, A Web Analytics Query Engine Implemented in Spark

This paper shares our experiences building and benchmarking Spindle as an open source Spark-based web analytics platform. Spindle's design has been motivated by real-world queries and data requiring concurrent, low latency query execution. We identify a search space of Spark tuning options and study their impact on Spark's performance. Results from a self-hosted six node cluster with one week of analytics data (13.1GB) indicate tuning options such as proper partitioning can cause a 5x performance improvement.

求助全文

通过发布文献求助，成功后即可免费获取论文全文。去求助

来源期刊

2014 IEEE 6th International Conference on Cloud Computing Technology and Science

自引率

0.00%

发文量