Foreseer:面向MapReduce的工作负载感知数据存储

2015 IEEE 35th International Conference on Distributed Computing Systems Pub Date : 2015-07-23 DOI:10.1109/ICDCS.2015.89

Jia Zou, Juwei Shi, Tongping Liu, Zhao Cao, Chen Wang

{"title":"Foreseer:面向MapReduce的工作负载感知数据存储","authors":"Jia Zou, Juwei Shi, Tongping Liu, Zhao Cao, Chen Wang","doi":"10.1109/ICDCS.2015.89","DOIUrl":null,"url":null,"abstract":"Inter-job Write once read many (WORM) scenario is ubiquitous in MapReduce applications that are widely deployed on enterprise production systems. However, traditional MapReduce auto-tuning techniques can not address the inter-job WORM scenario. To address the shortcomings in existing works, this work presents a novel online cross-layer solution, FORESEER. It can automatically predict workloads' data access information and tune data placement parameters to optimize the over-all performance for an inter-job WORM scenario. In our experiments, we observe that FORESEER can achieve significant performance speedup (up to 37%) compared with previous work.","PeriodicalId":129182,"journal":{"name":"2015 IEEE 35th International Conference on Distributed Computing Systems","volume":"23 1","pages":"0"},"PeriodicalIF":0.0000,"publicationDate":"2015-07-23","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"1","resultStr":"{\"title\":\"Foreseer: Workload-Aware Data Storage for MapReduce\",\"authors\":\"Jia Zou, Juwei Shi, Tongping Liu, Zhao Cao, Chen Wang\",\"doi\":\"10.1109/ICDCS.2015.89\",\"DOIUrl\":null,\"url\":null,\"abstract\":\"Inter-job Write once read many (WORM) scenario is ubiquitous in MapReduce applications that are widely deployed on enterprise production systems. However, traditional MapReduce auto-tuning techniques can not address the inter-job WORM scenario. To address the shortcomings in existing works, this work presents a novel online cross-layer solution, FORESEER. It can automatically predict workloads' data access information and tune data placement parameters to optimize the over-all performance for an inter-job WORM scenario. In our experiments, we observe that FORESEER can achieve significant performance speedup (up to 37%) compared with previous work.\",\"PeriodicalId\":129182,\"journal\":{\"name\":\"2015 IEEE 35th International Conference on Distributed Computing Systems\",\"volume\":\"23 1\",\"pages\":\"0\"},\"PeriodicalIF\":0.0000,\"publicationDate\":\"2015-07-23\",\"publicationTypes\":\"Journal Article\",\"fieldsOfStudy\":null,\"isOpenAccess\":false,\"openAccessPdf\":\"\",\"citationCount\":\"1\",\"resultStr\":null,\"platform\":\"Semanticscholar\",\"paperid\":null,\"PeriodicalName\":\"2015 IEEE 35th International Conference on Distributed Computing Systems\",\"FirstCategoryId\":\"1085\",\"ListUrlMain\":\"https://doi.org/10.1109/ICDCS.2015.89\",\"RegionNum\":0,\"RegionCategory\":null,\"ArticlePicture\":[],\"TitleCN\":null,\"AbstractTextCN\":null,\"PMCID\":null,\"EPubDate\":\"\",\"PubModel\":\"\",\"JCR\":\"\",\"JCRName\":\"\",\"Score\":null,\"Total\":0}","platform":"Semanticscholar","paperid":null,"PeriodicalName":"2015 IEEE 35th International Conference on Distributed Computing Systems","FirstCategoryId":"1085","ListUrlMain":"https://doi.org/10.1109/ICDCS.2015.89","RegionNum":0,"RegionCategory":null,"ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"","JCRName":"","Score":null,"Total":0}

引用次数: 1

摘要

WORM (Inter-job Write once read many)场景在广泛部署于企业生产系统的MapReduce应用中普遍存在。然而，传统的MapReduce自动调优技术不能解决作业间的WORM场景。为了解决现有工作中的不足，本工作提出了一种新颖的在线跨层解决方案FORESEER。它可以自动预测工作负载的数据访问信息，并调优数据放置参数，以优化作业间WORM场景的整体性能。在我们的实验中，我们观察到与以前的工作相比，FORESEER可以实现显着的性能加速(高达37%)。

本文章由计算机程序翻译，如有差异，请以英文原文为准。

查看原文本刊更多论文

Foreseer: Workload-Aware Data Storage for MapReduce

Inter-job Write once read many (WORM) scenario is ubiquitous in MapReduce applications that are widely deployed on enterprise production systems. However, traditional MapReduce auto-tuning techniques can not address the inter-job WORM scenario. To address the shortcomings in existing works, this work presents a novel online cross-layer solution, FORESEER. It can automatically predict workloads' data access information and tune data placement parameters to optimize the over-all performance for an inter-job WORM scenario. In our experiments, we observe that FORESEER can achieve significant performance speedup (up to 37%) compared with previous work.

求助全文

通过发布文献求助，成功后即可免费获取论文全文。去求助

来源期刊

2015 IEEE 35th International Conference on Distributed Computing Systems

自引率

0.00%

发文量