资源竞争和任务失败下MapReduce的性能建模

2013 IEEE 5th International Conference on Cloud Computing Technology and Science Pub Date : 2013-12-02 DOI:10.1109/CloudCom.2013.28

Xiaolong Cui, Xuelian Lin, Chunming Hu, Richong Zhang, Chengzhang Wang

{"title":"资源竞争和任务失败下MapReduce的性能建模","authors":"Xiaolong Cui, Xuelian Lin, Chunming Hu, Richong Zhang, Chengzhang Wang","doi":"10.1109/CloudCom.2013.28","DOIUrl":null,"url":null,"abstract":"MapReduce is a widely used programming model for large scale data processing. In order to estimate the performance of MapReduce job and analyze the bottleneck of MapReduce job, a practical performance model for MapReduce is needed. Many works have been done on modeling the performance of MapReduce jobs. However, existing performance models ignore some important factors, such as I/O congestions and task failures over cluster, which may significantly change the execution costs of MapReduce job. This paper, aiming at predicting the execution time of a MapReduce job, presents an enhanced performance model that takes the resource contention and task failures into consideration. In addition, the experimental results show that the model is more accurate than those without considering the contention and failure factors.","PeriodicalId":198053,"journal":{"name":"2013 IEEE 5th International Conference on Cloud Computing Technology and Science","volume":"26 1","pages":"0"},"PeriodicalIF":0.0000,"publicationDate":"2013-12-02","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"16","resultStr":"{\"title\":\"Modeling the Performance of MapReduce under Resource Contentions and Task Failures\",\"authors\":\"Xiaolong Cui, Xuelian Lin, Chunming Hu, Richong Zhang, Chengzhang Wang\",\"doi\":\"10.1109/CloudCom.2013.28\",\"DOIUrl\":null,\"url\":null,\"abstract\":\"MapReduce is a widely used programming model for large scale data processing. In order to estimate the performance of MapReduce job and analyze the bottleneck of MapReduce job, a practical performance model for MapReduce is needed. Many works have been done on modeling the performance of MapReduce jobs. However, existing performance models ignore some important factors, such as I/O congestions and task failures over cluster, which may significantly change the execution costs of MapReduce job. This paper, aiming at predicting the execution time of a MapReduce job, presents an enhanced performance model that takes the resource contention and task failures into consideration. In addition, the experimental results show that the model is more accurate than those without considering the contention and failure factors.\",\"PeriodicalId\":198053,\"journal\":{\"name\":\"2013 IEEE 5th International Conference on Cloud Computing Technology and Science\",\"volume\":\"26 1\",\"pages\":\"0\"},\"PeriodicalIF\":0.0000,\"publicationDate\":\"2013-12-02\",\"publicationTypes\":\"Journal Article\",\"fieldsOfStudy\":null,\"isOpenAccess\":false,\"openAccessPdf\":\"\",\"citationCount\":\"16\",\"resultStr\":null,\"platform\":\"Semanticscholar\",\"paperid\":null,\"PeriodicalName\":\"2013 IEEE 5th International Conference on Cloud Computing Technology and Science\",\"FirstCategoryId\":\"1085\",\"ListUrlMain\":\"https://doi.org/10.1109/CloudCom.2013.28\",\"RegionNum\":0,\"RegionCategory\":null,\"ArticlePicture\":[],\"TitleCN\":null,\"AbstractTextCN\":null,\"PMCID\":null,\"EPubDate\":\"\",\"PubModel\":\"\",\"JCR\":\"\",\"JCRName\":\"\",\"Score\":null,\"Total\":0}","platform":"Semanticscholar","paperid":null,"PeriodicalName":"2013 IEEE 5th International Conference on Cloud Computing Technology and Science","FirstCategoryId":"1085","ListUrlMain":"https://doi.org/10.1109/CloudCom.2013.28","RegionNum":0,"RegionCategory":null,"ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"","JCRName":"","Score":null,"Total":0}

引用次数: 16

摘要

MapReduce是一种广泛应用于大规模数据处理的编程模型。为了评估MapReduce作业的性能和分析MapReduce作业的瓶颈，需要一个实用的MapReduce性能模型。在MapReduce作业的性能建模方面已经做了很多工作。然而，现有的性能模型忽略了一些重要的因素，如集群上的I/O拥塞和任务失败，这些因素可能会极大地改变MapReduce作业的执行成本。本文针对MapReduce作业的执行时间预测，提出了一种考虑资源争用和任务失败的增强性能模型。此外，实验结果表明，该模型比不考虑竞争和失效因素的模型更准确。

本文章由计算机程序翻译，如有差异，请以英文原文为准。

查看原文本刊更多论文

Modeling the Performance of MapReduce under Resource Contentions and Task Failures

MapReduce is a widely used programming model for large scale data processing. In order to estimate the performance of MapReduce job and analyze the bottleneck of MapReduce job, a practical performance model for MapReduce is needed. Many works have been done on modeling the performance of MapReduce jobs. However, existing performance models ignore some important factors, such as I/O congestions and task failures over cluster, which may significantly change the execution costs of MapReduce job. This paper, aiming at predicting the execution time of a MapReduce job, presents an enhanced performance model that takes the resource contention and task failures into consideration. In addition, the experimental results show that the model is more accurate than those without considering the contention and failure factors.

求助全文

通过发布文献求助，成功后即可免费获取论文全文。去求助

来源期刊

2013 IEEE 5th International Conference on Cloud Computing Technology and Science

自引率

0.00%

发文量