Locality-Aware and Energy-Aware Job Pre-Assignment for Mapreduce

2016 International Conference on Intelligent Networking and Collaborative Systems (INCoS) Pub Date : 2016-09-01 DOI:10.1109/INCoS.2016.13

Lei Chen, Jing Zhang, Lijun Cai, Ziyun Deng, Tinqing He, Xu An Wang

{"title":"Locality-Aware and Energy-Aware Job Pre-Assignment for Mapreduce","authors":"Lei Chen, Jing Zhang, Lijun Cai, Ziyun Deng, Tinqing He, Xu An Wang","doi":"10.1109/INCoS.2016.13","DOIUrl":null,"url":null,"abstract":"Cloud Map-Reduce (CMR) is an advantage Map-Reduce platform and has been aroused more and more attention. To further balance the performance of job secluding among job cost, execution time and energy consumption, a locality-aware and energy-aware job pre-assignment algorithm is proposed for Map-Reduce of CMR in this paper. Firstly, the importance of rack in data locality and energy saving is analyzed. Secondly, a capacity pre-judged method is developed to measure the idea capacity of one rack for different jobs where the energy-efficient is defined to measure the balance statues of rack usage among job cost, execution time and energy consumption in job scheduling. Finally, based on the pre-judged idea capacity of racks, job pre-assignment method is proposed to centrally assign one job to virtual machines of several booked racks for saving energy and reducing communication. By comparing with other three algorithms, the extensive experimental results show our algorithm has good performance on job execution time, cross rack traffic, and energy consumption.","PeriodicalId":102056,"journal":{"name":"2016 International Conference on Intelligent Networking and Collaborative Systems (INCoS)","volume":"1 1","pages":"0"},"PeriodicalIF":0.0000,"publicationDate":"2016-09-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"3","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"2016 International Conference on Intelligent Networking and Collaborative Systems (INCoS)","FirstCategoryId":"1085","ListUrlMain":"https://doi.org/10.1109/INCoS.2016.13","RegionNum":0,"RegionCategory":null,"ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"","JCRName":"","Score":null,"Total":0}

引用次数: 3

Abstract

Cloud Map-Reduce (CMR) is an advantage Map-Reduce platform and has been aroused more and more attention. To further balance the performance of job secluding among job cost, execution time and energy consumption, a locality-aware and energy-aware job pre-assignment algorithm is proposed for Map-Reduce of CMR in this paper. Firstly, the importance of rack in data locality and energy saving is analyzed. Secondly, a capacity pre-judged method is developed to measure the idea capacity of one rack for different jobs where the energy-efficient is defined to measure the balance statues of rack usage among job cost, execution time and energy consumption in job scheduling. Finally, based on the pre-judged idea capacity of racks, job pre-assignment method is proposed to centrally assign one job to virtual machines of several booked racks for saving energy and reducing communication. By comparing with other three algorithms, the extensive experimental results show our algorithm has good performance on job execution time, cross rack traffic, and energy consumption.

查看原文本刊更多论文

Mapreduce的位置感知和能量感知作业预分配

Cloud Map-Reduce (CMR)是一种优势的Map-Reduce平台，已经引起了越来越多的关注。为了进一步平衡作业隔离性能在作业成本、执行时间和能耗之间的关系，本文提出了一种基于位置感知和能量感知的CMR Map-Reduce作业预分配算法。首先，分析了机架在数据局部性和节能方面的重要性。其次，提出了一种容量预判断方法来衡量一个机架对不同作业的思想容量，并定义了能效，以衡量作业调度中机架使用在作业成本、执行时间和能耗之间的平衡状态;最后，在预先判断机架思想容量的基础上，提出了作业预分配方法，将一个作业集中分配给多个已预订机架的虚拟机，以达到节能和减少通信的目的。通过与其他三种算法的比较，大量的实验结果表明，该算法在作业执行时间、跨机架流量和能耗方面具有良好的性能。

本文章由计算机程序翻译，如有差异，请以英文原文为准。

求助全文

约1分钟内获得全文求助全文

来源期刊

2016 International Conference on Intelligent Networking and Collaborative Systems (INCoS)

自引率

0.00%

发文量