S. Saravanan, V. Venkatachalam
{"title":"Advance Map Reduce Task Scheduling algorithm using mobile cloud multimedia services architecture","authors":"S. Saravanan, V. Venkatachalam","doi":"10.1109/ICOAC.2014.7229736","DOIUrl":null,"url":null,"abstract":"There is a massive improvement in the computer technology which leads to infinite number of resources in all over the world stored in the computer. Computing devices have several uses and are necessary for businesses, scientists, governments, engineers. These have to generate data that comes from anywhere. Sensors gathering climate data, a person posting to a social media site, or a cell phone. Furthermore, the data itself may be too large to store on a single machine. In order to reduce the time it takes to process the data, and to have the storage space to store the data, we introduce a technique called map reduce programming model. In this programming method, it has to divide the workload among computers in a network. As a result, the performance of Map Reduce strongly depends on how evenly it distributes this workload among the computer. In Map Reduce, workload distribution depends on the algorithm that partitions the data. To avoid the problems of uneven distribution of data we use data sampling. By using the partitioning mechanism, the partitioned distributes the data depends on how large and representative the sample is and on how well the samples are analyzed. Due to this improves load balancing and memory consumption of the computers. In addition to that we use micro-partitioning methods to divide the workload into small tasks that are dynamically scheduled at runtime. This approach is only effective in systems with high-throughput, low-latency task schedulers and efficient data materialization. To enhance the accuracy in scheduling we propose an innovative method called Map Reduce Task Scheduling algorithm for Deadline constraints. This method allows user specify a job's deadline and tries to make the job be finished before the deadline. Through measuring the node's computing capacity, a node classification algorithm is proposed in MTSD. This algorithm classifies the nodes into several levels in heterogeneous clusters. Under this algorithm, we firstly illuminate a novel data distribution model which distributes data according to the node's capacity level respectively.","PeriodicalId":325520,"journal":{"name":"2014 Sixth International Conference on Advanced Computing (ICoAC)","volume":"19 1","pages":"0"},"PeriodicalIF":0.0000,"publicationDate":"2014-12-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"16","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"2014 Sixth International Conference on Advanced Computing (ICoAC)","FirstCategoryId":"1085","ListUrlMain":"https://doi.org/10.1109/ICOAC.2014.7229736","RegionNum":0,"RegionCategory":null,"ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"","JCRName":"","Score":null,"Total":0}
引用次数: 16

摘要

计算机技术有了巨大的进步,这使得世界各地的无数资源都存储在计算机中。计算设备有多种用途,是企业、科学家、政府和工程师所必需的。它们必须生成来自任何地方的数据。收集气候数据的传感器,一个人在社交媒体网站上发帖,或者一部手机。此外,数据本身可能太大,无法存储在一台机器上。为了减少处理数据所需的时间,并有足够的存储空间来存储数据,我们引入了一种称为映射约简编程模型的技术。在这种编程方法中,它必须在网络中的计算机之间划分工作负载。因此,Map Reduce的性能在很大程度上取决于它在计算机之间均匀分配工作负载的程度。在Map Reduce中,工作负载分布取决于对数据进行分区的算法。为了避免数据分布不均匀的问题,我们使用数据抽样。通过使用分区机制,分区的数据分布取决于样本的大小和代表性,以及样本的分析效果。由于这提高了计算机的负载平衡和内存消耗。除此之外,我们还使用微分区方法将工作负载划分为在运行时动态调度的小任务。这种方法仅在具有高吞吐量、低延迟任务调度器和高效数据物化的系统中有效。为了提高任务调度的准确性,提出了一种基于时间约束的Map Reduce任务调度算法。此方法允许用户指定作业的截止日期,并尝试在截止日期之前完成作业。通过测量节点的计算能力,提出了一种MTSD中的节点分类算法。该算法将异构集群中的节点划分为若干级别。在该算法中,我们首先提出了一种新的数据分布模型,该模型根据节点的容量级别分别进行数据分布。
本文章由计算机程序翻译,如有差异,请以英文原文为准。
Advance Map Reduce Task Scheduling algorithm using mobile cloud multimedia services architecture
There is a massive improvement in the computer technology which leads to infinite number of resources in all over the world stored in the computer. Computing devices have several uses and are necessary for businesses, scientists, governments, engineers. These have to generate data that comes from anywhere. Sensors gathering climate data, a person posting to a social media site, or a cell phone. Furthermore, the data itself may be too large to store on a single machine. In order to reduce the time it takes to process the data, and to have the storage space to store the data, we introduce a technique called map reduce programming model. In this programming method, it has to divide the workload among computers in a network. As a result, the performance of Map Reduce strongly depends on how evenly it distributes this workload among the computer. In Map Reduce, workload distribution depends on the algorithm that partitions the data. To avoid the problems of uneven distribution of data we use data sampling. By using the partitioning mechanism, the partitioned distributes the data depends on how large and representative the sample is and on how well the samples are analyzed. Due to this improves load balancing and memory consumption of the computers. In addition to that we use micro-partitioning methods to divide the workload into small tasks that are dynamically scheduled at runtime. This approach is only effective in systems with high-throughput, low-latency task schedulers and efficient data materialization. To enhance the accuracy in scheduling we propose an innovative method called Map Reduce Task Scheduling algorithm for Deadline constraints. This method allows user specify a job's deadline and tries to make the job be finished before the deadline. Through measuring the node's computing capacity, a node classification algorithm is proposed in MTSD. This algorithm classifies the nodes into several levels in heterogeneous clusters. Under this algorithm, we firstly illuminate a novel data distribution model which distributes data according to the node's capacity level respectively.
求助全文
通过发布文献求助,成功后即可免费获取论文全文。 去求助
来源期刊
自引率
0.00%
发文量
0
×
引用
GB/T 7714-2015
复制
MLA
复制
APA
复制
导出至
BibTeX EndNote RefMan NoteFirst NoteExpress
×
提示
您的信息不完整,为了账户安全,请先补充。
现在去补充
×
提示
您因"违规操作"
具体请查看互助需知
我知道了
×
提示
确定
请完成安全验证×
copy
已复制链接
快去分享给好友吧!
我知道了
右上角分享
点击右上角分享
0
联系我们:info@booksci.cn Book学术提供免费学术资源搜索服务,方便国内外学者检索中英文文献。致力于提供最便捷和优质的服务体验。 Copyright © 2023 布克学术 All rights reserved.
京ICP备2023020795号-1
ghs 京公网安备 11010802042870号
Book学术文献互助
Book学术文献互助群
群 号:604180095
Book学术官方微信