面向 hadoop 框架的成本感知优化资源配置 Map-Reduce 调度器

Archana Bhaskar, Rajeev Ranjan
{"title":"面向 hadoop 框架的成本感知优化资源配置 Map-Reduce 调度器","authors":"Archana Bhaskar, Rajeev Ranjan","doi":"10.11591/ijai.v13.i2.pp1262-1271","DOIUrl":null,"url":null,"abstract":"Distributed data processing model has been one of the primary components in the case of data-intensive applications; furthermore, due to advancements in technologies, there has been a huge volume of data generation of diverse nature. Hadoop map reduce framework is responsible for adopting the ease of deployment mechanism in an open-source framework. The existing Hadoop MapReduce framework possesses high makespan time and high Input/Output overhead and it mainly affects the cost of a model. Thus, this research work presents an optimized cost aware resource provisioning MapReduce model also known as the cost-effective resource provisioning MapReduce (CRP-MR) model. CRP-MR model introduces the two integrated approaches to minimize the cost; at first, this model presents the optimal resource optimization and optimal Input/Output optimization cleansing in the Hadoop MapReduce (HMR) scheduler. CRP-MR is evaluated considering the bioinformatics dataset and CRP-MR performs better than the existing model. ","PeriodicalId":507934,"journal":{"name":"IAES International Journal of Artificial Intelligence (IJ-AI)","volume":"29 47","pages":""},"PeriodicalIF":0.0000,"publicationDate":"2024-06-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":"{\"title\":\"Cost-aware optimal resource provisioning Map-Reduce scheduler for hadoop framework\",\"authors\":\"Archana Bhaskar, Rajeev Ranjan\",\"doi\":\"10.11591/ijai.v13.i2.pp1262-1271\",\"DOIUrl\":null,\"url\":null,\"abstract\":\"Distributed data processing model has been one of the primary components in the case of data-intensive applications; furthermore, due to advancements in technologies, there has been a huge volume of data generation of diverse nature. Hadoop map reduce framework is responsible for adopting the ease of deployment mechanism in an open-source framework. The existing Hadoop MapReduce framework possesses high makespan time and high Input/Output overhead and it mainly affects the cost of a model. Thus, this research work presents an optimized cost aware resource provisioning MapReduce model also known as the cost-effective resource provisioning MapReduce (CRP-MR) model. CRP-MR model introduces the two integrated approaches to minimize the cost; at first, this model presents the optimal resource optimization and optimal Input/Output optimization cleansing in the Hadoop MapReduce (HMR) scheduler. CRP-MR is evaluated considering the bioinformatics dataset and CRP-MR performs better than the existing model. \",\"PeriodicalId\":507934,\"journal\":{\"name\":\"IAES International Journal of Artificial Intelligence (IJ-AI)\",\"volume\":\"29 47\",\"pages\":\"\"},\"PeriodicalIF\":0.0000,\"publicationDate\":\"2024-06-01\",\"publicationTypes\":\"Journal Article\",\"fieldsOfStudy\":null,\"isOpenAccess\":false,\"openAccessPdf\":\"\",\"citationCount\":\"0\",\"resultStr\":null,\"platform\":\"Semanticscholar\",\"paperid\":null,\"PeriodicalName\":\"IAES International Journal of Artificial Intelligence (IJ-AI)\",\"FirstCategoryId\":\"1085\",\"ListUrlMain\":\"https://doi.org/10.11591/ijai.v13.i2.pp1262-1271\",\"RegionNum\":0,\"RegionCategory\":null,\"ArticlePicture\":[],\"TitleCN\":null,\"AbstractTextCN\":null,\"PMCID\":null,\"EPubDate\":\"\",\"PubModel\":\"\",\"JCR\":\"\",\"JCRName\":\"\",\"Score\":null,\"Total\":0}","platform":"Semanticscholar","paperid":null,"PeriodicalName":"IAES International Journal of Artificial Intelligence (IJ-AI)","FirstCategoryId":"1085","ListUrlMain":"https://doi.org/10.11591/ijai.v13.i2.pp1262-1271","RegionNum":0,"RegionCategory":null,"ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"","JCRName":"","Score":null,"Total":0}
引用次数: 0

摘要

分布式数据处理模型一直是数据密集型应用的主要组成部分之一;此外,由于技术的进步,产生了大量不同性质的数据。Hadoop MapReduce 框架负责在开源框架中采用易于部署的机制。现有的 Hadoop MapReduce 框架具有较长的运行时间和较高的输入/输出开销,这主要会影响模型的成本。因此,本研究工作提出了一种优化的成本感知资源配置 MapReduce 模型,也称为高性价比资源配置 MapReduce(CRP-MR)模型。CRP-MR 模型引入了两种综合方法来最小化成本;首先,该模型提出了 Hadoop MapReduce(HMR)调度器中的最优资源优化和最优输入/输出优化清洗。考虑到生物信息学数据集,对 CRP-MR 进行了评估,结果显示 CRP-MR 比现有模型表现更好。
本文章由计算机程序翻译,如有差异,请以英文原文为准。
Cost-aware optimal resource provisioning Map-Reduce scheduler for hadoop framework
Distributed data processing model has been one of the primary components in the case of data-intensive applications; furthermore, due to advancements in technologies, there has been a huge volume of data generation of diverse nature. Hadoop map reduce framework is responsible for adopting the ease of deployment mechanism in an open-source framework. The existing Hadoop MapReduce framework possesses high makespan time and high Input/Output overhead and it mainly affects the cost of a model. Thus, this research work presents an optimized cost aware resource provisioning MapReduce model also known as the cost-effective resource provisioning MapReduce (CRP-MR) model. CRP-MR model introduces the two integrated approaches to minimize the cost; at first, this model presents the optimal resource optimization and optimal Input/Output optimization cleansing in the Hadoop MapReduce (HMR) scheduler. CRP-MR is evaluated considering the bioinformatics dataset and CRP-MR performs better than the existing model. 
求助全文
通过发布文献求助,成功后即可免费获取论文全文。 去求助
来源期刊
自引率
0.00%
发文量
0
×
引用
GB/T 7714-2015
复制
MLA
复制
APA
复制
导出至
BibTeX EndNote RefMan NoteFirst NoteExpress
×
提示
您的信息不完整,为了账户安全,请先补充。
现在去补充
×
提示
您因"违规操作"
具体请查看互助需知
我知道了
×
提示
确定
请完成安全验证×
copy
已复制链接
快去分享给好友吧!
我知道了
右上角分享
点击右上角分享
0
联系我们:info@booksci.cn Book学术提供免费学术资源搜索服务,方便国内外学者检索中英文文献。致力于提供最便捷和优质的服务体验。 Copyright © 2023 布克学术 All rights reserved.
京ICP备2023020795号-1
ghs 京公网安备 11010802042870号
Book学术文献互助
Book学术文献互助群
群 号:481959085
Book学术官方微信