Towards an Energy Complexity Model for Distributed Data Processing Algorithms

IF 7.5 3区 计算机科学 Q1 COMPUTER SCIENCE, INFORMATION SYSTEMS
Jie Song;Xingchen Zhao;Chaopeng Guo;Yu Gu;Ge Yu
{"title":"Towards an Energy Complexity Model for Distributed Data Processing Algorithms","authors":"Jie Song;Xingchen Zhao;Chaopeng Guo;Yu Gu;Ge Yu","doi":"10.1109/TBDATA.2023.3284259","DOIUrl":null,"url":null,"abstract":"Modern data centers exist as infrastructure in the era of Big Data. Big data processing applications are the major computing workload of data centers. Electricity cost accounts for about 50% of data centers’ operational costs. Therefore, the energy consumed for running distributed data processing algorithms on a data center is starting to attract both academia and industry. Most works study the energy consumption from the hardware perspective and only a few of them from the algorithm perspective. A general and hardware-independent energy evaluation model for the algorithms is in demand. With the model, algorithm designers can evaluate the energy consumption, compare energy consumption features and facilitate energy consumption optimization of distributed data processing algorithms. Inspired by the time complexity model, we propose an energy complexity model for describing the trends that an algorithm's energy consumption grows with the algorithm's input size. We argue that a good algorithm, especially for processing Big Data, should have a ‘small’ energy complexity. We define \n<inline-formula><tex-math>$E(n)$</tex-math></inline-formula>\n to represent the functional relationship that associates an algorithm's input size \n<inline-formula><tex-math>$n$</tex-math></inline-formula>\n with its notional energy consumption \n<inline-formula><tex-math>$E$</tex-math></inline-formula>\n. Based on the well-known abstract Bulk Synchronous Parallel (BSP) computer and programming model, we present a complete \n<inline-formula><tex-math>$E(n)$</tex-math></inline-formula>\n solution, including abstraction, generalization, quantification, derivation, comparison, analysis, examples, verification, and applications. Comprehensive experimental analysis shows that the proposed energy complexity model is practical, interestingly, and not equivalent to time complexity.","PeriodicalId":13106,"journal":{"name":"IEEE Transactions on Big Data","volume":"9 6","pages":"1510-1524"},"PeriodicalIF":7.5000,"publicationDate":"2023-06-08","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"IEEE Transactions on Big Data","FirstCategoryId":"94","ListUrlMain":"https://ieeexplore.ieee.org/document/10146456/","RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q1","JCRName":"COMPUTER SCIENCE, INFORMATION SYSTEMS","Score":null,"Total":0}
引用次数: 0

Abstract

Modern data centers exist as infrastructure in the era of Big Data. Big data processing applications are the major computing workload of data centers. Electricity cost accounts for about 50% of data centers’ operational costs. Therefore, the energy consumed for running distributed data processing algorithms on a data center is starting to attract both academia and industry. Most works study the energy consumption from the hardware perspective and only a few of them from the algorithm perspective. A general and hardware-independent energy evaluation model for the algorithms is in demand. With the model, algorithm designers can evaluate the energy consumption, compare energy consumption features and facilitate energy consumption optimization of distributed data processing algorithms. Inspired by the time complexity model, we propose an energy complexity model for describing the trends that an algorithm's energy consumption grows with the algorithm's input size. We argue that a good algorithm, especially for processing Big Data, should have a ‘small’ energy complexity. We define $E(n)$ to represent the functional relationship that associates an algorithm's input size $n$ with its notional energy consumption $E$ . Based on the well-known abstract Bulk Synchronous Parallel (BSP) computer and programming model, we present a complete $E(n)$ solution, including abstraction, generalization, quantification, derivation, comparison, analysis, examples, verification, and applications. Comprehensive experimental analysis shows that the proposed energy complexity model is practical, interestingly, and not equivalent to time complexity.
分布式数据处理算法的能量复杂度模型
现代数据中心作为大数据时代的基础设施而存在。大数据处理应用是数据中心的主要计算工作量。电力成本约占数据中心运营成本的50%。因此,在数据中心运行分布式数据处理算法所消耗的能量开始引起学术界和工业界的关注。大多数研究从硬件角度研究能耗,从算法角度研究能耗的研究很少。需要一种通用的、与硬件无关的算法能量评估模型。通过该模型,算法设计者可以对分布式数据处理算法的能耗进行评估,比较能耗特征,便于对分布式数据处理算法进行能耗优化。受时间复杂度模型的启发,我们提出了一个能量复杂度模型来描述算法的能量消耗随算法输入规模的增长趋势。我们认为,一个好的算法,尤其是处理大数据的算法,应该具有“小”的能量复杂度。我们定义$E(n)$来表示将算法的输入大小$n$与其名义能耗$E$相关联的函数关系。基于著名的批量同步并行(Bulk Synchronous Parallel, BSP)计算机和编程模型,我们提出了一个完整的$E(n)$解决方案,包括抽象、概括、量化、推导、比较、分析、实例、验证和应用。综合实验分析表明,所提出的能量复杂度模型具有实用性和趣味性,且不等同于时间复杂度。
本文章由计算机程序翻译,如有差异,请以英文原文为准。
求助全文
约1分钟内获得全文 求助全文
来源期刊
CiteScore
11.80
自引率
2.80%
发文量
114
期刊介绍: The IEEE Transactions on Big Data publishes peer-reviewed articles focusing on big data. These articles present innovative research ideas and application results across disciplines, including novel theories, algorithms, and applications. Research areas cover a wide range, such as big data analytics, visualization, curation, management, semantics, infrastructure, standards, performance analysis, intelligence extraction, scientific discovery, security, privacy, and legal issues specific to big data. The journal also prioritizes applications of big data in fields generating massive datasets.
×
引用
GB/T 7714-2015
复制
MLA
复制
APA
复制
导出至
BibTeX EndNote RefMan NoteFirst NoteExpress
×
提示
您的信息不完整,为了账户安全,请先补充。
现在去补充
×
提示
您因"违规操作"
具体请查看互助需知
我知道了
×
提示
确定
请完成安全验证×
copy
已复制链接
快去分享给好友吧!
我知道了
右上角分享
点击右上角分享
0
联系我们:info@booksci.cn Book学术提供免费学术资源搜索服务,方便国内外学者检索中英文文献。致力于提供最便捷和优质的服务体验。 Copyright © 2023 布克学术 All rights reserved.
京ICP备2023020795号-1
ghs 京公网安备 11010802042870号
Book学术文献互助
Book学术文献互助群
群 号:481959085
Book学术官方微信