Freyr $^+$+: Harvesting Idle Resources in Serverless Computing via Deep Reinforcement Learning

IF 5.6 2区计算机科学 Q1 COMPUTER SCIENCE, THEORY & METHODS

IEEE Transactions on Parallel and Distributed Systems Pub Date : 2024-09-17 DOI:10.1109/TPDS.2024.3462294

Hanfei Yu;Hao Wang;Jian Li;Xu Yuan;Seung-Jong Park

{"title":"Freyr $^+$+: Harvesting Idle Resources in Serverless Computing via Deep Reinforcement Learning","authors":"Hanfei Yu;Hao Wang;Jian Li;Xu Yuan;Seung-Jong Park","doi":"10.1109/TPDS.2024.3462294","DOIUrl":null,"url":null,"abstract":"Serverless computing has revolutionized online service development and deployment with ease-to-use operations, auto-scaling, fine-grained resource allocation, and pay-as-you-go pricing. However, a gap remains in configuring serverless functions—the actual resource consumption may vary due to function types, dependencies, and input data sizes, thus mismatching the static resource configuration by users. Dynamic resource consumption against static configuration may lead to either poor function execution performance or low utilization. This paper proposes \nFreyr\n<inline-formula><tex-math>$^+$</tex-math></inline-formula>\n, a novel resource manager (RM) that dynamically harvests idle resources from over-provisioned functions to accelerate under-provisioned functions for serverless platforms. \nFreyr\n<inline-formula><tex-math>$^+$</tex-math></inline-formula>\n monitors each function's resource utilization in real-time and detects the mismatches between user configuration and actual resource consumption. We design deep reinforcement learning (DRL) algorithms with attention-enhanced embedding, incremental learning, and safeguard mechanism for \nFreyr\n<inline-formula><tex-math>$^+$</tex-math></inline-formula>\n to harvest idle resources safely and accelerate functions efficiently. We have implemented and deployed a \nFreyr\n<inline-formula><tex-math>$^+$</tex-math></inline-formula>\n prototype in a 13-node Apache OpenWhisk cluster using AWS EC2. \nFreyr\n<inline-formula><tex-math>$^+$</tex-math></inline-formula>\n is evaluated on both large-scale simulation and real-world testbed. Experimental results show that \nFreyr\n<inline-formula><tex-math>$^+$</tex-math></inline-formula>\n harvests 38% of function invocations’ idle resources and accelerates 39% of invocations using harvested resources. \nFreyr\n<inline-formula><tex-math>$^+$</tex-math></inline-formula>\n reduces the 99th-percentile function response latency by 26% compared to the baseline RMs.","PeriodicalId":13257,"journal":{"name":"IEEE Transactions on Parallel and Distributed Systems","volume":"35 11","pages":"2254-2269"},"PeriodicalIF":5.6000,"publicationDate":"2024-09-17","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"IEEE Transactions on Parallel and Distributed Systems","FirstCategoryId":"94","ListUrlMain":"https://ieeexplore.ieee.org/document/10682062/","RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q1","JCRName":"COMPUTER SCIENCE, THEORY & METHODS","Score":null,"Total":0}

引用次数: 0

Abstract

Serverless computing has revolutionized online service development and deployment with ease-to-use operations, auto-scaling, fine-grained resource allocation, and pay-as-you-go pricing. However, a gap remains in configuring serverless functions—the actual resource consumption may vary due to function types, dependencies, and input data sizes, thus mismatching the static resource configuration by users. Dynamic resource consumption against static configuration may lead to either poor function execution performance or low utilization. This paper proposes Freyr

$^+$

, a novel resource manager (RM) that dynamically harvests idle resources from over-provisioned functions to accelerate under-provisioned functions for serverless platforms. Freyr

$^+$

monitors each function's resource utilization in real-time and detects the mismatches between user configuration and actual resource consumption. We design deep reinforcement learning (DRL) algorithms with attention-enhanced embedding, incremental learning, and safeguard mechanism for Freyr

$^+$

to harvest idle resources safely and accelerate functions efficiently. We have implemented and deployed a Freyr

$^+$

prototype in a 13-node Apache OpenWhisk cluster using AWS EC2. Freyr

$^+$

is evaluated on both large-scale simulation and real-world testbed. Experimental results show that Freyr

$^+$

harvests 38% of function invocations’ idle resources and accelerates 39% of invocations using harvested resources. Freyr

$^+$

reduces the 99th-percentile function response latency by 26% compared to the baseline RMs.

查看原文本刊更多论文

Freyr$^+$：通过深度强化学习挖掘无服务器计算中的闲置资源

无服务器计算凭借易于使用的操作、自动缩放、细粒度资源分配和即用即付的定价方式，彻底改变了在线服务的开发和部署。然而，在配置无服务器功能方面仍存在差距--实际资源消耗可能会因功能类型、依赖关系和输入数据大小而变化，从而与用户的静态资源配置不匹配。与静态配置不匹配的动态资源消耗可能导致函数执行性能低下或利用率低。本文提出了一种新型资源管理器（RM）--Freyr$^+$，它可以动态地从超额配置的函数中获取闲置资源，从而加速无服务器平台中配置不足的函数。Freyr$^+$ 实时监控每个函数的资源利用率，并检测用户配置与实际资源消耗之间的不匹配。我们为Freyr$^+$设计了具有注意力增强嵌入、增量学习和保障机制的深度强化学习（DRL）算法，以安全地获取闲置资源并高效地加速函数。我们在使用 AWS EC2 的 13 节点 Apache OpenWhisk 集群中实施并部署了 Freyr$^+$ 原型。Freyr$^+$ 在大规模模拟和真实世界测试平台上进行了评估。实验结果表明，Freyr$^+$ 可收集 38% 的函数调用闲置资源，并利用收集的资源加速 39% 的调用。与基准 RM 相比，Freyr$^+$ 将第 99 百分位函数响应延迟降低了 26%。

本文章由计算机程序翻译，如有差异，请以英文原文为准。

求助全文

约1分钟内获得全文求助全文

来源期刊

IEEE Transactions on Parallel and Distributed Systems 工程技术-工程：电子与电气

CiteScore

11.00

自引率

9.40%

发文量

281

审稿时长

5.6 months

期刊介绍： IEEE Transactions on Parallel and Distributed Systems (TPDS) is published monthly. It publishes a range of papers, comments on previously published papers, and survey articles that deal with the parallel and distributed systems research areas of current importance to our readers. Particular areas of interest include, but are not limited to: a) Parallel and distributed algorithms, focusing on topics such as: models of computation; numerical, combinatorial, and data-intensive parallel algorithms, scalability of algorithms and data structures for parallel and distributed systems, communication and synchronization protocols, network algorithms, scheduling, and load balancing. b) Applications of parallel and distributed computing, including computational and data-enabled science and engineering, big data applications, parallel crowd sourcing, large-scale social network analysis, management of big data, cloud and grid computing, scientific and biomedical applications, mobile computing, and cyber-physical systems. c) Parallel and distributed architectures, including architectures for instruction-level and thread-level parallelism; design, analysis, implementation, fault resilience and performance measurements of multiple-processor systems; multicore processors, heterogeneous many-core systems; petascale and exascale systems designs; novel big data architectures; special purpose architectures, including graphics processors, signal processors, network processors, media accelerators, and other special purpose processors and accelerators; impact of technology on architecture; network and interconnect architectures; parallel I/O and storage systems; architecture of the memory hierarchy; power-efficient and green computing architectures; dependable architectures; and performance modeling and evaluation. d) Parallel and distributed software, including parallel and multicore programming languages and compilers, runtime systems, operating systems, Internet computing and web services, resource management including green computing, middleware for grids, clouds, and data centers, libraries, performance modeling and evaluation, parallel programming paradigms, and programming environments and tools.