SWEEP: Accelerating Scientific Research Through Scalable Serverless Workflows

Proceedings of the 12th IEEE/ACM International Conference on Utility and Cloud Computing Companion Pub Date : 2019-12-02 DOI:10.1145/3368235.3368839

Aji John, Kristiina Ausmees, Kathleen Muenzen, Catherine Kuhn, A. Tan

{"title":"SWEEP: Accelerating Scientific Research Through Scalable Serverless Workflows","authors":"Aji John, Kristiina Ausmees, Kathleen Muenzen, Catherine Kuhn, A. Tan","doi":"10.1145/3368235.3368839","DOIUrl":null,"url":null,"abstract":"Scientific and commercial applications are increasingly being executed in the cloud, but the difficulties associated with cluster management render on-demand resources inaccessible or inefficient to many users. Recently, the serverless execution model, in which the provisioning of resources is abstracted from the user, has gained prominence as an alternative to traditional cyberinfrastructure solutions. With its inherent elasticity, the serverless paradigm constitutes a promising computational model for scientific workflows, allowing domain specialists to develop and deploy workflows that are subject to varying workloads and intermittent usage without the overhead of infrastructure maintenance. We present the Serverless Workflow Enablement and Execution Platform (SWEEP), a cloud-agnostic workflow management system with a purely serverless execution model that allows users to define, run and monitor generic cloud-native workflows. We demonstrate the use of SWEEP on workflows from two disparate scientific domains and present an evaluation of performance and scaling.","PeriodicalId":166357,"journal":{"name":"Proceedings of the 12th IEEE/ACM International Conference on Utility and Cloud Computing Companion","volume":"123 1","pages":"0"},"PeriodicalIF":0.0000,"publicationDate":"2019-12-02","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"18","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"Proceedings of the 12th IEEE/ACM International Conference on Utility and Cloud Computing Companion","FirstCategoryId":"1085","ListUrlMain":"https://doi.org/10.1145/3368235.3368839","RegionNum":0,"RegionCategory":null,"ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"","JCRName":"","Score":null,"Total":0}

引用次数: 18

Abstract

Scientific and commercial applications are increasingly being executed in the cloud, but the difficulties associated with cluster management render on-demand resources inaccessible or inefficient to many users. Recently, the serverless execution model, in which the provisioning of resources is abstracted from the user, has gained prominence as an alternative to traditional cyberinfrastructure solutions. With its inherent elasticity, the serverless paradigm constitutes a promising computational model for scientific workflows, allowing domain specialists to develop and deploy workflows that are subject to varying workloads and intermittent usage without the overhead of infrastructure maintenance. We present the Serverless Workflow Enablement and Execution Platform (SWEEP), a cloud-agnostic workflow management system with a purely serverless execution model that allows users to define, run and monitor generic cloud-native workflows. We demonstrate the use of SWEEP on workflows from two disparate scientific domains and present an evaluation of performance and scaling.

查看原文本刊更多论文

SWEEP:通过可扩展的无服务器工作流加速科学研究

科学和商业应用程序越来越多地在云中执行，但是与集群管理相关的困难使得许多用户无法访问按需资源或效率低下。最近，无服务器执行模型作为传统网络基础设施解决方案的替代方案获得了突出的地位，其中资源的供应是从用户中抽象出来的。由于其固有的弹性，无服务器范式为科学工作流构成了一个有前途的计算模型，允许领域专家开发和部署受不同工作负载和间歇性使用影响的工作流，而不需要基础设施维护的开销。我们提出了无服务器工作流启用和执行平台(SWEEP)，这是一个与云无关的工作流管理系统，具有纯粹的无服务器执行模型，允许用户定义、运行和监控通用的云原生工作流。我们演示了在两个不同的科学领域的工作流上使用SWEEP，并给出了性能和可伸缩性的评估。

本文章由计算机程序翻译，如有差异，请以英文原文为准。

求助全文

约1分钟内获得全文求助全文

来源期刊

Proceedings of the 12th IEEE/ACM International Conference on Utility and Cloud Computing Companion

自引率

0.00%

发文量