Gloss

Q1 Computer Science

ACM Sigplan Notices Pub Date : 2018-03-19 DOI:10.1145/3296957.3173170

S. Rajadurai, Jeffrey Bosboom, W. Wong, S. Amarasinghe

{"title":"Gloss","authors":"S. Rajadurai, Jeffrey Bosboom, W. Wong, S. Amarasinghe","doi":"10.1145/3296957.3173170","DOIUrl":null,"url":null,"abstract":"\n An important class of applications computes on long-running or infinite streams of data, often with known fixed data rates. The latter is referred to as\n synchronous data flow\n ~(SDF) streams. These stream applications need to run on clusters or the cloud due to the high performance requirement. Further, they require live reconfiguration and reoptimization for various reasons such as hardware maintenance, elastic computation, or to respond to fluctuations in resources or application workload. However, reconfiguration and reoptimization without downtime while accurately preserving program state in a distributed environment is difficult. In this paper, we introduce Gloss, a suite of compiler and runtime techniques for live reconfiguration of distributed stream programs. Gloss, for the first time, avoids periods of zero throughput during the reconfiguration of both stateless and stateful SDF based stream programs. Furthermore, unlike other systems, Gloss globally reoptimizes and completely recompiles the program during reconfiguration. This permits it to reoptimize the application for entirely new configurations that it may not have encountered before. All these Gloss operations happen in-situ, requiring no extra hardware resources. We show how Gloss allows stream programs to reconfigure and reoptimize with no downtime and minimal overhead, and demonstrate the wider applicability of it via a variety of experiments.\n","PeriodicalId":50923,"journal":{"name":"ACM Sigplan Notices","volume":"344 1","pages":""},"PeriodicalIF":0.0000,"publicationDate":"2018-03-19","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"4","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"ACM Sigplan Notices","FirstCategoryId":"1085","ListUrlMain":"https://doi.org/10.1145/3296957.3173170","RegionNum":0,"RegionCategory":null,"ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q1","JCRName":"Computer Science","Score":null,"Total":0}

引用次数: 4

Abstract

An important class of applications computes on long-running or infinite streams of data, often with known fixed data rates. The latter is referred to as synchronous data flow ~(SDF) streams. These stream applications need to run on clusters or the cloud due to the high performance requirement. Further, they require live reconfiguration and reoptimization for various reasons such as hardware maintenance, elastic computation, or to respond to fluctuations in resources or application workload. However, reconfiguration and reoptimization without downtime while accurately preserving program state in a distributed environment is difficult. In this paper, we introduce Gloss, a suite of compiler and runtime techniques for live reconfiguration of distributed stream programs. Gloss, for the first time, avoids periods of zero throughput during the reconfiguration of both stateless and stateful SDF based stream programs. Furthermore, unlike other systems, Gloss globally reoptimizes and completely recompiles the program during reconfiguration. This permits it to reoptimize the application for entirely new configurations that it may not have encountered before. All these Gloss operations happen in-situ, requiring no extra hardware resources. We show how Gloss allows stream programs to reconfigure and reoptimize with no downtime and minimal overhead, and demonstrate the wider applicability of it via a variety of experiments.

查看原文本刊更多论文

光泽

一类重要的应用程序对长时间运行或无限的数据流进行计算，通常具有已知的固定数据速率。后者被称为同步数据流(SDF)流。由于高性能需求，这些流应用程序需要在集群或云上运行。此外，由于硬件维护、弹性计算或响应资源或应用程序工作负载的波动等各种原因，它们需要实时重新配置和重新优化。然而，要在不停机的情况下重新配置和重新优化，同时在分布式环境中准确地保持程序状态是很困难的。在本文中，我们介绍了Gloss，这是一套用于分布式流程序实时重构的编译器和运行时技术。Gloss首次避免了在无状态和有状态的基于SDF的流程序重新配置期间的零吞吐量。此外，与其他系统不同，Gloss在重新配置期间全局重新优化并完全重新编译程序。这允许它重新优化应用程序，以获得以前可能没有遇到过的全新配置。所有这些Gloss操作都在现场进行，不需要额外的硬件资源。我们展示了Gloss如何允许流程序在没有停机时间和最小开销的情况下重新配置和重新优化，并通过各种实验证明了它的更广泛的适用性。

本文章由计算机程序翻译，如有差异，请以英文原文为准。

求助全文

约1分钟内获得全文求助全文

来源期刊

ACM Sigplan Notices 工程技术-计算机：软件工程

CiteScore

4.90

自引率

0.00%

发文量

审稿时长

2-4 weeks

期刊介绍： The ACM Special Interest Group on Programming Languages explores programming language concepts and tools, focusing on design, implementation, practice, and theory. Its members are programming language developers, educators, implementers, researchers, theoreticians, and users. SIGPLAN sponsors several major annual conferences, including the Symposium on Principles of Programming Languages (POPL), the Symposium on Principles and Practice of Parallel Programming (PPoPP), the Conference on Programming Language Design and Implementation (PLDI), the International Conference on Functional Programming (ICFP), the International Conference on Object-Oriented Programming, Systems, Languages, and Applications (OOPSLA), as well as more than a dozen other events of either smaller size or in-cooperation with other SIGs. The monthly "ACM SIGPLAN Notices" publishes proceedings of selected sponsored events and an annual report on SIGPLAN activities. Members receive discounts on conference registrations and free access to ACM SIGPLAN publications in the ACM Digital Library. SIGPLAN recognizes significant research and service contributions of individuals with a variety of awards, supports current members through the Professional Activities Committee, and encourages future programming language enthusiasts with frequent Programming Languages Mentoring Workshops (PLMW).