Experimental evaluation of a flexible I/O architecture for accelerating workflow engines in cloud environments

International Symposium on Design and Implementation of Symbolic Computation Systems Pub Date : 2015-11-15 DOI:10.1145/2831244.2831248

Francisco Rodrigo Duro, Francisco Javier García Blas, Florin Isaila, J. Carretero

{"title":"Experimental evaluation of a flexible I/O architecture for accelerating workflow engines in cloud environments","authors":"Francisco Rodrigo Duro, Francisco Javier García Blas, Florin Isaila, J. Carretero","doi":"10.1145/2831244.2831248","DOIUrl":null,"url":null,"abstract":"In the current scientific computing scenario storage systems are one of the main bottlenecks in computing platforms. This issue affects both traditional high performance computing systems and modern systems based on cloud platforms. Accelerating the I/O subsystems can improve the overall performance of the applications. In this paper, we present Hercules as an I/O accelerator specially designed for improving I/O access in workflow engines deployed over cloud-based infraestructures. Hercules provides a dynamic and flexible in-memory storage platform based on NoSQL-based distributed memory systems. In addition, Hercules offers a user-level interface based on POSIX for facilitating its usage on existing solutions and legacy applications. We have evaluated the proposed solution in a public cloud environment, in this case Amazon EC2. The results show that Hercules provides a scalable I/O solution with remarkable performance, especially for write operations, compared with classic I/O approaches for high performance computing in cloud environments.","PeriodicalId":166804,"journal":{"name":"International Symposium on Design and Implementation of Symbolic Computation Systems","volume":"78 1","pages":"0"},"PeriodicalIF":0.0000,"publicationDate":"2015-11-15","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"4","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"International Symposium on Design and Implementation of Symbolic Computation Systems","FirstCategoryId":"1085","ListUrlMain":"https://doi.org/10.1145/2831244.2831248","RegionNum":0,"RegionCategory":null,"ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"","JCRName":"","Score":null,"Total":0}

引用次数: 4

Abstract

In the current scientific computing scenario storage systems are one of the main bottlenecks in computing platforms. This issue affects both traditional high performance computing systems and modern systems based on cloud platforms. Accelerating the I/O subsystems can improve the overall performance of the applications. In this paper, we present Hercules as an I/O accelerator specially designed for improving I/O access in workflow engines deployed over cloud-based infraestructures. Hercules provides a dynamic and flexible in-memory storage platform based on NoSQL-based distributed memory systems. In addition, Hercules offers a user-level interface based on POSIX for facilitating its usage on existing solutions and legacy applications. We have evaluated the proposed solution in a public cloud environment, in this case Amazon EC2. The results show that Hercules provides a scalable I/O solution with remarkable performance, especially for write operations, compared with classic I/O approaches for high performance computing in cloud environments.

查看原文本刊更多论文

云环境中用于加速工作流引擎的灵活I/O架构的实验评估

在当前的科学计算场景中，存储系统是计算平台的主要瓶颈之一。这个问题既影响传统的高性能计算系统，也影响基于云平台的现代系统。加速I/O子系统可以提高应用程序的整体性能。在本文中，我们介绍了Hercules作为一个I/O加速器，专门设计用于改进部署在基于云的基础设施上的工作流引擎中的I/O访问。Hercules提供了一个基于nosql的分布式内存系统的动态、灵活的内存存储平台。此外，Hercules还提供了一个基于POSIX的用户级界面，以方便其在现有解决方案和遗留应用程序上的使用。我们已经在公共云环境中评估了建议的解决方案，在本例中是Amazon EC2。结果表明，与云环境中高性能计算的经典I/O方法相比，Hercules提供了一个性能卓越的可扩展I/O解决方案，特别是对于写操作。

本文章由计算机程序翻译，如有差异，请以英文原文为准。

求助全文

约1分钟内获得全文求助全文

来源期刊

International Symposium on Design and Implementation of Symbolic Computation Systems

自引率

0.00%

发文量