T. Dalman, Tim Dörnemann, Ernst Juhnke, M. Weitzel, Matthew Smith, W. Wiechert, K. Nöh, Bernd Freisleben
{"title":"云中的代谢通量分析","authors":"T. Dalman, Tim Dörnemann, Ernst Juhnke, M. Weitzel, Matthew Smith, W. Wiechert, K. Nöh, Bernd Freisleben","doi":"10.1109/ESCIENCE.2010.20","DOIUrl":null,"url":null,"abstract":"The MapReduce pattern popularized by Google has successfully been utilized in several scientific applications. In this paper, it is investigated whether a MapReduce approach utilizing on-demand resources from a Cloud is beneficial to perform simulation tasks in the area of Systems Biology and whether it can be seamlessly integrated into a service-oriented scientific workflow framework. In particular, an Amazon Elastic Map Reduce Cloud implementation of the 13C-MFA (Metabolix Flux Analysis) Monte Carlo bootstrap approach aimed at the integration into an existing BPEL-based scientific workflow system is presented. A comparison of a 64 node MapReduce cluster with a single node computation approach reveals a total performance gain up to a factor of 14, with a total cost for on-demand resources of $11. The most critical factor in terms of performance is I/O, i.e. our application suffers from the fact that I/O operations on many small files are expensive using Amazon S3 and the Hadoop DFS.","PeriodicalId":441488,"journal":{"name":"2010 IEEE Sixth International Conference on e-Science","volume":"23 1","pages":"0"},"PeriodicalIF":0.0000,"publicationDate":"2010-12-07","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"19","resultStr":"{\"title\":\"Metabolic Flux Analysis in the Cloud\",\"authors\":\"T. Dalman, Tim Dörnemann, Ernst Juhnke, M. Weitzel, Matthew Smith, W. Wiechert, K. Nöh, Bernd Freisleben\",\"doi\":\"10.1109/ESCIENCE.2010.20\",\"DOIUrl\":null,\"url\":null,\"abstract\":\"The MapReduce pattern popularized by Google has successfully been utilized in several scientific applications. In this paper, it is investigated whether a MapReduce approach utilizing on-demand resources from a Cloud is beneficial to perform simulation tasks in the area of Systems Biology and whether it can be seamlessly integrated into a service-oriented scientific workflow framework. In particular, an Amazon Elastic Map Reduce Cloud implementation of the 13C-MFA (Metabolix Flux Analysis) Monte Carlo bootstrap approach aimed at the integration into an existing BPEL-based scientific workflow system is presented. A comparison of a 64 node MapReduce cluster with a single node computation approach reveals a total performance gain up to a factor of 14, with a total cost for on-demand resources of $11. The most critical factor in terms of performance is I/O, i.e. our application suffers from the fact that I/O operations on many small files are expensive using Amazon S3 and the Hadoop DFS.\",\"PeriodicalId\":441488,\"journal\":{\"name\":\"2010 IEEE Sixth International Conference on e-Science\",\"volume\":\"23 1\",\"pages\":\"0\"},\"PeriodicalIF\":0.0000,\"publicationDate\":\"2010-12-07\",\"publicationTypes\":\"Journal Article\",\"fieldsOfStudy\":null,\"isOpenAccess\":false,\"openAccessPdf\":\"\",\"citationCount\":\"19\",\"resultStr\":null,\"platform\":\"Semanticscholar\",\"paperid\":null,\"PeriodicalName\":\"2010 IEEE Sixth International Conference on e-Science\",\"FirstCategoryId\":\"1085\",\"ListUrlMain\":\"https://doi.org/10.1109/ESCIENCE.2010.20\",\"RegionNum\":0,\"RegionCategory\":null,\"ArticlePicture\":[],\"TitleCN\":null,\"AbstractTextCN\":null,\"PMCID\":null,\"EPubDate\":\"\",\"PubModel\":\"\",\"JCR\":\"\",\"JCRName\":\"\",\"Score\":null,\"Total\":0}","platform":"Semanticscholar","paperid":null,"PeriodicalName":"2010 IEEE Sixth International Conference on e-Science","FirstCategoryId":"1085","ListUrlMain":"https://doi.org/10.1109/ESCIENCE.2010.20","RegionNum":0,"RegionCategory":null,"ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"","JCRName":"","Score":null,"Total":0}
引用次数: 19
摘要
由谷歌推广的MapReduce模式已经成功地应用于多个科学应用中。本文研究了利用云资源的MapReduce方法是否有利于执行系统生物学领域的模拟任务，以及它是否可以无缝集成到面向服务的科学工作流框架中。特别地，Amazon Elastic Map Reduce Cloud实现了13C-MFA (Metabolix Flux Analysis) Monte Carlo bootstrap方法，旨在集成到现有的基于bpel的科学工作流系统中。64节点MapReduce集群与单节点计算方法的比较显示，总性能增益高达14倍，按需资源的总成本为11美元。就性能而言，最关键的因素是I/O，也就是说，我们的应用程序在使用Amazon S3和Hadoop DFS时，对许多小文件进行I/O操作是非常昂贵的。
The MapReduce pattern popularized by Google has successfully been utilized in several scientific applications. In this paper, it is investigated whether a MapReduce approach utilizing on-demand resources from a Cloud is beneficial to perform simulation tasks in the area of Systems Biology and whether it can be seamlessly integrated into a service-oriented scientific workflow framework. In particular, an Amazon Elastic Map Reduce Cloud implementation of the 13C-MFA (Metabolix Flux Analysis) Monte Carlo bootstrap approach aimed at the integration into an existing BPEL-based scientific workflow system is presented. A comparison of a 64 node MapReduce cluster with a single node computation approach reveals a total performance gain up to a factor of 14, with a total cost for on-demand resources of $11. The most critical factor in terms of performance is I/O, i.e. our application suffers from the fact that I/O operations on many small files are expensive using Amazon S3 and the Hadoop DFS.
求助内容：
应助结果提醒方式：
京公网安备 11010802042870号
