发布求助
文献互助 智能选刊 最新文献

Water & sewage works最新文献

筛选
英文 中文
Failure prediction and localization in large scientific workflows 大型科研工作流程中的故障预测与定位
Water & sewage works Pub Date : 2011-11-14 DOI: 10.1145/2110497.2110510
T. Samak, D. Gunter, M. Goode, E. Deelman, Gaurang Mehta, Fabio Silva, K. Vahi
{"title":"Failure prediction and localization in large scientific workflows","authors":"T. Samak, D. Gunter, M. Goode, E. Deelman, Gaurang Mehta, Fabio Silva, K. Vahi","doi":"10.1145/2110497.2110510","DOIUrl":"https://doi.org/10.1145/2110497.2110510","url":null,"abstract":"Scientific workflows provide a portable representation for scientific applications' coordinated input, output, and execution management for highly parallel executions of interdependent computations, as well as support for sharing and validating the results. As scientific workflows scale to hundreds of thousands of distinct tasks, failures due to software and hardware faults become increasingly common. Real-time execution monitoring provides a foundation for improving the transparency and resilience of the workflows in the face of stochastic and systematic faults. Building on previous work on early detection of these failure scenarios, we describe methods for guiding remediation to stochastic errors through predictions of the impact on application performance. To complement this analysis, we also describe techniques for isolating systematic sources of failures. We evaluate our methods on a representative sample of large real-world workflows.","PeriodicalId":87858,"journal":{"name":"Water & sewage works","volume":"4 1","pages":"107-116"},"PeriodicalIF":0.0,"publicationDate":"2011-11-14","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"83364771","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 24
Workflow overhead analysis and optimizations 工作流开销分析和优化
Water & sewage works Pub Date : 2011-11-14 DOI: 10.1145/2110497.2110500
Weiwei Chen, E. Deelman
{"title":"Workflow overhead analysis and optimizations","authors":"Weiwei Chen, E. Deelman","doi":"10.1145/2110497.2110500","DOIUrl":"https://doi.org/10.1145/2110497.2110500","url":null,"abstract":"The execution of scientific workflows often suffers from a variety of overheads in distributed environments. It is essential to identify the different overheads and to evaluate how optimization methods help reduce overheads and improve runtime performance. In this paper, we present an overhead analysis for a set of workflow runs on cloud and grid platforms. We present the overhead distributions and conclude that they satisfy an exponential or uniform distribution. We compare three methods to calculate the cumulative sum of the overheads based on how they overlap. In addition, we indicate how experimental parameters impact the overhead and thereby the overall workflow performance. We then show how popular optimization methods improve runtime performance by reducing some or all types of overheads.","PeriodicalId":87858,"journal":{"name":"Water & sewage works","volume":"48 6 1","pages":"11-20"},"PeriodicalIF":0.0,"publicationDate":"2011-11-14","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"73019564","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 46
Exploring workflow interoperability tools for neuroimaging data analysis 探索神经成像数据分析的工作流互操作性工具
Water & sewage works Pub Date : 2011-11-14 DOI: 10.1145/2110497.2110508
V. Korkhov, D. Krefting, T. Kukla, G. Terstyánszky, M. Caan, S. Olabarriaga
{"title":"Exploring workflow interoperability tools for neuroimaging data analysis","authors":"V. Korkhov, D. Krefting, T. Kukla, G. Terstyánszky, M. Caan, S. Olabarriaga","doi":"10.1145/2110497.2110508","DOIUrl":"https://doi.org/10.1145/2110497.2110508","url":null,"abstract":"Neuroimaging is a field that benefits from distributed computing infrastructures (DCIs) to perform data processing and analysis, which is often achieved using grid workflow systems. Collaborative research in neuroimaging requires ways to facilitate exchange between different groups, in particular to enable sharing, re-use and interoperability of applications implemented as workflows. The SHIWA project provides solutions to facilitate sharing and exchange of workflows between workflow systems and DCI resources. In this paper we present and analyse how the SHIWA platform was used to implement various usage scenarios in which workflow exchange supports collaboration in neuroscience. The SHIWA platform and the implemented solutions are described and analysed from the \"user\" perspective, in this case the workflow developers and the neuroscientists. We conclude that the platform in its current form is valuable for the foreseen usage scenarios, and we identify remaining challenges concerning management of multiple credentials and data transfers across DCIs.","PeriodicalId":87858,"journal":{"name":"Water & sewage works","volume":"225 1","pages":"87-96"},"PeriodicalIF":0.0,"publicationDate":"2011-11-14","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"80110330","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 13
Provenance for MapReduce-based data-intensive workflows 基于mapreduce的数据密集型工作流的来源
Water & sewage works Pub Date : 2011-11-14 DOI: 10.1145/2110497.2110501
Daniel Crawl, Jianwu Wang, I. Altintas
{"title":"Provenance for MapReduce-based data-intensive workflows","authors":"Daniel Crawl, Jianwu Wang, I. Altintas","doi":"10.1145/2110497.2110501","DOIUrl":"https://doi.org/10.1145/2110497.2110501","url":null,"abstract":"MapReduce has been widely adopted by many business and scientific applications for data-intensive processing of large datasets. There are increasing efforts for workflows and systems to work with the MapReduce programming model and the Hadoop environment including our work on a higher-level programming model for MapReduce within the Kepler Scientific Workflow System. However, to date, provenance of MapReduce-based workflows and its effects on workflow execution performance have not been studied in depth. In this paper, we present an extension to our earlier work on MapReduce in Kepler to record the provenance of MapReduce workflows created using the Kepler+Hadoop framework. In particular, we present: (i) a data model that is able to capture provenance inside a MapReduce job as well as the provenance for the workflow that submitted it; (ii) an extension to the Kepler+Hadoop architecture to record provenance using this data model on MySQL Cluster; (iii) a programming interface to query the collected information; and (iv) an evaluation of the scalability of collecting and querying this provenance information using two scenarios with different characteristics.","PeriodicalId":87858,"journal":{"name":"Water & sewage works","volume":"17 1","pages":"21-30"},"PeriodicalIF":0.0,"publicationDate":"2011-11-14","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"89439577","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 45
Scientific workflow reuse through conceptual workflows on the virtual imaging platform 在虚拟成像平台上通过概念工作流实现科学的工作流重用
Water & sewage works Pub Date : 2011-11-01 DOI: 10.1145/2110497.2110499
Nadia Cerezo, J. Montagnat
{"title":"Scientific workflow reuse through conceptual workflows on the virtual imaging platform","authors":"Nadia Cerezo, J. Montagnat","doi":"10.1145/2110497.2110499","DOIUrl":"https://doi.org/10.1145/2110497.2110499","url":null,"abstract":"An increasing number of scientific experiments are \"in-silico\": carried out at least partially using computers. Scientific Workflows have become a key tool to model and implement such experiments, but they tangle domain knowledge, technical know-how and non-functional concerns and are, as a result, difficult to understand, reuse or repurpose.\u0000 In order to ease Scientific Workflow Reuse, this paper defines a Conceptual Workflow model that is closer to the end-user's domain and intentions. By placing our model higher on the abstraction scale, we can separate concerns and emphasize the in-silico experiment inside the workflow, thus improving readability and re-usability.\u0000 The conceptual representation can then be transformed into a regular Abstract Scientific Workflow, exploiting both domain and non-functional knowledge that are captured and harnessed through the use of Semantic Web technologies.","PeriodicalId":87858,"journal":{"name":"Water & sewage works","volume":"85 12 1","pages":"1-10"},"PeriodicalIF":0.0,"publicationDate":"2011-11-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"87671935","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 12
Workflow automation for processing plasma fusion simulation data 处理等离子体聚变模拟数据的工作流程自动化
Water & sewage works Pub Date : 2007-06-25 DOI: 10.1145/1273360.1273368
N. Podhorszki, Bertram Ludäscher, S. Klasky
{"title":"Workflow automation for processing plasma fusion simulation data","authors":"N. Podhorszki, Bertram Ludäscher, S. Klasky","doi":"10.1145/1273360.1273368","DOIUrl":"https://doi.org/10.1145/1273360.1273368","url":null,"abstract":"The Center for Plasma Edge Simulation project aims to automate the tedious tasks of monitoring the simulation, archiving and post-processing the output. This paper describes the tasks and requirements, the several components developed within the Kepler workflow system to provide the required functionality and the automated workflow solution. Besides functionality, the focus is on the robust execution of the workflow. A user-level checkpoint/restart model has been developed that allows the workflow to restart, moreover, it also performs operations that failed during the previous run.","PeriodicalId":87858,"journal":{"name":"Water & sewage works","volume":"35 1","pages":"35-44"},"PeriodicalIF":0.0,"publicationDate":"2007-06-25","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"74407918","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 39
myExperiment: social networking for workflow-using e-scientists myExperiment:使用工作流的电子科学家的社交网络
Water & sewage works Pub Date : 2007-06-25 DOI: 10.1145/1273360.1273361
C. Goble, D. D. Roure
{"title":"myExperiment: social networking for workflow-using e-scientists","authors":"C. Goble, D. D. Roure","doi":"10.1145/1273360.1273361","DOIUrl":"https://doi.org/10.1145/1273360.1273361","url":null,"abstract":"We present the Taverna workflow workbench and argue that scientific workflow environments need a rich ecosystem of tools that support the scientists. experimental lifecycle. Workflows are scientific objects in their own right, to be exchanged and reused. myExperiment is a new initiative to create a social networking environment for workflow workers. We present the motivation for myExperiment and sketch the proposed capabilities and challenges. We argue that actively engaging with a scientist's needs, fears and reward incentives is crucial for success.","PeriodicalId":87858,"journal":{"name":"Water & sewage works","volume":"50 1","pages":"1-2"},"PeriodicalIF":0.0,"publicationDate":"2007-06-25","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"84558231","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 2
A workflow approach to designed reservoir study 设计油藏研究的工作流方法
Water & sewage works Pub Date : 2007-06-25 DOI: 10.1145/1273360.1273374
Gabrielle Allen, Promita Chakraborty, Dayong Huang, Zhou Lei, John Lewis, Xin Li, C. White, Xiaoxi Xu, Chongjie Zhang
{"title":"A workflow approach to designed reservoir study","authors":"Gabrielle Allen, Promita Chakraborty, Dayong Huang, Zhou Lei, John Lewis, Xin Li, C. White, Xiaoxi Xu, Chongjie Zhang","doi":"10.1145/1273360.1273374","DOIUrl":"https://doi.org/10.1145/1273360.1273374","url":null,"abstract":"Reservoir simulations are commonly used to predict the performance of oil and gas reservoirs, taking into account a myriad of uncertainties in the geophysical structure of the reservoir as well as operational factors such as well location. Designed reservoir study provides a robust tool to quantify the impact of uncertainties in model input variables, and can be used to simulate, analyze, and optimize reservoir development. However, such studies are computationally challenging, involving massive (terabyte or petabyte) geographically distributed datasets and requiring hundreds or tens of thousands of simulation runs. Providing petroleum engineers with integrated workflow through a secure and easy-to-use user interface will enable new advanced reservoir studies. This paper describes the workflow solution and user interface designed and implemented for reservoir uncertainty analysis in the UCoMS project ( Ubiquitous Computing and Monitoring System for discovery and management of energy resources).","PeriodicalId":87858,"journal":{"name":"Water & sewage works","volume":"67 1","pages":"75-79"},"PeriodicalIF":0.0,"publicationDate":"2007-06-25","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"84040049","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 2
A semantic workflow authoring tool for programming grids 一个用于网格编程的语义工作流创作工具
Water & sewage works Pub Date : 2007-06-25 DOI: 10.1145/1273360.1273373
G. Avellino, S. Beco, B. Cantalupo, Andrea Cavallini
{"title":"A semantic workflow authoring tool for programming grids","authors":"G. Avellino, S. Beco, B. Cantalupo, Andrea Cavallini","doi":"10.1145/1273360.1273373","DOIUrl":"https://doi.org/10.1145/1273360.1273373","url":null,"abstract":"Workflows have an increasing role in scientific applications programming and business environment development because they are an effective technology to define composition of different pieces of knowledge, both in the application domain and in the business context. However, workflows are addressed in a number of different perspectives and no consensus has been reached yet on the language, modelling, and systems, to adopt in advanced contexts like Grid or semantic Web. In this paper, we describe a graphical tool for supporting different kinds of users in authoring workflows running on next generation Grids. The tool, named Semantic Workflow Programming Tool, is based on a reference workflow model that has been developed to provide advanced functionality with respect to the currently available workflow description. Design and functionality of the tool resulted from requirements and scenarios emerged in the EU funded NextGRID project, in whose context the tool is developed.","PeriodicalId":87858,"journal":{"name":"Water & sewage works","volume":"14 1","pages":"69-74"},"PeriodicalIF":0.0,"publicationDate":"2007-06-25","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"88284768","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 2
Supporting large-scale science with workflows 支持大规模的科学工作流程
Water & sewage works Pub Date : 2007-06-25 DOI: 10.1145/1273360.1273369
D. Pennington
{"title":"Supporting large-scale science with workflows","authors":"D. Pennington","doi":"10.1145/1273360.1273369","DOIUrl":"https://doi.org/10.1145/1273360.1273369","url":null,"abstract":"Current workflow systems support data flow through complex analyses in a distributed environment. The scope of scientific workflow systems could expand to support the entire scientific research cycle, which includes data flow, design flow, and knowledge flow tasks. Example tasks and requirements within each of these are explored, illustrated by a use case in environmental science. Nexus points between scientific tasks and potential workflow support mechanisms are identified and described. Several new ideas, or modifications of existing ideas are introduced: science dashboards, meta-provenance, event-driven workflows, extreme workflow development, and collective thinking workflows.","PeriodicalId":87858,"journal":{"name":"Water & sewage works","volume":"9 1","pages":"45-52"},"PeriodicalIF":0.0,"publicationDate":"2007-06-25","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"87390165","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 8
首页 上一页 下一页 尾页
0
×
引用
GB/T 7714-2015
复制
MLA
复制
APA
复制
导出至
BibTeX EndNote RefMan NoteFirst NoteExpress
×
提示
您的信息不完整，为了账户安全，请先补充。
现在去补充
×
提示
您因"违规操作"
具体请查看互助需知
我知道了
×
提示
现在去查看 取消
×
提示
确定
请完成安全验证×
相关产品
×
本文献相关产品
联系我们：info@booksci.cn Book学术提供免费学术资源搜索服务，方便国内外学者检索中英文文献。致力于提供最便捷和优质的服务体验。 Copyright © 2023 布克学术 All rights reserved.
京ICP备2023020795号-1
ghs 京公网安备 11010802042870号
Book学术文献互助
Book学术文献互助群
群 号：604180095
Book学术官方微信
小红书