{"title":"Tackling Complexity in High Performance Computing Applications.","authors":"J Darlington, A J Field, L Hakim","doi":"10.1007/s10766-016-0422-9","DOIUrl":"https://doi.org/10.1007/s10766-016-0422-9","url":null,"abstract":"<p><p>We present a software framework that supports the specification of user-definable configuration options in HPC applications independently of the application code itself. Such options include model parameter values, the selection of numerical algorithm, target platform etc. and additional <i>constraints</i> that prevent invalid combinations of options from being made. Such constraints, which are capable of describing complex cross-domain dependencies, are often crucial to the correct functioning of the application and are typically either completely absent from the code or a hard to recover from it. The framework uses a combination of functional workflows and constraint solvers. Application workflows are built from a combination of functional components: higher-order co-ordination forms and first-order data processing components which can be either concrete or abstract, i.e. without a specified implementation at the outset. A repository provides alternative implementations for these abstract components. A constraint solver, written in Prolog, guides a user in making valid choices of parameters, implementations, machines etc. for any given context. Partial designs can be stored and shared providing a systematic means of handling application use and maintenance. We describe our methodology and illustrate its application in two classes of application: a data intensive commercial video transcoding example and a numerically intensive incompressible Navier-Stokes solver.</p>","PeriodicalId":14313,"journal":{"name":"International Journal of Parallel Programming","volume":"45 2","pages":"402-420"},"PeriodicalIF":1.5,"publicationDate":"2017-01-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://sci-hub-pdf.com/10.1007/s10766-016-0422-9","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"37782858","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
George Teodoro, Tulio Tavares, Renato Ferreira, Tahsin Kurc, Wagner Meira, Dorgival Guedes, Tony Pan, Joel Saltz
{"title":"A Run-time System for Efficient Execution of Scientific Workflows on Distributed Environments.","authors":"George Teodoro, Tulio Tavares, Renato Ferreira, Tahsin Kurc, Wagner Meira, Dorgival Guedes, Tony Pan, Joel Saltz","doi":"10.1007/s10766-007-0068-8","DOIUrl":"https://doi.org/10.1007/s10766-007-0068-8","url":null,"abstract":"<p><p>Scientific workflow systems have been introduced in response to the demand of researchers from several domains of science who need to process and analyze increasingly larger datasets. The design of these systems is largely based on the observation that data analysis applications can be composed as pipelines or networks of computations on data. In this work, we present a runtime support system that is designed to facilitate this type of computation in distributed computing environments. Our system is optimized for data-intensive workflows, in which efficient management and retrieval of data, coordination of data processing and data movement, and check-pointing of intermediate results are critical and challenging issues. Experimental evaluation of our system shows that linear speedups can be achieved for sophisticated applications, which are implemented as a network of multiple data processing components.</p>","PeriodicalId":14313,"journal":{"name":"International Journal of Parallel Programming","volume":"36 2","pages":"250-266"},"PeriodicalIF":1.5,"publicationDate":"2008-04-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://sci-hub-pdf.com/10.1007/s10766-007-0068-8","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"30615270","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}