Composition of engineering web services with universal distributed data-flows framework based on ROA

Abstract

The problem of staging data in workflows has received much attention over the last decade, with a variety of user-directed and automatic solutions. The latter are the focus of the first contribution in this paper, where we propose a simple peer-to-peer solution adapted to the needs of RESTful services. The second contribution, is the combination of the data staging mechanism with a simple service deployment mechanism, that is designed to allow applications developed for the command-line to function as (RESTful) services without modification or (in some cases) recompilation. Thus, the aim of this paper is to describe the design and implementation of: (i) a peer-to-peer data-staging mechanism, that is itself RESTful, and (ii) a service deployment mechanism, also following REST design principles, which together form the Universal Distributed Data-flows framework, for the support of data-intensive (RESTful) workflows. We evaluate the framework by means of an engineering workflow developed for multi-disciplinary design optimization. The workflow itself is specified in Taverna, which is a conventional centralized data-staging enactment system. However, by virtue of the underlying services and staging mechanisms described here, the resulting enactment is peer-to-peer (for data), which furthermore permits asynchronous staging, with potential benefits for network utilization and end-to-end execution time.