Supporting dynamic parameter sweep in adaptive and user-steered workflow

Abstract

Large-scale experiments in computational science are complex to manage. Due to its exploratory nature, several iterations evaluate a large space of parameter combinations. Scientists analyze partial results and dynamically interfere on the next steps of the simulation. Scientific workflow management systems can execute those experiments by providing process management, distributed execution and provenance data. However, supporting scientists in complex exploratory processes involving dynamic workflows is still a challenge. Features, such as user steering on workflows to track, evaluate and adapt the execution need to be designed to support iterative methods. We provide an approach to support dynamic parameter sweep, in which scientists can use the results obtained in a slice of the parameter space to improve the remainder of the execution. We propose new control structures to enable adaptive and user-steered workflows supporting iterative methods using dynamic mechanisms. We evaluate our approach using a proof of concept (Lanczos algorithm) workflow and the results show up to 78% of execution time saved.