User-steering of HPC workflows: state-of-the-art and future directions

Abstract

In 2006 a group of leading researchers was gathered to discuss several challenges to scientific workflow supporting technologies and many of which still remain open challenges, such as the steering of workflows by users. Due to big data and long lasting workflows, many users demand steering features such as real-time monitoring, analysis and specially execution interference. The workflow execution should respond dynamically to such interference in the execution, to support the experimentation process in high performance computing. This paper revisits the issues in the user steering and dynamic workflows, presenting the state-of-the-art in it, and the open challenges. Our goal is to discuss research issues related to scientists' steering and present some ideas on how these demands may be supported in current scientific workflow technologies.