A compiler transformation-based approach to scientific workflow enactment

Abstract

We investigate in this paper the application of compiler transformations to workflow applications using the Manycore Workflow Runtime Environment (MWRE), a compiler-based workflow environment for modern manycore computing architectures. MWRE translates scientific workflows into equivalent C++ programs and efficiently executes them using a novel callback mechanism for dependency resolution and data transfers, with explicit support for full-ahead scheduling. We evaluate four different classes of compiler transformations, analyse their advantages and possible solutions to overcome their limitations, and present experimental results for improving the performance of a combination of real-world and synthetic workflows through compiler transformations. Our experiments were able to improve the workflow enactment by a factor of two and to reduce the memory usage of the engine by up to 33%. We achieved a speedup of up to 1.7 by eliminating unnecessary activity invocations, an improved parallel throughput up to 2.8 times by transforming the workflow structure, and a better performance of the HEFT scheduling algorithm by up to 36%.