Graph Templates for Dataflow Programming

Abstract

Current works on parallel programming models are trending towards the dataflow paradigm, which naturally exploits parallelism in programs. The Sucuri Python Library provides basic features for creation and execution of dataflow graphs in parallel environments. However, there is still a gap between dataflow programming and traditional parallel programming. In this paper we aim at narrowing that gap by introducing a set of templates for Sucuri that represent some of the most important parallel programming patterns. Through these templates programmers can implement applications that use patterns such as fork/join, pipeline and wave front just by instantiating and connecting sub-graph objects. Evaluation showed that the use of templates makes programming easier, while allowing a significant reduction in lines of code, compared to manually creating the dataflow graph.