Challenges and Opportunities for Dataflow Processing on Exascale Computers

Computational applications critical to society in areas such as materials design, climate modeling, and energy production and distribution must be developed quickly and correctly. Such studies are typically done via computational experiments, in which large numbers of simulation and analysis tasks are strung together into a workflow, formally or informally. Many innovative programming models to handle workflow specifications are based on an implicitly parallel dataflow language to support this model. Here, we posit that the dataflow model has the added benefit that it can exploit features of exascale computers, expected in ∼2023 [21]. Workflows and other outermost patterns such as parameter sweeps, searches, and optimizations can easily be expressed with dataflow languages. In this model, either statically or at runtime a dataflow structure is available to the runtime system, presenting the opportunity for many types of automated decisions for scheduling and resource management. Workflow applications on these systems will also introduce new requirements, such as high-performance data movement methods for in situ data analysis, and new challenges, such as varying reliability characterstics. In this paper, we describe three key exascale feature areas that will be exposed to users at the application level. First, a more complex storage hierarchy is expected. New cache types such as scratchpad memory may be available, and heterogeneous RAM systems may have differing performance and reliability characteristics. Node-local storage may be available. These systems are expected to be available to the application or middleware via advanced operating system and runtime features. Second, tighter power budgets and programmer-controlled power scaling will likely be available. These will require the application level to make decisions about performance/power tradeoffs. Third, a more complex