Towards the development of hierarchical data motion power cost models

Abstract

Data intensive applications comprise a considerable portion of HPC center workloads. Whether large amounts of data transfer occur before, during or after an application is executed, this cost must be considered. Not just in terms of performance (e.g. time to completion), but also in terms of power consumed to complete these necessary tasks. At the system level, scheduling and resource management tools are capable of recording performance metrics and other constraints, and making performance aware decisions. These tools are a natural choice for making power aware decisions, as well. More specifically, power aware decisions about data transfer costs for the entire application workflow. This research focuses on developing data motion power cost models and integrating these models into a task scheduler framework to enable complete power aware scheduling of an entire HPC workflow. We have taken an incremental approach to developing a hierarchical, system wide power model for data motion that starts with core data motion and will eventually encompass data motion across facilities. In this paper, we discuss our current research which addresses multicore data motion and data motion between nodes.