Analysis of Energy and Performance of PGAS-based Data Access Patterns

One of the factors associated with the usability of distributed programming models in exascale machines, is the energy and power cost associated with data movement across large-scale systems. PGAS implementations provide users with explicit interfaces for one-sided transfers to remote processes. However, a number of factors across the software stack have the potential of significantly impacting the energy signatures of communication-intensive applications that rely on such transfers. Performance characteristics like the use of non-blocking communication, the actual count of number of initiated transfers, the size of data payload packed within each transfer, as well as the use of pinned-down user buffers, all contribute to this impact. In this paper, we discuss a number of RDMA-based communication patterns that are frequently incorporated within applications and communication libraries and, that have the potential of significantly impacting the energy and performance characteristics. We present an empirical study of the potential energy savings achievable by studying the impact on the CPU and DRAM. Since performance is a major criteria for PGAS programming models, we use the energy-delay product as a metric to justify the feasibility of these transformations. We hope that this work motivates the incorporation of energy-based metrics for fine tuning PGAS implementations.