Task characterization-driven scheduling of multiple applications in a task-based runtime

Abstract

Task-based runtimes like OCR, X10 and Charm++ promise to address scalability challenges on Exascale machines due to their finegrained parallelism, inherent asynchrony, and consequent efficient localized synchronization. Although such runtimes are typically used to run a single application at a time, a common HPC scenario involves running a producer simulation application co-located with a consumer analytics application, to reduce data movement costs. The potentially diverse requirements of such co-located applications present challenges to the ability of the runtime to efficiently manage underlying resources, maintain application performance, and minimize sharing effects on application progress. To address this, we implement and study techniques based on application task characterization to improve resource utilization in shared task-based runtimes. We demonstrate that, by maintaining tasks characteristics, such as their compute, cache or memory intensity, using offline and/or online methods, we can improve task-based runtimes' ability to schedule and place tasks to minimize resource contention for co-running applications. Results are obtained via experimentation with the Open Community Runtime (OCR) on two distinct platforms ---an x86-based machine and a research platform based on the experimental Traleika Glacier (TG) architecture. On the x86, we see a performance improvement of 15% and on TG, we observe a reduction of energy usage by more than 50%, illustrating the potential benefits of the approach for next generation exascale platforms.