EVADyR: A new dynamic resampling algorithm for auto-tuning noisy High Performance Computing systems

Abstract

Black-box auto-tuning methods have been proven to be efficient for tuning configurable computer hardware, including those encountered within the High Performance Computing (HPC) ecosystem. However, because of the shared nature of HPC clusters and the complexity of the software and hardware stacks, the measurement of the performance function can be tainted by noise during the tuning process, which can reduce and sometimes prevent the benefit of the tuning approach. A usual choice for performing the tuning in spite of these interference is to add a resampling step at each iteration to reduce uncertainty, but this approach can be time-consuming and must be done carefully. In this paper, we propose a new resampling and filtering algorithm called EVADyR (Efficient Value Aware Dynamic Resampling). Compared to the state of the art, it finds a better exploration versus exploitation trade-off by resampling only promising configuration and increases the level of confidence around the suggested solution as the tuning process advances. This algorithm was able to tune efficiently two I/O accelerators highly sensitive to interference, in two different scenarios. Compared to Standard Error Dynamic Resampling (SEDR), a state of the art noise reduction strategy, we show that EVADyR is able to reduce the distance to the optimum by 93.5% and 24.7% for the two I/O accelerators respectively, as well as speed-up the experiment duration by 45.8% and 58.1% because less iterations are needed to reach the found optimum. Our results prove the importance of using noise reduction strategies whenever tuning systems running in production.