Minimizing the Average Job Completion Time for Acceleration Systems in Cloud Computing

Abstract

With the development of computation intensive applications, such as deep neural network inference and deep packet inspection, the conventional computation resources are exhausted by these computing tasks, which results in large application response time. To improve the user experience, more and more providers deploy accelerators in their computing clusters. Accordingly, there is a problem arising: how should we schedule the non-preemptive jobs such that the average job completion time can be minimized. To answer this question, we first formulate the problem to be a mathematical programming model. Based on solid analysis, we find that the problem we need to solve is NP-hard. Due to the hardness of this problem, we propose a (6 – 2/M)-approximation algorithm to solve it efficiently, where M is the number of accelerator servers in the system. Through extensive simulations, we find that the proposed algorithm outperforms the conventional scheduling algorithms, FIFO and Shortest Job First (SJF), by 24.24% and 29.07%, respectively.