Improvement of worker scaling-based scheduling algorithm to efficiently respond to explosion of micro function service requests

We present improved worker scheduler algorithm and architecture to minimize the scheduling cost and function service response time of FaaS platform. We try to solve the problem of performance degradation of previous scaling-based worker scheduling algorithm due to the explosion of the function invocation requests and the blocking-based communication channel thread with the computation node. To this end, two algorithms including Load-based function worker demand prediction algorithm and adoption of asynchronous communication channel integration and non-blocking allocation method have been developed. Through the performance evaluation of proposed worker scheduling algorithm on FaaS worker scheduler, the proposed worker scheduler achieves speedups as high as 10.1 and 16.8 compared to the baseline Bare-metal Computing nodes with the CPU and GPU devices using the in-house TSL Inferencing application.